File Safe

Goals and implementation

The collector file safe is a S3-like file storage with the following features:

  • upload and download over https

  • user or node authentication

  • fine-grained ACLs

  • immutable files

  • file integrity check upon download request

  • unique file version id

  • binary and text file types

Creating a safe file using the collector web interface

A « add safe file » tool is accessible directly from the navigation menu.

A file created using this tool has no initial content. An upload should follow.

Upload/Download using the collector web interface

Upload and download buttons are located in the safe file properties tab.

Upload/Download using nodemgr

The safe cli command Syntax

$ om node collector cli -- safe --help
Usage: nodemgr.py [options]

Upload, download and manage files in the collector safe. The safe is a file
sharing facility with access control rules for nodes and users making it
suitable to serve compliance reference files.

Options:
  -h, --help   show this help message and exit
  --ls         List the accessible files in the safe.
  --upload     Upload the file pointed by --file to the safe. Optionally give
               a name using --name.
  --download   Download from the safe the file pointed by --file to the file
               path or directory pointed by --to.
  --file=FILE  The safe file uuid to download, or the local file to upload.
  --to=TO      The local file path or directory name to download.
  --name=NAME  The user-friendly name to attach to the upload.

Example upload

Create a new file

$ om node collector cli -- safe --upload --file /etc/resolv.conf --name resolv.conf

Upload a new version of an existing file

$ om node collector cli -- safe --upload --file /etc/resolv.conf --name resolv.conf --id 59

Oneliner to store a secret in the safe

$ echo -n s3cr3t | om node collector cli -- safe --upload --name mysecret --file /dev/stdin
{
    "data": {
        "id": 56,
        "md5": "cc414bfc9c00475b59c87595299ff31d",
        "name": "mysecret",
        "size": 5,
        "uploaded_date": "2018-03-25 22:49:05",
        "uploaded_from": "10.0.3.1",
        "uploader": 1,
        "uuid": "safe.uuid.aaab49ba4f926d2a.706970653a5b3135343430323733335d.txt"
    }
}

Example download

By content uuid

$ om node collector cli -- safe --download --file safe.uuid.a44b45aa173dfe3c.7265736f6c762e636f6e66.conf --to /tmp/foo
.
downloaded

$ cat /tmp/foo
nameserver 8.8.8.8

By id (last content version)

$ om node collector cli -- safe --download --file 59 --to /tmp/foo
.
downloaded

$ cat /tmp/foo
nameserver 8.8.8.8

Safe support in compliance

The files compliance object is able to fetch file content from the safe. To that end, just use the safe://<file id> format in the files rule ref key. The path key specifies where the file should be installed.

Safe support in service configurations

The {safe://<id>} and {safe://<uuid>} service configuration references are automatically resolved if the service app code responsibles are allowed to read the corresponding secrets.

Those secrets are obfuscated in the service action logs.