External Storage Services

If you have media stored in other locations, such as your own S3 bucket, you can tell the platform to sync that media automatically so it’s available to be searched, tagged, shared, and edited. The original media will remain on your other storage, but proxies will be generated and stored on the platform for fast playback of your media.

To add a new External Storage source, navigate to the Workspace Settings (gear icon) > Exteral Storage. Note that you must be an Admin of the Workspace to edit these settings. Click the Add New External Storage button.

In the dialogue that pops up, enter the following values:

• Type: (currently only S3-compatable External Storage is supported)
• Name: a name for the External Storage. The Name value is used to create a Project with the same name for ingesting the media to.
• Path: if you only want to ingest media in a specific folder on the external storage, enter the folder name here (separate multiple hierarchical folders with a slash, for example videos/day1 or just videos). If you leave this field blank, the entire storage will be ingested
• Endpoint: If you are not using Amazon S3 and have a unique endpoint, such as with services like Wasabi, enter the bucket endpoint here. For Amazon S3, leave this field blank.
• Bucket: the name of the S3-compatible bucket
• Region: the region the bucket is located in
• Access Key and Secret Key: the credentials for accessing the bucket. The platform securely stores them in AWS Secrets Manager, so your credentials will remain safe. For permissions, ensure the credentials are granted List and Read permission. They do not need Write permission, and following the practice of least privilege, they ideally should not be given Write permission to your bucket.

When you hit Save, a new External Storage source will be created, and a new Project will be created to sync the files into. After a moment, the files will begin syncing. Please note that for stability reasons, there is a limit to how many files can be ingested at once.

If the sync completes and you see some files were not ingested (for example if you have many tens of thousands of files), hit the Resync button to sync the additional files. The process will look for tens of thousands more files and ingest anything that is new.

If any ingested files are moved or renamed, and you Resync the External Storage source, the changed files could get ingested again since a comparison is done on file name and folder location to determine if a file has already been ingested, and if either of those properties has changed it will be considered a new file.