I have 3 projects that I need to change over to our encrypted storage location. The project currently has files already uploaded.
What happens to the existing files in a project when I change the project's storage location? Do they move to the new (encrypted) location or stay at the same (unencrypted) location?
Created by gates-ki-service @gates-ki-service Yes that will work as well, although note that you will be purging the cache entries for all files in your synapse cache which match that date criteria, some of which may not be relevant to this storage location move. If that is not a concern (the consequence being that all such files would need to be downloaded if used from Synapse again rather than fetched from the cache) then that should be fine. @jordank I'm using the `syn.cache.purge` method with a future date. Is this the correct procedure? It appears to work. @pstoutdev The Python client will skip re-uploading an unchanged file that is already in its local cache. In this case that would have the undesired side effect of preventing the upload from depositing the unchanged file in the new storage location.
If you don't mind changing the timestamp of the file(s) on your local filesystem then an easy way of bypassing this and forcing it to re-upload the file is to reset the local file timestamps to the current time.
e.g. on a unix based system:
```
touch filename
```
Or on Windows:
```
copy /b filename +,,
```
If changing the local timestamps would be problematic let me know and I can supply an alternate way of removing the file(s) entries from the local cache completely. Through the web interface you can upload the same exact file and it will increment the version and store it in the new bucket. Is there an easy way to do this using the Python package?
The API changes will tentatively be available on production on the last week of the month,
Thanks
Marco Thanks for the info @marco.marasca. Do you know when the updates to the API will be available? Hello,
As Bruce indicated the storage location applies only for new uploads. The files can be moved to the new bucket (e.g. if the files are stored in an external S3 bucket using the S3 cp command) and relinked to the entities: this involves creating new file handles pointing to the new storage locations using the dedicated API: https://rest-docs.synapse.org/rest/POST/externalFileHandle/s3.html and updating the existing entities with the new file handle, using an update request specifying the new file handle: https://rest-docs.synapse.org/rest/PUT/entity/id.html. Currently this will automatically create a new version of the entity, the previous version will still point to the old location. If not needed the old version of the entity as well as the old file handle can be deleted.
For files stored in the default Synapse storage location, we will soon provide an API that will vend special AWS credentials allowing to copy the file from the synapse bucket into a new bucket using the S3 copy command. The process to update the entities will be the same as above.
Additionally we will also update the entity API to allow linking to a new file handle without automatically changing the version.
Thanks
Marco I will give a brief answer, but I'd like to connect @marco.marasca , a Synapse engineer who is working on solutions for bucket migration.
The storage location for a project indicates where new files will be uploaded. It doesn't affect existing files. For existing files you will have to have a migration strategy, and I invite Marco to comment on his effort to support such work.
I will also comment that if your files are currently stored in the default Synapse storage then the files are already encrypted with AES256 server side encryption, consistent with HIPAA requirements for protected health information.
Drop files to upload
Changing Storage Location on Project with files? page is loading…