Hello -
I'm looking for a way to use the command line tool to export a complete manifest of what is underneath it. The "synapse_metadata_manifest.tsv" file that is generated from a command like
```
synapse get -r syn27776531
```
is a great format, but I want to download a listing of the entire structure without actually downloading any of the data files. I can get a format that is somewhat like this by using this in Powershell:
```
synapse list -r syn26353105 | Out-File -FilePath .\tree.txt
```
But the tsv file generated from the get command would work best. Is there a way to do this? I wasn't able to find a flag in the CLI docs that did this.
Thanks!
Created by Wade DeJager wdejager @thomas.yu,
That's actually ok, no need to do anything further on this ticket. I was able to assemble what I needed from the CLI tool with a few additional formatting steps. I'll look forward to any future improvements to the process.
Best,
Wade Hi @wdejager ,
Apologies for the delays in response. Unfortunately, that particular command without downloading the files will result in that API endpoint being hit. We have been looking into improving the implementation so we batch get the information instead of calling that endpoint thousands of times.
In the meantime, let me try and connect you to the project data managers, maybe they can assist you with a file view and you would be able to do a table query on the information. Hello, @thomas.yu
For the python method: when I try the code as you suggested above, if I try it on a "low-level" synapse ID, syn26324313 for example, which is only 1 directory above three files, it works. If I try it on a high-level directory, like syn27201607 which contains the hierarchy I'm trying to export, I get the following error:
[WARNING] Requests are too frequent for API call: /entity/#/bundle2. Allowed 240 requests every 60 seconds....
Which repeats until I kill the process.
Any suggestions?
Best,
Wade Hello @wdejager,
Unfortunately, there isn't a command you can run to achieve this. but there are multiple ways to achieve this via Synapse or the Python client. In Synapse we have fileviews:
1. Please see this: https://help.synapse.org/docs/Views.2011070739.html. This will allow you to create a SQL like table using `syn26353105` as the scope to create a flat manifest file similar to the one that is generated by the `syn get -r` command without downloading the data.
2. Using python
```python
import pandas
import synapseclient
import synapseutils #this is installed when you install the python client
file_list = synapseutils.syncFromSynapse(syn, 'syn26353105', downloadFile=False)
manifest = pandas.DataFrame(file_list)
```
Please let me know if you have any questions @thomas.yu Do you want to provide some guidance for using the CLI?
Drop files to upload
CLI function to export only complete recursive manifest? page is loading…