Created by minmin luo minminluo Hi,
Thank you for your aid. However, I consulted our network office, they said they have NOT blocked any of the IP in the list.
Anyway, they set another channel specificly for these IPs, and we are trying again.
I was wondering is there any parameter I could use to resume the download once it failed. Just like when I use wget, there is -c or --continue, which could resume getting a partially-downloaded file.
Or should I retype 'synapse get syn52404044' in the same folder and it will automatically resume the download?
thanks. Hi @minminluo ,
Thanks for the information. I ran the command on my computer and was able to download the file. That rules out the file being corrupted on Synapse.
May I ask if you are on an institution network? If so, is it possible that one of these IP addressed are blocked by your institution? https://help.synapse.org/docs/Synapse-Infrastructure.2835382273.html.
Sorry for the inconvenience!
Best,
Tom Hi, I tried the new parameter 'get syn52404044'. But still failed.
first try with 'synapse get syn52404044'
(base) minminluo@Victor ~/DATA/yufengting $ synapse get syn52404044
Downloading [--------------------]0.06% 8.0MB/12.3GB (107.1kB/s) D19-12513.po
Downloading [--------------------]0.06% 8.0MB/12.3GB (108.8kB/s) D19-12513.po
Downloading [--------------------]0.13% 16.0MB/12.3GB (202.3kB/s) D19-12513.p
Downloading [--------------------]0.19% 24.0MB/12.3GB (299.7kB/s) D19-12513.p
Downloading [--------------------]0.06% 8.0MB/12.3GB (112.8kB/s) D19-12513.po
Downloading [--------------------]0.13% 16.0MB/12.3GB (215.5kB/s) D19-12513.p
Downloading [--------------------]0.06% 8.0MB/12.3GB (97.3kB/s) D19-12513.pos
Downloading [--------------------]0.13% 16.0MB/12.3GB (180.3kB/s) D19-12513.p
Downloading [--------------------]0.19% 24.0MB/12.3GB (267.8kB/s) D19-12513.p
Downloading [--------------------]0.06% 8.0MB/12.3GB (103.9kB/s) D19-12513.po
Downloading [--------------------]0.13% 16.0MB/12.3GB (206.3kB/s) D19-12513.p
Downloading [--------------------]0.19% 24.0MB/12.3GB (297.8kB/s) D19-12513.p
Downloading [--------------------]0.25% 32.0MB/12.3GB (388.7kB/s) D19-12513.possorted_bam.bam.synapse_download_128421089
ValueError: Failed downloading syn52404044 to /storage/minminluo/yufengting/D19-12513.possorted_bam.bam.synapse_download_128421089
caused by SynapseError: Could not download the file: D19-12513.possorted_bam.bam, please try again.
Because it says 'please try again', so I tried again.
Second try with ' synapse get syn52404044 '
(base) minminluo@Victor ~/DATA/yufengting $ synapse get syn52404044
Downloading [--------------------]0.06% 8.0MB/12.3GB (121.4kB/s) D19-12513.po
Downloading [--------------------]0.06% 8.0MB/12.3GB (86.9kB/s) D19-12513.pos
Downloading [--------------------]0.13% 16.0MB/12.3GB (173.4kB/s) D19-12513.p
Downloading [--------------------]0.19% 24.0MB/12.3GB (250.5kB/s) D19-12513.p
Downloading [--------------------]0.25% 32.0MB/12.3GB (324.4kB/s) D19-12513.p
Downloading [--------------------]0.06% 8.0MB/12.3GB (102.8kB/s) D19-12513.po
Downloading [--------------------]0.13% 16.0MB/12.3GB (197.8kB/s) D19-12513.p
Downloading [--------------------]0.19% 24.0MB/12.3GB (281.9kB/s) D19-12513.p
Downloading [--------------------]0.06% 8.0MB/12.3GB (92.5kB/s) D19-12513.pos
Downloading [--------------------]0.06% 8.0MB/12.3GB (81.0kB/s) D19-12513.pos
Downloading [--------------------]0.13% 16.0MB/12.3GB (159.1kB/s) D19-12513.p
Downloading [--------------------]0.19% 24.0MB/12.3GB (233.7kB/s) D19-12513.p
Downloading [--------------------]0.25% 32.0MB/12.3GB (310.0kB/s) D19-12513.p
Downloading [--------------------]0.32% 40.0MB/12.3GB (382.4kB/s) D19-12513.p
Downloading [--------------------]0.38% 48.0MB/12.3GB (455.2kB/s) D19-12513.p
Downloading [--------------------]0.44% 56.0MB/12.3GB (527.2kB/s) D19-12513.possorted_bam.bam.synapse_download_128421089
ValueError: Failed downloading syn52404044 to /storage/minminluo/yufengting/D19-12513.possorted_bam.bam.synapse_download_128421089
caused by SynapseError: Could not download the file: D19-12513.possorted_bam.bam, please try again.
This failed again. So I tried with '--debug'
result from 'synapse --debug get syn52404044'
Downloading [--------------------]0.06% 8.0MB/12.3GB (91.8kB/s) D19-12513.posDownloading [--------------------]0.1
3% 16.0MB/12.3GB (175.2kB/s) D19-12513.pDownloading [--------------------]0.19% 24.0MB/12.3GB (262.5kB/s) D19-125
13.pDownloading [--------------------]0.25% 32.0MB/12.3GB (348.5kB/s) D19-12513.possorted_bam.bam.synapse_download_
128421089 2024-01-09 17:35:23,402 [client:2655 - DEBUG]:
Retrying download on error: [] after progressing 0 bytes
Traceback (most recent call last):
File "/storage/minminluo/software/anaconda3/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/core/multithread_download/downloa
d_threads.py", line 366, in _get_response_with_retry
raise SynapseError(
synapseclient.core.exceptions.SynapseError: Could not download the file: D19-12513.possorted_bam.bam, please try again
.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/client.py", line 2631, in _downloadFileHandle
downloaded_path = self._download_from_url_multi_threaded(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/storage/minminluo/software/anaconda3/lib/python3.11/contextlib.py", line 81, in inner
return func(*args, **kwds)
^^^^^^^^^^^^^^^^^^^
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/client.py", line 2682, in _download_from_url_multi_threaded
multithread_download.download_file(self, request)
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/core/multithread_download/download_threads.py", line 247, in download_file
downloader.download_file(download_request)
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/core/multithread_download/download_threads.py", line 323, in download_file
self._check_for_errors(request, completed_futures)
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/core/multithread_download/download_threads.py", line 426, in _check_for_errors
raise ValueError(
ValueError: Failed downloading syn52404044 to /storage/minminluo/yufengting/D19-12513.possorted_bam.bam.synapse_download_128421089
Downloading [--------------------]0.06% 8.0MB/12.3GB (97.8kB/s) D19-12513.posDownloading [--------------------]0.13% 16.0MB/12.3GB (188.5kB/s) D19-12513.pDownloading [--------------------]0.19% 24.0MB/12.3GB (280.1kB/s) D19-12513.possorted_bam.bam.synapse_download_128421089 2024-01-09 17:37:37,845 [client:2655 - DEBUG]:
Retrying download on error: [] after progressing 0 bytes
Traceback (most recent call last):
File "/storage/minminluo/software/anaconda3/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/core/multithread_download/download_threads.py", line 366, in _get_response_with_retry
raise SynapseError(
synapseclient.core.exceptions.SynapseError: Could not download the file: D19-12513.possorted_bam.bam, please try again.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/client.py", line 2631, in _downloadFileHandle
downloaded_path = self._download_from_url_multi_threaded(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/storage/minminluo/software/anaconda3/lib/python3.11/contextlib.py", line 81, in inner
return func(*args, **kwds)
^^^^^^^^^^^^^^^^^^^
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/client.py", line 2682, in _download_from_url_multi_threaded
multithread_download.download_file(self, request)
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/core/multithread_download/download_threads.py", line 247, in download_file
downloader.download_file(download_request)
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/core/multithread_download/download_threads.py", line 323, in download_file
self._check_for_errors(request, completed_futures)
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/core/multithread_download/download_threads.py", line 426, in _check_for_errors
raise ValueError(
ValueError: Failed downloading syn52404044 to /storage/minminluo/yufengting/D19-12513.possorted_bam.bam.synapse_download_128421089
Downloading [--------------------]0.06% 8.0MB/12.3GB (90.5kB/s) D19-12513.posDownloading [--------------------]0.13% 16.0MB/12.3GB (173.6kB/s) D19-12513.pDownloading [--------------------]0.19% 24.0MB/12.3GB (253.8kB/s) D19-12513.pDownloading [--------------------]0.25% 32.0MB/12.3GB (336.7kB/s) D19-12513.possorted_bam.bam.synapse_download_128421089 2024-01-09 17:41:46,080 [client:2655 - DEBUG]:
Retrying download on error: [] after progressing 0 bytes
Traceback (most recent call last):
File "/storage/minminluo/software/anaconda3/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/core/multithread_download/download_threads.py", line 366, in _get_response_with_retry
raise SynapseError(
synapseclient.core.exceptions.SynapseError: Could not download the file: D19-12513.possorted_bam.bam, please try again.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/client.py", line 2631, in _downloadFileHandle
downloaded_path = self._download_from_url_multi_threaded(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/storage/minminluo/software/anaconda3/lib/python3.11/contextlib.py", line 81, in inner
return func(*args, **kwds)
^^^^^^^^^^^^^^^^^^^
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/client.py", line 2682, in _download_from_url_multi_threaded
multithread_download.download_file(self, request)
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/core/multithread_download/download_threads.py", line 247, in download_file
downloader.download_file(download_request)
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/core/multithread_download/download_threads.py", line 323, in download_file
self._check_for_errors(request, completed_futures)
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/core/multithread_download/download_threads.py", line 426, in _check_for_errors
raise ValueError(
ValueError: Failed downloading syn52404044 to /storage/minminluo/yufengting/D19-12513.possorted_bam.bam.synapse_download_128421089
Downloading [--------------------]0.06% 8.0MB/12.3GB (100.0kB/s) D19-12513.poDownloading [--------------------]0.13% 16.0MB/12.3GB (190.8kB/s) D19-12513.possorted_bam.bam.synapse_download_128421089 2024-01-09 17:43:56,527 [client:2655 - DEBUG]:
Retrying download on error: [] after progressing 0 bytes
Traceback (most recent call last):
File "/storage/minminluo/software/anaconda3/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/core/multithread_download/download_threads.py", line 366, in _get_response_with_retry
raise SynapseError(
synapseclient.core.exceptions.SynapseError: Could not download the file: D19-12513.possorted_bam.bam, please try again.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/client.py", line 2631, in _downloadFileHandle
downloaded_path = self._download_from_url_multi_threaded(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/storage/minminluo/software/anaconda3/lib/python3.11/contextlib.py", line 81, in inner
return func(*args, **kwds)
^^^^^^^^^^^^^^^^^^^
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/client.py", line 2682, in _download_from_url_multi_threaded
multithread_download.download_file(self, request)
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/core/multithread_download/download_threads.py", line 247, in download_file
downloader.download_file(download_request)
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/core/multithread_download/download_threads.py", line 323, in download_file
self._check_for_errors(request, completed_futures)
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/core/multithread_download/download_threads.py", line 426, in _check_for_errors
raise ValueError(
ValueError: Failed downloading syn52404044 to /storage/minminluo/yufengting/D19-12513.possorted_bam.bam.synapse_download_128421089
Downloading [--------------------]0.06% 8.0MB/12.3GB (116.9kB/s) D19-12513.possorted_bam.bam.synapse_download_128421089 2024-01-09 17:46:12,383 [client:2655 - DEBUG]:
Retrying download on error: [] after progressing 0 bytes
Traceback (most recent call last):
File "/storage/minminluo/software/anaconda3/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/core/multithread_download/download_threads.py", line 366, in _get_response_with_retry
raise SynapseError(
synapseclient.core.exceptions.SynapseError: Could not download the file: D19-12513.possorted_bam.bam, please try again.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/client.py", line 2631, in _downloadFileHandle
downloaded_path = self._download_from_url_multi_threaded(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/storage/minminluo/software/anaconda3/lib/python3.11/contextlib.py", line 81, in inner
return func(*args, **kwds)
^^^^^^^^^^^^^^^^^^^
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/client.py", line 2682, in _download_from_url_multi_threaded
multithread_download.download_file(self, request)
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/core/multithread_download/download_threads.py", line 247, in download_file
downloader.download_file(download_request)
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/core/multithread_download/download_threads.py", line 323, in download_file
self._check_for_errors(request, completed_futures)
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/core/multithread_download/download_threads.py", line 426, in _check_for_errors
raise ValueError(
ValueError: Failed downloading syn52404044 to /storage/minminluo/yufengting/D19-12513.possorted_bam.bam.synapse_download_128421089
Traceback (most recent call last):
File "/storage/minminluo/software/anaconda3/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/core/multithread_download/download_threads.py", line 366, in _get_response_with_retry
raise SynapseError(
synapseclient.core.exceptions.SynapseError: Could not download the file: D19-12513.possorted_bam.bam, please try again.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/storage/minminluo/software/anaconda3/bin/synapse", line 8, in
sys.exit(main())
^^^^^^
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/__main__.py", line 1889, in main
perform_main(args, syn)
File "/storage/minminluo/software/anaconda3/lib/python3.11/contextlib.py", line 81, in inner
return func(*args, **kwds)
^^^^^^^^^^^^^^^^^^^
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/__main__.py", line 1754, in perform_main
args.func(args, syn)
File "/storage/minminluo/software/anaconda3/lib/python3.11/contextlib.py", line 81, in inner
return func(*args, **kwds)
^^^^^^^^^^^^^^^^^^^
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/__main__.py", line 147, in get
entity = syn.get(
^^^^^^^^
File "/storage/minminluo/software/anaconda3/lib/python3.11/contextlib.py", line 81, in inner
return func(*args, **kwds)
^^^^^^^^^^^^^^^^^^^
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/client.py", line 977, in get
return_data = self._getWithEntityBundle(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/client.py", line 1136, in _getWithEntityBundle
self._download_file_entity(
File "/storage/minminluo/software/anaconda3/lib/python3.11/contextlib.py", line 81, in inner
return func(*args, **kwds)
^^^^^^^^^^^^^^^^^^^
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/client.py", line 1232, in _download_file_entity
downloadPath = self._downloadFileHandle(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/storage/minminluo/software/anaconda3/lib/python3.11/contextlib.py", line 81, in inner
return func(*args, **kwds)
^^^^^^^^^^^^^^^^^^^
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/client.py", line 2631, in _downloadFileHandle
downloaded_path = self._download_from_url_multi_threaded(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/storage/minminluo/software/anaconda3/lib/python3.11/contextlib.py", line 81, in inner
return func(*args, **kwds)
^^^^^^^^^^^^^^^^^^^
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/client.py", line 2682, in _download_from_url_multi_threaded
multithread_download.download_file(self, request)
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/core/multithread_download/download_threads.py", line 247, in download_file
downloader.download_file(download_request)
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/core/multithread_download/download_threads.py", line 323, in download_file
self._check_for_errors(request, completed_futures)
File "/storage/minminluo/software/anaconda3/lib/python3.11/site-packages/synapseclient/core/multithread_download/download_threads.py", line 426, in _check_for_errors
raise ValueError(
ValueError: Failed downloading syn52404044 to /storage/minminluo/yufengting/D19-12513.possorted_bam.bam.synapse_download_128421089
it is kind of the same like before. Just telling 'Could not download the file'. Please let me know what i can do to fix this. Hello @minminluo ,
Sorry for the trouble! Could you try and download one file and let me know the results?
```
synapse get syn52404044
```
Similarly, I notice that you are using your password to login still, please follow instructions [here](https://python-docs.synapse.org/tutorials/authentication/) to leverage a personal access token. Using your password will stop your download after a small period of time (~12 hours) as your session will expire.
Best,
Tom
Drop files to upload
Synapse client keep failing download page is loading…