STDERR: 2021-08-20T18:56:03.023122947Z 3aba68878ed9 2021-08-20 18:56:03,022 MainThread INFO cwltool: Resolved '/var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/brats-dream-challenge-infra-main/validation-phase-workflow.cwl' to 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/brats-dream-challenge-infra-main/validation-phase-workflow.cwl' STDERR: 2021-08-20T18:56:06.167901304Z 3aba68878ed9 2021-08-20 18:56:06,167 MainThread WARNING cwltool: Workflow checker warning: STDERR: 2021-08-20T18:56:06.167930818Z brats-dream-challenge-infra-main/validation-phase-workflow.cwl:55:9: Source 'filepath' of type STDERR: 2021-08-20T18:56:06.167933977Z ["null", "File"] may be STDERR: 2021-08-20T18:56:06.167936864Z incompatible STDERR: 2021-08-20T18:56:06.167939630Z brats-dream-challenge-infra-main/validation-phase-workflow.cwl:135:9: with sink 'input_file' of STDERR: 2021-08-20T18:56:06.167942205Z type "File" STDERR: 2021-08-20T18:56:06.167944811Z brats-dream-challenge-infra-main/validation-phase-workflow.cwl:55:9: Source 'filepath' of type STDERR: 2021-08-20T18:56:06.167947386Z ["null", "File"] may be STDERR: 2021-08-20T18:56:06.167949783Z incompatible STDERR: 2021-08-20T18:56:06.167952365Z brats-dream-challenge-infra-main/validation-phase-workflow.cwl:75:9: with sink 'input_file' of STDERR: 2021-08-20T18:56:06.167954789Z type "File" STDERR: 2021-08-20T18:56:08.498598014Z 3aba68878ed9 2021-08-20 18:56:08,498 MainThread INFO toil: Running Toil version 4.1.0-5ad5e77d98e1456b4f70f5b00e688a43cdce2ebe. STDERR: 2021-08-20T18:56:09.340556781Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/node-7f8bfa2a-953b-46b7-82aa-a245ed5dc070-16349ee66b573b0d6cff3dbc6912349f/tmp03qb7v4r/worker_log.txt STDERR: 2021-08-20T18:56:09.636183535Z 3aba68878ed9 2021-08-20 18:56:09,635 MainThread INFO toil.leader: 0 jobs are running, 0 jobs are issued and waiting to run STDERR: 2021-08-20T18:56:09.636700492Z 3aba68878ed9 2021-08-20 18:56:09,636 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/get_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_get_submission.cwl/instance-6a16uu9q with job batch system ID: 1 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-20T18:56:09.636754473Z 3aba68878ed9 2021-08-20 18:56:09,636 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/set_permissions.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_set_permissions.cwl/instance-0t9vl6jq with job batch system ID: 2 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-20T18:56:09.636831904Z 3aba68878ed9 2021-08-20 18:56:09,636 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks-Workflows/cwl-tool-synapseclient/v1.4/cwl/synapse-get-tool.cwl#synapse-get' synapse kind-https_raw.githubusercontent.com_Sage-Bionetworks-Workflows_cwl-tool-synapseclient_v1.4_cwl_synapse-get-tool.cwl_synapse-get/instance-cno425h_ with job batch system ID: 3 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-20T18:56:10.518394515Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/node-7f8bfa2a-953b-46b7-82aa-a245ed5dc070-16349ee66b573b0d6cff3dbc6912349f/tmp7ufy_n8c/worker_log.txt STDERR: 2021-08-20T18:56:10.532003897Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/node-7f8bfa2a-953b-46b7-82aa-a245ed5dc070-16349ee66b573b0d6cff3dbc6912349f/tmpf7x4qqo2/worker_log.txt STDERR: 2021-08-20T18:56:10.569624179Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/node-7f8bfa2a-953b-46b7-82aa-a245ed5dc070-16349ee66b573b0d6cff3dbc6912349f/tmponezafrg/worker_log.txt STDERR: 2021-08-20T18:57:20.920180512Z 3aba68878ed9 2021-08-20 18:57:20,918 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks-Workflows/cwl-tool-synapseclient/v1.4/cwl/synapse-get-tool.cwl#synapse-get' synapse kind-https_raw.githubusercontent.com_Sage-Bionetworks-Workflows_cwl-tool-synapseclient_v1.4_cwl_synapse-get-tool.cwl_synapse-get/instance-cno425h_ STDERR: 2021-08-20T18:57:33.193906796Z 3aba68878ed9 2021-08-20 18:57:33,193 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/set_permissions.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_set_permissions.cwl/instance-0t9vl6jq STDERR: 2021-08-20T18:57:37.294414194Z 3aba68878ed9 2021-08-20 18:57:37,294 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/get_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_get_submission.cwl/instance-6a16uu9q STDERR: 2021-08-20T18:57:37.295123348Z 3aba68878ed9 2021-08-20 18:57:37,294 MainThread INFO toil.leader: Issued job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/brats-dream-challenge-infra-main/validate.cwl' validate.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_3bf8a211-954d-4c33-91eb-10bd38a9cab7_brats-dream-challenge-infra-main_validate.cwl/instance-ch7wcpd_ with job batch system ID: 4 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-20T18:57:37.994278271Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/node-7f8bfa2a-953b-46b7-82aa-a245ed5dc070-16349ee66b573b0d6cff3dbc6912349f/tmp2o6cq_y1/worker_log.txt STDERR: 2021-08-20T18:58:10.923606915Z 3aba68878ed9 2021-08-20 18:58:10,923 MainThread INFO toil.leader: Job ended: 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/brats-dream-challenge-infra-main/validate.cwl' validate.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_3bf8a211-954d-4c33-91eb-10bd38a9cab7_brats-dream-challenge-infra-main_validate.cwl/instance-ch7wcpd_ STDERR: 2021-08-20T18:58:10.924266492Z 3aba68878ed9 2021-08-20 18:58:10,924 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/validate_email.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_validate_email.cwl/instance-m89lws9d with job batch system ID: 5 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-20T18:58:10.924326601Z 3aba68878ed9 2021-08-20 18:58:10,924 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_annotate_submission.cwl/instance-ya0n06eo with job batch system ID: 6 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-20T18:58:11.866141976Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/node-7f8bfa2a-953b-46b7-82aa-a245ed5dc070-16349ee66b573b0d6cff3dbc6912349f/tmp5zqh2wv6/worker_log.txt STDERR: 2021-08-20T18:58:11.878686270Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/node-7f8bfa2a-953b-46b7-82aa-a245ed5dc070-16349ee66b573b0d6cff3dbc6912349f/tmpitlqh_fd/worker_log.txt STDERR: 2021-08-20T18:59:09.913416030Z 3aba68878ed9 2021-08-20 18:59:09,912 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_annotate_submission.cwl/instance-ya0n06eo STDERR: 2021-08-20T18:59:13.691885338Z 3aba68878ed9 2021-08-20 18:59:13,691 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/validate_email.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_validate_email.cwl/instance-m89lws9d STDERR: 2021-08-20T18:59:13.692371401Z 3aba68878ed9 2021-08-20 18:59:13,692 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-4ldgoofe with job batch system ID: 7 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-20T18:59:14.748157159Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/node-7f8bfa2a-953b-46b7-82aa-a245ed5dc070-16349ee66b573b0d6cff3dbc6912349f/tmp6iw7xfgj/worker_log.txt STDERR: 2021-08-20T18:59:15.107307267Z 3aba68878ed9 2021-08-20 18:59:15,106 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-4ldgoofe STDERR: 2021-08-20T18:59:15.108052391Z 3aba68878ed9 2021-08-20 18:59:15,107 MainThread INFO toil.leader: Issued job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/brats-dream-challenge-infra-main/score.cwl' score.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_3bf8a211-954d-4c33-91eb-10bd38a9cab7_brats-dream-challenge-infra-main_score.cwl/instance-8o9_glg5 with job batch system ID: 8 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-20T18:59:15.789725929Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/node-7f8bfa2a-953b-46b7-82aa-a245ed5dc070-16349ee66b573b0d6cff3dbc6912349f/tmp5fspo6kb/worker_log.txt STDERR: 2021-08-20T19:00:46.039953472Z 3aba68878ed9 2021-08-20 19:00:46,039 MainThread INFO toil.leader: Job ended: 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/brats-dream-challenge-infra-main/score.cwl' score.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_3bf8a211-954d-4c33-91eb-10bd38a9cab7_brats-dream-challenge-infra-main_score.cwl/instance-8o9_glg5 STDERR: 2021-08-20T19:00:46.040475285Z 3aba68878ed9 2021-08-20 19:00:46,040 MainThread INFO toil.leader: Issued job 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/brats-dream-challenge-infra-main/score_email.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_3bf8a211-954d-4c33-91eb-10bd38a9cab7_brats-dream-challenge-infra-main_score_email.cwl/instance-ztd1f476 with job batch system ID: 9 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-20T19:00:46.040530798Z 3aba68878ed9 2021-08-20 19:00:46,040 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_annotate_submission.cwl/instance-hz65dpjv with job batch system ID: 10 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-20T19:00:46.705352422Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/node-7f8bfa2a-953b-46b7-82aa-a245ed5dc070-16349ee66b573b0d6cff3dbc6912349f/tmpisszt8ol/worker_log.txt STDERR: 2021-08-20T19:00:46.839907292Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/node-7f8bfa2a-953b-46b7-82aa-a245ed5dc070-16349ee66b573b0d6cff3dbc6912349f/tmpeily9nve/worker_log.txt STDERR: 2021-08-20T19:01:58.789687721Z 3aba68878ed9 2021-08-20 19:01:58,789 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_annotate_submission.cwl/instance-hz65dpjv STDERR: 2021-08-20T19:02:07.859573248Z 3aba68878ed9 2021-08-20 19:02:07,859 MainThread INFO toil.leader: Job ended: 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/brats-dream-challenge-infra-main/score_email.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_3bf8a211-954d-4c33-91eb-10bd38a9cab7_brats-dream-challenge-infra-main_score_email.cwl/instance-ztd1f476 STDERR: 2021-08-20T19:02:07.860119047Z 3aba68878ed9 2021-08-20 19:02:07,859 MainThread INFO toil.leader: Issued job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-vmq32hng with job batch system ID: 11 and cores: 1, disk: 1.0 G, and memory: 100.0 M STDERR: 2021-08-20T19:02:08.612933617Z INFO:toil.worker:Redirecting logging to /var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/node-7f8bfa2a-953b-46b7-82aa-a245ed5dc070-16349ee66b573b0d6cff3dbc6912349f/tmpznthr1yy/worker_log.txt STDERR: 2021-08-20T19:02:08.871579165Z 3aba68878ed9 2021-08-20 19:02:08,871 MainThread INFO toil.leader: Job ended: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-vmq32hng STDERR: 2021-08-20T19:02:08.871980877Z 3aba68878ed9 2021-08-20 19:02:08,871 MainThread WARNING toil.leader: The job seems to have left a log file, indicating failure: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-vmq32hng STDERR: 2021-08-20T19:02:08.872109504Z 3aba68878ed9 2021-08-20 19:02:08,872 MainThread WARNING toil.leader: Log from job kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-vmq32hng follows: STDERR: 2021-08-20T19:02:08.872116776Z =========> STDERR: 2021-08-20T19:02:08.872120198Z INFO:toil.worker:---TOIL WORKER OUTPUT LOG--- STDERR: 2021-08-20T19:02:08.872123229Z INFO:toil:Running Toil version 4.1.0-5ad5e77d98e1456b4f70f5b00e688a43cdce2ebe. STDERR: 2021-08-20T19:02:08.872126254Z Failed to evaluate expression: STDERR: 2021-08-20T19:02:08.872129071Z Expression evaluation error: STDERR: 2021-08-20T19:02:08.872131888Z Expecting value: line 1 column 1 (char 0) STDERR: 2021-08-20T19:02:08.872134663Z script was: STDERR: 2021-08-20T19:02:08.872137339Z 01 "use strict"; STDERR: 2021-08-20T19:02:08.872140226Z 02 var inputs = { STDERR: 2021-08-20T19:02:08.872142910Z 03 "status": "INVALID", STDERR: 2021-08-20T19:02:08.872145945Z 04 "previous_annotation_finished": true, STDERR: 2021-08-20T19:02:08.872149065Z 05 "previous_email_finished": true STDERR: 2021-08-20T19:02:08.872151859Z 06 }; STDERR: 2021-08-20T19:02:08.872154678Z 07 var self = null; STDERR: 2021-08-20T19:02:08.872157410Z 08 var runtime = { STDERR: 2021-08-20T19:02:08.872160237Z 09 "cores": 1, STDERR: 2021-08-20T19:02:08.872162942Z 10 "ram": 100.0, STDERR: 2021-08-20T19:02:08.872165755Z 11 "tmpdirSize": 1024, STDERR: 2021-08-20T19:02:08.872168986Z 12 "outdirSize": 0.95367431640625, STDERR: 2021-08-20T19:02:08.872171785Z 13 "tmpdir": "/var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/tmpa6fulw8m", STDERR: 2021-08-20T19:02:08.872174999Z 14 "outdir": "/var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/node-7f8bfa2a-953b-46b7-82aa-a245ed5dc070-16349ee66b573b0d6cff3dbc6912349f/tmpznthr1yy/133b821b-5934-4764-9238-ae23059faa81/t7wthzac3/tmp-outrxcjxjyv" STDERR: 2021-08-20T19:02:08.872178786Z 15 }; STDERR: 2021-08-20T19:02:08.872183192Z 16 (function(){ STDERR: 2021-08-20T19:02:08.872186220Z 17 if(inputs.status == "INVALID"){ STDERR: 2021-08-20T19:02:08.872189205Z 18 throw 'invalid submission'; STDERR: 2021-08-20T19:02:08.872191900Z 19 } else { STDERR: 2021-08-20T19:02:08.872194694Z 20 return {finished: true}; STDERR: 2021-08-20T19:02:08.872197465Z 21 } STDERR: 2021-08-20T19:02:08.872200065Z 22 })() STDERR: 2021-08-20T19:02:08.872202766Z stdout was: '' STDERR: 2021-08-20T19:02:08.872210180Z stderr was: 'invalid submission' STDERR: 2021-08-20T19:02:08.872213029Z STDERR: 2021-08-20T19:02:08.872215640Z WARNING:cwltool:Failed to evaluate expression: STDERR: 2021-08-20T19:02:08.872218455Z Expression evaluation error: STDERR: 2021-08-20T19:02:08.872221488Z Expecting value: line 1 column 1 (char 0) STDERR: 2021-08-20T19:02:08.872224380Z script was: STDERR: 2021-08-20T19:02:08.872227010Z 01 "use strict"; STDERR: 2021-08-20T19:02:08.872229712Z 02 var inputs = { STDERR: 2021-08-20T19:02:08.872232478Z 03 "status": "INVALID", STDERR: 2021-08-20T19:02:08.872235298Z 04 "previous_annotation_finished": true, STDERR: 2021-08-20T19:02:08.872238230Z 05 "previous_email_finished": true STDERR: 2021-08-20T19:02:08.872241123Z 06 }; STDERR: 2021-08-20T19:02:08.872243698Z 07 var self = null; STDERR: 2021-08-20T19:02:08.872246320Z 08 var runtime = { STDERR: 2021-08-20T19:02:08.872248886Z 09 "cores": 1, STDERR: 2021-08-20T19:02:08.872251685Z 10 "ram": 100.0, STDERR: 2021-08-20T19:02:08.872254362Z 11 "tmpdirSize": 1024, STDERR: 2021-08-20T19:02:08.872257056Z 12 "outdirSize": 0.95367431640625, STDERR: 2021-08-20T19:02:08.872259883Z 13 "tmpdir": "/var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/tmpa6fulw8m", STDERR: 2021-08-20T19:02:08.872263494Z 14 "outdir": "/var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/node-7f8bfa2a-953b-46b7-82aa-a245ed5dc070-16349ee66b573b0d6cff3dbc6912349f/tmpznthr1yy/133b821b-5934-4764-9238-ae23059faa81/t7wthzac3/tmp-outrxcjxjyv" STDERR: 2021-08-20T19:02:08.872266922Z 15 }; STDERR: 2021-08-20T19:02:08.872269630Z 16 (function(){ STDERR: 2021-08-20T19:02:08.872272288Z 17 if(inputs.status == "INVALID"){ STDERR: 2021-08-20T19:02:08.872286345Z 18 throw 'invalid submission'; STDERR: 2021-08-20T19:02:08.872289485Z 19 } else { STDERR: 2021-08-20T19:02:08.872292107Z 20 return {finished: true}; STDERR: 2021-08-20T19:02:08.872294908Z 21 } STDERR: 2021-08-20T19:02:08.872297516Z 22 })() STDERR: 2021-08-20T19:02:08.872300086Z stdout was: '' STDERR: 2021-08-20T19:02:08.872302704Z stderr was: 'invalid submission' STDERR: 2021-08-20T19:02:08.872305415Z STDERR: 2021-08-20T19:02:08.872308155Z Traceback (most recent call last): STDERR: 2021-08-20T19:02:08.872310951Z File "/usr/local/lib/python3.6/site-packages/toil/worker.py", line 366, in workerScript STDERR: 2021-08-20T19:02:08.872314287Z job._runner(jobGraph=jobGraph, jobStore=jobStore, fileStore=fileStore, defer=defer) STDERR: 2021-08-20T19:02:08.872317294Z File "/usr/local/lib/python3.6/site-packages/toil/job.py", line 1392, in _runner STDERR: 2021-08-20T19:02:08.872320256Z returnValues = self._run(jobGraph, fileStore) STDERR: 2021-08-20T19:02:08.872325923Z File "/usr/local/lib/python3.6/site-packages/toil/job.py", line 1329, in _run STDERR: 2021-08-20T19:02:08.872329202Z return self.run(fileStore) STDERR: 2021-08-20T19:02:08.872331950Z File "/usr/local/lib/python3.6/site-packages/toil/cwl/cwltoil.py", line 937, in run STDERR: 2021-08-20T19:02:08.872335012Z raise cwltool.errors.WorkflowException(status) STDERR: 2021-08-20T19:02:08.872337752Z cwltool.errors.WorkflowException: permanentFail STDERR: 2021-08-20T19:02:08.872340520Z ERROR:toil.worker:Exiting the worker because of a failed job on host 3aba68878ed9 STDERR: 2021-08-20T19:02:08.872343578Z WARNING:toil.jobGraph:Due to failure we are reducing the remaining retry count of job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-vmq32hng with ID kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-vmq32hng to 0 STDERR: 2021-08-20T19:02:08.872348018Z <========= STDERR: 2021-08-20T19:02:08.872354703Z 3aba68878ed9 2021-08-20 19:02:08,872 MainThread WARNING toil.leader: Job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-vmq32hng with ID kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-vmq32hng is completely failed STDERR: 2021-08-20T19:02:26.039548543Z 3aba68878ed9 2021-08-20 19:02:26,039 MainThread INFO toil.leader: Finished toil run with 11 failed jobs. STDERR: 2021-08-20T19:02:26.039630388Z 3aba68878ed9 2021-08-20 19:02:26,039 MainThread INFO toil.leader: Failed jobs at end of the run: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-4ldgoofe 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/brats-dream-challenge-infra-main/score_email.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_3bf8a211-954d-4c33-91eb-10bd38a9cab7_brats-dream-challenge-infra-main_score_email.cwl/instance-ztd1f476 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/brats-dream-challenge-infra-main/validate.cwl' validate.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_3bf8a211-954d-4c33-91eb-10bd38a9cab7_brats-dream-challenge-infra-main_validate.cwl/instance-ch7wcpd_ 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-vmq32hng 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_annotate_submission.cwl/instance-hz65dpjv 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/validate_email.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_validate_email.cwl/instance-m89lws9d 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/get_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_get_submission.cwl/instance-6a16uu9q 'CWLWorkflow' kind-CWLWorkflow/instance-ejatbv3a 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/brats-dream-challenge-infra-main/score.cwl' score.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_3bf8a211-954d-4c33-91eb-10bd38a9cab7_brats-dream-challenge-infra-main_score.cwl/instance-8o9_glg5 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_annotate_submission.cwl/instance-ya0n06eo 'https://raw.githubusercontent.com/Sage-Bionetworks-Workflows/cwl-tool-synapseclient/v1.4/cwl/synapse-get-tool.cwl#synapse-get' synapse kind-https_raw.githubusercontent.com_Sage-Bionetworks-Workflows_cwl-tool-synapseclient_v1.4_cwl_synapse-get-tool.cwl_synapse-get/instance-cno425h_ STDERR: 2021-08-20T19:02:26.050847858Z Traceback (most recent call last): STDERR: 2021-08-20T19:02:26.050874188Z File "/usr/local/bin/toil-cwl-runner", line 8, in STDERR: 2021-08-20T19:02:26.050877885Z sys.exit(main()) STDERR: 2021-08-20T19:02:26.050880509Z File "/usr/local/lib/python3.6/site-packages/toil/cwl/cwltoil.py", line 1691, in main STDERR: 2021-08-20T19:02:26.050883520Z outobj = toil.start(wf1) STDERR: 2021-08-20T19:02:26.050886201Z File "/usr/local/lib/python3.6/site-packages/toil/common.py", line 829, in start STDERR: 2021-08-20T19:02:26.050890447Z return self._runMainLoop(rootJobGraph) STDERR: 2021-08-20T19:02:26.050893022Z File "/usr/local/lib/python3.6/site-packages/toil/common.py", line 1120, in _runMainLoop STDERR: 2021-08-20T19:02:26.050895845Z jobCache=self._jobCache).run() STDERR: 2021-08-20T19:02:26.050898588Z File "/usr/local/lib/python3.6/site-packages/toil/leader.py", line 269, in run STDERR: 2021-08-20T19:02:26.050901374Z raise FailedJobsException(self.config.jobStore, self.toilState.totalFailedJobs, self.jobStore) STDERR: 2021-08-20T19:02:26.050905652Z toil.leader.FailedJobsException: The job store 'file:/var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/tmp4myyaw__' contains 11 failed jobs: 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-4ldgoofe, 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/brats-dream-challenge-infra-main/score_email.cwl' python3 kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_3bf8a211-954d-4c33-91eb-10bd38a9cab7_brats-dream-challenge-infra-main_score_email.cwl/instance-ztd1f476, 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/brats-dream-challenge-infra-main/validate.cwl' validate.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_3bf8a211-954d-4c33-91eb-10bd38a9cab7_brats-dream-challenge-infra-main_validate.cwl/instance-ch7wcpd_, 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-vmq32hng, 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_annotate_submission.cwl/instance-hz65dpjv, 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/validate_email.cwl' python3 kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_validate_email.cwl/instance-m89lws9d, 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/get_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_get_submission.cwl/instance-6a16uu9q, 'CWLWorkflow' kind-CWLWorkflow/instance-ejatbv3a, 'file:///var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/brats-dream-challenge-infra-main/score.cwl' score.py kind-file_var_lib_docker_volumes_workflow_orchestrator_shared__data_3bf8a211-954d-4c33-91eb-10bd38a9cab7_brats-dream-challenge-infra-main_score.cwl/instance-8o9_glg5, 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/annotate_submission.cwl' challengeutils kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_annotate_submission.cwl/instance-ya0n06eo, 'https://raw.githubusercontent.com/Sage-Bionetworks-Workflows/cwl-tool-synapseclient/v1.4/cwl/synapse-get-tool.cwl#synapse-get' synapse kind-https_raw.githubusercontent.com_Sage-Bionetworks-Workflows_cwl-tool-synapseclient_v1.4_cwl_synapse-get-tool.cwl_synapse-get/instance-cno425h_ STDERR: 2021-08-20T19:02:26.050927191Z Log from job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-vmq32hng follows: STDERR: 2021-08-20T19:02:26.050930366Z =========> STDERR: 2021-08-20T19:02:26.050932852Z INFO:toil.worker:---TOIL WORKER OUTPUT LOG--- STDERR: 2021-08-20T19:02:26.050935378Z INFO:toil:Running Toil version 4.1.0-5ad5e77d98e1456b4f70f5b00e688a43cdce2ebe. STDERR: 2021-08-20T19:02:26.050938152Z Failed to evaluate expression: STDERR: 2021-08-20T19:02:26.050940905Z Expression evaluation error: STDERR: 2021-08-20T19:02:26.050943421Z Expecting value: line 1 column 1 (char 0) STDERR: 2021-08-20T19:02:26.050945911Z script was: STDERR: 2021-08-20T19:02:26.050948435Z 01 "use strict"; STDERR: 2021-08-20T19:02:26.050950920Z 02 var inputs = { STDERR: 2021-08-20T19:02:26.050953408Z 03 "status": "INVALID", STDERR: 2021-08-20T19:02:26.050955900Z 04 "previous_annotation_finished": true, STDERR: 2021-08-20T19:02:26.050958531Z 05 "previous_email_finished": true STDERR: 2021-08-20T19:02:26.050961182Z 06 }; STDERR: 2021-08-20T19:02:26.050963571Z 07 var self = null; STDERR: 2021-08-20T19:02:26.050965982Z 08 var runtime = { STDERR: 2021-08-20T19:02:26.050968318Z 09 "cores": 1, STDERR: 2021-08-20T19:02:26.050970796Z 10 "ram": 100.0, STDERR: 2021-08-20T19:02:26.050973195Z 11 "tmpdirSize": 1024, STDERR: 2021-08-20T19:02:26.050975907Z 12 "outdirSize": 0.95367431640625, STDERR: 2021-08-20T19:02:26.050978427Z 13 "tmpdir": "/var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/tmpa6fulw8m", STDERR: 2021-08-20T19:02:26.050983559Z 14 "outdir": "/var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/node-7f8bfa2a-953b-46b7-82aa-a245ed5dc070-16349ee66b573b0d6cff3dbc6912349f/tmpznthr1yy/133b821b-5934-4764-9238-ae23059faa81/t7wthzac3/tmp-outrxcjxjyv" STDERR: 2021-08-20T19:02:26.050986756Z 15 }; STDERR: 2021-08-20T19:02:26.050989084Z 16 (function(){ STDERR: 2021-08-20T19:02:26.050991628Z 17 if(inputs.status == "INVALID"){ STDERR: 2021-08-20T19:02:26.050994188Z 18 throw 'invalid submission'; STDERR: 2021-08-20T19:02:26.050996612Z 19 } else { STDERR: 2021-08-20T19:02:26.050999037Z 20 return {finished: true}; STDERR: 2021-08-20T19:02:26.051001529Z 21 } STDERR: 2021-08-20T19:02:26.051003887Z 22 })() STDERR: 2021-08-20T19:02:26.051006262Z stdout was: '' STDERR: 2021-08-20T19:02:26.051008669Z stderr was: 'invalid submission' STDERR: 2021-08-20T19:02:26.051011123Z STDERR: 2021-08-20T19:02:26.051013448Z WARNING:cwltool:Failed to evaluate expression: STDERR: 2021-08-20T19:02:26.051015900Z Expression evaluation error: STDERR: 2021-08-20T19:02:26.051018336Z Expecting value: line 1 column 1 (char 0) STDERR: 2021-08-20T19:02:26.051020800Z script was: STDERR: 2021-08-20T19:02:26.051023152Z 01 "use strict"; STDERR: 2021-08-20T19:02:26.051025572Z 02 var inputs = { STDERR: 2021-08-20T19:02:26.051027966Z 03 "status": "INVALID", STDERR: 2021-08-20T19:02:26.051030446Z 04 "previous_annotation_finished": true, STDERR: 2021-08-20T19:02:26.051032972Z 05 "previous_email_finished": true STDERR: 2021-08-20T19:02:26.051035557Z 06 }; STDERR: 2021-08-20T19:02:26.051037871Z 07 var self = null; STDERR: 2021-08-20T19:02:26.051040230Z 08 var runtime = { STDERR: 2021-08-20T19:02:26.051042575Z 09 "cores": 1, STDERR: 2021-08-20T19:02:26.051044990Z 10 "ram": 100.0, STDERR: 2021-08-20T19:02:26.051047412Z 11 "tmpdirSize": 1024, STDERR: 2021-08-20T19:02:26.051049914Z 12 "outdirSize": 0.95367431640625, STDERR: 2021-08-20T19:02:26.051052397Z 13 "tmpdir": "/var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/tmpa6fulw8m", STDERR: 2021-08-20T19:02:26.051055260Z 14 "outdir": "/var/lib/docker/volumes/workflow_orchestrator_shared/_data/3bf8a211-954d-4c33-91eb-10bd38a9cab7/node-7f8bfa2a-953b-46b7-82aa-a245ed5dc070-16349ee66b573b0d6cff3dbc6912349f/tmpznthr1yy/133b821b-5934-4764-9238-ae23059faa81/t7wthzac3/tmp-outrxcjxjyv" STDERR: 2021-08-20T19:02:26.051058361Z 15 }; STDERR: 2021-08-20T19:02:26.051060651Z 16 (function(){ STDERR: 2021-08-20T19:02:26.051063026Z 17 if(inputs.status == "INVALID"){ STDERR: 2021-08-20T19:02:26.051065600Z 18 throw 'invalid submission'; STDERR: 2021-08-20T19:02:26.051068000Z 19 } else { STDERR: 2021-08-20T19:02:26.051070472Z 20 return {finished: true}; STDERR: 2021-08-20T19:02:26.051080583Z 21 } STDERR: 2021-08-20T19:02:26.051082997Z 22 })() STDERR: 2021-08-20T19:02:26.051085361Z stdout was: '' STDERR: 2021-08-20T19:02:26.051087761Z stderr was: 'invalid submission' STDERR: 2021-08-20T19:02:26.051090317Z STDERR: 2021-08-20T19:02:26.051092887Z Traceback (most recent call last): STDERR: 2021-08-20T19:02:26.051095313Z File "/usr/local/lib/python3.6/site-packages/toil/worker.py", line 366, in workerScript STDERR: 2021-08-20T19:02:26.051098033Z job._runner(jobGraph=jobGraph, jobStore=jobStore, fileStore=fileStore, defer=defer) STDERR: 2021-08-20T19:02:26.051100626Z File "/usr/local/lib/python3.6/site-packages/toil/job.py", line 1392, in _runner STDERR: 2021-08-20T19:02:26.051103266Z returnValues = self._run(jobGraph, fileStore) STDERR: 2021-08-20T19:02:26.051105779Z File "/usr/local/lib/python3.6/site-packages/toil/job.py", line 1329, in _run STDERR: 2021-08-20T19:02:26.051108348Z return self.run(fileStore) STDERR: 2021-08-20T19:02:26.051110793Z File "/usr/local/lib/python3.6/site-packages/toil/cwl/cwltoil.py", line 937, in run STDERR: 2021-08-20T19:02:26.051113395Z raise cwltool.errors.WorkflowException(status) STDERR: 2021-08-20T19:02:26.051115826Z cwltool.errors.WorkflowException: permanentFail STDERR: 2021-08-20T19:02:26.051118411Z ERROR:toil.worker:Exiting the worker because of a failed job on host 3aba68878ed9 STDERR: 2021-08-20T19:02:26.051121034Z WARNING:toil.jobGraph:Due to failure we are reducing the remaining retry count of job 'https://raw.githubusercontent.com/Sage-Bionetworks/ChallengeWorkflowTemplates/v3.1/cwl/check_status.cwl' kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-vmq32hng with ID kind-https_raw.githubusercontent.com_Sage-Bionetworks_ChallengeWorkflowTemplates_v3.1_cwl_check_status.cwl/instance-vmq32hng to 0 STDERR: 2021-08-20T19:02:26.051124610Z <========= The submitted file : https://www.synapse.org/#!Synapse:syn26118296/files/