The Digital Mammography DREAM Challenge


Created By Thea Norman Thea

Deep Learning to Improve Breast Cancer Detection on Screening Mammography
Deep Neural Networks Improve Radiologists' Performance in Breast Cancer Screening
couldnt download pilot data
Postdoctoral researcher position in deep learning for medical image analysis at NYU
End-to-end training all convolutional design v2
End-to-end training all convolutional network with AUC: 0.91 on DDSM and 0.96 on INbreast
It says that they have chosen a winner from the challenge. Can a team still work on this and submit a solution?
Concerns about comparing the performance with radiologists
question about winners' writeups
About samples of metadata for london dataset
Ribli's Detection Network
About labeling of cancer exams and false negatives
Open question
suggestion about preprocessing process
Community Phase Baseline - averaging the predictions ?
Publishing the images
Command to retrive pushed images...
Open Source Breast MRI
Timeline - End of scoring period - 16th May
a question on training data distribution
Validation error in method write-up
How to download and upload Docker submissions?
No feedback of the write up
About submiting src
Deadline for write-up Monday, May 16 at 23:59:59 PDT.
Error in submission 9605233
Method write-up
When scores will be provided?
precision and confusion matrix
Postprocessing used on Mammograms
Error encountered during training
Congratulations to the winners of round 3!
DM Challenge: Please check that predictions are reproducible
Model State File Unlock
Images in validation rounds
submission 9604802 restarted
Repeated entries in inference submission
No progress % for SC2 express lane?
VALIDATED BUT NOT YET EXECUTED for about 20 hours...
Replace individual submission 9605169 with team submission?
Are servers getting slower than usual?
No log file generated
Inference Lane delays
kill submissions 9603296 and 9603095
Scores from Express Lane Inference Submissions
Submission 9603582 stuck?
Unlock model state
extremely slow docker upload
server busy ?
Inference Data in Validation Phase
The training log updated
I'm not getting the model state through email
Access to database infrastructure after competition ?
Frozen Submission?
Can you plot the GPU usage for submission 8623843
Invalid submission 8505126
Commit is not found in docker registry
Invalid signature model state
Round 3 Inference Submission was Mistakenly Counted Towards Validation Round
submission 8592460 sees wrong /preprocessedData content
Server problem?
submitted one job but got 3 in the queue
Unable to validate your submission to the Digital Mammography Prediction - Sub-Challenge 1 Express Lane
Submission 8504086
Speed difference between inference jobs
Possible to send private message to the organizers?
Can we have more submissions in the Final Round?
about images ID in validation phase
preprocessing rerun
Progress on last round 3 jobs
Express lane waiting time
Dash board is gone?
Training Job is not showed in dashboard
Timeline of the Final Round in Validation Phase
submission 8510126 is stuck, need help
Expired Download Token
Submission 8505031
Why my existing preprocessing was reruned?
inference submission failed
Can't access Model file
slow training and crash in inference
submission failed but OK
model state file NA associated with submission NA
Problems with CPU usage in inference
One column is missing from the image crosswalk in the test set
SC2 inference failed
stop a running submission
Is it possible to cancel one inference job that is already in progress?
Preprocessing in submission taking longer than usual
Is intermediary logs available?
Please kill Inference 8507924
No duplicated submissions allowed?
ModelState Download Failed
model state file NA associated with submission NA
Inference submission after 3rd round deadline.
The update cycle is abnormal.
exceded number of submisions
Start the training before the preprocessing completes
Several questions about Final Round
Please remove a job.
corrupted docker?
Please kill submission 8464238
Kill submission 8464317
Validation phase
Stuck in validated mode
training submission log not being updated
Job is not canceled
Scoring run timed out?
Express lane seriously lags
Preprocessing is Gone
Round 3 deadline
Is it ok to look at exam_meta.tsv as a log?
Training Quota
Inference Scratch Space
Inference submission times
Conformity of CCID, MLOID terminology in dataset for prosthesis exams ?
Should the leaderboard results be penalized for the number of submissions?
Preprocessing is GONE and rerunning
Interesting paper in European Congress of Radiology
Inference counting
Unlabeled training data
How many submissions can each team submit in the last round (scoring round)?
8370829 never progressed. i called cancel 2 hours ago never responded.
Error in multithreading using matlab 2014B
External data policy.
Clarification on external/private data
Invalid signature retrieving model state
Scoring script
System outage?
help with multiple gnu and theano
Scoring - what is better than chance
How can we test the new SC2 requirement?
Model State Files Missing
Cached file is in the directory, but can not be opened by training script
Delay in Express Lane
Express Lane Training Pre-Processing
Failed to pull the images in infernce express line
Invalid submission after scoring on corresponding Express Lanes
Progress on inference submissions
Team-up policy?
Finding the region of interest in a given image
about Feb 12, 2017 news letter
Logs for inference phase
Submission 8263435 logs
Number of training images?
Recieve the email said submission faild but dash board shows submission validated
More environment variables?
Inquiry on SC2 submission and /metadata/exams_metadata.tsv file
Training quota remaining is only "1D:19H:24M"?
Enquiry on SC1 Submission
submission limit question
Round 3
Cancelling submissions
What happens to the "VALIDATED BUT NOT YET EXECUTED" inference submissions now?
What "INVALID" and "VALIDATED BUT NOT YET EXECUTED" simultaneously mean?
Why was my job killed?
Submissions went smoothly
Please.. cancel all old submissions except last one
Submission was killed
More capacity?
can you please cancel submission 8135352?
Please delete my submission 8223598, Thanks!
Cancel Sub Challenge 2 submission
Please delete my submission id 8203512, because it does not pass the express line. Thanks!
TensorFlow version problem
servers stacked
Please kill submission 8212984 and 8209713.
Exponential values in predictions.tsv
Sub-challange 1 Express Lane shows very different score for the SAME model
Please kill/remove submissions 8136121, 8129849, 8135352
Please kill submission 8157778
Pixel values in source images
cancel a job on scoring lane?
Log file of (submission ID 8220999)
Express lane no longer recognizes my preprocessing Docker
Strange error...
Timeout Pushing Docker Image
can you please check sc2 express lane? it has been stuck for a day!
Submission 8160926 stuck in validated state
Cancel inference submission: got submitted twice
Model State exceeded 1GB per day quota: can I access it the next day
Log file of running job has not changed in the last 22 hours
Can I get the model state during the middle of a training?
kill scoring submission 8160085
could you kill my submission 8101074 please?
sc1 express lane has been 'RECEIVED' for a long time, can you check what happened?
**URGENT, PLEASE READ: **8115015 costed 12 hours with little log update for a very very long time
Submission on hold for two days - can you please check
Just to confirm, the age column is DEFINITELY at the 8th column right?
At what time does Round 2 close on Feb. 7?
Training submissions stuck with validated status
preprocessing is not allowd in a training submissionfile
Using matlab available on the server?
Sub-challenge1 wating time
Permission denied error
Cancel scoring submission
Inference phase disk limit
Which submissions count toward Round 2?
SC2 : chronological sort of previous exams ?
Running out of compute hour, can phase 3 and/or final round please double compute hour?
Can we use downloaded modelstate in later training submissions
Please what happens with this JOUB 8104082
How many images are in the leaderboard data set?
Can you please cancel submission 8118422 to SC1?
Job 8116168 stucked in validated state
Occur error in 'inference' phase relating 'SCORING' docker
Stalled preprocessing jobs?
completely stopped: my jobs today
validation error from the same code that was successful
Cancel request got stuck
No log update from running model
Gold Standard
Any scratch space available during Scoring?
Can we conveniently assume all .dcm have a depth of 12 bits?
Huge queue?
Preprocess chaining
preprocessing validated bot no log files
"Model State File" for full jobs
JUST to make sure, in the challenge dictionary are we supposed to use column 6 and 7 or column 8 and 9 as gold standard?
Invalid Docker Commit Error
Cannot start from pretrained model using Tensorflow 0.11.0, please help
preprocessing results in round 1 still available in round 2?
no full log from preprocessing and now stuck
discrepancy between pilot_data and real data
Sub-Challenge 2 Scoring - last update was 10 hours ago
Community Phase Milestones
Scoring Process
Express queue: problem exceeding alloted time
Inference scored in express lane SC1 but no output file in leaderboard SC1
Canceling training not working?
submission failed on prediction express lane
Why my job is killed?
Some thoughts for round 2
I/O Timeout Error
current lead time
Two exactly same submission, but one report ERR
computational quota
file access error, Express Training queue
No space left on device:
URGENT: 8044454 never progressed, i called cancel, never responded
Preprocessing started again, although same hash
Submissions waiting in VALIDATED STATE
NaN Issues
Tensorflow prediction code
Is there any chance that the server's capacity will be temporarily lifted to accommodate a lot of submissions?
Submission got stuck without any log file update
Just to make sure, can we use metadata extracted from DICOM header in sub-challenge 1?
evaluation code to how many digits
Log file recently stopped getting updated
Submission without log file
Round1 Result/LeaderBoard
2nd phase 336 hours or 712 hours training hours?
submitting as part of a team
Cancel Requested
Logfile not updated
Errors with no stacktrace
Strange CUDNN errors
Filesystem question
TensorFlow Code For Loading Pre-Trained Model Timed out in Express Lane
Access modelState File
Preprocessing already preprocessed data ?
Stuck in validation mode....
Is the SC1 express lane working?
Submission stuck in validated state for 8 hours?
Running Two TensorFlow Training Processes using Different GPU Cores
There must at least be one confidence value error
Does a model state file have to be obtained after a successful run?
Log file not being updated
Modifying preProcessed folder files
Why there is no leaderboard until now?
Do undownloaded model states count towards quota?
Where do I send the submission file?
Submission Failed with no STDERR on SC1 but was successful locally and on SC1 Express lane
IO Error on scoring SC-1
Validation error in submission to Digital Mammography Prediction - Sub-Challenge 1
Is the express lane closed during system maintenance too?
Received a strange error from scoring submission...
Round Two
Full Scoring SC-1 & Leaderboards
Errors - is there any hope?
Training submission - no termination error
Submission NOT Processes
Exception while scoring submission to Digital Mammography Prediction - Sub-Challenge 2
Good Luck - and Happy Christmas
Restarting a docker on the training lane
Exception while scoring submission to Digital Mammography Prediction - Sub-Challenge 1 and 2
Are we allowed to use the trained models for academic research purposes after the competition?
Training Requests Stuck at VALIDATED
No confirmation of submission
Custom cancel request - easy to do, please help
i submitted to sc1 scoring express lane (i think) but never received any response
Preprocessing seem being stuck for several days
What will happen the day after tomorrow?
outcomes for subchallenge 1 training? In metadata file?
Read image extremely slow.
Strange error: OpenBLAS blas_thread_init: pthread_create: Resource temporarily unavailable
Validation Express lane error
Cant find NVCC
syn7894342 not shown
How is the 1GB per day for modelState calculated?
Is /output the right place to write intermediate data during inference phase?
Why there is no email or feedback when I submitted several models to train?
confidence or predictions? What's the right name of the last column in predictions.tsv
Submission runs fine locally but stops in express lane
Can't get email when submit preprocessing model to express lane
Model state exceeded maximum size.
implementing modelState periodic update
clarification on 'the teams can continue to train their model for up to 336 hours and score it 3 more times.'
What's the "VALIDATED" status and why is my job stuck?
Validation error in submission to Digital Mammography Prediction - Sub-Challenge 1: SubjectIds must be part of the gold standard
How to cancel the running job?
"preprocessing is not allowed in a training submission file"
Submission in team : has a conflicting submission
Round 1 extension?
Preprocessing folder persistent and future suggestion
Time to complete preprocessing of the given docker images
Problem with upload image
Issue training on main training lane when model finishes fine on express lane.
Access to preprocessed information in the inference image
Submission failed in the express lane
Predictions file clarification
Scoring express lane no confirmation email or status update needs improvement
Inconsistent inference script name for Tensorflow example
Invalid literal error when parsing metadata file for Express Lane data
i submitted something to the test fast lane hours ago, and never received anything. does that mean my submission is successful?
Exception scoring submission Sub-Challenge 2 Express Lane: File "", line 222 not enough arguments for format string
can't download pilot images anymore!
a job has been hang for over 10 hours, no progress in log, no response to cancel, can you please terminate and deduct from quata?
dcm files can be found when running preprocessing pipeline
Training submission no response or updates
Webinar #2 Q&A: Can we use age information in first sub-challenge?
May be we need some priority queues for model training?
Our team have a weird error ("exec format error")
Submission Failed
GPUs unavailable?
Cannot connect to Docker daemon error
Job Status
Sub-challenge 2 express-lane scoring failed: training is not allowed
No saving preprocessed images in testing?
No email for submission to express training queue.
Signal 9 Termination
Our job (7837197) has no update for over 10 hours. Can you help check?
dm-preprocess-png fails with error message "exec user process caused "exec format error""
EVALUATION_IN_PROGRESS stuck and pending "Cancel Requested" - what to do?
submission quota exceeded
i encountered this error multiple times, each with different images, what could be the possible reason?
Questions related to Scoring submission, and test dataset
Replacing previous scoring submission within a round
Long queuing time-questions
Webinar #2 Q&A: Is it a problem if the model is not exactly deterministic?
Submission stucked (11 hours) on EVALUATION_IN_PROGRESS
"Permission denied" error during preprocessing
monitor submission in regular lane
Submission terminated by there is no error in log
modelState folder independence at different docker submissions
Webinar #2 Q&A: Where is located scratch space?
Accessing the already preprocessed data during training
Entry Point for Preproecssing and Training
detailed information about error in express lane
Webinar #2 Q&A: Could we install some python library in our preprocessing pipeline?
Webinar #2 Q&A: How can we optimize model configuration or hyper-parameters?
Webinar #2 Q&A: Can we use other type of deep learning frameworks in express lane?
Webinar #2 Q&A: Can I use pre-trained models like inception to do the training?
Webinar #2 Q&A: What are the baseline AUROC values?
Webinar #2 Q&A: Do you have docker for "torch" framework?
Webinar #2 Q&A: What data will your model have access to if you run it locally with nvidia-docker?
Webinar #2 Q&A: How can I have access to sample input data? i.e. images/metadata etc
Webinar #2 Q&A: Is it possible to use an ensemble of models?
Webinar #2 Q&A: The problem of preprocessing data
Webinar #2 Q&A: How can we change the random seed?
Webinar #2 Q&A: Can we run two docker containers at the same time in order to train in parallel?
Webinar #2 Q&A: Are the deadlines for submissions or finalized submissions?
Webinar #2 Q&A: Is the prediction submission time also reducing the computation time?
Webinar #2 Q&A: I can not test my program in test phase. Can I use your provided docker containers for caffe, torch or matlab?
Webinar #2 Q&A: Can the model state be used for the initial condition of the training?
Webinar #2 Q&A: Is the persistence of the previously preprocessed data is not guaranteed? If so, how can be work around this?
Webinar #2 Q&A: Can the content of /preprocessedData be updated, meaning adding new data and not deleting the old one?
Webinar #2 Q&A: Can you tell us an estimate of how long it takes to run the example docker containers?
Webinar #2 Q&A: Will the 14 day training time be enforced in the first round?
Webinar #2 Q&A: What happens with patients with only left or right image?
Webinar #2 Q&A: Will the time limits on training be extended?
Model training (regular lane) stuck in validation (3 times in a row)
NotFoundError: /modelState
Detect and take care of stuck preprocessing/training submission
Link for QUEUE status
About predictions file
Learning submission stuck , EVALUATION_IN_PROGRESS with no logs (and time quota question)
The available space of /modelState directory
How long should it take to convert dcm to png?
Training submissions after an INVALID submission are not shown
Confusing scoring feedback
job stuck in "Evaluation In Progress" state
Clarification of model training status and status detail
Docker CPU allocation
Error encountered during training (dm-preprocess-caffe)
No feedback for Digital Mammography Prediction - Sub-Challenge 1 Express Lane
training cancelled
Docker push : authentication required
Is pre-processing submission not included in limit quota?
inference is not allowed
Wait too long to cancel a job ?
Open phase input vs. Leaderboard phase input
Download older model state from open phase
Missing Uploaded Docker Image
Conflicting Submission
Issue with Theano example using CUDA
Date Dictionary - express queue and training and leaderboards
Reference base of images
submitting as individual vs. team
Express Lane Down?
Fields in metadata - in the 'Digital Mammography Model Training' lane
Prediction submission guide
Any news on when the challenger submissions will be possible?
syn7498325 is not accessible by the submission creators.
No training log
40 same emails within half hours, could you stop it immediately !!!
Could you stop sending the same email 10 times
Error when saving model in express lane: no space left on device
truncated log file
I'd like to join but I have a number of concerns about this challenge.
IOError: [Errno 2] No such file or directory: '/trainingData/cmkg906a.dcm'
Dummy dataset
Duplicated images in training set?
truncated log file
Re-use of /preprocessedData/ volume. Has anyone has success with it?
Log Files for Express Lane
Final training runs - where can we submit them when they need longer than the express lane?
model final output
Same submission finishes with Express Lane, but fails full Model Training
Can not access model state file
Has the competition already begun?
Is the pilot set part of the training set? not running
IndexError: list index out of range: no label (7th column) in /metadata/images_crosswalk.tsv in express lane
Examples of testing and instructions for downloading trained models
Parallel command not found error in dm-preprocess-png
where can I find the new data?
Jobs should appear in the list immediately after submitting, not when they start running
Trained Model Retrieval Update - - your Newsletter on the 7th of November
Submissions closed?
Express Lane for preprocessing and training?
Urgent Testing Related Questions
Submission failed with no such file or directory
What's the result of the competition last request output form?
Running jobs
Cannot see list of currently running jobs
evaluation on patient or breast?
GPU already preallocated at start of processing - problem still present
image data corruption???
File not found error
Too Many Users
If not submitted in 1st round can also continue to take part in the challenge?
Data integrity - errors in image files
Seemingly conflicting information regarding the trained models
Submission failed
Problems in submission
Messages on completion of submissions - 'has completed its training phase'
Confused about training process
Running tensorflow example container on my local machine
KeyError when running tensorflow example
Model Submitted is not progressing
TokenMgrError: Lexical error at line 1, column 48. Encountered: "@" (64), after : ""
What is the best practice to deal with intermediate data? - some serious concerns
Access to output - /modelstate
.dcm or .dcm.gz files under the /trainingData folder?
Tensorflow example kept running out of memory
dm-tensorflow-example ValueError
Timeline update
Error:: The repository for image docker.... does not exist or is not accessible
Not mounting expected directories
Order of execution of images - should we only be submitting the training image after completion of preprocessing?
Periodic Logs, Where to find - not seeing directories
Directories missing - do we have to mount them?
How to test docker images locally?
Log file page empty - fixed itself spontaneously
Unnecessary restart of preprocessing stage
Is there another way?
No message received about my submission and express lane
Timeline Update
Running job stuck in "Stop Requested" state
No email notification and doubt regarding output.
Why are both GPUs preoccupied at the start of training?
Problem in push an docker image on windows machine
Feature request: Please put submission ID and docker image name in ENV vars in each container
Our team can not download the photo? pilot_images_all.tgz?, who can send this to my email ?
Time available
When do we get access to the full training set?
Submission in Docker for Windows
Error Unknown
Log file has not been updated for a long time
Row id: 20 has been changed since last read. failure
Limit of submission during the challenge phases
Errors in push - reloading after a restart
Can python and matlab code be submited through docker?
couldnt download pilot data
Trained model
Saved JPG images?
Authentication error
Push timeout
IOError: [Errno 21] Is a directory: '/metadata/images_crosswalk.tsv'
haven't been able to try anything, would there be an extension of time line?
Row id: has been changed since last read.
Preprocessing container fails of Row id problem?
Docker privileged mode?
Error during preprocessing with example scripts
Disk I/O speed?
Error in push
Updating a file in your project
Can I join now
Container validation before putting it in the submission queue
Example code runs too slow + Queue of submitted scripts
pending submission for over 24h
Which step is missing?
cannot push in new image
preprocessing pipeline written in MATLAB
Can't find pushed docker image
How to interpret repeated imaging in the same exam?
Misunderstunding of the processing system
Pilot data
Does a preprocessing docker image have to include
Monitoring page: Too Many Users
providing a short queue
An SDK for this competition
Submission Failed on preprocessing job
Tensorflow GPU allocation problem?
throwing in the towel
docker push failed: unauthorized: authentication required (solved)
London calling
Log file not getting updated
{"message":"unauthorized: authentication required"}
No notifications regarding stdout logs
What is "exec format error"?
No email notification
Do we need to predict Cancer and Invasive Cancer separately?
What are the rules for collaboration in the competitive phase?
Sign up now to attend first webinar for DM Challenge on Wed Sep 14
Authentication error
Locating log files in the absence of notification emails
Queue times
How many permitted submissions in each round?
why there is no theano based docker?
Error while Pushing/Pulling Docker Images and Doubt regarding Submission data.
Views of Test Images
Various issues
No email received after submiting the model in the synapse project
Sharing pilot data
Job status on Wiki not updating
Not getting any notification emails
Submission terminated log file generated long after the actual termination time
About times and queue statuses
Submission failed without clear clues as to why
Submission failed due to duplicate entry?
continuing preprocessing
docker build error while installing pydicom
How to determine the positive or negative?
positive or negative digital mammography
couldnt download images
docker pull is giving permission denied.
labels format for SC1
metadata format
Webinar #1 Q&A: Would it be possible to add a docker image with a higher level framework such as keras?
Problems setting up Docker
Image submission
Image Sizes
Logic in example pre-processing scripts
Submission during open phase
Any easy ways to manipulate the files on the /preprocessedData folder?
Webinar 1 Posted
modelState persistency/lifetime
how can i remove an existing docker from synapse?
When will the Caffe example code become available?
Webinar #1 Q&A: Can we preprocess the original data more than once?
Webinar #1 Q&A: Will we get access to our trained models?
False positives
Is it possible to use additional mammography data (not given from DM Dream challenge) ?
Webinar #1 Q&A
Webinar #1 Q&A: For the open phase, will we get log-files regarding the model training accuracy?
Webinar #1 Q&A: You've been using python for your examples. Can we use other languages - or is it more sensible to stick with python too?
Webinar #1 Q&A: Is there a baseline algorithm?
Webinar #1 Q&A: If we submit a preprocessing Docker container, will we be able to review the preprocessed images?
Webinar #1 Q&A: Are all the images at the same physical resolution per pixel?
Webinar #1 Q&A: Are we provided BI-RADS scores or a physician's impression?
Webinar #1 Q&A: Are the DICOM metadata are available for the Sub-Challenge 1?
Webinar #1 Q&A: Is there a deadline for forming teams? E.g. can individual participant join together during any phase?
Webinar #1 Q&A: Will the provided example container have example input data/directories, intermediate log files, and output?
Webinar #1 Q&A: Is it OK for us to have partial images or partially processed images in the 10TB local storage?
Webinar #1 Q&A: Is Caffe the recommended deep learning framework to use?
Webinar #1 Q&A: When will the basic caffe, tensorflow examples be available ?
Webinar #1 Q&A: What models will your internal team build and when do we expect release?
Webinar #1 Q&A: Are you going to release more subjects for the training set?
Webinar #1 Q&A: Where can we find information regarding the "reference algorithm" ?
Webinar #1 Q&A: The open phase doesn't have any scoring right? so can we test our models in this phase?
Webinar #1 Q&A: What is the distribution of exams by manufacturer?
Webinar #1 Q&A: What proportion of exams have more than the 4 standard views?
Webinar #1 Q&A: Can we expect that every exam will have at least the 4 standard views?
Docker Nvidia CUDA Driver/Toolkit Problem
Pre-processing in parallel CPUs?
Language options - API & GPU preferences
Open phase file upload
Team in Zurich
Image Metadata Clarification
Will you provide both the images of the left breast and right breast in the main data?
Is 2D mammography here to stay?
Manufacturer of imaging systems
Clarification about white circled parts in the images.
Do we have a file describing details about malignancy/benign and tumor coordinates?
OpenCV vs DICOM image depth
Is there a date for the first webinar?
About when the disease is diagnosed
Disk space limitation
Docker submission mechanism?
Deep Learning only?
MonBUG hackathon in Old Port of Montreal
Challenge Participation Photos
More subjects are expected in training data set
view information is avaible or not in the training and testing
tumor vs invasive cancers
Clarification of Subchallenge 2
Regarding computational resources
localization ground truth for training data?
Pilot Set request
Image Verification
Showing mammographs to a 3rd person
dead link to "about the challange"
Participating without submitting/sharing model code
Training set image sizes
Label in image
Model development, experimentation, and testing
False negatives in pilot data?
Circle on image from pilot data set
Please do not post mammogram images or metadata from the Pilot Set in this Forum
pilot data: ftp?
"Register a Team for the Digital Mammography Challenge" link not working

The Digital Mammography DREAM Challenge page is loading…