We have received a couple requests to release the results from our evaluation on the STIR 2024 Challenge test split. Feel free to use these results in your publications. A challenge paper will follow later on
2D:
Place | Team Name | Average Accuracy | Type
1 | MFT | 0.776 | Baseline
2 | CUHK | 0.774 | Post-challenge
3 | MFTIQ_SEARAFT (ROMA) | 0.768 (0.772) | Post-challenge
4 | MedTrack | 0.765 | Pre-challenge
5 | CCG_DGIST | 0.738 | Pre-challenge
6 | Jmees | 0.69 | Pre-challenge
7 | ICVS | 0.664 | Pre-challenge
8 | CSRT | 0.586 | Baseline
9 | UBC_RCL | 0.586 | Post-challenge
10 | RAFT | 0.424 | Baseline
11 | Control | 0.411 | Baseline
12 | SRV | 0.344 | Post-challenge
3D and latency results are the same as those presented on the challenge day, available in the powerpoint on the homepage, as there were no new submissions in these categories.
Test Dataset for STIR Challenge 2024:
We received a request to release the dataset used for evaluation. We have done so, and it is available here:
[STIR Challenge 2024 Dataset](https://zenodo.org/records/14803158)
STIR Dataset:
We also note the ~10x larger STIR dataset can also be used for evaluation purpose if you create your own test/train splits within it. Here is that [dataset](https://ieee-dataport.org/open-access/stir-surgical-tattoos-infrared) and [paper](https://arxiv.org/abs/2309.16782).