Biomedical image analysis challenges should be considered as an academic exercise, not an instrument that will move the field forward in a real, practical way

The biomedical imaging community has witnessed innovative algorithmic developments in quantitative imaging biomarkers taking advantage of modern multimodality imaging technologies. These developments encompass a wide methodological portfolio including but not limited to image registration, segmentation, classification, and prediction. Despite the enormous progress in technical developments, including rigorous peer-review preceding publication in the scientific literature, extensive testing, and feedback from users of the associated open-source software tools, validation of these advanced image analysis tools prior to their deployment in the clinic is still one of the main challenges faced by developers and end-users.