I need help finding a source, cuz there are so many fluff articles about medical AI out there…
I recall that one of the medical AIs that the cancer VC gremlins have been hyping turned out to have horribly biased training data. They had scans of cancer vs. not-cancer, but they were from completely different models of scanners. So instead of being calibrated to identify cancer, it became calibrated to identify what model of scanner took the scan.
I am failing to find source, but there is also a story about an older predictive model that worked great at one hospital, but failed miserably at the next. There was just enough variation in everything that the model broke.
(I think the New England Journal of Medicine podcast, but I am not finding the episode.)
I need help finding a source, cuz there are so many fluff articles about medical AI out there…
I recall that one of the medical AIs that the cancer VC gremlins have been hyping turned out to have horribly biased training data. They had scans of cancer vs. not-cancer, but they were from completely different models of scanners. So instead of being calibrated to identify cancer, it became calibrated to identify what model of scanner took the scan.
Versions of this dataset bias have been circulating since the 1960s.
I am failing to find source, but there is also a story about an older predictive model that worked great at one hospital, but failed miserably at the next. There was just enough variation in everything that the model broke.
(I think the New England Journal of Medicine podcast, but I am not finding the episode.)