Predicting Medical AI in 2017(lukeoakdenrayner.wordpress.com)
lukeoakdenrayner.wordpress.com
Predicting Medical AI in 2017
https://lukeoakdenrayner.wordpress.com/2016/12/31/predicting-medical-ai-in-2017/
3 comments
Hi, I'm the author of the blogpost (didn't expect to see it here!) and I largely agree with you.
I think I would say that the use of private datasets isn't actually a problem though, the problem is that the methods and analysis in many of the papers is horribly flawed. I could reasonably trust a paper with good methodology - all of medical research is on private data essentially. But the majority is, as you say, pure hype.
I actually wrote another blog post a bit back that was popular on some simple rules to separate hype from reality in medical AI work - https://lukeoakdenrayner.wordpress.com/2016/11/27/do-compute...
I think I would say that the use of private datasets isn't actually a problem though, the problem is that the methods and analysis in many of the papers is horribly flawed. I could reasonably trust a paper with good methodology - all of medical research is on private data essentially. But the majority is, as you say, pure hype.
I actually wrote another blog post a bit back that was popular on some simple rules to separate hype from reality in medical AI work - https://lukeoakdenrayner.wordpress.com/2016/11/27/do-compute...
Thanks for the links--I wasn't aware of these resources.
I do research in a neighboring domain, and have sort of dabbled in these sorts of datasets before, with a tiny bit of ML-ish research on different topics. It's an area I've thought about expanding into.
My sense is that the sorts of EMR/EHR records you're linking to aren't fine-grained enough for the most part to be useful for the sorts of things that will constitute AI's presence in healthcare. I think they are incredibly, incredibly useful, and will find a place in AI research (I've edited papers using these sorts of datasets with AI), but my sense is that they're not the sort of thing that will give AI a foothold.
I think your sentiment and that of the OP is right in that there isn't likely to be anything major this year, but I think AI/ML will have a bigger presence over the next 10 years than people are currently expecting. There's too much research showing that many tasks currently done by clinicians can be done better algorithmically by a machine, and in contrast to a lot of fields, the content matter is supposed to be approached that way--quantitatively and scientifically.
What is needed is a lot of very fine-grained data. Not visit data or event summaries, diagnoses, procedures, providers, but actual specific image files, etc. and so forth. My experience is that this stuff is in EMRs sometimes, but not consistently, and is never made publicly available. Some of the data--much of it maybe--is not actually in hospital records at all. It might be sitting in R&D or university labs somewhere, or isn't being collected at all.
My guess is that big corporations will start collecting this data one way or another, or researchers will establish consortiums, or apps will start amassing it as they gain traction, and that's how this will proceed.
I do research in a neighboring domain, and have sort of dabbled in these sorts of datasets before, with a tiny bit of ML-ish research on different topics. It's an area I've thought about expanding into.
My sense is that the sorts of EMR/EHR records you're linking to aren't fine-grained enough for the most part to be useful for the sorts of things that will constitute AI's presence in healthcare. I think they are incredibly, incredibly useful, and will find a place in AI research (I've edited papers using these sorts of datasets with AI), but my sense is that they're not the sort of thing that will give AI a foothold.
I think your sentiment and that of the OP is right in that there isn't likely to be anything major this year, but I think AI/ML will have a bigger presence over the next 10 years than people are currently expecting. There's too much research showing that many tasks currently done by clinicians can be done better algorithmically by a machine, and in contrast to a lot of fields, the content matter is supposed to be approached that way--quantitatively and scientifically.
What is needed is a lot of very fine-grained data. Not visit data or event summaries, diagnoses, procedures, providers, but actual specific image files, etc. and so forth. My experience is that this stuff is in EMRs sometimes, but not consistently, and is never made publicly available. Some of the data--much of it maybe--is not actually in hospital records at all. It might be sitting in R&D or university labs somewhere, or isn't being collected at all.
My guess is that big corporations will start collecting this data one way or another, or researchers will establish consortiums, or apps will start amassing it as they gain traction, and that's how this will proceed.
RE: "What is needed is a lot of very fine-grained data. Not visit data or event summaries, diagnoses, procedures, providers, but actual specific image files, etc. and so forth."
Interesting; can you offer other examples of "very fine-grained data" you are thinking of, besides image files?
I'm also a researcher in medical AI and I would just add one other area that I'm optimistic about (which I happen to work on): automated EEG interpretation. We've seen some really big steps forward in the past few years and automatic epileptogenic spike detection is now as good as a human neurologist. Seizure detection is not quite there yet, but we've made a lot of progress and within another year or two it might get there. It's not a huge market so it doesn't really get much publicity outside of neurology circles. But if we can continue to cut down the time it takes to read an EEG it will be a huge boon for neurologists.
I think there are immediate ML applications in medicine in addition to radiology.
Drug discovery, any kind of imaging (retinopathy, neuroimaging, ultrasound etc).
I think the reason DeepMind are also doing a non-AI related project with NHS is because it's a quick win (current hospital IT is horrible) and they are in some ways trading it in exchange for access to lots of medical image data.
I agree there need to be more public anonymised medical datasets, so research results can be benchmarked against a common metric.
Drug discovery, any kind of imaging (retinopathy, neuroimaging, ultrasound etc).
I think the reason DeepMind are also doing a non-AI related project with NHS is because it's a quick win (current hospital IT is horrible) and they are in some ways trading it in exchange for access to lots of medical image data.
I agree there need to be more public anonymised medical datasets, so research results can be benchmarked against a common metric.
So whats happening in radiology field?
For those interested in AI in medicine, the most recent NIPS held a workshop on exactly that topic: https://nipsml4hc.ws
There's quite a bit of active research applying LSTMs, CNNs, and the like to medical data sets. I'd say about two thirds of the work I saw was from academia, but there are active efforts from industry: Google Brain, Empatica, Cardiogram (us), and Evidation were all in attendance or presenting work in progress.
There's quite a bit of active research applying LSTMs, CNNs, and the like to medical data sets. I'd say about two thirds of the work I saw was from academia, but there are active efforts from industry: Google Brain, Empatica, Cardiogram (us), and Evidation were all in attendance or presenting work in progress.
The link is refusing to connect. Can you update the link?
I can't edit my post anymore, but as of right now, the link works over http rather than https:
http://www.nipsml4hc.ws/
[deleted]
I actually have a very pessimistic outlook. I think outside of few specific applications in Radiology and maybe pathology, AI/ML is unlikely to succeed in Medicine. Even Deep Mind which started its health division by building a Non-AI app. The major issue in medicine is figuring out the right kind of problem/intervention/interface.
Worse a lot of research in this field (outside of radiology) is pure hype. It typically involves researchers teaming up with their friends in university associated hospital to publish studies on private datasets which can never be reproduced. Why worry whether the problem definition is correct or clinically meaningful when you can claim anything and the community lets you get away with calling your work Deepest Patient or AI Doctor. This is why Physicians & medical researcher mock machine learning in healthcare research.
It is going to take another half a decade or even a decade untill unform EMR/EHR datasets become available. And some Imagenet style competition weeds out the current crop of GRU-LSTM-RNN-Stacked-Autoencode-GAE papers whose goal is to stuff resumes with "hot" keywords rather than actually improve the state of the art in an emiprical manner.
[1] https://github.com/AKSHAYUBHAT/ComputationalHealthcare