Show HN: I Built AskMedically – Get Research-Backed Answers to Medical Queries
8 comments
How do you handle the very well known limits of LLMs in your especially-sensitive use case? Hallucinations are the leading example. Health queries are a really bad place to do even mild “imagining” of responses.
I completely agree with you! LLMs are not good for medical queries, and that's exactly the reason I built this tool. I have used a simple RAG mechanism where I feed only research papers from trusted sources to the LLM to summarise them. In short, every answer is grounded in research papers. The idea is to cut down the time to do research for daily medical queries. I am still working on refining the answers, though - many new features are coming soon to make the research process easy.
Did you do any kind of validation? For example, do you have a testset of questions with criteria for what a right answer would be?
I don't have any test sets because I haven't trained any model from scratch. I have built a simple RAG, and my validation comes from users directly, like whether they find the answer useful or not.
The real value of these tools is in the validation, and I mean not just the face validity. User feedback is just face validity.
If you were a doctor and you needed to make a real treatment decision for a real patient, would you use this tool without checking the answer thoroughly, reading the literature yourself and checking to see if it didn't miss any relevant sources? If no, then you might as well skip the tool and do the work yourself. If yes, then you need to know for certain that the answer is correct.
And I don't think it matters if you trained the model yourself. You validate the tool as a whole.
The problem with using user feedback as validation is that users ask questions they don't know the answer to. Therefore, they are unable to judge the correctness of an answer. What you need is a gold standard, and validate against that.
If you were a doctor and you needed to make a real treatment decision for a real patient, would you use this tool without checking the answer thoroughly, reading the literature yourself and checking to see if it didn't miss any relevant sources? If no, then you might as well skip the tool and do the work yourself. If yes, then you need to know for certain that the answer is correct.
And I don't think it matters if you trained the model yourself. You validate the tool as a whole.
The problem with using user feedback as validation is that users ask questions they don't know the answer to. Therefore, they are unable to judge the correctness of an answer. What you need is a gold standard, and validate against that.
Interesting! I'd love to know more about how you built this. Also as a small note the twitter / linkedin buttons at the bottom of the page don't seem to do anything but point to the page itself.
Thanks, glad that you liked it! It's a simple RAG w/o indexing, which fetches the relevant research papers and summarizes them as per the question.
I have just launched the POC last week, and mostly focused on improving the search results, so I didn't get time to create the social pages yet. Will do it soon! But, thanks for pointing that out :)
I have just launched the POC last week, and mostly focused on improving the search results, so I didn't get time to create the social pages yet. Will do it soon! But, thanks for pointing that out :)
Does medically provide better answers than Gemini for any of those examples questions?
AskMedically only "summarises" answers from research papers, and Gemini or any other LLM "generates" answers. For medical queries, the citations are important and should be grounded in research papers - this is the only difference between AskMedically and Gemini or any other equivalent LLM.
Great idea. I’m pro scientific medicine. I was therefore a bit put off by my first test:
Treatment for knee osteoarthritis
The second response was a Chinese study about acupuncture. AFAIK that is a pseudoscience.
Treatment for knee osteoarthritis
The second response was a Chinese study about acupuncture. AFAIK that is a pseudoscience.
Thanks for the feedback! I sort of agree with you on acupuncture being a bit pseudoscience, and that's the reason I built this tool to have evidence-based answers. Seems like that Chinese study got published in PubMed, and that's why it got picked up by the tool: https://pubmed.ncbi.nlm.nih.gov/40557042/
I am working on more features to get deeper insights and more refined search results.
I am working on more features to get deeper insights and more refined search results.
how does this get on the first page of hacker news with 4 upvotes ????
No idea! I am also wondering the same.
I’ve built AskMedically – an AI-powered assistant that answers health and medical questions using real research papers from trusted medical sources like PubMed, Cochrane, etc.
Whether you’re a healthcare enthusiast, patient, student, or professional – AskMedically helps you explore trusted medical knowledge without needing a medical degree or slogging through dozens of PDFs.
Examples:
• “Does intermittent fasting improve insulin sensitivity?”
• “What are the benefits of creatine for brain health?”
• “Is ashwagandha safe to take long-term?”
• “How does ADHD present in adult women?”
• “Are cold plunges actually effective or just hype?”
It gives you:
• A clear, summarized answer
• Citations to real, peer-reviewed studies
• A link to read more if you want to dive deeper
Why I built it: We’re drowning in health advice on social media and wellness blogs, and it’s hard to know what’s evidence-based. I wanted to create a calm, reliable place where anyone curious about health could explore answers grounded in science.
It’s free to use, mobile-friendly, and optimized for both quick questions and deep dives.
Try it out: https://www.askmedically.com
Would love to hear what you think – especially what features or questions you’d like it to handle better. Contact: [email protected]