Show HN: Autoretrieval – Autoresearch for RAG Pipelines(github.com)
github.com
Show HN: Autoretrieval – Autoresearch for RAG Pipelines
https://github.com/daly2211/autoretrieval
2 comments
What are the core knobs for a RAG pipeline? See any interesting patterns? Eg in practice, what did the agents tend to tune? Did domain matter?
interesting approach can you share some insights of your experiments in different domains
The agent edits the retrieval pipeline, runs an eval, checks whether the F-beta score improved, keeps or discards the change, and repeats. It can try different chunking strategies, chunk sizes, the number of retrieved chunks, embedding models, keyword retrieval, and more, while keeping a log of every experiment.
The repo also includes a script that generates question and reference-highlight pairs from any document collection. This makes it possible to optimize a RAG pipeline for domain-specific data.
On one of my own RAG pipelines, it more than doubled the F3 score after a few hours of running experiments.
I'd love to hear any feedback or ideas.