That looks really interesting, thank you for sharing !
I have to admit, I am not exactly sure what a feature store does yet, but I will read more and watch your presentation.
I have this idea of taking my whole Zotero library full of PDFs/websites, use my own notes and tags and prepare it for RAG applications, so I can query my documents, make connections between documents, classify them by topic, etc and have a way to add my new notes/tags back to Zotero (this is all possible with their API).
Do you think Feast is a good framework for that ?
It would be worthwhile to clarify the term "tracing" to distinguish between live aggregation and post-processing approaches.
The general confusion around the "tracing" terminology seems to imply a competition between these two, while they should rather be seen as complementary.
DTrace, SystemTap and eBPF/BCC are designed to aggregate data in the critical path and compute a summary of the activity. Ftrace and LTTng are designed to extract traces of execution for high resolution post-processing with as small overhead as possible.
Aggregation is very powerful and gives a quick overview of the current activity of the system.
Tracing extracts the detailed activity at various levels and allows in-depth understanding of a particular behaviour after the fact by allowing to run as many analyses as necessary on the captured trace.
In terms of impact on the traced system, trace buffering scales better with the number of cores than aggregation approaches due its ability to partition the trace data into per-core buffers.
Both approaches have upsides and downsides and should not be seen as being in competition, they address different use-cases and can even complement each other.
Great tool !
It is interesting because we are working to extract the same kind of information but with LTTng for low intrusiveness and remote analysis of production servers.
If you are interested, you can have a look at the following repository :
https://github.com/jdesfossez/lttng-analyses
I have this idea of taking my whole Zotero library full of PDFs/websites, use my own notes and tags and prepare it for RAG applications, so I can query my documents, make connections between documents, classify them by topic, etc and have a way to add my new notes/tags back to Zotero (this is all possible with their API). Do you think Feast is a good framework for that ?