Hi HN! Elastiknn is an Elasticsearch plugin that lets users store numerical vectors within documents and run exact and approximate nearest neighbor queries on them. It supports dense and sparse vectors with five similarity functions (L1, L2, Angular, Jaccard, Hamming).
It's a hobby project of mine, based on an application where I needed to to continuously update an index of vectors for approximate searches. Very few of the existing approximate search libraries satisfy this use-case; most require re-building the index from scratch.
Elastiknn lets you add/update/delete vectors without rebuilding the index and combine vector queries with existing Elasticsearch queries, all through the standard JSON/HTTP API, all in your existing cluster. You just specify a field type of `elastiknn_sparse_bool_vector` or `elastiknn_dense_float_vector`, specify some optional settings for approximate indexing, index vectors as JSON lists, and then use the `elastiknn_nearest_neighbors` query in your search requests.
Nothing is without tradeoffs. In terms of raw throughput, Elastiknn is slower than many existing solutions. If you need a high performance solution for offline batch jobs, it's likely not for you. There are some benchmark results at https://elastiknn.com/performance/.
My current goals for the project are: integrating with the ann-benchmarks project, releasing builds for multiple Elasticsearch versions, and creating some example applications.
All feedback is appreciated. At this point there are several teams evaluating the plugin. There's a pretty thorough test suite, but nothing beats real-world usage and feedback. Cheers!
It's a hobby project of mine, based on an application where I needed to to continuously update an index of vectors for approximate searches. Very few of the existing approximate search libraries satisfy this use-case; most require re-building the index from scratch.
Elastiknn lets you add/update/delete vectors without rebuilding the index and combine vector queries with existing Elasticsearch queries, all through the standard JSON/HTTP API, all in your existing cluster. You just specify a field type of `elastiknn_sparse_bool_vector` or `elastiknn_dense_float_vector`, specify some optional settings for approximate indexing, index vectors as JSON lists, and then use the `elastiknn_nearest_neighbors` query in your search requests.
Nothing is without tradeoffs. In terms of raw throughput, Elastiknn is slower than many existing solutions. If you need a high performance solution for offline batch jobs, it's likely not for you. There are some benchmark results at https://elastiknn.com/performance/.
My current goals for the project are: integrating with the ann-benchmarks project, releasing builds for multiple Elasticsearch versions, and creating some example applications.
All feedback is appreciated. At this point there are several teams evaluating the plugin. There's a pretty thorough test suite, but nothing beats real-world usage and feedback. Cheers!