There currently isn't a way to filter docs alongside a KNN query, and the dimension support is limited to 1024 (a Lucene limitation) and OpenAI embeddings are 1536 dimensions - also indexing performance is not comparable. Wishing this changes, as they're a good stack for the reasons you state
We offer a hosted/managed app search experience on top of Elastic or OpenSearch with appbase.io.
You can leverage the best of Elasticsearch (state-of-the-art) engine while getting Algolia like features like search relevance control plane, query rules, out-of-the-box analytics / insights, caching, UI components across all FE platforms (React, Vue, React Native, Flutter).
We're really excited about this launch, as it has been some time in the making. Our goal with appbase.io is to offer a supercharged Elasticsearch experience - we do this by offering an advanced search relevance control plane that allows a user to create a relevant search with no code. Since this experience is powered under the hood with Elasticsearch - so you aren't limited in the future if you wish to build upon what appbase.io already offers. You can host with us, or run it alongside your existing Elasticsearch cluster.
What can you can do with appbase.io:
- Build Auto suggestions (powered by n-grams, edge n-grams),
- Highlighting support,
- Support for 39 languages as well as support for multi-language search,
- Popular suggestions based on analytics,
- Set synonyms,
- Use rank feature in addition to text relevance to optimize relevance tuning,
- Advanced query rules to extend search relevance.
In addition to our no code control plane, one can use our JS UI components (4,000+ Github stars, 1MM+ downloads, actively used by more than 1,000 projects) to design the search experience for web and mobile. We offer these components for React, Vue, React Native and Vanilla JS. You can read more about it over here: https://appbase.io/product/search-ui/
When a search experience is powered with appbase.io, we additionally support collecting telemetry on search and click data and provide analytics/insights on areas to improve search engagement and conversion metrics. Read more over here: https://www.appbase.io/product/analytics/.
Searchbox is designed with the goal of being a lightweight (16KB to 35KB when minified + gzipped), performance focused library for building search UIs declaratively. We have variants for React, Vue and a headless library that can be used with any framework.
With this release version, Searchbox assumes the backend to be an appbase.io/Elasticsearch server. While there are other libraries that are backend agnostic, making this assumption allows the library to support tracking search analytics, displaying popular suggestions, and configuring advanced search relevance options. Searchbox comes from 3 years of learnings we've had in maintaining ReactiveSearch - one of the most popular search UI libraries out there.
Key benefits of using it:
- Searchbox lets you create versatile search experiences. Think auto suggestions, highlighting, incorporating fuzzy search, voice search, setting search relevance (fields to search on, weights to apply), showing default suggestions, rendering your own UIs.
- In addition, it lets you measure search/click analytics, set query rules, show popular suggestions. Most other searchbox libraries don't offer these capabilities.
- It's built with the open web in mind. Search is designed to work with any design system (Bootstrap/Material/Antd/your own), and it can integrate with any kind of UI components. See a demo here for an integration with React Table: https://codesandbox.io/embed/github/appbaseio/searchbox/tree....
- It supports tree shaking. If the built-in searchbox component were not used, the size of the library comes down to 10KB when minified + gzipped.
This pricing change from Algolia makes it easier for low-volume search users or someone who is just getting started to use their enterprise features, but at the same time as the parent mentions, makes it an order of magnitude more expensive for mid-to-high volume users. Imo, this is a mismatch as there is already a trial in place for someone who's just getting started to try those features out.
For anyone interested in creating a search at scale, I would recommend checking us out at appbase.io (founder here :wave:) - we provide relevant search, analytics, and access control for search and are built transparently on top of Elasticsearch. It also doesn't hurt that we author some of the most popular open-source search UI libraries: https://github.com/appbaseio/reactivesearch
Hey, addressing some of the concerns raised here directly:
> - client side query construction basically puts you in a position ..
We recommend using a proxy server for these cases. It's on our roadmap to add first-class support for search templates which would completely prevent query generation to happen client-side.
> - it's extremely aggressive in sending lots of queries to elasticsearch ..
This is configurable by using the debounce prop. And the number of queries depend on what components you would like to see updated based on a change in the search query or a facet value, for example.
> - the queries it constructs are a bit convoluted and under-use such features as filters ..
We would be happy to address specific scenarios if you're seeing, please raise an issue. That said, I believe we already do this to the extent it's generalizable. And since as a user, you can change the underlying query - it should be addressable by users.
---
To sum up, the DOS and querying part is completely configurable and just comes down to usage. We would love to address any generalizable issues.
There are merits to your point about client-side query generation. They are already addressed for users who are using:
1. Appbase.io - as you can set ACLs and rules (e.g. only allow search requests, max X requests per IP per hour, set max size of Y),
2. For users outside of appbase.io, we recommend using a proxy server like you did to implement an authorization logic.
Once we have first-class support for search templates in ReactiveSearch, this should be effectively addressed.
Thanks for the compliment. :-) The UI components would always remain open-source.
We offer a hosted search service at appbase.io which funds this and have very recently started offering it more broadly as a managed platform for any ElasticSearch upstream. It offers a better search development experience, out-of-the-box search/click analytics and more fine-grained security features: https://medium.appbase.io/appbase-io-clusters-a-supercharged....
I am not sure about the merits of the case to comment on this directly (blog post is sparse on details), but this is certainly unfortunate for the ElasticSearch community.
I want to share Arc - https://github.com/appbaseio/arc, an API gateway for ElasticSearch with security features that we have been actively developing starting this year. It's Apache 2.0 licensed, built in Go and we use it for providing security features for all of our customers. It's not as feature-rich as X-Pack / SearchGuard today, but we're happy to accept any PRs.
ReactiveSearch is a UI components library for creating powerful search experiences. While we originally created it with React in mind, we have added components for Vue.JS and React Native.
This release is a major rewrite of the library for more extensible component rendering support and adds support for controlled components, state access and also comes with experimental support for voice search, search operators and GraphQL. Here are the release notes for 3.0 - https://github.com/appbaseio/reactivesearch/releases/tag/v3.....
"The more advanced multi-tenant use case – where RediSearch was able to complete 25 million indices in just 201 seconds or ~125K indices/sec, while Elasticsearch crashed after it indexed 921 documents, showing that it was not designed to cope with this level of load." previously stated that "Elasticsearch crashed after 921 indices and just couldn’t cope with this load."
It's hard to mistake documents for indices. Both original and the currently edited statement sound strongly suspect and make me question the benchmarking methodology used. What caused the ES to crash after indexing 921 documents? Why is comparing indexing speeds on a 1-node setup even a legit benchmarking test?
As someone who is involved in the ecosystem and maintains some open-source projects based on ElasticSearch, I am similarly curious about the move to not build upon existing projects (there could be a good reason, but haven't heard it yet).
This is a good project to happen, and kudos to AWS for doing this! However, like some in the thread, I suspect AWS has been building this for a while for their own product and Elastic's license change may have influenced the decision to open-source.
Getting a good out-of-the-box granular security has been a long overdue pain point with ElasticSearch, there have been good alternatives for the other problems it is replicating.
Would very much like to hear from OP why these weren't contributed to instead of creating another alternative.
We've been working on a similar project in Golang and now well may be as good a day to open-source and put it out there: https://github.com/appbaseio/arc. It's an extremely light-weight API gateway for ElasticSearch that at the core comes with an out-of-the-box security system based on users and permissions (set granular ACLs, Rate Limits, TTL) inspired from a proprietary security system we built over the past year - https://appbase.io/features/security/.
Docker cloud offered CI/CD integrations and management features similar to K8S that were proprietary to them. While Docker images themselves are portable, this is a whole different beast.
