Horizon 1.0: a realtime, open-source JavaScript back end from RethinkDB(horizon.io)
horizon.io
Horizon 1.0: a realtime, open-source JavaScript back end from RethinkDB
http://horizon.io/#1.0-release
132 comments
Hi, Slava, I was a web developer and now I am a mobile developer. Nobody is developing web applications anymore (at least in developing countries). We are all mobile first. How does android or ios developers take advantage of Horizon ? Can you give a few examples? How does horizon make mobile (native) developers life easier ? web development is slowing down and mobile development is on the rise. Mobile developers don't want to deal with backend development. But it seems like web development is your main focus and mobile is a second thought. Please make it the focus of your work. Meteor is useless to us because it is optimized for web client. I hope Horizon won't fall for this trap. Your bigger audience and potential users are android or IOS developers. Please optimize for them and provide lots of examples to show them how their life can be easier. Thanks.
In the FAQ page http://horizon.io/faq/
Can I use Horizon on mobile devices?
Horizon ships with a JavaScript client library that can be used for building web and mobile apps with frameworks like React, Angular, or React Native. We’re also working on specific platform client libraries that support the Horizon protocol. Client libraries for iOS and Android are coming soon.
Can I use Horizon on mobile devices?
Horizon ships with a JavaScript client library that can be used for building web and mobile apps with frameworks like React, Angular, or React Native. We’re also working on specific platform client libraries that support the Horizon protocol. Client libraries for iOS and Android are coming soon.
I'm from a developing country too and that is such an exaggeration. Maybe your career has been focused on mobile, but no way there's overall more native development than web.
So what does your dumb mobile client connect to? If it's any good, probably a web backend.
Web developer here. Some of the most complicated application are being built for web vs mobile. Within the enterprise web applications, I fail to see how mobile is anywhere near web's presence especially for data intensive apps.
Very fitting handle.
Im really stoked for this and thanks a lot for your hard work.
Im really stoked for this and thanks a lot for your hard work.
Hi Slava!
What do you guys think about the approach taken here, and what are the major differences in approach theat you took with Horizon?
https://github.com/mikemintz/react-rethinkdb
I've had really good luck with that whitelisting approach, so I'm wondering what the biggest wins would be from potentially switching to Horizon.
What do you guys think about the approach taken here, and what are the major differences in approach theat you took with Horizon?
https://github.com/mikemintz/react-rethinkdb
I've had really good luck with that whitelisting approach, so I'm wondering what the biggest wins would be from potentially switching to Horizon.
Mike's `react-rethinkdb` project is really cool, but it's based on exposing the entire ReQL language to the browser. We thought this approach wouldn't work well as an industrial-strength product -- ReQL is designed to be accessed from the backend, and exposing a full query language to the browser introduces a number of challenges. Security is really hard, learning curve is steep, there are DoS challenges, the protocol is too complex to write platform-specific extensions, etc.
On the other hand, Horizon exposes a simple API/protocol that's easy to learn and easy to secure. If you need more functionality, you can import Horizon as a Node.js module and start writing backend code that accesses RethinkDB directly. The learning curve is much easier, it provides an upgrade path for sophisticated apps, and feels like a more robust product.
(I don't mean to disparage `react-rethinkdb`, I think the project is awesome, it's just a philosophical difference in approach)
On the other hand, Horizon exposes a simple API/protocol that's easy to learn and easy to secure. If you need more functionality, you can import Horizon as a Node.js module and start writing backend code that accesses RethinkDB directly. The learning curve is much easier, it provides an upgrade path for sophisticated apps, and feels like a more robust product.
(I don't mean to disparage `react-rethinkdb`, I think the project is awesome, it's just a philosophical difference in approach)
Thanks for your time!
Could you give me a quick comparison of Horizon and Meteor? I've used Meteor in the past (and loved it), and it looks like Horizon might be fun to learn.
Could you give me a quick comparison of Horizon and Meteor? I've used Meteor in the past (and loved it), and it looks like Horizon might be fun to learn.
It looks to me like Horizon and Meteor are solving problems in different scopes of the application stack.
Horizon looks to be more like the Minimongo/DDP/Accounts parts of Meteor, and doesn't help with any more of the stack like Meteor does (eg: In Horizon its more up to you to wire it into Angular/React/etc). Meteor is more of a platform and Horizon is more of a real-time client DB and account framework.
If I had to choose between the two, Horizon would be good for if you had a strong opinion/technical need for very specific coupling between UI and DB, and Meteor for when you don't or need to get something out very quickly (eg: this weeks internal CRUD app for a few dozen users).
Horizon looks to be more like the Minimongo/DDP/Accounts parts of Meteor, and doesn't help with any more of the stack like Meteor does (eg: In Horizon its more up to you to wire it into Angular/React/etc). Meteor is more of a platform and Horizon is more of a real-time client DB and account framework.
If I had to choose between the two, Horizon would be good for if you had a strong opinion/technical need for very specific coupling between UI and DB, and Meteor for when you don't or need to get something out very quickly (eg: this weeks internal CRUD app for a few dozen users).
There's one on the website:
http://horizon.io/faq/
http://horizon.io/faq/
IMO it hand waves Meteor away without acknowledging (perhaps out of ignorance) some of it's other strengths.
I'll watch Horizon but I'm taking it all with a grain of salt. Standard new javascript framework protocol people.
I'll watch Horizon but I'm taking it all with a grain of salt. Standard new javascript framework protocol people.
The FAQ repeats some things I've heard about Meteor -- specifically that its architecture doesn't scale.
But Meteor isn't completely prescriptive. There are several pieces that can be swapped out: You can use Angular, React, or Blaze on the client, for instance.
From what I've read, it doesn't look like Horizon is a complete framework, just a database client/server component that can be used with any framework.
But Meteor isn't completely prescriptive. There are several pieces that can be swapped out: You can use Angular, React, or Blaze on the client, for instance.
From what I've read, it doesn't look like Horizon is a complete framework, just a database client/server component that can be used with any framework.
We have a short-burst, write-heavy, mostly cpu-bound, work-load, which according to the various financially incentivized critics would completely knee-cap us for our use of Meteor.
Except we:
1) optimized our database operations
2) relegated cpu-bound work to micro-services that could process work from a queue at their own pace
3) Leveraged the ecosystem to writer smarter and more concise code
Know your enemy and know yourself and you need not fear the result of a thousand battles (Sun Tzu). Meteor is an amazingly productive ecosystem for writing real-time applications. No it will not solve every problem you throw at it.
Just to be clear I'm not hating on Horizon. I definitely will try it in the future.
Except we:
1) optimized our database operations
2) relegated cpu-bound work to micro-services that could process work from a queue at their own pace
3) Leveraged the ecosystem to writer smarter and more concise code
Know your enemy and know yourself and you need not fear the result of a thousand battles (Sun Tzu). Meteor is an amazingly productive ecosystem for writing real-time applications. No it will not solve every problem you throw at it.
Just to be clear I'm not hating on Horizon. I definitely will try it in the future.
OK, glad to hear that it does work for you. Sounds like you have a robust architecture.
I don't actually know what is supposed to "not scale" about Meteor. I've just read it multiple times. But because of the nature of Meteor (Really easy for anyone to set up! Anyone can create a real time app in ten minutes or less!), it probably attracts a lot of junior developers. It therefore should have occurred to me to translate "It doesn't scale!" to "I don't know how to scale it and my app is slow!!!". :)
I don't actually know what is supposed to "not scale" about Meteor. I've just read it multiple times. But because of the nature of Meteor (Really easy for anyone to set up! Anyone can create a real time app in ten minutes or less!), it probably attracts a lot of junior developers. It therefore should have occurred to me to translate "It doesn't scale!" to "I don't know how to scale it and my app is slow!!!". :)
There are two major issues that crop up in my experience.
* Node as a runtime is deeply mediocre, in my experience. You can usually solve this by throwing more (usually virtualized) hardware at the problem.
* Meteor's architecture is (at present) tied into MongoDB's oplog tailing. You can scale it up through sharding, through alternate libraries, and DDP itself is not tied to MongoDB at all. IIRC fixing this was a goal for Meteor 2.0.
Both of these can be worked around. If nothing else, the Meteor web side will talk to anything doing DDP, and DDP is not that hard to deal with.
* Node as a runtime is deeply mediocre, in my experience. You can usually solve this by throwing more (usually virtualized) hardware at the problem.
* Meteor's architecture is (at present) tied into MongoDB's oplog tailing. You can scale it up through sharding, through alternate libraries, and DDP itself is not tied to MongoDB at all. IIRC fixing this was a goal for Meteor 2.0.
Both of these can be worked around. If nothing else, the Meteor web side will talk to anything doing DDP, and DDP is not that hard to deal with.
It comes up because Meteor gets slow faster then most other solutions will get slow. It kind of reminds me in Django how a common mis-step for beginners is using an O(n^2) algorithm on the Django ORM. Meteor is the same way if you aren't using proper indices, or are trying do something that SEEMS doable to the uninitiated (like say pushing ~n thousand JSON objects into a MongoDB set), it's going to give you a bad time.
"2) relegated cpu-bound work to micro-services that could process work from a queue at their own pace"
doesn't this defeat the primary selling point of meteor?
The three key features we derive from Meteor are:
1) real-time view layer
2) diverse and highly integrated ecosystem (libs are written just for meteor)
3) productive development environment (es2015 modules on client/server, no fuss build pipline for sass, jade, angular-templates, angular dependency injection)
There are a ton of secondary benefits, but thats the crux of it. Yes we do sacrifice some of the benefits of Meteor as a monolithic (homogenous?) full-stack solution, but I would argue that they are in fact small, and if you dive into Meteor you'll find it to be pretty modular.
1) real-time view layer
2) diverse and highly integrated ecosystem (libs are written just for meteor)
3) productive development environment (es2015 modules on client/server, no fuss build pipline for sass, jade, angular-templates, angular dependency injection)
There are a ton of secondary benefits, but thats the crux of it. Yes we do sacrifice some of the benefits of Meteor as a monolithic (homogenous?) full-stack solution, but I would argue that they are in fact small, and if you dive into Meteor you'll find it to be pretty modular.
Congrats on the release! I've only hada little time to play with it so far as a beta user, but was super impressed and can't wait to use it on a real project soon. Cheers!
Are the transactions optimistic? How does the logic that concurrently applies/rolls-back these transactions over multiple clients work? I'm very interested in this, do you have some internal documentation on it?
Take a look at this comment: https://news.ycombinator.com/item?id=11716145. TL;DR: currently not optimistic but will be soon; it's fairly simple to do because RethinkDB core has built in support for it. Once the feature lands we'll talk about the implementation in detail.
[deleted]
Okay I can wait :)
But without optimistic transactions, what sets Horizon apart from just running the transaction-code on the server? If I understand it correctly, the latency will be the same in both cases, i.e., one network round-trip.
But without optimistic transactions, what sets Horizon apart from just running the transaction-code on the server? If I understand it correctly, the latency will be the same in both cases, i.e., one network round-trip.
Ease of use.
Lack of optimistic updates is a temporary state, though. The vision for Horizon definitely includes this functionality, and we'll make sure it lands ASAP.
Lack of optimistic updates is a temporary state, though. The vision for Horizon definitely includes this functionality, and we'll make sure it lands ASAP.
> I've been up for about twenty-four hours, but I'll be around to answer any questions for the rest of the day
Make sure you don't regret about your answers later :) Answering questions on HN after being 24 hours awake is generally not a good idea :)
Make sure you don't regret about your answers later :) Answering questions on HN after being 24 hours awake is generally not a good idea :)
Hey Slava. Interesting stuff - two questions:
1. Do you have something analogous to https://www.firebase.com/docs/web/api/ondisconnect? Real-time apps often have some manner of presence indicator. Firebase allows you to say "when this client disconnects, set the value of this key to $BLAH". I noticed that Horizon has http://horizon.io/api/horizon/#ondisconnected, but this is just a client-side indication of when you've disconnected.
2. What are Horizon's transaction properties? How about ordering of appends to a collection?
1. Do you have something analogous to https://www.firebase.com/docs/web/api/ondisconnect? Real-time apps often have some manner of presence indicator. Firebase allows you to say "when this client disconnects, set the value of this key to $BLAH". I noticed that Horizon has http://horizon.io/api/horizon/#ondisconnected, but this is just a client-side indication of when you've disconnected.
2. What are Horizon's transaction properties? How about ordering of appends to a collection?
There is currently no "disconnect-and-set" functionality, but it's on the roadmap and will happen soon. It just didn't land into v1.
Since Horizon is backed by RethinkDB, it's transaction properties are the same as that of RethinkDB. Check out this document for the specifics -- http://rethinkdb.com/docs/consistency/.
Since Horizon is backed by RethinkDB, it's transaction properties are the same as that of RethinkDB. Check out this document for the specifics -- http://rethinkdb.com/docs/consistency/.
Thanks for that. Can you clarify disconnect behavior in general? Do subscribed changesets resume if your internet blips?
Also, there doesn't seem to be a callback arg in Collection#store: http://horizon.io/api/collection/#store. This is worrying. How do I know if a write succeeded?
Edit: found http://horizon.io/api/collection/#subscribe - you chain subscribe to writes when you care about their success
Also, there doesn't seem to be a callback arg in Collection#store: http://horizon.io/api/collection/#store. This is worrying. How do I know if a write succeeded?
Edit: found http://horizon.io/api/collection/#subscribe - you chain subscribe to writes when you care about their success
Once you guys made it big, you should rename yourselves to RethoughtDB :)
Does it handle conflict resolution, ike when two people modified the same field at almost the same time?
Edit: While Horizon assigns versions internally, it looks like they are not currently used for catching concurrent client-side modifications by different users. (I previously wrote that they would cause the "losing" write to fail if it was based on an outdated version of the data, but that doesn't seem to be the case yet.)
I opened an issue against our docs to add information on this, as I'm a bit fuzzy on the detailed behavior myself: https://github.com/rethinkdb/horizon-docs/issues/50
I opened an issue against our docs to add information on this, as I'm a bit fuzzy on the detailed behavior myself: https://github.com/rethinkdb/horizon-docs/issues/50
> Horizon is distributed as a standalone server for browser-based JavaScript apps. When you're ready to write custom backend code, you can load the Horizon modules directly in Node.js
This is key. Firebase seems great for quick prototypes, but I never built against it because it was unclear (to me at least) how to deal with cases where you want to leave the sandbox and add custom backend functionality that they haven't built out for you. Horizon seems ideal -- it gives you the ease of use of a turnkey system starting out, with the flexibility to customize backend logic/validation later when you need to. That, plus the fact that it's put together by the awesome RethinkDB team, makes it very interesting to me!
This is key. Firebase seems great for quick prototypes, but I never built against it because it was unclear (to me at least) how to deal with cases where you want to leave the sandbox and add custom backend functionality that they haven't built out for you. Horizon seems ideal -- it gives you the ease of use of a turnkey system starting out, with the flexibility to customize backend logic/validation later when you need to. That, plus the fact that it's put together by the awesome RethinkDB team, makes it very interesting to me!
Firebase does support a REST API that I've used successfully from Rails. So back-end access works pretty well.
I've setup background workers that connect to a firebase DB to perform business logic / backendy type stuff. It works rather well and they even have a queue collection specifically for this type of thing. You could even setup a daemon on a VPS / EC2 that would in turn fire up AWS lambda procs (node or w/e) to perform w/ your firebase data.
Regardless, I'm very excited to play around w/ Horizon as well.
Regardless, I'm very excited to play around w/ Horizon as well.
For apps that don't necessarily have the realtime requirement, Kinto [1] is another really cool open-source backend solution that focuses on the offline-first scenario. It's built by the Firefox Sync team and used in production at Mozilla.
That said, at first glance there doesn't seem to be anything stopping anyone from building a caching layer on top of Horizon to enable offline-first functionality. Definitely going to be keeping an eye on this as well!
EDIT: It looks like they are in fact considering offline-first support already [2]!
[1] http://kinto.readthedocs.io/en/latest/overview.html
[2] https://github.com/rethinkdb/horizon/issues/58
That said, at first glance there doesn't seem to be anything stopping anyone from building a caching layer on top of Horizon to enable offline-first functionality. Definitely going to be keeping an eye on this as well!
EDIT: It looks like they are in fact considering offline-first support already [2]!
[1] http://kinto.readthedocs.io/en/latest/overview.html
[2] https://github.com/rethinkdb/horizon/issues/58
Offline is much harder than one initially think. More specifically, the big issue us that you almost certainly need a solid algorithm to solve conflicts in order for it to work, otherwise it's quite likely you'll update your data in a matter that is unexpected from a user perspective (assuming that two users edit the same data). The only systems i know of which seem to handle this gracefully is Derby/ShareJS (using operational transformation but still lacking offline support) and Amelisa (using CRDTs and in a very early phase development wise). If this is not built in from the ground up, I fear it's going to be very complicated to build in.
How is Kinto vs Horizon different, anyone from the Horizon team want to comment.
(Former Meteor core dev here) This is cool!
Does Horizon also solve "optimistic updates"? If so I'd love to learn more details. For comparison, Meteor keeps a local datastore that updates immediately when data is mutated and then reconciled with the real database.
Does Horizon also solve "optimistic updates"? If so I'd love to learn more details. For comparison, Meteor keeps a local datastore that updates immediately when data is mutated and then reconciled with the real database.
The current version of Horizon doesn't do optimistic updates, but it's on the roadmap. Check out https://github.com/rethinkdb/horizon/issues/23. This is relatively easy because RethinkDB itself has support for notifying the client via a feed when a particular update has landed (i.e. the RethinkDB client can correlate the write with a feed message). It just didn't make it into v1, but will happen soon.
See also my issue about implementing the equivalent of Meteor's write fence: https://github.com/rethinkdb/horizon/issues/344
I'm continuously impressed by Rethink. When I first used it, I ran into a few issues with changefeeds. Since they work in the open, I could quickly look at the Github issues and see that the fixes/features were already slated for future release milestones. To me, that's a huge win.
Slava, Horizon looks cool, and I'm a huge fanboy of RethinkDB. However, in the demo video you write queries client side. How do you protect against users modifying front-end JavaScript and thus the queries?
// ex
this.props.horizon.order('datetime', 'descending').limit(8).watch()This is the question I always get stuck on with these "front-end only" frameworks. How do you prevent DoS attacks or scrapers downloading your whole database? What is the information security model?
(Josh from RethinkDB here)
The sibling comment by ubercore has the right link. And your concern is important, since without security, why not just expose your db to the frontend directly?
At a high level, the permission system:
1. Is a whitelist approach (you can't do anything by default)
2. Allows you to specify query shapes that are allowed to be run by the current user. This prevents queries of the wrong shape from ever hitting the database, helping with scalability.
3. Has a fallback to full javascript rules which can depend on the data on disk. A little slower than the query rules, but allows much more flexibility where you need it. And you can combine them with a query rule to do a rough first pass of what is acceptable.
Ultimately, the permissions system is designed to prevent unfettered database access, but it also has an eye towards performance in the case of DOS attacks. Horizon overall attempts to use only indexed queries, low resource usage operations etc, so you don't accidentally invite DOS attacks .
Now, I'm not claiming we're DOS proof or anything, but I just intend to mention we have an eye towards it when we talk about security.
The sibling comment by ubercore has the right link. And your concern is important, since without security, why not just expose your db to the frontend directly?
At a high level, the permission system:
1. Is a whitelist approach (you can't do anything by default)
2. Allows you to specify query shapes that are allowed to be run by the current user. This prevents queries of the wrong shape from ever hitting the database, helping with scalability.
3. Has a fallback to full javascript rules which can depend on the data on disk. A little slower than the query rules, but allows much more flexibility where you need it. And you can combine them with a query rule to do a rough first pass of what is acceptable.
Ultimately, the permissions system is designed to prevent unfettered database access, but it also has an eye towards performance in the case of DOS attacks. Horizon overall attempts to use only indexed queries, low resource usage operations etc, so you don't accidentally invite DOS attacks .
Now, I'm not claiming we're DOS proof or anything, but I just intend to mention we have an eye towards it when we talk about security.
I believe http://horizon.io/docs/permissions/ is the answer
I would think you'd do this server-side, I've had similar issues outside of JavaScript: users play a game and create a custom client that connects to my server, they send custom packets thereby I have to make sure that they: don't access too much data at once (a user asking for 100 different things in 3 seconds is not even normal in certain scenarios) or the data they're requesting is formatted wrong (a string where a integer is expected?) the solution I guess is server-sided. This issue isn't unique only to client-side javascript, it is relevant in networking as well. So long as the back-end is designed intelligently it could all work out.
Out of the box Meteor does publish the to the browser. But I believe if you remove the insecure package you then control what is published to the browser e.g. usually only a subset of the db.
On the Meteor App I'm currently working on, we limit the data the client sees to what they need, so while it makes a scrapers job slightly easier as they can query the DB, they don't get anymore data than any other approach. If data is displayed to the user they can scrape it.
Security is the same as the clients DB writes are rejected if they are modifying fields in ways that are not permitted (eg: changing a trouble ticket from "submitted" to "approved"). This can make some of the security easier to audit as most of the business logic security is in one place rather than scattered about in dozens of API calls.
While the exact technical details for these real-time DB based front ends are different, the basic thinking is the same of never trust the client and its costly to transmit info to the client.
The nice thing about having the DB directly on the client is that it makes CRUD apps much easier to make as you have to build less bespoke framework to get data to/from the client and verifying things conform to business logic. If you are not building a CRUD app you will want to evaluate other methods for a better fit.
Security is the same as the clients DB writes are rejected if they are modifying fields in ways that are not permitted (eg: changing a trouble ticket from "submitted" to "approved"). This can make some of the security easier to audit as most of the business logic security is in one place rather than scattered about in dozens of API calls.
While the exact technical details for these real-time DB based front ends are different, the basic thinking is the same of never trust the client and its costly to transmit info to the client.
The nice thing about having the DB directly on the client is that it makes CRUD apps much easier to make as you have to build less bespoke framework to get data to/from the client and verifying things conform to business logic. If you are not building a CRUD app you will want to evaluate other methods for a better fit.
You can't protect from the client modifying the JavaScript code, and that's ok. Horizon ships with a full security model (http://horizon.io/docs/permissions/) that lets you specify exactly what data each user can and cannot access, which queries they can run, etc. We're going to continue expanding the security model to include DoS protection, and any other issues that may come up.
So while the user can change JavaScript as they please, that's ok; if they try to do something they're not supposed to do, the server will just reject the query.
So while the user can change JavaScript as they please, that's ok; if they try to do something they're not supposed to do, the server will just reject the query.
If you're looking for a way to check out Horizon and don't already have RethinkDB up and running locally, we've put a One-Click app together over on DigitalOcean: https://www.digitalocean.com/community/tutorials/how-to-use-...
From their blog post:
> Horizon isn’t prescriptive or particularly opinionated–it’s designed to work well with the JavaScript frameworks that you already know and love.
It is seriously a breath of fresh air to read something like that. Bravo, Horizon/RethinkDB team.
> Horizon isn’t prescriptive or particularly opinionated–it’s designed to work well with the JavaScript frameworks that you already know and love.
It is seriously a breath of fresh air to read something like that. Bravo, Horizon/RethinkDB team.
I think it's amazing that it's working with real RxJS streams for changes now - given how Redux store now returns an observable thing, Cycle is Rx based and Angular 2.0 uses a lot of Rx.
This guarantees that things will interop nicely with eachother and we'll be able to use Rx all around.
This guarantees that things will interop nicely with eachother and we'll be able to use Rx all around.
You might want to check out lovli.js (https://github.com/flipace/lovli.js) -- a Horizon+Redux integration. It's really cool and demonstrates how well Horizon can interop with modern web technologies.
Oh, this is exactly what I want to do with Horizon! So cool.
Although this is about horizon, if you haven't looked at RethinkdB you should definitely check it out.
It's an awesome database for a variety of use-cases.
Personally, I think their query language + support for joins makes it much more useable than MongoDB (which I've used quite a bit).
It's an awesome database for a variety of use-cases.
Personally, I think their query language + support for joins makes it much more useable than MongoDB (which I've used quite a bit).
Agreed! I had tons of fun building my own CMS on top of RethinkDB (as well as a logging/journal app for personal use). The only reason I ended up with a wordpress/MySQL solution for the CMS was that building the CMS interface is a bitch and migrating from SQL to SQL was a bit easier.
But man was I sad to leave RethinkDB for that!
But man was I sad to leave RethinkDB for that!
Sorry, but coming from an embedded world the use of Real-time here seems borderline offensive to me. Anyone care to explain what it means in this context? From the dawn of computer science it has been about strict time guarantees - and I highly doubt this is what's being referred to here.
AFAIK, soft-real time has been a well known term since the 80s. It means that the quality of service degrades the longer it takes for a response to be returned, and eventually a response is worthless if enough time passes.
I agree with that, but how does this apply to a javascript framework? You have no control over network latency or device CPU load.
> Horizon is built on RethinkDB, a massively scalable, open-source database capable of millions of realtime updates per second.
Have there been any performance benchmarks or reviews of RethinkDB yet? Especially at this scale?
Have there been any performance benchmarks or reviews of RethinkDB yet? Especially at this scale?
Checkout the Jepson reports by Aphyr. They go into incredible detail.
https://aphyr.com/posts/330-jepsen-rethinkdb-2-2-3-reconfigu...
https://aphyr.com/posts/330-jepsen-rethinkdb-2-2-3-reconfigu...
That's a good report and I've read it before - but still looking for something about the actual performance (latency, throughput, concurrency) that it can handle.
Hey manigandham, Dalan (dalanmiller) from RethinkDB here,
We have a performance benchmark report we did a couple months ago basically locked and loaded to publish with accompanying scripts and data for transparent review of our methodology. We just have been so busy with Horizon, we had put this on the back burner. Expect to see something within the next couple weeks when we can put some serious attention to it to ensure we do everything just right.
In the meantime, you should check out the whitepaper by BigchainDB which uses RethinkDB for the purposes of consistency and automatic change notifications [1].
- [1] https://www.bigchaindb.com/whitepaper/
We have a performance benchmark report we did a couple months ago basically locked and loaded to publish with accompanying scripts and data for transparent review of our methodology. We just have been so busy with Horizon, we had put this on the back burner. Expect to see something within the next couple weeks when we can put some serious attention to it to ensure we do everything just right.
In the meantime, you should check out the whitepaper by BigchainDB which uses RethinkDB for the purposes of consistency and automatic change notifications [1].
- [1] https://www.bigchaindb.com/whitepaper/
This is awesome, really looking forward to it. Rethinkdb is my goto database for everything nowadays, and I tell everyone I meet about it.
Also: kudos to whoever is handling the artwork/marketing side of things. Love the vector art and website designs.
Also: kudos to whoever is handling the artwork/marketing side of things. Love the vector art and website designs.
Christina from RethinkDB here (I work on all things community)... I'll pass the kind words along to our co-founder Mike G. who worked countless hours on the website, plus Annie Ruygt who does the artwork! It's wonderful to hear positive feedback since we do most of our design in-house :D
I'll second deepanchor. It might seem like such a minor detail, but at least for some people, including myself, it does matter. It signifies something.
Hey Slava. Promising stuff! One question:
A websocket connection between client and server is great for real-time apps, but for non real-time app, the unnecessary websocket connections create more work for devops -- specifically DNS and load balancer. Does/will Horizon support that clients only use the Collections API to get/set data without websocket (for real-time sync)?
A websocket connection between client and server is great for real-time apps, but for non real-time app, the unnecessary websocket connections create more work for devops -- specifically DNS and load balancer. Does/will Horizon support that clients only use the Collections API to get/set data without websocket (for real-time sync)?
Launch blog post: http://horizon.io/blog/horizon-launch/
Saw a demo of this at a realtime meetup in Palo Alto and was pretty impressed. Like everything RethinkDB does I anticipate it will be a fantastic one-two punch of sound engineering and smooth user experience.
Awesome! Was it Mike Seid's demo? Will have video up soon. Thanks for stopping by our meetup :D
Could you post it here when it's up? I'd like to see it.
Yeah it was Mike's!
At last, an open source alternative to Firebase.
How does Horizon handle offline network conditions? Are there network + app states (e.g., cold boot, refresh) under which users might not have access to data?
I'm wondering how this works (or if it does) with GraphQL subscriptions. https://github.com/rethinkdb/rethinkdb/issues/3711
There's currently no GraphQL support in Horizon, but it's on the roadmap. See https://github.com/rethinkdb/horizon/issues/125.
Thanks!
Looks really promising to me, relatively similar to www.deepstream.io . However, will there be additional clients /are they already in development by the community? So far I only found support for vue.js and React. Polymer elements would be pretty dope... ;)
Congrats! I remember seeing someone from your team doing a demo at an SF JavaScript meetup. Looks very interesting.
Great move, RethinkDB folks!
I'm looking forward start prototyping some apps using Horizon. On a first look seems very promising.
Congrats!
I'm looking forward start prototyping some apps using Horizon. On a first look seems very promising.
Congrats!
This looks really cool, but I still don’t understand how Rethink plans to cash out their VCs, though—~$12M, according to Angel List. Cloud hosting is trending toward zero and support/consulting only scales with people. I’m not to trying to be (intentionally) critical; I’m genuinely curious and hoping there is some business model innovation going on here too.
Excellent work Slava! Have been planning to migrate to RethinkDB for one of the existing projects, great timing with Horizon :D
Slava @ Rethink here. Just wanted to point out that only a tiny amount of credit for Horizon belongs to me, as most of it has been a product of tireless work of many people (hey Marc, Dalan, Josh, Daniel, Michael, Annie, Christina, Ryan, Mlucy, Chris, Marshall, and all the awesome beta contributors!)
This isn't false humility either -- I've been super busy with some other aspects of RethinkDB over the past few months, so while I did have a hand in designing and shipping Horizon, maybe less than 5% of it was my doing.
This isn't false humility either -- I've been super busy with some other aspects of RethinkDB over the past few months, so while I did have a hand in designing and shipping Horizon, maybe less than 5% of it was my doing.
I love Firebase, but this is very exciting. Congrats on 1.0!
How well does this play with Electron (http://electron.atom.io/ ) as a front-end? I didn't find that many references to it (mostly questions about putting them together).
Congrats on the launch! As a long-time user of Meteor I'm always excited to see other approaches to similar issues.
One thing that I always found a bit lacking with Meteor was pagination. The problem in a nutshell is that the client doesn't know how much data is potentially available on the server unless it requests it, making things like showing "page 1 of 12" type result counts tricky (I talk more about it here: https://www.discovermeteor.com/blog/pagination-problems-mete... )
I'd be curious to know how Horizon deals with that problem?
One thing that I always found a bit lacking with Meteor was pagination. The problem in a nutshell is that the client doesn't know how much data is potentially available on the server unless it requests it, making things like showing "page 1 of 12" type result counts tricky (I talk more about it here: https://www.discovermeteor.com/blog/pagination-problems-mete... )
I'd be curious to know how Horizon deals with that problem?
https://github.com/aldeed/meteor-tabular handles this with custom subscriptions, and it stays up to date by continuously polling the collection without a limit. If you want to get more real-time or more scalable, the problem is that if the total-count subscriptions can't fit in Meteor's memory, every subscription would need to re-request its count from Mongo on every incoming oplog entry, no matter whether it ends up mattering.
Sadly, Horizon doesn't seem to support aggregate queries yet, reactive or not, without loading the records with findAll. Theoretically, Rethink could be even more scalable at this than Mongo, since it could use its index to only "wake up" relevant count subscribers. But this is still on the roadmap:
https://github.com/rethinkdb/rethinkdb/issues/3735 https://github.com/rethinkdb/rethinkdb/issues/1118
Generally, it seems Horizon is very limited compared to Meteor, but it's a great fresh start, with a much more well-thought-out and scalable architecture, and I'm looking forward to seeing where it goes.
Sadly, Horizon doesn't seem to support aggregate queries yet, reactive or not, without loading the records with findAll. Theoretically, Rethink could be even more scalable at this than Mongo, since it could use its index to only "wake up" relevant count subscribers. But this is still on the roadmap:
https://github.com/rethinkdb/rethinkdb/issues/3735 https://github.com/rethinkdb/rethinkdb/issues/1118
Generally, it seems Horizon is very limited compared to Meteor, but it's a great fresh start, with a much more well-thought-out and scalable architecture, and I'm looking forward to seeing where it goes.
How does this differ from Meteor?
See the FAQ http://horizon.io/faq/
For convenience, this is the section taken from it that addresses this exact question! (I imagine other people may be thinking this)
How is Horizon different from Meteor?
Horizon has philosophical and technical differences with Meteor that will result in vastly different developer experiences.
Horizon is a small layer on top of RethinkDB with a narrow, purpose-built API designed to make it very easy to get started building realtime apps without backend code. Horizon isn’t prescriptive – you can use any front-end framework without any magic/customizations, and once your app outgrows the Horizon API you can use any backend technology (e.g. Node.js, Python, Ruby) and any backend framework (e.g. Rails, Express, Koa, etc.)
By contrast, Meteor is a much more prescriptive framework. Horizon is a reasonably small component that has a very clean separation with the database and the frontend, while Meteor is a much more monolithic experience.
Another major difference is architectural – Meteor uses a LiveQuery component built by tailing MongoDB’s oplog. This approach is fundamentally limited – it’s impossible to do many operations efficiently, and even the basic functionality is extremely difficult to scale.
Horizon is built on RethinkDB, so the LiveQuery functionality is in the database. This allows for much more sophisticated streaming operations, and scalability is dramatically simpler because the database has all the necessary information to allow for a scalable feeds implementation.
For convenience, this is the section taken from it that addresses this exact question! (I imagine other people may be thinking this)
How is Horizon different from Meteor?
Horizon has philosophical and technical differences with Meteor that will result in vastly different developer experiences.
Horizon is a small layer on top of RethinkDB with a narrow, purpose-built API designed to make it very easy to get started building realtime apps without backend code. Horizon isn’t prescriptive – you can use any front-end framework without any magic/customizations, and once your app outgrows the Horizon API you can use any backend technology (e.g. Node.js, Python, Ruby) and any backend framework (e.g. Rails, Express, Koa, etc.)
By contrast, Meteor is a much more prescriptive framework. Horizon is a reasonably small component that has a very clean separation with the database and the frontend, while Meteor is a much more monolithic experience.
Another major difference is architectural – Meteor uses a LiveQuery component built by tailing MongoDB’s oplog. This approach is fundamentally limited – it’s impossible to do many operations efficiently, and even the basic functionality is extremely difficult to scale.
Horizon is built on RethinkDB, so the LiveQuery functionality is in the database. This allows for much more sophisticated streaming operations, and scalability is dramatically simpler because the database has all the necessary information to allow for a scalable feeds implementation.
Congrats on the launch, this looks like an amazing product that made immediate sense to me. I have a feeling the Ionic community is going to be quite drawn to this over time.
Are there any dockerized versions of this yet?
Are there any dockerized versions of this yet?
This looks really great! Does Horizon support pluggable transport? I would like to use Ajax paired with EventSource/SSE over HTTP/2 for bidirectional communication instead of Web Sockets.
There is currently no EventSource/SSE support, but this came up a couple of times. It should be pretty easy to add, and is currently on the roadmap (though for now still unscheduled).
Would it be a change in Horizon or in Engine.io?
How does access control work? For example, if I have a database containing a table "users", and a table "files", and the files table contains a column "owner" (pointing to the users table), then how do I prevent the client-side database engine from accessing the files which do not belong to the user that is currently logged in?
Check out the docs on security and permissions -- http://horizon.io/docs/permissions/. Horizon ships with a full security model that allows you to protect your data and specify exactly what each user can and cannot access and which queries they can run.
This is super exciting. I have been playing around with it in closed beta and I'm really impressed. Open Source FTW.
The Horizon blog could really use a feed that I can subscribe too. I would love to head about all of your future news.
Good idea! We forgot to push a RSS feed. Here it is: http://horizon.io/feed.xml
Thanks! I'm now subscribed :)
It would be even awesomer if you could include the articles in the feed so I don't have to leave my cozy RSS reader :)
It would be even awesomer if you could include the articles in the feed so I don't have to leave my cozy RSS reader :)
I agree! I'll take a look at updating our RSS feed in a bit to include the full articles.
Congrats guys. As usual, amazing work!
Great thinking from the RethinkDB side. One answer I couldn't find on their website is about the offline and optimistic update. It's really tricky to get right. Having a feed of changes is just a small part of having a robust solution.
Take a look at this comment: https://news.ycombinator.com/item?id=11716145. TL;DR: currently not optimistic but will be soon; it's fairly simple to do because RethinkDB core has built in support for it. Once the feature lands in Horizon we'll talk about the implementation in detail.
This is on our list : https://github.com/rethinkdb/horizon/issues/23
The big piece remaining that will help is write timestamps in the database, which we're going to try to land in RethinkDB 2.4. If we were controlling a traditional MVC frontend model and had control over the objects themselves, it would be easier to do optimistic updates. Since we're aiming to be compatible with a lot of frontend frameworks where immutability of the model is important, we have to carefully consider how to do it.
That being said, it's super important, so it's near the top of our priorities
The big piece remaining that will help is write timestamps in the database, which we're going to try to land in RethinkDB 2.4. If we were controlling a traditional MVC frontend model and had control over the objects themselves, it would be easier to do optimistic updates. Since we're aiming to be compatible with a lot of frontend frameworks where immutability of the model is important, we have to carefully consider how to do it.
That being said, it's super important, so it's near the top of our priorities
This looks great. I wrote my first complicated Firebase app a couple weeks ago, and looking forward to rewriting a version of it with Horizon to compare the two.
I couldn't find it in the docs, but what is the equivalent to Firebase.ServerValue.TIMESTAMP?
I couldn't find it in the docs, but what is the equivalent to Firebase.ServerValue.TIMESTAMP?
Have you ever looked at Meteor?
I have, they're not really the same thing: https://www.quora.com/What-is-the-difference-between-Firebas...
@slava - how would you compare a pouchdb <-> couchdb setup vs horizon?
I'm intrigued but after reading a couple of pages it's not really clear to me what benefits I get or really exactly what this is....
Why should I use it over my current approach ? Why is this compelling?
Why should I use it over my current approach ? Why is this compelling?
Think of it as an open-source Firebase. Not only you can host your own data, you can also do complex queries because it uses RethinkDB underneath.
[deleted]
It would be cool to have a object oriented database with prototype support, rather then table oriented. I think Rethink is a step in the right direction.
This looks cool. Congrats on the launch! Is it a project or a business? Do you intend to make money from hosting? Or something else?
Horizon is an open-source project. Horizon Cloud (currently in development) is a business -- http://horizon.io/cloud/
Looks very interesting!
You may want to make your homepage clearer, before navigating to the FAQ I did not have a clue of what Horizon really does.
You may want to make your homepage clearer, before navigating to the FAQ I did not have a clue of what Horizon really does.
I'm really excited to see a framework make security and authentication part of it's foundation.
Amazing work Horizon team!
Amazing work Horizon team!
Anyone got the chat example working?
Hey @nileshtrivedi,
We pushed a hotfix a few minutes after launch that resolved this issue [1] which you may be experiencing. You should try doing `npm update -g` if you're still on `horizon 1.0.0`. You can check with `hz version`.
If this doesn't fix it for you, please open up an issue [2] and I will help you out.
- [1] https://github.com/rethinkdb/horizon/issues/392
- [2] https://github.com/rethinkdb/horizon/issues/new
We pushed a hotfix a few minutes after launch that resolved this issue [1] which you may be experiencing. You should try doing `npm update -g` if you're still on `horizon 1.0.0`. You can check with `hz version`.
If this doesn't fix it for you, please open up an issue [2] and I will help you out.
- [1] https://github.com/rethinkdb/horizon/issues/392
- [2] https://github.com/rethinkdb/horizon/issues/new
so .. can one write a spreadsheet app that is able to show results reactively in the front-end and will still reflect that same logic in some kind of backend api ?
Kind of being able to have the same logic in my spreadsheet as well as my api ?
This could be huge!
Kind of being able to have the same logic in my spreadsheet as well as my api ?
This could be huge!
Does Horizon handle using npm packages, requires or imports on the front end?
Do you have a timeline on when will GraphQL be supported in horizon?
The faq is really pretty strong.
Looks cool!
Looks cool!
I've been up for about twenty-four hours, but I'll be around to answer any questions for the rest of the day.