Ask HN: What's the best data storage to use along with Node.js?
9 comments
CouchDB requires a different mindset. I'm going to write a lot about it over the months as I use it and go to production with it.
I found that if you just think about CouchDB as as soup and you build views to build indicies that organize your data, then it gets really easy to work with.
I'm designing my WIN ( https://github.com/mathgladiator/win ) framework around node.js and CouchDB.
What I am doing is controlling my design documents via node.js using the fact that v8 will keep a string copy of the function for it's toString().
If you look at line 112 of win.js ( https://github.com/mathgladiator/win/blame/master/lib/win.js ), I define my indexer function to basically serialize a function from node.js into the design document.
I use this in the user system ( https://github.com/mathgladiator/win/blob/master/pages/page.... ) on line 202 to index the user documents by their id so I can use another function ($.get_index) to pull them out by their key.
I'm using this on some internal projects, and I'm getting very good velocity at working with CouchDB. Granted, there is a huge (and now) obvious flaw to my indexer function which I'm going to fix soon, but it works great.
I found that if you just think about CouchDB as as soup and you build views to build indicies that organize your data, then it gets really easy to work with.
I'm designing my WIN ( https://github.com/mathgladiator/win ) framework around node.js and CouchDB.
What I am doing is controlling my design documents via node.js using the fact that v8 will keep a string copy of the function for it's toString().
If you look at line 112 of win.js ( https://github.com/mathgladiator/win/blame/master/lib/win.js ), I define my indexer function to basically serialize a function from node.js into the design document.
I use this in the user system ( https://github.com/mathgladiator/win/blob/master/pages/page.... ) on line 202 to index the user documents by their id so I can use another function ($.get_index) to pull them out by their key.
I'm using this on some internal projects, and I'm getting very good velocity at working with CouchDB. Granted, there is a huge (and now) obvious flaw to my indexer function which I'm going to fix soon, but it works great.
CouchDB / Endtable
but I would like to stay away from relational databases
What's the specific reason for that, if any? I'm not trying to criticize you, although people really do keep overestimating their need for non-relational storage solutions. Outlining your requirements would help.
What's the specific reason for that, if any? I'm not trying to criticize you, although people really do keep overestimating their need for non-relational storage solutions. Outlining your requirements would help.
I'm going to be using JSON to represent data on pretty much all the parts of my App. (Since both the client and server are JavaScript, that's an obvious choice) I didn't want to copy all fields of my objects to a lot of enormous queries or something like that.
Also, my app will deal with a lot of tree-like structures, and representing them on a Relational DB would suck, unless I stored JSON on a text field. That seems like an ugly hack to me.
Also, my app will deal with a lot of tree-like structures, and representing them on a Relational DB would suck, unless I stored JSON on a text field. That seems like an ugly hack to me.
Hey
Sorry but I don't understand this answer. I too like to hear reasons for decisions to stay away from non-relational databases.
Your first point, that you'll be using JSON, feels strange. When I build stuff, the data storage and the usage of the data are very separate areas. The web part using the JSON doesn't know or give a damn if the data is stored in MySQL or MongoDB or written in sand and translated to computer-speak by monkeys.
Second - Maybe I'm misunderstanding you, but isn't this tree structure a number of entities that have relationships with other entities (as parent / child / sibling / whatever). How is that not applicable to a relational database?
Sorry but I don't understand this answer. I too like to hear reasons for decisions to stay away from non-relational databases.
Your first point, that you'll be using JSON, feels strange. When I build stuff, the data storage and the usage of the data are very separate areas. The web part using the JSON doesn't know or give a damn if the data is stored in MySQL or MongoDB or written in sand and translated to computer-speak by monkeys.
Second - Maybe I'm misunderstanding you, but isn't this tree structure a number of entities that have relationships with other entities (as parent / child / sibling / whatever). How is that not applicable to a relational database?
Since many NoSQL DBs use JSON to store the data, it seems you could save complicated conversion code between JSON and the db format by sticking to JSON throughout. That would be my line of thinking (I'm not the OP).
I think dbslayer would be perfect for this. Mysql is super easy to set up on a vps (there are lots of scripts to set up the whole server on sites like lowendbox or vpsbible) and then with dbslayer you get json and clustering if you want it so you can scale smoothly. It works great with node and there's already a module for it on Github @:
https://github.com/Guille/node.dbslayer.js/
https://github.com/Guille/node.dbslayer.js/
one issue with node.js is that it is so early in the lifecycle of the product that there aren't a lot of 'clear winners' in the add-on modules department. it's hard to tell which will be maintained in the long term and which are just hobby projects created so that the developer could learn node.
i would guess that mongoose is your best bet because it was written by a commercial company that is using it in their commercial product (learnboost).
you've probably already considered this, but i would consider google app engine. no database configuration necessary and i watched a video from the 2009 google io conference the other day that showed how insanely easy is it to do things like social graphs and follower using their datastore (assuming each person has 5,000 or fewer friends -- a reasonable number for 99% of the companies out there).
i would guess that mongoose is your best bet because it was written by a commercial company that is using it in their commercial product (learnboost).
you've probably already considered this, but i would consider google app engine. no database configuration necessary and i watched a video from the 2009 google io conference the other day that showed how insanely easy is it to do things like social graphs and follower using their datastore (assuming each person has 5,000 or fewer friends -- a reasonable number for 99% of the companies out there).
You mean using Google App Engine (either Python or Java) as an "external" storage, separate from your Node server?
Goodbye Google Appengine: http://news.ycombinator.com/item?id=1927903
Obviously bias, but I do think CouchDB and node.js make for a particularly nice couple
* sorry, I was sure I read your post but managed to miss that you already tried couch
* sorry, I was sure I read your post but managed to miss that you already tried couch
It really depends on what you're storing and how you plan on interacting with it. If you don't need ad hoc queries CouchDB w/ defined views may work well. Otherwise consider another solution like mongo.
If the available solutions are immature / undocumented feel free to contribute to them. That's what FOSS is all about ;)
If the available solutions are immature / undocumented feel free to contribute to them. That's what FOSS is all about ;)
How about Redis? If someone more knowledgable could please answer say... a Redis vs. CouchDB argument for node from a data storage/semantic perspective. Having contributed to redis-node-client, I know it has good support. And Redis's orientation as a data-structure store does seem more appealing.
Although I almost never comment on HN, I thought I'd chime in because well, all I do is node.js at Nodejitsu.
For CouchDB clients, I find cradle to be the best (http://github.com/cloudhead/cradle). Besides Alexis (cloudhead) being pretty responsive to fixes and new features, its a high-level Couch library as opposed some of the other offerings out there.
Before, I go too much more into I'll finish up by saying this isn't really the best place to ask this question. Jump into the #node.js channel on IRC and a bunch of people would be more than happy to answer your question.
For CouchDB clients, I find cradle to be the best (http://github.com/cloudhead/cradle). Besides Alexis (cloudhead) being pretty responsive to fixes and new features, its a high-level Couch library as opposed some of the other offerings out there.
Before, I go too much more into I'll finish up by saying this isn't really the best place to ask this question. Jump into the #node.js channel on IRC and a bunch of people would be more than happy to answer your question.
I've found the key value-based redis to be very easy to use, powerful and well-documented. Redis also seems to fit in well with the async nature of node.js apps.
While it's mostly known as an in-memory store, redis also has a virtual-memory option which can be useful when you have a (relatively) small number of keys -all of which can fit into memory- that are associated with much larger values which cannot fit into memory. Sure, there is a swap cost, etc, but I found this to be quite acceptable for a small sync server app I'm building (after prototyping with MongoDB and SQLite).
While it's mostly known as an in-memory store, redis also has a virtual-memory option which can be useful when you have a (relatively) small number of keys -all of which can fit into memory- that are associated with much larger values which cannot fit into memory. Sure, there is a swap cost, etc, but I found this to be quite acceptable for a small sync server app I'm building (after prototyping with MongoDB and SQLite).
Originally I was planning to use mongodb, since I liked its design, but the native driver has almost no documentation and is too cumbersome to use directly. Mongoose looks very promising, but doesn't seem quite ready for primetime, at least IMO.
So I found out about CouchDB. I decided to give it a go, but it's somewhat hard to configure beyond the very basic setup.
I was wondering: What are the other options and how well do they perform along with node.js? Ideally I would like something that's lightweight and easy to setup (like SQLite) but I would like to stay away from relational databases (like SQLite :-P) I would also like something that has at least some documentation.
Thanks.