Show HN: Dashku - a real-time dashboard app, built with SocketStream(dashku.com)
dashku.com
Show HN: Dashku - a real-time dashboard app, built with SocketStream
https://dashku.com
28 comments
Hi carlsednaoui
As the creator of SocketStream, I am very keen to know how this error occurred. I believe it relates to the ulimit issue Paul mentioned in another comment which was fixed some time ago (i.e. the file was there, but the user was out of file descriptors)... not sure yet. Either way, we need to catch the exception properly.
As a general tip to all those hosting a Node app in production, you can catch uncaught exceptions with:
process.on('uncaughtException', function(err) { console.log(err); });
This ensures anything which goes wrong unexpectedly will be logged to your console, rather than sent to the browser - not good!
SocketStream doesn't currently do this for you by default, but maybe it should (in production mode). It's something I will consider implementing in the near future.
So far my focus has been on creating a great dev environment. It's very encouraging to see Dashku stand up to thousands of simultaneous users (once the ulimit issue was fixed), but I'm the first to admit more work is needed to ensure SocketStream is rock solid in production. It will take time but, thanks in part to experiences like today's, we'll get there.
Great job Paul.
Owen
As the creator of SocketStream, I am very keen to know how this error occurred. I believe it relates to the ulimit issue Paul mentioned in another comment which was fixed some time ago (i.e. the file was there, but the user was out of file descriptors)... not sure yet. Either way, we need to catch the exception properly.
As a general tip to all those hosting a Node app in production, you can catch uncaught exceptions with:
process.on('uncaughtException', function(err) { console.log(err); });
This ensures anything which goes wrong unexpectedly will be logged to your console, rather than sent to the browser - not good!
SocketStream doesn't currently do this for you by default, but maybe it should (in production mode). It's something I will consider implementing in the near future.
So far my focus has been on creating a great dev environment. It's very encouraging to see Dashku stand up to thousands of simultaneous users (once the ulimit issue was fixed), but I'm the first to admit more work is needed to ensure SocketStream is rock solid in production. It will take time but, thanks in part to experiences like today's, we'll get there.
Great job Paul.
Owen
I'm building a fairly large app right now with SocketStream and I just want to say that it is a joy to use. It really is a great dev. environment...flexible, useful, lightweight, and intuitive even without a ton of documentation.
I hope the /root/ refers to the site's root directory and not root user's home directory.
At least this confirms that it's actually running socketstream...
At least this confirms that it's actually running socketstream...
Probably not a good idea to run NodeJS as root.
Why not... It's an age-old concept, one I've enforced over decades, but seriously..... nowadays, once you are in, you are in, and systems (at least the ones I've put together in recent yeras, I can only assume the same of others) are very regularly refreshed from scratch from servers that can't be reached from the public facing stuff. (so even if you manage to find that one old php server and find some bug and drop some php thingy on it, it's going to get squished in 30 seconds, give or take, as regular housekeeping takes place. If you managed to actually focus on the attack and go deeper, it might take longer -but root or no root, the damage you can do is the same either way, and limited hte same either way.
EDIT: For the record - I still follow this practice, I'm just debating it's relevance these days. Things change, right?
EDIT: For the record - I still follow this practice, I'm just debating it's relevance these days. Things change, right?
Hi everyone,
My apologies, I didn't expect to end up Top of HN. I'll try and get it back up.
In the meantime, here is the YouTube video: http://www.youtube.com/watch?v=cxf8f9AQBgM&feature=plcp
My apologies, I didn't expect to end up Top of HN. I'll try and get it back up.
In the meantime, here is the YouTube video: http://www.youtube.com/watch?v=cxf8f9AQBgM&feature=plcp
Thanks for posting, I'll have to come back later and check out the site. From the video I can't really tell if it's opensource or a hosted service.
I'm also curious if there's an easy code import/export feature for individual widgets and/or the entire dashboard (all widgets, global styling, settings, etc). I don't think I'd use it for anything serious without being able to export the entire dashboard page and add it to my git repo (and being able to reload it if somethings gets lost or messed up). And being able to import/export individual widgets would make it easy for the community to share custom widgets so that in practice end users wouldn't actually have to write much code.
Anyway, it looks like a cool project!
EDIT: I see now there are REST api endpoints for creating & getting a dashboard, awesome.
I'm also curious if there's an easy code import/export feature for individual widgets and/or the entire dashboard (all widgets, global styling, settings, etc). I don't think I'd use it for anything serious without being able to export the entire dashboard page and add it to my git repo (and being able to reload it if somethings gets lost or messed up). And being able to import/export individual widgets would make it easy for the community to share custom widgets so that in practice end users wouldn't actually have to write much code.
Anyway, it looks like a cool project!
EDIT: I see now there are REST api endpoints for creating & getting a dashboard, awesome.
Hi everyone,
Once again, my apologies for the downtime.
For some reason, this was the case:
The site is alive, but may still be intermittent. Will keep you updated.
Once again, my apologies for the downtime.
For some reason, this was the case:
$ ulimit -n
> 1024
The site is alive, but may still be intermittent. Will keep you updated.
Did you set the limit on the CLI or within your limits.conf file? limits.conf is how you persist the value between reboots.
CLI (facepalm). Thanks for the tip.
so, no disrespect, innovate innovate innovate, keep on going, looks great.
but.
In general terms - what does a potential user of this service conclude if the server appears to be hosted on a single machine and the admin staff doesn't seem to know how ulimit works?
(Yes, I realize this is ycombinator and about ideas and VC and all that.... just throwing the concept out there.)
Great product, but total lack of either understanding or preparedness on the lower levels.
but.
In general terms - what does a potential user of this service conclude if the server appears to be hosted on a single machine and the admin staff doesn't seem to know how ulimit works?
(Yes, I realize this is ycombinator and about ideas and VC and all that.... just throwing the concept out there.)
Great product, but total lack of either understanding or preparedness on the lower levels.
Hi,
It was unfortunate that this ulimit issue occurred and I apologise for that.
I knew about ulimit from 2 sources (a LinkedIn article on Node.js practices) and a 2nd source from a hosting service's documentation.
The fault (and it was mine) was that I didn't realise that in order to persist the ulimit setting between reboots, you had to set it in a configuration file.
Originally, the application was hosted across multiple instances and proxied, but due to the nature of the WebSocket library and it's use of transports with the proxy library, the connection was intermittent, so I reverted to a single instance. Since resolving the ulimit issue, this application has stayed up.
As for admin staff, you may be surprised to learn there is no company with multiple devs, designers, product people and sysadmin behind all this. This is all my work, all 2 and a bit months of it.
It was unfortunate that this ulimit issue occurred and I apologise for that.
I knew about ulimit from 2 sources (a LinkedIn article on Node.js practices) and a 2nd source from a hosting service's documentation.
The fault (and it was mine) was that I didn't realise that in order to persist the ulimit setting between reboots, you had to set it in a configuration file.
Originally, the application was hosted across multiple instances and proxied, but due to the nature of the WebSocket library and it's use of transports with the proxy library, the connection was intermittent, so I reverted to a single instance. Since resolving the ulimit issue, this application has stayed up.
As for admin staff, you may be surprised to learn there is no company with multiple devs, designers, product people and sysadmin behind all this. This is all my work, all 2 and a bit months of it.
No problem.
Fantastic site, by the way.
Fantastic site, by the way.
I've been building a project that's very similar to this using AngularJS, socket.io, and the Highcharts/D3 graphing libraries. If you decide to open this project up, I'd happily throw some cycles at features & fixes. Great work!
What's your next step? Open-source the code, set up billing, improve the app (maybe move it off to a naturally-scaling cloud service)?
I mean, it's neat (understatement of the week) and all; but where do you plan on going from here?
I mean, it's neat (understatement of the week) and all; but where do you plan on going from here?
Hi,
Thanks. At this point, I need to go and find some work as my bank balance doesn't look great.
I'm going to continue working on it in some capacity, and I will investigate turning it into a SaaS.
There are some components of the app which would be worth open-sourcing, in order to assist others with building SocketStream applications.
Thanks again.
Thanks. At this point, I need to go and find some work as my bank balance doesn't look great.
I'm going to continue working on it in some capacity, and I will investigate turning it into a SaaS.
There are some components of the app which would be worth open-sourcing, in order to assist others with building SocketStream applications.
Thanks again.
Can you please put up a tickbox somewhere, that says "I agree to get pinged via email once the service gets a clear commercial roadmap"? I'd really appreciate that.
Good luck with your other gigs as well.
Good luck with your other gigs as well.
You shouldn't be serving your static landing pages and assets over node.js. Ideally you should have an nginx server reverse proxy dynamic requests to your node.js server. Nginx should serve the static assets before the requests hit the node server.
You have some data to back up this recommendation?
Indeed.
I mean, it may very well be true, and without time to analyze, if you are highly confident in nginx and the rest of your server setup (it's not some other user definiable limit) you might as well.
That said - building websites to handle load has no magic bullet. Sure, nginx is good. Far-future expires headers and cache-busting schemes to go with it are good too. CDN's are good. Varnish is good. Clusters are great too, for capacity and redundancy, but you need to be aware of stuff in all these cases, and every layer you add is also something else to manage and something else to break..... so really it has to come from test-driven results regarldess, so you are spot on.
My beef with services like this (granted I haven't investigated fully) is I don't want hosted stuff. I get that's a huge market, but for confidentiality and other reasons, there is also a market for slightly more traditional stuff.
I'd love a toolkit to make it easy to build arbitrary animated control panel-y things a-la Quest Spotlight on <whatever> for my apps... havent' found it yet, anyone know one?
That said - building websites to handle load has no magic bullet. Sure, nginx is good. Far-future expires headers and cache-busting schemes to go with it are good too. CDN's are good. Varnish is good. Clusters are great too, for capacity and redundancy, but you need to be aware of stuff in all these cases, and every layer you add is also something else to manage and something else to break..... so really it has to come from test-driven results regarldess, so you are spot on.
My beef with services like this (granted I haven't investigated fully) is I don't want hosted stuff. I get that's a huge market, but for confidentiality and other reasons, there is also a market for slightly more traditional stuff.
I'd love a toolkit to make it easy to build arbitrary animated control panel-y things a-la Quest Spotlight on <whatever> for my apps... havent' found it yet, anyone know one?
Here is a little piece of a script I wrote to push the limits up on an Ubuntu box during setup (AWS).
# Increase File Descriptor limits.
export FILEMAX=`sysctl -n fs.file-max`
sudo mv /etc/security/limits.conf /etc/security/limits.conf.bak
sudo touch /etc/security/limits.conf
sudo chmod 666 /etc/security/limits.conf
echo "root soft nofile $FILEMAX" >> /etc/security/limits.conf
echo "root hard nofile $FILEMAX" >> /etc/security/limits.conf
echo "* soft nofile $FILEMAX" >> /etc/security/limits.conf
echo "* hard nofile $FILEMAX" >> /etc/security/limits.conf
sudo chmod 644 /etc/security/limits.conf
touch /home/ubuntu/.bash_profile
echo "ulimit -n $FILEMAX" >> /home/ubuntu/.bash_profile
# Increase Ephemeral Ports.
sudo chmod 666 /etc/sysctl.conf
echo "net.ipv4.ip_local_port_range = 1024 65535" >> /etc/sysctl.conf
sudo chmod 644 /etc/sysctl.confYou know it's odd.
One one hand, the almost-greybeard in me wants people to stop screwing around with what I take to be standard unix concepts... what are the ephemeral ports, etc.
On the other hand, when I take a close look at something like what you just posted, the level of control is hard to argue with - nothing wrong with that.
The temptation is still to basically do what you did (thanks BTW) and just revert everything back to what it rightfully should be.... it served us just fine for ages.
Or I'll switch to FreeBSD ,unless they're doing the same thing.
On the other hand, when I take a close look at something like what you just posted, the level of control is hard to argue with - nothing wrong with that.
The temptation is still to basically do what you did (thanks BTW) and just revert everything back to what it rightfully should be.... it served us just fine for ages.
Or I'll switch to FreeBSD ,unless they're doing the same thing.
Seen a couple of startup's sites go down when they make the HN homepage lately, and both seem to be using node.js (according to the error messages). Not dissing node.js, but maybe its the wrong tool for the job.. Just use a static site for your landing page!
14 minute video? Why? On the homepage have a video that is 30-60 seconds. here is an example http://www.youtube.com/watch?v=qFcZjtGS1CE
Otherwise looks good.
Otherwise looks good.
I hadn't seen that before, and I've sent a batch of photos. Really well done!
Site seems to be having problems. Suggestion to the OP (your page also seems to be down) is to link to a static site with a demo video or description of the product. Or, test the load your app can handle before posting here.
Looks interesting. I looked at Geckoboard a while ago but gave up since the dashboards didn't offer the flexibility I needed at the time, and they were just too slow to render (I don't have a dedicated screen, just want to it the page often).
One issue: I tried to sign up using Chrome 20 (Windows 7) and the dialog that appears is below the video in the Z-order.
One issue: I tried to sign up using Chrome 20 (Windows 7) and the dialog that appears is below the video in the Z-order.
(tangent) I've got a feeling this is a bug in Chrome; z-order is not respected in some embedded <object>s. I've had this problem with sites that i've developed and haven't found a solution.
I am the developer of InfoCaptor and would be pleased if you can take a look at https://my.infocaptor.com
Looks very feature rich ... but to be honest the aesthetics need work. A huge part of this space relates to appearance obviously and the sample dashboards just don't look as polished as the competitors, even if they offer more widgets and options.
thanks, i know need to work on getting more polished dashboards. It just taking time to churn everything out.
My suggestion, is that if you're going to build a product for a tech audience, and you use bootstrap (which is a great idea btw), spend some time to make it not look like every other bootstrap site. I'm looking at you black fixed global nav bar.
the actual app is not based on bootstrap. just the website is bootstrap which is like godsend. it is only couple weeks since i released it publicly and i am not done launching.
But I absolutely hear your suggestion that website needs lot of work.
But I absolutely hear your suggestion that website needs lot of work.
This is great. I really like how you can test your widgets with sample data.
Because of the lack of a decent dashboard solution, we've been running our own. We absolutely wanted to use D3 and we wanted to have multiple, toggleable pages.
This might just do the trick.
Because of the lack of a decent dashboard solution, we've been running our own. We absolutely wanted to use D3 and we wanted to have multiple, toggleable pages.
This might just do the trick.
I have tried most of those dashboard services and the best one by far for me is http://klipfolio.com/
I see nothing but a link to self. Did something break?
The youtube video was blocking the sign up form for me -- I had to remove that div in "Inspect Element" before I could make an account...
The title remembers me on http://beautifulmind.io/ which was on HN some days ago
The app's assets (main.js and main.css) aren't being served properly. So right now I can only see a link to self on the homepage.
Is this always going to be free? Because there are other competitors in this space currently with premium solutions...
Hi,
For the beta, yes, which should last a couple of months. I will investigate turning it into a SaaS, but for now I will go and find some work.
For the beta, yes, which should last a couple of months. I will investigate turning it into a SaaS, but for now I will go and find some work.
This looks very slick, kudos!
Small nitpick: why do I need to give myself a username when signing up?
Small nitpick: why do I need to give myself a username when signing up?
Much of the text in the page is not visible, and the layout is broken.
Chrome 20 on Ubuntu 12.04.
Chrome 20 on Ubuntu 12.04.
Independent of what it does and all, you have designed the UI really good. Kudos for that.
https://dashku.com/ is not working for me.
Website is still down, that's a shame - OP can't even get the homepage back online before showing HN.
30 minutes to bring back a homepage is pretty bad man, also your personal homepage http://paulbjensen.co.uk is returning NO CONTENT, is that on the same (bricked) server?
30 minutes to bring back a homepage is pretty bad man, also your personal homepage http://paulbjensen.co.uk is returning NO CONTENT, is that on the same (bricked) server?
Hi,
Apologies for the downtime.
It turns out ulimit was set to 1024 when I thought I'd configured it to unlimited.
As for paulbjensen.co.uk, I removed that site about 3 months ago but didn't remove the link to it in HN. As for your question about the same bricked server, the answer is no, I run Dashku on a different server to the server that was hosting paulbjensen.co.uk.
Apologies for the downtime.
It turns out ulimit was set to 1024 when I thought I'd configured it to unlimited.
As for paulbjensen.co.uk, I removed that site about 3 months ago but didn't remove the link to it in HN. As for your question about the same bricked server, the answer is no, I run Dashku on a different server to the server that was hosting paulbjensen.co.uk.
thank you for this! just created a awesome dashboard very quick! :)
why not create a kickstarter project and raise enough money so you can open source the code? i would definitely contribute!
why not create a kickstarter project and raise enough money so you can open source the code? i would definitely contribute!
So this is free for now, what are you thinking of charging?
demo page doesn't work in Chrome. Map doesn't do anything. updating text is in a thin column down the left side that scrolls off below the main image.
(Error code: ssl_error_no_cypher_overlap)
Not loading for me.
website is down
Error: ENOENT, no such file or directory '/root/apps/status/client/static/assets/main/1340976517230.html' at Object.openSync (fs.js:240:18) at Object.readFileSync (fs.js:128:15) at ServerResponse.<anonymous> (/root/apps/status/node_modules/socketstream/lib/client/http.js:34:26) at EventEmitter.<anonymous> (/root/apps/status/app.coffee:48:16) at EventEmitter.emit (/root/apps/status/node_modules/socketstream/node_modules/eventemitter2/lib/eventemitter2.js:319:22) at Router.route (/root/apps/status/node_modules/socketstream/lib/http/router.js:18:15) at Object.handle (/root/apps/status/node_modules/socketstream/lib/http/index.js:96:54) at next (/root/apps/status/node_modules/socketstream/node_modules/connect/lib/proto.js:190:15) at /root/apps/status/node_modules/socketstream/node_modules/connect/lib/middleware/session.js:299:9 at /root/apps/status/node_modules/socketstream/node_modules/connect/lib/middleware/session.js:322:9