Overlock – IoT Exception Tracking(overlock.io)
overlock.io
Overlock – IoT Exception Tracking
https://overlock.io/
6 comments
From our research, it seems that a lot more of the hardware is running linux than we would've expected [1]. This makes linux a great starting point - but you are right there's other platforms to look at too.
Regarding data use - you're completely right. In previous production IoT projects we've worked on we've had in excess of 10X more debug data than actual useful information coming out, lest we're not able to debug problems! We've got a few tricks up our sleeve, the main one being that this only sends data when there is an exception.
[1]: https://ianskerrett.wordpress.com/2017/04/19/iot-developer-t... (Point 4)
Regarding data use - you're completely right. In previous production IoT projects we've worked on we've had in excess of 10X more debug data than actual useful information coming out, lest we're not able to debug problems! We've got a few tricks up our sleeve, the main one being that this only sends data when there is an exception.
[1]: https://ianskerrett.wordpress.com/2017/04/19/iot-developer-t... (Point 4)
In my experience, the Linux-based devices tend to be low-unit-number type devices such as gateways (or prototypes).
Also, that point 4 in the link does not have the most common type of embedded OS... i.e. "None". Bare metal is very common.
Another point here is that (according to that link), Raspbian is the most common. That indicates the data is coming from the "Maker" community, not the professional embedded developers... we don't tend to use RPi's for much.
Anything that will be remotely cost-sensitive (as anything produced in large numbers will be) will tend to be microcontroller-based.
Like you say, part of the solution is only sending the data that is needed. I would imagine another part of the solution that you could implement is some sort of shared string-table and hence transmitting only event-codes.
Anything that will be remotely cost-sensitive (as anything produced in large numbers will be) will tend to be microcontroller-based.
Like you say, part of the solution is only sending the data that is needed. I would imagine another part of the solution that you could implement is some sort of shared string-table and hence transmitting only event-codes.
We're really targeting gateways at the moment. Obviously, having overlock running on end devices would be ideal, and is certainly a future goal. For the moment though, running on the gateway seems to provide the most utility and is easily acheivable given the proliferation of linux based gateways.
Compression is also certainly on the cards, but really comes in to its own once we have the clients running on resource-constrained devices.
Compression is also certainly on the cards, but really comes in to its own once we have the clients running on resource-constrained devices.
I more or less wondered the same.
The site has not a lot of info, but shows a screenshot around a NPE. These won't happen on most embedded IoT devices. Instead they just either crash or are behaving totally weird, thanks to C, No-OS or an RTOS and no memory protection.
How is this specifically different and/or more well suited to IoT/IoS than a regular logging/APM service?
There are 2 main areas we're focusing on for an MVP:
1) Automatically pulling in state and logs for associated nodes because IoT devices are all about how data moves through a system. Errors are often due to mistakes in code in other nodes, so we're trying to make it easier to pin this class of problem down. They're generally really hard to reproduce and only occur 'in the field'.
2) Allowing really simple logging of 'lifecycle events' because IoT devices are generally powered for months/years at a time in an uncontrolled environment compared to other software.
We've got loads of other things we want to do in this space too, but we feel that these two are what's missing from exception tracking at the moment. We believe that IoT needs better tooling to get more reliable and secure.
We've got loads of other things we want to do in this space too, but we feel that these two are what's missing from exception tracking at the moment. We believe that IoT needs better tooling to get more reliable and secure.
The question was how your solution is better than a hosted logging solution. An "automatically pulling in state and logs for associated nodes" requirement is achieved in cloud log hosting by placing tags on the logs. This tag and this tag are correlated, so show these things to the user, for example. This is accomplished with a search engine. This is assuming all data is sent to centralized storage.
I was, in part, responsible for creating a cloud logging service. I've never heard the term "lifecycle events" before, but it sounds like something where a device has a life and over time it has events occur, some of which are logged, and some of which are not. Whether or not sending these logs in a controlled way or not would not seem to matter much if the logging was aggregated in a centralized location. The "benefits" of not sending all log data from a device are not presumed to be that important, given bandwidth is cheap and storage even cheaper, increasingly over time for both.
Controlling what information is and is not sent from a device might be interesting (assuming correlation with this and this device, log to here), but you can't depend on the device to have the room for storing that stuff for long, or having much intelligence in sending it on (unless you run something like Particle's devices which have some cloud connected logic built in).
A dedicated edge device for this task might be interesting, but some IoT use cases won't have those edge devices available (at least for some time) on customer premise (home?), so logging to the cloud is presumed to be the only solution here. This reminds me, Microsoft just released some IoT stuff recently that may be related.
The question for how your product compares to existing cloud logging solutions will continue to resurface for y'all. Figuring out a good answer to that may make all the difference in how you go to market!
I was, in part, responsible for creating a cloud logging service. I've never heard the term "lifecycle events" before, but it sounds like something where a device has a life and over time it has events occur, some of which are logged, and some of which are not. Whether or not sending these logs in a controlled way or not would not seem to matter much if the logging was aggregated in a centralized location. The "benefits" of not sending all log data from a device are not presumed to be that important, given bandwidth is cheap and storage even cheaper, increasingly over time for both.
Controlling what information is and is not sent from a device might be interesting (assuming correlation with this and this device, log to here), but you can't depend on the device to have the room for storing that stuff for long, or having much intelligence in sending it on (unless you run something like Particle's devices which have some cloud connected logic built in).
A dedicated edge device for this task might be interesting, but some IoT use cases won't have those edge devices available (at least for some time) on customer premise (home?), so logging to the cloud is presumed to be the only solution here. This reminds me, Microsoft just released some IoT stuff recently that may be related.
The question for how your product compares to existing cloud logging solutions will continue to resurface for y'all. Figuring out a good answer to that may make all the difference in how you go to market!
I work for overlock, we're trying to build a debugging tool which specifically addresses the unique needs of distributed systems. We're really looking for early feedback and for companies who are deploying IoT systems. Happy to answer any questions!
Congratulations on your beta launch! Really interesting.
Some questions: FAQ reads that channel from devices to Overlock employs MQTT(S). I guess that means MQTT-SN, using TLS layer for authentication. Which auth methods do you support? Do you support validation of the client and the server? Can the gateway act as such also for Overlock or do devices need to open the MQTT channel directly?
Some questions: FAQ reads that channel from devices to Overlock employs MQTT(S). I guess that means MQTT-SN, using TLS layer for authentication. Which auth methods do you support? Do you support validation of the client and the server? Can the gateway act as such also for Overlock or do devices need to open the MQTT channel directly?
Thanks!
That should probably be written as secure-MQTT, I'll get that changed. So we are just using vanilla MQTT over TLS.
We're currently using token-based authentication but certainly open to supporting other options. Did you have any suggestions?
I'm not quite sure what you mean about validation? In general, the gateway will already be in communication with the end devices over whatever protocol they're using. So it makes sense to log state etc. from the gateway for those devices. In that sense, the gateway does act as a gateway for overlock.
That should probably be written as secure-MQTT, I'll get that changed. So we are just using vanilla MQTT over TLS.
We're currently using token-based authentication but certainly open to supporting other options. Did you have any suggestions?
I'm not quite sure what you mean about validation? In general, the gateway will already be in communication with the end devices over whatever protocol they're using. So it makes sense to log state etc. from the gateway for those devices. In that sense, the gateway does act as a gateway for overlock.
What's your pricing like? This is the one thing that I'm always cautious about with these kinds of services as most of them are designed with web in mind. This lends the pricing to be per node/machine, which makes it untenable for an IoT deployment.
Yeah, you're completely right. We've designed everything else from the ground up to be for IoT, so that follows for pricing too. We're not quite ready to announce it just yet though.
What unique needs have you identified?
In an IoT deployment, an issue that is detected on one device (e.g. an exception on the server) may actually originate on an entirely different device (e.g. the end device sends some incorrect data). Overlock collects together the state of all the devices involved in a given request and the associated logs in one place so that you can track the error back to where it originated.
So this is Sentry but closed source and specifically focusing on embedded platforms?
We'll be open sourcing large parts of it when it's out of private beta
I can run a full copy of Sentry from source, which is a major difference from "large parts" being open sourced. Admittedly, I can't do any of the payment stuff and run a direct competitor, but the rest is there.
If your IoT platform is even _capable_ of exceptions, you’re doing it wrong and at too high a level.
I'm not sure I understand this comment at all. What do you mean "_capable_ of exceptions"? And why would that be doing it wrong. I can't tell if you are saying anything at all or just being inflammatory.
If you’re not doing minimalist bare metal (which will not support exceptions), you’re burdening your users with greater power requirements, bugs, and required firmware updates (or security holes) for the life of the device. Don’t put Linux on there. Make it do the bare minimum it has to do, expose minimal interfaces, and flash it into your MCU forever. Folks who design IoT should be planning for a mean lifetime of 20 years for their products. And no, I’m not going to buy shit that requires me to update firmware on three dozen different things in my house.
the firmware update angel is orthogonal to wether or not you have linux or bare metal. I don't have a strong opinion about Linux on IoT, but I do think that if you really want a mean lifetime of 20 years then your IoT device is going to need to be upgradeable, the internet is going to change and you are going to have to make changes. Also I'm not insane enough to think that I could write network drivers for bare metal and DNS resolvers without pulling in libraries, and HTTP code, etc. At some point you will realize that you will save yourself a lot of headaches with Linux and be far more likely to get your 20 year life out of your IoT devices.
Well, if I were to speak as a homeowner and not an engineer, I don’t give a rats ass what’s going to change. At most, I’d agree to upgrade _one_ device in my house, not every single little IoT trinket embedded in the walls. And then not too often, say once every 5 years or so. That necessitates simple, well tested code that does the bare minimum it has to do, and runs on a low power MCU where Linux in its full form can’t run.
Also, data-reduction is of prime importance for IoT devices, does Overlock have any tricks to play here?