@jabiko thanks for the note. Glad our product is working so well for you.
re:Azure we are working on some new things :) . Feel free to drop me a message if you'd like to discuss further ([email protected]).
I should have mentioned that correlating observability data (or sometimes product metrics collected via Prometheus) with product data (or any other data really like business data) can be super useful and totally possible with Promscale because PostgreSQL is under the hood. So you could have a copy that data into the same PostgreSQL instance used by Promscale or maybe use Foreign Data Wrappers (1). This would allow you to analyze, for example, api request latency by product plan the customer is subscribed to or based on which feature flags are enabled for their account, etc. without having to add all those attributes as labels to all your metrics which can be technically complex and also costly.
Thanks for sharing. Makes total sense. As soon as you have to query across span attributes AND span parent/child relationships you need a more sophisticated query language.
100% agree. OpenTelemetry solves half of the problem by standardizing and unifying instrumentation. With Promscale our aim is to solve the other half with a unified storage and query language for metrics, traces and logs. This release is a big step towards realizing that vision. It includes:
- Integrated with Grafana for dashboarding as well as for the tracing experience. Note that Grafana offers a very similar experience to Jaeger for traces (they reused the Jaeger code) and we extend that experience with a set of dashboards that deliver additional visibility into your services (the blog post has more details about those)
- SQL for querying and correlating traces and metrics.
So pretty much everything you said except logging which is in our roadmap.
That's definitely our goal: to offer a solution that is very easy to self-host that covers most of the features APM products provide within Grafana so you don't have to go back and forth between different tools. And a key advantage is that you can customize the experience and extend it in any way you want because it's made of editable Grafana dashboards populated with data from SQL queries.
We are also super excited to offer this capability to all our TimescaleDB users.
Feel free to ping us on our community Slack (https://slack.timescale.com #promscale channel or drop me an email ramon[at]timescale.com). We would be happy to answer any question you have, learn more about your needs and get your feedback about the product.
SQL definitely opens lots of possibilities to analyze the data. And lots of visualization tools already integrate with PostgreSQL/TimescaleDB which Promscale uses to store data so you have a lot of options to pick from.
Would you mind sharing some of the problems you've had to use SQL for?
As @akulkarni said, Promscale supports Prometheus metrics and OpenTelemetry traces natively and there are different ways to correlate both signals. I am actually delivering a talk that goes over the different ways you can correlate Prometheus and OpenTelemetry data tomorrow at Prometheus Day :)
One is via adding exemplars to your Prometheus metrics that link to specific traces that are representative of the value of those metrics. In Promscale you can store all that information and then display it in Grafana as explained in 1. That's the way that is most often discussed but typically involves deploying one backend for metrics and one backend for traces instead of just one as is the case with Promscale.
With Promscale you can also correlate metrics and traces using SQL joins. That opens a whole set of new possibilities. For example, imagine if you could retrieve the slowest requests happening on services running on nodes where CPU usage is high to understand how they are impacting their performance. Or imagine you are seeing a specific OOM error often in your traces and you could run a SQL query to look at the evolution of memory usage in the last 24 hours of nodes where those OOM errors are more frequent to see if you spot anything strange happening. You could even go a step forward andretrieve in the same query what processes are consuming the most memory in those nodes to pinpoint the processes that could be causing the issue.
Hey rudasn, I'm Ramon, the author of the blog post.
You can definitely use the tools mentioned in the blog post on bare metal as well. All components in the demo environments described in the blog post run on Docker on a host, no Kubernetes or cloud services. It would also be possible to run them directly on the host with no Docker.
Usually the challenge is that many of those tools have have been designed with microservices and cloud native environments in mind. So typically they are easier to set up in those environments and most of the documentation you'll find online is about those types of setups.