Quick tip when working with Claude Code and Git: When you're happy with an intermediate result, stage the changes by running `git add` (no commit). That makes it possible to always go back to the staged changes when Claude messes up. You can then just discard the unstaged changes and don't have to roll back to the latest commit.
This type of service is becoming increasingly prevalent among European energy suppliers for their residential customers. Beyond providing a revenue stream for consumers this model aggregates distributed energy resources (home batteries, EV's, PV systems) into a one virtual power plant. This enables the storage of surplus energy generated during solar peaks and dispatch back to the grid during periods of high demand. I find it a fascinating domain to work in!
Totally irrelevant on the subject, but the screen recording in the article caught my attention: what tool does one use to create the screen recording with zoom and mouse tracking?
Otel seems complicated because different observability vendors make implementing observability super easy with their proprietary SDK’s, agents and API’s. This is what Otel wants to solve and I think the people behind it are doing a great job.
Also kudos to grafana for adopting OpenTelemetry as a first class citizen of their ecosystem.
I’ve been pushing the use of Datadog for years but their pricing is out of control for anyone between mid size company and large enterprises. So as years passed and OpenTelemetry API’s and SDK’s stabilized it became our standard for application observability.
To be honest the documentation could be better overall and the onboarding docs differ per programming language, which is not ideal.
My current team is on a NodeJS/Typescript stack and we’ve created a set of packages and an example Grafana stack to get started with OpenTelemetry real quick.
Maybe it’s useful to anyone here: https://github.com/zonneplan/open-telemetry-js
This seems like a great tool! I like how it "simply" works by putting a view on top of the actual table. This concept is widely used in the data engineering world (in dbt).
There must be an easier way to write migrations for pgroll though. I mean, JSON, really?
What a nice writeup. I've listened to jungle and related styles on and off over the years but I wasn't aware the music had played such a big role in 90's game music!
I see your point. Some of our measurements are counters and interpolate missing data via certain rules. We store these interpolations in a separate table with exactly the same format and join the two tables directly when querying the data. Since the real value always takes precedence over the interpolated value this is just a COALESCE(realmeasure0000, interpolation0000) statement.
This is super fast when taking advantage of postgres' partition-wise joins.
The schema is organized by how we need to query it which is mostly by fetching all measurements for an entire day or month. Querying statistics on those metrics is not often needed, maybe a few times per week or day, so we do that in Snowflake if necessary. Or create a specialized streaming data flow when it the stats are queried more often.
This system contains measurements and state of physical devices (time series). It’s designed for both heavy write and read, with slight emphasis on write.
Each table is one type of device and contains 1 to 5 different measurements/states. But here’s the trick: because data is queried with minimum bucket size of 15minutes I figured we could just create a column for each measurement + quarter of the day (i.e. measure0000, measure0015), so that’s 100 columns for each measurement (96 quarter + 4 for DST), include the date in the key, et voila: excellent write performance (because it’s mainly UPDATE queries) and good read performance.
Okay, the queries to make sense of the data aren’t pretty, but can be generated.
I find it really cool how effective this is for time-series data without Postgres extensions (we’re on RDS).
It’s really something you don’t know how this applies to your Postgres DB, until you run into a situation where you do.
The author explains this very well, it’s a good read! I’ve learned about this padding little over a year ago, while I was designing a data intensive application with a colleague. I was skeptical about the advantage at first, but for our specific design, where we have 100 to 480+ columns in one table it makes a huge difference on table store size. Not so much on the indexes, though.
Where I work we’ve set up OpenTelemetry SDK in the applications to expose traces, logs and metrics.
Grafana agent as OTEL collector on the application hosts, Grafana Tempo as backend for traces, Loki for logs and Prometheus for Metrics.
The cool thing about Tempo it generates metrics for ingested spans and their labels (spanmetrics) so this allows us to explore “unknown unknowns” as the author calls it in a very cost efficient way.
This is awesome, can’t wait to play TH again.
Totally forgot about this game, but i remember playing a demo version of the original game. The game was good fun. What impressed me most back then was the quality and “realism” of the videos (mind you, it was 1997)
Which projects are standing out in this space right now?