Show HN: UPID – a unique ID with built-in prefix(github.com)
github.com
Show HN: UPID – a unique ID with built-in prefix
https://github.com/carderne/upid
1 comments
Blog post:
UPID is as UPID does
https://rdrn.me/upid/
---
BTW: Hi/Lo scheme for Primary Keys / IDs was invented long before Stripe existed. Partcularly, I used short up to 3 letters prefixes to make IDs Human-readable (and writable!) more than 20 years ago. 5 years later Amazon started using similar scheme in their AWS product, i.e i-NNNNN for instances, etc.[1].
In theory there are 2 kinds of ID schemes: technical/surrogate and natural/business/semantic (i.e. those derived from the natural properties of the object the ID refers to).
Adding a prefix with the type, makes ID scheme hybrid. "Hi" part (prefix) is natural, and "Lo" part is technical. In case of UPID/ULID/UUIDv7 even the "Lo" part is hybrid, since it includes an information when the entity was created (or at least inserted into the database/system).
---
[1] List of some AWS ID prefixes:
UPID is as UPID does
https://rdrn.me/upid/
---
BTW: Hi/Lo scheme for Primary Keys / IDs was invented long before Stripe existed. Partcularly, I used short up to 3 letters prefixes to make IDs Human-readable (and writable!) more than 20 years ago. 5 years later Amazon started using similar scheme in their AWS product, i.e i-NNNNN for instances, etc.[1].
In theory there are 2 kinds of ID schemes: technical/surrogate and natural/business/semantic (i.e. those derived from the natural properties of the object the ID refers to).
Adding a prefix with the type, makes ID scheme hybrid. "Hi" part (prefix) is natural, and "Lo" part is technical. In case of UPID/ULID/UUIDv7 even the "Lo" part is hybrid, since it includes an information when the entity was created (or at least inserted into the database/system).
---
[1] List of some AWS ID prefixes:
i-
vol-
snap-
ami-
eni-
sg-
vpc-
subnet-
acl-
rtb-
igw-
dopt-
vpn-
vgw-
cgw-
nat-
rds-
lb-Thanks for this, hadn’t thought of this in terms of Hi/Lo. The similarity is limited though because there isn’t any sequence of Hi values…?
AWS also a great example! But I wonder if they’re storing them as fixed 128bit and getting the juicy performance benefits.
AWS also a great example! But I wonder if they’re storing them as fixed 128bit and getting the juicy performance benefits.
Just because the "Hi" part is categorical rather than numeric and has a low cardinality doesn't make it any less of a Hi/Lo scheme. This is my own classification, but I think I've read at least one blog where the author called similar prefixes a Hi/Lo scheme
The prefixes are primarily used to create human-readable IDs, which will help avoid human errors and make debugging easier for developers
The original purpose of the Hi/Lo algorithm[1] was mainly to allow the client side to generate PKs in batches (as a performance optimization, offline operation, or building entire data structures on the client side), all of which can be replaced today by UUIDs/ULIDs and the like
---
[1] https://stackoverflow.com/questions/282099/whats-the-hi-lo-a...
The prefixes are primarily used to create human-readable IDs, which will help avoid human errors and make debugging easier for developers
The original purpose of the Hi/Lo algorithm[1] was mainly to allow the client side to generate PKs in batches (as a performance optimization, offline operation, or building entire data structures on the client side), all of which can be replaced today by UUIDs/ULIDs and the like
---
[1] https://stackoverflow.com/questions/282099/whats-the-hi-lo-a...
The biggest downside (imo) of the Stripe approach is that you either have to store them as strings, which isn't great, or strip/prepend in your API layer.
So I made a new thing called UPID. It's most similar to ULID, as it includes a time-component (but with lower precision) and also encodes to a base32 alphabet. But it also includes a 20-bit prefix (allowing four characters). So it can be stored as a UUID or anywhere a 128-bit blob goes, and it always knows what its prefix is because it's baked in.
I've made implementations for Python, Rust and TypeScript, as well as a Postgres extension (with a upid datatype). Mentioned it a few weeks ago in a HN discussion and people seemed pretty interested so thought I'd post it as a submission.
In Rust it looks like this:
There are more examples in the repositories.
Repo (Python, Rust, Postgres): https://github.com/carderne/upid
TypeScript: https://github.com/carderne/upid-ts
Demo site: https://upid.rdrn.me/