I'm working on Pocketdata - a personal, private AI data plane.
The idea is to take boring components: PostgreSQL, Bifrost (LLM gateway), Open WebUI, LanceDB, Agentgateway (MCP and OpenAPI gateway) and deploy them in Fly.io. One Fly.io "org" per user. The closest equivalent is blaxel.ai, but it caters for AI SaaS startups, not individual customers.
The combination of the fact that Fly secrets are visible only from within the apps, distroless containers, and transparent data encryption for PostgreSQL assures that the service (Pocketdata) provider cannot access their data, only the infrastructure provider (Fly.io) theoretically can, but practically speaking, this gives an extremely high degree of privacy assurance.
> It seems like a web version of claude code + skills + marketplace, but with an encrypted database, and permissions that let you feel safe using yolo mode. I’m going to try it.
Architecture TLDR: In Fly.io, org-per user, manage keys in Fly secrets. A Postgres db with transparent data encryption (TDE), the master key is stored on user's computer in the keychain or in the password manager. Thus the nobody can read the data at rest. All containers are distroless so nobody can ssh onto them. Postgres is backed up via pgBackRest to Wasabi object storage with customer-provided encryption keys that are injected into the containers via Fly secrets.
Apart from the database for record-like things (chats, emails, tables), vectors for larger things (such as web pages) are stored in serverless LanceDB on Wasabi, too.
Agents/apps themselves (Open WebUI, Zero Mail, etc.) are separate Fly apps, and have their separate schemas and users in Postgres. They also cannot go to public internet directly (prohibited via https://community.fly.io/t/new-feature-network-policies/1917...), only to Postgres, Bifrost, and Agentgateway.
Postgres, pgBackRest, pgBouncer and another Go sidecar for pg (a la Pocketbase, but with Postgres backend), live in a single container managed by Horust. Bifrost and Agentgateway live in separate containers, but the same Fly machine. This machine might be 1.5gb, 4 vCPUs. LanceDB is on a separate machine because it needs burstable memory and may be infrequently used.
All machines (core, lance, and all individual app/agent machines) are suspendable, so they almost don't cost anything when not in use.
It's not super easy to port a library with domain-specific ML/DS algorithms built around Pandas, such as accepting Pandas dataframes as arguments to its functions and class constructors.
I don't buy this "object storage + Iceberg is all you need for OLAP" hype. If the application has sustained query load, it makes sense to provision servers rather than go pure serverless. If there are already provisioned servers, it makes sense to cache data on them (either in their memory or SSDs) to avoid round-trips to object storage. This is the architecture of the latest breed of OLAP databases such as Databend and Firebolt, as well as the latest iterations of Redshift's and Snowflake's architecture. Also, this is the approach of the newest breed of vector databases, such as Turbopuffer.
For OLAP use cases with real-time data ingestion requirements, object-storage-only approach also leads to write amplification. Therefore, I don't think that architectures like Apache Pinot, Apache Paimon, and Apache Druid are going anywhere.
Another problem with "open table formats" like Iceberg, Hudi, and Delta Lake is their slow innovation speed.
Every once in a while, before falling asleep I enter a "mellow" state in which I can conjure mental images dramatically more vividly than normally. I would say that normally it varies between 10-50% of vividness (also depends on the state of mind, the day, whether I'm under the influence of different chemicals), and in this state of mind it's 90%+ vivid.
And I don't confuse this state of mind with lucid dreaming: I had spontaneous lucid dreams a few times. They were not very vivid, just as my normal dreams (I dream in images, and they are maybe around 50% vivid).
So, there is definitely a way to experience variation of vividness of mental imagery, by a single observer.
> This one seems theoretically more interesting than some others but practically less useful.
Are there other factors why you think AutoBNN is not practically useful, apart from being based on the wrong foundation (which was a mistaken belief of yours)?
The big question is whether we really want more centralised cloud applications of this sort in the AI future, with its potentially exacerbated risks of data breaches, identity theft, spam, etc.
Having a "Rama" for local-first, truly distributed (Solid Project style), self-sovereign-identity based apps would be differentially better probably.
You can't do it in your basement as of 2023. Very important qualification. It's entirely plausible that continuous evolution of ML architectures will lead to general AI which anyone can start on their phone and computer and learn online from there.
You have excellently described the job of business analysts and system architects, but this is not the job of 90% of programmers today, including senior-level. Part of this is already done by other people and doesn't require specific programming skills, hence, at the very least, programmers will lose their privileged position. Another part of it is actually too hard for most people who are currently employed as programmers to do on a decent level (such as meaningfully hacking on Linux kernel).
There will be just a short transition period. In 10 years, AI will be writing most of code, and in 20 years - nearly all code. People will do only architecture/business analysis.
Perhaps people on HN start sensing that successors of Github Copilot will take their programming job. Rightly so.
Personally, I think that in the age of AI programming any notions of code licensing should be abolished. There is no copyright for genes in nature or memes in culture; similarly, these shouldn't be copyright for code.
The idea is to take boring components: PostgreSQL, Bifrost (LLM gateway), Open WebUI, LanceDB, Agentgateway (MCP and OpenAPI gateway) and deploy them in Fly.io. One Fly.io "org" per user. The closest equivalent is blaxel.ai, but it caters for AI SaaS startups, not individual customers.
The combination of the fact that Fly secrets are visible only from within the apps, distroless containers, and transparent data encryption for PostgreSQL assures that the service (Pocketdata) provider cannot access their data, only the infrastructure provider (Fly.io) theoretically can, but practically speaking, this gives an extremely high degree of privacy assurance.
The latest update on the project: https://engineeringideas.substack.com/p/tasklet-is-the-o1-mo...