no_circuit · HackerTrans

no_circuit

no profile record

コメント

9 日前議論

Sounds like the lesson learned is using the right tool for the job -- reusing Kubernetes in an existing cluster to spin up sandboxes is a fair initial path to start offering the service. But Kubernetes likely isn't meant for rapid churn of workloads, here sandboxes.

The architecture to me seemed very similar to SeaweedFS [1] (Facebook Haystack [2]) except with an extra layer for sandbox-hosting nodes. Like requests go into a master, or the global load balancer, then to a volume server, which in turn knows where the files/sandboxes should go. There is no need for sandboxes to be managed with the Kubernetes overhead since the the nodes/bare metal servers probably have scheduling taints on them to preserve the memory/cpu for the sandboxes.

[1] https://github.com/seaweedfs/seaweedfs

[2] https://www.usenix.org/legacy/event/osdi10/tech/full_papers/...

先月議論

My impression with this space is that you'd need to fundraise startup-style, which I'm assuming you'll do, to catch up with everyone that is doing a similar thing.

The problem space and solution has been around for a while in big tech, and now there is a handful known products publicly known, and probably a couple dozen still secret ones. It is just now with AI/agents volume, there probably needs to be an easier solution for quick narrowly focused VCS views.

For filesystem mount, usually FUSE-FS, of a version control system to enable multiple viewers without transferring a lot of data see some current/previous implementations:

- Google: Piper via CitC (Clients in the Cloud) often used with Cider (web IDE)

- Meta: Sapling on EdenFS (from what I read, never worked there)

- Rational Clearcase, anyone else remember mounting VOBs?

The main issue I see is with the site -- it just seems like a big blob of AI-generated text I need to understand what is going on. The cool part wasn't even shown off: your GitHub UI clone that you can get to from seeing the benchmark code.

FYI, I also think the 4-way arrows logo has been used before, and still might be in use. I tried searching, but I think I saw a multi-colored one, maybe in a UK-based IT corporate training company's class I attended.

6 か月前議論

Why stop at restricting pull requests? I wouldn't want spam issues either. New issues and contributors should be gated at the "discussion" stage.

6 か月前議論

Impressive looking project generated with AI help. Have similar goals of having an artifacts system myself.

I think the approach of multi-format, multi-UI, and new (to you) programming language isn't optimal even with AI help. Any mistake that is made in the API design or internal architecture will impact time and cost since everything will need to be refactored and tested.

The approach I'm trying to take for my own projects is to create a polished vertical slice and then ask the AI to replicate it for other formats / vertical slices. Are there any immediate use cases to even use and maintain a UI?

So a few comments on the code:

- feature claims rate limiting, but the code seems unused other than in unit tests... if so why wasn't this dead code detected?

- should probably follow Google/Buf style guide on protos and directory structure for them

- besides protos, we probably need to rely more on openapi spec as well for code generation to save on AI costs, I see openapi spec was only used as task input for the AI?

- if the AI isn't writing a postgres replacement for us, why have it write anything to do with auth as well? perhaps have setup instructions to use something like Keycloak or the Ory system?

8 か月前議論

Yes, it is huge to spread the work out on embedded UIs in chat interfaces. But I think the design direction is going, is exactly the same direction of how Google Assistant, Amazon Alexa, or any of the other assistants work, ifkyk.

The MCP community is just reinventing, but yes, improving, what we've done before in the previous generation: Microsoft Bot Framework, Speaktoit aka Google Dialogflow, Siri App Shortcuts / Spotlight.

And interactive UIs in chats go back at least 20 years, maybe not with an AI agent attached...

The next thing that will be reinvented is the memory/tool combination, aka a world model.

9 か月前議論

If your binary has a small function set, probably not. But in a use case if you want to proxy/intercept cloud APIs, then something like Google APIs has 34K message types:

    git clone https://github.com/googleapis/googleapis.git
    cd googleapis
    find . -name '*.proto' -and -not -name '*test*' -and -not -name '*example*' -exec grep '^message' {} \; | wc -l

I think this more speaks to the tradeoff of not having an IDL where the deserializer either knows what type to expect if it was built with the IDL file version that defined it, e.g., this recent issue:

https://github.com/apache/fory/issues/2818

But now I do see that the 4096 is just arbitrary:

    If schema consistent mode is enabled globally when creating fory, type meta will be written as a fory unsigned varint of type_id. Schema evolution related meta will be ignored.

9 か月前議論

Yes, I agree that protos usually should only be used at the serialization boundary, as well as the slightly off-topic idea that the generated code should be private to the package and/or binary.

So to reflect the real‑world practices, the benchmark code should then allocate and give the protobuf serializer an 8K Vec like in tonic, and not an empty one that may require multiple re-allocations?

9 か月前議論

IMO, not a fair benchmark.

I can see the source of an 10x improvement on an Intel(R) Xeon(R) Gold 6136 CPU @ 3.00GHz, but it drops to 3x improvement when I remove the to/from that clones or collects Vecs, and always allocate an 8K Vec instead of a ::Default for the writable buffer.

If anything, the benches should be updated in a tower service / codec generics style where other formats like protobuf do not use any Fory-related code at all.

Note also that Fory has some writer pool that is utilized during the tests:

https://github.com/apache/fory/blob/fd1d53bd0fbbc5e0ce6d53ef...

Original bench selection for Fory:

    Benchmarking ecommerce_data/fory_serialize/medium: Collecting 100 samples in estimated 5.0494 s (197k it
    ecommerce_data/fory_serialize/medium
                            time:   [25.373 µs 25.605 µs 25.916 µs]
                            change: [-2.0973% -0.9263% +0.2852%] (p = 0.15 > 0.05)
                            No change in performance detected.
    Found 4 outliers among 100 measurements (4.00%)
      2 (2.00%) high mild
      2 (2.00%) high severe

Compared to original bench for Protobuf/Prost:

    Benchmarking ecommerce_data/protobuf_serialize/medium: Collecting 100 samples in estimated 5.0419 s (20k
    ecommerce_data/protobuf_serialize/medium
                            time:   [248.85 µs 251.04 µs 253.86 µs]
    Found 18 outliers among 100 measurements (18.00%)
      8 (8.00%) high mild
      10 (10.00%) high severe

However after allocating 8K instead of ::Default and removing to/from it for an updated protobuf bench:

    fair_ecommerce_data/protobuf_serialize/medium
                            time:   [73.114 µs 73.885 µs 74.911 µs]
                            change: [-1.8410% -0.6702% +0.5190%] (p = 0.30 > 0.05)
                            No change in performance detected.
    Found 14 outliers among 100 measurements (14.00%)
      2 (2.00%) high mild
      12 (12.00%) high severe

9 か月前議論

Is 4096 types enough for everyone?

https://github.com/apache/fory/blob/fd1d53bd0fbbc5e0ce6d53ef...

9 か月前議論

Are the benchmarks actually fair? See:

https://github.com/apache/fory/blob/fd1d53bd0fbbc5e0ce6d53ef...

It seems if the serialization object is not a "Fory" struct, then it is forced to go through to/from conversion as part of the measured serialization work:

https://github.com/apache/fory/blob/fd1d53bd0fbbc5e0ce6d53ef...

The to/from type of work includes cloning Strings:

https://github.com/apache/fory/blob/fd1d53bd0fbbc5e0ce6d53ef...

reallocating growing arrays with collect:

https://github.com/apache/fory/blob/fd1d53bd0fbbc5e0ce6d53ef...

I'd think that the to/from Fory types is shouldn't be part of the tests.

Also, when used in an actual system tonic would be providing a 8KB buffer to write into, not just a Vec::default() that may need to be resized multiple times:

https://github.com/hyperium/tonic/blob/147c94cd661c0015af2e5...

10 か月前議論

Yes, from https://www.txse.com/solutions:

  TXSE's goal is to provide greater alignment with issuers and investors and address the high cost of going and staying public.

The alignment part translates IMO to avoiding political / social science policy issues like avoiding affirmative action listing requirements like the Nasdaq Board Diversity Rules that was just recently repealed: https://corpgov.law.harvard.edu/2025/01/12/fifth-circuit-vac....

So it is as one might imagine, the formation was probably for similar reasons why owners are moving their company registration out of Delaware.

10 か月前議論

Right, for example, there are no comments on why something there is something called a PromiseIdAllocator that starts with the magic number "1":

https://github.com/currentspace/capn-rs/blob/a816bfca5fb6ae5...

Yet there is a public interface that allows for initialization with "0":

https://github.com/currentspace/capn-rs/blob/a816bfca5fb6ae5...

It's like the LLM was able to predict that 1 is needed in the protocol, but wasn't relevant to check in the boilerplate.

I don't have a problem with newtype-all-the-things to ensure correctness in some areas, but no comments/constants does not lead to confidence.

10 か月前議論

Regarding the concept, it's cool to see you using LLMs to quickly generate protocol versions.

But asking the community to review an AI-generated implementation week-old announced protocol, is more or less putting the recently coined-term AI "workslop" upon others. It doesn't really matter if it happens to be a good implementation or not.

There are two main issues I can think of right now:

1) Work going into the protocol is only useful for your implementation of it. The capnweb-core crate depends on the tokio runtime, and parts of the protocol/definitions are in the client crate:

https://github.com/currentspace/capn-rs/blob/a816bfca5fb6ae5...

What if someone wants to leverage work into the core parts of the protocol to use a different runtime or no-std?

2) The project has namespace squatted/swiped the best name for the official implementation of the project. I understand Rust/Crates-IO allows for this free-for-all, but isn't it entirely possible that Cloudflare already has Rust crates for this that they might open source? Or if someone else wants to make a competing implementation? Maybe it's just me, but I like to put my organization prefix on all my crates in the event I ever open source any of them.

Would you offer to transfer the crate names to Cloudflare if they were going to offer an implementation -- just like what happened with protobuf/Google?

12 か月前議論

How long ago did you try SQLx? Not necessarily promoting SQLX, but the `query_as` which lets one make queries without the live database macro has been around for 5 years [1].

For lower level libraries there is also the more downloaded SQLite library, rusqlite [2] who is also the maintainer of libsqlite3-sys which is what the sqlite library wraps.

The most pleasant ORM experience, when you want one, IMO is the SeaQl ecosystem [3] (which also has a nice migrations library), since it uses derive macros. Even with an ORM I don't try to make databases swappable via the ORM so I can support database-specific enhancements.

The most Rust-like in an idealist sense is Diesel, but its well-defined path is to use a live database to generate Rust code that uses macros to then define the schema-defining types which are used in the row structs type/member checking. If the auto-detect does not work, then you have to use its patch_file system that can't be maintained automatically just through Cargo [4] (I wrote a Makefile scheme for myself). You most likely will have to use the patch_file if you want to use the chrono::DateTime<chrono::Utc> for timestamps with time zones, e.g., Timestamp -> Timestamptz for postgres. And if you do anything advanced like multiple schemas, you may be out of luck [5]. And it may not be the best library for you if want large denormalized tables [6] because compile times, and because a database that is not normalized [7], is considered an anti-pattern by project.

If you are just starting out with Rust, I'd recommend checking out SeaQl. And then if you can benchmark that you need faster performance, swap out for one of the lower level libraries for the affected methods/services.

[1] https://github.com/launchbadge/sqlx/commit/47f3d77e599043bc2...

[2] https://crates.io/crates/rusqlite

[3] https://www.sea-ql.org/SeaORM/

[4] https://github.com/diesel-rs/diesel/issues/2078

[5] https://github.com/diesel-rs/diesel/issues/1728

[6] https://github.com/diesel-rs/diesel/discussions/4160

[7] https://en.wikipedia.org/wiki/Database_normalization

昨年議論

I'm assuming that the PaaS/IaaS providers already have solutions for secrets. So a new centralized system may help with just dev and DIY bare metal?

But the centralized method, as in secretspec, not everyone will accept reading secrets in environment variables, as is also done with the 1password cli run command [1]. They also may need to be injected as files or less secure command line parameters. In the Kubernetes world one solution the is External Secrets Operator [2]. Secrets may also be pulled from an API as well from the cloud host. I won't comment on how that works in k8s.

To note, the reason for reading from file handles is so that the app can watch for changes and reload, e.g., key/token rotations without restarting the server.

But what could be useful to some developers is a secretspec inject subcommand (the universal version of the op inject command). I use op inject / dotenvy with Rust apps -- pretty easy to manage and share credentials. Previously I had something similiar written in Rust that also handled things like base64 / percent-encoding transforms.

If you aren't tied to Rust, probably could just fork external-secrets and get all the provider code for free.

[1] https://developer.1password.com/docs/cli/reference/commands/...

[2] https://external-secrets.io

昨年議論

That's correct, the example you are giving represents bike-shedding among request path variations.

I assumed most readers of my comment would get that the idea that /users/signup is ambiguous whether or not that is supposed to be another resource, while /users:signup is less so.

昨年議論

The system won't be able remember why the user was created unless the content of the post includes data saying it was a signup. That's important for any type of reporting like telemetry and billing.

So then one gets to bike-shed if "signup" it is in the request path, query parameters, or the body. Or that since the user resource doesn't exist yet perhaps one can't call a method on it, so it really should be /users:signup (on the users collection, like /users:add).

Provided one isn't opposed to adopting what was bike-shedded elsewhere, there is a fairly well specified way of doing something RESTful, here is a link to its custom methods page: https://google.aip.dev/136. Its approach would be to add information about signup in a request to the post to /users: https://google.aip.dev/133. More or less it describes a way to be RESTful with HTTP/1.1+JSON or gRPC.

昨年議論

I wouldn't recommend Cargo as something to copy for a real project, even though I've a fan of and have been using Rust exclusively lately. It suffers from not being able to handle global features without manually/conditionally propagating features to dependencies, as well as not being able to propagate metadata to dependencies without abusing the links functionality.

Why is that important? Well that's useful if you want something like json/serde or not in all transitive dependencies for a particular artifact you are generating like a library or a binary. That applies for other configurability that C/C++ developers bake into their libraries too.

Is this an educational learning experience as part of Hackclub which is a linked organization on your GitHub profile? Whether or not if so, trying to build this will be a good learning experience.

Think beyond just C/C++ and maybe Rust...

The entire set of ideas of things to implement is just to look at the feature set of Bazel and Buck 2 (which happens to also be written in Rust). Those offer functionality to build complete products in any language, locally or distributed across a build farm of servers, and glue them all together in any format. For example you can't build a multi-arch OCI/Docker container image for a Rust-binary server in a single command with Cargo.

Except for the initial learning curve, using them could be as simple as including their "build" files in your published git repo. No central repository needed.

https://github.com/hackclub https://bazel.build/about/why https://buck2.build/docs/about/why/

昨年議論

The VR version of network management already existed [1]. It was called CA [2] Unicenter TNG. It really could use an update with some rendering with Unreal Engine! :D

Unrelated but could be confused with what was seen in Jurassic Park as "Unix".

[1] https://archive.org/details/vw_ca-unicenter-tng-demo

[2] https://en.wikipedia.org/wiki/CA_Technologies

2 年前議論

Task is in a similar problem space.

Unlike Just which clearly states it is not a build system [1], Task can be told about expected files so tasks can be skipped to avoid unnecessary work [2]. So if your task is to build software, IMO make and the others like Task would be better.

If your tasks only care about the success code from a process, and/or are a Rust fan instead of Go, then Just should be fine. Otherwise, for specific use-cases like CI, you are likely already coding in a proprietary YAML/JSON/XML format.

[1] https://github.com/casey/just/blob/e1b85d9d0bc160c1ac8ca3bca...

[2] https://taskfile.dev/usage/#prevent-unnecessary-work