Images pass through as they are considered main content. Same with tables.
Pulpie will return all main content on a page as HTML/Markdown. I’m not sure I fully understand “which one this is good at?”. perhaps you can try the model on hugging face and let me know if the results look good?
We see far better performance with models. Heuristics break on richer content like codeblocks, formulae, quotes, etc. In our testing, our model was 25 F1 points better than Trafilatura.
We haven't run a targeted eval against SEO spam yet. However, with Pulpie, each block gets labeled by what the text actually says rather than what the tags look like. Wrapping boilerplate in semantic tags fools rule based extractors precisely because they judge structure. Pulpie doesn't. The closest benchmark we have for this is the WebMainBench difficulty split, where pulpie-orange-small holds 0.813 on the hard subset. For comparison, trafilatura scores a 0.526.
For quantization, we haven't benchmarked INT8 or FP8. Everything in the post ran on L4 and A100. That said, I expect it to go well for a few reasons. It's a single forward pass over the page, so the workload is compute bound rather than bandwidth bound, which is why the L4 held up so well against the A100 and why cheaper cards should degrade gracefully. At 210M params the small model is roughly 420MB in FP16 and half that in INT8. So it fits on any consumer GPU with room to spare. Also, one pass classification tends to survive 8 bit quantization better than autoregressive generation since there is no error accumulation across decode steps.
We've found that maximizing chunk size gives the best retrieval performance and is easier to maintain since you don't have to customize chunking strategy per document type.
The upper limit for chunk size is set by your embedding model. After a certain size, encoding becomes too lossy and performance degrades.
There is a downside: blindly splitting into large chunks may cut a sentence or word off mid-way. We handle this by splitting at delimiters and adding overlap to cover abbreviations and other edge cases.
As the other comment said, its a practice in good enough chunks quality. We focus on big chunks (largest we can make without hurting embedding quality) as fast as possible. In our experience, retrieval accuracy is mostly driven by embedding quality, so perfect splits don't move the needle much.
But as the number of files to ingest grows, chunking speed does become a bottleneck. We want faster everything (chunking, embedding, retrieval) but chunking was the first piece we tackled. Memchunk is the fastest we could build.
Which language are you thinking of? Ideally, how would you identify split points in this language?
I suppose we've only tested this with languages that do have delimiters - Hindi, English, Spanish, and French
There are two ways to control the splitting point. First is through delimiters, and the second is by setting chunk size. If you're parsing a language where chunks can't be described by either of those params, then I suppose memchunk wouldn't work. I'd be curious to see what does work though!
> Chunking is generally a one-time process where users aren't latency sensitive.
This is not necessarily true. For example, in our use case we are constantly monitoring websites, blogs, and other sources for changes. When a new page is added, we need to chunk and embed it fast so it's searchable immediately. Chunking speed matters for us.
When you're processing changes constantly, chunking is in the hot path. I think as LLMs get used more in real time workflows, every part of the stack will start facing latency pressure.
We're the maintainers of Chonkie, a chunking library for RAG pipelines.
Recently, we've been using Chonkie to build deep research agents that watch topics for new developments and automatically update their reports. This requires chunking a large amount of data constantly.
While building this, we noticed Chonkie felt slow. We started wondering: what's the theoretical limit here? How fast can text chunking actually get if we throw out all the abstractions and go straight to the metal?
This post is about that rabbit hole and how it led us to build memchunk - the fastest chunking library, capable of chunking text at 1TB/s.
We want to be the platform that connects documents to AI for all applications. Consequently, we want to cover all use cases, including the ones you mentioned :)