chopratejas

2 miesiące temudiscuss

Hi -

I am the core maintainer of Headroom. Headroom remains OSS and free to use :)

If youre a dev that wants to try it out, use the Github pip install directly.

We are building it for multiple languages, harnesses and models.

Extraheadroom was started by one of the initial users of Headroom to attract a crowd of non devs who would not like to get into pip installs etc. they have attributed Headroom OSS on their webpage - and being OSS, Headroom is in principle available to folks to build on top of it.

chopratejas

6 miesięcy temudiscuss

Fair enough. Trying to keep it concise here - This is how you install it:

pip install "headroom-ai[proxy]"

headroom proxy --port 8787

It will:

* Check all the data going into the LLM and apply intelligent compression based on the content type - different for JSONs, code etc.

* If the LLM is not getting what it is seeking, there is reversible compression - so the LLM will not lose accuracy

* When you think of MCP tools, code function calls etc. that fill up the context window and cause needle in haystack problems - they get eliminated.

There is also an SDK which works like this:

from langchain_openai import ChatOpenAI from headroom.integrations import HeadroomChatModel

# Wrap your model - that's it!

llm = HeadroomChatModel(ChatOpenAI(model="gpt-4o"))

# Use exactly like before response = llm.invoke("Hello!")

Ive personally used it with Claude Code and Cursor and seen the benefits.

chopratejas

6 miesięcy temudiscuss

Some results from real world data so far:

  ┌─────────────────┬─────────────┬──────────────────────────────┐
  │    Data Type    │ Compression │             Why              │
  ├─────────────────┼─────────────┼──────────────────────────────┤
  │ Server logs     │ 90%+        │ Highly repetitive patterns   │
  ├─────────────────┼─────────────┼──────────────────────────────┤
  │ MCP tool output │ 70%+        │ JSON structure overhead      │
  ├─────────────────┼─────────────┼──────────────────────────────┤
  │ Database rows   │ 50-70%      │ Same schema, many records    │
  ├─────────────────┼─────────────┼──────────────────────────────┤
  │ File trees      │ 40-50%      │ Repeated metadata            │
  ├─────────────────┼─────────────┼──────────────────────────────┤
  │ Code diffs      │ 0%          │ Every line unique            │
  ├─────────────────┼─────────────┼──────────────────────────────┤
  │ Dense prose     │ -0.3%       │ No patterns, slight overhead │
  ├─────────────────┼─────────────┼──────────────────────────────┤
  │ Encrypted       │ 0%          │ Incompressible               │
  └─────────────────┴─────────────┴──────────────────────────────┘

chopratejas

6 miesięcy temudiscuss

What is it?

- Context Compression (with Reversibility - this part is the difference) for LLMs

- very different than any compression or summarization tools that promise cost savings and speed!

- claude code costs / cursor costs - reduced by 50-60%

- ideal for startups and Enterprises!!

- integration with LangChain

- Memory as a first class citizen

- its OSS! So Free!

Give it a try, Its OSS - if you love it, star it. If you don't, lets make it better, together!

chopratejas

6 miesięcy temudiscuss

Author here. I built Headroom because I was spending $200/day running agents with tool calls.

The problem: tools return huge JSON (search results, DB queries, file listings). Each response bloats context. By turn 10, you're paying for 100k+ tokens on every LLM call.

Existing solutions have a fundamental tradeoff: - Truncation: fast but you might cut data the model needs - Summarization: slow (~500ms) and still lossy - Bigger context: just delays the problem, costs more

The insight behind Headroom:

You can't know which data matters until the model tries to use it. So instead of guessing, compress aggressively AND keep a retrieval path.

  1. Smart compression - not random truncation. For JSON arrays, we keep errors (100%), statistical anomalies, items matching the user's query (BM25 + embeddings), first/last items. For code, we use tree-sitter AST parsing to preserve imports, signatures, types - output is guaranteed syntactically valid. For logs, we keep errors and state transitions.

  2. CCR (Compress-Cache-Retrieve) - everything compressed gets cached locally. We inject a `headroom_retrieve` tool. If the model needs more data, it asks and gets it in <1ms.

  The retrieval is what makes aggressive compression safe. In practice, the model almost never retrieves because the smart compression keeps what matters. But when it does need more, it can get it.

Results on my workloads: - Search results (1000 items): 45k → 4.5k tokens (90%) - Agent with tools (10 calls): 100k → 15k tokens (85%) - Overhead: 1-5ms per request

Usage:

  As a proxy (zero code changes):
  pip install "headroom-ai[proxy]"
  headroom proxy --port 8787
  ANTHROPIC_BASE_URL=http://localhost:8787 claude

Or wrap your client: from headroom import HeadroomClient client = HeadroomClient(OpenAI())

LangChain integration is one line.

Limitations I'm aware of: - CCR adds memory overhead (LRU cache, configurable) - AST compression requires tree-sitter (~50MB) - Not battle-tested on all edge cases yet

Happy to answer questions about the compression algorithms, the retrieval mechanism, or anything else.

chopratejas

7 miesięcy temudiscuss

More detail / what it does + what it doesn’t:

What it actually changes:

Tool output compression is deterministic and schema-preserving: it returns a subset of the original array items (no invented summaries, no wrapper keys).

It supports both OpenAI-style role="tool" messages and Anthropic-style tool_result blocks.

“Fail open”: if JSON parsing/compression fails, it passes through unchanged.

Why another context tool? Most “context compression” projects focus on prose. The thing that killed my agent runs was valid tool calling + tool payload bloat. The goal here was: reduce tokens without breaking the contract.

Typical savings On tool-heavy runs, the big wins come from crushing large arrays (search results, traces, lists). In my traces I’m seeing ~70–90% reduction on tool payload tokens depending on how repetitive the payload is. (If you have a better benchmark harness, I’m happy to adopt it.)

Escape hatch when compression drops something you need When a tool output is compressed, Headroom stores the original briefly and can expose a retrieve tool (headroom_retrieve) so the model (or you) can pull the full uncompressed payload by hash. (There’s also an MCP server for this.)

Shortcomings / where it can be the wrong idea

SmartCrusher is intentionally conservative: it focuses on JSON arrays. If your tool returns a giant nested object or long free-text, Headroom won’t magically solve that (text compression utilities exist but are opt-in).

If your downstream logic requires “the full list of 1,000 items,” then any reduction strategy can be wrong—use the retrieve tool or disable for that tool.

Relevance scoring is heuristic/optional; it can miss “the one weird item you cared about” if it doesn’t look anomalous/relevant.

Running as a proxy means your prompts/tool outputs flow through a local service (privacy/security tradeoff; logging is off by default for full content).

Happy to answer any comparisons—tell me what you’re using (prompt compression, truncation, provider caching tricks, etc.) and I’ll map it to how Headroom behaves.

chopratejas

7 miesięcy temudiscuss

I kept hitting context limits building coding agents because tool outputs explode (search/log/API payloads), and prompt compression breaks strict tool calling.

Headroom is a local proxy / SDK wrapper that:

- compresses tool outputs (schema-preserving subset of JSON arrays; keeps errors/anomalies/top/relevant)

- trims history as whole tool-call units (no broken function calls)

- stabilizes prefixes so provider caching stops getting invalidated by drift

Try it locally: pip install "headroom-ai[proxy]" && headroom proxy --port 8787 then point your client’s base URL at it.

Repo: https://github.com/chopratejas/headroom

Limitations: best on JSON arrays; text compression is opt-in; if you truly need every row, you’ll need the retrieval escape hatch or per-tool disable.

Do give it a 'star' if you like it.

chopratejas

Submissions

Show HN: Headroom (OSS): Cuts LLM costs by 85%

Show HN: Headroom – Reversible context compression for LLMs(~60% cost reduction)

Headroom – context optimization layer for tool-using agents

comments