MintMCP | Founding Engineer / Founding GTM | SF Bay Area (ONSITE)
MintMCP helps enterprises safely roll out agents to the entire organization (e.g, governance around MCP tools and agents like Claude Code / Codex / Cowork). We're building an MCP gateway with abstractions useful for 1) preventing agents from doing the unintentionally bad things you hear about every week without constraining them to be useless, 2) getting the rest of your teams and organizations to adopt them with minimal friction.
We're a small team of humans with dozens of agents: our company operates on the infrastructure we build for enterprises. We're looking for agent-forward engineers and GTM folks to help solve scaling problems, maintain the highest standards for security, make agents and MCPs usable and useful.
If you have experience in infra, security, and agents, and believe MCP, while not perfect, will evolve to become critical AI infrastructure, you might find the problems we're working on exciting!
MintMCP | Founding Engineer / Founding GTM | SF Bay Area (ONSITE)
MintMCP helps enterprises safely do more with Agents (e.g, governance around MCP tools and agents like Claude Code / Cowork, and others). We're building an MCP gateway as the foundation with the proper abstractions that enterprises need to allow long running agents to be safely deployed across an entire organization; organizations can easily and clearly limit the blast radius of what an agent can do, while giving it the right combination of tools to be useful.
We're a small team of humans with dozens of agents: our company operates on the infrastructure we build for enterprises. We're looking for agent-forward engineers and GTM folks to help solve scaling problems, maintain the highest standards for security, make agents and MCPs usable and useful.
Problems we're solving right now: Agent Identity, ReBaC for tool resources, scalable hosting of private MCP servers, and our agent platform that vertically integrates with the foundational MCP gateway layer.
If you have experience in infra, security, and agents, and MCPs, you might find the problems we're working on exciting!
Agreed, OAuth is certainly preferred for many reasons, but replace "API keys" with "OAuth access tokens" and you have the same fundamental challenge of ensuring an LLM or untrusted code never has access to the user's secrets.
If the output schema specifies an id field, the LLM can write a code snippet that references it based on the context of the subsequent request, but the LLM doesn't need to observe the underlying value unless necessary. E.g., it can pass the 'id' opaquely to another call that receives the "id" as an input. If the user specifically wants to see the "id", the code orchestration approach can have the LLM just print the content.
bfloat16 was first in DistBelief, so it actually predates TensorFlow and TPUs (I worked on both systems). IIRC the motivation was more about minimizing parameter exchange bandwidth for large-scale CPU clusters rather than minimizing memory bandwidth within accelerators, but the idea generalized.
Thanks for mentioning that Sam, I appreciate it. Speaking for just myself:
I personally don't care what tools or frameworks people use to get work done and have repeatedly suggested people use whatever works best for them.
It also wouldn't make any sense to hoard any good stuff internally if we wanted to provide a useful framework that people wanted to use externally.
We actually don't have a huge amount of tooling internally that we hold back. The only tooling I really use is the [timeline](https://github.com/tensorflow/tensorflow/blob/f488419cd6d925...) for debugging performance (not the EEG tool in the whitepaper, I've never used it). That's available externally though, as you can see.
On some of the comments in this thread in general, I'm pretty sad at the lack of scientific rigor in the community, and that goes for any person who publishes code that differs from the results they claim, regardless of affiliation. I am happy about projects like OpenAI's RL baselines, as I know others are too.
Most of the papers and articles comparing performance of frameworks have lots of bugs and aren't even comparing the same model computation between frameworks. In fact, CNTK's article points to an external benchmark showing CNTK in a good light, but those benchmarks have bugs in them rendering the comparison incorrect (we've been sending PRs to fix them). I find it disappointing that the culture of the organization promotes calling others out for being 'irresponsible' except when it suits them.
The TensorFlow team hasn't published many articles comparing performance directly to others because it's honestly a lot of hard work to verify that you are comparing fairly. Of course, I do think TensorFlow needs to improve performance out of the box for people, and the team is working on that.
I'm sorry to hear of that experience, and it's certainly not intentional. We should (and I will) always try to be better (I know some will forever view Google as an antagonistic behemoth but usually the engineers on both sides are trying to do the right thing; it just takes time to come to consensus and understanding sometimes).
Regarding internal version: we built TensorFlow to support devices as modular plugins: the CPU and GPU devices are built this way (you can read the source code to see how device registration works), and the same registration mechanism is used for the TPU code, which can't be opensourced due to internal dependencies. Internal customers just link in an additional library to get TPU support, but it still uses the same core codebase that is available in the opensource world. I know this because I wrote a lot of the device modularity and TPU binding code :)
The codebase in GitHub is pretty much exactly the same as the internal, the main exceptions being things like having to rewrite include paths for files, filesystem plugins for internal cluster filesystems, etc; and those things are modularized so that we can have equivalent implementations in the OSS build to support things like HDFS and GCS filesystems, RDMA network layer communication, etc.
We daily sync the code between the two repositories using a suite of tools we've built. I'm on sync rotation this week and you can see all of my commits and activity on GitHub as proof; I merged something like 60 commits from the community just this week. It wouldn't make any sense for anybody (Google or the community) to maintain two different versions for something that is actively being developed by so many contributors. I've also directly worked with NVidia engineers on improvements they've made (and merged) to the system; the ones I've dealt with are great, so that statement is also false.
I'll be giving a talk about all the work we do to make this possible at OSCON next week, and if you are there feel free to catch me to ask me questions.
wrt GPU memory: If we made InteractiveSession by default grow memory, would that work for you? Seems sensible to change the default for InteractiveSession (e.g., for use in notebooks), but not the default for Session (production training / inference) due to the potential memory fragmentation.
TensorFlow doesn't yet do loop fusion (though I believe the specific example shown in that article may already be done via constant folding). But if you have a bunch of elementwise operations, JIT-techniques can reduce the number of memory passes over a buffer. If your model is already very computationally dense (time dominated by matmul or convolution), then this won't help as much, but otherwise, JIT techniques can help.
Windows support is definitely being worked on and lots of progress has been made, so it will eventually arrive -- just lots of little details to work out, but we're optimistic it'll come soon.
For what it's worth, we're hoping to integrate these APIs into our iOS version of the TensorFlow runtime, so you can maintain graph portability but still get the benefits of the optimized implementation on the platform.
The arxiv paper is just a repost of the preliminary paper, which is really more a design outline and rationale -- we may work on a more substantial paper later, we've been busy building the software.
(Do you have an example link that doesn't work? I clicked a bunch of links there and they were all working. Feel free to file a bug at github.com/tensorflow/tensorflow)
The field is moving quickly enough that many published benchmarks are stale within 3 months, and it's a lot of hard work to maintain up to date benchmarks, given how many frameworks there are. Also there are performance/memory/scalability/flexibility tradeoffs everywhere, so it's hard to capture everything in one number without a tremendous number of caveats.
"GitHub issues are for bugs / installation problems / feature requests.
For general support from the community, see [StackOverflow](link)"
However, we still get plenty of issues filed that are more appropriate for StackOverflow, so not sure there's anything technical you can do without integrating more closely with StackOverflow :)
MintMCP helps enterprises safely roll out agents to the entire organization (e.g, governance around MCP tools and agents like Claude Code / Codex / Cowork). We're building an MCP gateway with abstractions useful for 1) preventing agents from doing the unintentionally bad things you hear about every week without constraining them to be useless, 2) getting the rest of your teams and organizations to adopt them with minimal friction.
We're a small team of humans with dozens of agents: our company operates on the infrastructure we build for enterprises. We're looking for agent-forward engineers and GTM folks to help solve scaling problems, maintain the highest standards for security, make agents and MCPs usable and useful.
If you have experience in infra, security, and agents, and believe MCP, while not perfect, will evolve to become critical AI infrastructure, you might find the problems we're working on exciting!
You can apply via MCP here: https://www.mintmcp.com/hiring