> Defaults are bad. Agents can be expected to read the documentation, register what good starting values are, and fill them all in, in place.
That is pretty bad, when you now need to look at this code, and you have 95% of the output being craft, but sometimes it isn't, and you need to understand the difference between each.
A great example of that is:
int fd = open("log.txt", O_RDONLY);
Versus:
// 1. Manually build and initialize the Security Attributes structure
SECURITY_ATTRIBUTES sa;
sa.nLength = sizeof(SECURITY_ATTRIBUTES);
sa.lpSecurityDescriptor = NULL; // Explicitly no custom security descriptor (inherits default)
sa.bInheritHandle = FALSE; // Explicitly state this handle cannot be inherited by child processes
// 2. We need a handle for the template file parameter.
// Win32 requires this to be an active file handle opened with GENERIC_READ, or explicitly INVALID_HANDLE_VALUE.
HANDLE hTemplateFile = INVALID_HANDLE_VALUE;
// 3. Now we call CreateFile with every single parameter fully populated
HANDLE hFile = CreateFile(
"log.txt", // 1. lpFileName: The file we want to open
GENERIC_READ, // 2. dwDesiredAccess: Read-only access
FILE_SHARE_READ, // 3. dwShareMode: Prevents any other process from writing to it
&sa, // 4. lpSecurityAttributes: Pointer to our explicitly defined struct
OPEN_EXISTING, // 5. dwCreationDisposition: Only open if it already exists
FILE_ATTRIBUTE_NORMAL, // 6. dwFlagsAndAttributes: Normal file, no special caching or async flags
hTemplateFile // 7. hTemplateFile: Passing our explicit invalid handle instead of NULL
);
An agent can output the second just as well, sure.
However... which one do you think is better to read or understand?
Did you catch the fact that this is blocking concurrent writers? Or that we expected the file to exist?
Nevertheless, you are still talking about 15B / year - when the initial investment was < 30B.
That investment also probably covers things like power, salaries, etc.
Prety much all financial transactions are settled with a given date, not instantly.
Go sell some stocks, it takes 2 days to actually settle. (May be hidden by your provider, but that how it works).
For that matter, the ultimate in BASE for financial transactions is the humble check.
That is a great example of "money out" that will only be settled at some time in the future.
There is a reason there is this notion of a "business day" and re-processing transactions that arrived out of order.
Committing to NVMe drive properly is really costly. I'm talking using O_DIRECT | OSYNC or fsync here.
Can be in the order of whole milliseconds, easily. And it is much worse if you are using cloud systems.
You must have the data upfront, you cannot build this in an incremental fashion
There is also bo mention on how this would handle updates, and from the description, even if updates are possible, this will degrade over time, requiring new indexing batch
This is wrong, because your mmap code is being stalled for page faults (including soft page faults that you have when the data is in memory, but not mapped to your process).
The io_uring code looks like it is doing all the fetch work in the background (with 6 threads), then just handing the completed buffers to the counter.
Do the same with 6 threads that would first read the first byte on each page and then hand that page section to the counter, you'll find similar performance.
And you can use both madvice / huge pages to control the mmap behavior
Another way to do that is to look at the tech empower benchmarks.
It tests on big machines, but you can get > 1 M req/sec across a wide variety of environments.
That is the wrong abstraction to think at. The problem is not _which_ tools you give the LLM, the problem is what action it can do.
For example, in the book-a-ticket scenario - I want it to be able to check a few websites to compare prices, and I want it to be able to pay for me.
I don't want it to decide to send me to a 37 hour trip with three stops because it is 3$ cheaper.
Alternatively, I want to be able to lookup my benefits status, but the LLM should physically not be able to provide me any details about the benefits status of my coworkers.
That is the _same_ tool cool, but in a different scope.
For that matter, if I'm in HR - I _should_ be able to look at the benefits status of employees that I am responsible for, of course, but that creates an audit log, etc.
In other words, it isn't the action that matters, but what is the intent.
LLM should be placed in the same box as the user it is acting on-behalf-of.
FWIW, I once got a cease and desist letter because "company-xyz" found that we were using a subdomain "company-xyz.customers.our-service.com".
They discovered that because they were monitoring the CT logs.
And they were concerned about trademark issues.
It ended up being one of the teams in "company-xyz" that had opened an account (under the company name, of course).
But that is just a small note that people _are_ monitoring those.
If you are checking the cert logs, it is a very tiny bit to validate the key as well.
If you aren't checking... well, that isn't a concern anyway, now is it?
And the whole _point_ of the cert transparency log is that it only take _one_ such instance to ruin the credibility of a CA.
The fact that you do that in the public, and that it is _forever_, make it very hard to do in the shadows.
One thing that is ignored here is that for crypto, there are additional concerns
For example, are the instructions cost depend on the data? If so, you have a timing leak and a hole in your system
One of the reasons to use assembly is specifically to avoid timing dependency and the compiler being "smart" about things and thus leading to timing attacks
That is pretty bad, when you now need to look at this code, and you have 95% of the output being craft, but sometimes it isn't, and you need to understand the difference between each.
A great example of that is:
Versus:
An agent can output the second just as well, sure. However... which one do you think is better to read or understand?
Did you catch the fact that this is blocking concurrent writers? Or that we expected the file to exist?
Or what about:
Versus:
Same thing, but actually understanding what is going on is orders of magnitude different.