I get a better first pass at code by asking it to write code at the level of a "staff level" or "principal" engineer.
For any task, whether code or a legal document, immediately asking "What can be done to make it better?" and/or "Are there any problems with this?" typically leads to improvement.
One helpful addition would be Requests Per Minute (RPM), which varies wildly and is critical for streaming use cases -- especially with Bedrock where the quota is account wide.
After years of raising 3 kids, you would think if I ask to add diapers to the cart, it would know something. But no, it would just go with whatever is the top recommended, or first in a search, or something like that. Nothing using the brand or most recent sizes we purchased.
There was no serious attempt to drive real commerce. Instead, Alexa became full of recommendation slots that PMs would battle over. "I set that timer for you. Do you want to try the Yoga skill?"
On the other hand, they have taken on messy problems and solved them well, but not using technology, and for no real financial gain. For example, if you ask for the score of the Tigers game, Alexa has to reconcile which "Tigers" sports team you mean among both your own geography and the worldwide teams, at all levels from worldwide to local, across all sports, might have had games of interest. People worked behind the scenes to manage this manually, tracking teams of interest and filling intent slots daily.
Asking even a top-notch LLM to output well formed JSON simply fails sometimes. And when you’re running LLMs at high volume in the background, you can’t use the best available until the last mile.
You work around it with post-processing and retries. But it’s still a bit brittle given how much stuff happens downstream without supervision.
Processing high volumes of unstructured data (text)… we’re using a STAG architecture.
- Generate targeted LLM micro summaries of every record (ticket, call, etc.) continually
- Use layers of regex, semantic embeddings, and scoring enrichments to identify report rows (pivots on aggregates) worth attention, running on a schedule
- Proactively explain each report row by identifying what’s unusual about it and LLM summarizing a subset of the microsummaries.
- Push the result to webhook
Lack of JSON schema restriction is a significant barrier to entry on hooking LLMs up to a multi step process.
Another is preventing LLMs from adding intro or conclusion text.
The context is a for-profit company. If a third-party using the API is doing so as a substitute for someone using the ad-supported site, the API will be priced to equal the revenue lost from using the site. If the people switching from UI to API are more valuable than usual, the API will cost more than usual.
If the API is being used to support something new -- i.e., not a replacement for a user on the site, but as a new revenue source -- it will be priced to maximize overall API revenue, or overall revenue as an indirect result of growth supported by the API. (Maybe it's one massive customer using it or 100,000 freemium developers.)
The idea that a company "must" do something doesn't make any sense. And the assertion that the company is harming itself is ultimately something the Board, acting on behalf of shareholders, is responsible for.
(FWIW, I agree Reddit is shooting itself in the foot.)
The letter is a demand for action that assumes a conflict exists and the client is correct. It is standard. The recipient can acknowledge and agree or acknowledge and disagree.
Same! I had a directory of info coded horribly by some CRM into HTML DIVs (not a table) and copy/pasting it into a text file resulted in a single column of names, contact info, kids, grades, etc.
I asked ChatGPT to reformat it into a CSV, noted the useful breaks, requested some transformations and filtering, and specified a delimiter. After 5 minutes of experimentation, it worked like a charm. Absolutely amazing.
Executives has larger, permanent desks (or offices) that don’t get bothered, and never get kicked out of a meeting room — but their assistants will kick you out of yours. The problem is invisible to them.
You can connect code steps to other actions. For example, every Python code step has a reserved `output` variable which allows you to pass a list out to the next steps. It gets passed as stringified JSON: if it contains dicts, those values are made available automatically as inputs to other steps. You can also pass more complex, nested JSON and just json.loads it later.