Getting the UX to work well enough is a major challenge. I’m redesigning that currently, as I got negative feedback from early testers on my initial experimental UX. There’s a balance to be struck between giving users a low latency response, giving the models time to work together and call tools, and not overloading the user with too much information.
I wonder if combinators could be useful for neurosymbolic AI—either in the backward pass (e.g., training models on synthetic data) or the forward pass (e.g., iterative code generation with evolutionary algorithms). Combinators feel alien, making even Haskell or APL seem intuitive, but maybe that’s because they don’t align with human working memory. Language models, with their massive context windows, handle long-range dependencies in sequences well, even if their understanding is shallower in some ways. Could combinators, with their compositional and deductive nature, be a better fit for machines than humans? For example, instead of generating Python functions in an evolutionary approach[0], could we use combinators as the building blocks? They’re compact, formal, and inherently step-by-step, which might make them ideal for tasks requiring structured reasoning and generalization. What do you think?
To throw shade without directly showing where Wolfram goes wrong is pathetic. This blog reads like a post about basketball in the 90s that doesn't mention Michael Jordan, titled "Bull in a China Shop".
I feel like I'm in a parallel universe where Stephen Wolfram doesn't exist. But he does, he's a living legend, and his seminal work, A New Kind of Science, and its core concepts, computational equivalence and computational irreducibility, are the answers you're looking for here.
Thanks, but that comparison is for old models, a different, non-shipped version of Devin called “Devin-base”, and doesn’t include Claude.
Slack integration, automatically pushing to CI, etc., are relatively low-value compared to the questions of “does it write better code than alternatives?”, “can I depend on it to solve hard problems?”, “will I still need a Cursor and/or ChatGPT Pro subscription to debug Devin’s mistakes?”
Looking for comprehensive benchmarks with Devin vs Cursor + Claude 3.6 vs ChatGPT o1 Pro.
In my own experience using Cursor with Claude 3.5 Sonnet (new) and o1-preview, Claude is sufficient for most things, but there are times when Claude gets stumped. Invariably that means I asked it to do too much. But sometimes, maybe 10-20% of the time, o1-preview is able to do what Claude couldn’t.
I haven’t signed up for o1 Pro because going from Cursor to copy/pasting from ChatGPT is a big DevX downgrade. But from what I’ve heard o1 Pro can solve harder coding problems that would stump Claude or o1-preview.
My solution is just to split the problem into smaller chunks that make it tractable for Claude. I assume this is what Devin’s doing. Or is Devin using custom models or an early version of the o1 (full or pro) API?
The reason it makes sense to compare them is there are problems that Claude 3.5 (or o1) can’t solve. Can Devin solve them? If yes, it’s easily worth the $500. If no, it’s a harder sell.
This has a lot of potential. I've been using Cursor Composer heavily and it's great but buggy, and could be more agentic.
After about an hour with Windsurf, I find myself frustrated with how it deals with context. If you add a directory to your Cascade, it's reluctant to actually read all the files in the directory.
I understand that they don't want to pay for a ton of long-context queries, but please, let users control the context, and pass the costs to the user.
It's very annoying to have the LLM try to create a file that already exists, it just didn't know about it.
Also, comments on the terminal management reflect a real issue. One solution is to expose the Cascade terminal to the user, letting the user configure the terminal in a working state, so that it has access to the correct dependencies and the PATH is properly sourced.
Much of the criticism comes with a tinge of envy. "How did those guys get funding and not me?"
Envy is insidious. Focusing on how other people have unfair advantages eats away at your principles, motivation, creativity, and presence of mind to capture the opportunities available to you.
As a founder myself, I appreciate the fact that it's so easy (for some people) to raise money. It means that the competition is softer than it seems.
OpenAI’s strategy has been bizarre since at least last November, when they launched custom GPTs, then had the boardroom coup.
Since the launch of Claude 3 Opus, and then Claude 3.5 Sonnet, they have been significantly behind Anthropic in terms of the general intelligence of their models. And instead of deploying something on par or better, they are making demos of video generation (Sora) or audio-to-audio models, not releasing anything.
GPT-4o is quite bad at coding, often getting stuck in a loop, and “fixing” buggy code by rewriting it without any changes.
GPT-4o is speculated to be a distillation of a larger model, and now GPT-4o-mini is an even dumber smaller model. But what’s the point?
Who is actually using small/fast/cheap/dumb models in production apps? Most real apps require higher reliability than even the biggest/slowest/priciest/smartest models can provide today. For the use case of transformers that has taken off, aiding students and knowledge workers in one-off tasks like writing code and prose, most users want smarter, more reliable outputs, even at the expense of speed and cost.
GPT-4o-mini seems like a move to increase margins, not make customers happier. That, like demoing products without launching them, is what big old slow corporations do, not how world-leading startups operate.
Intergenerational conflict is particularly salient for wealthy Americans. The death of the boomers enables the millennials’ inheritance. Millennials blame boomers for not gracefully ceding power in their own lifetimes. But boomers see millennials as waiting for their parents to die, as complacent and heartless.
This conflict does not exist to nearly the same degree in families without wealth to inherit, and in cultures that respect the wisdom of one’s elders; rejection of one’s elders/youth worship is one value that millennial Americans inherited from Baby Boomers.
As to the emails, Clegg was right when he said this:
> For Millennials, authenticity, agency, autonomy, idealism, altruism etc all seem to be top of their list of desirable virtues – for better or worse, they are difficult to reconcile with Silicon Valley these days.
Well, it’s worse, not better.
Anyway, this kind of self-conscious persona-construction is opposite to what millennials vibe with. No matter how much Zuck styles himself like a rapper or athlete, he only appeals to boomer or at best Gen X sensibilities. This is great for META stock price, but doesn’t make Zuck appealing to millennials.
Replying to signal boost this. I too get hair loss from DHT-boosters such as creatine or the herb tribulus terrestris. I should note that I have a higher than average amount of body and facial hair, so likely have naturally elevated DHT levels.
When I stopped taking creatine, I did lose a few reps in all my lifts; it does work.
One structural problem with macroeconomics as a discipline is that if you really understand reality better than prevailing macroeconomic theory, you have many better opportunities than being a theorist or commentator. And the value of these opportunities depend on you not disclosing your insights.
Eg trading, investing, entrepreneurship, working as an in-house economist bound by NDAs.
I see a lot of talk about retrieval over long context. Some even think this replaces RAG.
I don't care if the model can tell me which page in the book or which code file has a particular concept. RAG already does this. I want the model to notice how a concept is distributed throughout a text, and be able to connect, compare, contrast, synthesize, and understand all the ways that a book touches on a theme, or to rewrite multiple code files in one pass, without introducing bugs.
How does Gemini 1.5's reasoning compare to GPT-4? GPT-4 already has superhuman memory; its bottleneck is its relatively weak reasoning.
Quora is, I think, part of the reason that OpenAI has a big lead over their rivals. Quora has, specifically for training language models, very high-signal data, when compared to Reddit, twitter, gmail, and Meta’s platforms. And OpenAI is afaik, the only AI lab with a license to Quora’s data.
Quora has long been a world leader of SEO and dark UX patterns — it was never as valuable to users as its Google ranking would indicate — so it’s hard to mourn its demise. But yes, it does stink to see the quality of information going down while the quantity grows at an ever-increasing exponential rate.
The GPT-4 API costs significantly more than the $20/mo ChatGPT Plus subscription, but it does have benefits. Namely, 128k tokens of context with the latest GPT-4-turbo model, so you can put a whole (small) software project or a short book into your prompt.
Getting the UX to work well enough is a major challenge. I’m redesigning that currently, as I got negative feedback from early testers on my initial experimental UX. There’s a balance to be struck between giving users a low latency response, giving the models time to work together and call tools, and not overloading the user with too much information.