This integration is way more limited and frictioned. Whereas with search Apple's fully outsourced and queries go straight to your 3rd-party default, Siri escalates to GPT only for certain queries and with one-off permissions. They seem to be calculating that their cross-app context, custom silicon, and privacy branding give them a still-worthwhile shot at winning the Assistant War. I think this is reasonable, especially if open source AI continues to keep pace with the frontier.
The trendline is definitely toward increasing dynamic routing, but I suspect it's more so that MoE/MoD/MoDE enable models to embed additional facts with less superposition within their weights than enable deeper reasoning. Instead I expect deeper reasoning will come through token-wise dynamism rather than layer-wise -- e.g., this recent Quiet-STaR paper in which the model outputs throwaway rationale tokens: https://arxiv.org/abs/2403.09629
I think it's reasonable as an early-ish stage startup to say to a candidate that ~"eventually, with scale, there will be cool and impactful ML opportunities here" as long as you're realistic and upfront about the facts that ~"right now most of the impact is in simple but foundational analyses" and ~"there'll be some amount of fires to put out and rote work to automate".
Seems like permanent remote-optional is rapidly becoming the competitive equilibrium in tech.
As tooling improves, there is some point at which gains from expanded recruiting pool + real estate savings outweigh coordination/culture costs of remote. Beyond this it will be irrational to force physical presences (for many roles, at least) and there'll be no going back.
Approximately once per week I observe a comment on HN saying approximately "search doesn't work anymore." Why is this? Why do I only observe such complaints here?
As with so many things, it seems like incentives are at the root of this problem: manufacturers want to sell more foods, and adding sugar (whose addictive properties are likely to increase consumption x retention) is an easy tactic to pursue.
What should be done to counter this? Tax sugar and it will be substituted for some other unhealthy, manipulative thing. Perhaps only shifting preferences via culture / education (s.t. sweetened products sell relatively worse) can work?
Sure, not releasing the full trained model probably delays it, but sooner or later a bad actor will do their own scraping and train their own model and share it around and the genie will be out of the bottle. Then what?
I think we need to be conducting AI research (and building software generally) under the assumption that all of it will eventually be repurposed by bad actors. How would our practices be different if we consistently and cautiously did this?
Here's a thought experiment: how would the Manhattan project have been different if it were carried out in the open and its products were instantaneously and infinitely reproducible? What is the MAD equilibrium of AI research? I think the impact potential is similar even before AGI.
1) low signal-to-noise ratio - i.e., lots of duplicative stories, publishers are commoditized, and there are no mechanisms to distinguish (and reward!) high quality content
2) paywalls and/or extreme ad loads lurking everywhere
3) local news is a second-class citizen
4) lack of user control in customizing topics, while at the same time all the (uncontrollable) targeted "for you" stories tend to overfit (e.g., Google seems to think tech news and the Patriots are all I care about)
In general I'm extremely dissatisfied with the functionality of available news products and with the consequences of attention-optimized digital news media on our world. There's a lot of data which indicates most people feel some degree of similarly [1][2][3][4] and I think there's opportunity and demand for creating better.
As such, last summer I set out to build Gatherscope - a news ecosystem designed from first principles to serve readers and sustain quality journalism. At the moment, the high-level plan is:
1) build an aggregator layer that's differentiated in the breadth, depth, quality, and customizability of content that's surfaced and in applying machine clustering and summarization to make it faster and easier for busy consumers to digest what current events are happening, see varying perspectives about them, go deeper where worthwhile, and then get on with their lives
2) stack other layers around this - either by building them or through 3rd party relationships - to further deliver a rich, insightful, frictionless UX (e.g., bundled multi-publisher subscriptions, community quality review, synthesized audio, contextual search and feedback loops, publisher tools for local journalists)…these should all work independently but best together
Today there's a (quite-rough) MVP of part 1 live at https://www.gatherscope.com and a very long way to go :)
All this to say: if any of this sounds interesting and/or you have strong, thoughtful opinions about news media, I'd love to hear from you. I've recently begun a hunt for collaborators, and even if that's not for you it would be good to hear how to build you a useful product faster.
I find the ethical boundary between colorizing old footage and 'deepfakes' to be fascinating to consider. The former is celebrated here while simultaneously the latter is an object of increasing apprehension, abhorrence, and calls for regulation.
An obvious difference is intent. The intent of They Shall Not Grow Old is (I assume) to memorialize historic events and enable modern audiences to understand them more viscerally. The intent of (most) deepfakes is deception.
Still, what should we do when the same technology that enables deepfakes also enables better They Shall Not Grow Old's, perhaps concerning subjects more poorly-documented than World War I? It's acceptable to make assumptions about the colors and sounds of a moment from 100 years ago and represent them indistinguishably from truths, but how about assumptions as to who was present and what actions they took and words they spoke?
I mean...corporations consist of people (i.e., employees) with feelings and values which - at least to some extent - govern their collective choices and actions. Granted it's tough / probably impossible to disentangle this from profit motive.
For now.