This is incorrect. In the process of producing each token, activations are produced at each layer which are made available to future token production processes via the attention mechanism. The overall depth of computations that use this latent information without passing through output tokens is limited to the depth of the network, but there has been ample evidence that models can do limited "planning" and related capabilities purely in this latent space.
I think this ignores the fact that an agent can be meaningfully embodied in the internet, using APIs for sensors and actuators. OpenAI's training of large language models with reinforcement learning, recent retrieval augmented models and "chain of thought" reasoning are all meaningful steps in this direction in my opinion.
Microsoft (Semantic Machines) | Senior Software Engineer | Berkeley, CA or Boston, MA or Redmond, WA or fully REMOTE | Full Time
The Semantic Machines group is bringing next-generation natural language processing (NLP) technologies to products used by hundreds of millions of people worldwide. You can learn more about how Microsoft is using this technology to create entirely new kinds of user experiences here: http://semanticmachines.com/.
At the core of our platform is a new programming language, designed to support programs structured like human commands which are predicted by machine learning models. This language includes ideas from functional and logic/constraint programming, as well as novel features for meta-computation and introspection. We’re looking for engineers to work alongside product and research teams to help guide the evolution of this platform, including improvements to the core programming language, runtime, constraint system, and tooling.
The ideal candidate should be passionate about designing and evolving programming languages and/or practical formal reasoning systems, supporting users with high quality tools, and working on a rapidly evolving product-driven platform. No experience with machine learning or natural language processing is required – we’d love to work with people who are excited about the promise of these technologies and the opportunity to make them more accessible, regardless of their previous exposure to them.
At Semantic Machines [0] we rely heavily on probabilistic programming to build state-of-the-art dialogue systems. In particular, we use a library called PNP (probabilistic neural programming) on top of Dynet to allow us to express structured prediction problems in a simple and elegant form. If there are questions I am happy to elaborate to the extent I can. (Also, we are hiring! My email is jwolfe@.)
Yep, that sounds about right. For us, API communication and some global events flow through channels into the app state, where it's distributed to components. For routing we've built up a library around secretary, where we write down a map of uris, handlers, and metadata (can be a modal, etc). Info about the current page and such goes into the app state and that drives the transitions between page components.
We're still figuring out exactly where the boundary lies, but our current approach is much like yours -- view state that's purely local to a component goes in local state, and everything else goes in app state. This feels more natural, but means you lose some of the nice benefits of things like replay/undo for this local state.
For client/server, we've started building up some nice abstractions around our API (like fetching more data in a generic paginated list stored in a cursor), but the write side is still mostly manual. We're trying to use the same API for iOS and web, which probably prevents us from doing some fancier things that continue the same architecture back into the server.
No, you are still free to organize your application state however you wish. For example, our app dynamically pulls in content as you browse from feed to feed and use infinite scroll. It's up to you to decide what to fetch when, and what to cache/discard as the user navigates around.
I'd have to think about that a bit -- I'm hesitant to create names not provided by the user, but maybe there is another way.
Quite honestly that part of things (turning validation on and off, etc) isn't really done yet -- we have plans to make it much more flexible and powerful, but haven't got there yet.
Schemas are already optional -- you can have a s/defn with no, some, or full schematization. You can also put the schemas in the meta if you don't like the :- syntax. I'm not quite sure what you mean by the other point though. My email is in my profile, let's talk :)
I've been really pumped about core.typed since I first heard about it, and I'm excited about the possibilities of combining (or eventually replacing) schema with core.typed.
I think core.typed has come a long way since last time I looked at it (when we started developing schema). Obviously it's a much bigger and more serious effort, and schema was about getting as much bang right away as we could for a few bucks.
I'd love to talk in more detail about how we can collaborate, or at least make sure we play nice together.