It is. I tried it out but it doesn't have the fidelity w.r.t I/O and 'hooks' I was looking for. Then again, I'm working beyond reinforcement learning. More advanced A.I necessitates more advanced gym.
I highlighted box2d as its an opensource physics/sim engine.
The sim/physics engine is 90% of the 'gym'. I'm sure if there was money involved, the sims/hooks/etc that are being sourced and packaged around could get a lot more advanced more quickly. i.e :
https://github.com/erincatto/Box2D/issues
To test out increasingly 'general' and advanced A.I, you need a sandbox/playground environment...
You can use a real-world environment with a robot (noisy, more latency, computationally restrictive , more headaches, and extra development time)
Or you can utilize a 'virtual' world.
Minecraft is being used as Microsoft's sandbox/playground.
Video games are a good sandbox/playground : physics engine+sim+a.i hook.
You can generalize this into an observation->action loop platform.
There are many open source platforms out there for this.
Take Box2d as an example which OpenAI put a custom wrapper around.. It's a 2d Physics simulation engine. 2-Dimensions = less complexity than 3-D. You have your 'physical environment' and then you hook that into your A.I .. observation/command loop.
There are public Sandboxes/playgrounds and there are private ones. Many of these engines aren't hard to tie into. I'm sure this was one of the major things to resolve at OpenAI as it is for anybody in the space : make a 'playground environment for testing your A.I'...
I'm also sure that OpenAI has a more advanced private gym with higher fidelity links to the A.I like everyone in the space does. Members only ;)
So, OpenAI has packaged a bunch of open source engines/etc into an approachable platform to make development easier. Also, OpenAI hopes to get people to upload their results and have them detail how they achieved their results... Interesting.
A.I development is at the point IMO where you don't need a name or accolades to contribute. You don't need a PHD. You don't need to be an expert in the field. You don't need to be some award winning coder. In some ways, that can harm you w.r.t to having an ingrained view on how to approach problems in a space that is begging for new paradigms.
Dedicate a solid month and you can have make a virtual A.I gym setup and be off and running.
If you have done any serious code development, you can easily break into this space.
The thing that will be the most time consuming will be wrestling with these packages, dependencies, understanding them, and figuring out how to hook/in out of them.
It seems openAI has tried to reduce this pain with the release of openAI gym.
However, you'll find, if you get into any serious A.I dev, you're going to want to start cutting through people's wrappers and add-on layers that add latency, increase complexity, and keep your A.I away from the heart of the sim.
You'll want your own custom hooks....
You'll want the code to be as low level as possible.
It depends on what your aim is. If you want to jump into the field, follow what the field is doing and/or suggesting.
If you want to go down the rabbit hole, i'd suggest
Jeff Hawkin's book (On Intelligence)
The essential takeaway will be that the current focus is on cortical algorithms representing the neocortex. Even as such, the current models are only partially inspired.
From there, something might stand out to you.
Caution : This is the harder and unpaved road.
Studies in :
Neuroscience.
Neurobiology.
Computational Neuroscience.
Information theory.
For conversion work to computational land, a creative mind and a broad array of knowledge and experience in software engineering. Might take years but you'll maintain a depth of understanding and much greater capability of tackling AGI.
On AGI efforts, I personally can't imagine how one can maintain they are on a path to AGI by way of instantiating an artificial form of it yet have not even a basic understanding of how the biological form of it functions or how far off their models are from the essential parts that make it tick.
I guess to some people, it's cortex all the way down.
Basically yes. But that's not only incredibly effective, it's quite possibly how real brains work too. A lot of people do believe it is a path to AGI.
> It's only incredibly effective in a static world. The world is not static nor is the human brain. Nor is the interplay between a human brain and the world and A.I. It's a very disjoint, dynamic, and interdependent relationship with far more complexities than could ever be represented in a statistical flow map much less in the incomplete mathematics and statistics that underly them. I have no doubt that people believe they can make a statistical map of the world. It wont be the first nor will it be the last time people try to make an 'effective' one. The dynamics of the world will change and they will be invalid as they in no way are structured based on a true understanding of what's going on. Nor is there any awareness of what's going. Aren't the overly complex and flawed risk models that no one could explain what caused the crash in 2007/2008? You think the dynamics in the world are less or more? So, I say to people subscribed to this provenly flawed thinking : Good luck.
That's not an accurate description at all, there are no "agents". In fact your whole description of NNs sounds off.
And backprop wasn't invented or named in silicon valley. In fact it's been around since the 80's.
> Agents/node.. Tomato/Tomato .. they are partial computation nodes receiving and instantiating fed back partial derivatives based on computing an error between expected/actual. Where's the intelligence? Hindsight is 20/20 ... You're brute forcing the partial elements that contribute to a desired answer by slamming a cheese grater (NN) in forward and reverse flow ... hoping the important stuff sticks somewhere. Don't try to make it seem any more complex than that. Curve fitting at its finest. Constraint optimization. Gradient Descent. Regression. statistics all packaged up with fancy buzzwords.
The Backpropagation algorithm is used to find a local minimum of an error function. Flashback to grad school where there were a laundry list of methods in constraint optimization courses. There's nothing special about it. What is special is the thinking behind it.
You rightfully stated, most of these methods were developed in the 70s' 80s'. While people are off copying and instantiating the works of that time and relabeling it with buzzwords, there is little attention being paid to the thinking that yielded those methods. That's what matters .. the actual intelligence and thinking.. not what pops out the other end.
With little focus/money being put into expanding upon the thinking of that time period, those focused on it are not going to get any further than they .. Even worse, you maintain no understanding as to why they did what they did. Which is why no one can tell you how NN works. The intelligent people who defined them are dead.
Every "AGI" project is a bunch of pseudoscience. They have no idea how to build an intelligence. They have no idea how the brain works. They have no results to show with their algorithms, they aren't beating benchmarks. The theories are always vague and ad hoc and include a million special cases to make their systems do anything.
> People in the weak A.I space may have the lay person fooled by slapping buzzwords and A.I on everything. However, anyone who has spent anytime doing grad work in this area before it took on fancy names knows better... It's distributed gradient descent. The objective function is broken down into partial forms and instantiated at the distributed computational points in the gradient descent flow path defined by a NN. You slam it in forward and reverse and eventually enough stuff gets jammed into the lines for future flows.
I recall something named Genetic algorithms/evolutionary programming that were supposed to be the keys to the future...
So, Strong A.I ... AGI. I'm thinking those who have the best shot at it are people who know how NNs work on down to the mathematics and statistics, theory, philosophy and pseudoscience. Given this understanding, they have the ability to formulate new math/statistics/computational models and frankly whatever else it takes to represent a true form of intelligence.
I imagine they are working hard at doing that very thing in the shadows while others busy themselves trying to beat cooked benchmarks and fight over coin and the spotlight.
So, you go down your path and they will seemingly go down their path... But don't for a second think they don't understand exactly where your path is likely to lead you.. Many of them have gone down it and found nothing of value at the end. I guess the new crop of individuals who have no understanding of the thinking behind these algorithms they're copying/instantiating have to take this journey for themselves.
Call AGI a bunch of pseudoscience and foolishness. I have no doubt a good number of people will be praising and following it like religious zealots much like the work of those 'crazies' from the 70s'/80s that everyone laughed at but now can't wait to copy/relabel and call their own.
> No. It's a piece of a much larger puzzle and only a partial piece at that. An overfit piece that people are over-applying. This is why things are overly complex and filled with statistics...
I mean we are trying to emulate biological neural networks which is fundamental to intelligence.
> There is far more functional complexity to the underlying biology. This is why studying neurobiology/neuroscience have value as opposed to resorting to ever more complex statistical models that no one understands.
Do you think that the artificial neural networks we are using right now are not rigorous enough?
> Of course not. Out of all the amazing people centered on it, no one can say why/how it works. Is it magic? lol... That should tell you something and trigger a red flag. I can state why it works and already have. It's just not something that's convenient and would necessarily cause one to admit that its not the broad general answer were looking for....
So, for some time, those centered on this paradigm are probably going to build out wildly elaborate NNs. They will require boat loads of data and computational power and achieve great outputs. Coincidentally this fits nicely in the cloud computing model that the big tech titans maintain.
Somewhere down the line, more solid and thought through computational models inspired by actual understanding will come out that will shake the very foundation of said approaches and so will go another page in tech history.
NNs are biological inspired. With all of the fanfare surrounding them, You maybe never stopped to question how inspired.
Drop Paul King/Paul Bush a line a Quora or dig through some of their posts.
It's better to talk w/ a Neuroscientist/Computational Neuroscientist about this stuff IMO.
Lots of theoretical models, philosophical, and metaphysical thinking... The scientific method... Actual in-depth studies centered on the system you're trying to understand and duplicate in another form. You know.. Everything but the foolishness employers and VC firms look for in people.
How all the greats did it :
> Einstein
> Von Neumann
> Alan Turing
> etc
If you have a proven coding background, you're off and running in the production lab. The problem with the layers of hype, academic jargon, overly complex white-papers, and hand waving is that it makes people believe this is unapproachable outside the narrow scope of thinking that everyone is currently subscribed to. Scopes change with time and the people who tend to progress and widen these scopes are often those who think outside the narrow box everyone else is set upon. This is what Jobs meant by 'think different'.
Or you can attempt to brute force it with statistical models, PHDs, computing power, and truckloads of data hoping something miraculously emerges.
So, what type of thinking is used to develop strong A.I?
Strong thinking... Something that most and the industry aren't set upon which is why it most always takes an outside to usher in such new paradigms.
As is the history of the Googles of the world ...
Which is why I say you are more likely to hear about strong A.I once someone has developed it in the dark. It isn't going to be a 'thing' until it is a thing for people don't know how to recognize, value, or back undefined things until someone goes out of their way to make it into a reality.
"Very few people today would have the audacity to explicitly try building human-level AI."
Hmmm, there are a considerable number of people/groups who have this audacity... Have for decades... and they have been explicitly trying with much incremental success.
One thing such people speak about is ridding the space of outlandish buzz words/promotions that mask the true nature of how things function. This 'hype' creates barriers to contribution, learning, and progress.
Furthermore, the difficult efforts have been overshadowed by statistically mapped input/output flow models currently being called "A.I".
There is no mystery w.r.t how deep learning/etc work I.M.O.
You have inputs 'X'.
You take a known solution space 'Y' (supervised learning) or you create an arbitrary one (unsupervised learning) 'Z'.
You break apart the input space and map it to nodes in a graph.
You break apart the output space and map it to nodes in a graph.
Input flows are decomposed into minimal component parts and recomposed into higher orders of correlation. This is then compared (via flow restricted weighting) to increasing orders of the output space.
How does this magical 'piece apart and and piece back together' process work during active flows? It works based on guided encoding of 'importance' weights on the partial information represented by individual nodes in the flow graph network. Thus why under/over fitting can occur if you have too many/too little nodes.
How are the weights codified? By encoding the partial derivative (partial contribution) a node has w.r.t to the accuracy of the solution ... Error function (Desired - Actual). Curve fitting.
It's essentially distributed brute force statistical gradient descent which is why you have to beat on it, tune it, anneal it, and cram hoards of data through for it for it to yield anything of value. "throw enough dirt and it will stick"
Frankly, there is nothing to understand ... NNs/etc are distributed optimizers guided by partial objective information. The resultant network/weights are a spaghetti jumble of 'whatever gets the right output out the other end'.
You're throwing a bunch of 'agents' at a solution space and having them gradually combine their results to a final solution. This was previously known as constraint optimization before it got the silicon valley treatment of buzzwords :
This is not A.I. I don't feel anyone who has a grain of integrity ever thought it was.
It's very slimmed down version of cortical Algorithms with lots of missing pieces at best.
Strong A.I is being developed far from such thinking and is a totally different animal. People who work in this space are necessarily guarded and un open as there is a lack of appreciation, value, and funding for their 'audacious' efforts.
Of course, once more solid systems are developed, I'm sure you'll hear from them again in the form of 'blackbox' capability presentations.
Currently, the spotlight and money are being thrown at PHDs and names as no one has a clear understanding of what they're looking for. Namely because no one wants to spend the time/money on defining that. People are moreso interested in getting products/results out the door.
".... lets get the best minds, throw them in a room, throw money at them and hopefully a solution will come about"
Seems very similar to distributed brute forcing of a problem space with a made up objective function...."throw enough dirt and it will stick"
There are a lack of generalist being brought into these A.I labs and efforts as they are perceived to have little value. Yet, its the 'generality' and 'fuzzy' stuff that underlies our very intelligence. From general to specific or specific to general...
So, the industry wants to brute force this w/ money/PHDs/Buzzwords/industry names...
The more complex and disjoint a problem space is, the harder it becomes to brute force....
Time will tell. All roads eventually lead to Rome.
Though, some will take considerably longer.
So, a probability driven dynamic call-stack FSM.
I have seen this concept used in many ways before without the fanfare (i.e : code coverage/test suites and game A.I Engines)
It is indeed a cool approach and no doubt deeply inspired.
Rip apart the traditional program execution model and add statistical branching ...
I highlighted box2d as its an opensource physics/sim engine.
The sim/physics engine is 90% of the 'gym'. I'm sure if there was money involved, the sims/hooks/etc that are being sourced and packaged around could get a lot more advanced more quickly. i.e : https://github.com/erincatto/Box2D/issues