Thanks for the recommendation, ordered. Hartley is very valuable to know how we got to where we are, from hand tuned features and RANSAC to more modern full photometric optimization and ML. It answers a lot of "why don't we just... " type questions while still providing the foundations.
This is a complex question. As you pointed out, having a high fidelity map helps to identify the part of the environment that change, creating 'deltas'.
Another reason is that autonomous vehicles take data from a wide variety of sensors, including cameras, RADAR, LIDAR and others.
None of these modalities is perfectly, and there is always ambiguity and drift. High fidelity, high resolution maps provide a strong prior that helps to resolve these issues.
Besides the sensing issues, high fidelity maps also provide priors with respect to planning and prediction of the behaviors of other road users. A busy intersection should be approaches differently to a small backroad.
In the end, autonomy needs both real-time processing and high fidelity maps to perform effectively.
Somewhat different to the suggestions so far, but Thrun et. al's 'Probabilistic Robotics' is a very good applied probability text, with a focus on physical systems.
Plenty of worked examples and problem sets as well.
These are all questions to be addressed, along with 'how many of those miles should we upgrade', 'what will we get out of it', and 'how do we make a self-driving car'.
There are many many startups working on some fraction of the autonomy stack (e.g. Aurora) or the entire vertical (e.g. Zoox). I call out several specific ones in my comment if you care to read it through.
I work in autonomous vehicle R&D, and if we had an environment filled with reliable sensors it would considerably simplify our problem.
There is no such thing as a _complete, detailed 3D map of driveable areas_. Most current efforts rely heavily on high resolution, large scale, semantically labeled maps, but on any given stretch of road this is only a first approximation of the environment. Live sensors embedded in infrastructure that could pass along real-time information and updates, particularly from directions our on-board sensors can't capture, would be very useful indeed.
And yes, road markings are part of the semantics we use.
Without revealing too much, German automakers have expertise primarily in logistics, with some expertise in automotive design.
There is a vast pyramid of Tier-1 and Tier-2 suppliers that feed into that logistics chain. A lot of those suppliers are promising automakers that they can deliver either components that feed into autonomy, or else autonomy in full. This fragments the effort across literally hundreds of small teams, many of whom have little to no expertise at all in the necessary disciplines.
Additionally, every German automaker also has its own in-house autonomy team, and the relationships between in-house, contracting, and supplier teams is chaotic. Many managers see autonomy as another way to build a small empire and make a name for themselves, and it just results in a huge organizational snarl.
The fact that this is an area no one knows how to execute on yet since it's brand new, coupled with these gross inefficiencies, will mean autonomy efforts from the automakers will be stillborn. They will eventually learn from it though, acquire the right teams, and get on with putting it into their cars just as they're doing with EV.
Not sure why you're getting downvoted, this is a legitimate take.
My thinking is this is two-pronged. It won't make sense to make the infrastructure investment until there are sufficient proof points that this is 1.) something people want and 2.) something that's economically viable.
It's likely that the early geofenced version of passenger autonomy will demonstrate what further infrastructure is needed and how much it would cost.
One interesting side point; dedicated lanes for autonomous traffic are already being proposed on some roadways, particularly interstates in the U.S and highways in Europe. The economic benefits from autonomous logistics (e.g. trucking) are more readily capturable, so the infrastructure investment might make more sense there up front.
Similar background and profession - putting aside whether an individual would ever want to own one, general purpose passenger autonomy won't happen for probably another decade. Limited domain autonomy will be here in before 5 years, though. It'll just suck.
But the first step to getting good at something is sucking at it.
We are at the peak of the autonomous driving hype cycle, and the trough of disillusionment is going to wipe out a few players, and see the massive consolidation of others. These start-ups are very capital intensive and once the funding environment shifts, it's going to be a big problem for a lot of them.
The initial launches of autonomous taxi services will be underwhelming - relative slow, sometimes frightening (there will be accidents for sure), and with limited deployment areas in limited weather. The novelty will wear off quickly, and then the real work is going to start.
There are basically going to be two ways to survive it - be part of a larger organization that can shoulder a long-term R&D burden to go from 'toy' to 'real infrastructure'. I can see Cruise and Waymo making it that way, unless GM gets cold feet.
The other survivors will be either super lean like Voyage, who went straight to revenue generating niches like retirement communities and rely on being downstream of the technical innovation being done elsewhere (no in-house vehicle or sensor development to cut down R&D costs and move quickly).
Or else they'll be in niches like logistics (e.g. Peloton, Nuro) where the parameters of the game are different, and the structure is more B2B than B2C.
One other play is the autonomy-tech licensing structure like Aurora is trying, but that's a hard sell, especially since they're dealing with German automakers who (from first hand experience in this domain in particular) are clueless about autonomy.
This is an incredibly exciting, risky time to be a part of this new, emerging industry. It feels in many ways like the very very early P.C era, where everything is very much still in play. I'm glad I made the career shift to get there.
One interesting thing about the 'zero' approach is that it learns through self-play. So you can use a database to bootstrap the learning, but in principle you'd only need the game rules and enough training time.
Almost everything you interact with was designed and built by people like you. Understanding why those things work the way they do, and are made the way they are, is powerful.
You don't need permission from someone else to start working on something that interests you, nor do you need to know everything about how to start.
~10 years experience in software development, everything from huge government telecoms projects to my own startup. I now work in robotics/computer vision research. Here's my advice:
Programming is not hard. Don't focus on software development as a career path.
Software is only a tool, and most production software is badly written because the customer doesn't care about design patterns, only whether the product works or not. Despite the prevailing opinion on HN, learning to program is not difficult and a reasonable person with technical training in any field can pick it up in a few months from scratch. The current high salaries paid for writing CRUD apps (and almost everything is CRUD apps no matter how much JavaScript you put in front of it) won't last as the barrier to entry is ever lower. Very little that could be called 'engineering' gets done in a software context.
Once you've learned two programming languages with very different concepts (e.g. C++/Scala), you'll be able to pick up any language/programming concept easily. Don't waste time learning the flavor-of-the-month JavaScript framework, it's a waste of mental energy and if you ever need to know it you'll be able to learn everything anyone actually uses it for in a week or two of concentrated study. For the time you spent learning React in and out, you could have taught yourself rocket propulsion,
Focus instead on developing the patience needed to learn complex ideas in depth, alongside a broad range of soft skills.
If you go to university, pick a scientific or engineering discipline - ECE will keep you close to computing, but ME is a good choice too. Physics or Mathematics are both also good choices. Don't major in CS unless you really care about the mathematical foundations of computing (which have nothing to do with most programming you're ever likely to encounter). In my experience it's very easy to teach a physicist to program, but the converse is not true.
Mentors are everywhere, but they're not necessarily going to mentor you 1:1. Professors at university are mentors. Coworkers are mentors. Anyone smarter than you that you have the opportunity to spend time with is a mentor. There's no need to codify the relationship. To get the most out of these relationships, spend time figuring out the most important, concrete things you'd like to learn and pose them as questions.
To develop leadership skills, you need to first master self-examination. This is very difficult and takes a lot of mental fortitude, and life experience you may not have had the chance to develop yet. This is a journey I'm still on so I don't have much useful advice, other than that.