What was the motivation? Honestly, I was too lazy to get a job and staying in academia for another 3+ years seemed amazing (probably not recommended, but it worked out OK for me).
What helped get me through it:
1) Doing something I genuinely enjoyed - I approached the Computer Vision professor who gave me some ideas. I super enjoy writing code, and the idea of processing gigabytes of video to produce answers seemed cool. I treated it as a super difficult programming project.
2) Breaking my leg - Just before starting, I broke my leg badly. And that meant working from home with a weekly visit from the professor with a stack of reading papers. That time spent understanding state of the art was super useful.
3) Funding - At some point, DARPA gave enough money for me not to worry about funding, so I never had to work a job or get distracted.
4) Marriage - The final straight of writing a thesis was tough and I was super lucky to have a supportive wife who pushed me to get-shit-done.
I remember attending a tech event at MSR Cambridge, and a speaker made some disparaging comment about older developers not being able to keep up in this modern world of programming.
An older gentleman stood up and politely mentioned they knew a thing or two.
Interestingly they used to attach a sponge to the end. You might think that was because it doesn’t break the glass, but really it was to ensure the nearby houses don’t get woken up for free!
It takes you through implementing some classic AI examples (such as Eliza) and teaches you how to write code in a data-first style through Lisp.
I worked through the whole thing building it in Clojure and learnt loads (not just about AI, but about writing readable code in general). I can't recommend it highly enough!
There's some research in the "Accelerate" book that talks about this. For example, having an automated CI/CD pipeline decreases stress measures (iirc).
For me it's the mindset that differs. Too often as software engineers we find a bug and just fix it. Aviation goes a step deeper and finds the environment that created the bug and stops that.
Unfortunately, the recent 737 MAX incidents seem to have changed this. From what I understand the reaction to the problems sounds more like what I'd expect a software business to do, rather than the airline industry!
I haven't read the Stephen Pinker book, but it sounds similar to one of my favourite books "Factfulness: Ten Reasons We're Wrong About the World--and Why Things Are Better Than You Think" which sounds like it covers similar topics.
As douglaswlance says, the world is getting better faster than ever!