I think that ego/status is a big part of it. And the loss aversion bias. Our thought process is "This thing is going to be able to do software as well as a human, therefor it is going to take my job." When a more optimistic and probably more realistic take is "This thing is going to do the boring parts of software, so I can spend my time building out the more interesting, big-picture parts of code. I'm going to be so much more productive." But the fear of loss is so much more distracting than the potential gain. ... And what is it that we fear losing? Status! Software development is in high demand, it's a sought after role, we feel "special".
I wrote a book (https://www.manning.com/books/relevant-search). Considerations: are you willing to write the book assuming that you will effectively make nothing in direct proceeds from the book? (Close to true.) The upside from a book is in monetizing on it indirectly. For instance, as a "though leader" you'll be able to garner a higher salary at your next job or you'll be able to ask for higher rates for consulting. But the catch here is that you've got to be willing to put yourself out there, otherwise it's a nice bullet point on your resume.
Oh yeah - and don't underestimate the time and effort involved. (I know you mentioned it in your post, but still.) A due date, even if fairly far off, looms ominous. For a year you'll have trouble relaxing and having fun because you know you should be getting work done on the book. You won't have weekend back for quite some time.
Would I do it again? Yep.
Will I do it again? Nope. (At least not likely :D )
Elasticsearch might seem like a strange option at first since it's historically a text search engine, but it's main datastructure is a compressed bit array which is ideal for OLAP processing.