I thought about doing this years ago, but without AI, the only way was to have people do the analysis. I thought this was bad karma, as I would end up paying people in Bangladesh to look at the stuff that Americans eat.
Embedded systems often have crappy compilers. And you sometimes have to pay crazy money to be abused, as well.
Years ago, we were building an embedded vehicle tracker for commercial vehicles. The hardware used an ARM7 CPU, GPS, and GPRS modem, running uClinux.
We ran into a tricky bug in the initial application startup process. The program that read from the GPS and sent location updates to the network was failing. When it did, the console stopped working, so we could not see what was happening. Writing to a log file gave the same results.
For regular programmers, if your machine won't boot up, you are having a bad day. For embedded developers, that's just a typical Tuesday, and your only debugging option may be staring at the code and thinking hard.
This board had no Ethernet and only two serial ports, one for the console and one hard-wired for the GPS. The ROM was almost full (it had a whopping 2 MB of flash, 1 MB for the Linux kernel, 750 KB for apps, and 250 KB for storage). The lack of MMU meant no shared libraries, so every binary was statically linked and huge. We couldn't install much else to help us.
A colleague came up with the idea of running gdb (the text mode debugger) over the cellular network. It took multiple tries due to packet loss and high latency, but suddenly, we got a stack backtrace. It turned out `printf()` was failing when it tried to print the latitude and longitude from the GPS, a floating point number.
A few hours of debugging and scouring five-year-old mailing list posts turned up a patch to GCC (never applied), which fixed a bug on the ARM7 that affected uclibc.
This made me think of how the folks who make the space probes debug their problems. If you can't be an astronaut, at least you can be a programmer, right? :-)
Apple making cars was an interesting play when they had a bunch of cash overseas and good relationships with Chinese manufacturers. Tax law changes made bringing money back to the US less punishing, and manufacturing in China is becoming problematic.
You can do something similar with docker compose, driving the system from the outside. Create dockerized versions of dependencies like the database, build and run tests, and then run tests against the production app container.
It's particularly useful for testing a set of microservices.
A few years ago I did an embedded Linux project on an ARM7 CPU without an MMU, a vehicle tracker for trucks with GPS, cellular modem, ODB2 interface, and other I/O.
Lack of MMU caused some unexpected issues:
* Because there is no MMU, there are no shared libraries. All the binaries have to be statically linked. We had a total of 2MB of ROM, of which about 1M was the Linux kernel. So we ran out of space for the programs we needed for the application about 3/4 of the way through the project. The solution was to use Lua, which was a 200k binary, and it could run scripts to do some things.
* Lack of MMU means no memory protection between programs. One program can step on another program's memory. Just like the good old days of DOS. Ouch!
* We started with the ucLinux distribution, which is pretty lame. A lot of programs in the tree didn't build. We ended up rebuilding with Yocto, and making patches to things where necessary.
It's a bad market, probably the worst since 2000.
Some of this is overhiring and zero interest rate behavior coming to an end.
Companies are shifting from prioritizing growth to profitability.
There is also a lot of asshole behavior. They are laying people off to please Wall Street, not because they need to for financial reasons. And they are doing layoffs in extremely callous ways, e.g., right before Christmas.
My daughter's data engineering team in finance was reduced from five to two, and they are not giving raises that match inflation. She is almost singlehandedly responsible for servicing million-dollar accounts. It's not like there isn't money.
There is also a lot of hiring of remote people outside the US. The companies figured out how to do remote work during Covid, and they are using it to reduce costs.
Things will probably improve in the next year, but it will be rough for a while.
Sure, you can make deadlocks in any language, but it's uncommon in Erlang.
Shared state is the exception, and message passing means that things that manage state, such as gen_servers, only process one message at a time from their inbox.
Contrast this with languages like Java where every object is a potential concurrency problem. Or the 10+ years of trying to make Python async (see Twisted).
In some companies, staff positions can only be effectively done by someone who has been there for 10 years. They know the specific tech stack that the company uses, know the application, know where the bodies are buried, and everyone in the company trusts them. It's basically impossible for someone to parachute in and do the job.
Erlang is based on virtual threads (confusingly called processes).
The Erlang virtual machine schedules them on OS threads.
Erlang processes communicate using message passing, preventing deadlocks.
You can use millions of Erlang processes without problems, e.g., to handle millions of Elixir LiveView sessions.
The fundamental problem is that domain name trademark enforcement is handled by the legal department when it should be handled by marketing, business development, and/or DevRel.
While companies can enforce their trademarks, it's generally better to allow people who are using domains in a reasonable way to continue. Taking the domain away and then doing nothing with it doesn't benefit anyone.
Even in parasitical scenarios, allowing squatters to get affiliate revenue by redirecting traffic to the company is better than fooling around with the legal process.
I think Scrum is a transitional phase. It is a solution to the problem of having a dev team shared between multiple business functions.
The dev team keeps getting yanked around and interrupted. So every two weeks you get all the business people in a room and you have them make a list of things that the dev team will work on this cycle. Then the dev team can execute in peace. The Scrum Master's job is to protect the dev team. The Product Owner's job is to get the business people to converge on the prioritized list of work.
In a healthy organization, Scrum is much less useful. Scrum combines three different cycles: defining work, executing work, and reviewing work. In a flow-based process like Kanban, you can split them up.
The product manager builds a prioritized backlog of work, and the dev team estimates tasks to help do business ROI calculations. It's a collaboration between product and development.
The dev team implements tasks from the backlog one by one. You can batch up work to make a release, deliver incrementally, or use tools like feature flags. You evaluate whether features are effective working based on metrics.
When I was a kid I ran a lawn service, and I got all my engines out of the trash.
When the mowers are cheap, it's rational. They get the lawnmower out of the shed the first time in spring and it won't start, it would cost a significant amount to get it fixed, and they have to waste your time and wrestle a dirty lawnmower in and out of your car. So they buy a cheap new one and throw the old one away.
Paying someone is still more expensive than doing it yourself.
I wish that everyone in the US had affordable, convenient, and private access to a doctor who could help them choose contraceptives and ensure no side effects.
Unfortunately, that's not the case, so getting them over the counter is an improvement.
Linux from Scratch is educational but fundamentally impractical to use.
I recommend using embedded Linux instead, e.g. with Buildroot. You get an understanding of the fundamentals of the hardware, kernel, and build toolchain. And you can create small hardware systems that do fun things.
I ran Gentoo for a few years, and enjoyed it. Doing the install from stage1 gives you a similar understanding of how the pieces fit together and enables you to fix low level system problems, e.g. with disk failures. The community is also smart, as it self-selects for people who have skills and good attitude. Unfortunately Gentoo is too slow to install and quirky to deploy on servers.