Every currency has onshore and offshore rates. For example, you can buy dollars or Eurodollars, which is simple US currency kept offshore.
Eurodollars have different exchange rates because they don't have the reserve requirements from the Fed. i.e: it's less regulated.
The RMB has a heavier regulation whereby the exchange rate isn't allowed to fluctuate. This does increase the difference in value for the onshore and offshore currencies.
But having that distinction does not by itself make it a currency manipulator.
Yes this was very surprising. The system was working fine after the cluster restart. There was no need for an emergency rollback.
Doing a large rollback based on a hunch seems like an overreaction.
It's totally normal for engineers to commit these errors. That's fine. The detail that's missing in this PM is what kind of operational culture, procedures and automation is in place to reduce operator errors.
Did the engineer making this decision have access to other team members to review their plan of action? I believe that a group (2-3) of experienced engineers sharing information in real-time and coordinating the response could have reacted better.
Of course, I wasn't there so I could be completely off.
I ran the production systems for various Google products and this was an interesting challenge there because there were many engineers and many changes done every day.
One imperfect but helpful technique is to use release notes for your application.
Ask the team to write good commits when changing code, and aggregate all code changes done to the application into a page that describes what went in that release. GitHub releases do that for you automatically. When a release is pushed that breaks something, it's easier to scan the release notes and find who changed what.
Another technique is to use explicit feature flags to protect all code changes. Then use a dashboard that shows recent changes to feature flags – again this helps tracking down the culprit.
I worked at Google for almost 10 years. In my experience, these end-of-life projects often get shutdown because nobody wants to work on them.
There are a ton of interesting projects at Google for people to work on, so people select for interesting things with potential high-impact. If a team can't be formed to own the product, maybe someone will volunteer to take care of it in their 20% time. But there are these company-wide mandates to move production systems from a storage system to a newer one, and a part-timer doesn't see the point of doing such thankless job.
Management can also play a role. If a VP wanted to support the product, it would get supported. But they use the basic heuristic as above ("is this an interesting project with high potential impact"). In this analysis a VP is just a proxy for 100 engineers making the same decision.
IMHE, generations are a nightmare to operate for high performance servers at scale because you have to balance the sizes of those heaps manually and it can change abruptly with code changes or workload fluctuation.
Go allocations are indeed costlier but the performance critical sections of applications can be profiled and optimized accordingly to remove allocations.
I'd rather have Go's amazing low GC latency and slightly higher allocation costs vs the operational nightmare from HotSpot.
Automatic management of generations has never fully worked in Java. Every new JDK version just adds more knobs. Sounds like you have a different experience?
Is there a way for companies to tell if they are on the Advisory Track, besides the emails?
That's totally OK, mistakes happen, but I'm not sure what the emails say. If startupschool.org perhaps had the final status (advisory track or not), that would be super helpful.