1) There are plenty of standards in the pallet industry, especially in the EU. (http://www.palettes-cp.com/welcome-en.html) The US has the 48x40 GMA which for many years was used almost exclusive in the grocery chain. These standards were born of similar or exactly the same needs from different customers. Things like 2x2 barrels, poly supersacks, square and octagonal boxes, etc. All those things are standardized and so the pallet that goes underneath them is largely the same.
2) Customization is a factor of efficiency. Why would someone shipping a product want to buy a standard size pallet that doesn't fit or hold the weight requirement for his use case? Instead, he consults with either internal packaging specialists (in big companies) or a manufacturer directly to design a pallet to fit his load. By doing this he actually knows the limitations of the packaging (stacked x high, racked, weight distribution, unstacker and rollerbed compatibility, etc) and also has some general idea that the product he is receiving is correctly priced for his requirements. Many times those requirements are customer driven, meaning my customer's customer.
There's a massive PR engine at work for both the wooden and plastic side, but the things brought up most are fire and chemical composition. You would think that nothing could worse for a fire than wooden pallets (It's a constant concern for us, not so much pallets but just wood products in general) but there's a serious hazard with the manufacture of some plastic pallets. (Think oil fire.) I'm sure some of this has been overcome or can be, but it was an issue just a few years ago for sure.
In order to combat the fire front manufacturers starting using chemical composition to make their pallets flame retardant. One of the those chemicals is called deca bromine and is considered to be extremely hazardous. This creates a problem specifically in scenarios where food contact is common.
This is a dated article, but it references some of the concerns.
While CHEP definitely has changed the way some retail supply chains work, overall the pallet industry largely works the same as it did before CHEP was on the map. We build upwards of 2mil new manufacture wooden pallets per year and I never run into CHEP as an alternative for the sectors I service.
Source: I run one of the largest pallet manufacturers in the US, 50+ year old family business.
Disclosure: I run one of the largest wooden pallet manufacturing companies in the United States. My family has been in this business for 50+ years.
Your perspective on whitewood comes from a 3rd tier perspective (grocery, as you say). In other words your various suppliers buy their pallets from various sources and then ship their product into you on those pallets. You and your (former?) employer are not directly involved in their procurement so you have little say in what you get, and I'm sure the quality varies widely.
That being said whitewood pallets (or just wooden as we would say) are actually a very cost effective and reliable way to ship product in the US, much more so than plastic. There are many quality manufacturers around the entire nation that provide great product and service, much better than your experience seems to suggest. Plastic is expensive to buy, more expensive to lose, and has various other problems and concerns.
To compare a built CHEP pallet with a "whitewood" pallet is a bit misleading however in several directions, especially to come to the conclusion that they are the "good stuff". They are the good stuff in terms of build quality because (by my estimation) they are built roughly 3x as strong as a generic 48x40 (standard GMA pallet.) They also cost upwards of 3x as much to manufacture and the process is no different than the bad quality pallets that you have had experience with. (Estimated GMA is about $10-11 per, CHEP block pallet would be above $30, depending on locale.) Of course they hold up better, they are built to do so. However, the pallet industry has gone from standardization on sizes in the last 20 years (GMA 48x40, Coke pallet 36x36 mentioned below) to being specifically engineered to the load it was made to carry.
TL;DR: There are wooden pallet manufacturers out there that produce a high quality product. CHEP and plastic are both expensive but have their own issues.