Though I don't enjoy the current sad state of affairs with regards to the security and validation of CAs, there's something to be said for the old adage that no security is better than false security, and trusting all self-signed certificates would definitely be false security, since eavesdroppers could just do a man-in-the-middle with their own self-signed certificate.
I'd personally be really happy to see something like http://perspectives-project.org/ instead of the current web of mistrust.