I make a much higher income than almost all my friends and most acquaintances. (I'm a programmer, employed full-time by a company, got about $350k in 2010, living and working in NYC.)
I never really took the taboo against talking about income seriously either, until I realized that answering these kinds of questions honestly was seriously poisoning my interactions with people. Some people would just get jealous and resentful, others would make silent, unwarranted assumptions about me, still others just became more prone to "hey, can you cover me and I'll pay you back?" me with no intention of doing so and for higher and higher amounts.
So, I now discuss my income only with people who I know have a high one as well. Everyone else gets a very frank "I'm sorry, but that's none of your business." What's interesting is that I get more pleasant reactions now than when I answered honestly.
So yeah, based on my experience, it is in your interest to obey the taboo and keep your mouth shut.