All it takes is a moment to hook up a profiler, and you will know FOR SURE where your performance problems are. Don't sit there thrashing away at shadows when you could be done and on to more important things. Diagnose, THEN fix; always in that order.
I couldn't have put it better myself.