Resources and lots of like minded people. That is what you'd be missing if you skipped SV and headed to San Diego. I'm an entrepreneur myself and moved to San Diego (from NY) almost 2 years ago.
I was working on a startup coming out here and the beach and weather did not help me focus (but other distractions including living with my college friends directly ON the beach).
That aside you can find like minded people, but it is much MUCH MUCH harder. I've joined a couple startup groups on meetup.com and actually that is how my current partner and I met. But I haven't been to any meetups since.
Personally, the weather is beautiful and does give me inspiration, but I wouldn't sacrifice all the resources that you would get in SV.