Probably the rampant unemployment, underemployment, low paying jobs if you can actually get one, and the huge homeless population which is more a reflection on how abysmal the city and local economy is than those unfortunate enough to be homeless themselves. Don't forget the nonstop rain and suicide hotline numbers posted on every bridge or parking garage higher than 20 feet, since nearly everyone is cranked out on antidepressants or else they'd be leaping into the Willamette river with cement shoes on too.
When I lived there and had visitors from out of the area, everyone commented on how depressing the place was. Not a good place to live or raise a family. If you like the Pacific Northwest, go with Seattle, yes the weather is still crap but the city is vibrant, has real jobs with real wages, and you can actually build a career there.
Portland? WTF? Why? It's a hipster dump. Rains 9 months a year, has 12% perpetual unemployment, plus the USA's largest resident homeless population, which isn't terribly shocking given the awful job market. But, if you can actually get a job the salary will be circa 1994 and yet the property and rent prices are in fantasy land stuck in the 2006 housing bubble. Oh, plus the city has one of the highest suicide rates in the nation. Great place to live if you like being wet, underemployed, and depressed.
Glad I got out of that dump, if you want to have a future you need to be in Boston, NYC, or The Valley.