It's probably one of my favourite shows, just loved it.
Yeah a friend of mine has some family that stay around Baltimore who have said that while the impoverished areas are probably among some of the worst in the country, those areas are quite isolated away from the areas of the city that anyone else would chose to explore - I guess that's all part of a cycle of an area becoming further impoverished. I have heard that Baltimore itself is a very nice city, may need to visit one day.
I think the show probably served as way to show a more overall picture of the struggles in inner city American cities, as opposed to it being solely a representative of 1 city that had been struggling.
Seasons 3/4 of The Wire are probably the best of the series, if you haven't seen that far in yet.