>Why is being "priced out" of same place you already live at perceived as something worse than not being able to afford to move into some neighborhood to begin with?

Because there are costs involved with relocating, which the former aren't being given any choice about. Finding a new place to live and moving there involves a lot of time, effort and stress. Not to mention any friends and family they're now less able to keep in touch with.