I think it's highly unlikely that my choice to buy will end up not looking like a good decision. I'm about 2 years from having it paid in full and already rent in the area is far higher than my mortgage payment. If my house suddenly dropped 90% in value it stands to reason that other houses in the region will likely drop similarly so I still have an asset worth a relatively constant percentage of the price of whatever house I move to. Since I always need a place to live I don't view the value of my house as wealth, it is insurance on always having a place to live.
If something really blew up and I couldn't afford to live in my house, I could always rent it out, either divide it up and rent out part of it to someone else, or live with a friend, relative or even out of my car for the duration of the difficulty while retaining ownership of the house.