a >3000 s.f. house (which are usually built very ticky tack and inefficient these days) -once energy becomes very expensive- that sets its own limits on housing. When gas at the pump is $10+ per gallon, that sets its own limits on commuting- or even finding essential goods and services.
Have to feed lots of children on a budget that's been eaten up by dramatic inflation for the price of (among other things) food? That sets its own limits on family planning.
Some places in the country (think Oregon) had the foresight to adopt mostly responsible land use planning that (mostly) kept sprawl to a minimum- and only within "urban growth boundaries." Development (mostly) proceeds with mass transit and centralized communities in mind. Valuable farmland is (mostly) preserved. Not so in the SouthEast- or other places.
True, there are plenty of McMansions packed together in Oregon- and plenty of shortsighted developers- and suckers who buy into their scams, based on the assumption that current conditions are more or less permanent. They'll not be happy when they discover their assumptions were wrong.
Unfortunately, you have to draw lines somewhere- and even using the best urban planning and rationally, well thought out codes- you'll end up with what look like arbitrary limits in some cases.
Personally, I'd rather see those limits set responsibly- and if that limits people "freedom" for the the greater public benefit (including theirs) too bad. That's what governments do. That's why complex societies create laws in the first place. That goes back (in written form) to ancient Mesopotamia and Code of Hammurabi.
See, e.g.
http://www.wsu.edu/~dee/MESO/CODE.HTM