Always Room For More

One common argument against infill: "but there isn't room for any more people!"  (or, alternatively, "we can't have more people without turning into a skyscraper monoculture!"

Manhattan is far from a skyscraper monoculture- even in midtown there are lots of 2-6 story buildings of all types.  And yet our housing density is 70,000 people per square mile- more than four times that of San Francisco, more than seven times that of Washington.

In other words, at Manhattan densities San Francisco could accommodate more than 3 million people. 

And Manhattan isn't even the densest place in the affluent world- Mongkok, a neighborhood in Hong Kong, has over 300,000 people per square mile.  So San Francisco could accommodate about 14 million people at Mongkok densities.  And even the tiniest look at Google Street View should persuade you that even Mongkok isn't all skyscrapers (though it does have 5-10 story buildings where Manhattan might have 2-4 stories)