They should tax houses that sit vacant really really heavily. I think that is the #1 beef that many East Coast Americans have with Europeans, their buying up housing and not living in it. There is a similar problem on the West Coast with so called 'ghost houses'. many of them are not kept up and they just sit there, a constant reminder of growing inequality.
I've heard Europeans accused of many things by Americans, but believe me, this isn't one of them! I have absolutely no idea where you got this idea, but it's got no foundation in reality. I can't rule out that it might be the case in some very specific locations, but it's absolutely not the case on the east coast as a whole. Heck, Europeans don't even buy homes in their home countries nearly as much as Americans do, never mind vacation or investment homes in USA. (Most of USA is not that attractive to Europeans, to be blunt.) I'm not saying that no Europeans own US real estate, but this just is not a reason why Americans hate on Europeans!!!
I don't think you can generalize about the west coast, either, but in San Francisco specifically, you have a weird situation where there is a severe housing shortage AND high vacancy, because their tenancy laws make it essentially impossible to evict a tenant, so when people buy a building they want to convert back into a single-family house, or merge a few apartments into a bigger one for their own use, they cannot have anyone living in it. So as tenants move of their own volition, they don't re-rent, but just keep it vacant until the building (or the adjacent units) is free and they can begin construction.
Meanwhile, look at places like London and New York, which really do have tons of high-end real estate being bought as investment properties by ultrawealthy investors from China, Russia, and the Middle East. (Switzerland largely put an end to this, to stop such investment from making mountain resort villages desolate outside of the ski season, as well as pricing locals out of the real estate market. Some of those villages passed laws outlawing vacation home sales, i.e. mandating that it must be a primary residence.)