I'm not going to believe that every phone charger and tiny device is PFC. OK they are small loads, but they take all of that power at peak so they appear like 10-20 times greater which is the whole point of PFC, if you add up all of the small devices that may have an inductor at best you may be surprise at how much unclean power is being drawn.
Last time i looked PFC required another stage of SMPS, a recent industrial unit I looked at showed a seperate block and was only 90% efficient but looking at a DC/DC that is 96% efficient because it does not need two stages.
The typical small power supply PFC/multivolt section additional parts are exactly one each of a suitable power mosfet, inductor and diode, plus of course the power supply controller chip must be a PFC model--but those likely cost 0.10 or less in bulk. Obviously a small amount of power is dissipated, but I'm not sure an efficiency comparison between PFC and non-PFC is apples-to-apples because the non-PFC just shifts the inefficiency elsewhere. I've seen exactly one 'efficient' non-PFC ATX supply, by Antec, that I used in a build about 8 years ago--every other non-PFC unit has been total crap for efficiency compared to any newer models. And even that Antec isn't as good as the better PFC models.
Now if you are talking wall warts, although I suspect many except the very cheapest are actually PFC, I've no proof. I suppose I should tear one down. But almost everything recent I've stuck in my Killawatt device has shown a pretty good power factor.
As for what causes all that harmonic distortion, I don't really know but if you look in the list of exceptions to the CE power factor requirement, there's a whole list of suspects like cheap LED lights under 25W, heating controls under 200W, large arc welders, etc. In addition, some requirements only specify a power factor >0.7, which is pretty bad. Then there's the whole issue of the legacy installed equipment.