Why will I be bothered by that ?
And it doesn't bother you at all that there is no circuit you can put in region A in the diagram below that can extract energy from the wires that surround it? Even a DC/DC convertor? Even if you can connect it to a GND? But extracting energy from regions B or C is a piece of cake?
And I guess that it doesn't bother you if an isolated charge is placed in region A and it stays where it is put, but if the same charge is placed in B or C it will accelerate, acquiring energy ultimately suppled from the battery, without being connected to it?
And it doesn't both you if you charge a capacitor between +110V and +100V is has exactly the same stored energy as one charged between 0V and -10V? Even though one has been charged at a higher energy? And the other has been charged at a completely different polarity?
And it doesn't bother you that a transformer can get 95%+ transfer of energy from one wire to the other, even though the wires don't touch, and no charges from the input wire get transferred to the output wire?
And it doesn't bother you that for your version of electrostatics (sum of force between charges), every charge needs to be in consistent communication with every other charge in the universe, to work out how far away they are, and at what direction?
And it doesn't bother you that a transmission line is a series of inductors and capacitors, however on inspection those capacitors and inductors can neither be identified or isolated?
And it doesn't bother you that commercial radio transmitters can get kilowatts of energy to disappear into literally thin air?
You must be a firm believer in the Lumped Element Model. It seems to work, so to you it must reflect the mechanics of reality, rather than an useful abstraction and approximation that allows you to get stuff done.