Many, many moons ago, I was building a high gain audio preamp with RIAA-equalization, for playing LPs.
RIAA-equalization required of course, RC networks.
For optimal curve fit, the audio press recommended precision tolerance capacitors, which meant of course plastic foil types.
Long story short: one of the channels had significantly higher hum than the other, and that would occur even with the inputs shorted to ground.
In those pre-Google, pre-Siri, pre-Cortana days, the only way to solve the problem was by tedious and time consuming troubleshooting.
I discovered that, when I touched with my finger one of those capacitors, the hum would increase.
I said "aha! a defective capacitor", and proceeded to replace it -unknowingly- in the correct direction, which solved the problem. I strongly believed that it had been a defective capacitor.
Many years later, I read a paper which mentioned the pitfalls of capacitors in high-gain circuits, which mentioned the external foil issue.
Fortunately, I still had the preamp, which I was no longer using. Out came the soldering iron, and removed and placed the capacitor "backwards".
To my astonishment, the hum came back!!!
So yes, place the foil side to the lowest potential, preferably ground.