At optical frequencies, metals have a very high, imaginary index of refraction.
At low frequencies (say, < THz), we express that as resistivity. Resistivity includes a length dimension (e.g., ohm-meters), meaning it depends on scale. So, long waves encounter a lot of material, and therefore a low resistance, so are easily reflected by metal surfaces -- hence waveguides (TE or TM modes) and transmission lines (TEM00 mode). But short waves encounter much less, much more often -- so the attenuation is much higher.
Essentially, the attenuation of a waveguide or transmission line, scales proportionally to width. At radio frequencies, we use coax and other cables, of various human-scale dimensions. Something like RG-8 is good up to some tens of GHz.
Transmission lines stop being useful, when higher order waveguide modes take over. You lose the valuable assumptions (flat frequency response, impedance, low dispersion) and so either thinner cross sections need to be used (the lowest waveguide mode is inversely proportional to width), or proper waveguides must be used (which have significant dispersion even in the intended operating band, making them a challenge for some applications.
Transmission line or waveguide can only be made so small, until, well for one it's just so hard to handle (and must be made to such exacting tolerances, connectors especially), but also the number of waves interacting with it per unit length goes up with frequency. Basically, the losses, the attenuation per length, goes up and up and up, and pretty quickly it becomes futile -- useless over any real distance.
Waveguide has higher efficiency (lower loss per length), so is useful to higher frequencies than TLs, but the same problem once again applies.
At some point, we stop using wave physics, and geometric optics takes over. We note that a waveguide is simply a shell of very high index of refraction, with a low index inside of it. The reflection off that high index, is where the low loss comes from. Where does the remainder go? It is absorbed. At low frequencies, we use skin effect; at high frequencies, the skin is so thin (fractional um) that we really don't care anymore (it's only relevant for metal films, perhaps*) and can consider its index of refraction instead.
*Which, if you've ever held a CD up to a bright light, you'll have noticed they transmit blue light, of all things. This is probably partly due to thickness (interference blocking red light from passing?) but also partly due to plasma frequency or something like that (metals stop being metallic at high enough frequencies).
The index of refraction is given by the dielectric permittivity and magnetic permeability of a material. Well, for something like copper, permeability is 1, that's easy. What's permittivity? Well, in the same way that a resistance becomes an inductance when multiplied by the imaginary constant j = sqrt(-1), a capacitance becomes a resistance when multiplied by j. So we can encompass both dielectric permittivity, and resistivity, as a complex-valued permittivity. So, when I said earlier that metals have a high, imaginary index of refraction, this is what I mean -- it's a roundabout way of saying, the resistivity is much lower than that of free space, and indeed dominant over the permittivity of free space as well.
And it is indeed an index of refraction; if we look at metal films, the thickness required for attenuating certain frequencies, is very thin indeed (10s, 100s nm?). We might wonder about total internal reflection, but the loss means it's not going very far, so, nevermind that.
Oh yes, loss -- an imaginary index, means it attenuates as it goes -- the wave vectors point in on themselves and decay exponentially; whereas for the real case, they continue spiraling around as normal. It's the opposite of how we use complex numbers for AC signals, with real values corresponding to growth or decay, and complex values corresponding to oscillation (i.e., the value of exp(a + jb)). Simply put, when the index appears in wave equations, it happens to be paired with a j, swapping the real and imaginary components around to give the expected mathematical correspondence.
And, as it happens, magnetic materials aren't much better; YIG is good in the low GHz, but I don't think is useful much higher? (I see a few articles with YIG and THz keywords, which look to be pretty crazy stuff well above my pay grade..!) Not sure what else, if anything, extends up there.
So we use dielectric waveguides, to go higher. We can use these just fine at ~GHz frequencies too, they're just rather bulky -- the waves aren't strongly contained, the field is nonzero outside of the main body and tends to leak into anything nearby. We need to reserve a lot of space to prevent that, or add additional layers (with thickness on the order of 1/4 wavelength each!). Or use a properly shielded waveguide, which, well, just wrap the thing with metal right, can put anything right up outside one of those -- hence standard metallic waveguide, eh?
But at optical frequencies, not only can we afford extra shielding, in terms of available space -- we are obliged to use it!
So, we make optical fiber for instance, with a high index n in the core, surrounded by lower n, surrounded by a jacket, or additional layers, or air. This exhibits total internal reflection (just how, underwater, the water surface looks nearly perfectly shiny at glancing angles), and the material losses can be amazingly low.
So, all this is a big part of why high frequencies "don't like to stay inside wires". There's some additional weirdness as well, when it comes to metals, in the THz range. Metals work, because electrons are free to move within them; at some frequency, this assumption no longer holds.
Imagine incident waves, with a frequency on the order of the effective mass of electrons in the metal. This... is a bit of an abuse of units, you can't compare mass and frequency after all. Well, the electrons have some energy level (thermal or Fermi) which gives them a velocity and equivalent frequency, the plasma frequency. (I'm really winging it by this point, by the way; I know very little about the underlying statistical mechanics in this regime.)
Also something about Debye length, which has to do with the shielding effect of electrons in the metal. That is, if we apply a (static) electric field, electrons will be drawn or repelled by that field, at the surface; but how thick is that "surface" really? The answer is, this length. The electric field penetrates into the (atomic) surface, on the order of this length. 10s of nm for most metals, I think it is?
Effectively, weird things happen when the classical skin effect depth, is of the same order as the Debye shielding length. It is no coincidence, that these effects take over, at frequencies where quantum energies are substantial -- the band structure of the metal might be some eV across, so we expect to see something interesting happen for photons on the same order -- i.e. visible to UV light. Indeed, all metals (as far as I know?) have the effect of looking "pink", like copper does: the only difference is what wavelength is "pink" to them. Copper happens to have the lowest cutoff (plasma frequency), so looks pink in visible light. Gold, and to an even lesser extent cesium, looks well, golden, having a somewhat higher cutoff. Most metals look silvery because the cutoff lands in the UV range (say 5-10eV). But if your eyes could see into the UV spectrum, you would see exactly the same cutoff for everything else.
The plasma frequency generalizes to any conductor (or gas of charged particles, at least -- including actual plasmas, hence the name). The lower the carrier density, or interaction rate or something, the lower the frequency -- as it happens, Earth's ionosphere has a cutoff around 30MHz, below which it is reflective (hence the long distance propagation of short- and long-wave radio signals), above which it is transparent (hence VHF+ for space communication and radioastronomy). The interplanetary medium is also a plasma, with even lower frequency cutoff -- we might expect to get effective shortwave, even very long wave (10s kHz?) radioastronomy at a moon base (there have been proposals to use craters as huge antenna dishes substantially dwarfing the former Arecibo!) -- but not much below that, as the interplanetary medium is filled with plasma from the solar wind, with a cutoff frequency of some... kHz? 100s Hz? Not sure, somewhere around there.
Which means, down in the ELF, we well and truly are, by ourselves, all alone in the dark -- well, in the white noise of solar and planetary EM activity at least. We will most definitely never see communications from extraplanetary sources in the <kHz.
Tim