Using a smaller LED is no problem. If anything it’s desirable because it gives you more tolerance for positioning inaccuracy. What you do want, as much as practical, is to minimize the gap between the light pipe and LED. Like… you want a half mm or so for positioning tolerance, but beyond that, the larger the gap, the more light can exit laterally and illuminate other segments.
I recently used the panel-press-in light pipes for the first time. Worked great, but to have them spaced closely as I did, I did find it necessary to prevent optical crosstalk by slipping black heat-shrink tubing over the pipes after insertion. (I then used a fine hot air nozzle to ever so slightly shrink the tube right next to the panel. (Heating it at the open/PCB end would cause the tube to shrink over the end of the light pipe, which I didn’t want.)