No I removed the shortcut from the ferrite between the 5V line and the 5V LDO output (feedback is still from the 5V line). I then remeausered the noise pulses. The 3.3V line on both sides of it's ferrite were not changed at all and only a very minor change at the 5V line. But at the 5V LDO output side of the ferrite there is a very clear change. The p-p dropped from ~30 mV to 9 mV. Pictures attached so the same as before + the other one showing LDO output pin in yellow. With a multimeter I can get more noise at the 5V line (~4 mV RMS) after this modification and it is showing only 200 uV RMS at the LDO output.
Actually this is the same thing I noticed in the very beginning, that the LDO output is quieter than the 5V line after the ferrite, which was put there to make things reversed.
There is no capasitance between at the LDO ouput, only on after the ferrite. So I can only imagine, that 5V LDO is not making any part of this noise, it all comes through the 3.3V LDO. So I'm guessing, that connectig 3.3 LDO to the other side of the 5V LDO (5.6 V line) would eliminate all the noise on the 5V line. Also maybe getting 5V line directly from the 5V LDO output and 3.3V LDO input through ferrite might work quite well.
Those are a bit more difficult ro rework on the PCB I have, thus I don't know yet will I test them or not. After all I don't really have a noise problem anymore, this is just trying to learn something.