It’s perfectly normal, it’s capacitive coupling through the air. The high input impedance of digital multimeters makes them capable of measuring voltages with practically no current.
They even make adapters to suppress these “phantom” voltages.
Also, for fun, watch what happens if you touch the common test lead while the red V lead is connected to live.
(IN VOLTAGE MODE ONLY!! DO NOT ATTEMPT IN CURRENT MODE!)
This is the same effect that lets you take a modern ultra-efficient LED, like a white or emerald green one, connect one lead to earth ground, and touch the other lead, and in a dimly lit room you’ll see it light up!