The program resides on the magnetic drum while it is executed. (And the data too, it's a von Neumann architecture.) A drum rotation takes 17 ms; with carefully optimized programs, up to 7 instructions per revolution can be executed. But that's a somewhat theoretical optimum, a couple of instructions per rev are probably more typical.
Edit: As a nice side effect of the magnetic drum architecture, all information is non-volatile, including the CPU registers (which reside on the magnetic drum too). Stop the program, switch the computer off, and restart execution in the exact same place tomorrow. "Hibernate" mode is not a 1990s Windows innovation!
The hardware supports 31-bit fixed-point arithmetic, including division and multiplication in hardware. But there are floating point libraries, and even a compiler for a basic Fortran-style language, ACT.