Here's a simple constant current sink. At 1-5mA, you can use almost any little NPN transistor.
Regulation won't be as good as the three terminal regulator method mentioned by another poster, but it'll do for your niece's needs. You do need more than 24V to accommodate the headroom for the regulator, but that's true of any method you'd use to achieve constant current. If you don't have a pot, replace the combination of R3 and R4 with a single resistor that gives your desired current with 5.1V across it.
You can probably use three 9V alkaline batteries to power the experiment. Current will fall with battery voltage, but that may not be an issue over the duration of the experiment.
ETA: How do I get the attached image to show full size in the post? I'm new here!
ETA2: Thanks for the tip, Brumby.