I would use a lead acid battery, because it's easy to use and robust. The charging voltage should be limited by a shunt regulator, which will depend on the size of the solar panels. A single transistor can be used to turn the LEDs off, when the solar panel isn't charging the battery, with a diode in series with the solar panels to prevent the battery discharging, when it's dark. Connect the LEDs in strings of three in series, to save energy. Note that 5mm LEDs don't produce much light. You might want something brighter.