An old low frequency transformer mains adaptor would do fine but as you've said it may be too big to be practical and they're getting rarer nowadays and are banned in some countries because they're to inefficient.
A soft start needn't be complicated. A MOSFET, resistor and capacitor are all that's required. The attached example uses the Miller effect to make C1 look much larger, when the MOSFET is turning on. The simulation shows it will take about 3s for 90% of the supply voltage to appear across RL. If this is too slow, use a lower value capacitor or resistor.
Note that it isn't a logic level MOSFET (I chose it because it will be slow) but should be fine for 10 or so Amps when the battery voltage is 6V. Check the MOSFET will pass enough current when given a gate voltage of 6V.
I've used a similar circuit before to power a motor off a switched mode power supply and it worked quite well.