The battery looks a bit bigger, but 1/2 AA and 2/3 AA 3V and 3.6V Lithium cells are commonly available with solder tabs or wires. Search for instructions on the HP 3478A and 3468A, which use such a battery to store their calibration values.
My technique has been to both solder a backup battery holder with 2xAA (I don't remember if I used a series diode) and also use a floating soldering iron so I can keep the unit powered. Obviously only do this if the battery contacts are well away from any mains wiring. Now I have a TS-100 soldering iron (similar to
this) that I can power from a bench power supply or other floating supply. What I used to do before that with my Hakko 936-clone was to heat up the iron, then unplug it from the station, quickly solder using the heat capacity of the tip, and then plug it back in to heat up again. This works best with big tips. Again, only do this if you feel comfortable working on live equipment.
I don't think it's possible to read parallel RAM in-circuit, though I can't say I tried. For the HP 3478A and 3457A people figured out commands to send via GPIB to copy the contents, although I'm not sure if they also commands to restore the contents. What might work if you're motivated is record all address and data lines with a logic analyzer, and reconstruct the contents from that. If you're lucky, it will read the entire RAM on start up to calculate the checksum.