Yes it seem to be a thermoelectric effect (but not necessary only from the battery), we are in this 10µV range, I removed the battery and replaced it with a wire (on crocodile clips)
And I could reproduce the effect, by cooling one end it went up, by cooling the other end it went down, although one end was more sensitive than the other.