Apologies for not spotting this sooner and sharing my own related experience.
At one point I tried to implement digital control of the Vadj on an OCXO with an integrator instead of a DAC, to avoid the oven around the DAC+Vref.
That is basically the same thing you are trying to do, only I had an input which could be be brought high or low for some amount of time to adjust the voltage as necessary.
My direct inspiration was the integrator circuit in the HP5061 and HP5065 atom frequency standards, which use a 5µF Polyethylene capacitor.
My target time constant was much longer than HP's, hours and days rather than milliseconds, so I spent (too much) time trying to find a better capacitor.
My findings were roughly:
I didn't try vacuum capacitors - too bulky for their small capacitance, but I suppose they would be close to perfect (if kept in darkness!)
Silver-mica was very stable, but since they come only in very low capacitance, even fA leakage dooms the idea.
Tiny ceramics have the same problem, but worse stability.
High capacitance ceramics are sensitive to everything, up to and including what you ate the day before.
Anything with liquids in it is hideously sensitive to temperature and air-pressure.
Stabilizing the temperature only got me to a nonlinear barometric sensor.
(I only tried ultracaps up to about 5F in the usual cylindrical housing.)
That more or less left film capacitors, and they were all over the place, even for what is supposed to be the same kind of film from the same manufacturer.
Dont even think about paper.
Teflon is supposed to be the superior plastic, but impossible to get hold of in relevant capacitance. (Probably because of Wassenaar 3.A.1.e.2)
Back then eBay sellers from Russia sold what they claimed was teflon capacitors, but at prices and with appraisals which literally screamed audiohomoeopathy, so I figured they didn't have any teflon in them in to begin with.
Taking a clue from HP's use of PE, I spent a fair bit of time on plastic film capacitors.
Around this time AoE III came out and said basically the same thing: kV plastic films.
The good news is that those are cheap, so you can afford to experiment.
With µF sized capacitors rated for 1kV voltage, and everything washed in iso-propanol after soldering, leakage current is a non-issue, my HP3458 had a hard time measuring the self-discharge on my timescale of hours and days.
They were heavy enough that I saw no signs of microphonics until I literally struck them with a pencil.
I could not attribute anything to humidity, but could also not yet rule it out.
Some of them were light sensitive, but a good box fixes that.
Air pressure mattered, not much, but relevant. I saw both signs, so I figured I could probably compensate that out with a set of handpicked capacitors.
But temperature mattered a lot, and with the selection of dielectrics I tried, the sign was always the same, so ovenizing would be required.
Since that was exactly what I had tried to avoid in the first place, I sort of gave up around that point.
I should note that I saw "pop-corn" changes in the voltage, and since their rate seemed to be roughly proportional to the physical volume of the capacitors I made a note to find out if cosmic rays or background radiation were relevant. Never got to it.
As I understand it, you are targeting a timescale of "a day in the lab"-ish.
You can probably ignore the air-pressure, provided you check the barometer before and after to see that it didn't change too much[1].
I'm less convinced you can avoid an oven, or at least thermal insulation, so I'll conclude with a cheap trick for high stability work in both frequency and voltage:
Find an old fridge, ignore the compressor.
Convenient door. Good thermal insulation. Faraday shielded. Easy to drill holes for wires.
Preinstalled tubing to run water through, if you want to do active temperature control[2]
Add a couple of bricks on the top and bottom shelves for increased thermal impedance.
Have fun :-)
/phk
[1] My thoughts go out to Kleinstein and PTB: Looks like some really bad weather coming your way.
[2] Please get a qualified cooling technician to evacuate the gas so you don't contribute to atmospheric pollution.