Depending on your requirements (voltage, current, definition of affordable) the cheapest option would probably be an analog* precision power supply (e.g. HP 611x, some of the Power Designs units, Lambda LR-61x-DM). They are probably a couple of decades old, but generally well made. Obviously no digital programmability. Available with voltage ranges from 10 V to 200 V, and power generally 20-40W.
Other options that might be affordable is voltage sources. The EDC ones (e.g. MV-105/6) are available fairly cheap, but very limited current capability (50 mA?). The IET VI-700 is available with GPIB option (can not do remote range switch), slightly higher current capability (70 mA guaranteed, up to 140 mA with reduced accuracy). The Keithley 230 is more expensive, goes up to 100 mA and is fully programmable (including current limit).
The third option that comes to mind is old source measure units. Definitely more expensive, but I have seen the Keithley 23x series go for ~300 in as-is state (schematics are available), obviously more in tested state. Step size goes down to 100 µV. The 236/7 go up to 100 mA, the 238 up to 1A.
Keep in mind that if you want an accurate voltage with a decent current draw, you should generally use remote sensing. Precision power supplies / SMUs will generally have remote sensing connections, voltage sources usually not.
You should go through the datasheets and do the math on the accuracy at 2 mV. Offset error may contribute substantially. If you can adjust the value based on readback, then resolution and stability may be more important than accuracy. Especially if the source can also go negative.
*) I guess technically setting voltage via thumb wheels / rotary switches could be considered digital control.