You can use digital oscilloscope or digital analyzer with analog input with high enough input bandwidth and capability to record long enough continuous sample. Then you can analyze it in Octave or Matlab.
For example my SDS1102X allows to record 14 Mpts record at 1 GHz, 500 MHz, 50 MHz, 10 MHz, 5 MHz, 2 MHz, 1 MHz, 500 kHz, 200 kHz, 100 kHz, 50 kHz, 20 kHz sample rate (sample rate below 50 MHz needs to manually disable Roll mode).
RBW = SampelRate / N
where N is point count
As you can see at 10 MHz sampel rate it has RBW = 10M / 14M = 0.71 Hz.
But this is for rectangular window (or no window) function.
If you're applying some window function before FFT, it affects RBW. So it depends on your window function choice, which depends on what is more critical for you - frequency resolution or amplitude resolution.
With window function RBW = RBWrectangular * ENBWwindow.
For Kaiser window it can be estimated as: ENBWKaiser ≈ 1 + β/2
where β is β parameter for Kaiser window, which depends on your requirements for frequency or amplitude resolution.
For example you can estimate Kaiser β parameter with this Octave function:
% Oppenheim, Schafer, "Discrete-Time Signal Processing", p.475-476.
% A - attenuation in dB
function beta = kaiserBeta(A)
if (A >= 50)
beta = 0.1102 * (A - 8.7);
elseif (A < 50 && A > 21)
beta = 0.5842 * (A - 21)^0.4 + 0.07886 * (A - 21);
else
beta = 0;
end
end
PS: as you can see, there is no universal solution and it all depends on your requirements.