GPU accelerated video rendering is a hot mess since years. As far as i understood it NVEnc is a nVidia only thing will only utilize GPU, but not the CPU. Handbrake H.264, FFmpeg, and according to
Wikipedia allegedly MAGIX Vegas too, support OpenCL and could theoretically utilize CPU and GPU (even on
nVidia GPUs with latest drivers), but no idea if the encoder can or does select that.
The thing is, the libx264 (used in handbrake and ffmpeg,...) does
some lookaheads via OpenCL, but not the main encoding cause they did not want to rewrite the whole code. So it won´t drastically benefit from it. Handbrake can move scaling to OpenCL, ffmpeg can filter with it, but itself can not beat libx264 to fully use OpenCL. So "support for OpenCL" is to be taken with caution, although the
list of encoders with OpenCL support is long.
Handbrake and
ffmpeg however also support NVEnc. Why mention ffmpeg? Because it always represents a good state of open source support for these things and is actively developed.
AMD allegedly worked on a fork/version of h.264 that uses OpenCL, but nothing is to be found except some interviews and an old anandtech article.