I'm sorry the results were disappointing, but I didn't expect anything else. GPU-accelerated functions require tight integration between the OS and it's APIs, the hardware, and the application. This is difficult to do. Somewhat ironically, even though you couldn't get big benefits, Apple re-wrote a lot of it's software to use intel's quick sync video accelerators built into its latest processors. Ironic that a 10 year old with iMovie and a "slower" computer can do HD iMovie exports faster than your professional software. It's also used for FaceTime and in-home video streaming (airplay).
If you really want hardware accelerated encoding, your best bet is trying to get Intel Quick Sync working. It's very fast, but only some applications can use it. The problem with hardware cores is that the encoders are, well, written in hardware, and limited by that hardware. Quick Sync doesn't have the same quality as x264, which is by far the best AVC encoder out there, IMHO. Unfortunately, it's written for x86 with lots of assembler built in, so while some hardware acceleration is being beta'ed, it's not ready for primetime and indeed the benefits aren't that large due to the crazy level of optimization that the x264 team has done. Handbrake uses x264 as it's encoder, so you're good to go there.
GPU acceleration, at this point, seems mostly focused on real-time recording (for gamers) and large-scale encoding (for broadcasters such as twitch.tv that need to re-encode lots of streams at once, in real time).
I use an AMD graphics card for my gaming PC and they've rolled out some software to do real-time recording. It really works shockingly well. I've been able to do 1080p60 h.264 15-20mbit/sec real-time recordings of my games and noticed maybe a 10-20% performance hit to the actual game.
The technology is there. You just can't access it.