Yes, that's overkill for anything except precision cell testing.
Fancy is usually any hard plastic tube you have handy that's a reasonably good loose fit for the battery diameter, with a couple of holes in it for nails across it. A conical spring for contact pressure is nice, but at a pinch, a bit of aluminum foil wrapped round a chunk of closed cell foam will do. It gets a bit unwieldy if you want 12V as for an AA pack that ends up being about 17" long!