The most common solution is to spot weld tabs to your batteries. You can even buy batteries that already have these tabs and can be soldered directly into the PCB.
Some people claim to have success with soldering wires directly to the battery by using a hot iron, keeping the heating time to an absolute minimum, and cool with water (wet cloth) directly after soldering. But I'm not sure what that does to battery internals and it's probably a bad idea.
You can also buy batteries (or battery packs) with wires and a plug on them.
For the rest, I'm quite happy you can buy decent battery holders for 18650. For AA for example the battery holders are usually so bad that they become very unreliable.