I would glue some thick mounting posts to the back of the meter, maybe with 2 part "Araldite", that do not obscure the battery compartment and screws, then make sure the ends that mate to the PCB are all sanded flush and level, then align the "stood-off" meter to the board and bolt/screw into the posts from the other side of the PCB. You could have some form of quick-release fasteners for this, so that removing the meter and changing the battery was easily achieved.
I wouldn't be thrilled about drilling holes into rubber/PVC holsters - if they're drilled through a flexible medium, what's to say that medium won't stretch open and allow the screws to slip through, if the meter is knocked or subjected to vibration.