I'd use something that's not easy to remove, but not permanent either, SD cards can wear out and fail.
I tried a 2mm blob of epoxy in a corner with burned board & card, not good, it tore apart the resist layer of both pcb and card when removing it.
Better a hard, rubbery compount you can easily cut.
If this is meant to be definitive (Unless maybe a mainteinance routine), then why the fear of breaking it, when people is not supposed to access it?
I like hinge slots quite a bit, to me they're actually more robust unless you break it when handling, easy to replace when broken...
The latching type can fail more easily, they're more prone to have issues with dirt / flux / contamination, and I found out that lots of these don't actually hold the card, you can simply pull the card out with tweezers.
This was the problem we had with these PC104 boards, the slots were still "clicked in" but the cards were falling off.
You could solder the card directly into the board, though not funny if needing replacing:
https://hackaday.com/2015/08/18/reflow-solder-your-micro-sd-to-ensure-it-doesnt-go-anywhere/I'd put a small pad nearby and solder the hinge to lock it in place.
Make the pad part of ground plane (Solid) or it will lift easily, small so it holds just enough solder to lock the hinge in place (About 2-3mm / 80-120th seems reasonable), and make sure it doesn't go under the SD slot or it will be hard to clean.