Just because it has the output negative connected to the neutral pin doesn't mean that there is a always a capacitor connection to both sides of the mains AC. You can verify this by measuring the current AC DVM ammeter to earth ground connected to the ungrounded wallwart power supply.
Let's assume it does have a wallwart with two small-value caps to the neutral(it is not really likely that this is the case).
When you connect the power supply to a "sensitive circuit" and it is, for some reason, grounded to earth, it should not bother the circuit. It is connected to ground, so no problem. If you connect the ground of the power supply first (which is always a good practice), then there is no problem.
Assume you are forgetful and connect the positive lead of your wall-wart powered LM317 power supply first, would you not be connecting it to a large value electrolytic bypass filter cap. on your "sensitive circuit" that is earth grounded, so this bypass cap. would shunt the tiny AC current from the wallwart to ground..again no problem.
Assume your "sensitive circuit" is not grounded to earth ground, now there is no problem because no current from the AC mains will flow.
Assume your "sensitive circuit" is connected to your DIY power supply and not grounded to earth ground and you decide to connect the grounded lead of your bench oscilloscope the gate of a unprotected MOSFET on your "sensitive circuit for some strange reason)..it could be for some destructive reason..maybe a problem.
Assume your "sensitive circuit" is connected solidly to a earth ground and you walk across the room on a rug and build up upon your body a nice static electricity high voltage and then you touch the "sensitive circuit" in a sensitive place..maybe a problem, even without your custom power supply connected at all.
Assume your "sensitive circuit" is connected to an earth ground by your oscilloscope that is 3-wire grounded and you connect your "sensitive circuit" to a long wire going to your entertainment center across the room, also earth grounded, but to a different AC circuit in the building. There could be a high current, low-voltage difference between the grounds an you burn out your "sensitive circuit" because of this earth grounded AC difference.. big problem.
Assume your "sensitive circuit" and your power supply are both connect by 3-wire cords to earth ground neutral..no problem.
Finally, assume you've got y our custom power supply securely earth grounded with a 3-wire cord connected to the earth ground. Now for some misadventure reason you are holding the power supply ground clip and by neglect or accident you somehow come in contact with your other hand to a live AC mains connected wire on something you are working with, even though it is also earth grounded. Now the "safe earth ground" of your power supply serves to complete the path of the high current AC mains voltage through your body..big problem. It is interesting to note that if your power supply was not earth grounded, then you would not have a chance to suffer a fatal shock in this same unfortunate situation.
In Denmark, they sell and always supply laptops and printers (and many other equipment) with 3-wire and two-wire cords to connect to their wallwarts, and I reuse these wallwarts all the time (with the supplied two-prong power cords) to power breadboards on my electronics bench with no problem and I don't have anything earth grounded. Most houses here do not have 3-wire wall outlets.
There have been many other discussions before on this topic and some responders will take much time to disagree with me again on this topic, but I always welcome both sides to the story.
https://www.eevblog.com/forum/beginners/is-it-safe-to-use-a-bench-power-supply-not-grounded/msg254146/#msg254146