The ONLY thing that ground serves, is to sink protective fault currents from the line.
If you have a metal enclosure, you must either:
1. Ground it, so if the enclosure becomes energized from a loose wire inside, it shorts out and blows the fuse, or
2. Insulate it, so if it becomes energized, hazardous voltage is still not exposed to a user. (This is called a "double insulated" appliance, and may not be legal everywhere.)
Note that your Arduino is grounded, or can be (via USB to an also probably grounded PC). It would probably be a good idea to maintain that, so that if a line wire goes somewhere it's not supposed to, you don't get shocked (or cause damage) plugging in a USB connector, or touching the low voltage parts of your circuit.
Speaking of, there is also terminology for the low voltage side. If there are two insulation layers between mains and LV, you have reinforced insulation, and it is SELV, safe extremely low voltage (no mains connection, insulated to >= 2.5kV RMS, not susceptible to a single-point failure). It can remain isolated, or you can ground it as needed.
If you didn't have reinforced insulation, but a single layer ("basic"), you would have to ground your SELV circuit, because otherwise a single failure (of that single "basic" insulation layer) could connect it to mains. Grounding it provides a path for fault current, blowing the fuse and returning it to a safe condition.
Speaking of, you need to be sure there is fusing on the hot mains lead (sometimes, the neutral requires a fuse as well, in case of possibly reversed wiring). Usually, those power supplies have this inside... but you can't very well verify that, can you? It's in a solid black box...
So, best practice also to add a fuse.
Finally, keep all the mains connections well insulated, tied down, away from fingers. Preferably closed under a cover that requires a tool, such as a screwdriver, to open. (And guess what, if the cover is insulating, it counts as another layer; if no live wires are physically touching it, then there's another layer of insulation as well, be it air or wire insulation or what; and the two combined therefore counts as reinforced insulation, so you're golden! Or if it's metal, simply ground it, and you're still golden.)
Follow all these rules, and you will have a good chance of getting UL approval, which means if anyone
else touches the thing and gets zapped, you aren't liable for damages. A big deal, if you later discover you can make and sell a million of something!
Even if it's just a one-off, these rules should at least make you familiar with what's needed to build and handle a project, safely. No, I certainly wouldn't expect you to get UL approval for a stack of proto boards -- testing is in the $10k range! -- but if you ever develop something for market, this also gives you a good idea of what's needed and worthwhile.
Tim