You cant necessarily use magnetism to determine a material. Brass and most Copper alloys are weakly magnetic to modern Strong magnets.
A small (about 10 mm diameter, 5 mm thick, NIB) magnet will not noticeably attract high purity piece of copper or brass. I learned about this trick from an engineer who worked on MRI development. It will easily and unmistakably attract something with a thin nickel plate, which is used in the vast majority of gold-plated copper, precisely because you can use a much thinner gold plate. Otherwise, you need a much thicker and more expensive gold plating that very few are willing to pay for, unless they absolutely feel they need it.
Note that I specifically said attract here, not move. If something is magnetic, or paramagnetic, the force is attractive.
Brass is another story. Lots of brass has some iron content, especially cheap brass, because they melt a lot of scrap, and they are not going to spend a lot of money extracting every steel nail, screw, or hinge pin from the pile of scrap. This was a sad discovery by my MRI colleague, who discovered that hundreds of brass machine screws used in an MRI were magnetic. He learned to test them by laying them on a flat surface and seeing if a magnet would make them roll across the surface. The ones that did were not used.
It is true that copper has a non-zero magnetic moment. A much stronger effect is the eddy currents induced by relative motion of the copper and magnet, but this can be alleviated by moving the magnet or copper piece very slowly.
Cheers,
John
Edit: fixed a typo