While I agree that pencil erasers are not the appropriate cleaning product for gold pads, your blanket statement that all pencil erasers are abrasive is just nonsense. Pencil erasers work by binding the graphite particles, which are not bonded to the paper. (Remember, the first product used to erase graphite was bread.) My understanding is that natural rubber erasers use powdered pumice not for its abrasive properties, but to make the eraser more crumbly. Plastic erasers don't need that, since they just choose a polymer that already has the desired crumbing properties.
Ink erasers, in contrast, contain large amounts of coarser abrasives. That's because they can't just bind the ink, but must actually debrade the surface of the paper. (We mostly just encounter this as the blue side of a double eraser.)