Do some research with the online patent systems. Google patents is OK, I find the European espacenet.com site a bit easier to use.
Very few patents / inventions are totally new. Most will have a whole family tree of prior art. Just search for relevant terms and see what comes up. Most will also have cited and citing references, plus the patent description usually starts with a list of already known prior art patent numbers. All patents are also grouped by classification numbers, sometimes searching through a particular class can also bring up results. Searching for other work by a particular inventor is also worth doing, since they often have other patents for similar ideas. Basically, you need to spend a good bit of time going through the archives to see what's already been done. (It's not to bad these days. Before everything went online you had to search the records on microfiche, which was not much fun!)
As for what you can get away with, anything mentioned ("disclosed") can't be patented, but can possibly be used without infringement. Anything contained in the claims is what's been protected, so you can't copy that.
However, all patents expire after 20 years, so anything older is now public domain. A lot of patents are also abandoned before 20 years (check the legal status links), so they're OK to copy also. On top of that, patents are country specific, so even if a live patent exists in one country you can still copy it elsewhere. (You just couldn't produce or sell anything covered by the patent in a country that is covered.)
As for the "I didn't know" ignorance excuse, that won't work. Also be aware that in patent law there's no statute of limitations, so you could be sued years down the line, even after any infringed patent has expired. If you were copying it while it was still in force, then you're liable.
If you feel like hiring a patent attorney, they will simply do exactly what I've described above. So DIY and save yourself a lot of money.