Hey eevblog!
Wanted to add my two cents on this one. If you find it fun, take it as slowly as your budget permits, but never give up on something you find entertaining.
I started a break fix computer company back in 2000 and, over time, moved into network infrastructure design and eventually .Net C# app design. What I do for a living uses electronics but really has nothing to do with electronics design and repair. It has been a hobby of mine though since I was a little kid when my dad would bring all kinds of electronics that failed tests at his job including the kewl capacitor I use to charge to shock my little sister (in hind site, that may be what is wrong with her today, hehe).
I had been running with an Ideal 61-361 DMM I bought back in 2004. I think I bought it for $100 @ Home Depot in the states. I learned a lot with the meter and, at one point, went through a lot of freaking fuses. Mainly bought and used it to do some additional work while running Cat5e or building phone switch boxes. Didn't really get back into electronics until a year or two ago (from when i was a kid).
When I did get back into it, I started by pulling out the old Radio Shack electronics kit, then moved to the Basic Stamp2, and then started playing with Arduino. Thanks to Dave, I have jumped into the Renesas products and have really loved them so far.
Fast forward to today: I realized over the past few months that I really needed a second DMM with the hobby projects I am working on. I guess as you get older, you find a determination to follow something through and you also learn that buying cheap Chinese crap will cost you in the end. I just picked up a brand new Fluke 87-v due to that methodology. Fluke products are great and have had plenty of fun using them in the networking side of stuff. I do plan to keep it away from my cell phone for sure. Since it will sit on my hobby bench though, I am not too worried.
BTW a little tip. I picked up the DMM off of CraigsList brand new for $125. Apparently the guy I bought it from receives 6 or so free DMMs a year. Major steel (although I wish I had been reading the forum when Granger was selling that Agilent DMM for cheap)!
Point is, if you are serious about any hobby, do spend good money on the right tools. You will thank yourself later.
If you don't have the time and/or the money, build up your lab and parts slowly. Salvage anything and everything you can get your hands on.
If you want to test the waters though, get a $30-50 or so meter, an arduino/pic/basicstamp starter board, a few components and a breadboard and go to town. You shouldn't spend more than $100-150 to test the waters.
Realize you will fail. Take a note that you will have to pick your head up off of the floor. And do try-try again.
Electronics knowledge is a very useful thing considering we live in such a disposal society these days. Even if it is just for fun.
Good luck and go build something!