I am a hobbist like you.
Some people here will say you can never have too many handheld DMMs! I believe you need at least 2 average ones to begin with. The Fluke 17B+ is a good entry level DMM. There are good and inexpensive ones from UNI-T as well.
One DMM that is surprisingly good is the Mastech MS8260E (or F, G). It is a cheap Chinese DMM, but does the job done. Its latest revision, MS8260G, has auto-range. It is great as a hobbist's first DMM. When you can, upgrade them to more advanced DMMs, from Agilent, Fluke and so on.
The o-scope is a good thing to have as well, but I believe it can wait.
Before that, you need a good adjustable or programmable power supply, preferably with negative output too (if you plan on working with opamps). You can roll your own, which is a good thing and you'll learn a lot in the process. For those in North America, there are often good used units (HP, BK Precision, etc) on eBay.
You can also make your own basic signal generator, which won't feature as many bells and whistles as a commercial model, but that will also serve as a valuable learning experience.