Places like DigiKey, etc sell assortments.
http://www.digikey.com/product-search/en/kits/resistor-kits/2490615Years ago I bought small boxes from Radio Shack and the first couple assortments I ordered from DigiKey came in the same size as well.
I started with a resistor and capacitor kit, then when I'd run out of one size I'd replace it with a 100pack. For example the kit might come with 10 .1uF caps so I'd order 100.
I'm in the process of re starting the hobby again now that my kids are old enough to be interested. So I'm using some of my old stock and then adding to it.
In the picture you can see the little boxes (opened here holding resistors), there are several inside the toolbox I've had since I was a kid, then I added the (fishing) bag to the right with mechanical parts.
I also saved up and bought one nice big protoboard and seem to have lost it somewhere over the years. I've now bought a few smaller ones so I can put a project I'm working on away until I get the rest of the parts it needs.
I'd start with a resistor and capacitor assortment, a few regulators, a couple smaller protoboards and wire kits. Then decide what you want to make and buy stuff specific to that. For example I'm re-learning PIC's after selling my programmer after college so I bought the Pickit3 and 44 pin demo board and a few other pics and they are in the microchip cardboard box.