It sounds like Microchip programmers have a lot of bugs that drive users crazy.
I'm not looking to spend a hundreds of dollars. A cheap beginner kit would be good for me to get started.
I would not say 'lots of bugs'. I have a Pickit3 and occasionally will have to unplug/plug back in, but not very often. I also have a SNAP programmer which has never had any problems so far.
This is what I would do- buy a curiosity board (~$20)-
https://www.microchip.com/DevelopmentTools/ProductDetails/PartNO/DM164137it has a programmer/debugger built in (basically low-voltage program only pickit3), includes a chip, you can swap out the micro to any low-voltage programmable 8bit pic from 8pin to 20pin, has led's, a switch, a pot, and you can get familiar with it without anything else needed.
The next step would be to buy a SNAP programmer (~$15), which is basically a faster version of the on board programmer/debugger- which you can connect to the curiosity board. Not necessary, but nice.
There is no advantage in using the older 'high voltage programmable only' devices, and although the newer devices also allow high voltage programming, the only advantage is you then get to use the reset pin as an input pin- which can be nice with a low pin count device (with low-voltage programming, the reset pin is exclusively a reset pin), but usually the better solution is to simply move up to the next size pin count device in the same series (and there WILL be one).
When searching for pic 8bit parts (parametric search on their web site), select the parts with 'PPS' and you will be looking at the newer parts- they are better, cheaper.