I've got a proxxon mf70 that I converted to CNC. I mostly use it for precision drilling, but I have used it to mill a few simple single-sided boards and a few other things. It's really easy to do a CNC conversion, but I probably wouldn't buy it again. It is *really* small.
I also have an X1 from Little Machine Shop. http://littlemachineshop.com/products/product_view.php?ProductID=4700&category=1387807683, and I really like it. I haven't converted it to CNC, and I probably won't, but it's a nice small machine. I have some unboxing videos on my blog. I got the unit with the MT2 spindle and bought an MT2 to ER-20 adapter for it.
I also have a old sherline lathe that I got a really good deal on through ebay. I bought an MT1 to ER20 adapter for it, so I can use the same set of collets on my mill and lathe. I really like the sherline, though I run into its size limitation more often than I'd like.
I'm running similar, sort of... I have a harbor freight X1 that I got from craigslist for a few hundred bucks w/ basic tooling, cheap mics, mills, vice and some bits and parts from LMS (along with a 7x10 lathe as well) I ended up converting it the X1 to CNC, It does aluminum, but not well, a part that would probably take a average machinist a couple hours on a bridgeport takes all day on the X1 CNC lol.
On the other hand I also have a extended x/y X2 hybrid CNC conversion that I am building from some LMS parts (extended x/y table kit, solid post conversion, head stock/R8 spindle from another machine), ebay ball screws, budget steppers, and a brushless 750w sewing machine motor + controller.
As for tool holding, with the X2 and bigger IMO I would look to getting 3/4 shank holders (china) or the Tormach tool holders, then really you can run the tool holders on anything with a 3/4" collet in it, I almost picked up a 5000lb Van-Norman mill with some odd spindle that has collets that run in the $200+ range with that idea (but it needed way too much work...)
If I ever get around to getting the X2 don, I'll probably look in to doing a brushless conversion on the X1, new bearings and up the spindle speed to use it more for just engraving (really too much flex for much else)