8-Bit Micros will likeing be more costly to work with than Microcontrollers because:
1. You need to add external RAM and ROM to work with. You will also need to add logic to assign address space to the CPU. Usually the ROM is at the high address from FFFF and down, and RAM is at the low addresses 0000 and UP (assuming a 16 Bit address bus)
2. If you use EPROMs to store your program code, you will need to buy a EPROM programmer and EPROM eraser. You can use EEPROMs (but they can be a lot slower), You will likely need to add Wait states when addressing\accessing EPROM and EEPROMs since they may not be fast enough to operating at full processor speed. FLASH devices are better option, but you need a FLASH Programmer.
3. CPU tools for assemblers and compilers for 8 bit processors such as the Z80 may be hard to find. I purchased a x86 compiler\IDE more than two decades ago and it wasn't cheap.
Microcontrollers usually have built in FLASH and RAM, so you saving you the cost and design time. FLASH Memory size is rarely a problem, but RAM size in MCU's tend to be very limited, usually in the 2K to 4K range. You can find MCU's with larger memory, but these are usually in Surface mount packages. I assume your looking for DIP components so you can either make use a Bread board or Wirewrap board.
I have successfully built many CPU and MCU circuits using both breadboards and wirewrap. I still use breadboards for testing some designed, since its fast and easy to test designs, although I rarely use wirewrap. You can build simple MCU circuiton a breadboard in less than 30 minutes, but a CPU with RAM and ROM will take several hours since you have to connect the RAM and ROM to both the data bus (8 parallel connections) and the database (16 parallel connections) and add logic gates to handle the address space for the RAM and ROM. A 8 Bit MCU using a 16 pin dip will take a very small workspace on a breadboard, but a 8-Bit CPU will take up most of the board space.
Most of the MCU manufacturers offer low cost IDEs and compilers for there products, They usually have free student versions, but are limited in the number of lines of code you can compile. Assemblers for any size code are usually free. The only issue is that you need a MCU programmer, but this is far easier and cheaper to use than a Flash\EPROM\EEPROM programmer. MCU are programmed using a serial I/O to limit the number of pins used on the MCU, and they allow in-circuit programming so you don't have to remove the MCU from the circuit to reprogram it when you want to make a change. The two most common are the JTAG and the MicroChip (PIC) ICD. these programming interfaces usually also provide debugging access to the MCU that interfaces into the IDE so you can debug your code easier. Debugging with CPUs is usually much harder. The only commerical option for debugging embedded CPU code are expensive CPU software simulators, or hardware CPU emulators that are plugged into your circut board. The software simulators lack access to the physical hardware, which make have a design flaw and don't necessary help that much.
For a Beginneer I would recommend the Microchip since they are very easy to get up and running. You can download the IDE and student version of thier C compiler for free and run simulations in the IDE simulator without spending a dime on hardware or software. To move to real hardware, you will just need to buy a ICD (about $150 USD) and MCU (from 0.99 USD to $20 USD) depending on size, speed and features, and a breadboard. I would start off with the PIC-18F2525 which comes in a 28 Pin Dip and has 25 I/O pins. it has 16K to 48K or flash and just under 5K of RAM. It also has a built in SPI port, ADCs, PWMs, Timers, Interrupts, etc. You will be able to do a lot with this MCU and I think its only about $5 USD. I think as a beginner you could have a simple system running in less than 8 hours using the PIC 18F MCU. if you go with the Z80 your learning curve and wiring time is substantially larger. I would estimate it will take you 40 to 100 hours.
Concerning your question on how a CPU (or MCU) starts. It uses the reset vector. Every CPU/MCU has a set of vectors, or fixed addresses in the address range that is used to handle special events, such as a reset or an interrupt. When you turn on a CPU, a softstart circuit is used to the Reset pin that triggers it when the power is turned on This is usually a simple RC circuit that holds the Reset pin momentarly low or high so that if forces the CPU/MCU to reset it self upon power on. The Cap takes time to charge and it takes a bit of time for the cap to reach full charge. During this time the RESET pin is held until the cap charges enough to flip the logic state from 0 to 1 or 1 to 0, depending on how you wire the RC to Vcc and gnd.
Most modern CPUs/MCUs have a builtin softstart reset, but CPUs like the Z80 do not. When the RESET signal is triggered the CPU/MCU will load into the Address register the value stored in the Reset vector and start executing the code at that location. The reset vector is usually located in the FLASH\ROM space of the system so that it remains hardcoded. Most CPUs place the reset vector in the high most address space (for typical 8-bit CPUS it in addresses FFFE and FFFF) You have to read the CPU/MCU datasheet to find out where the reset vector is located. FWIW: MCU IDE usually handle the bootstrap code for you, You just specify the start of the program and it set the reset vector for you when you assemble/compile your code in the IDE.
Since your a beginner, I would avoid anything like FPGA devices. Keep it simple! I would recommend a breadboard over a prebuilt learner kit, since you'll quickly want to experiment with devices that aren't built into the kit, Perhaps it will be an SPI temperature sensor, or a ADC converter. You can also add LCDs, 8-Segment LEDs, Serial I/O, varies sensors (temp, voltage, light), Audio, relay, MOSFET switches, etc. A breadboard offers you much more flexiblity than a prefab PCB learning kit.
If you decide to go with a breadboard I would recommend buying the PB-105 from global specialities. I have two of these boards I bought in the late 1980's and still use today. Not cheap but considering that I still use them after more than twenty years, worth the investment
http://www.globalspecialties.com/categories.php?category=2&ct=Premium%20Prototyping%20Breadboards