BrianHG mentioned previously a list of commands/functions that the module would perform and I have a basic checklist myself, like CIRCLE, LINE, FILL, etc and even perhaps basic sprite collision - how would this be implemented and where do I start?
If you want to start with a geometry engine, then:
Begin with writing your code written in basic.
Try to get as many of the functions possible done with the same 2 for loops. IE (x & y) counter.
Try to make the drawng codes always draw from left to right as your ram is 16 bit which may be as many as 16 pixels per word.
Start this far. Make sure the final output resembles addressing screen memory from a pointer, not X&Y pointers.
Then we optimize a bit.
Then we make a verilog equivilant.
Now, because of memory constraints, even though another said no, I do recommend the 'FIFO command plotting drawing port'. This doesn't mean you will stick with this, all it means is that lateron, you will get rid of the first command FIFO and replace it with a programmable loop sequence counter pointing into system graphics memory to fetch the commands automatically (which would probably still feed fill this FIFO anyways
).
Now when I say basic, I mean use the 'Free Basic' compiler which I used for the RS232_Debugger. It takes less than a second to compile and opening a graphics screen is 1 line command which emulating a dumb point placement is also 1 command. You may use a 'C' compiler for windows to create your code, but, for this type of code fiddling around, you wont have such play-ability. Also, you might end up coding too well and create code which will be more difficult to translate to verilog.
For the 'FIFO' port, the basic emulation code should read a file which contains the stream the Z80 would be sending to generate the graphics.
As we go further, you will need to add thing like screen base memory, limit X&Y coordinates so your commands don't end up erasing out of screen memory. You also need to store the pixel bit-plane mode drawing type so the final section knows how to fill the ram.
The final part, written in verilog, pixel writer, takes the plotted memory coordinates and edits the ram's contents. Because of the biplane system, you will need to read-modify-write each byte coming in. In this small sequencer, you will want a smart 1 byte cache which will read a byte if needed, and edit contents. It will not write the byte until it receives the next pixel write since it may be the same memory address as in that cache. If so, it will re-edit the cache. If not, it will write the current cache and read the next pixel byte to be edited. (Now I know we talked about single bit masking for writing bitplane data, however, this may work inside the FPGA, however, if you want external DRAM or SRAM, we can only narrow this down to 8 bits. So might as well put in the effort and do the cache for universal compatibility.)
I think I said enough for you to begin thinking....
As for every control listed above, I would probably engineer 2 banks of controls for 2 the drawing engine since you can display multiple graphics modes simultaneously on the screen, refilling all that data with 2 different window modes going on might offer an advantage. This doesn't mean program it that way to begin with, just make allowances when storing all the control registers to hold the 2 banks. The other choice is 2 distinct drawing engines taking up more FPGA resources.
As for compiling graphics command data. I do have a universal configurable compiler I created for another project which can build 16 bit opcodes functions with data to create your 'FIFO' port file, however, you would be expected to create a Z80 driven controller of your own in the future.
A lot to think about...
-------------------------------------------------------------------------------------------
Sprite collision is handled inside the pixel-mixer just before the palette since part of the code already exists to discern if a pixel is solid or not and which once is drawn on-top of one-another. As for software rendered sprites in the Geometry engine, that will be located in the 'pixel writer which takes the plotted memory coordinates and edits the ram's contents.' Another reason the read-modify-write pixel drawing command has additional use above simple blind writes into memory which contains 'bit' for 'bit' write enables to access individual bit-planes.