Go for the cheapest version of the ADV7125 DAC, the 50MHz version, though the TI THS8136PHP is cheaper, faster and higher quality, but you need to run the IO and a bank on the FPGA at 1.8v, though, no voltage reference.
https://www.mouser.com/ProductDetail/Texas-Instruments/THS8136PHP?qs=sGAEpiMZZMswix2y39yldVRXhfgYHmM93ATDG4%252BVYYs%3Dvs
https://www.mouser.com/ProductDetail/Analog-Devices/ADV7125KSTZ50?qs=sGAEpiMZZMswix2y39yldfRSUiTQKynFVgceQfpCL2c%3DWhy are you clocking the DAC from your oscillator. You should be using an output from the FPGA, preferably, one of the dedicated PLL output pins. Use the second PLL CLK output if you have the reference oscillator on CLK1 input.
Why haven't you wired the 'blank' signal on the DAC to the video enable out. It would help and also is necessary if you change the DAC for the DVI transmitter.
Things you can use the FPGA dedicated CLK inputs for:
clock oscillator <must be on a dedicated clock input for PLL to function at high quality>
z80 clk input
Z80 buss controls, IE RD WR MREQ M1 INTA, IORQ, so long as all the buss controls are on the dedicated clock inputs.
reset
serial port RXD
I'm trusting you FPGA IO pins and part number. Remember the EP4CE10E22C8N may have a few different IO pins than the EP4CE6E22C8N, typically a few less IOs changed into VCCint and GNDint.
What about the unused IOs on the SD-Card reader.
For the next revision, note that going straight to BGA means you cannot make a mistakes as you wont have access to pins to lift.
What size are your VIAs on your PCB. are you sure any fab house can handle that in cheap PCB mode? I know it's not a problem with higher quality expensive PCBs and if you are already using fine VIAs, you might as well go to a 4 layer PCB as it will be only a few dollars more from some China PCB manufacturers.
Are your power traces thin little traces like the data lines?
Your AMS1117-1.2 is ok for the EP4CE10E22C8N and you should expect it to get warm since it is linear with an almost 4v drop and a potential 0.25w VCCint load when switching your core to 15 layers. This means around 208ma draw on a 800ma regulator. With a 3.8v drop from the 5v supply = around 0.8 watts of dissipated waste heat. (I'm assuming worst case with your current design).
Where is your 3.3v regulator?
You are going to get noise/ripples on you analog output with that ground layout and positioning of resistors.
Also, ADV7125 has an internal voltage reference mode.
With the ADV7125, also make sure the analog 5V is well filtered and isolated from the rest of your 5v design.