Latest 2-layer PCB design with changes made to position of the DAC, number of colour bits to the DAC, etc. (I've dropped the two LSBs from each channel).
Has taken me an hour or two to get component positioning/orientation in a way that allows a successful auto-route. Still had to wire two of the power feeds to the FPGA manually.
I haven't bothered working on ground planes yet - I thought I'd seek some feedback on the positioning of the DAC and those pull-down resistors for R, G and B before I start stressing about ground plane placement.
Ohh boy, are you auto routing and auto placing this?
Move the VGA connector to the right a little.
Make the DAC at 90 degrees so that the analog out faces the VGA plug.
Make the DAC's 75 ohm resistors and dac wiring straight to the VGA plug. Make the proper VGA plug's Analog R&G&B GND return pin paths feed on the top through/wired to the other side of the 75 ohms and to the DAC's R&G&B negative signals. Place as many of the decoupling caps on top to the sides of these signals. Also, you should be using 0603 parts, or if your hands are steady enough, go for 0402 for these DAC components so everything is in a straight line.
Under the dac, bottom layer, a fill GND should be used throughout. Use 2 adjacent VIAs for every GND connection to the top layer.
For a 2 layer board, this should make a really good analog signal for your frequency range.
Remember, match red's return GND signal path with red's 75 ohm resistor, then to the dac's inverted red GNS input. You do not want a signal reflection crosstalk across your video outputs since you didn't match GNDs. The GND plane feeding the VGA connector's GND pins will be good enough for your bandwidth.