I got half way through reverse engineering the N64 PCB in order to make my own portable version. This information may be useful to some people (not my own):
On the early N64's (NUS-CPU-01 to NUS-CPU-04) the DAC and Video Encoder chip are separate. the DAC is labelled 'VDC-NUS' (Or 'VDC-NUS A') and takes the digital video data directly from the RCP and outputs RGB. This RGB is then sent to the separate 'ENC-NUS' video encoder chip wheres its encoded into Composite and S-Video. It is possible to perform an RGB mod on these consoles because you can tap the RGB signals from inbetween these two chips.
The AVDC-NUS (NUS-CPU-05) combines the DAC and the Video Encoder together into one chip (It actually also combines the Audio DAC aswell) Which means that it converts to RGB internally and then encodes directly into Composite and S-Video - so there is no way for us to tap into the RGB signals anymore
The Video/Audio chip progression seems to go like this:-
NUS-CPU-01 to NUS-CPU-04 (Early-ish NTSC N64's)
VDC-NUS - Video DAC
ENC NUS - Video Encoder
BU9480F - Audio DAC
-------------------------------------------------------------------------
NUS-CPU(P)-01 (Early PAL N64's)
DENC-NUS - Combined Video DAC and Video Encoder
(The DENC-NUS chip was only ever used in PAL consoles)
BU9480F - Audio DAC
-------------------------------------------------------------------------
NUS-CPU-05 (Mid NTSC N64's)
AVDC-NUS - Combines Video DAC, Video Encoder and Audio DAC into one chip
(The AVDC-NUS was only ever used in NTSC consoles)
-------------------------------------------------------------------------
NUS-CPU-05-1 onwards (Late NTSC N64's) and NUS-CPU(P)-02 (Late PAL N64's)
MAV-NUS - Combines Video DAC, Video Encoder and Audio DAC into one chip
-------------------------------------------------------------------------
The AVDC-NUS and MAV-NUS chips seem to share exactly the same pinout. It is my guess that they are both almost identical and only differ in that AVDC-NUS chip can only use NTSC color encoding (Just a guess) whereas the MAV-NUS can do both PAL and NTSC color encoding by making one pin high or low.