Purely for video, using a VGA-HDMI adapter plus a HDMI over fiber transmitter, such as this: https://www.amazon.com/LornCeng-Extension-Uncompressed-Transmission-Singlemode/dp/B0BN536RN1 , would do for probably much cheaper than my above link.
There are actually "HDMI fiber cables" that are cheaper than that, I have an 8m one, they sell 30m ones.
Rolling your own is an interesting project but it's not happening under $200 unless analog, and even then...
For starters if you go digital you're looking at a 1.13 Gb/s stream unless you want to compromise on quality. That will need a mid-level FPGA with gigabit serdes'es to format the streams, maybe an ECP5 can do it. If you implement 8b10b coding this becomes 1.41 Gb/s not counting overhead.
So count in two dev boards with enough differential I/Os for the ADCs, DACs and the transceivers, maybe $150 a pop? If they are available, that is. Pray you don't need any terminating resistors for your differential connections to get the particular kind of signaling the ~1.5 Gbps transceivers want, otherwise you'll have to spin a board.
To digitize the VGA you need a fast ADC or maybe 3 and an analog front-end and the layout can't bee too trash or you'll have disgusting ringing effects around edges. At the other end you need 3 DACs and also analog front-ends for each channel and the same layout comment applies. The UART is easy compared to the rest, you'll just slip it in the stream.
I don't even know how much the fiber itself costs.
Cheapest transceiver modules on Mouser start at 15 bucks. Unless someone finds a full duplex one that'll be 4 modules at $60. Not counting time obviously.
If you can use multiple fibers and the line isn't too long the analog route seems more feasible to me.
The cheaper transmitter modules use LEDs, those are quoted as having 10ns rise/fall times but that's for full scale, so the bandwidth should be enough for an XGA signal. With proper readout PIN photodiodes can be pretty fast. So you have a transconductance amplifier to drive the LED and a transimpedance amplifier at the other end and some level shifting, but it should go down to DC. I don't even think you need a chopper anymore. You may have to include some trimmers to adjust the signal levels so that the sync pulses have the proper levels. You'll have a little bit of skew between channels so the edges may show some slight fuzzy coloring, but if you cut your fibers to within 10 cm this should be minimal.
If you go monochrome you can reduce the video signal to a single fiber.
Duplex on a fiber in analog would require a dichroic beam splitter at each end, you could use RGB cubes used in video projectors and RGB LEDs as a poor man's WDM, you do have two optical tables right?
![Smiley :)](https://www.eevblog.com/forum/Smileys/default/smiley.gif)
Or maybe an optical circulator? Digital you can do TDM.
On second thought, since the return channel is only used for 1 Mbaud UART, maybe a single beam splitter can do it, possibly by adjusting the timing of the UART signal w.r.t. the video signal (transmit during blanking kind of thing), this could be done with a small FGPA or maybe even a µC.
Another avenue could be something that converts VGA to RF, then you RF to fiber and back.
Would be nice if you gave us some more details about the constraints!