Choice of microcontroller or FPGA for DIY "Ambilight"

[NiHaoMike]

I'm building a DIY "Ambilight" and although I'm aware of the implementation that uses a Raspberry Pi, I would like to try doing it in "bare metal". I have a downscaling splitter to tap off a 1080p version of the 4K signal driving the display, then either a HDMI to VGA or HDMI to Svideo converter to convert that into something easy to work with using low cost DIY electronics.

I'm leaning towards the Svideo option with a TVP5150A or similar to interpret it since that gives a data rate that could possibly be processed by a microcontroller. Output will be WS2812 LEDs, about 60 along the top edge and 35 on each side.

I haven't looked very far into the exact algorithms used for the Ambilight effect, but what sort of processing power should I be looking at if I want the LEDs to update at 30Hz? I'm most familiar with PIC so would a dsPIC or PIC32 be able to do it? Or should I step up to a STM32F4 or even a FPGA?

[Berni]

Well video input buses tend to be a rarity on MCUs. Usually they are found on bigger Linux capable chips from TI,NXP etc... So to look at 1080p video you probably would be best off with a FPGA.

But for what you are doing id assume you are mostly just interested in finding the average color in sections of the display. This means you could add glue logic that clock divides the video bus out of a HDMI receiver chip so that it only clocks every 32th pixel cutting down the massive 400MB/s bandwith down to only about 10MB/s

But if you have VGA that makes things even simpler. There the red green blue components of the image are carried in analog form over 3 separate wires. There is no clock to worry about keeping up with the data rate. In fact if you simply terminated the lines into 75Ohm (VGA needs that) and fed the signal into a RC low pass filter you would get roughly the average color of the whole screen as a analog voltage. To do more advanced things you could use a less restrictive low pass filter to only smooth out the time it takes VGA to draw 1/4 of one video row and then feed that into the slow ADC on a MCU. If you time sync your MCU to the Hsync and Vsync signals in VGA you can then know exactly what part of the frame is currently being shown and take it in.

[JPortici]

Output will be WS2812 LEDs, about 60 along the top edge and 35 on each side.

if you're willing to change the LEDs, a guy in the microchip forum came up with an hardware driver for WS2412B using the CLC module in the newest dsPIC33C it can run with DMA so zero cpu time is spent on drawing the screen
https://www.microchip.com/forums/m1083323.aspx

[Berni]

Oh neat. Finally i see a cool use for those "baby CPLD blocks" that Microchip sticks onto there MCUs.

I haven't worked with these addressable RGB LEDs all that much, but it does look like its a case of a slow communication bus that makes them a pain in the ass to drive in large numbers, not the actual amount of data. So being able to offload an entire string update to the DMA is a huge help.

[james_s]

Somewhere I saw a project to do just that with an FPGA connected to a HDMI source driving an array of addressable RGB LEDs. I don't recall where I saw it though.

[hamster_nz]

If you did want to go FPGA, then this is the level of board you would need.

https://www.scarabhardware.com/minispartan6/

Updating the LEDs isn't the problem, it is the HDMI clock rates 1485 Mb/s , vs the FPGA SERDES performance.

You will also need a splitter that strips HDCP from the HDMI stream, if you intend to use it with copy protected sources....

[NiHaoMike]

I'm using a cheap HDMI to Svideo converter specifically so that I can take advantage of cheap ASICs to downsample with and take care of supporting all the different formats. Then I'm thinking I could just use a TVP5150A or similar to convert that into a parallel stream that could be interpreted by the parallel slave port on some microcontrollers.

[Berni]

Yes exactly, its best to just use a whatever to RGB parallel converter chip to do the high speed conversion for you.

But do be aware that  HDMI 1080p 60Hz uses a pixel clock of about 130 MHz at 24bit colors. So that better be a blazing fast parallel slave port to keep up. When you consider each pixel is 3 bytes you get 390 MB/s. hence why my suggestion to use external logic to skip pixels.

[tsman]

If you did want to go FPGA, then this is the level of board you would need.

https://www.scarabhardware.com/minispartan6/

I found the old blog post I remember from before. Somebody already made an Ambilight clone based on that board. They say the FPGA limits them to 720p operation though. EDID, CEC and HDCP aren't implemented.

[NiHaoMike]

But do be aware that  HDMI 1080p 60Hz uses a pixel clock of about 130 MHz at 24bit colors. So that better be a blazing fast parallel slave port to keep up. When you consider each pixel is 3 bytes you get 390 MB/s. hence why my suggestion to use external logic to skip pixels.

Hence the route of using a HDMI to Svideo converter and then some video ADC, to take the data rate down to a much more manageable 27MHz on an 8 bit bus. (The Raspberry Pi based Ambilight also does that to avoid having to get an expensive HDMI capture box.)

I found the old blog post I remember from before. Somebody already made an Ambilight clone based on that board. They say the FPGA limits them to 720p operation though.

1080p support is a must have, so that means the only affordable FPGA able to directly interface with the HDMI would be an Artix 7, quite overkill for this project. Additionally, the existence of affordable splitters with downscaling (to get 1080p from 4K) is what made the project feasible at all - other ways of dealing with that firehose of data are a lot more expensive.

[lucazader]

If you already have it split into a parallel bus then the DCMI interface on some of the stm32 processors might be useable..?
I have never really done anything with this, but just thought i'd plonk it here with an app note in case it is of use..
https://www.st.com/resource/en/application_note/dm00373474.pdf

[NiHaoMike]

if you're willing to change the LEDs, a guy in the microchip forum came up with an hardware driver for WS2412B using the CLC module in the newest dsPIC33C it can run with DMA so zero cpu time is spent on drawing the screen
https://www.microchip.com/forums/m1083323.aspx

Haven't found much info on WS2412B LEDs (appear to be a higher voltage version for more brightness), but the protocol looks similar enough that the idea can easily be adapted for the WS2812 strips. I already have some ideas on how to clock out the data using two channels of PWM, a SPI peripheral in slave mode, and a little bit of external logic, so the CLC is not a hard requirement. I'm focusing on the video capture for now since that's the harder problem to solve. (Worst case, I wouldn't mind having to use a second microcontroller just for driving the LEDs, although that appears to not be necessary.)

[Berni]

If you are doing for Svideo then you might as well just use Composite instead. The only difference in Svideo is that the chroma and luminance signal is carried on different wires. The chroma signal is still phase and amplitude modulated onto a 3.5MHz carrier, in composite this carrier is simply mixed into the lower frequency luminance signal. So by using Svideo you just make more work for you by needing 2 ADCs that need to run just as fast as 1 ADC used for composite video.

So you do still need a ADC runni at least 10 MSPS to capture a NTSC signal. And make essentially a software defined radio to demodulate the chroma signal out of it. You also need to extract the synchronization signals from it to know where you are on the screen and also synchronize yourself to the phase of the color burst at the beginning of each line to get consistent color hue in your colors. And once you have captured the information you have to then convert the HCL color space into the RGB color space.

In contrast VGA has the synchronization signals carried on separate pins so they can go right into a interrupt pin on a MCU and the analog R,G and B signals have no modulation on them, just feed it straight into a ADC and grab the RGB value of the current pixel. VGA running at 1080p still needs a 130 MSPS ADC to fully capture, but because its just analog you can simply put the RGB signal trough a low pass filter to average about 100 pixels together and then feed that into a 1.3 MSPS ADC. This then gets into the territory of ADCs that are built into MCUs. So this makes it a 2 chip solution of HDMI to VGA converter chip and MCU driving the LEDs.

[NiHaoMike]

Video ADCs like the TVP5150A have the color decoding built in.

[asmi]

Lol what's the point converting digital signal to analog only to be converted back to digital?
Use FPGA to downscale FullHD to whatever resolution you want while still keeping it all-digital. The simplest 4x averaging (2x2 square) will result in 960x540 frame, if that's still too much - do it again to get  480x270 frame. Or you can use non-cardinal dividers for more flexibility, but slightly higher complexity. If you don't feel like coding this in HDL, you can use HLS tools to generate the code for you.

[janoc]

This has been done quite a few times:

http://zerocharactersleft.blogspot.com/2015/04/diy-fpga-based-hdmi-ambient-lighting.html

http://hacks.esar.org.uk/hdmi-light-v2/

https://hackaday.io/project/694-ambicontroller

And probably a few more. You won't find one using a microcontroller and a HDMI/DisplayPort style input for the reasons described by others above.

The implementations that use a micro/Raspberry Pi rely on a modified media player that extracts the necessary color information from the image and feeds it to the device e.g. over USB or GPIO (Raspberry Pi). As a consequence, such implementation doesn't work with anything else but that modified media player program.

[NiHaoMike]

Hyperion supports using USB capture boxes, most users opting for the HDMI to Svideo converter since that plus a Svideo capture box ends up way cheaper than any HDMI capture box. But I would like to try an implementation that does not require an OS.

Lol what's the point converting digital signal to analog only to be converted back to digital?
Use FPGA to downscale FullHD to whatever resolution you want while still keeping it all-digital. The simplest 4x averaging (2x2 square) will result in 960x540 frame, if that's still too much - do it again to get  480x270 frame. Or you can use non-cardinal dividers for more flexibility, but slightly higher complexity. If you don't feel like coding this in HDL, you can use HLS tools to generate the code for you.

The cheapest FPGA that can handle 1080p HDMI directly is the Artix 7, the cheapest board with that I can find is about $60. (Using a universal HDMI input board allows the use of a much cheaper FPGA but that board itself costs about $20.) The output is such low resolution that even Svideo is overkill. A HDMI to Svideo converter is about $8 and Svideo ADCs are super cheap.

[asmi]

The cheapest FPGA that can handle 1080p HDMI directly is the Artix 7, the cheapest board with that I can find is about $60. (Using a universal HDMI input board allows the use of a much cheaper FPGA but that board itself costs about $20.) The output is such low resolution that even Svideo is overkill. A HDMI to Svideo converter is about $8 and Svideo ADCs are super cheap.

But in case of FPGA you can implement LED control logic directly inside that FPGA and will not need any external components. This will also allow achieving very high refresh rates and/or very sophisticated control algorithms. In any case needless conversions are just wrong thing to do IMHO regardless of price.

[NiHaoMike]

I think someone tried a HDMI capture box with the Raspberry Pi Ambilight and found it had basically no advantage over the HDMI to Svideo converter + Svideo capture box solution for that use. As for latency, the Raspberry Pi is going to be at a big disadvantage - a microcontroller solution should have no problem getting less than one frame of latency. At this point, it looks like my best bet is to start with a high end dsPIC or STM32.

I found this device that claims to do it with some M4 microcontroller, sadly not open source.
https://www.scimolight.de/index-en.html

[james_s]

I have not tried this, but you may be able to get away without capturing anywhere near the whole image. Ambilight is very low resolution, effectively just a handful of "pixels" and all that really matters is the edges of the picture. You could probably do some sampling at approximate locations in the picture and average the color value. I doubt you'd ever see a difference between grabbing a full 1080p frame in one shot over HDMI or capturing at a much lower rate over composite video at the other extreme. A technique as used by the old sampling oscilloscopes might let you get by with much lower performance hardware, video is roughly a repetitive signal, changes in the image occurring at a far slower rate than the pixel clock.

[NiHaoMike]

Doing equivalent time averaging over multiple frames would not work for more than a few frames as it would cause ghosting. But it is correct that the data near the middle can simply be ignored - that's time better spent processing the samples previously captured.

One advantage of using Svideo instead of composite is the possibility of adding some analog filtering to the Y (intensity) signal in order to be able to skip some of the Y samples. That won't work on the C (color) signal since it's modulated on a carrier. I have looked at HDMI to YPbPr converters but the cheap ones seem to just be HDMI to RGB converters with some additional circuits to convert the RGB to YPbPr, so no downscaling that I'm looking for.

[Berni]

Yes but the point of analog RGB is that no modulation is used, so that your downsampling can simply be done with a analog low pass filter. But as far as i know YPbPr used in component video are also non modulated just like RGB so you could also use the same analog low pass filter downsampling trick. Tho i do admit its a bit of a hack to use analog video just to downsample it and then use a ADC on it right after. But it is a low cost and low effort solution.

The more proper solution would involve a HDMI to DPI RGB converter chip feeding into a cheap FPGA that then drives the LED strings directly. Its a bit more expensive and involves annoying FPGA software development but it would give optimal performance.

Yes you can feed HDMI directly into a FPGA but the datarate is about 1.5Gbit per lane so that means a pretty expensive FPGA that has some fast enough pins or transceivers to handle that. So its likely cheaper to simply add a HDMI to DSI RGB converter chip and use the cheapest reasonable FPGA you can find.

[NiHaoMike]

If I were to target mass production (I'm not), most likely the cheapest way to make it work would be one of those HDMI mini monitor chipsets that can downscale very low, plus a Lattice FPGA. But for a hobbyist, it's basically impossible to get the former already mounted on a board for a reasonable price.

[Berni]

I have a 10 inch 1920x1200 touchscreen monitor with one of these boards in it, but because i wanted to modify mine i went and bought a replacement board for it to be safe. Manged to get it for 6 USD on aliexpress i think. Its a board that takes in HDMI,VGA, Composite etc and outputs LVDS at the correct resolution to drive the LCD panel. It also provides menus for setting stuff, generates the power rails from 12V input and drives stereo speakers.

It uses a chinese chip that all of these boards use and you can buy only the chip if you look around for long enough. Tho it does also need firmware to go along with it, that's what configures it how to drive the LCD panel correctly and what resolutions to use.

[james_s]

I hadn't even thought of those. Some of those small TFT driver boards are not even LVDS, their output is simple 3.3V logic, digital RGB lines, hsync, vsync and a dotclock. Piece of cake to interface to an FPGA.