Cheap 250Mbs Link between Boads

[ali_asadzadeh]

Hi,
I want to design a new project for the customer, I want to send Data between boards with the least amount of wires, there are around 200 boards and they should send their data to the next Board, and the data rate is around 250Mbs, also the boards are around 50-60cm away from each other,
I was thinking of using RS485, But the data Rate is limited to 50Mb, so I was searching other options, Ethernet is out of question because of the cost, So I thought maybe Select IO from spartan 6 parts Like the LX9 part can come handy, is there a way to make a UART liked interface @250Mbs, what about the Physical layer? should I go with LVDS? any Ideas or hints would help a lot.

[laugensalm]

You might want to forget any asynchronous solution and use a 10b8b coding on the physical layer, as DVI does. If you're not stuck with Xilinx, you might want to look at the gear boxes in the mach XO[2-3] FPGAs, there are also neat ref designs that work over a Ethernet cable (without the Ethernet protocol). I've also abused DVI-alike protocols over SATA cables and HDMI (off standard) for such hacks with no issues. And not to forget some real time issues with deviating clocks on sender/receiver, but I assume you've thought about that already.

[glenenglish]

2 x DDR LVDS pairs
125 MHz clock and  data.  will easily make it between two boards on twisted pair. Use two pairs out of an RJ45.....

any fpga would do that.

why would you even consider an Spartan6 , unsupported tools ?? , unsupported silicon
You are designing a product for a customer ! not a doorbell for your house...
There is plenty of QFP  FPGA around.... Trion T20  .... , Lattice XO2,

[BrianHG]

Even every single slowest worst lowest power Altera FPGAs can achieve those bit-rates on every pair of IO pins.

[glenenglish]

And you'll need to use shielded Cat6 cable to meet EMI. (and EMC), with shields connected each end.
Like, you will be very unpopular (and in alot of trouble) if your clock/data radiates enough to cause trouble in the aviation band, or public service radio bands

[BrianHG]

And you'll need to use shielded Cat6 cable to meet EMI. (and EMC), with shields connected each end.
Like, you will be very unpopular (and in alot of trouble) if your clock/data radiates enough to cause trouble in the aviation band, or public service radio bands

There is always fiber optic.... LOL, talk about overkill.

[langwadt]

And you'll need to use shielded Cat6 cable to meet EMI. (and EMC), with shields connected each end.
Like, you will be very unpopular (and in alot of trouble) if your clock/data radiates enough to cause trouble in the aviation band, or public service radio bands

then how does GB ethernet do without?

[SiliconWizard]

Cost of Ethernet? How so? Gigabit Ethernet PHY+mag jack can be pretty cheap. We don't know much about your system, does each board already have a FPGA? A MCU? A SoC?
Even a CH32V307 has a Gigabit MAC, so with an external PHY + mag jack, you'd get your (robust) data link for a couple extra bucks per board (and the CH32V307 also has a FMC that can be used to implement a FIFO to some other MCU or SoC. Of course other more powerful MCUs can support that too.

Otherwise yes, a modest FPGA and a few IOs set up in LVDS mode.

[BrianHG]

And you'll need to use shielded Cat6 cable to meet EMI. (and EMC), with shields connected each end.
Like, you will be very unpopular (and in alot of trouble) if your clock/data radiates enough to cause trouble in the aviation band, or public service radio bands

then how does GB ethernet do without?

It's a balanced signal with no continuous carrier clock, it's seen as weak spread-spectrum noise.

[langwadt]

And you'll need to use shielded Cat6 cable to meet EMI. (and EMC), with shields connected each end.
Like, you will be very unpopular (and in alot of trouble) if your clock/data radiates enough to cause trouble in the aviation band, or public service radio bands

then how does GB ethernet do without?

It's a balanced signal with no continuous carrier clock, it's seen as weak spread-spectrum noise.

and so can LVDS be ....

[ali_asadzadeh]

Cost of Ethernet? How so? Gigabit Ethernet PHY+mag jack can be pretty cheap. We don't know much about your system, does each board already have a FPGA? A MCU? A SoC?
Even a CH32V307 has a Gigabit MAC, so with an external PHY + mag jack, you'd get your (robust) data link for a couple extra bucks per board (and the CH32V307 also has a FMC that can be used to implement a FIFO to some other MCU or SoC. Of course other more powerful MCUs can support that too.

Otherwise yes, a modest FPGA and a few IOs set up in LVDS mode.

I was considering CH32V307, But Since I need two such ports, One receiver that gets it's data from last board in the chain and one transmitter that would add it's own data to the received one and resend it to next stage so I moved on it!

There is always fiber optic.... LOL, talk about overkill.

I have seen some lower speed parts,
https://www.broadcom.com/products/fiber-optic-modules-components/industrial/industrial-control-general-purpose/650nm
But since I need the system to be as cheap as possible, I forget about them.

2 x DDR LVDS pairs
125 MHz clock and  data.  will easily make it between two boards on twisted pair. Use two pairs out of an RJ45.....

any fpga would do that.

Do you suggest any app note, code or open source project for that or You mean I try doing something like a SPI based system? the clock and Data! this seems not a bad option, But I prefer to use only two pairs of wires to make it even simpler.

You might want to forget any asynchronous solution and use a 10b8b coding on the physical layer, as DVI does. If you're not stuck with Xilinx, you might want to look at the gear boxes in the mach XO[2-3] FPGAs, there are also neat ref designs that work over a Ethernet cable (without the Ethernet protocol). I've also abused DVI-alike protocols over SATA cables and HDMI (off standard) for such hacks with no issues. And not to forget some real time issues with deviating clocks on sender/receiver, but I assume you've thought about that already.

I'm not stuck to AMD, I'm open to options, But I like AMD and Gowin parts, I have not done 10b8b coding before, So I'm a nobe in this area, any open source project or app note do you suggest?

[langwadt]

2 x DDR LVDS pairs
125 MHz clock and  data.  will easily make it between two boards on twisted pair. Use two pairs out of an RJ45.....

any fpga would do that.

Do you suggest any app note, code or open source project for that or You mean I try doing something like a SPI based system? the clock and Data! this seems not a bad option, But I prefer to use only two pairs of wires to make it even simpler.

it is two pairs, one pair for data, one pair for clock

you could probably do a 250Mbaud uart over a single LVDS pair, but clock recovery and drift could get interesting. Much easier if you have the clock

[glenenglish]

The gigabit ethernet is a controlled , band limited waveform.
it's not super fast hard edges like LVDS.
the LVDS drivers will do at 1000 MHz.... and with rise times to match.

even band limited, the clock on LVDS would need real shielding
HOWEVER, at 125 MHz LVDS (250 Mbps) , the FPGA could easily operate its own clock recovery, so you would not need to transmit clock, just data.....  so, just shielded CAT-6.
You could do DPLL clock recovery . Or you could transmit manchester.... but manchester would generate a clock line which you are trying to avoid ....
Glen.
RF engineer.

[mariush]

Some clarification would be helpful.... do you need that 250mbps because data has to reach the last device in the series in some reasonable time ...

I have a suspicion that you're making some RGB led tiles or something like that, for a big display, and you're trying to push 2k-4k pixels x a few hundred lines worth of information so many times a second.

Not sure why you think ethernet is too expensive and complicated... you can get used 48 port 100 mbps ethernet switches for something like 30-50$ , you'd only need 4 of those in a stack to control your 200 boards and you'd also be able to use ready made patch cables to limit the problems caused by making manual cables.

ethernet chips are like 1-1.5$ if you get 100+, one per card.

I'm assuming that your bandwidth would be much less than 100 mbps if you don't have to pass the data to the next board and so on... you could have a beefier fpga split the data into chunks for each board and do broadcast or push the data through udp and the switches would send only the data you want to individual boards.

At 100 mbps you could have the unused pairs repurposed for sending power to each board, like in passive poe  (one pair 12v , one ground, something like that)

[ali_asadzadeh]

thanks for the feedbacks,

Some clarification would be helpful.... do you need that 250mbps because data has to reach the last device in the series in some reasonable time ...

Yes, I need to reach this, Because every board should add it's data to the last part of the stream, and as the stream passes from the first board to the last board, it would get bigger, and they should do it in a limited time, since new data would arrive at fixed time, so The 250mbps is a must to meet all the required timings.

Not sure why you think ethernet is too expensive and complicated... you can get used 48 port 100 mbps ethernet switches for something like 30-50$ , you'd only need 4 of those in a stack to control your 200 boards and you'd also be able to use ready made patch cables to limit the problems caused by making manual cables.

Note that the space the boars suppose to have is limited, so I can not use ethernet cable for every board to a switch then get the final result, also money wise, it does not make sense either, since I need to use lot's of wire to run things.

HOWEVER, at 125 MHz LVDS (250 Mbps) , the FPGA could easily operate its own clock recovery, so you would not need to transmit clock, just data.....  so, just shielded CAT-6.
You could do DPLL clock recovery . Or you could transmit manchester.... but manchester would generate a clock line which you are trying to avoid ....
Glen.

Would you please explain more on how to recover clock, since I have not done related projects and I would be a noob there.

you could probably do a 250Mbaud uart over a single LVDS pair, but clock recovery and drift could get interesting. Much easier if you have the clock

I do not know how to do a 250mbps baud uart, since at least I need 8x clock, inside the FPGA, and the clock usually can not reach more than 100Mhz for low cost FPGA's, do you suggest any tricks?

[BrianHG]

I do not know how to do a 250mbps baud uart, since at least I need 8x clock, inside the FPGA, and the clock usually can not reach more than 100Mhz for low cost FPGA's, do you suggest any tricks?

Most FPGA vendors have their own exclusive IP SERDES functions to handle all of this for you, including embedding and recovering clocks.  At least Altera and Xilinx do so.  It is not a thing you will be engineering or coding on your own unless you are making your own ASIC.

You have to go through each vendor's data sheets, read their LVDS transceiver's electrical section for wiring and loading the IO and then read and select the SERDES IP primitive which is right for your clocking and coding requirements.

Otherwise, we are at a point where you are asking us to do your homework/research and funnel you a bunch of vendor's .pdfs and page numbers.


You can also use single ended coax, like BNC or smaller RF connectors at 250mb for a good 100cm without cable-length equalization amplification as these solutions with proper termination will achieve above 3gbps.

Don't forget to diode protect your ports.

[langwadt]

you could probably do a 250Mbaud uart over a single LVDS pair, but clock recovery and drift could get interesting. Much easier if you have the clock

I do not know how to do a 250mbps baud uart, since at least I need 8x clock, inside the FPGA, and the clock usually can not reach more than 100Mhz for low cost FPGA's, do you suggest any tricks?

oldschool 12Mbit USB got away with 4x sampling, either with a 48MHz SDR or 24MHz DDR, A spartan6 can definitely do much more then 100MHz DDR, though 500MHz DDR might b pushing it.

how are you going to wire it? why is limiting the number of pairs so important?  repurpose cables from something like HDMI, DP, USB-C, hell even RPi camera/display flatflex, and you'll get plenty of pairs cheap and easy

[glenenglish]

I think at this point, you probably need to read some books, do some tutorials,

maybe try this project you want at say 1/4 the bit rate here, say 62.5 Mbps, and learn about the process, then try 250 Mbps. This way you will learn and conquer each speed raising step and the associated challenges.

and even $4 Trion T8  FPGAs (can get in QFP) can reach 300 MHz clock and 400 Mbps LVDS ....and 250 Mbps  DDR  only needs 125 MHz clock at the edge, and internally, 8 bits wide, 31.25 MHz clock.

I know your blog signature says "I'm a Digital Expert from 8-bits to 64-bits" but, if this is a commercial project, I would suggest you employ a consultant to get you up to speed .

[mtwieg]

Sounds like SATA cables are a good fit. Two shielded differential pairs rated for >1Gbps, well suited to LVDS.  Connectors and cables are quite cheap IMO. Lots of cables available in the 15-100cm range.

[glenenglish]

sata cables - that's a good idea.
yeah clock recovery for a 125 MHz clock using a DPLL should be a cinch for an experienced  fpga person.

otherwise.....1 gig ethernet could do it. or serdes. . the op can buy  5:1 serdes gigabit stream chips from TI  . (LVDS to serdes) . The are all higher latency, intensive overkill options.

but, I think given the op is a beginner, data and clock on two LVDS  shielded pairs would be the easiest. IE 1 x sata cable - cheap and cheap connectors as you pointed out mtwieg.  Use a Trion T8 (6 lvds in each direction per QFP, $5) .  Obviously bigger FPGA if more work or memory is required in the FPGA.
 

[langwadt]

sata cables - that's a good idea.
yeah clock recovery for a 125 MHz clock using a DPLL should be a cinch for an experienced  fpga person.

otherwise.....1 gig ethernet could do it. or serdes. . the op can buy  5:1 serdes gigabit stream chips from TI  . (LVDS to serdes) . The are all higher latency, intensive overkill options.

but, I think given the op is a beginner, data and clock on two LVDS  shielded pairs would be the easiest. IE 1 x sata cable - cheap and cheap connectors as you pointed out mtwieg.  Use a Trion T8 (6 lvds in each direction per QFP, $5) .  Obviously bigger FPGA if more work or memory is required in the FPGA.

I think if you are really penny pinching you could do SATA as edge connectors in the PCB, though I think you need slightly thinner than the usual 1.6mm PCB

[ali_asadzadeh]

Most FPGA vendors have their own exclusive IP SERDES functions to handle all of this for you, including embedding and recovering clocks.  At least Altera and Xilinx do so.  It is not a thing you will be engineering or coding on your own unless you are making your own ASIC.

I know I can use SERDES  to solve it all, the problem is the cost, I think I would go for XC6SLX9-2TQG144I part from AMD, since it's one of the cheapest Industrial parts available, unless you suggest something else with similar or better prices, (they are around 8$ @ 100 units)
https://www.lcsc.com/product-detail/Programmable-Logic-Device-CPLDs-FPGAs_AMD-XILINX-XC6SLX9-2TQG144I_C415799.html

Sounds like SATA cables are a good fit. Two shielded differential pairs rated for >1Gbps, well suited to LVDS.  Connectors and cables are quite cheap IMO. Lots of cables available in the 15-100cm range.

Thanks for the suggestion, It seems a very good choice indeed.

[glenenglish]

why do you need industrial parts ?
sure you can use an old  obsolete part in a new customer product... but really, should you ?
(where there are plenty of good  an dmodern $4 parts like I gave examples of in QFP that are much faster than spartan 6 )

serdes is a ridiculous overkill. and might not run slow enough to work for you down your available cable....

[langwadt]

why do you need industrial parts ?
sure you can use an old  obsolete part in a new customer product... but really, should you ?
(where there are plenty of good  an dmodern $4 parts like I gave examples of in QFP that are much faster than spartan 6 )

serdes is a ridiculous overkill. and might not run slow enough to work for you down your available cable....

spartan6 has suspport until atleast 2030

serdes is just  a fancy shift register, you can run as slow as you like

[BrianHG]

Most FPGA vendors have their own exclusive IP SERDES functions to handle all of this for you, including embedding and recovering clocks.  At least Altera and Xilinx do so.  It is not a thing you will be engineering or coding on your own unless you are making your own ASIC.

I know I can use SERDES  to solve it all, the problem is the cost, I think I would go for XC6SLX9-2TQG144I part from AMD, since it's one of the cheapest Industrial parts available, unless you suggest something else with similar or better prices, (they are around 8$ @ 100 units)
https://www.lcsc.com/product-detail/Programmable-Logic-Device-CPLDs-FPGAs_AMD-XILINX-XC6SLX9-2TQG144I_C415799.html

Sounds like SATA cables are a good fit. Two shielded differential pairs rated for >1Gbps, well suited to LVDS.  Connectors and cables are quite cheap IMO. Lots of cables available in the 15-100cm range.

Thanks for the suggestion, It seems a very good choice indeed.

All Xilinx, Lattice, Altera parts have built in SERDES capabilities in every one of their parts on almost on every one of their IOs.  Read the data sheets.  You do not need the expensive ones with the dedicated multi-gigabit dedicated SERDES pins.  The difference is that the cheaper IO pin's SERDES capabilities are limited to 500-900mbps.  Also, you can use the free dev tools when running the cheap 500-800mbps IO pins.

MFM pr FSK your serial bit pattern running at 500megabit and your clock will be encoded with your data with the easiest means of clock recovery.