Kintex-7 FPGA board with DDR3 memory and pci-e

[OwO]

I'm thinking of designing something like this since I can get the chips for pretty cheap. 3 years ago a 160k LE FPGA with pci-e x8 for $40 and 100k LE FPGA with pci-e x4 for $20 would have been a pipe dream.

I'm trying to brainstorm possible applications: pci-e compute card? software defined radio? The trouble is all these rely on being integrated into a PC which is bulky and ex🅱ensive. I would like a e.g. standalone SDR system with its own SoC running Linux, which would be able to act as a network gateway as well as something I can log in to remotely to do analysis of the radio spectrum. I'm already doing that with a Zynq, but the hard memory controller there is limited to 1GB of total ram and only 32 bit width which is really limiting.

One of the things I'm working on is computing huge FFTs of data from a SDR receiver for detailed spectrum analysis (in realtime). For example a size 64M FFT would require >512MB of ram just for the input buffer. Maybe a good solution is a Kintex board with DIMMs and a softcore running Linux? The softcore isn't doing anything intensive, it just coordinates things and allows remote access. Or maybe a separate SBC with a slow (USB) interface to the FPGA board? Some SBCs actually have pci-e but those would cost about as much as the FPGA.

Anyone got any more ideas?

[dmendesf]

I'd like to know we're you can find such low prices.

A cheap pci-e dogs card is a long dream of mine. Never saw one for less than 200usd with decent chips from altera or xilinx.

Enviado de meu moto g(6) plus usando o Tapatalk

[blacksheeplogic]

If you are willing to forgo PCIe a SOC FPGA with an ARM HPS is one option that's relatively low cost. If PCIe is as must, I've seen some kits that use a small SOC and PCIe x4  M2 adapter as the computer. None of these however are in your $40 quoted range, I just don't think your being realistic on price.

[OwO]

I've already built custom Zynq boards and they are fine for lower end compute, but the 1GB ram limit (imposed by the Zynq HPS memory controller) and only 32 bit DDR3 total width is kind of limiting, which is why I'm looking at other possibilities as well. Unfortunately the Altera SoCs are not nearly as easy to source, so I'm mostly limited to Xilinx stuff.

The Artix-7 is limited to a DDR3 PHY rate of 800MT/s, so I'm more gravitating towards the Kintex-7 which can do 1600MT/s (because of its "high performance" 1.8V/1.5V-only IO banks). The smallest Kintex-7 that is easy to get right now is the 160K LE XC7K160T-2FFG676I for just under $40, and I'm thinking about whether I should get a few for future experimentations. The Artix 7 with 100K LEs go for about $20. The effort to lay out >32 bit DDR3 isn't trivial, so I should pick only one family/package combination and stick with it.

I don't think it's worth it economically to build one or two because from my experience it takes at least two iterations and 3x wasted chips to get a design with lots of BGA working. The reasons being you can't test bodge fixes, and the soldering is always suspect so you end up assembling at least 2 of each design to troubleshoot any issues. But if there is a card/module design that will be reusable over and over, then it might make sense. I think the breakeven point with this project will be 5 units.

[OwO]

The basic idea I have right now is a pci-e card (not mini-pcie) with laptop size DIMM slots on it. There will need to be some kind of expansion port with lots of properly matched differential GPIO pairs so that I can add a e.g. AD9363 daughterboard for SDR experiments. I think I'll probably abuse the mini-pcie connector or some other board edge connector for the expansion port.

The board can be operative with or without pci-e being connected, so I can use it both as a compute card or standalone. I've already successfully routed the Zynq DDR3 on a 4-layer board, so this should be very doable on 4 layers too especially considering the "soft" memory controller has far more flexibility in pin assignments. I estimate prototyping cost to be 2 board revisions + 3 wasted chips, or 2*$60 + 3*$40 = $240 to get a working design, so if this is a design that can be reused more than ~3 times it's worth it.

[ali_asadzadeh]

The Artix-7 is limited to a DDR3 PHY rate of 800MT/s, so I'm more gravitating towards the Kintex-7 which can do 1600MT/s (because of its "high performance" 1.8V/1.5V-only IO banks). The smallest Kintex-7 that is easy to get right now is the 160K LE XC7K160T-2FFG676I for just under $40, and I'm thinking about whether I should get a few for future experimentations. The Artix 7 with 100K LEs go for about $20. The effort to lay out >32 bit DDR3 isn't trivial, so I should pick only one family/package combination and stick with it.

That's great, I was considering Artix 7 for their prices in china! you enlighten me up where do you get your chips in china?  also 676pin BGA, is it doable on 4 layer? 484 can be done for sure...

[OwO]

That's great, I was considering Artix 7 for their prices in china! you enlighten me up where do you get your chips in china?  also 676pin BGA, is it doable on 4 layer? 484 can be done for sure...

For prototyping I just get them from taobao.
https://item.taobao.com/item.htm?spm=a1z0k.7386009.0.0.63775eeavoMBkB&id=545807301276&_u=t2dmg8j26111
Notice the words "可直拍". This is important because that means the list price is the correct price (rather than having to message the seller for the real price).
For actual production you will want to find the upstream dealer, both to get better guarantees of consistency and usually better prices.

So far I've only done 400-BGA on 4 layers, but pin count isn't a big problem because you don't need to route all the I/Os, only what you need. So the real limit isn't how many pins but rather how many I/Os you want to break out and how wide the DDR3 is. In fact given the same number of broken out I/Os higher pin count is easier because you have more "channels" between balls to route the signals out through.

[ali_asadzadeh]

For prototyping I just get them from taobao.
https://item.taobao.com/item.htm?spm=a1z0k.7386009.0.0.63775eeavoMBkB&id=545807301276&_u=t2dmg8j26111
Notice the words "可直拍". This is important because that means the list price is the correct price (rather than having to message the seller for the real price).
For actual production you will want to find the upstream dealer, both to get better guarantees of consistency and usually better prices.

So far I've only done 400-BGA on 4 layers, but pin count isn't a big problem because you don't need to route all the I/Os, only what you need. So the real limit isn't how many pins but rather how many I/Os you want to break out and how wide the DDR3 is. In fact given the same number of broken out I/Os higher pin count is easier because you have more "channels" between balls to route the signals out through

Thanks for the info, how do you find the upstream dealer? Also taobao does not let us search nay more, it says you should register and it needs a china cell number

[aldi]

How can the XC7K160T-2FFG676I sell for only 40 dollar, the DigiKey price is 400 !?

[asmi]

How can the XC7K160T-2FFG676I sell for only 40 dollar, the DigiKey price is 400 !?

They are very likely counterfeit. if you search on Aliexpress, you can also find some lots, but Xilinx logo isn't visible on either of them. I'm itching to order a couple (I'm just a hobbyist so don't really care if it's genuine as long as it works), but kind of afraid if customs catch the counterfeit and I get into troubles.

[OwO]

Most likely either new old stock, surplus that have been dumped onto the market, or just reballed used chips. I've seen listings for used Zynqs that still comes with the PCB attached. In my case my main concern would be continued supply, because in this case the taobao prices are lower than the average market price quoted on hqyun, meaning once stock runs out likely no supplier can supply at these prices anymore. The market prices are still far lower than digikey prices, but around 1.5x what they go for on taobao currently.

Counterfeit FPGAs do not exist, it is too much work to reverse engineer the bitstream format and it makes more sense to just sell your own line of FPGAs like anlogic is doing.

[NorthGuy]

I'm itching to order a couple (I'm just a hobbyist so don't really care if it's genuine as long as it works), but kind of afraid if customs catch the counterfeit and I get into troubles.

If you decide to try, let us know what you get.

[asmi]

If you decide to try, let us know what you get.

I've ordered a XC7A50T-2FTG256I for ~50 CAD (which is half of DK's list price), as I actually have some spare empty boards for this footprint, so I can put it on once it arrives and test it immediately. It's ePacket, so it should be here in a couple of weeks. If the part will turn out OK, I will order a few K160's and design a board for it. PDS is going to be quite a pain as it can easily eat 10 Amps of current over Vccint, plus about 3 Amps for MGTs. Getting 8x10Gbps MGTs for a 100 CAD would be an awesome deal if it works out!

[OwO]

If you decide to try, let us know what you get.

I've bought Zynq-7010s twice from taobao, 2 chips each time, from different sellers. About $13 each. All the chips test out fine and all the functionality seems to work, but I do suspect they may be reballed chips.

[ali_asadzadeh]

ounterfeit FPGAs do not exist, it is too much work to reverse engineer the bitstream format and it makes more sense to just sell your own line of FPGAs like anlogic is doing.

Thanks for sharing, which website or app is this picture? I can use QQ

[nick_d]

My application for this is to implement MiniSAT in Verilog and use for massively parallel SAT solving. See "glucose syrup" solver to understand about parallel SAT. This idea needs embedded SRAM in the FPGA and would benefit from a Linux SoC to supervise. Not sure if it benefits from PCI-e but possibly from SDRAM.
cheers, Nick

[anovickis]

usually these are reballed pulls, not counterfeit

[technix]

Another idea is to use a separate PCIe interface chip like WCH CH368. That chip provides a single 32-bit PCIe 1.0 x1 device interface over a 32-bit parallel bus, allowing you to build whatever you want using cheap FPGA chips like Spartan-6 or Zynq 7010.

[mrflibble]

My application for this is to implement MiniSAT in Verilog and use for massively parallel SAT solving. See "glucose syrup" solver to understand about parallel SAT.

Interesting! Any particular design you're planning to use for the fpga implementation?

[OwO]

Another idea is to use a separate PCIe interface chip like WCH CH368. That chip provides a single 32-bit PCIe 1.0 x1 device interface over a 32-bit parallel bus, allowing you to build whatever you want using cheap FPGA chips like Spartan-6 or Zynq 7010.

The main issue with that is the bandwidth; if you do the math even a x8 pci-e interface (gen 2) has only 4GB/s of bandwidth each way, which is about the same as the memory bandwidth on a Zynq 7010 (DDR3 1066 32-bit). I would consider only x4 and above to be worth it, and x16 is ideal. Unfortunately even the Kintex-7 only supports x8, which means DDR3/4 memory on the card is mandatory if you want to do any real processing. I think up to one channel (64 bits) is routable on a 4 layer board, and two channels (128 bit total) might be doable on 6 layers. On the Zynq-7010 the bottleneck for FFT performance is the DDR3 bandwidth even with the most memory bandwidth efficient algorithm known (Bailey's 2-pass FFT algorithm).

[legacy]

isn't there any miniPCI prototyping card? I know there is a miniPCIe kit 

[asmi]

The main issue with that is the bandwidth; if you do the math even a x8 pci-e interface (gen 2) has only 4GB/s of bandwidth each way, which is about the same as the memory bandwidth on a Zynq 7010 (DDR3 1066 32-bit). I would consider only x4 and above to be worth it, and x16 is ideal. Unfortunately even the Kintex-7 only supports x8, which means DDR3/4 memory on the card is mandatory if you want to do any real processing. I think up to one channel (64 bits) is routable on a 4 layer board, and two channels (128 bit total) might be doable on 6 layers. On the Zynq-7010 the bottleneck for FFT performance is the DDR3 bandwidth even with the most memory bandwidth efficient algorithm known (Bailey's 2-pass FFT algorithm).

Each PCIe Gen 2 lane provides 5 Gbps of bandwidth, so x8 can theoretically provide 5 GBytes/s. But Kintex MGTs have a max line rate of 10 Gbps (depending on package and speed grade), so they can be used to implement PCIe Gen 3 connection, but in this case you will have to do it all by yourself as built-in PCIe block only supports Gen 2. There are commercial PCIe Gen 3 x8 IP cores for Kintex, so it's totally possible from technical standpoint. This will give you max bandwidth of about 8 GBytes/s.

[legacy]

My application for this is to implement MiniSAT in Verilog and use for massively parallel SAT solving. See "glucose syrup" solver to understand about parallel SAT.

Interesting! Any particular design you're planning to use for the fpga implementation?

never heard about SAT before, new thing learned (thanks), but now the question in my head is: for what? 

[mrflibble]

never heard about SAT before, new thing learned (thanks), but now the question in my head is: for what? 

Solving minesweeper of course.

[technix]

I have a feeling that for your specific uses going with high end graphics cards can be cheaper than FPGA, and it is less export controlled. If Taobao pricing is to be trusted, you can buy two or three nVidia RTX 2080 Ti cards or half a dozen AMD Radeon Vega 64 cards for the price of a single high gate count Kinetix-7 development board with PCIe x8 interface. Also for nVidia they provide pre-tuned, manually optimized implementations of some common algorithms like FFT as concrete CUDA libraries for those high end cards.

The FPGA board used in a system that has a high end GPU only handles interfacing the PC to high speed ADC and DAC, allowing you to use a lower gate count, slower and cheaper part with either less internal SerDes or an external PCIe interface chip.