Lattice iCE40 Bitstream Reverse-Engineered

[krivx]

I just saw this project: http://www.clifford.at/icestorm/

Here is the demo video: https://www.youtube.com/watch?v=u1ZHcSNDQMM&feature=youtu.be

It looks like bitstreams can be converted back to Verilog already.

[free_electron]

This is pretty much useless as all it does is pull a netlist of the interconnects. The generated output is not readable nor portable. You don't get the source. And it is dependent on the actual luts of the used chip. If your mew target uses a different lut architecture it won't even synthesize.

[clifford]

This is pretty much useless [...] If your mew target uses a different lut architecture it won't even synthesize.

This is just plain wrong.

In the video I used the following code for initial synthesis:

Code: [Select]

module top (
input  clk,
output LED1,
output LED2,
output LED3,
output LED4,
output LED5
);

localparam BITS = 5;
localparam LOG2DELAY = 22;

function [BITS-1:0] bin2gray(input [BITS-1:0] in);
integer i;
reg [BITS:0] temp;
begin
temp = in;
for (i=0; i<BITS; i=i+1)
bin2gray[i] = ^temp[i +: 2];
end
endfunction

reg [BITS+LOG2DELAY-1:0] counter = 0;

always@(posedge clk)
counter <= counter + 1;

assign {LED1, LED2, LED3, LED4, LED5} = bin2gray(counter >> LOG2DELAY);
endmodule
And this was the output of icebox_vlog:

Code: [Select]

module chip (output LED1, output LED3, output LED4, output LED5, output LED2, input clk);

wire n1;
reg LED1, n3, n4, n5, n6, n7, n8, n9, n10, n11, n12, n13, n14, n15, n16, n17, n18, n19, n20, n21, n22, n23, n24, n25, n26, n27, n28;
wire n29, n30, n31, n32, n33, n34, n35, n36, n37, n38, n39, n40, n41, n42, n43, n44, n45, LED3, LED4, LED5, n49, n50, n51, n52, n53, n54, n55, n56, n57, n58, LED2;
assign n1 = clk, n29 = 1;
wire n60, n61, n62, n63, n64, n65, n66, n67, n68, n69, n70, n71, n72, n73, n74, n75, n76, n77, n78, n79, n80, n81, n82, n83, n84, n85, n86;

assign n60  = /* LUT    8  9  0 */ n29 ? !n4 : n4;
assign LED5 = /* LUT    9 11  3 */ n26 ? !n27 : n27;
assign n61  = /* LUT    8  9  1 */ n30 ? !n5 : n5;
assign LED4 = /* LUT    9 11  2 */ n28 ? !n27 : n27;
assign n62  = /* LUT    8  9  2 */ n31 ? !n6 : n6;
assign LED3 = /* LUT    9 11  1 */ n3 ? !n28 : n28;
assign n63  = /* LUT    8  9  3 */ n32 ? !n7 : n7;
assign n64  = /* LUT    8  9  4 */ n33 ? !n8 : n8;
assign n65  = /* LUT    8 10  1 */ n38 ? !n13 : n13;
assign n66  = /* LUT    8  9  5 */ n34 ? !n9 : n9;
assign n67  = /* LUT    8 10  0 */ n37 ? !n12 : n12;
assign n68  = /* LUT    8  9  6 */ n35 ? !n10 : n10;
assign n69  = /* LUT    8 10  3 */ n40 ? !n15 : n15;
assign n70  = /* LUT    8  9  7 */ n36 ? !n11 : n11;
assign n71  = /* LUT    8 10  2 */ n39 ? !n14 : n14;
assign n72  = /* LUT    8 10  5 */ n42 ? !n17 : n17;
assign n73  = /* LUT    8 11  2 */ n50 ? !n22 : n22;
assign n74  = /* LUT    8 10  4 */ n41 ? !n16 : n16;
assign n75  = /* LUT    8 11  3 */ n51 ? !n23 : n23;
assign n76  = /* LUT    8 10  7 */ n44 ? !n19 : n19;
assign n77  = /* LUT    8 11  0 */ n45 ? !n20 : n20;
assign n78  = /* LUT    8 10  6 */ n43 ? !n18 : n18;
assign n79  = /* LUT    8 11  1 */ n49 ? !n21 : n21;
assign n80  = /* LUT    8 11  6 */ n54 ? !n26 : n26;
assign n81  = /* LUT    8 11  7 */ n55 ? !n27 : n27;
assign n82  = /* LUT    8 12  2 */ n58 ? !LED1 : LED1;
assign n83  = /* LUT    8 11  4 */ n52 ? !n24 : n24;
assign n84  = /* LUT    8 12  1 */ n57 ? !n3 : n3;
assign LED2 = /* LUT   12 12  7 */ n3 ? !LED1 : LED1;
assign n85  = /* LUT    8 11  5 */ n53 ? !n25 : n25;
assign n86  = /* LUT    8 12  0 */ n56 ? !n28 : n28;
assign n30  = /* CARRY  8  9  0 */ (0 & n4) | ((0 | n4) & n29);
assign n31  = /* CARRY  8  9  1 */ (0 & n5) | ((0 | n5) & n30);
assign n32  = /* CARRY  8  9  2 */ (0 & n6) | ((0 | n6) & n31);
assign n33  = /* CARRY  8  9  3 */ (0 & n7) | ((0 | n7) & n32);
assign n34  = /* CARRY  8  9  4 */ (0 & n8) | ((0 | n8) & n33);
assign n39  = /* CARRY  8 10  1 */ (0 & n13) | ((0 | n13) & n38);
assign n35  = /* CARRY  8  9  5 */ (0 & n9) | ((0 | n9) & n34);
assign n38  = /* CARRY  8 10  0 */ (0 & n12) | ((0 | n12) & n37);
assign n36  = /* CARRY  8  9  6 */ (0 & n10) | ((0 | n10) & n35);
assign n41  = /* CARRY  8 10  3 */ (0 & n15) | ((0 | n15) & n40);
assign n37  = /* CARRY  8  9  7 */ (0 & n11) | ((0 | n11) & n36);
assign n40  = /* CARRY  8 10  2 */ (0 & n14) | ((0 | n14) & n39);
assign n43  = /* CARRY  8 10  5 */ (0 & n17) | ((0 | n17) & n42);
assign n51  = /* CARRY  8 11  2 */ (0 & n22) | ((0 | n22) & n50);
assign n42  = /* CARRY  8 10  4 */ (0 & n16) | ((0 | n16) & n41);
assign n52  = /* CARRY  8 11  3 */ (0 & n23) | ((0 | n23) & n51);
assign n45  = /* CARRY  8 10  7 */ (0 & n19) | ((0 | n19) & n44);
assign n49  = /* CARRY  8 11  0 */ (0 & n20) | ((0 | n20) & n45);
assign n44  = /* CARRY  8 10  6 */ (0 & n18) | ((0 | n18) & n43);
assign n50  = /* CARRY  8 11  1 */ (0 & n21) | ((0 | n21) & n49);
assign n55  = /* CARRY  8 11  6 */ (0 & n26) | ((0 | n26) & n54);
assign n56  = /* CARRY  8 11  7 */ (0 & n27) | ((0 | n27) & n55);
assign n53  = /* CARRY  8 11  4 */ (0 & n24) | ((0 | n24) & n52);
assign n58  = /* CARRY  8 12  1 */ (0 & n3) | ((0 | n3) & n57);
assign n54  = /* CARRY  8 11  5 */ (0 & n25) | ((0 | n25) & n53);
assign n57  = /* CARRY  8 12  0 */ (0 & n28) | ((0 | n28) & n56);
/* FF  8  9  0 */ always @(posedge n1) if (1) n4 <= 0 ? 0 : n60;
/* FF  8  9  1 */ always @(posedge n1) if (1) n5 <= 0 ? 0 : n61;
/* FF  8  9  2 */ always @(posedge n1) if (1) n6 <= 0 ? 0 : n62;
/* FF  8  9  3 */ always @(posedge n1) if (1) n7 <= 0 ? 0 : n63;
/* FF  8  9  4 */ always @(posedge n1) if (1) n8 <= 0 ? 0 : n64;
/* FF  8 10  1 */ always @(posedge n1) if (1) n13 <= 0 ? 0 : n65;
/* FF  8  9  5 */ always @(posedge n1) if (1) n9 <= 0 ? 0 : n66;
/* FF  8 10  0 */ always @(posedge n1) if (1) n12 <= 0 ? 0 : n67;
/* FF  8  9  6 */ always @(posedge n1) if (1) n10 <= 0 ? 0 : n68;
/* FF  8 10  3 */ always @(posedge n1) if (1) n15 <= 0 ? 0 : n69;
/* FF  8  9  7 */ always @(posedge n1) if (1) n11 <= 0 ? 0 : n70;
/* FF  8 10  2 */ always @(posedge n1) if (1) n14 <= 0 ? 0 : n71;
/* FF  8 10  5 */ always @(posedge n1) if (1) n17 <= 0 ? 0 : n72;
/* FF  8 11  2 */ always @(posedge n1) if (1) n22 <= 0 ? 0 : n73;
/* FF  8 10  4 */ always @(posedge n1) if (1) n16 <= 0 ? 0 : n74;
/* FF  8 11  3 */ always @(posedge n1) if (1) n23 <= 0 ? 0 : n75;
/* FF  8 10  7 */ always @(posedge n1) if (1) n19 <= 0 ? 0 : n76;
/* FF  8 11  0 */ always @(posedge n1) if (1) n20 <= 0 ? 0 : n77;
/* FF  8 10  6 */ always @(posedge n1) if (1) n18 <= 0 ? 0 : n78;
/* FF  8 11  1 */ always @(posedge n1) if (1) n21 <= 0 ? 0 : n79;
/* FF  8 11  6 */ always @(posedge n1) if (1) n26 <= 0 ? 0 : n80;
/* FF  8 11  7 */ always @(posedge n1) if (1) n27 <= 0 ? 0 : n81;
/* FF  8 12  2 */ always @(posedge n1) if (1) LED1 <= 0 ? 0 : n82;
/* FF  8 11  4 */ always @(posedge n1) if (1) n24 <= 0 ? 0 : n83;
/* FF  8 12  1 */ always @(posedge n1) if (1) n3 <= 0 ? 0 : n84;
/* FF  8 11  5 */ always @(posedge n1) if (1) n25 <= 0 ? 0 : n85;
/* FF  8 12  0 */ always @(posedge n1) if (1) n28 <= 0 ? 0 : n86;

endmodule
Now please point to the construct in that verilog code that will not synthesize on some targets!

I remember from theamphour that you are not very keen on open source eda software, so you might not realize the importance of our work in that larger picture, but regardless of that I think its still pretty rude to say something like "this is pretty much useless" about someone else's work without actually looking at it first..

[ogoun]

Hi,
Great work!

I was looking at the Xilinx XC3000 series (dinosaur fpgas), with a view to doing something similar but never got very far.

Given the large number of orphaned designs using these ancient parts, have you considered doing something similar for them?

Cheers,

Pete

[mikeselectricstuff]

Even if the resulting code is not readable, I can see how it might be useful to simulate it.
And extract contents of block-ram initialisation data

[mikeselectricstuff]

This is pretty much useless [...] If your mew target uses a different lut architecture it won't even synthesize.

This is just plain wrong.

But if it used any device-specific features  it would be of much more limited use, though it may be possible to generate HDL that emulates the functions of those features.
Of course ICE40 doesn't really have much in the way of device-specific features....

I remember from theamphour that you are not very keen on open source eda software, so you might not realize the importance of our work in that larger picture,

How exactly is it important? To whom?
Whilst i'd be the first to say that the whole FPGA software process is a pain, archaic, bloated  and generally crap, I can't see any realistic chance of an open-source solution ever becoming  a useable alternative for anything but the most trivial cases.

The ability to discover & understand  internal functionality from a bitstream is of course interesting and potentially useful for reverse-engineering, but considering the size and complexity of even low-end FPGAs these days, and how little progress has been made on this in the last 20-odd years it seems like it's still a long way away...

[mikeselectricstuff]

Open source is a faith, a way to freedom to OSS believers.

And often about as useful as other religions

Windows and VS won't be so cheap if there is no GNU/Linux and GCC.

Debatable - the market will ultimately decide prices, and I'm not sure it makes much difference if competition comes from OSS or a commercial competitor.

For the same reason, some people want to challenge the dominance of Cadence and Synopsis.

The reason they are dominant is they are the experts - any newcomer has a huge learning curve

Many works has been done on open source SoC, such as OR1k and lowRISC.

And how many people are using it in real products? Can I buy a fully tested & qualified chip based on these?

Open source is a "nice to have", but most people just want to get a job done.

IMO, instead of spending time basically duplicating vendor tool functionality, it would be a much better use of resources to work on new innovative front-end design methods to replace the current HDL for driving the vendor place & route tools.
The vendor will always be in the best position to know how to use the parts their design - there is a two-way design process between the silicon & the tools. No Open Source effort stands any chance of doing better for anything of useful functionality. 
 

[clifford]

I remember from theamphour that you are not very keen on open source eda software, so you might not realize the importance of our work in that larger picture,

How exactly is it important? To whom?

We will develop a reference open-source place&route flow for iCE40 for the GSoC. That would not be possible without knowing the bitstream format. THIS is the main motivation for icestorm, not reverse engineering fpga bitstreams. The reverse engineering part just demonstrates well that we do in fact understand the bitstream to a very large extent.

This is not only relevant for iCE40 as architecture, but the main motivation is to deliver a reference full open-source end-to-end FPGA flow, to demonstrate that this can be done. There is a lot of dogmatic thinking about how huge and complex and impossibly complicated this kind of stuff is, primarily by people who them self have zero experience in writing software for this kind of problems.

Btw: Being told that I'm working on things that cannot be done, by people who clearly are not qualified to be a judge of that because they do not have the relevant background, is the story of my life in the last couple of years. (The relevant background in this case is writing FPGA and ASIC design tools btw., not using them, and it really does not matter how many decades of experience you have using them.. A secretary might also have years and years of experience in using word processors. That does not make him or her an expert on estimating the complexity of writing one either.) The vendors of commercial EDA tools are very careful to market even the simplest programs as super-complex ones. And especially the people who are using those programs day-in day-out seem to have a tendency to buy into that marketing BS.

So here is an important truth: Writing synthesis tools is about as complex as writing compilers. I know, because I've worked on both. In both cases you have no chance if you don't know what you are doing. An most people would not even know where to begin. But that does not mean it is an impossible task or even that it is inherently complex. It just means you have to read the relevant books and learn the relevant algorithms and methods before you start.

The pure existence of an Open Source end-to-end FPGA flow can change how willing vendors are to share low-level specifications. If we only had the compilers provided by CPU vendors, and no Open Source compilers, vendors maybe would take a different road about specifying things like instruction sets. But with compilers we have reached a point where it is not only a no-brainer to release all your instruction set documentation, you also don't write your own compiler from scratch. Noone does. You take GCC or Clang+LLVM and port it to your architecture (in the last few years we have seen Clang+LLVM to become more and more important). I can see a similar shift happening in FPGA synthesis within the next 10-15 years.

And this is not only about ideology. You can do great things today by using code generators on a CPU to create code for the CPU. JITs are a good example for that. You cannot do something like run Vivado on the ARM processor in a Zynq and on-demand synthesize a circuit and load it into the FPGA fabric. (An example for something like that would be a complex trigger circuit for a logic analyzer.) There are many applications that will profit from an open source flow, even when the OSS flow is not as optimized as the competing proprietary flow.

Our iCE40 flow will use Yosys as HDL synthesis front-end. We can already synthesize to netlists for Xilinx 7 series FPGAs with Yosys and I've implemented the basic stuff for iCE40 synthesis a while back and will improve on that while a student is working on the place&route tool.

, but considering the size and complexity of even low-end FPGAs these days

that's why we chose the iCE40 as target. it is not complex at all, and even the largest part in the family is not very large (which limits the complexity of the place&route problem to something that I would consider reasonable for a student project).

, and how little progress has been made on this in the last 20-odd years it seems like it's still a long way away...

As author of Yosys I must say a statement like this really hurts. In the last two years there have been a couple of (industrial and accademic) ASIC tapeouts of designs that have been synthesized with Yosys. On the iCE40 side I expect similar results from Yosys and Lattice LSE by the end of the summer, so it might still be a long way until we can compare our synthesis solution with Synpilfy Pro, but we are in the range of what you would expect from an industry tool. (The iCE40 backend is pretty new, but by comparing Yosys and Synpilfy Pro on the Xilinx side and comparing Lattice LSE and Synpilfy Pro on the iCE40 side i would say that this is a realistic goal.)

When it comes to ASIC and FGPA flows, the amount of things we can do with OSS right now would have been unthinkable 5 years ago, 20 years ago there wasn't even a discussion about things like that..

PS: There are a lot of FPGA architectures that most people here have never heard of, that are used as IP blocks in larger ASICs and cannot be bought as a parts but only as ASIC IP. Yosys plays an increasingly larger role in this industry. But unfortunately I'm not at liberty to share any details about that.. ASIC people are a very secretive folk..

[clifford]

And how many people are using it in real products? Can I buy a fully tested & qualified chip based on these?

Thats exactly what the lowRISC project is about, as well as some other RISC-V projects like Shakti (but I'm not sure if Shakti CPUs will be available on the free market or Indian government only).

Open source is a "nice to have", but most people just want to get a job done.

For some projects open source is not a question of ideology. Its about can I use that tool in the way that is required by my application. And in this cases you simply cannot do your project without having an open source tool.

Think of Linux for example. Of course you could just go and compare Linux with Windows CE feature-wise and then come to one conclusion or another. But for some project you simply cannot go with something that isn't Open Source, for example if you have to port the whole thing to a new processor architecture, or recompile it to use your hardware extension that add special protection for return addresses on the stack, but requires different instructions to be used by the compiled code to access those return addresses.

IMO, instead of spending time basically duplicating vendor tool functionality, it would be a much better use of resources to work on new innovative front-end design methods to replace the current HDL for driving the vendor place & route tools.

There are people working on this as well (see Chisel for example, which is used by more and more projects, including the rocket core generator).

You are free to participate in this kind of projects if you think it is important. I think this is important too, but I also think the stuff I'm working on is important and a nice think about not getting paid is that I can decide for myself where my resources are used best..

The vendor will always be in the best position to know how to use the parts their design - there is a two-way design process between the silicon & the tools. No Open Source effort stands any chance of doing better for anything of useful functionality.

And what if you ARE the vendor? Look at compilers: noone is writing their own compiler from scratch because it is just plain stupid to replicate to work of everyone else who has written a compiler before you, just so you can add your one little special feature. You know what is really really hard about making your own small FPGA? Its writing the design tools.

Why do we have processors with built in vector floating point units and megabytes of cache, but no FPGA fabric? It would be really easy to create a uC with a small and simple FPGA fabric that can do things like serialisation/deserialisation for your bus protocols? One reason is of course power (hard IPs are way more power efficient), but that's not the entire truth. The main reason is: Because it is impractical to write your own synthesis toolchain from scratch for something that is just an extra-feature of a chip that's main purpose is something else and you won't be able to negotiate licences for packages like Synplify Pro that are reasonable for a applications like this.

Btw: lowRISC is considering a small FPGA fabric for their minion cores. Guess what: Noone talks about licencing Synplify Pro for that.

[mikeselectricstuff]

The pure existence of an Open Source end-to-end FPGA flow can change how willing vendors are to share low-level specifications.

I seriously doubt that

If we only had the compilers provided by CPU vendors, and no Open Source compilers, vendors maybe would take a different road about specifying things like instruction sets. But with compilers we have reached a point where it is not only a no-brainer to release all your instruction set documentation, you also don't write your own compiler from scratch. Noone does. You take GCC or Clang+LLVM and port it to your architecture (in the last few years we have seen Clang+LLVM to become more and more important). I can see a similar shift happening in FPGA synthesis within the next 10-15 years.

The problem is the FPGA market (in terms of number of designs) is miniscule compared to CPUs, and there are far fewer players. Vendors supply tools that work. Any competition will have too small a market share to attain critical mass for anything outside a few niche areas.

As author of Yosys I must say a statement like this really hurts. In the last two years there have been a couple of (industrial and accademic) ASIC tapeouts of designs that have been synthesized with Yosys.

One tapeout a year - that's hardly significant.
And I'm not referring to synthesis tools, just the place & route side of things.

ASIC people are a very secretive folk..

Which is Exactly why you'll never see an FPGA vendor publish the essential details. Apart from competietion and customer IP concerns, another reason is they don't want to reveal things that competitors may claim violates their IP rights.

I can certainly see that a Open Source synthesis and front-end tools could potentially be very useful, however IMO place and route/fitting will always be the domain of vendor tools. The vast majority of users simply don't care if it's OSS or not, they just want to get the job done.

What you are doing is undoubtedly interesting and cool,  I just can't see much in the way of practical use, let alone "important" to anyone outside adademia . ICE40 is so far behind the state of the art that there is a huge gap between being able to place & route such a simple device is a long way from being able to do anything useful with more versatile devices, and by the time it;s done the device will have been superceded.

Maybe I'm just getting too cynical in my old age

[amyk]

The ability to discover & understand  internal functionality from a bitstream is of course interesting and potentially useful for reverse-engineering, but considering the size and complexity of even low-end FPGAs these days, and how little progress has been made on this in the last 20-odd years it seems like it's still a long way away...

You mean "how little public progress"...

FPGA bitstreams are very regular, because FPGAs themselves are. The major issues are legal/political, not technical.

[nctnico]

The problem is the FPGA market (in terms of number of designs) is miniscule compared to CPUs, and there are far fewer players. Vendors supply tools that work. Any competition will have too small a market share to attain critical mass for anything outside a few niche areas.

I'd put that as 'barely work'. I have been using Xilinx' tools for close to 15 years and for designs with a high density it takes tweaking the parameters from numbers obtained by rolling the dice to get a design routed properly and quickly.

In an open source environment it is much easier to get to a better generic place & routing tool than vendors can. Let the vendors concentrate on creating silicon. For example: from a technical point of view the Linux kernel is lightyears ahead compared to what is under the hood of Windows (which is made by a multi billion dollar firm!).

edit: typos

[mikeselectricstuff]

The problem is the FPGA market (in terms of number of designs) is miniscule compared to CPUs, and there are far fewer players. Vendors supply tools that work. Any competition will have too small a market share to attain critical mass for anything outside a few niche areas.

I'd put that as 'barely work'. I have been using Xilinx' tools for close to 15 years and for designs with a high density it takes tweaking the parameters from number obtained by rolling the dice to get a design routed properly and quickly.

In an open source environment it is much easier to get better to a better generic place & routing tool than vendors can. Let the vendors concentrate on creating silicon. For example: from a technical point of view the Linux kernel is lightyears ahead compared to what is under the hood of Windows (which is made by a multi billion dollar firm!).

But is a generic P&R process ever going to get anywhere near something which is (presumably) highly optimised to the particular device?

[nctnico]

AFAIK an FPGA is like a grid of components with routing resources in between. I think you can compare it with a PCB design where you have a fixed amount of room for wires between components so many problems you find with placement and routing on a PCB design also apply to placing and routing an FPGA.

[Someone]

The vendor will always be in the best position to know how to use the parts their design - there is a two-way design process between the silicon & the tools. No Open Source effort stands any chance of doing better for anything of useful functionality.

And what if you ARE the vendor? Look at compilers: noone is writing their own compiler from scratch because it is just plain stupid to replicate to work of everyone else who has written a compiler before you, just so you can add your one little special feature. You know what is really really hard about making your own small FPGA? Its writing the design tools.

Why do we have processors with built in vector floating point units and megabytes of cache, but no FPGA fabric? It would be really easy to create a uC with a small and simple FPGA fabric that can do things like serialisation/deserialisation for your bus protocols? One reason is of course power (hard IPs are way more power efficient), but that's not the entire truth. The main reason is: Because it is impractical to write your own synthesis toolchain from scratch for something that is just an extra-feature of a chip that's main purpose is something else and you won't be able to negotiate licences for packages like Synplify Pro that are reasonable for a applications like this.

Btw: lowRISC is considering a small FPGA fabric for their minion cores. Guess what: Noone talks about licencing Synplify Pro for that.

We can look at the example of processors in the last decade where we have transitioned from expensive vendor tools and lacking documentation, to an explosion of vendors offering silicon backed by open source toolchains and competing to offer low cost processors. Hopefully an open source toolchain for FPGAs can help spur on small semi firms to start releasing FPGAs (or as above FPGA fabric on/with/inside other existing products).

[mikeselectricstuff]

We can look at the example of processors in the last decade where we have transitioned from expensive vendor tools and lacking documentation, to an explosion of vendors offering silicon backed by open source toolchains and competing to offer low cost processors.

I'd disagree that availability of OS toolchains has much to do with it - nobody said "hey, there's an OSS compiler, let's sell a chip to use it"
It's the other way round - they make use of OS toolchains for a new product because they're there, and the easiest/cheapest way to support the product.
If they weren't there, the manufacturers would have worked with one of the commercial companies like IAR,Keil etc. to do a compiler.

Hopefully an open source toolchain for FPGAs can help spur on small semi firms to start releasing FPGAs (or as above FPGA fabric on/with/inside other existing products).

I can't really see that happenning - the existing players are too dominant for it to be worth a new player risking entry to the market - several have tried & failed over the years.
The only recent-ish "new" player I can recall in recent memory is SiliconBlue, and they are now the Lattice Ice40 range. Actel got taken over by Microsemi. The number of players is shrinking, not expanding.

Even if someone new found a new niche (like low pin-count devices) , it would be easy for the existing players to add to their range to compete with it.

I'm sure tools are a major part of an FPGA company's cost, but doubt availability tools would be enough to swing the viability of entry to the market.

Linux is successful because so many people have a use for it. the FPGA area is just too specialist for enough people to be interested, and the vast majority of users don't care what flavour of tools they need to use to do the job.

Considering we've not yet got fully professional-grade OSS PCB software, what are the chances of seeing something in a field that's way more niche, and more complex than that ?

[Rasz]

Maybe I'm just getting too cynical in my old age

Yes. You (and free electron) sound like Sun 2 workstation salesman in 1984 telling us we will never get free compilers because nobody works for free, they are extremely hard to write and need to support mountains of hardware, not to mention why bother when you get perfectly fine CC shipped with your $40K box.

[hamster_nz]

I ponder on how hard it is to characterise the timing of the iCE40 devices (propagation, loading due to fanout, switching, temperature, voltage, different speed grades) to allow accurate enough timing. You can get away with nearly anything at 1MHz, but what at about 200MHz?

Sure you can make some assumptions, but you wouldn't want to do that for a commercial product where you have liability issues.

At least with a CPU once you have the part number you pretty much know the performance limits for the entire device, and as over-clockers nuts will tell you, the limits can be pretty close.

[free_electron]

This is pretty much useless [...] If your mew target uses a different lut architecture it won't even synthesize.

This is just plain wrong.

In the video I used the following code for initial synthesis:

Code: [Select]

module top (
input  clk,
output LED1,
output LED2,
output LED3,
output LED4,
output LED5
);

localparam BITS = 5;
localparam LOG2DELAY = 22;

function [BITS-1:0] bin2gray(input [BITS-1:0] in);
integer i;
reg [BITS:0] temp;
begin
temp = in;
for (i=0; i<BITS; i=i+1)
bin2gray[i] = ^temp[i +: 2];
end
endfunction

reg [BITS+LOG2DELAY-1:0] counter = 0;

always@(posedge clk)
counter <= counter + 1;

assign {LED1, LED2, LED3, LED4, LED5} = bin2gray(counter >> LOG2DELAY);
endmodule
And this was the output of icebox_vlog:

Code: [Select]

module chip (output LED1, output LED3, output LED4, output LED5, output LED2, input clk);

wire n1;
reg LED1, n3, n4, n5, n6, n7, n8, n9, n10, n11, n12, n13, n14, n15, n16, n17, n18, n19, n20, n21, n22, n23, n24, n25, n26, n27, n28;
wire n29, n30, n31, n32, n33, n34, n35, n36, n37, n38, n39, n40, n41, n42, n43, n44, n45, LED3, LED4, LED5, n49, n50, n51, n52, n53, n54, n55, n56, n57, n58, LED2;
assign n1 = clk, n29 = 1;
wire n60, n61, n62, n63, n64, n65, n66, n67, n68, n69, n70, n71, n72, n73, n74, n75, n76, n77, n78, n79, n80, n81, n82, n83, n84, n85, n86;

assign n60  = /* LUT    8  9  0 */ n29 ? !n4 : n4;
assign LED5 = /* LUT    9 11  3 */ n26 ? !n27 : n27;
assign n61  = /* LUT    8  9  1 */ n30 ? !n5 : n5;
assign LED4 = /* LUT    9 11  2 */ n28 ? !n27 : n27;
assign n62  = /* LUT    8  9  2 */ n31 ? !n6 : n6;
assign LED3 = /* LUT    9 11  1 */ n3 ? !n28 : n28;
assign n63  = /* LUT    8  9  3 */ n32 ? !n7 : n7;
assign n64  = /* LUT    8  9  4 */ n33 ? !n8 : n8;
assign n65  = /* LUT    8 10  1 */ n38 ? !n13 : n13;
assign n66  = /* LUT    8  9  5 */ n34 ? !n9 : n9;
assign n67  = /* LUT    8 10  0 */ n37 ? !n12 : n12;
assign n68  = /* LUT    8  9  6 */ n35 ? !n10 : n10;
assign n69  = /* LUT    8 10  3 */ n40 ? !n15 : n15;
assign n70  = /* LUT    8  9  7 */ n36 ? !n11 : n11;
assign n71  = /* LUT    8 10  2 */ n39 ? !n14 : n14;
assign n72  = /* LUT    8 10  5 */ n42 ? !n17 : n17;
assign n73  = /* LUT    8 11  2 */ n50 ? !n22 : n22;
assign n74  = /* LUT    8 10  4 */ n41 ? !n16 : n16;
assign n75  = /* LUT    8 11  3 */ n51 ? !n23 : n23;
assign n76  = /* LUT    8 10  7 */ n44 ? !n19 : n19;
assign n77  = /* LUT    8 11  0 */ n45 ? !n20 : n20;
assign n78  = /* LUT    8 10  6 */ n43 ? !n18 : n18;
assign n79  = /* LUT    8 11  1 */ n49 ? !n21 : n21;
assign n80  = /* LUT    8 11  6 */ n54 ? !n26 : n26;
assign n81  = /* LUT    8 11  7 */ n55 ? !n27 : n27;
assign n82  = /* LUT    8 12  2 */ n58 ? !LED1 : LED1;
assign n83  = /* LUT    8 11  4 */ n52 ? !n24 : n24;
assign n84  = /* LUT    8 12  1 */ n57 ? !n3 : n3;
assign LED2 = /* LUT   12 12  7 */ n3 ? !LED1 : LED1;
assign n85  = /* LUT    8 11  5 */ n53 ? !n25 : n25;
assign n86  = /* LUT    8 12  0 */ n56 ? !n28 : n28;
assign n30  = /* CARRY  8  9  0 */ (0 & n4) | ((0 | n4) & n29);
assign n31  = /* CARRY  8  9  1 */ (0 & n5) | ((0 | n5) & n30);
assign n32  = /* CARRY  8  9  2 */ (0 & n6) | ((0 | n6) & n31);
assign n33  = /* CARRY  8  9  3 */ (0 & n7) | ((0 | n7) & n32);
assign n34  = /* CARRY  8  9  4 */ (0 & n8) | ((0 | n8) & n33);
assign n39  = /* CARRY  8 10  1 */ (0 & n13) | ((0 | n13) & n38);
assign n35  = /* CARRY  8  9  5 */ (0 & n9) | ((0 | n9) & n34);
assign n38  = /* CARRY  8 10  0 */ (0 & n12) | ((0 | n12) & n37);
assign n36  = /* CARRY  8  9  6 */ (0 & n10) | ((0 | n10) & n35);
assign n41  = /* CARRY  8 10  3 */ (0 & n15) | ((0 | n15) & n40);
assign n37  = /* CARRY  8  9  7 */ (0 & n11) | ((0 | n11) & n36);
assign n40  = /* CARRY  8 10  2 */ (0 & n14) | ((0 | n14) & n39);
assign n43  = /* CARRY  8 10  5 */ (0 & n17) | ((0 | n17) & n42);
assign n51  = /* CARRY  8 11  2 */ (0 & n22) | ((0 | n22) & n50);
assign n42  = /* CARRY  8 10  4 */ (0 & n16) | ((0 | n16) & n41);
assign n52  = /* CARRY  8 11  3 */ (0 & n23) | ((0 | n23) & n51);
assign n45  = /* CARRY  8 10  7 */ (0 & n19) | ((0 | n19) & n44);
assign n49  = /* CARRY  8 11  0 */ (0 & n20) | ((0 | n20) & n45);
assign n44  = /* CARRY  8 10  6 */ (0 & n18) | ((0 | n18) & n43);
assign n50  = /* CARRY  8 11  1 */ (0 & n21) | ((0 | n21) & n49);
assign n55  = /* CARRY  8 11  6 */ (0 & n26) | ((0 | n26) & n54);
assign n56  = /* CARRY  8 11  7 */ (0 & n27) | ((0 | n27) & n55);
assign n53  = /* CARRY  8 11  4 */ (0 & n24) | ((0 | n24) & n52);
assign n58  = /* CARRY  8 12  1 */ (0 & n3) | ((0 | n3) & n57);
assign n54  = /* CARRY  8 11  5 */ (0 & n25) | ((0 | n25) & n53);
assign n57  = /* CARRY  8 12  0 */ (0 & n28) | ((0 | n28) & n56);
/* FF  8  9  0 */ always @(posedge n1) if (1) n4 <= 0 ? 0 : n60;
/* FF  8  9  1 */ always @(posedge n1) if (1) n5 <= 0 ? 0 : n61;
/* FF  8  9  2 */ always @(posedge n1) if (1) n6 <= 0 ? 0 : n62;
/* FF  8  9  3 */ always @(posedge n1) if (1) n7 <= 0 ? 0 : n63;
/* FF  8  9  4 */ always @(posedge n1) if (1) n8 <= 0 ? 0 : n64;
/* FF  8 10  1 */ always @(posedge n1) if (1) n13 <= 0 ? 0 : n65;
/* FF  8  9  5 */ always @(posedge n1) if (1) n9 <= 0 ? 0 : n66;
/* FF  8 10  0 */ always @(posedge n1) if (1) n12 <= 0 ? 0 : n67;
/* FF  8  9  6 */ always @(posedge n1) if (1) n10 <= 0 ? 0 : n68;
/* FF  8 10  3 */ always @(posedge n1) if (1) n15 <= 0 ? 0 : n69;
/* FF  8  9  7 */ always @(posedge n1) if (1) n11 <= 0 ? 0 : n70;
/* FF  8 10  2 */ always @(posedge n1) if (1) n14 <= 0 ? 0 : n71;
/* FF  8 10  5 */ always @(posedge n1) if (1) n17 <= 0 ? 0 : n72;
/* FF  8 11  2 */ always @(posedge n1) if (1) n22 <= 0 ? 0 : n73;
/* FF  8 10  4 */ always @(posedge n1) if (1) n16 <= 0 ? 0 : n74;
/* FF  8 11  3 */ always @(posedge n1) if (1) n23 <= 0 ? 0 : n75;
/* FF  8 10  7 */ always @(posedge n1) if (1) n19 <= 0 ? 0 : n76;
/* FF  8 11  0 */ always @(posedge n1) if (1) n20 <= 0 ? 0 : n77;
/* FF  8 10  6 */ always @(posedge n1) if (1) n18 <= 0 ? 0 : n78;
/* FF  8 11  1 */ always @(posedge n1) if (1) n21 <= 0 ? 0 : n79;
/* FF  8 11  6 */ always @(posedge n1) if (1) n26 <= 0 ? 0 : n80;
/* FF  8 11  7 */ always @(posedge n1) if (1) n27 <= 0 ? 0 : n81;
/* FF  8 12  2 */ always @(posedge n1) if (1) LED1 <= 0 ? 0 : n82;
/* FF  8 11  4 */ always @(posedge n1) if (1) n24 <= 0 ? 0 : n83;
/* FF  8 12  1 */ always @(posedge n1) if (1) n3 <= 0 ? 0 : n84;
/* FF  8 11  5 */ always @(posedge n1) if (1) n25 <= 0 ? 0 : n85;
/* FF  8 12  0 */ always @(posedge n1) if (1) n28 <= 0 ? 0 : n86;

endmodule
Now please point to the construct in that verilog code that will not synthesize on some targets!

Note : do no confuse LUT. LUT in this context = logic unit , not 'look-up-table). Some families hav a LUT that is 2 fliplfops with a combinatorial cloud before it with 10 inputs. , others have 4 flipflops with a larger cloud. every manufacturer and family has its own mix of such LUT blocks.
take any fpga where the LUT is constructed differently from the Lattice LUT used for this specific family.

If i understand it correctly what you do is pull a reverse netlist and wire it into the LUT inputs and outputs. And then you unmap the lut matrix into NOt AND and OR operations and multiplexers and assign those to wires. you know what the LUT of this particular device looks like so you can resolve down to the flipflop inputs and outputs. correct ?

Assume one architecture where 1 lut consists of an 8 input and/or matrix feeding 2 flipflops.
Assume a different architecture where 1 lut is a 20 input and/or matrix feeding 4 flipflops

The lut architecture is different between families of the same manufacturers. Compare a lut from Lattice against an altera or a Xilinx one. They are different. Even though you have a logic cloud of and/not/or and multiplexers ( the ? operand in verilog ) feeding flipflops , that will not map correctly on an architecture with a different lut structure. You may end up with such large differences that the recompiled entity does not run as timing is completely off.

It gets more complex once code starts using specialistic macros.
How will you trans-map the bitstream if it uses embedded memory blocks , embedded adders / multipliers or other hard-macros that don't exist in other families or manufacturers ?

It s great you found a way to decompile the bit stream, but its output is a massive logic cloud of and/or/not that is pretty much unreadable. Yes you can recompile it , providing you don't switch architecture or families, if you are lucky it may work on some other families.

So what you have is a bitstream to logic equation resolver ( you could take your output one step further and get rid of the intermediate wires and write out the boolean equations driving the fliplfop control lines. You can compare your output to opening the final , minimized logic netlist in the schematic viewer of the FPGA tools.

that's not really decompiling in my book.

Anyway, cool you figured out how the bitstream works , but i don't see a practical use.
Let's say i have a board with a xilinx fpga that has, amongst other functionality, some logic function in it i really want.  i want to 'decompile the bitstream' , steal that interesting block and slap it in an altera.

The system works for simple devices but is not really portable ( mainly due to architectural differences between manufacturers and their use of hard-macro's that do not exist in other families.

As for 'free' compilers for FPGA : you can synthesize down to a point but then you really need the manufacturer tools. Only the manufacturer tools have access to those hard macro's.
The tools are optimized to recognize certain functions and map those onto available hardware in the particular device. you may code a simple counter . The compiler detects this and knows it has a certain number of LUTS that have optimized interconnects to implement counters.
Every FPGA manufacturer has their secret sauce of  'accelerators'. Only they know how to translate the source to an optimized output.
With old, simple CPLD's and fpga's that only held simple gates and simple flipflops it doesn't matter.
Fact is that devices produced in the last 10 years have 'accelerators'. your code is not flattened to it shortest form logic equation like they did 20 years ago. Instead it is smartly mapped on those accelerators.  every vendor has their own 'secret mix'.

most devices these days have special functionality to even accelerate relatively simple things like counters. unless you know those exist, and how they work ( for example the carry propagation logic may exist as a hard , optimized block that is much faster than its synthesized equivalent ) you don't stand a chance reversing that , nor porting it.

While this is a cool experiment , it doesn't really have a practical application. The output is pretty much unreadable and it will only work for simple devices.

[clifford]

ASIC people are a very secretive folk..

Which is Exactly why you'll never see an FPGA vendor publish the essential details.

Aehmm.. never? do you remember JBits? Xilinx did release the bitstream format of the XC4000 devices (and I think also the Virtex I, which might just have been called Virtex without a number). But back then there was no Open Source infrastructure to do anything useful with that.

FPGA Vendors have a long standing tradition of using BSD-licenced open source components in their tool chains. Look at ABC for example and all the commercial tools that are essentially a GUI and a little front-end that feed stuff through ABC for logic optimization or verification. They don't tell you about it of course, because why should they? On occasion they release very detailed information about old very large devices, so the academic world has something to play and advance the tools the industry is using. (I remember VTR using bitstream docs released by Xilinx for newer devices than Virtex I, but I can't find anything yet.). But so far there never was an open-source end-to-end flow, mostly because of the lack of a HDL front-end.

(Yes, there is Odin-II, and vl2m, and HANA, but have you tried any of those? At best they can be used as netlist parsers. I couple of years ago I've been asked to compare those to each other (and to yosys). This have been my results: http://scratch.clifford.at/vlogev.pdf)

So FPGA vendors publishing essential details is nothing new. They just not advertise it to end-users and usually you need a few years old dev board for a chip that has been top-of-the-line at its time. With our work you can buy an icestick for 25 USD and play with an FPGA bitstream format.

FPGA vendors using open source software in their tools is nothing new either. But usually this is mostly stuff for logic optimization, because this is the domain where we have really good open source tools for a long time now. But I see no reason why they would not start using open source in other problem domains, once it is available.

[mikeselectricstuff]

ASIC people are a very secretive folk..

Which is Exactly why you'll never see an FPGA vendor publish the essential details.

Aehmm.. never? do you remember JBits? Xilinx did release the bitstream format of the XC4000 devices (and I think also the Virtex I, which might just have been called Virtex without a number). But back then there was no Open Source infrastructure to do anything useful with that.

OK if you want to argue semantics I'll rephrase -  you'll never see an FPGA vendor publish the essential details on any remotely current device that anyone would think of using in a new design.

All I'm saying is there are much more useful things to spend OSS development time on than place & route.
e.g. place & route will always need updating as  new devices appear, whereas synthesis & other front-end tools are mostly device-independent, so unless an OSS effort was actively funded, chances are it wouldn't sustain enough development effort over time to continue to be useful.
It's not inconceivable that a manufacturer may develop  new tools and open source them, but it's hard to see that they would see enough benefit for them to do it, certainly towards the back-end aspects specific to their products.
 
Another  aspect, which I believe was a significant factor back in the PLD/GAL days was simply that by staying closed, it is much  quicker and easier for them to make changes, e.g. to adapt to new processs & fabs - as soon as you commit to publically documenting things you take on significant work just keeping public documentation up to date, and risks of bad publicity from problems with 3rd-parties using outdated info.

[wumpus]

Linux is successful because so many people have a use for it. the FPGA area is just too specialist for enough people to be interested, and the vast majority of users don't care what flavour of tools they need to use to do the job.

Many people would disagree. Due to various reasons (crowdfunding, distrust, scaling limits of CPUs among others), interest in open source hardware has picked up enormously recently.

The unavailability of general, open source FPGA tooling has always been a limiting factor there. Instead of applying tunnel vision to current (sort of) happy users of vendor tools, take a wider view. This can open up completely new markets.

There is nothing inherently esoteric or specialist about programmable logic, it could be said to be more fundamental than programming CPUs. But the bulky, hard to install, restrictively licensed tools have kept many people from even reaching the "hello world" phase - see the Parallela forums for enough examples, even if they have an FPGA available and are interested.

A freely available "gcc for FPGA" could make a huge difference here, smash down the adoption barrier. And one look at gcc (and clang) is enough to be convinced that the open source community is able to build huge, complex code generation pipelines. There is a lot of specialist knowledge represented.

Just as in a compiler, the chief part of the pipeline is generic. Place and route, seen abstractly, is a matter of having a correct definition of the units and connections of the target hardware then optimizing within those constraints. Even if the result is less optimized for a specific vendor, the generality and availability is a great boon for other applications (the same could be said about vendor compilers versus open source compilers).

Anyhow, arguing about this is useless. We'll see where this goes. I'm excited about it, at least.

[Mechanical Menace]

Bookmarked the IceStorm page. Amazing work Clifford, thanks. I can see this (and supported FPGAs) becoming very popular in the admittedly small homebrew computer and cpu scene before the year's out.

[mikeselectricstuff]

Linux is successful because so many people have a use for it. the FPGA area is just too specialist for enough people to be interested, and the vast majority of users don't care what flavour of tools they need to use to do the job.

Many people would disagree. Due to various reasons (crowdfunding, distrust, scaling limits of CPUs among others), interest in open source hardware has picked up enormously recently.

The unavailability of general, open source FPGA tooling has always been a limiting factor there.

Nonsense. There is no real barrier to using an FPGA in an OSHW project that an OS toolchain would remove. Everyone can access FPGA tools at minimal cost.

[daqq]

Nonsense. There is no real barrier to using an FPGA in an OSHW project that an OS toolchain would remove. Everyone can access FPGA tools at minimal cost.

To be fair, the bigger devices are supported only by the costly versions of stuff - atleast for Xilinx. Dunno about the rest, Altera, erm, Intertra?.

I never really got this - what's the point of not publishing stuff like this? From a chip manufacturers point of view it seems the best course of action would be to ensure that ALL of my tools area available to everyone, free of charge. Or atleast MASSIVELY support open source initiatives like this.

The availability of free or cheaply priced tools is a big issue for me, I assume the same goes for others.