BrianHG_DDR3_CONTROLLER open source DDR3 controller. NEW v1.60.

[BrianHG]

Here is the true operation tuning time when parameter 'SKIP_PUP_TIMER = 0'.  This is normal operation, however, to simulate this, it takes a few minutes as Modelsim needs to simulate 0.2seconds of real time since we now go through the entire required power-up delays listed in the DDR3 specifications.



As you can see here, the 'phase_step' is pulsed for 2000ns.  The time after that is 3000ns before the read check takes place.  So, if you step at the rise of 'phase_step, your PLL needs to have a new valid output phase within 5000ns of the step.

[SpacedCowboy]

As it stands it’s running off CLK_IN, and 90ns @ ~25MHz is approx 2 clocks. Since the PLLs are going to be well up and running before the phase change will occur, I could slave the logic off the 100MHz clock instead, then there’s plenty of time

And then I saw your 2nd post… 5000ns is indeed plenty

[BrianHG]

My tuning section clk is actually hard tied the DDR3_CK / 4 output.

[SpacedCowboy]

So I'm not 100% sure, but I think this looks good ?

I can see the WRITE commands, eg:

Code: [Select]

WRITE @ DQS= bank = 7 row = 0004 col = 00000000 data = 8888)
... corresponding to later READ data, eg:

Code: [Select]

READ @ DQS= bank = 7 row = 0004 col = 00000000 data = 8888

[BrianHG]

Looks ok.  If the tuning wasn't doing anything, the sim would get stuck in an infinite loop during initialization.  No reads or writes would take place.

IE: Break the phase step signal to the PLL, make the PLL's input set to '0' and see if the sim will run.

Also, all you had to do was compare the output when running with 'FPGA_VENDOR = Altera' mode.

To be absolutely sure, I need to inspect the waveform during tuning.

If this checks out, then all you have left is the DDR_IO primitive to be inserted into my IO port file.
Your best bet is to read my code's Cyclone DDR_IO primitive and replicate in Gowin.  So long as you can specify the data input read clock and data output write clock, you should be able to get this to work.
(Oh, and dont forget proper .SDC file restraints.  The values I have may be different for Gowin on the output side as the delays have been tuned to maximize Altera's FMAX.)

[BrianHG]

Another test would be to invert the phase up/down input and see if the startup matches the Altera startup.

[SpacedCowboy]

Looks ok.  If the tuning wasn't doing anything, the sim would get stuck in an infinite loop during initialization.  No reads or writes would take place.

IE: Break the phase step signal to the PLL, make the PLL's input set to '0' and see if the sim will run.

Yep, if I set ".phase_step(1'b0)" in the BrianHG_DDR3_PLL (...) instantiation, I get an endless series of read data 0, read data 1 lines. Presumably the tuning.

Also, all you had to do was compare the output when running with 'FPGA_VENDOR = Altera' mode.

That's ... totally fair.

To be absolutely sure, I need to inspect the waveform during tuning.

So tomorrow, I'll fork your repo on GitHub into my 'Spaced-cowboy' GitHub account, and upload the code that I have here. Then it's a 'git clone' away.

If this checks out, then all you have left is the DDR_IO primitive to be inserted into my IO port file.
Your best bet is to read my code's Cyclone DDR_IO primitive and replicate in Gowin.  So long as you can specify the data input read clock and data output write clock, you should be able to get this to work.

This is at line 358 of BrianHG_DDR3_IO_PORT_ALTERA.sv, right ?

The DDR facilities of the Gowin parts seem a bit primitive compared to the Altera ones, and ODDR is a bit strange - there's a TX input that I haven't figured out yet - might be a clock-enable signal perhaps. The PDF documentation isn't what I'd call 'great' (see attached), but there's probably some use of DDR outputs in their example code that I can go look at and figure out what the parameters do.

(Oh, and dont forget proper .SDC file restraints.  The values I have may be different for Gowin on the output side as the delays have been tuned to maximize Altera's FMAX.)

And here do the dragons dwell....

Another test would be to invert the phase up/down input and see if the startup matches the Altera startup.

I'll give that a go tomorrow. Off to bed

Thanks for all the help Brian, this is actually going a lot smoother than I thought it would...

[SpacedCowboy]

Actually maybe ODDR doubles as IODDR, and TX sets the direction, since Q1 can connect to an IOBUF (which is bidirectional).

[BrianHG]

Looks like the TX is the output enable.
Then you have the Q output and the Q for the OE.

something like :

assign pin_name = Q1 ? Q0 : 1'bz ;

Where as the DDR input primitive would also wire to 'pin_name'.

Anyways, look at my 'BrianHG_DDR3_IO_PORT_ALTERA.sv'.

Remove lines 228 through 358 while concentrating on lines 360 through 461.
(note that lines 228 through 358 use HW DIFFERENTIAL drivers for the DDR3_CK and DDR3 DQS pins.  360 through 461 uses a software emulated dumb differential driver.)

Altera documentation:
https://www.intel.com/programmable/technical-pdfs/683148.pdf

[SpacedCowboy]

To be absolutely sure, I need to inspect the waveform during tuning.

Ok, so the repo is forked and the code changes uploaded, the fork is here

Another test would be to invert the phase up/down input and see if the startup matches the Altera startup.

Did that this morning, and all the phases seem to align between the Gowin and Altera PLL instances

This video has me selecting phase_done (rising edge) as a convenient click-to-move-to-point, and one clock subsequent to that signal going high is when the Altera PLL has shifted to the new phase.

• The dark blue trace is the DDR3_CLK_RDQ clock, and it aligns with the dark blue trace from the Gowin PLLs below (clk_ddrRead).
• The 'phase' counter above is actually from the Gowin module and represents what is sent to the Gowin PLL as the phase to use.
  

[BrianHG]

To be absolutely sure, I need to inspect the waveform during tuning.

Ok, so the repo is forked and the code changes uploaded, the fork is here

That's nice and all, but, why did you use Gowin's black box ' pll_ddr1 & 2 ' instead of directly instantiating 'rPLL' yourself directly?
Also, why didn't you feed into the rPLL my 'CLK_KHZ_IN/1000', 'CLK_IN_MULT-1' and 'CLK_IN_DIV-1' ?
Without those 3, you cannot change the frequency of my DDR3 controller.

(Cant see your video, it is a privileged page.)

*** Additional: Did you try a power-up sequence with the parameter 'SKIP_PUP_TIMER = 0'?  In this configuration, I pulse the phase_step really slow, and, there should be no DDR3 RST_N & CKE warnings from Micron's DDR3 model.  Note that this will take a few minutes to run and it will appear to be cycling through the configuration non-stop at the end before the read/write tests begin.

[BrianHG]

Thanks for all the help Brian, this is actually going a lot smoother than I thought it would...

Well, I designed my controller to work on bottom end 20 year old FPGAs, those without IO delay functions, relying 100% on PLL phase capabilities, plus, multiple configurable proper end-to-end simulations test-benchs.  It's as easy as it can get with the negative that you cannot stack too many DDR3s in parallel without dropping the clock rate, or taking care about your trace lengths.  My guess is that I would feel safe with maximum a single laptop DDR3 sodim module with 4 ram chips, 64 bit DDR3 bus running at 400MHz, 800mtps.  2 chips and I would not go beyond 600MHz, 1.2gtps.

Though, I do not know Gowin's IO port's capabilities.

[SpacedCowboy]

There's a couple of reasons why I was using the drop-in IP from Gowin

• I didn't think it made much difference - I wasn't aware of the significance of 'CLK_KHZ_IN/1000', 'CLK_IN_MULT-1' and 'CLK_IN_DIV-1' and that you were trying to make that a cross-architecture feature.
  
  You'd also said "Do not worry about how I managed/strong armed Altera's PLLs into doing what I like from provided parameters.  With Gowin, it is ok if you are left with no choice but to provide only a selection of source clocks and output clocks per-generated by Gowin's IP tool" and I thought those parameters where part of that setup process.
  
  
• It was also kind of handy, when I was re-generating the PLLs that it would just overwrite the same files and I'm done - just hit up-arrow/return in modelsim to see any change.

That said, now that I'm happier with the end-result, it's a reasonable ask. I'll see if I can figure out how to do it.

The video not being visible is weird - it's fine here on the internal network, my ISP must be doing something to filter external access because there's no protection on the page at all.I've converted it into a GIF, so perhaps it'll work when uploaded instead.

[edit] No such luck. I've loaded it onto Imgur, and if this doesn't work, sod it.

I had tried a sequence with  'SKIP_PUP_TIMER = 0', I did it last night, but maybe because it was late, I looked in BrianHG_DDR3_PHY_SEQ_v16.sv not BrianHG_DDR3_PHY_SEQ_v16_tb.sv for the parameter  ... I thought it was running faster than you'd mentioned.

So, running it with 'SKIP_PUP_TIMER = 0' for real this time, it does show a couple of warnings regarding RST_N - which presumably it oughtn't. The log is attached (as a .zip) but the offending lines are:

Code: [Select]

# ** Error: (vsim-8630) ddr3.v(542): Infinity results from division operation.
#
# ** Error: (vsim-8630) ddr3.v(543): Infinity results from division operation.
#
# ** Error: (vsim-8630) ddr3.v(544): Infinity results from division operation.
#
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.reset at time 7697500.0 ps WARNING: 200 us is required before RST_N goes inactive.
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.cmd_task at time 8698750.0 ps WARNING: 500 us is required after RST_N goes inactive before CKE goes active.

So, something else to look at there.

[BrianHG]

There's a couple of reasons why I was using the drop-in IP from Gowin

• I didn't think it made much difference - I wasn't aware of the significance of 'CLK_KHZ_IN/1000', 'CLK_IN_MULT-1' and 'CLK_IN_DIV-1' and that you were trying to make that a cross-architecture feature.
  
  You'd also said "Do not worry about how I managed/strong armed Altera's PLLs into doing what I like from provided parameters.  With Gowin, it is ok if you are left with no choice but to provide only a selection of source clocks and output clocks per-generated by Gowin's IP tool" and I thought those parameters where part of that setup process.
  
  
• It was also kind of handy, when I was re-generating the PLLs that it would just overwrite the same files and I'm done - just hit up-arrow/return in modelsim to see any change.

That said, now that I'm happier with the end-result, it's a reasonable ask. I'll see if I can figure out how to do it.

Ok, when I said strong-armed, I meant the stupid 'localparam Altera_Dummy_String' I had to create as this is a parameter limitation bug in Altera's 20 year old pll primitive and their HDL design team's issues expecting a number encoded as a string embedded into a 64 bit integer, and in more than one place.  (Do not ask, I do not want to go into this BS hell.)

Yes, I did say begin with the simple fixed 400MHz pll, however, you now must tune Gowin's PLL to my clocks and multipliers.  The rest of my DDR3 controller uses these figures to tune all the DDR3 delays, like the number of clocks between RAC/CAS/ refresh clock cycle timing based on selected parameter 'DDR3_SPEED_GRADE'.  All my power-up timers also tune themselves based these PLL settings as well.

The video not being visible is weird - it's fine here on the internal network, my ISP must be doing something to filter external access because there's no protection on the page at all. I've converted it into a GIF, so perhaps it'll work when uploaded instead.

I had tried a sequence with  'SKIP_PUP_TIMER = 0', I did it last night, but maybe because it was late, I looked in BrianHG_DDR3_PHY_SEQ_v16.sv not BrianHG_DDR3_PHY_SEQ_v16_tb.sv for the parameter  ... I thought it was running faster than you'd mentioned.

So, running it with 'SKIP_PUP_TIMER = 0' for real this time, it does show a couple of warnings regarding RST_N - which presumably it oughtn't. The log is attached (as a .zip) but the offending lines are:

Code: [Select]

# ** Error: (vsim-8630) ddr3.v(542): Infinity results from division operation.
#
# ** Error: (vsim-8630) ddr3.v(543): Infinity results from division operation.
#
# ** Error: (vsim-8630) ddr3.v(544): Infinity results from division operation.
#
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.reset at time 7697500.0 ps WARNING: 200 us is required before RST_N goes inactive.
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.cmd_task at time 8698750.0 ps WARNING: 500 us is required after RST_N goes inactive before CKE goes active.


So, something else to look at there.

Looking at your attached .txt file, your sim begins at line 2548.  Reading from there:

Code: [Select]

# restart -force
# Loading sv_std.std
# Loading work.BrianHG_DDR3_PHY_SEQ_v16_tb
# Loading work.BrianHG_DDR3_PLL
# Loading work.BrianHG_DDR3_PHY_SEQ_v16
# Loading work.BrianHG_DDR3_GEN_tCK
# Loading work.BrianHG_DDR3_CMD_SEQUENCER_v16
# Loading work.gowin_ddr_clocking
# Loading work.pll_ddr1
# Loading work.rPLL
# Loading work.pll_ddr2
# Loading work.BrianHG_DDR3_IO_PORT_ALTERA
# ** Warning: (vsim-3017) BrianHG_DDR3_PLL.sv(548): [TFMPC] - Too few port connections. Expected 11, found 10.
#
#         Region: /BrianHG_DDR3_PHY_SEQ_v16_tb/DUT_DDR3_PLL/genblk6/gowin_ddr_clocks
# ** Warning: (vsim-3722) BrianHG_DDR3_PLL.sv(548): [TFMPC] - Missing connection for port 'fdly'.
#
# ** Warning: (vsim-3839) BrianHG_DDR3_PHY_SEQ_v16.sv(450): Variable '/BrianHG_DDR3_PHY_SEQ_v16_tb/DUT_PHY_SEQ/CMD_TXB', driven via a port connection, is multiply driven. See BrianHG_DDR3_PHY_SEQ_v16.sv(480).
#
#         Region: /BrianHG_DDR3_PHY_SEQ_v16_tb/DUT_PHY_SEQ
# ** Warning: (vsim-3839) BrianHG_DDR3_PHY_SEQ_v16.sv(148): Variable '/BrianHG_DDR3_PHY_SEQ_v16_tb/DUT_PHY_SEQ/SEQ_RDATA_VECT_OUT', driven via a port connection, is multiply driven. See BrianHG_DDR3_PHY_SEQ_v16.sv(480).
#
#         Region: /BrianHG_DDR3_PHY_SEQ_v16_tb/DUT_PHY_SEQ
# ** Warning: (vsim-3839) BrianHG_DDR3_PHY_SEQ_v16.sv(147): Variable '/BrianHG_DDR3_PHY_SEQ_v16_tb/DUT_PHY_SEQ/SEQ_RDATA', driven via a port connection, is multiply driven. See BrianHG_DDR3_PHY_SEQ_v16.sv(480).
#
#         Region: /BrianHG_DDR3_PHY_SEQ_v16_tb/DUT_PHY_SEQ
# ** Warning: (vsim-3839) BrianHG_DDR3_PHY_SEQ_v16.sv(146): Variable '/BrianHG_DDR3_PHY_SEQ_v16_tb/DUT_PHY_SEQ/SEQ_RDATA_RDY_t', driven via a port connection, is multiply driven. See BrianHG_DDR3_PHY_SEQ_v16.sv(480).
#
#         Region: /BrianHG_DDR3_PHY_SEQ_v16_tb/DUT_PHY_SEQ
# run -all
# ** Warning: *****************************
#    Time: 0 ps  Scope: BrianHG_DDR3_PHY_SEQ_v16_tb.DUT_DDR3_PLL File: BrianHG_DDR3_PLL.sv Line: 616
# ** Warning: *** BrianHG_DDR3_PLL Info ***
#    Time: 0 ps  Scope: BrianHG_DDR3_PHY_SEQ_v16_tb.DUT_DDR3_PLL File: BrianHG_DDR3_PLL.sv Line: 617
# ** Warning: *********************************************
#    Time: 0 ps  Scope: BrianHG_DDR3_PHY_SEQ_v16_tb.DUT_DDR3_PLL File: BrianHG_DDR3_PLL.sv Line: 618
# ** Warning: ***      CLK_IN           =    50 MHz.     ***
#    Time: 0 ps  Scope: BrianHG_DDR3_PHY_SEQ_v16_tb.DUT_DDR3_PLL File: BrianHG_DDR3_PLL.sv Line: 619
# ** Warning: ***      DDR3_RDQ/WDQ     =   800 MTPS.    ***
#    Time: 0 ps  Scope: BrianHG_DDR3_PHY_SEQ_v16_tb.DUT_DDR3_PLL File: BrianHG_DDR3_PLL.sv Line: 620
# ** Warning: ***      DDR3_CLK/RDQ/WDQ =   400 MHz.     ***
#    Time: 0 ps  Scope: BrianHG_DDR3_PHY_SEQ_v16_tb.DUT_DDR3_PLL File: BrianHG_DDR3_PLL.sv Line: 621
# ** Warning: ***      DDR3_WDQ_PHASE   =   270 degrees. ***
#    Time: 0 ps  Scope: BrianHG_DDR3_PHY_SEQ_v16_tb.DUT_DDR3_PLL File: BrianHG_DDR3_PLL.sv Line: 622
# ** Warning: *** True DDR3_WDQ_PHASE   =  1875 ps.     ***
#    Time: 0 ps  Scope: BrianHG_DDR3_PHY_SEQ_v16_tb.DUT_DDR3_PLL File: BrianHG_DDR3_PLL.sv Line: 623
# ** Warning: ***      DDR3_RDQ_PHASE   =     0 degrees. ***
#    Time: 0 ps  Scope: BrianHG_DDR3_PHY_SEQ_v16_tb.DUT_DDR3_PLL File: BrianHG_DDR3_PLL.sv Line: 624
# ** Warning: *** True DDR3_RDQ_PHASE   =  0000 ps.     ***
#    Time: 0 ps  Scope: BrianHG_DDR3_PHY_SEQ_v16_tb.DUT_DDR3_PLL File: BrianHG_DDR3_PLL.sv Line: 625
# ** Warning: ***      CMD_CLK          =   200 MHz.     ***
#    Time: 0 ps  Scope: BrianHG_DDR3_PHY_SEQ_v16_tb.DUT_DDR3_PLL File: BrianHG_DDR3_PLL.sv Line: 626
# ** Warning: ***      DDR3_CLK_50      =   200 MHz.     ***
#    Time: 0 ps  Scope: BrianHG_DDR3_PHY_SEQ_v16_tb.DUT_DDR3_PLL File: BrianHG_DDR3_PLL.sv Line: 627
# ** Warning: ***      DDR3_CLK_25      =   100 MHz.     ***
#    Time: 0 ps  Scope: BrianHG_DDR3_PHY_SEQ_v16_tb.DUT_DDR3_PLL File: BrianHG_DDR3_PLL.sv Line: 628
# ** Warning: *********************************************
#    Time: 0 ps  Scope: BrianHG_DDR3_PHY_SEQ_v16_tb.DUT_DDR3_PLL File: BrianHG_DDR3_PLL.sv Line: 629
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.file_io_open: at time                    0 WARNING: no +model_data option specified, using /tmp.
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file: at time 0 INFO: opening /tmp/BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file.0.
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file: at time 0 INFO: opening /tmp/BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file.1.
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file: at time 0 INFO: opening /tmp/BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file.2.
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file: at time 0 INFO: opening /tmp/BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file.3.
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file: at time 0 INFO: opening /tmp/BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file.4.
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file: at time 0 INFO: opening /tmp/BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file.5.
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file: at time 0 INFO: opening /tmp/BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file.6.
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file: at time 0 INFO: opening /tmp/BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file.7.
# ** Error: (vsim-8630) ddr3.v(542): Infinity results from division operation.
#
# ** Error: (vsim-8630) ddr3.v(543): Infinity results from division operation.
#
# ** Error: (vsim-8630) ddr3.v(544): Infinity results from division operation.
#
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.cmd_task: at time 1205718750.0 ps INFO: Load Mode 2
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.cmd_task: at time 1205718750.0 ps INFO: Load Mode 2 Partial Array Self Refresh = Bank 0-7
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.cmd_task: at time 1205718750.0 ps INFO: Load Mode 2 CAS Write Latency =           5
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.cmd_task: at time 1205718750.0 ps INFO: Load Mode 2 Auto Self Refresh = Disabled
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.cmd_task: at time 1205718750.0 ps INFO: Load Mode 2 Self Refresh Temperature = Normal
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.cmd_task: at time 1205718750.0 ps INFO: Load Mode 2 Dynamic ODT = Disabled


Ok, the first 6 warnings should not be there.
The first 2 seem to be associated with your Gowin pll.

The next 4 for some reason comes from my code, and it should not.
Did you modify my test bench HDL?

The next set of warnings are just a print-out from my PLL.  I guess I should have used $display instead of $warning, though the $warning show better in Quartus' compiler.

The divide by zero error is in Micron's DDR3 model, I tend to ignore.

Then we reach the load MRS2, without the RST_N or CKE required delay warning, and everything runs from there correctly.


Again, read your 'Gowin/pll_ddr#/pll_ddr#.v' to see what's inside those black boxes.
The only thing you might have trouble with is configuring the adjustable 270deg in PLL #2 based on my source parameter.

Also don't forget to pass my string  .FPGA_FAMILY    to     rpll_inst.DEVICE, and in your sims set my .FPGA_FAMILY to "GW2A-18".

(it looks like you will have to strong-arm Gowin's 'rpll_inst.PSDA_SEL = "1100";' as it looks to be a string, not a binary number, though it looks easy enough.)

From my system PLL module's parameter DDR3_WDQ_PHASE, multiply by 16, divide by 360, shrink to 4 bit logic, then make a string = 0 through 15 "0000", "0001", "0010",...  Remember, this must all be done as localparams.

[SpacedCowboy]

Looking at your attached .txt file, your sim begins at line 2548.  Reading from there:

Code: [Select]

# restart -force
# Loading sv_std.std
# Loading work.BrianHG_DDR3_PHY_SEQ_v16_tb
# Loading work.BrianHG_DDR3_PLL
# Loading work.BrianHG_DDR3_PHY_SEQ_v16
# Loading work.BrianHG_DDR3_GEN_tCK
# Loading work.BrianHG_DDR3_CMD_SEQUENCER_v16
# Loading work.gowin_ddr_clocking
# Loading work.pll_ddr1
# Loading work.rPLL
# Loading work.pll_ddr2
# Loading work.BrianHG_DDR3_IO_PORT_ALTERA
# ** Warning: (vsim-3017) BrianHG_DDR3_PLL.sv(548): [TFMPC] - Too few port connections. Expected 11, found 10.
#
#         Region: /BrianHG_DDR3_PHY_SEQ_v16_tb/DUT_DDR3_PLL/genblk6/gowin_ddr_clocks
# ** Warning: (vsim-3722) BrianHG_DDR3_PLL.sv(548): [TFMPC] - Missing connection for port 'fdly'.
#
# ** Warning: (vsim-3839) BrianHG_DDR3_PHY_SEQ_v16.sv(450): Variable '/BrianHG_DDR3_PHY_SEQ_v16_tb/DUT_PHY_SEQ/CMD_TXB', driven via a port connection, is multiply driven. See BrianHG_DDR3_PHY_SEQ_v16.sv(480).
#
#         Region: /BrianHG_DDR3_PHY_SEQ_v16_tb/DUT_PHY_SEQ
# ** Warning: (vsim-3839) BrianHG_DDR3_PHY_SEQ_v16.sv(148): Variable '/BrianHG_DDR3_PHY_SEQ_v16_tb/DUT_PHY_SEQ/SEQ_RDATA_VECT_OUT', driven via a port connection, is multiply driven. See BrianHG_DDR3_PHY_SEQ_v16.sv(480).
#
#         Region: /BrianHG_DDR3_PHY_SEQ_v16_tb/DUT_PHY_SEQ
# ** Warning: (vsim-3839) BrianHG_DDR3_PHY_SEQ_v16.sv(147): Variable '/BrianHG_DDR3_PHY_SEQ_v16_tb/DUT_PHY_SEQ/SEQ_RDATA', driven via a port connection, is multiply driven. See BrianHG_DDR3_PHY_SEQ_v16.sv(480).
#
#         Region: /BrianHG_DDR3_PHY_SEQ_v16_tb/DUT_PHY_SEQ
# ** Warning: (vsim-3839) BrianHG_DDR3_PHY_SEQ_v16.sv(146): Variable '/BrianHG_DDR3_PHY_SEQ_v16_tb/DUT_PHY_SEQ/SEQ_RDATA_RDY_t', driven via a port connection, is multiply driven. See BrianHG_DDR3_PHY_SEQ_v16.sv(480).
#
#         Region: /BrianHG_DDR3_PHY_SEQ_v16_tb/DUT_PHY_SEQ
# run -all
# ** Warning: *****************************
#    Time: 0 ps  Scope: BrianHG_DDR3_PHY_SEQ_v16_tb.DUT_DDR3_PLL File: BrianHG_DDR3_PLL.sv Line: 616
# ** Warning: *** BrianHG_DDR3_PLL Info ***
#    Time: 0 ps  Scope: BrianHG_DDR3_PHY_SEQ_v16_tb.DUT_DDR3_PLL File: BrianHG_DDR3_PLL.sv Line: 617
# ** Warning: *********************************************
#    Time: 0 ps  Scope: BrianHG_DDR3_PHY_SEQ_v16_tb.DUT_DDR3_PLL File: BrianHG_DDR3_PLL.sv Line: 618
# ** Warning: ***      CLK_IN           =    50 MHz.     ***
#    Time: 0 ps  Scope: BrianHG_DDR3_PHY_SEQ_v16_tb.DUT_DDR3_PLL File: BrianHG_DDR3_PLL.sv Line: 619
# ** Warning: ***      DDR3_RDQ/WDQ     =   800 MTPS.    ***
#    Time: 0 ps  Scope: BrianHG_DDR3_PHY_SEQ_v16_tb.DUT_DDR3_PLL File: BrianHG_DDR3_PLL.sv Line: 620
# ** Warning: ***      DDR3_CLK/RDQ/WDQ =   400 MHz.     ***
#    Time: 0 ps  Scope: BrianHG_DDR3_PHY_SEQ_v16_tb.DUT_DDR3_PLL File: BrianHG_DDR3_PLL.sv Line: 621
# ** Warning: ***      DDR3_WDQ_PHASE   =   270 degrees. ***
#    Time: 0 ps  Scope: BrianHG_DDR3_PHY_SEQ_v16_tb.DUT_DDR3_PLL File: BrianHG_DDR3_PLL.sv Line: 622
# ** Warning: *** True DDR3_WDQ_PHASE   =  1875 ps.     ***
#    Time: 0 ps  Scope: BrianHG_DDR3_PHY_SEQ_v16_tb.DUT_DDR3_PLL File: BrianHG_DDR3_PLL.sv Line: 623
# ** Warning: ***      DDR3_RDQ_PHASE   =     0 degrees. ***
#    Time: 0 ps  Scope: BrianHG_DDR3_PHY_SEQ_v16_tb.DUT_DDR3_PLL File: BrianHG_DDR3_PLL.sv Line: 624
# ** Warning: *** True DDR3_RDQ_PHASE   =  0000 ps.     ***
#    Time: 0 ps  Scope: BrianHG_DDR3_PHY_SEQ_v16_tb.DUT_DDR3_PLL File: BrianHG_DDR3_PLL.sv Line: 625
# ** Warning: ***      CMD_CLK          =   200 MHz.     ***
#    Time: 0 ps  Scope: BrianHG_DDR3_PHY_SEQ_v16_tb.DUT_DDR3_PLL File: BrianHG_DDR3_PLL.sv Line: 626
# ** Warning: ***      DDR3_CLK_50      =   200 MHz.     ***
#    Time: 0 ps  Scope: BrianHG_DDR3_PHY_SEQ_v16_tb.DUT_DDR3_PLL File: BrianHG_DDR3_PLL.sv Line: 627
# ** Warning: ***      DDR3_CLK_25      =   100 MHz.     ***
#    Time: 0 ps  Scope: BrianHG_DDR3_PHY_SEQ_v16_tb.DUT_DDR3_PLL File: BrianHG_DDR3_PLL.sv Line: 628
# ** Warning: *********************************************
#    Time: 0 ps  Scope: BrianHG_DDR3_PHY_SEQ_v16_tb.DUT_DDR3_PLL File: BrianHG_DDR3_PLL.sv Line: 629
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.file_io_open: at time                    0 WARNING: no +model_data option specified, using /tmp.
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file: at time 0 INFO: opening /tmp/BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file.0.
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file: at time 0 INFO: opening /tmp/BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file.1.
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file: at time 0 INFO: opening /tmp/BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file.2.
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file: at time 0 INFO: opening /tmp/BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file.3.
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file: at time 0 INFO: opening /tmp/BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file.4.
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file: at time 0 INFO: opening /tmp/BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file.5.
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file: at time 0 INFO: opening /tmp/BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file.6.
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file: at time 0 INFO: opening /tmp/BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.open_bank_file.7.
# ** Error: (vsim-8630) ddr3.v(542): Infinity results from division operation.
#
# ** Error: (vsim-8630) ddr3.v(543): Infinity results from division operation.
#
# ** Error: (vsim-8630) ddr3.v(544): Infinity results from division operation.
#
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.cmd_task: at time 1205718750.0 ps INFO: Load Mode 2
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.cmd_task: at time 1205718750.0 ps INFO: Load Mode 2 Partial Array Self Refresh = Bank 0-7
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.cmd_task: at time 1205718750.0 ps INFO: Load Mode 2 CAS Write Latency =           5
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.cmd_task: at time 1205718750.0 ps INFO: Load Mode 2 Auto Self Refresh = Disabled
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.cmd_task: at time 1205718750.0 ps INFO: Load Mode 2 Self Refresh Temperature = Normal
# BrianHG_DDR3_PHY_SEQ_v16_tb.sdramddr3_0.cmd_task: at time 1205718750.0 ps INFO: Load Mode 2 Dynamic ODT = Disabled


Ok, the first 6 warnings should not be there.
The first 2 seem to be associated with your Gowin pll.

Yep - it's the fdly [3:0] input to gowin_ddr_clocking. Apparently I forgot to take out the declaration of the input port at the top of the gowin_ddr_clocking module. We're never changing the delay on the output port, so I'm hardwiring those to 4'b0 and there was no need to pass it in. Fixed locally.

The next 4 for some reason comes from my code, and it should not.
Did you modify my test bench HDL?

Not that I'm aware of. The only changes that ought to have been made in the test bench are at the top of the file wrt the PLL instance. You can see the diffs here on GitHub

running 'git diff' on the local version of what's at GitHub is just showing me the line where I set SKIP_PUP_TIMER to 0, so I don't think there's any changes. Could be a Modelsim version ? I'm running

Code: [Select]


ModelSim ALTERA STARTER EDITION 10.3c

Revision: 2014.09
Date: Sep 20 2014



The next set of warnings are just a print-out from my PLL.  I guess I should have used $display instead of $warning, though the $warning show better in Quartus' compiler.

The divide by zero error is in Micron's DDR3 model, I tend to ignore.

Then we reach the load MRS2, without the RST_N or CKE error, and everything runs from there correctly.

Again, read your 'Gowin/pll_ddr#/pll_ddr#.v' to see what's inside those black boxes.
The only thing you might have trouble with is configuring the adjustable 270deg in PLL #2 based on my source parameter.

Also don't forget to pass my string  .FPGA_FAMILY    to     rpll_inst.DEVICE, and in your sims set my .FPGA_FAMILY to "GW2A-18".

Yep, but if I'm varying the input clock, I'm presumably going to have to do maths on the parameters to get the various divisors set up. Shouldn't be impossible, just means I need a better understanding of what your constants mean and how the PLL parameters work to configure it than I do right now.

Speaking of which:

Code: [Select]

parameter int        CLK_KHZ_IN              = 50000,          // PLL source input clock frequency in KHz.
parameter int        CLK_IN_MULT             = 32,             // Multiply factor to generate the DDR MTPS speed divided by 2.
parameter int        CLK_IN_DIV              = 4,              // Divide factor.  When CLK_KHZ_IN is 25000,50000,75000,100000,125000,150000, use 2,4,6,8,10,12.

The first and third are fairly obvious, the second seems to scale the wrong way. 50 x 32 = 1600. Isn't the DDR3 clock running at 400MHZ, so with DDR it is 800MT/s, so 1600 is x2 not /2 ? Maybe just perspective, and probably doesn't matter as long as it's consistent
 
So I think what you're saying is that from the above 3 figures, I ought to be deriving any of the numerical PLL parameters such as FBDIV_SEL in

Code: [Select]

defparam rpll_inst.FCLKIN = "50";
defparam rpll_inst.DYN_IDIV_SEL = "false";
defparam rpll_inst.IDIV_SEL = 0;
defparam rpll_inst.DYN_FBDIV_SEL = "false";
defparam rpll_inst.FBDIV_SEL = 7;
defparam rpll_inst.DYN_ODIV_SEL = "false";
defparam rpll_inst.ODIV_SEL = 2;
defparam rpll_inst.PSDA_SEL = "0000";
defparam rpll_inst.DYN_DA_EN = "true";
defparam rpll_inst.DUTYDA_SEL = "1000";
defparam rpll_inst.CLKOUT_FT_DIR = 1'b1;
defparam rpll_inst.CLKOUTP_FT_DIR = 1'b1;
defparam rpll_inst.CLKOUT_DLY_STEP = 0;
defparam rpll_inst.CLKOUTP_DLY_STEP = 0;
defparam rpll_inst.CLKFB_SEL = "internal";
defparam rpll_inst.CLKOUT_BYPASS = "false";
defparam rpll_inst.CLKOUTP_BYPASS = "false";
defparam rpll_inst.CLKOUTD_BYPASS = "false";
defparam rpll_inst.DYN_SDIV_SEL = 2;
defparam rpll_inst.CLKOUTD_SRC = "CLKOUT";
defparam rpll_inst.CLKOUTD3_SRC = "CLKOUT";
defparam rpll_inst.DEVICE = "GW2A-18";


which, as I said, doesn't look impossible. And certainly updating the device to be passed through is fine too.

Is there any difference between doing it as "type (parameters) instance (signals)" rather than as "type instance (signals) defparams" ?

[BrianHG]

Yep, but if I'm varying the input clock, I'm presumably going to have to do maths on the parameters to get the various divisors set up. Shouldn't be impossible, just means I need a better understanding of what your constants mean and how the PLL parameters work to configure it than I do right now.

What math?
I gave you the math, you just had to...

defparam rpll_inst.FCLKIN = (CLK_KHZ_IN /1000);
defparam rpll_inst.IDIV_SEL = (CLK_IN_DIV-1);
defparam rpll_inst.FBDIV_SEL = (CLK_IN_MULT-1);

How hard was that?

Also:

localparam string gowin_phase[0:15] ='{"0000","0001","0010",...,"1111"};
localparam  phase_set = (DDR3_WDQ_PHASE * 16 / 360);

and
defparam rpll_inst.PSDA_SEL = gowin_phase_string[phase_set[3:0]];

Only that you might need to remove the 'string' in the first localparam as this is the same hard-handed manipulation I has to use to make Altera's PLL work as they have some input parameters which arent true integers or strings.

You will need to test everything in the first simple PLL testbench.

Is there any difference between doing it as "type (parameters) instance (signals)" rather than as "type instance (signals) defparams" ?

No difference.

If you like, you may find the common remainder of the CLK_IN_DIV & CLK_IN_MULT to divide both by that number if you like.  I always prefer having the CLK_IN_DIV set to at least 2 as this makes a true 50:50 source reference as the rise and fall times of your source crystal oscillator are now ignored.  You are now relying on the rise-to-next-rise of the crystal feeding your PLL source.

[BrianHG]

Clicking on 'Help / About Modelsim' gives me this:

[BrianHG]

Looking at the:
defparam rpll_inst.FCLKIN = "50";

Notice that the '50' is in quotes.
It may be another one of those BS fake string things.
We may need to strong-arm a phony string for that one too.

(If this is it, it isn't too bad.  Not until you take a look at LATTICE, they actually rely on some figures written in their 'COMMENTS', yes, inside their /* xxx=xxxMHz */ to make their PLL primitive design compile properly.  WTF?  How.  What kind of backwater design is this?)

[SpacedCowboy]

What math?
I gave you the math, you just had to...

defparam rpll_inst.FCLKIN = (CLK_KHZ_IN /1000);
defparam rpll_inst.IDIV_SEL = (CLK_IN_DIV-1);
defparam rpll_inst.FBDIV_SEL = (CLK_IN_MULT-1);

How hard was that?

Ok, I think the confusion is because those figures don't match what the tool produces for a 50MHz input clock (you can see the generated ones in my last post, it's 50/8/1 not 50/32/4) so it wasn't clear that you'd done it already, I thought you were just referring to some code of your own. Looking at the formula, and at the values (50,32,4) yours will of course produce a 400MHz clock, assuming there's no parameter-out-of-range in some in-the-middle calculation.

Also:

localparam string gowin_phase[0:15] ='{"0000","0001","0010",...,"1111"};
localparam  phase_set = (DDR3_WDQ_PHASE * 16 / 360);

and
defparam rpll_inst.PSDA_SEL = gowin_phase_string[phase_set[3:0]];

Only that you might need to remove the 'string' in the first localparam as this is the same hard-handed manipulation I has to use to make Altera's PLL work as they have some input parameters which arent true integers or strings.

You will need to test everything in the first simple PLL testbench.

Yep, the "joys" of stringification are heretofore unknown to me. I'm sure it'll be fun.

If you like, you may find the common remainder of the CLK_IN_DIV & CLK_IN_MULT to divide both by that number if you like.  I always prefer having the CLK_IN_DIV set to at least 2 as this makes a true 50:50 source reference as the rise and fall times of your source crystal oscillator are now ignored.  You are now relying on the rise-to-next-rise of the crystal feeding your PLL source.

That makes sense.

Looks like my modelsim is a little out of date as well - I can rectify that.

Looking at the:
defparam rpll_inst.FCLKIN = "50";

Notice that the '50' is in quotes.
It may be another one of those BS fake string things.
We may need to strong-arm a phony string for that one too.

(If this is it, it isn't too bad.  Not until you take a look at LATTICE, they actually rely on some figures written in their 'COMMENTS', yes, inside their /* xxx=xxxMHz */ to make their PLL primitive design compile properly.  WTF?  How.  What kind of backwater design is this?)

Is there really no function in verilog to accept an integer and convert it to a string format so it can be passed to a parameter ? Seems like a great potential extension to $sformat(), or some compile-time equivalent if it doesn't do it now.

And yes, that is nuts from Lattice. They also don't let you run Modelsim on a PC, serving the display to the nice big 3-monitor Mac using Microsoft Remote Desktop. They're just fine with Linux serving the display over X11 though...

[BrianHG]

Yep, but if I'm varying the input clock, I'm presumably going to have to do maths on the parameters to get the various divisors set up. Shouldn't be impossible, just means I need a better understanding of what your constants mean and how the PLL parameters work to configure it than I do right now.

Speaking of which:

Code: [Select]

parameter int        CLK_KHZ_IN              = 50000,          // PLL source input clock frequency in KHz.
parameter int        CLK_IN_MULT             = 32,             // Multiply factor to generate the DDR MTPS speed divided by 2.
parameter int        CLK_IN_DIV              = 4,              // Divide factor.  When CLK_KHZ_IN is 25000,50000,75000,100000,125000,150000, use 2,4,6,8,10,12.
The first and third are fairly obvious, the second seems to scale the wrong way. 50 x 32 = 1600. Isn't the DDR3 clock running at 400MHZ, so with DDR it is 800MT/s, so 1600 is x2 not /2 ? Maybe just perspective, and probably doesn't matter as long as it's consistent
 

  Is this what you were asking, FOUT = CLK_KHZ_IN  * CLK_IN_MULT / CLK_IN_DIV ?
50 * 32 / 4 = 400 MHz DDR3 CK.

So, if you want a 25Mhz source, just change the CLK_IN_DIV to 2 and you will get the same result.

[BrianHG]

Is there really no function in verilog to accept an integer and convert it to a string format so it can be passed to a parameter ? Seems like a great potential extension to $sformat(), or some compile-time equivalent if it doesn't do it now.

My quote:

Ok, when I said strong-armed, I meant the stupid 'localparam Altera_Dummy_String' I had to create as this is a parameter limitation bug in Altera's 20 year old pll primitive and their HDL design team's issues expecting a number encoded as a string embedded into a 64 bit integer, and in more than one place.  (Do not ask, I do not want to go into this BS hell.)

[SpacedCowboy]

Only got as far as the first two lines. I'm going to stop talking now, it's just getting embarrassing.

Ok, when I said strong-armed, I meant the stupid 'localparam Altera_Dummy_String' I had to create as this is a parameter limitation bug in Altera's 20 year old pll primitive and their HDL design team's issues expecting a number encoded as a string embedded into a 64 bit integer, and in more than one place.  (Do not ask, I do not want to go into this BS hell.)

 

[BrianHG]

This `ifdef shouldn't be necessary: Gowin/gowin_ddr_clocking.sv#L212

Did you try renaming the word 'string' to 'int' on this line: Gowin/gowin_ddr_clocking.sv#L65

This is what I had to do for Altera's PLL: BrianHG_DDR3/BrianHG_DDR3_PLL.sv#L193

And right after, I had to: BrianHG_DDR3/BrianHG_DDR3_PLL.sv#L278

Then I sent the 'DDR3_WDQ_PHASE_pss' into the PLL's parameters.
(Obviously, you would choose your own names._

The key factors in making both ModelSim and Quartus Prime happy when they compile, is that my dummy string array = '{"xxxx"} was a 'localparam int', and I made a new 'localparam param_to_be_sent' = dummy_string_array[sel#];.

If I did not go through this, even though ModelSim would accept the numbers for simulation, Quartus Prime would compile away, not saying a thing accepting a default "0" for that parameter.

------
Note, for generating your: Gowin/gowin_ddr_clocking.sv#L191 as a string, just copy & rename my int Altera_Dummy_String, except, trim it to 0-255 as this will probably be the frequency range input.

I have a feeling the "xxx' for the Gowin's frequency input may be because they might accept fractions, like "14.31818".  This might make things messy.  Not all FPGA compilers can accommodate 'real' floating point numbers.  Anyways, you will need to compile in Gowin a simple PLL clock to see if everything works out OK.

[SpacedCowboy]

This `ifdef shouldn't be necessary: Gowin/gowin_ddr_clocking.sv#L212

Did you try renaming the word 'string' to 'int' on this line: Gowin/gowin_ddr_clocking.sv#L65

This is what I had to do for Altera's PLL: BrianHG_DDR3/BrianHG_DDR3_PLL.sv#L193

And right after, I had to: BrianHG_DDR3/BrianHG_DDR3_PLL.sv#L278

Then I sent the 'DDR3_WDQ_PHASE_pss' into the PLL's parameters.
(Obviously, you would choose your own names._

The key factors in making both ModelSim and Quartus Prime happy when they compile, is that my dummy string array = '{"xxxx"} was a 'localparam int', and I made a new 'localparam param_to_be_sent' = dummy_string_array[sel#];.

If I did not go through this, even though ModelSim would accept the numbers for simulation, Quartus Prime would compile away, not saying a thing accepting a default "0" for that parameter.

Worked like a charm  - no synthesis errors and modelsim is happy too.

[edit] And having said that, and posted it, I immediately noticed that the write-clock duty-cycle isn't working, it's always falling at 0 now, not at (phase + 8/16). I'll look at that...

------
Note, for generating your: Gowin/gowin_ddr_clocking.sv#L191 as a string, just copy & rename my int Altera_Dummy_String, except, trim it to 0-255 as this will probably be the frequency range input.

I have a feeling the "xxx' for the Gowin's frequency input may be because they might accept fractions, like "14.31818".  This might make things messy.  Not all FPGA compilers can accommodate 'real' floating point numbers.  Anyways, you will need to compile in Gowin a simple PLL clock to see if everything works out OK.

I'm not actually having any problems with the input frequency for the clock, both simulation and synthesis seem quite happy with the expression as-is. I did back-port the gowin_ddr_clockings changes to the PLL test project and compare the Modelsim output for the Altera PLL with the Gowin one, and they both match up (see attached example).

The timing report in the Gowin synthesis seems to indicate that it understood the ask for the PLL outputs - again, see attached. I think this is a good indicator it's doing "the right thing".

[BrianHG]

Umm, shouldn't the 133MHz be 100MHz?
Maybe a typo in the divider.
Or if it's another output, maybe set it to 100MHz and place it on the first PLL reserving the write clock to PLL2.
(This is a plus in the future as you may use the 'delay' output primitive for the write data removing the need for the second PLL all together.  Though, get my DDR3 working firstly as intended.)

Also for the duty cycle, why not just hard write it in the parameter line just as if it came from the rPLL GUI generator?

Next, take a look at IO buffer primitives, then, the DDR primitives to drive those buffers.

DDR3_CK needs a differential LVDS output.  (May also be 2 output pin buffers, but this is lower quality.)
DDR3_DQS needs a differential LVDS bidirectional.  (May also be 2 bidir pin buffers, but this is lower quality.)