Xilinx Microblaze - bootloading from flash

[JohnnyMalaria]

Hi,

I'm struggling to understand how to configure a Microblaze-based Artix 7 design to successfully load a compiled C program from flash into DDR and run.

What I have managed to do successfully:

1. Create a Microblaze design that connects to my dev board's LEDs, buttons etc (Numato Mimas A7 Mini).
2. Run a Hello World program from the block RAM that responds to me pressing buttons etc.
3. Create a boot loader that does appear to load the SREC-formatted ELF for the Hello World program.

But at the end of the boot loading, it says it is calling the code at address 0x00000000 and my Hello World doesn't run.

The Hello World program is configured to use the DDR memory (using the MIG IP) which has a base address of 0x80000000.

I watched/read many tutorials but none are specific to my board and so I don't know what settings need to be changed, especially memory addresses etc.

Any suggestions for a clear explanation/tutorial on programming a Microblaze design into flash such that it boot-loads a program to run in DDR?

Thanks,

John.

[asmi]

Why would it call code at address zero?
Also - my memory is rusty and I don't have access to Vitis right now, but I seem to recall that you will need to update bootloader to specify offset in flash where the executable begins. Have you tried running your bootloader in debugger to see what's actually going on?

[JohnnyMalaria]

The boot loader runs in verbose mode so I can see via the COM terminal that it is successfully reading data from the flash. I modified the boot loader so that the flash offset was correct. Once the final SREC line has been read, the boot loader states it's running code starting at 0x0000000. That's the bit that's puzzling me but I haven't found a simple explanation.

BTW, I'm using Vivado 2018.1 with SDK.

[asmi]

The boot loader runs in verbose mode so I can see via the COM terminal that it is successfully reading data from the flash. I modified the boot loader so that the flash offset was correct. Once the final SREC line has been read, the boot loader states it's running code starting at 0x0000000. That's the bit that's puzzling me but I haven't found a simple explanation.

Have you tried changing the bootloader to instead jump to whatever address it unpacks the executable to (somewhere in DDR I guess)?

[asmi]

Oh, one more thing - make sure the application you are burning into the flash is set up to run from DDR, not from BRAM. Your point 2 seems to indicate that it's not set up as such. That's probably why it attempts to jump to address zero upon unpacking it into memory.

[JohnnyMalaria]

Thanks - I have two versions of the Hello World ELF. One for BRAM and one for DDR.

I did try hard-coding the bootloader to call 0x80000000 (DDR base address) but no joy.

[asmi]

Thanks - I have two versions of the Hello World ELF. One for BRAM and one for DDR.

I did try hard-coding the bootloader to call 0x80000000 (DDR base address) but no joy.

OK, I will do some experiments tonight once I'm done with the work for the day. I don't remember there being any serious issues getting it to work.

[JohnnyMalaria]

Thank you

My main problem is I don't understand exactly what Vivado's memory configuration files are, how they work and the relationships between bit, bin and elf files etc.

This is the board I have: https://numato.com/product/mimas-a7-mini-fpga-development-board/

I don't have a JTAG cable etc. Instead, the board supports Xilinx's virtual cable and I use Numato's Tenagra software that acts as the hardware server.

[asmi]

Thank you

My main problem is I don't understand exactly what Vivado's memory configuration files are, how they work and the relationships between bit, bin and elf files etc.

You can read about that stuff in UG908, chapter 3 and below. Basically it's an image of what is supposed to go to the flash.

This is the board I have: https://numato.com/product/mimas-a7-mini-fpga-development-board/

I don't have a JTAG cable etc. Instead, the board supports Xilinx's virtual cable and I use Numato's Tenagra software that acts as the hardware server.

This is why I always recommend avoiding boards which have these homebred configuration tools. As I understand, you can still buy Digilent's HS2 JTAG cable, connect it to your board and use Vivado's native tools for programming as there is a JTAG connector on your board, even if it's got non-standard pinout - if you lower the frequency, you should be able to get away with a bunch of flying jumper leads.

[SiliconWizard]

I don't have a JTAG cable etc. Instead, the board supports Xilinx's virtual cable and I use Numato's Tenagra software that acts as the hardware server.

This is why I always recommend avoiding boards which have these homebred configuration tools. As I understand, you can still buy Digilent's HS2 JTAG cable, connect it to your board and use Vivado's native tools for programming as there is a JTAG connector on your board, even if it's got non-standard pinout - if you lower the frequency, you should be able to get away with a bunch of flying jumper leads.

The Mimas A7 has a JTAG header. You  can absolutely use a Digilent HS2 with it (or any other JTAG probe). The only thing required is to make a small adapter because the header on board is different, but a few Dupont wires do the trick. That's what I've done with my Numato Saturn board.

[JohnnyMalaria]

Thanks for the link to the guide - a little light reading for the evening

Looks like I should buy the cable, too.

[SiliconWizard]

Thanks for the link to the guide - a little light reading for the evening

Looks like I should buy the cable, too.

I don't know for the Mimas A7, but for the Saturn, Numato's provided tool only allowed to program the Flash through USB - not to directly configure the FPGA (volatile). That was annoying because during development you want to frequently try things without wearing off the Flash, and most of all, Flash programming is slow. I think it was between 30s - 1min to program the Flash, whereas configuring the FPGA through JTAG with Digilent HS2 takes less than 2s!

[JohnnyMalaria]

I've made some progress (of sorts).

The bootloader loads a small amount of data (about 0x24 bytes) into BRAM at 0x00000000 and then loads more starting at the DDR base (0x80000000). Through the debugger, I can see the "Hello World" text string in the DDR so I'm confident it is loading okay.

The disassembly of the BRAM following bootloading is:

Code: [Select]

          _start:
00000000:   imm           -32768
00000004:   brai          0                 ; addr=0x80000000
          _vector_sw_exception:
00000008:   imm           -32768
0000000c:   brai          1628              ; addr=0x8000065c
          _vector_interrupt:
00000010:   imm           -32768
00000014:   brai          2112              ; addr=0x80000840
00000018:   add           r0 , r0 , r0
0000001c:   add           r0 , r0 , r0
          _vector_hw_exception:
00000020:   imm           -32768
00000024:   brai          812               ; addr=0x8000032c
And the start of the DDR is:

Code: [Select]

80000000:   imm           -32768
80000004:   addik         r13, r0 , 3416
80000008:   imm           -32768
8000000c:   addik         r2 , r0 , 3096
80000010:   imm           -32768
80000014:   addik         r1 , r0 , 6504
80000018:   imm           0
8000001c:   brlid         r15, 608          ; addr=0x8000027c
80000020:   or            r0 , r0 , r0
80000024:   imm           0
80000028:   brlid         r15, 892          ; addr=0x800003a4
8000002c:   addik         r5 , r3 , 0
80000030:   bri           0                 ; addr=0x80000030
80000034:   imm           -32768
80000038:   addik         r5 , r0 , 3412
I don't know Microblaze assembler so I don't know if the code starting in DDR (which should be the Hello World program) make sense. However, it closely resembles the disassembly of the start of the bootloader itself:

Code: [Select]

          _start1:
00000050:   imm           0
00000054:   addik         r13, r0 , 16120
00000058:   imm           0
0000005c:   addik         r2 , r0 , 15824
00000060:   imm           0
00000064:   addik         r1 , r0 , 18032
00000068:   imm           0
0000006c:   brlid         r15, 608          ; addr=0x000002cc: _crtinit
00000070:   or            r0 , r0 , r0
00000074:   imm           0
00000078:   brlid         r15, 772          ; addr=0x0000037c: exit
0000007c:   addik         r5 , r3 , 0
I assume there's some way to load symbols for the program in DDR so I can see the function names etc as I can for the bootloader (?)

If I break into the program, it stops at program counter 0x80000000. If I then single-step the debugger it just goes in a loop comprising the first two lines of the DDR code. i.e., following:

Code: [Select]

80000000:   imm           -32768
80000004:   addik         r13, r0 , 3416
it goes back to 0x80000000.

This may give me a clue but it's bed time https://forums.xilinx.com/t5/Processor-System-Design-and-AXI/MicroBlaze-hangs-on-first-instruction-in-DDR3/td-p/483840

[asmi]

Ok, so here is what I did. I used the board from my signature, with Ethernet addon board, and used Vivado/Vitis 2020.2 IDE.
1. Created a system diagram which includes DDR controller (DDR2 in my case), Microblaze, Timer, UART, EthernetLite (for 100 Mbit Ethernet) and Quad SPI (configured in "Performance Mode" and Quad).
2. Once generated bitstream and exported HW, went in Vitis and created a platform project based on that xsa file
3. Created a baremetal echo application using LwIp echo template. It's set to run from DDR (0x8000_0000).
4. Using "Xilinx -> Program Flash" menu item, converted the executable into SREC and burnt it into flash at offset 0x40_0000 (I have 128 Mbit flash, so 0x100_0000 bytes, full bitstream is 0x21_72F7 bytes).
5. Created another baremetal application using "SREC SPI Bootloader" template. It's automatically set to run off BRAM.
6. Opened blconfig.h file inside generated project and changed FLASH_IMAGE_BASEADDR to 0x00400000
7. Launched bootloader under debugger and observed what's going on.

I noticed that it does write some data into BRAM at offset 0 before proceeding to DDR memory. It also does jump to address 0, but it's apparently patched by the loader such that it launches the executable from DDR at 0x8000_0000.

And it worked with no problems! Now - I didn't try burning the bootloader into the bitstream, but that is a trivial step and I'm sure it will work - just make sure you associate the bootloader's elf file with the BRAM before generating a final bitstream. I also had to use 32KB of local BRAM, because debug version of bootloader was over 16KB.

[JohnnyMalaria]

I started from scratch and ended up with the same result - i.e., an apparent loop at 0x80000000.

The steps I followed seem the same as yours. I may need to just by a board that I know can do this.

[asmi]

I started from scratch and ended up with the same result - i.e., an apparent loop at 0x80000000.

Launch bootloader under debugger and see what's going on.

The steps I followed seem the same as yours. I may need to just by a board that I know can do this.

The board doesn't matter as long as you have access to live debugging. If not - buy a Digilent's HS2 cable - it's about $50, and connect it to JTAG connector on your board.
And in the future keep this in mind when choosing a board to buy - don't buy something just because it's cheap, as in many cases the reason it's cheap is exactly what's happening here - lack of good debugging facilities, or in general crappy design. For novices I generally recommend Digilent's boards because they usually have more samples and support materials, and they also have a support forum where their employees actually respond to users' questions and do their best to help them out.

[JohnnyMalaria]

I did use the debugger to monitor the bootloader. And I've also ordered the Digilent HS2 cable which should arrive tomorrow (courtesy of yet another Mouser free shipping upgrade).

Though these kinds of things are frustrating, they are also excellent ways to force you to really understand how it works. It's just the mountain of documentation that makes it difficult to know where to look for explanations.

Thanks for your guidance

[JohnnyMalaria]

So now I have my eye on the Zedboard. I suspect having a built-in ARM processor along with the FPGA will be better in the long run instead of using a MicroBlaze soft processor. But I need more GPIO than appears to be on offer. I assume an FMC breakout board would give me access to more of the FPGA's pins. Can you suggest any?

[asmi]

So now I have my eye on the Zedboard. I suspect having a built-in ARM processor along with the FPGA will be better in the long run instead of using a MicroBlaze soft processor. But I need more GPIO than appears to be on offer. I assume an FMC breakout board would give me access to more of the FPGA's pins. Can you suggest any?

I prefer working with pure FPGAs as opposed to Zynqs, and I have never felt limited by Microblaze because I use it mostly to orchestrate hardware blocks and respond to user interface (and none of these tasks require a lot of CPU horsepower), however I have to warn you that FMC cards tend to be very expensive and sometimes have compatibility issues. Also important factor is whether you going to be making an addon boards for it yourself, or prefer a board with a ton of peripherals already onboard. I belong to the former category, so if I were to shop for a Zynq board today, I'd likely get Eclypse Z7 board for their Syzygy connectors, which are kind of a middle ground between super low-end PMOD connectors and high-end FMC - they are good enough to let you send a full-on 1.25 Gbps over diff pairs through connector, yet are cheaper and smaller than FMC connectors, and easier to work with. But if you prefer your board to have a ton of gadgets out of box, ZedBoard is not a bad choice. It's bit old by modern standards, but it's very popular and you can find a lot of materials for it.

That said, like I said above, I prefer working with pure FPGAs, so if I were shopping for a board in this price range, I'd go for Nexys Video board. Though my personal favorite is Genesys 2 board, it's significantly more expensive, but it's got a kick-ass fast and big FPGA (Kintex-325T) and a good assortment of peripherals for wide range of applications.

But then again - I prefer making my own devboards with exactly the stuff I need on them. It's certainly not the cheapest nor easiest approach, but it's by far the most satisfying one

[miken]

I personally own, uh, three different Zynq dev boards. Zedboard is your typical academic type learning board, with a bunch of PMODs and stuff like buttons for the beginner labs. It looks like it exposes 33 pin pairs or so on the FMC. I'm not a big fan of FMC connectors for cost reasons but you can buy certain expansion boards for them off the shelf.

MicroZed is a good option if you are willing to design a carrier board for BergStak 0.8mm pitch connectors. It has 48 pairs and a few more single ended I think.

I also own a Snickerdoodle, which has fewer but friendlier (?) I/O pins (0.05"/1.27mm). Samtec claims them good up to 8Gbps.

In most cases I've used a Linux with Zynq, since it's a powerful enough processor to make that approach convenient. I've done MicroBlaze in an Artix before, but that was just for early hardware bringup.

[JohnnyMalaria]

Well, having eyed the Zedboard, I just ordered an Arty Z7-20.

[miken]

That looks like a reasonably priced option, but how much I/O were you looking for?

[iMo]

FYI - I bought a $30 Spartan6 board with $20 xilinx programmer clone off ebay some 5-6y back, fired up the ISE14 with their mb SDK and had none problem to run microblaze wired to ie. the cordic module or my own stuff. Also the on-chip logic analyzer worked fine. I ran PDP-8 with an sdcard as the disk unit(s) too on it.. So it works with low cost parts as well..

[asmi]

Well, having eyed the Zedboard, I just ordered an Arty Z7-20.

Oh well, I hope it will work for you. It wouldn't be anywhere near the top of my list, as it's about as barebone as it gets. And I like my gadgets

[JohnnyMalaria]

The Zedboard has a lot of things I can't imagine using for my application and the Arty has the Arduino-esque headers. I need to control a number of programmable switched capacitor network bandpass/lowpass filters, some programmable instrumentation amplifiers plus a few SPI devices including a 4-channel DDS. The onboard dual 12-bit ADC is also attractive since my application involves half a dozen relatively low speed analog signals I have to acquire.

I'm creating a more robust version of my instrument described in the attached. The existing version is very, well, Heath Robinson/Rube Goldberg, to put it mildly.