Padding output hex file with objcopy

[HwAoRrDk]

I'm compiling some code with GCC and then as a post-build step outputting an Intel Hex file from the ELF with objcopy. I'd like to pad the resulting Hex file to fill up the remaining flash with 0x00 (an illegal RISC-V instruction, so that if PC ever gets into the weeds, the default infinite-loop hard-fault illegal instruction ISR is hit and the watchdog triggers). I thought maybe I could do this with the --gap-fill and --pad-to options of objcopy, like so:

riscv-none-elf-objcopy.exe --gap-fill 0x00 --pad-to 0x3FFF -O ihex <input.elf> <output.hex>

But this doesn't seem to work how I expected. It produced a 1.4 gigabyte hex file!

The objcopy documentation is not very detailed regarding this option. It simply says:

Pad the output file up to the load address address. This is done by increasing the size of the last section. The extra space is filled in with the value specified by --gap-fill (default zero).

I guess perhaps the address I'm giving is not right when interpreted as a 'load address', although I'm not sure of that because the 'FLASH' origin address is 0x00000000 in the linker script, so I would expect 0x3FFF to be correct in that context. Or maybe objcopy's definition of the "last section" is not the last one in the ELF file, but the one at the highest address, which I guess would be .data at 0x20000000. The latter could explain the enormous Hex file if it's padding out to there.

Can objcopy actually do what I want? Or do I need to find another way? (I suppose I shall have to see if srec_cat is available for Windows...)

[wek]

Get a life. Use srecord.

[T3sl4co1l]

Get a life.

Ummm?

[HwAoRrDk]

Get a life. Use srecord.

Polite reply:

Oh, gee, thanks, I'll look into that.

Not-so-polite reply:

Excuse me? Fucking what? Did someone piss in your breakfast cereal this morning? Of course I fucking know that srecord utilities can do this job, otherwise, y'know, maybe I wouldn't have mentioned srec_cat in my original post, eh? I'm not an ignorant dumbass. Unlike some who would prefer to spit denigratory vitriol instead of just ignoring something they consider unworthy of their time.

All I'm trying to do is verify whether objcopy won't actually do this job, or if I'm just using it wrong.

But no, apparently I'm wasting people's time making a post that is an affront to the intelligence of the reader.

[Siwastaja]

which I guess would be .data at 0x20000000

Probably - that would generate 512 megabytes of padding, which, given the overhead of .hex file would be close to 1.4 gigabytes as you say.

Just a hunch: try adding: --remove-section=.data* --remove-section=.bss*
And also any other weird section, which you can see using objdump -h. For example, my makefiles now always contain --remove-section=.ARM* since some tool I don't understand is generating weird sections I don't understand and definitely do not need in my binary.

[abeyer]

I guess perhaps the address I'm giving is not right when interpreted as a 'load address', although I'm not sure of that because the 'FLASH' origin address is 0x00000000 in the linker script, so I would expect 0x3FFF to be correct in that context. Or maybe objcopy's definition of the "last section" is not the last one in the ELF file, but the one at the highest address, which I guess would be .data at 0x20000000. The latter could explain the enormous Hex file if it's padding out to there.

Can objcopy actually do what I want? Or do I need to find another way? (I suppose I shall have to see if srec_cat is available for Windows...)

I think you're correct in your analysis of why that's happening. I don't know of a way to do this in one pass with objcopy. You can exclude the data section and just pad out the rest, and then create a separate hex of just the data. But then you need to combine them again at some point. Doesn't seem worth the hassle vs just using srecord.

[eutectique]

Let the linker do it for you.

https://mcuoneclipse.com/2014/06/23/filling-unused-memory-with-the-gnu-linker/

[SiliconWizard]

Isn't 0xFFFFFFFF also an illegal instruction (TBC)? Just mass erase flash before programming.

[dkonigs]

Let the linker do it for you.

https://mcuoneclipse.com/2014/06/23/filling-unused-memory-with-the-gnu-linker/

I've actually had to use a combination of both methods.  Have something in the linker script to pad out the binary image to the flash size, then use "objcopy --gap-fill" to fix all of the little holes that sometimes appear between the sections.  Though in my case I wanted to fill with 0xFF (instead of 0x00) as its the default value for unprogrammed flash, and I needed consistent and repeatable results so hashes would match.

(Originally just used an "end fill" section in the linker script, but after a GCC tooling update a few padding holes started appearing within the binary that I couldn't do anything about, so I discovered "--gap-fill" to take care of them.)

[wek]

otherwise, y'know, maybe I wouldn't have mentioned srec_cat in my original post, eh?

I genuinely overlooked that, sorry.

denigratory vitriol

That was not the intention.

The long version would have been, perhaps:

Don't waste your time.

Binutils are notoriously overcomplicated, with some of the features gotten there either very soon and then simply never been properly maintained for lack of mainstream need, until gotten unusable for some other related and more important development; or as some vanity/marginal feature forming part of some particular effort for some very particular target/toolchain. Those features then function weird, depend on circumstances (e.g. target, host, (non)existence and particular naming of some particular section, etc.) and are incompletely/insufficiently/unusably documented.

Attempting to use those features then leads to frustration, as they often "almost work".

Been there, although not with --gap-fill. For example, I've tried the linker-script version but as others mentioned, it's not it either, so I quickly gave up. Wasted substantial time on trying to comprehend orphan sections and actually trying to utilize them for "no need to update linker script" approach, just to having that failed miserably and having to rework that build setup after linker/toolchain update (forced by client requirement). Sucessfully using injection of binary data tables through converting bin to elf to be linked together with "normal" objects in one project, but that too was a massive waste of time to get it working with walking down quite a couple of dead ends (with the repeated lure of "oh this time it almost perfectly worked"), and I won't update the linker for that project, ever. Etc.

To find out the idiosyncracies and real working of such features, often the "real documentation" (i.e. sources) have to be consulted, which, needless to say, is a painful experience in itself, for the same historico-layering reasons.

srecord is so much, much, much more fit to this particular task, even if its documentation is ehm weird.

JW

[HwAoRrDk]

Isn't 0xFFFFFFFF also an illegal instruction (TBC)?

You know, I'm not sure. I was thinking it wasn't and that it actually decoded to something, but I realise now I might be thinking of a different architecture than RISC-V. To save having to wade through the ISA specification, does anyone know of any online RISC-V decoder/disassembler so I can check?

Let the linker do it for you.

https://mcuoneclipse.com/2014/06/23/filling-unused-memory-with-the-gnu-linker/

Ah, that's handy, I will try that out.

I'm getting a strange sense of deja vu reading that article. Perhaps I have seen it before and just didn't remember it.

That was not the intention.

If that wasn't the intention, then next time perhaps you should stop and think about the meaning of the words you're using.

Now, maybe English isn't your first language, and the implicit meaning and impact of bluntly telling someone to "get a life" is not apparent to you. If not, know that doing that is tantamount to telling someone to fuck off; that "you are annoying me and idiotic to spend your time on something I consider boring and unimportant"; that you are criticising or ridiculing that person's pursuit as insipid, uninteresting, and/or unrealistic.

If you were just trying to tell me "don't waste your time", then "get a life" is not an appropriate similitude.

even if its documentation is ehm weird.

We can agree on that, at least. I despair having to look anything up in Srecord's documentation. It's exasperating the way they split things up and spread things around multiple pages.

[HwAoRrDk]

Isn't 0xFFFFFFFF also an illegal instruction (TBC)?

You know, I'm not sure. I was thinking it wasn't and that it actually decoded to something, but I realise now I might be thinking of a different architecture than RISC-V. To save having to wade through the ISA specification, does anyone know of any online RISC-V decoder/disassembler so I can check?

Oh, wait, there is a table in the back of the RISC-V ISA manual. I suppose really it is the opcode field in any given sequence of bytes being decoded that determines whether or not it is a valid instruction. According to the tables in the manual, a 32-bit instruction with all 1s in the opcode (lower seven bits) is reserved for extended instructions greater than 80 bits in length, so probably considered invalid for an RV32EC architecture. And a 16-bit instruction with 1s in the lower 2 bits is not a valid compact instruction, as that is what denotes a 32-bit instruction.

So perhaps you're correct, and I don't need to do anything at all apart from ensure unused flash memory areas are erased.

[wek]

Now, maybe English isn't your first language,  and the implicit meaning and impact of bluntly telling someone to "get a life" is not apparent to you.

When and where I learned that phrase, it was meant literally, as an advice, and definitively not an insult. I'll remember it's not that way; thanks, and sorry.

  [we agree on that srecord documentation is weird]

Besides that, srecord is also unusual in that the command-line, even if appears to contain things like input/output file name and switches, as so many other programs; here, in fact, the command-line is a script, and it is to be written (and is parsed) strictly in the order it's written, with each "element" (consisting of multiple "keywords" with no visual/logical clues of arrangement) acting upon an internal buffer (which in turn consists of two arrays, "values" and "masks").

So, in your case, you may want to write something like:

Code: [Select]

srec_cat.exe     -gen 0x0000 0x0x3FFF --constant 0     -exclude -within input.hex -i    input.hex -i    -o output.hex -i
which can be seen as a script to:

Code: [Select]

-gen 0x0000 0x0x3FFF --constant 0     // create a mask of valid bytes between 0x0000-0x3FFF, and fill the corresponding value buffer with all zeros
-exclude -within input.hex -i         // in the mask, "punch holes" (invalidate) all ranges which are defined in input.hex (-i = intelhex)
                                      //   -- this is needed as srecord by default doesn't allow overlaps
input.hex -i                          // now read in input.hex (-i = intelhex),
                                      //   i.e. in mask mark ranges used by input.hex as valid and read values into the value buffer
-o output.hex -i                      // now output all things marked in mask as valid, into output.hex (-i = intelhex)

In this way, you can add and remove various pieces of binary data multiple times within one command.

In contrast, while it may appear that the purpose of objcopy is to do something similar, the procedure is different: objcopy takes the command-line switches and - using internally defined and not always well described rules -  a selection of sections from the .elf, and then performs operations in an order again defined by some internal rules, to generate the output.

I don't say one is better than the other, by no means; it's just that for some particular purposes one might have advantages over the other, and vice versa.

JW