How bloody hard can it be to program an AVR chip?

[EEVblog]

It's amazing how we adapt to the internet. In the past most went to the book shelf to look up something they didn't know. With the internet we google, it's so convenient. An now we even google first before thinking at all :-)

In this case I probably thought (and assumed) a bit too much!
I don't recall precisely, but my mind when I saw the ".HEX file format not supported" message would have likely been thinking along the lines of the most complex solution. Like that the .HEX file really was in some HEX format incompatible with AVR Studio, and that there was no point me actually checking the .HEX file itself because I simply wouldn't know what I'd bee looking for in terms of some format incompatibility, and that would have required research to learn all that stuff again etc.
I was in a hurry so I simply redownloaded it to make sure it wasn't a corrupted and tried again, same result. I tried some other makerbot .HEX files, same result.
So then I would have quickly googled to see if anyone had the problem, and if there was another place I could download the .HEX from.
And of course that lead me to avrdude, and I forgot about AVR studio for a while and tried that, and ran into all those troubles there as well.
It's not the first time I've had issues with AVR Studio, so a genuine .HEX file incompatibility didn't seem unreasonable to me at the time.

So although it seems I was completely stupid on the surface, there were many factors based on my experience and the circumstances that lead me down the path I took.
And yes, with hindsight it's bloody obvious I should have at least took a peak in the .HEX file and I would have noticed straight away it was a HTML file, but really that didn't cross my mind at all it would be such a gross problem like that.
Shit happens.

[c4757p]

Oh, Dave, pampered with good internet... You must have gotten it earlier than I did. I wouldn't have thought the file was HTML (well, OK, I would, but I'm familiar with GitHub...), but I sure would have suspected I got a corrupted download or something! It's been a while since that's happened to me, but it sure used to!

[EEVblog]

I think the blame with this lies on Github for linking to HTML files with a .HEX extension.
If I right-click and Save As, I get "Type: HTML document" in the dialog and it gets saved with the extension ".hex.htm" AND an HTML icon, which would just show as ".hex" if you had file extensions hidden

When you move your cursor over the .HEX filename it gives you the path:
https://github.com/makerbot/MightyBoardFirmware/blob/master/dist/MightyBoard/mighty_one_v7.3.13.hex
That, plus the filename, to me for all the world indicated I was downloading the direct .hex file.
Chrome then saves the file as a .hex, not some hidden .hex.html
So everything lead me to believe I had the genuine file.
Had I clicked on the filename instead I might have realised something was amiss of course, but I didn't.

[c4757p]

Chrome then saves the file as a .hex, not some hidden .hex.html

Indeed it does! Fail... Web sites use nonstandard extensions all the time! They should know better than to keep the one it's got.

[lewis]

So how do you actually download a hex file from that page Dave linked to in his first post?

[c4757p]

Click "Raw". No, it's not intuitive at all...

[lewis]

Aha! Easy when you know how!

[EEVblog]

Click "Raw". No, it's not intuitive at all...

No, you have to right click and "Save as" on the raw button.
Well, at least in Chrome anyway, otherwise it opens the HEX file in the browser window instead of downloading.
You can save it from the browser windows of course, but it defaults to a .txt extension then.

[lewis]

What an absolute pain in the arse (and/or ass). It's not difficult to make things easy.

[c4757p]

I'd just click Raw - that way I can see the code in the browser window and verify that it's truly what I think I'm getting. Then Ctrl-S to download. Yeah, it defaults to .txt, but it takes me two seconds to change that...

Arse and ass.

This sort of web interface is very common among VCS hosts. Launchpad and SourceForge do it the same way. The developers were probably so accustomed to it that they didn't really consider that it's not "normal" for people who aren't developers.

[firewalker]

The extension thing on Windows is really stupid. KDE4 for example reports that the download file is html and select the appropriate icon.



Alexander.

[c4757p]

Yes, Linux (at least, in my experience, KDE, GNOME and XFCE) and IIRC Mac OS all look at the file's contents, not its extension. Extensions can be used to differentiate between similar files (most of them, for instance, can recognize the difference between a C header and C source), but that's only used as a last resort when the file type can't be deduced quickly from its contents.

[metalphreak]

Windows is by default set to hide known file extensions.

filename.hex.html would show up as filename.hex since html is a known file type... makes things very misleading.

NotAVirus.txt.exe for example 

[jancumps]

...These OS* groups really need to set proper release deadlines and prioritize polishing the UI ...

They usualy focus on those things that contributers work on.

[c4757p]

NotAVirus.txt.exe for example 

Precisely. Hiding the file extensions was one of the dumbest ideas Microsoft has ever had, and that's saying something...

They finally took a hint from Unix and started marking executables that come from the Internet in the file system as "ask first before running" (Unix won't execute anything that isn't marked executable in the file system, and this goes all the way to the kernel - you can't fire off an "exec" system call to a non-executable file, it'll just raise EACCES - and nothing is ever marked executable by a browser), but they went with the "I'll click OK without even reading this" style of error message...

[mrflibble]

This sort of web interface is very common among VCS hosts. Launchpad and SourceForge do it the same way. The developers were probably so accustomed to it that they didn't really consider that it's not "normal" for people who aren't developers.

I think that about sums it up. IMO this is not so much an issue with github. That web interface is pretty much working as expected I'd say. It is that if this really is Makerbot's intended way of distributing firmware updates, then, ehr, yes... At the very least they should add a clear description about this in the FAQ. And also assume that noone actually reads the FAQ, so add some checks to the firmware update software as well. Then when the software barfs on the hex file (aka html file), instead of only a "unsupported hex format, jump to conclusions .... NOW!" message, they could also print the url for "hey, you might look here" that points to the FAQ. Users will usually follow the path of least resistance, so if least resistance is to actually read that entry of the FAQ then things might actually work. Sortof. But best way is to just provide a "proper" distribution channel for updates.

Well that, and everything Mike wrote about how to actually do firmware updates.

[c4757p]

I don't think there's anything wrong with GitHub at all. I've used it before and it's perfectly intuitive to a developer. I'm actually a pretty big fan of it - I'm probably going to throw my KiCad library that I'm working on up on it, though it will have a separate home page and a download link that doesn't require farting around with GitHub. (Using a .hex extension for what's really an HTML page about a hex file is a bit dodgy, but oh well, nothing is perfect.) But I don't think anybody has complained about GitHub itself, just having to use it.

Also, everything Mike said.

[c4757p]

Er... why do you have two folders with blank names?

[lewis]

He's photoshopped out 'porn' and 'unusual porn'. Fits alphabetically...

[c4757p]

"Unusual porn"...   

[free_electron]

Here's how i write bootloaders:

- Loader sits in top block of flash and protected.

Powerup

Step 1:
1) check hardware pin. if low : erase topmost byte of user flash.
2) check topmost byte of application flash. If it says 0x55 : last upload succeeded and flashed: if not : wait for firmware
3) check topmost word of user ram. if it says 0xDEAD the application failed to start properly : erase topmost byte of user flash and wait for firmware. ost likely the flash is corrupt.
4) set topmost ram to 0XDEAD , program hardware watchdog to 3 seconds and vector to application code.

Flash code:
write contents to flash , do integrity check ( checksum ) write 0x55 to topmost flash : hardware reset

normal boot process:

the firmware has 3 seconds to start and then alter the 0xDEAD to 0x55AA. if it fails to do that the system will reset and the bootloader will see the 0xDEAD signifying the system failed to start properly we have flash corruption or other problems. If 0x55AA is there we know at least the init code worked and we crashed because of some software bug. No reflash needed. Simply restart. We write 0xDEAD and restart.

oh, and i always use hardware watchdogs.

[firewalker]

He's photoshopped out 'porn' and 'unusual porn'. Fits alphabetically...

Ooooops, busted! 

Actually it was unziped folders from Win98 zip files. Ugly. I erased the long file name.



Alexander.

[EEVblog]

Windows is by default set to hide known file extensions.
filename.hex.html would show up as filename.hex since html is a known file type... makes things very misleading.

But the point is that windows on github does not add the HTML extension, it does save it as a .hex file. So unless you know that fact, you can get easily fooled, regardless of your windows settings.

[mrflibble]

Actually it's also a bit of "how does your browser handle it". Yesterday I checked the http headers with wget just for the fun of it, and it had content type: text/html utf-8 ... not much more you can do server side. After that it's up to the browser to do something sensible with that.

But user friendly it's not. They would probably have been better off plonking it in a .zip, and then bundle the hex file with an md5sum file. That way they could do something clever in their update software. Hell, they could even sign it although that is probably one bridge too far. Anyways, update distribution for random non-dev users could be a wee bit better here.

In fact, I almost cannot believe that this is the suggested way... O_o

[amyk]

Actually it's also a bit of "how does your browser handle it". Yesterday I checked the http headers with wget just for the fun of it, and it had content type: text/html utf-8 ... not much more you can do server side. After that it's up to the browser to do something sensible with that.

At least they could've appended the .htm/.html extension to all links which lead to HTML and NOT raw file contents, like many file hosting sites do.