Flir E4 Thermal imaging camera teardown

[mikeselectricstuff]

In the meantime, somebody please try to find that magic menu...!

[mrflibble]

When dealing with potential lockout situations (like fixing something remotely) I tend to start a timer of say 1 hour so if somehow whatever I did locks me out, when the timer runs out it either undoes it, reboots etc.

More or less ditto for remote systems. For embedded like this I like the file existence approach so you can toggle execution branch.

But who cares about all that, I see Mike just undid it. Nice one!! 

[mrflibble]

Just tried adding the copy commands to applaunch.dat and no effect.
however commenting out appcore does stop the main app running.

Maybe applaunch.dat can only run executables

cmd /? :
CMD [/[C|K] command][/P][/Q][/T:bf]

  /C command  Runs the specified command and terminates.
  /K command  Runs the specified command and remains.
  /P          CMD becomes permanent and runs autoexec.bat
              (cannot be terminated).
  /T:bf       Sets the background/foreground color (see COLOR command).
\>Bad command or filename

hangonaminute! background processing in brain just finished. Note the absence of /R in that description. I bet someone grabbed the source of stock command, plonked in some code for serial console, handled /R argument, and didn't even update the help text. So for all you know the entire /A to /Z range is full of interesting functionality.

[BravoV]

Even more RESPECT 

+1 

Another great moment of reverse engineering event documented live in this forum, thanks Mike. 

Meanwhile, and again, when everytimes I watched great forum thread such like this one, just worry it might be locked or gone because of cease & desist .. love letter Dave received from lawyer. 

Time to save or update an OFFLINE version of this thread. 

[Fraser]

Mike,

You asked about the Germanium lens and its effects on calibration etc.

Some thoughts on the matter.

1. If FLIR are combining pixels in the 80x60 modes, the whole FPA will be 'in play' and illuminated by the lens.
2. If only the centre 80x60 pixels of the FPA were being used, the lens would be designed to illluminate that area of the FPA only ?
3. You have produced a hi-res image at 320x240 with the whole FPA 'in play'. I saw no vigneting so I believe the lens is illuminating the whole FPA. A lens change is therefore not essential which is good as they are not a common item on their own.
4. The quality of the lens used on the 80x60 E4 may be inferior to that of the E8 but whether most users would notice even at 320x240 is questionable.
5. FLIR calibrate their TICs in rows of test jigs, facing black body sources. The FPA thermal response should have been captured across all pixels and mapped. The lens is part of that calibration routine so a change of lens would reduce accuracy. The deviation would depend upon the similarity of the replacement lens to the original. Materials used, len thickness, coating and quality etc. Again, I doubt the change would be noticed by many users for the tasks that they had in mind. Adding a close-up lens to my PM695 made very little difference to the temperature calibration (but it is a very large and expensive close-up lens).
6. I have read of a company that is producing a new range of synthetic thermal camera lenses that would cost much less than Germanium. It is possible that FLIR might be using such in their low end cameras and a Germanium in their higher resolution units. This would be similar to web cams....some use plastic lenses, others HQ glass. Changing from a synthetic lens to a Germanium would likely be a major calibration difference due to differences in matrial 'through' losses.
7. A final thought. Lens diameter is very important in LW microbolometer cameras. To get good sensitivity the largest lens possible should be used. From your tear down I was not impressed with the tiny lens used in the E4, but the FPA is obviously very sensitive when compared to my units from the late 1990's so a small lens gives acceptable specs. There appeared little option for a larger lens in the E8 unless several parts were changed to accomodate such. I just don't see FLIR doing this on such a 'budget' line. 

[mrflibble]

Even more RESPECT 

+1 

definitely  ++

Time to save or update an OFFLINE version of this thread. 

What? You mean you didn't mirror this thread and the entire Flir firmware exe/zip/iso/pdf collection? 

Gotta be prepared for when the pointy haired CatOutOfBag denial posse orders a fix with the next firmware update, only to fail horribly. Too late.

@Aurora:
So your educated guess is that most likely the E4 and E8 will use the same lens? That would be even better news. If so, that would seem to indicate the difference in temperature resolution between models is purely a processing issue.

[Fraser]

If lens diameter is fixed due to the mechanical constraints, the possibilities for differences in temperature resolution are

1. Different lens material
2. Different lens coating
3. Deliberate hobbling of lower spec E4 to encourage sales of higher spec units 
4. Different FPA used in different models (there are two FPA models detailed in the OS)

I am suspecting FLIR may have gone for option 3 !

Mass production of a product like the E series is most cost effective if most mechanical and electronic parts are the same, with just firmware differences. The BoM is then kept as small as possible and the manufacturing line is simplified.

[BravoV]

What? You mean you didn't mirror this thread and the entire Flir firmware exe/zip/iso/pdf collection? 

That post meant for provoking a further mass generation of the off-line mirrors, silly. 

[mikeselectricstuff]

There is a ton of stuff about lens calibration in the service menu.

The E series also seems to have this 'added noise' thing, disabled in service mode.
It could be that this makes the downsampled modes and the 60Hz-9Hz downsampling  work better with some added noise to reduce banding. Will do some playing if it will let me fiddle with the config files.
The 320x240 image looks fairly free of noise, but not tried setting a narrow band.
incidentally you can manually set the span and zero  by first locking the temp range and then using the 'level' and 'span' commands from the console.

I will be interested to see the E8 lens - there is potentially space for a bigger lens, but not hugely bigger.
At 9fps they can probably get away with the small lens as they have 6 frames to work with, so in principle 6x the SNR - the Ex0 range lenses look a lot bigger

[mikeselectricstuff]

Mass production of a product like the E series is most cost effective if most mechanical and electronic parts are the same, with just firmware differences. The BoM is then kept as small as possible and the manufacturing line is simplified.

Yes, though screwing on a different lens when a unit is calibrated and configured to be a specific model wouldn't be a big deal.
I'm pretty sure the present lens moulding envelope could accommodate a bigger lens

[Fraser]

With regard to deletion of this thread....... would Dave do that ? He supports that Wiki Leaks chap doesn't he ? Freedom of speech and all that.

But seriously, the cat is well and tuly out of the bag, and FLIR know it. I suspect meetings have already taken place to decide what, if any, action will be taken    Sometimes less action is best in such cases, as it avoids negative publicity and FLIR will know that suppression of Mike's discoveries would be nigh on impossible. Sadly what is likely to happen is a change in firmware to block the 'hack' as that is cost effective and limits the damage to only the initial production batch.

The PR fallout for FLIR will revolve around the proven fact that they are using a 'full fat' 320x240 pixel FPA in their budget E4 camera. It will be apparent to all that the E8 is massively over priced as the hardware costs will be similar, if not the same. Historically I have had the view that the major cost in a TIC is the FPA and Gemanium lens. This E4 revelation tends to change that view as the 320x240 FPA and lens is obviously cheap enough to place in a lowly 80x60 spec camera.....if it were an expensive part due to resolution, such action would make no financial sense, and an 80x60 or 160x120 FPA would have been used instead.

The E4 in its 'hacked' state would make a very capable TIC for experimentation and consumer grade work. I do not see professionals in industry using hacked units though so the E8 will still sell to its target market. This is very much like the Rigol DSO hack, and I suspect it will play out in much the same way. One big diffrence with the Rigol was that it was using overclocked parts to achieve its spec, the FLIR is effectively using overspecified parts which is the inverse !

For me, as a TIC geek, this is a facinating develpment. I only wish I had the software skill-set that has been displayed here by Mike and the other members on ths 'Hack the E4' team. RESPECT to you all   

[BravoV]

With regard to deletion of this thread....... would Dave do that ? He supports that Wiki Leaks chap doesn't he ? Freedom of speech and all that.

Personally I believe Dave won't do that, but what can I say if he suffered enough legal pressures that are too hard to bear for him personally ? 

I think we should move this discussion at another newly created thread maybe at General section, enough OOT contaminations in here already, sorry Mike, won't talk about it further here.

[equinoxe]

 

Well done!!
BTW:
The word 3vlig (trevlig) is used in Swedish meaning nice!

[mrflibble]

What? You mean you didn't mirror this thread and the entire Flir firmware exe/zip/iso/pdf collection? 

That post meant for provoking a further mass generation of the off-line mirrors, silly. 

Recursive: see recursive. Mission accomplished! 

You did provoke me to feed the wayback machine however. So that's up to date as of now. I wish them the best of luck getting that deleted as well as random dropbox locations that may or may not pop up.

And yes, cat's out of the bag. Best move IMO is just to accept it. Yup, it happened, lesson learned. Silly fixes in future firmware will not work. They might as well enjoy the significant rise in E4 sales, and be happy. 

[mrflibble]

So when's the video coming out? O0

BTW:
The word 3vlig (trevlig) is used in Swedish meaning nice!

Interesting coincidence.

Or confirmation of my .rcc file induced suspicion that this was made by their Swedish software team. That resource file is full of se.flir references.

[mikeselectricstuff]

I suspect a likely outcome is more sales of E4, Rigol style, not loss of E8 sales - The E8  looks overpriced for teh benefit it gives.
And it may also seriously piss off FLIR's competition. Neither of which would cause FLIR to lose much sleep. They can effectively gain sales and market share without openly price-slashing.
With the loosening of export controls on 320x240 cams I think we'll be seeing the price of this resolution falling a lot soon (notwithstanding hacks) - it's just a matter of which manufacturer jumps first, as 32x240 cams clearly have disproportionately  high margins at the moment.
 At this res, MSX type tech is much less of a bonus so manufacturers without it won't be at much of a disadvantage.

There are many ways they could have totally locked this down if they'd really wanted to. However big companies often don't give much consideration to it unless the're actively selling upgrades.

[mikeselectricstuff]

So when's the video coming out? O0

BTW:
The word 3vlig (trevlig) is used in Swedish meaning nice!

Interesting coincidence.

Or confirmation of my .rcc file induced suspicion that this was made by their Swedish software team. That resource file is full of se.flir references.

I think it actually says something like "designed by Flir Sweden" on the label.

As regards "action" - what could they do? maybe Dave won't get the review units, but after the Rigol experience they would be insane to try anything heavy due to the Striesand effect.
All that has been done is revealing of how to use existing functionality, with reference to published documents. Though there were some hints from disassemblies, these were not essential, and no copyright information has been re-published.

If Flir are pissed off with anyone, it should be with their own developers for lazily re-using code without taking out unnecessary functionality. Assuming of course it was unintentional.

[Fraser]

Mike,

I may be totally wrong as you know more about the firmware than me, but I have not herad of this noise insertion technique in a TIC before. Bit padding, yes, but not noise insertion.

The Uncooled Microbolometer is a noisy beast and much time and money has been spent trying to tame it  in order to provide nice clean images with as little internally generated noise as possible. My 1st Generation FPA PM570 is pretty noisy when compared to the later 3rd Generation FPA PM695 on this front. FLIR have spent a lot of time developing the FPA and also the noise REDUCTION algorithms and capabilities. As you know, my PM695 has 3 modes for noise reduction.

1. Off - for fast moving targets like cars and animals etc.
2. Normal - Everyday observation, hand held, non fast moving targets.
3. High - Lowest noise mode for static targets only and tripod mounting of camera is essential.

Having used these modes, it appears the noise reduction works by comparing captured frames and deleting the random elements before display. The frame rate appears much slower as a result. The PM695 provides a very nice image in the 'Noise redcuction OFF' mode and you can pan the camera easily. With the mode set to 'High', any movement of the camera or target causes really impressive image smearing and pixelation.

I would be amazed if FLIR did not employ noise suppression in the E series. Is it not possible that the noise suppression is switched OFF in service mode to provide the raw, fastest update without interferance from the noise algorithms and processes. That is what I would do in a service mode. Keep it to raw basics, without any bells or whistles active to confuse the testing or CALIBRATION 

[mikeselectricstuff]

Mike,

I may be totally wrong as you know more about the firmware than me, but I have not herad of this noise insertion technique in a TIC before. Bit padding, yes, but not noise insertion.

The Uncooled Microbolometer is a noisy beast and much time and money has been spent trying to tame it  in order to provide nice clean images with as little internally generated noise as possible. My 1st Generation FPA PM570 is pretty noisy when compared to the later 3rd Generation FPA PM695 on this front. FLIR have spent a lot of time developing the FPA and also the noise REDUCTION algorithms and capabilities. As you know, my PM695 has 3 modes for noise reduction.

1. Off - for fast moving targets like cars and animals etc.
2. Normal - Everyday observation, hand held, non fast moving targets.
3. High - Lowest noise mode for static targets only and tripod mounting of camera is essential.

Having used these modes, it appears the noise reduction works by comparing captured frames and deleting the random elements before display. The frame rate appears much slower as a result. The PM695 provides a very nice image in the 'Noise redcuction OFF' mode and you can pan the camera easily. With the mode set to 'High', any movement of the camera or target causes really impressive image smearing and pixelation.

I would be amazed if FLIR did not employ noise suppression in the E series. Is it not possible that the noise suppression is switched OFF in service mode to provide the raw, fastest update without interferance from the noise algorithms and processes. That is what I would do in a service mode. Keep it to raw basics, without any bells or whistles active to confuse the testing or CALIBRATION 

I've not played - the only reason I assumed it was noise addition was a comment/assumption from the guy whol emailed about the i series hack.

.caps.config.image.targetNoise.enabled bool true
.caps.config.image.targetNoise.targetNoiseMk int32 135

It could actually be noise reduction or shaping - maybe this is a threshold to decide when to apply noise reduction, i.e. adjust filter until noise is below the target value.  "Mk" - mask? Perhaps something like a hysteresis function?
Either way it would make sense to not do it in service/cal  mode.

Also don't forget that there may be other issues in service mode that haven't been spotted yet - need to compare some known reference images to check cal. I've left it running all day at home to see if there's any timeout etc. 

[Fraser]

I can't help feeling that, with the 320x240 resolution, MSX has just 'crashed and burned'   

I still like the idea on low resolution TICs and when there is low thermal difference between objects in the field of view.

[mikeselectricstuff]

I can't help feeling that, with the 320x240 resolution, MSX has just 'crashed and burned'   

I still like the idea on low resolution TICs and when there is low thermal difference between objects in the field of view.

Absolutely - MSX on the 80x60 makes it much more useful, and for many applications, is close to as useful as higher thermal resolution, so for low-budget users it's a viable alternative to higher res cams. Without it, even at £800 an 80x60 cam would be disappointing. Considering the minimal cost of a visible camera it's a no-brainer.
The only time it really fails is when light levels are low, but all you need is a torch. Higher-end models have illuminators.

One enhacemant that would be nice is automatic distance calibration - this could be done in 2 ways - point at an object with vertical edge in the thermal image and presss a button to make it look for teh best correlation with edges in the visual, or a second visible camera below the main lens - by correlating images from the two visible cameras it could continuously auto-scale the distance.   

Something I don't really feel any need for is higher framerate - even without any ITAR issues, the cost saving of being able to use a smaller lens to improve SNR is probably a pragmatic solution. I guess framerate becomes more important for night-vision type apps, or if you have autofocus. 
 It will be interesting to look at the raw 60FPs data though, but I won't have time for that for a while.

[Fraser]

60fps is a must for high speed thermography of targets moving through the field of view. 9fps is fine for static objects.

I recall that someone saw an E5 camera running at 60 fps at a German exhibition ? I don't think I dreampt it. If so, the lens may not be such a limiting factor.

I remain surprised that the UI is so sluggish on this camera. From your investigations, it appears to use a powerful main processor. I wonder if the image processing hogs the MIPS ? FLIR need to fix that. Possibly verbose, inefficient programming combined with Win CE OS ?

[Fraser]

Now heres a question for you all......

The E4 is fast giving up its secrets and the MSX capability is becoming mute.

There are a lot of FLIR i3's out there........Hmmmmm I wonder if they can be upgraded in a similar manner ? i3's should be being discounted soon, and cheaper on the used marketplace due to the E4's arrival. We know the i7 can be hacked.

FLIR would not be able to lock the firmware on those old i3 stocks, or units already sold   

Ask yourself whether the E4 offers ANYTHING better than an i3 when MSX is removed from the equation 

Cheap 320x240 TIC anyone 

[Pinkus]

If you want one, you might better hurry up to get one. I ordered my E4 this night which is (luckily) already on its way.
I just checked the supplier again: the E4 suddenly is out of stock at this supplier and also at another supplier - both had them on stock this night.
Either the E4 are now sold out quickly because of this thread or Flir is pulling the stock back for implementing a new firmware.

[mikeselectricstuff]

60fps is a must for high speed thermography of targets moving through the field of view. 9fps is fine for static objects.

I recall that someone saw an E5 camera running at 60 fps at a German exhibition ? I don't think I deampt it. If so, the lens may not be such a limiting factor.

I remain surprised that the UI is so sluggish on this camera. From your investigations, it appears to use a powerful main processor. I wonder if the image processing hogs the MIPS ? FLIR need to fix that. Possibly verbose, inefficient programming combined with Win CE OS ?

Probably - when inside a fullscreen menu it shouldn't actually be doing anything else at all so no reason for it to be slow. I suspect the issue is there are many layers in the way of doing what is a simple task - another casualty of using a software framework designed for higher-end products.
I saw plenty of non-optimalities in the ARM code, but not sure to what extent this is due to the compiler itself or the API layers hiding things that the compiler could optimise if it knew about them.

Sluggishness in thefullscreen option menus isn't a major deal as you hardly ever need to go there, and the onscreen popups are just about acceptable
My biggest complaint by far is that you need to pull up the menus to select between simple functions, in particular image mode and target distance.
I was told that FLIR are working on improving this in a future release. 

It may be that it is being slowed by image processing, but this should be turned off when inside fullscreen menus.
My belief is that all the processing happens in the FPGA, and all the SoC does is overlay the menus.