Yet another cheap thermal imager incoming.. Seek Thermal

[eneuro]

Three consecutive frames, 11_6, 12_1 and 13_3. 16bit .png

Thx 
It looks like there were a very small time distance between those 0x6 0x1 0x3 frames so perfect to compare with router thermal images with gradient clery visibble in the corners.
We'll see what happends there 
I'll remove those hexagon patterns and dead pixels using methods described above and attach those updated images there, so every one can try play with them in imagej2 while there shouldn't be 0x0000 pixels no more, so with images provided by you we'll have set of two series: oryginal sensor captured data and preprocessed hexagon & dead pixels, but no additional corrections bluring etc made 

BTW: Which ambient room temperature could be when you recorded those hot iron thermal images?

Update: Attached preprocessed @cynfab files with hexagon & dead dots filled with neighbours avarage and a few imagej2 processing which might be interesting excercise while everybody can do it with those attached 16bit PNG sensor files  in imagej2 to get @cynafab hot iron thermal signature like this below with iron256 LUT I've posted above


Seek sensor frames 0x1 and 0x6 looks very similar and in this experiment made its average as new calibration frame used with image frame 0x3.


After substracting averaged shutter frames (0x1+0x6)/2 above from scene frame 0x3 and as before small median for noise removal and blur to get gradients more visible ended this time with this after applying my new iron LUT


Gradient is less visible while there is higher temperature range in image from room ambients temperatures and small amount of this hot iron copper wires, but it will be interesting experiment remove from scene image 0x3 those hot temperatures and make air temp there and after adding much more strong averaging we'll see that in the corners those gradients exists, but are not such clear visible in scene above 

Which is interesting average sensor values for shutter calibration frames are about 8000 while average temperature in thermal scene is lower about 7500, so this is what we could expect since inside Seek dongle were 35*C in one of @Aurora tests, and while ambient room temperature was 23*C as @cynfab report below it looks ok, however we still do not have algorithm to calculate absolute pixels value and match with thermal LUT display, but it gives some hints and it is time investigate those hexagon pattent pixels in first and last row in sensor  raw frames, as well as agani review 207th column which on those calibration frames and object frames average is: 4559 which by using one of my formula showed before leads to 71.2*F ~ 21.8*C so close to 23*C estimated by @cynfab but inside this dongle must be higher temperature, so for the moment no conclusions on this just speculations.
Lets examine another hexagon dot[55]... etc pixels for some hints and read Melexis IR - thermophile sensors datasheet for some hints   

[cynfab]

Ambient was about 23C

[-jeffB]

I can confirm that amazon orders are shipped out immediately.
If I was in USA I would already have it in hand because the package has arrived to my virtual US address.
(And is now on it's way to Europe...)

I checked on Amazon just now, and the only result it returns for "Seek Thermal" is the Lightning (Apple) version. (One seller claims to have it, and is asking US$799. Yeah, right.)

I can confirm that US orders direct from Seek no longer go to back-order status. Unfortunately, while my order (placed last Monday) was marked as "Shipped" on Wednesday the 19th, FedEx still says "Label created", which means Seek hasn't actually bothered to drop it off for shipment yet.

If I thought I could, I'd consider canceling the direct-from-Seek order and buying from Amazon. I've got an application in mind for which I'd need to have it by next Wednesday, and while Seek pretty clearly isn't going to make that happen, Amazon could easily do it -- IF the camera were actually in stock. So I'm starting off on a bit of a frustrated note here...

[miguelvp]

Decompiled, SEEK thermal app.

Decompiled or unzipped only? 
JAR (file format)

JAR files are fundamental archive files, built on the ZIP file format and have the .jar file extension.

It has java source, so as he stated it's decompiled. Of course the image processing and filtering routines are in the library so no source for those, but you can see where they are used, same with the max min temperature.

[eneuro]

It has java source, so as he stated it's decompiled.

@blackboxdisease  Yep, I had problems to view this google folder so had no idea that there are very nice looking text files 
I have latest com.tyriansystems.SeekThermal_1.4.0.2-20141113.apk  unzipped in other folder and didn't looked into your sources what decompiled apk version can be in this source.zip ?

Update: It looks like they use http://en.wikipedia.org/wiki/Kelvin as absolute temperature and it is converted to android user interface according to choosen units:  Kelvin default, Celcius and Fahrenheit

Code: [Select]

private float kelvinToChosenUnits(float f)
    {
        switch ($SWITCH_TABLE$com$tyriansystems$Seekware$enums$SeekwareThermographyUnit()[units.ordinal()])
        {
        case 3: // '\003'
        default:
            return f;

        case 2: // '\002'
            return f - 273.15F;

        case 1: // '\001'
            return 32F + 1.8F * (f - 273.15F);
        }
    }


They hardcoded maximum threshold temperature to 523.15*K ~ 250*C and minimum to 253.15*K ~ -20*C 

Code: [Select]

public float getMaximumThresholdTemperature()
    {
        return kelvinToChosenUnits(523.15F);
    }

public float getMinimumThresholdTemperature()
    {
        return kelvinToChosenUnits(253.15F);
    }

Choosen emissivity in Android app with a few other parameters is passed to this *.so native Linux library on Android device:

Code: [Select]

SeekwareNativeLib.ThermographyCalculate(bytebuffer, bytebuffer1, floatbuffer, 32448, 32448, 0, 0, 208, 156, emissivity, minMaxTemperatureBuffer, bytebuffer2);

So, this java source code do not show us how absolute temperatures are obtained on Seek output without disassembling this libSeekware.so  or designing own alghorithm for this based on data we can catch from USB frames.

Code: [Select]

libSeekware.so: ELF 32-bit LSB shared object, ARM, version 1 (SYSV), dynamically linked, not stripped

But ok, it is nice to see any source code to have overview of program flow 

[blackboxdisease]

@eneuro it's the latest from the playstore. I was hoping that maybe though one cannot get access to the firmware, it may be possible to change how it is processed/rendered in software. I find it hard to believe the camera itself can process real time data the way it does, and think the apk uses the phones hardware to do all the image processing. I'll post the source on dropbox as well then.

here is the dropbox link. archived in tar.gz
https://www.dropbox.com/s/s9bj6k9w7v9zmo9/source.tar.gz?dl=0

[miguelvp]

The actual processing is done by the phone, but the code is in libSeekware.so

Do you have any way to decompile that, or is it part of that new drop that you posted?

The decompiled source is not useless because you can at least tell what frames they are passed into the library.

[eneuro]

@miguelvp Yep, didn't tracked what is pased there and from which frame it comes, but 32448=208*156 might be buffers size and calculated temperatures stored in floatbuffer.

Code: [Select]

SeekwareNativeLib.ThermographyCalculate(bytebuffer, bytebuffer1, floatbuffer, 32448, 32448, 0, 0, 208, 156, emissivity, minMaxTemperatureBuffer, bytebuffer2);

Now we can look for these unknown buffers in source code, while emissivity is float number as remember from quick review of this code and it is known.
We image size as well and not only 206x156 oryginal Seek sensor resolution is passed there, so maybe they use this 207th column in those calculations too.

@blackboxdisease I've already downloaded source.zip file you provided before so source part I've included are from there

[eneuro]

After removed hot iron by random noise at histogram average and some stronger averaging of this previous output than of course as expected nice circular inside gradients in the corners detected, while now there is is whole iron LUT applied to smaller thermal range while hotest iron was removed from oryginal output.

We have ONLY around 200 pixels min max range now, so if we applied only part of this iron lUT like before of course such strong gradients were not visible, but OK it shows that they exists.

It could be interesting to see some Flir E? at 320x240 resolution output, apply reverse LUT to have oryginal temp and then remove hotest objects from scene like in this experiment and apply the same full iron LUT without absolute values mapped to LUT colors and maybe we could get supprised what we get  too
Need some Flir 320x240 raw data 16bit  PNG image in similar scene like this without any LUTs applied 

So lets try to apply corrections, but at the end of the month will publish this, while I'd like to see first how good Seek app update as they promised will be.

[mikeselectricstuff]

The actual processing is done by the phone, but the code is in libSeekware.so

Do you have any way to decompile that, or is it part of that new drop that you posted?

The decompiled source is not useless because you can at least tell what frames they are passed into the library.

It would be interesting to know the balance of what happens in the phone vs. the firmware.
I'd be surprised if the 30->9fps filtering wasn't in the firmware - what framerate are people seeing coming over USB?

[miguelvp]

It would be interesting to know the balance of what happens in the phone vs. the firmware.
I'd be surprised if the 30->9fps filtering wasn't in the firmware - what framerate are people seeing coming over USB?

You are right on the firmware doing the 30 to 9fps filtering and as you showed in your video it grabs many frames into one.

The fastest time from frame to frame I've seen it's around 123 ms and that places it at 8fps, but that could be partially because of the processing. I'll  try a version that doesn't do any processing at all but just dumps delta frame times.

155 ms is the norm for what I've seen, making it 6.45 fps, pre-cal frame takes 185ms and the cal frame takes 186ms, but I'm not using a very accurate clock like the query performance counter, just c# DateTime class.

I'm not sure if the multiple frames are an attempt for super resolution.

[cynfab]

Here is an image of my hot soldering gun taken from the same point as my Seek, but with my E4+.
Differences are in the field of view, orientation, and ambient temp which was about 18C

  enjoy
   ...ken...

[eneuro]

but I'm not using a very accurate clock like the query performance counter, just c# DateTime class.

No precision clocks needed if 20000 frame counter in image sensor data dot[40] had this value after about 10 minutes as  @cynfab  posted many posts ago.
We've seen 4 increments in this field after longer time period, so quick calculation leads to conclusion that:
20000/10min/60s= 33.3/s
While we have 1 frame per 4 increments  than we have 8.3 fps available when catching different frame types 0x3 0x6 etc.
Now question is if this value in sensor frame 208x156 in dot[40] is really frame counter and it looks like it is about 4 difference between next catched from USB, so maybe it is droped in Seek firmware and inserting a few NOPs (usually 0x00 depending on MPU) could disable this frame rate reduction 

@cynfab Thx for this E4+ iron test data. Maybe do you have this Flir's  Hot Iron oryginal LUT, while I'd like to make reverse LUT and obtain grayscale source thermal scene catched by Flir before this LUT was applied?

Anyway this Flir note max>280*C means that iron temperature was 300*C as you mentioned before, while it shows on temperature LUt scale maxium value 138*C only? 

And the most important thing-What emissivity you had set in Flir equipment?
 
Update: Flir have only jpeg file formats? Thermal image destruction close to those LUT scale black labels looks very bad and a lot of more work needed to remove unwanted data from this scene including  iron image inside...

It looks like not only image around labels is destroyed by thermal image itself too 


@cynfab Did you changed jpeg compression level and have such option in Flir E4+ , or modified somehow this thermal shot taked from Flir in image editor and saved in too low quality, so there is so bad jpeg quality which simply destroyed this thermal image and there is no sense to try to do any reverse LUTs on this data?

8x8 jpeg compresion blocks destoyed details in this Flir E4+ 320x240 thermal image 

[cynfab]

The emissivity was the default 0.95.
I have no clue why the temp scale was 138C while the max tip temp was >280C.
That is an image from the camera, so you can extract the thermal image (as well as the visible camera image) with exiftools or Flirtools as detailed in the exceeding long E4 teardown thread, or maybe E4 useful info thread. IIRC, the iron palette file is also readily available.

IMHO it remains to be seen as to how useful the Seek Imager will be to pro's. To the average joe, who can part with ~US200, it's a cool toy. To folks who can't afford an E4 or don't want a Flir One, it is at least something. To folks like me and probably a few others on this forum, it is a really interesting device which maybe some day it will produce data that MAY be useful in an embedded application.
Time, research and innovative/motivated folks will make the difference.

   ...ken...

[miguelvp]

Yes, I did look at that data posted just 3 pages ago and it does increment by 4 at the beginning but then it increments by 8, also frames ID 6 are missing, and calibration takes 17 counts per calibration.

Frame count at boot is 4 frames per ID 3, but from ID 1 (calibration) to ID3 visual it goes up by 5, but from the last visible until the calibration frame it goes up by 12.

So calibration takes 17 counts, visible frames are 4 count at boot and later on 8 count, therefore halving the frame rate.

But that count is interesting, I wonder when it slows down to half of the initial frame rate so we can figure out the real frame rate. I'll investigate that value as well as putting some precision timing.

[eneuro]

That is an image from the camera, so you can extract the thermal image (as well as the visible camera image)...

OK. The question was if is it possible to set jpeg compression level in Flir E4+ as well as maybe set other lossless image formats like our nice 16bit PNGs, so one can grab high quality image not jpeg like above which for futher image processing is useless due to those 8x8 compression blocks which as I've shown simply destroyed thermal image.
It is not such a big deal to apply reverse thermal palette with a few lines of code in OpenCV, but problem with this image from Flir E4+ camera is it is jpeg at bad quality, so is it possible set in this camera jpeg quality to 100% etc or switch to other lossless thermal image formats and simply store them on SD card and then plug to PC and have high quality thermal images from this 320x240 camera?
It looks like newbe question, but didn't use this Flir E4+ device, but while they lock resolution to 80x60 in oryginal not hacked version it wouldn't suprise me if there were no chance to overwrite this jpeg image quality to 100% to make those images taken from this camera more usefull 
But ok, I hope I will find Flir E4 manual and will review its options, while probably it is available for download 

Update: Ok we are on a good way there flir: Flir E4,E5,E6,E8 with MSX Enhancement

File format: radiometric jpg
Simultaneus storage of IR/Visual/MSX images

Only this radiometric jpg available? No other file formats supported? 

Hopefully our investigation goes in good direction:
Understanding Proprietary Infrared Image Files

This paper outlines the limitations and hurdles that thermographers must face when dealing with infrared images where the radiometric data is stored/encrypted within a proprietary file format (.IMG, .JPG, .SIT, .IS2, .IRI, .ANA, .TIF, .FTS, etc.).

But we have choosen in this thread non-patented replacement for GIF http://en.wikipedia.org/wiki/Portable_Network_Graphics - 16bit PNGs and it looks like is good choice, while we can do whatever we want with those thermal images taken from Seek camera and have also FITS image format in hands with contures supported back to 80's even by old NASA archives

[cynfab]

Dump the exif data for the .jpg using exiftool and you will see that it is not your usual .jpg There is lots inside to play with.

Example of exiftool usage:
( I didn't come up with this command, it was posted by someone far more skilled at exiftool than I) (eevblog is just full of those folks)

Code: [Select]

exiftool Hot_Iron_E4.jpg -RawThermalImage -b | convert - -interlace none -depth 16 r:- | convert -size 320x240 -interlace none -depth 16 -endian msb gray:-  Hot_gray_16.png

Results in the attached 16bit .png

[eneuro]

Dump the exif data for the .jpg using exiftool and you will see that it is not your usual .jpg

No plans to use Flir thermal cameras for the moment, just wanted to do some image processing with this attached by you file, but from link I've provided above:

Note: Do not be confused. FLIR uses file extensions of .JPG, and .TIF which are also used as extensions for digital photo files; however, the FLIR file structure for the radiometric image is completely different. You cannot get radiometric data out of FLIR .JPG or .TIF files without using FLIR’s Proprietary software).

It explains everything, I was a little bit supprised what I saw in this attached .jpg using Gimp-will try imagej2 later 

[miguelvp]

That has a lot of detail. Here is in 8bits so people can see it

[tomas123]

Example of exiftool usage:
( I didn't come up with this command, it was posted by someone far more skilled at exiftool than I)

you can use my tutorial
https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg348398/#msg348398

or see my signature for more informations

@miguelvp
in level stretched 8 bit you can show more informations 

[miguelvp]

Thanks, I'm at work so only had ACDSee to work with it and it's really not that powerful using 16 bit greyscale

[blackboxdisease]

I'm not sure if this will help. I couldn't find a way to save it but there is an online disassembler.

http://www2.onlinedisassembler.com/odaweb/owtD0r/0

good luck.

[eneuro]

in level stretched 8 bit you can show more informations 

But one can miss that there is a lot of bluring in this Flir E4 image and 8bit images-no 
Overlayed hotest iron object in the scene with oryginal 16bit gray PNG histogram average and left pixels around  untouched than applied iron LUT.

Of course nothing to compare to Seek while only 2 frames were used without any rows corrections etc.
However, as expected when you download first file RGB version with LUT applied hot_iron_flir_e4.iron_replaced_with_small_noise_around_average.lut.png that this image was blured or another way smooth filtered, so while pixels added by my software later are not blured huge averaging visible-2 to 3 pixels affected at the edges.
This image is created for Flir's MSX 

Anyway not a great scene for looking for gradients in Flir Ex while there is a lot of stuff in background.

NOTE: black image is 16bit gray untouched oryginal Flir PNG posted there by @cynfab before whole iron LUT were applied to its min max values.

Update: When looked again to this oryginal .jpg taken from Flir E4 by @cynfab it looks like Flir made such trick there that... they simply made temperature scale up to 138* and then they land with something similar to my modyfication, while now thermal LUT is able to display much more details at lower temperatures from the scene, but of course this white 138*C temp is not iron temperature but only SATURATED to this value to be able see anything while they had this very hot iron close to 300*C .
Tricky Flir, so probably this mark at the left is notice that there are temperatures higher than >280* in the scene. 
Any other explanation for this trick they made with very similar iron LUT I've implemented in my software as Flir's ones in this thermal image posted by @cynfab ? 

It was very interesting LUT manipulating excercise 

[tomas123]

there is a lot of bluring in this Flir E4 image

most of blurring comes from the lense
see my images here
https://www.eevblog.com/forum/testgear/flir-one-thermal-imaging-camera-teardown-and-hacks/msg551882/#msg551882

one square = one sensor pixel of Flir Exx

the result is really sharp (thanks to good optics of the Flir Exx)
Wow

https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg343791/#msg343791

[Fraser]

@Eneuro,

It would appear that you are discovering how FLIR make their E4 images look 'nice'. My 1997 FLIR PM570's were the first Microbolometer handheld units that they made. The images contain all the detail needed for professional thermography but are the images 'pretty' ? Nope, they contain noise as FLIR had not conquered that particular challenge. They admitted as much to me when I tested their first camera to arrive in the UK.

Since the PM570 the FLIR microbolometers and image processing techniques have developed. My PM695's offer better performance and so they should at $56000 each ! Do the images still have noise in them....yep you bet they do. If you select a span of 2C you see the noise as I would expect. When increasing the span to 10C or so the noise disappears. Crucially for a thermographer, the professional cameras present the DATA that is needed for radiometric measurements.

Along comes the FLIR E4 at a very appealing price and I expect poor imagery from it. When it arrived and had been upgraded to 320x240 I was very surprised at the clean images that it produced. The noise was heavily suppressed making the image attractive for the casual user. The Ex series have very small lenses and I would have expected quite high image noise which could bother some users. If FLIR were intending to make a camera that produces 'pretty' thermal images, then they have, IMHO, succeeded.

Now as to whether a thermal image should be heavily processed or not very much depends upon the intended purpose of said images. A thermographer needs clean data with which to work and the presence of limited noise in an image is not an impediment to such use. Obviously high noise levels are not acceptable ! Now a photographer or artist has different needs and wants the images to be as 'clean' as possible without losing definition.

OK this is where I comment on the fine work that you guys have been doing with the SEEK. IMHO the SEEK is a ground breaking unit that may make others in the TIC OEM world reconsider sales strategies, then again maybe not. I am impressed with what has been achieved at the price point but I am disappointed with the image in terms of noise content, clarity and thermal gradient. As received the SEEK leaves a great deal to be desired when compared to other thermal cameras of similar resolution such as my NEC F30S, Testo 880-1 or the FLIR E4. There is only so much allowance that can be made for the low cost if the image proves unusable for many tasks. An old saying of "spoiling the ship for a ha'porth of tar" may apply to the SEEK product.

I see much work being done to reduce the noise content of the images. I remain impressed by the expertise that is being demonstrated here. The SEEK certainly could benefit from better image processing. But when is image processing too much processing ? As you point out, the FLIR E4 appears to be heavily processed to obscure its optical block shortcomings. Does it still do its job, well yes it likely does. Is it a perfect solution ? maybe not. Let us not forget that we are working with an uncooled microbolometer and not a cryo-cooled FPA. Microbolometers are inherently noisy beasts and need taming with processing....just the right amount of processing though 

If you want to see a really nice thermal image, take a look at a scene with a cryo-cooled thermal camera like the PM550 from circa 1995....... noise, what noise    Sadly I do not own a PM550 at this time but I do have a FLIR SC3000 that is currently on the repair bench. If/when I repair its Stirling cooler controller I hope to get some very nice quality images from its Quantum Well FPA.

I am not a FLIR 'fan-boy', just a long term user of their cameras (and the AGEMA's before them). They have a history of making some of the best thermal cameras that money can buy outside of the military.

Aurora