Yet another cheap thermal imager incoming.. Seek Thermal

[callipso]

Weird thing about flatfield calibration...if i do the magnet trick, obviously it takes whatever it sees and uses that to sub out each frame. But on the very next flat field where i let the shutter work, a ghost image of the last flat field (like hot objects) is still present, but less. l think Seeks software is averaging flat fields. A bug?

Yea, I've noticed that in Mike's video too... They seem to be averaging just about everything they can to produce a less noisy image, but in this case that would be a wrong assumption...

[eneuro]

Not sure I can agree about the focus statement.

Could you in spare time try to make thermal images with Flir which has this MSX available of hot gorund like those from http://wildfiretoday.com/2014/10/27/a-thermal-infrared-camera-attachment-for-smart-phone/   shown by me in prev posts and its grayed visual there?
Grayed image there in attachment as well as attempt to add edges in pure way simply just by overlay so easy to improve, but in the case of ground it could be interesting to see what this "amazing" Flir's MSX will provide  in this case.
It could be great to make small fire on the ground similar to this and do those thermal shots after it burned to get at least one nice hot spot.
First image attached is Seek Thermal from link above with such simple overlayed visual edges-lets call it visual curves, to avoid any Flir's patents 

Without adding hi res visual context those low res Flir's thermal cameras as well as Seek Thermal probably would give out only thermal blobs of averaged temperatures not image 
External lens can help but Seek Thermal gradient issues make this thermal output not reliable.

[Fraser]

@eneuro,

I am not sure my wife would take kindly to me setting a fire in her manicured garden 

Edit:

Post was too long and rambling - removed. Copy kept in case content wanted by anyone.

Aurora

[jaybeez]

I've read that these lenses work with the therm-app, as a cheaper alternative to opgal's offerings.
http://eom.umicore.com/en/infrared-optics/product-range/35-mm-f-1.1/
with 3D printing would it be worth looking into making new housings for the seek to incorporate a lens like this? Mike? your unit already had the lens housing removed. anyone know which lens gives a 1x image?

[bktemp]

Does anybode know how to convert the 16bit image data to absolute temperature values?

[efahrenholz]

I hope some of the seek engineers are still following this thread, because I have a pretty good question for them...

Why is there a ghost image that slowly reappears strongest right before a flat field event? It doesn't have to be an intentional image  (like holding the shutter open during a flat field to see it), I notice it creeps back in after a calibration, no matter what it is. Sometimes the flatfield image shows up as blocky hotspots, and it gets hotter right before a fresh flat. Whatever the sensor looked like during the calibration, that image slowly appears right before a new flat field. Even fixed pattern noise shows up. What...is going on?!

I just want to know the math involved in how you subtract the flat from each frame. I don't think its a trade secret or anything, but its clearly some kind of bug. I know sometimes I get 5 frames on a flatfield, sometimes it's only one frame. I tested this while waving the camera with the shutter forced open.

[eneuro]

To be clear on MSX. 
...
The idea of fusion was enhanced by allowing the transparency of the visible light image to be adjusted in order to have lower impact on the thermal image data that was being displayed.

I saw this Flir's patent claim and while they make this transparency they have to modify thermal image in a way I do not like at all to show transpareny on their output images,
so they claim in their patent file that they avoid thermal image change but they have to do it while nobody could notice those added edges if they do not changed thermal image 

So, thank you very much for long overview, but just developing my own software and I have another vision of how this fusion might be and do not care too much what Flir claims in their patents while I wouldn't like again use the same word for this since those patent claim is... b*llshit   

I've choosen this difficult ground gray image as example just to see what Flir's MSX will do in this case?
I could do such test but do not plan to buy any Flir cams with their MSX, because I do not like those artifical edges on thermal image affected by added edges from visual in a way they did it.
I prefer simply make those edges white or black depending on average temperatures on thermal image while I used to use iron palette where whit eis top hot and black is minimum cold.
It can be done better and even simply overwrite with black or white edge from hi-res visual 5MP-8MP camera (full HD) can in many cases look better than messing with this Flir's MSX.
So, that is why I'd like to see how grass or ground in the garden with hot spots will look like with Flir's MSX enabled, while I hope it possible turn it off or change its tarnsparency level etc., but there are endless amount of edges when one looks at green grass, leafs in the wood or even ground with stones and other small things...which will make more challenging to detect edges and remove unwanted from output image for fusion with thermal IR and visual camera. 

Flir's MSX? No thanks, I can do it in a way I've did many years before any Flir software exists using classic image processing tools available for years and will be able to find science publications many years old if they tried to prewent me from doing this now...

[miguelvp]

Does anybode know how to convert the 16bit image data to absolute temperature values?

https://www.eevblog.com/forum/testgear/yet-another-cheap-thermal-imager-incoming/msg524602/#msg524602

Those are pre getting the real images but the range is about the same.

208x156 16 bits unsigned only 14 bits used.
Pretty much you want to capture the max and min value (ignoring dead pixels and reference ones, under 0x800) and then stretch them ramp them from min to max to fit on a 256 look up table. So (max-min)/256 should get you started.

Edit: Look at sgstairs code posted somewhere in this thread too, also frenzy on post 123 has code but that was before we got the real raw data.

Edit for link to sgstairs post with link to the git depot and what is needed to make it work on a PC running windows:
https://www.eevblog.com/forum/testgear/yet-another-cheap-thermal-imager-incoming/msg534497/#msg534497

[Hyperion]

So I've managed to reduce the noise using a convoluted method. You guys who have programming skills can probably automate/integrate this whole process. You will need Adobe Photoshop CC, or Photoshop CS6 Extended, or any earlier version of photoshop that supports the Smart Object Stack Mode(s). Note that some versions such as CS6 Standard featured a bug that showed the Stack Mode option in the menu, but it was actually disabled and could not be used in the standard version.

Although the final image will be much better than any image or still from a video from the Seek, you must manage your expectations, as this really just removes the noise and does not improve sharpness etc. It is also time consuming.

1. Start recording a video and setup Seek camera in a tripod or on something stable. You can only record stationary scenes and any moving objects will end up blurred / blended, so it's best if everything is stationary.
2. Leave it alone for ~3 to 5 minutes. We are trying to capture many shutter events after which the noise pattern will change, they usually happen every 2 to 10 seconds.
3. Stop recording and put file on computer.
4. Follow this guide (https://www.isimonbrown.co.uk/vlc-export-frames/) to use VLC media player to save its output as images. In step 7, set the value to something between 60 and 90; given the framerate of the video is 14 fps, this works out to be several seconds and enough for 1 or 2 shutter events to have taken place. For the example below, I used a value of 60 on a roughly 5 minute video and ended up with 64 usable images. You could also use VirtualDub to export the frames. (You can choose to output every single frame of the video however this will greatly increase processing (CPU) time of later steps.)
5. Go to the folder it exported the images to. You have to delete those at the start and end which are not spatially aligned with the bulk of the rest. eg. when you moved the camera to start/top recording.
6. Open images in Windows Photo Viewer or your application of choice and hold whichever shortcut scans rapidly through the images (eg, Right arrow key) to ensure they are all aligned
7. Open one of the versions of Photoshop listed above. Go File -> Scripts -> Load Files into Stack
8. Go to Select -> All layers, then, Layer -> Smart Objects -> Convert to Smart object
9. You can now go into the Smart Objects menu and choose the Stacking mode. For the example below I used Mean which is arguably the best.

Alternatively, programs like PhotoAcute that can do stacking / blending and superresolution can achieve the same result, however, these kind of programs tend to require subscriptions / purchases to use them fully. That is why I suggest photoshop and although it too isn't free most people have better access to it.

Below is my computer tower's internals made from 64 stacked images: I will look into using smaller numbers of frames after work tomorrow.

• 

[RAWebb]

Close up lenses are used to provide detailed images of PCB's etc.

I've put up a simple holder for the 20mm ZnSe lenses over at Thingiverse. It's just a friction fit which depends on the asymmetry of the camera face to hold it in place, but it seems to do the trick. Should do well enough to keep handy in the toolbag. http://www.thingiverse.com/thing:525605

have any pictures you've taken with this setup?

Yep, there are a couple at Thingiverse. I'll stick 'em over here as well, one each with a 100mm and 50mm aux lens.

They are noisy and the resolution isn't world-changing but it's better than the "Ow! $#!% my finger!" test to answer the question "What's pulling all the current on this damned board?"

[efahrenholz]

@Hyperion,

The noise on your non stacked frame is pretty acceptable. That's probably because they already do a ton of post processing on the image we get through the seek app. I'm downloading vb express and I'll see if I can mess around with the code to do some stacking and black pixel fill in, but I'm not well versed in vb...so don't expect anything amazing. We need to get cracking on this if we are going to improve the image quality beyond what the seek software engineers have done. There may not be much left to be improved on...but I hope there is more we can do.

[eneuro]

Does anybode know how to convert the 16bit image data to absolute temperature values?

Those are pre getting the real images but the range is about the same.
208x156 16 bits unsigned only 14 bits used.
Pretty much you want to capture the max and min value (ignoring dead pixels and reference ones, under 0x800) and then stretch them ramp them from min to max to fit on a 256 look up table. So (max-min)/256 should get you started.

I made a look into that code since I'm interested in own Linux USB driver if they resolve those gradient issues.

This hardcoded 0x800 looks bad in this code, as well as fitting to 256 LUT- while we have 14bit using 1024 LUT could give much better results.
Also averaging/convolution  methods used in this code are computational not efficient- it can be done much faster using a little bit memory which is not concern on modern devices while we'are processing 208x156 raw sensor data.
Also it does not use any moving average between frames, so no chance to make this image less noisy by mounting this thermal dongle on tripod to avoid any movements while it does not have any stabilization.

I suggest reading image processing publications to make this code more efficient, however USB communication part might be usefull if someone have no chance to make USB sniffing 

[eneuro]

So I've managed to reduce the noise using a convoluted method.

You mean http://en.wikipedia.org/wiki/Moving_average ? How many frames used and what is output frame rate from this sensor hardware catched by USB?

[eneuro]

Yep, there are a couple at Thingiverse. I'll stick 'em over here as well, one each with a 100mm and 50mm aux lens.

They are noisy and the resolution isn't world-changing but it's better than the "Ow! $#!% my finger!" test to answer the question "What's pulling all the current on this damned board?"

One question-is it possible from Seek Thermal application save output images like those you provided not in JPEG format, but in other losless like RGB PNG for example?
Aren't those saved JPEG images infected by JPEG compression level-what I mean-do they looks the same noisy on the screen before saving them?
Is it possible to force save them with 100% quality level as JPEG or losless PNG?
Those images have wisible 8x8 pixel boxes and it is interesting it is JPEG quality level less than 100% or simply it is intentionally made to fit into those military ITAR standards.
This gradient effect can also be deliberate action pro ITAR rules,
but it is only speculation, while i'm not thermal cameras manufacturer and never do not followed those ITAR rules http://en.wikipedia.org/wiki/International_Traffic_in_Arms_Regulations which might affect somehow such kind of equipment designed for mass production...

Probably Seek Thermal must hate this found gradient issue which could be more difficult to catch without USB row data from sensor (as well as dead pixels) and @Mike made additional video on that  gradient

[efahrenholz]

So I've managed to reduce the noise using a convoluted method.

You mean http://en.wikipedia.org/wiki/Moving_average ? How many frames used and what is output frame rate from this sensor hardware catched by USB?

It looks like he converted the video to still frames and just stacked 64 frames using averaging math. This is the same as using a webcam on a telescope to get sharper planetary imaging. It gets the actual true image by averaging out abnormalities caused by atmosphere and noise. Many webcams can capture at over 30 fps...this fast frame rate helps overcome tracking errors or the need to track at all. Basically you can get a really good image every second.

For us, we need the ability to get the full framerate off the sensor, because right now it's only allowing around 9fps. Its averaging 5 frames together for each visible frame. By my math its about a 45fps sensor. Its a shame ITAR limits the fps on such a low resolution sensor.

I did manage to compile the seeker windows program on a dell venue tablet and run it with the camera plugged directly into the usb port. I notice a lot of white specs along with dead pixels and the null pixels. We'll just call those patent pixels.

[miguelvp]

I agree, larger than 256 LUTs can be used specially if they are color

[datsthat]

This is the reason why gradient is an issue for my useage is because I can't set temp low enough to ensure it catches lifeform without the gradient interfering.


Temp 50 degrees.  Wooded area with my son

[eneuro]

For us, we need the ability to get the full framerate off the sensor, because right now it's only allowing around 9fps. Its averaging 5 frames together for each visible frame. By my math its about a 45fps sensor. Its a shame ITAR limits the fps on such a low resolution sensor.

9 fps not so bad -if moving average of 8 were used than even on MPU it can be fast while divide by 8 in assembler can be performed by shifting bits and will be fast -no integer divide needed, and output 1 Hz frequency could be fine in many applications if we were able improve image quality 

I did manage to compile the seeker windows program on a dell venue tablet and run it with the camera plugged directly into the usb port.

Nice, so if you could add a few lines of code where raw sensor data is read and output it simply as unsigned int or uint16_t to text file (than compress) without any comparisions and changing to 0 those pixels with 0x800 condition in this code-raw data as it comes from the sensor in format below (and include also thermal image of catched object as well as its visual image to see what light conditions it was: dark, normal working light) they maybe someone from the forum could be able to figure out what is in those last 2 columns in those raw sensor images.

Code: [Select]

/**
 * (C) 2014 Eneuro
 */
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include <string.h>
#include <time.h>

void random_init() {
srand(time(0) );
}

double random_double() {
return ((double)rand())/RAND_MAX;
}

unsigned int random_14bit() {
// 2^14= 16384
unsigned int pixel= random_double()*16384;

if(pixel>=16384) {
pixel= 16384-1;
}

return pixel;
}

int main(int argc, char *argv ) {


unsigned int i, j, k, l, m, n;
unsigned int y,x, w= 208, h= 156, wh= w*h;
unsigned int img0, *img;
unsigned int f, frames= 8;

random_init();
img= (unsigned int*)malloc(sizeof(unsigned int)*wh );


// frames
for(f=0; f<frames; f++ ) {

// frame
// Create random image with 14bit data or read from sensor
for(i=0; i<wh; i++ ) {
img[i]= random_14bit();
} // Create random image with 14bit data or read from sensor

// Output frame
// rows
for(y=0; y<h; y++ ) {
// column
for(x=0; x<w; x++ ) {
img0= (unsigned int)img[y*w +x];
printf("%05u ", img0 );
} // column
printf("\n");
} // rows
printf("\n\n");
// Output frame

// frame

} // frames

return 0;
}
so, each frame in this text file could look like this below separated by 2 empty lines (it is LF 10 dec codes on Linux and CRLF 13 dec 10 dec on Window$ as text new line character(s) ).
Output text file looks like this in the case of C source code shown above:

This way if we compress such text file with many frames of row data from sensor one could for example to investigate how this gradient evolves in time.
You can add delay and output to such text file for example sequence of 8 frames than delay 10 seconds another one etc, for 10 minutes and this way if you had this thermal cam on tripod pointed to flat surface we could try to see how it changes in time, by applying moving average to each 8 frames and another ones after delay.

When applied from 4-1024 convolutions to some raw USB sensor image pointed to such flat surface I've got something like this

but it was 8bit per channel RGB, so 14bit degraded to 8bit by software we are talking about and gradient image made by @marshallh,
but it could be interesting to see how it changes in time without any postprocessing-just raw 14bit sensor data with iron 1024 LUT or 14bit.

BTW: Compressed 8 frames of 14 bit raw data in text format shown above is <1MB so easy to exchange for investigation of gradient issue and those strange 2 last columns in each frame 

[ricksastro]

Does anybode know how to convert the 16bit image data to absolute temperature values?

Those are pre getting the real images but the range is about the same.
208x156 16 bits unsigned only 14 bits used.
Pretty much you want to capture the max and min value (ignoring dead pixels and reference ones, under 0x800) and then stretch them ramp them from min to max to fit on a 256 look up table. So (max-min)/256 should get you started.

I made a look into that code since I'm interested in own Linux USB driver if they resolve those gradient issues.

This hardcoded 0x800 looks bad in this code, as well as fitting to 256 LUT- while we have 14bit using 1024 LUT could give much better results.
Also averaging/convolution  methods used in this code are computational not efficient- it can be done much faster using a little bit memory which is not concern on modern devices while we'are processing 208x156 raw sensor data.
Also it does not use any moving average between frames, so no chance to make this image less noisy by mounting this thermal dongle on tripod to avoid any movements while it does not have any stabilization.

I suggest reading image processing publications to make this code more efficient, however USB communication part might be usefull if someone have no chance to make USB sniffing 

I'd also suggest dithering (making slight movements occasionally) the camera if on a tripod and then aligning the images on the subject is an excellent way to minimize FP noise (especially if you use a Sigma Reject type of combining algorithm).    I've been playing a bit the the PC SW from here and using my astro SW (which works internally with .fits files which are B&W and uses up to 32 bit ints or even 64 bit Float pixel values).   Need to dedicate some more attention here when I get some time.   It's a simple format, so I should be able to output in the .FITs format.

[miguelvp]

I did manage to compile the seeker windows program on a dell venue tablet and run it with the camera plugged directly into the usb port. I notice a lot of white specs along with dead pixels and the null pixels. We'll just call those patent pixels.

That was my intent for usage as well, I bought my wife a Venue tablet last year and she doesn't use it for fear to damage the screen.

It's been a busy weekend so no much progress from me. I'm starting an app from scratch with C++, so far I got the USB enumerated and very close to open it.

[efahrenholz]

Yes please do code in c++. I can actually read that a lot better than c#.

[ricksastro]

I did get the C# program modified to save a 16 bit unmolested, unscaled image from the Seek.    Since 16 bit grayscale doesn't seem to be supported, I had to do 48 bit color (with R=G=B).    But it's a true 48 bit PNG file, openable by Photoshop (or my astro software).

Seems to be using the range from 31,500 = ice to 33,000 = 97F router.

Early days on this...more playing to come.  Need sleep

[eneuro]

Since 16 bit grayscale doesn't seem to be supported, I had to do 48 bit color (with R=G=B).

Gray Alpha channels in PNG could be used-easy to write using libPNG 
http://www.libpng.org/pub/png/book/chapter08.html#png.ch08.div.5.4

However, this FITS image format (http://en.wikipedia.org/wiki/FITS) can be very usefull there 

"Flexible Image Transport System (FITS) is an open standard[3] defining a digital file format useful for storage, transmission and processing of scientific and other images."

There is a few links to FITS docs, libs, sample files on NASA web pages:
The FITS Support Office at NASA/GSFC  http://fits.gsfc.nasa.gov/
http://fits.gsfc.nasa.gov/fits_documentation.html
http://fits.gsfc.nasa.gov/fits_libraries.html

I've downloaded for research this simple:
The Interactive FITS File Editor   fv: http://heasarc.gsfc.nasa.gov/docs/software/ftools/fv

http://heasarc.gsfc.nasa.gov/docs/software/ftools/fv/fv_download.html
ftp://heasarc.gsfc.nasa.gov/software/lheasoft/fv/fv53.exe
ftp://heasarc.gsfc.nasa.gov/software/lheasoft/fv/fv53_pc_linux64.tar.gz

With funtools library it should be easy add support for FITS images to application
https://www.cfa.harvard.edu/~john/funtools/
https://www.cfa.harvard.edu/~john/funtools/funtools.pdf

There is Definition of the Flexible Image Transport System (FITS)

Using mentioned above fv it is easy view and modify eg. sample Hubble telescope image I've downloaded from NASA website for testing (it is just 16bit):


Which is more interesting contours can be added to those FITS image files as well as other data incuding tables, etc, so Flir's MSX no longer needed to add edges while we can output thermal image in this FITS format with contours


No problem to load such FITS image even using GIMP, but probably this data is scalled to 8bit I guess,
so using software like fv is adwantage.

I've suggested simple text format for investigation purposes while no problem with bits order in different machines and it is easy to add only few lines of code to output this raw data while we have fixed size Seek Thermal sensor image data, so easier than include FITS library, but while NASA is using FITS and it i sdesigned with backward compatibility for archives purposes, so it is well defined image format 

[ricksastro]

Since 16 bit grayscale doesn't seem to be supported, I had to do 48 bit color (with R=G=B).

Gray Alpha channels in PNG could be used-easy to write using libPNG 
http://www.libpng.org/pub/png/book/chapter08.html#png.ch08.div.5.4

However, this FITS image format (http://en.wikipedia.org/wiki/FITS) can be very usefull there 

"Flexible Image Transport System (FITS) is an open standard[3] defining a digital file format useful for storage, transmission and processing of scientific and other images."

There is a few links to FITS docs, libs, sample files on NASA web pages:
The FITS Support Office at NASA/GSFC  http://fits.gsfc.nasa.gov/
http://fits.gsfc.nasa.gov/fits_documentation.html
http://fits.gsfc.nasa.gov/fits_libraries.html

I've downloaded for research this simple:
The Interactive FITS File Editor   fv: http://heasarc.gsfc.nasa.gov/docs/software/ftools/fv

http://heasarc.gsfc.nasa.gov/docs/software/ftools/fv/fv_download.html
ftp://heasarc.gsfc.nasa.gov/software/lheasoft/fv/fv53.exe
ftp://heasarc.gsfc.nasa.gov/software/lheasoft/fv/fv53_pc_linux64.tar.gz

With funtools library it should be easy add support for FITS images to application
https://www.cfa.harvard.edu/~john/funtools/
https://www.cfa.harvard.edu/~john/funtools/funtools.pdf

There is Definition of the Flexible Image Transport System (FITS)

Using mentioned above fv it is easy view and modify eg. sample Hubble telescope image I've downloaded from NASA website for testing (it is just 16bit):


Which is more interesting contours can be added to those FITS image files as well as other data incuding tables, etc, so Flir's MSX no longer needed to add edges while we can output thermal image in this FITS format with contours


No problem to load such FITS image even using GIMP, but probably this data is scalled to 8bit I guess,
so using software like fv is adwantage.

I've suggested simple text format for investigation purposes while no problem with bits order in different machines and it is easy to add only few lines of code to output this raw data while we have fixed size Seek Thermal sensor image data, so easier than include FITS library, but while NASA is using FITS and it is designed with backward compatibility for archives purposes, so it is well defined image format 

Definitely quite familiar with FITS, as I mentioned in a previous post.   I've used it for years in the astrophotography world.    I chose PNG since it was simpler and supported, worked with both the astro SW as well as Photoshop and just wanted to see the data quickly.   I tried TIFF, but the Studio graphics libraries defaulted to 24bit when saving rather than 48 bit.   PNG just worked.    And neither worked with the 16 bit grayscale natively with Studio.   After some googling, it seems to just be unsupported.    FITS is easily created from the ground up since it's pretty simple, so if I find a reason to do so, I'll switch to it.   It would be more efficient for sure, but it's such a small amount of data we're dealing with I'm not so worried about it if it's simpler.

[ricksastro]

BTW, the gradient is indeed still easily seen on the 16 bit raw output from the camera.  Just set the white and blackpoints to give high contrast and it shows up just like in the app.
CORRECTION TO ABOVE: I was using calibrated frames, so the above statement may not be true...will report and hopefully have examples tonight.

One interesting artifact I notices is a oval light patter in the upper left corner just like sgstairs shown in post 312.

https://www.eevblog.com/forum/testgear/yet-another-cheap-thermal-imager-incoming/312/

Too coincidental to be a random manufacturing glue or dirt I would think...but marshallh's image next to it didn't seem to show it, so it may be a batch thing.

But it's invisible in the app, so it must get compensated for in the calibration.