Yet another cheap thermal imager incoming.. Seek Thermal

[eneuro]

the gradient is indeed still easily seen on the 16 bit raw output from the camera.

How many dead pixels has your sensor? 
I mean those close to black pixels which does not fit into the "good black" hexagonal pattern.

Post your 14bit sensor output to compare with those other 8bit raw sensor images.
It could be interesting to compute percentage of dead pixels in another Seek Thermal camera, but using not touched 14bit raw sensor data-nobody showed such data so far 

Just developing FITS module for 16bit pixel images with contures and PNG RGBA output as well for transparency to avoid Flir's MSX issues in output imaginery from any thermal camera 
fits2rgba tool will apply 14bit iron palette on resized image and optionally add contures transparency in Aplha channel from hi res visual camera 

[efahrenholz]

BTW, the gradient is indeed still easily seen on the 16 bit raw output from the camera.  Just set the white and blackpoints to give high contrast and it shows up just like in the app.

One interesting artifact I notices is a oval light patter in the upper left corner just like sgstairs shown in post 312.

https://www.eevblog.com/forum/testgear/yet-another-cheap-thermal-imager-incoming/312/

Too coincidental to be a random manufacturing glue or dirt I would think...but marshallh's image next to it didn't seem to show it, so it may be a batch thing.

But it's invisible in the app, so it must get compensated for in the calibration.

My raw images don't have any circles in the field but plenty of bad pixels. These things claim to pass QC before they are shipped. I think they are "allowing" a greater amount of bad sensors through because the demand is so high.

[miguelvp]

My raw images don't have any circles in the field but plenty of bad pixels. These things claim to pass QC before they are shipped. I think they are "allowing" a greater amount of bad sensors through because the demand is so high.

At 12 micro meters those dead pixels are not a big concern as long as they don't form large deadzones, it's not a display after all and the whole thing costs less than just the lepton sensor by itself.

So I don't think I would use the term "allowing", it's what it is at this development stage. As long as it can be software corrected the sensors they are shipping are fine for the cost they are offered at.
You can always spend $1000 for the therm-app one if you want a better camera/sensor/optics/etc...

Edit: and they probably have dead pixels as well in their raw data for all we know.

[ricksastro]

One correction...

After more sleep and looking at the code more, I was looking at calibrated frames, where x8000 was added as a bias and the last ambient calibration frame subtracted.  This explains why the ice was < 0x8000 and the router was > 0x8000 (colder and hotter than ambient)

So the thermal gradient may still be introduced by the subtraction of the calibration frame...I'll see tonight when I have time.

I made some mods so I will be outputting 100% using uncalibrated, unbiased frames as well as raw calibration frames themselves.   Then I can perform proper FF calibration (and bad pixel mapping) in my astro software and create calibration frames taken

Trying to get the errors to a bare minimum before  applying any filtering to retain as much detail as possible.

Sorry for any confusion.

[miguelvp]

You can ignore all of the calibration frames if you want. If you look at sgstair's code byte 20 of the raw data (pixel 10) is used to determine if it's a cal frame or a good frame.

Edit: I wonder if the horizontal bands are due to the flag introducing noise:



Edit: picture of Mike's shutter (doesn't seem to be the same pattern on the whole shutter so I don't know) might be just noise from the sensor itself:

[ricksastro]

You can ignore all of the calibration frames if you want. If you look at sgstair's code byte 20 of the raw data (pixel 10) is used to determine if it's a cal frame or a good frame.

I'm going to save off the cal frames separately so I can analyze them and possibly use them for image processing.

[miguelvp]

They already are on a separate buffer

[ricksastro]

They already are on a separate buffer

Yes, they are saved off in the code, but not saved as an image file to the PC.  I'll save multiple copies of both cal and image frames to I can look at them over time and better understand the dynamics of the situation.

[eneuro]

I'm going to save off the cal frames separately so I can analyze them and possibly use them for image processing.

It looks like libPNG outputs very nice 16bit gray images with a few lines of C code:

$ file seek_thermal_sensor_gray_16bit_test_by_eneuro.png
seek_thermal_sensor_gray_16bit_test_by_eneuro.png: PNG image, 208 x 156, 16-bit grayscale, non-interlaced

Probably you wish you had such sensor output from your Seek Thermal on flat surface without any gradients and dead pixels 

They can be read and viewed and processed in 16bit gray mode using ImageJ2 image processing software 


If someone is interested there is C source
http://stackoverflow.com/questions/8818206/16-bit-grayscale-png
it requires PNG library.

It looks like optimum image format for sensor output data, while it can be opened in GIMP and other tools in 8bit mode, while people interested in more details can look into its 16bit data in more sophisticated image processing software capable to play with 16bit gray images.
Than all we need after post processing sensor data frames is output augumented FITS with contours or RGBA with 1024 iron LUT.

If someone could output some 16bit gray 208x156  PNG images (not touched by any preprocessing) using tools I've shown above from Seek Thermal sensor raw image it could be nice to  those last 2 columns or try to predict this gradient 

For my development those test 16bit gray PNG images I've made are fine to simulate Seek Thermal output at given frame rate and see how fast I can process those frames 

[efahrenholz]

@eneuro,

The raw sensor image you are showing looks pretty good compared to all the sensors raw outputs I've seen. Lots of dead pixels, areas where the pixel data is bleached. I honestly don't know what's going on with the white blobs on the sensor. They don't appear to be usable data. So that mixed with dead pixels and the null pixels means these sensors are pretty grade c.

[miguelvp]

he doesn't have a camera, it's just a test image the simulates a raw perfect image

[eneuro]

The raw sensor image you are showing looks pretty good compared to all the sensors raw outputs I've seen.

Yep, this is of course simulation, while tested which PC software can read those 16bit gray images 
So, that is why included links to this libPNG based soft-people who tweak this C# or C++ code can add support for 16bit gray images and output from their real Seek Thermal sensors raw data while nobody so far posted there its not manipulated 14bit output 
A few frames from different sensors could show how those dead pixels are distributed maybe even between different orders numbers and how many it is
More than 1.5%? 

[miguelvp]

Reference pixels or dead pixels, the ones in the pattern are purposely turned off and they are every 15 pixels from this series modulus height: 10, 6, 2, 13, 9, 5, 1, 12, 8, 4, 0, 11, 7, 3, 14
So on the first row is: 10, 25, 40 ...
on the second row is: 6, 21, 36 ...
on the sixteenth row is the same as the first row: 10, 25, 40 ...

Not sure why it never starts on pixel 15 but it just doesn't.

I added code back on page 32, to interpolate those pixels, the same loop can be done to generate them.
Just declare a uint16 array of 208*156,  and fill it with 0x8000, then use that loop and instead of averaging set those values to 0.

https://www.eevblog.com/forum/testgear/yet-another-cheap-thermal-imager-incoming/msg538322/#msg538322

[efahrenholz]

Reference pixels or dead pixels, the ones in the pattern are purposely turned off and they are every 15 pixels from this series modulus height: 10, 6, 2, 13, 9, 5, 1, 12, 8, 4, 0, 11, 7, 3, 14
So on the first row is: 10, 25, 40 ...
on the second row is: 6, 21, 36 ...
on the sixteenth row is the same as the first row: 10, 25, 40 ...

Not sure why it never starts on pixel 15 but it just doesn't.

I added code back on page 32, to interpolate those pixels, the same loop can be done to generate them.
Just declare a uint16 array of 208*156,  and fill it with 0x8000, then use that loop and instead of averaging set those values to 0.

https://www.eevblog.com/forum/testgear/yet-another-cheap-thermal-imager-incoming/msg538322/#msg538322

You did precisely what I described, minus the corners. But 4 data points should be good enough, and from your images there, they look VERY good. In fact, I'd say they look sharper than any other images. The next problem to tackle is the hot pixels. Are they pegged at the same values?

[miguelvp]

I do the corners, not the last two non active columns because, well, they are not active.
Edit: nevermind you mean the diagonals, I didn't do those because it was good enough.

If you only have 3 neighbors then it will use those, if only 2 then only those two.

I've been busy so not a lot of progress, but I want to take several reference images at different temperatures to characterize my camera.

If linear, then on dead pixels the slope will be 0, as in y=0*x+b because the pixel is dead and will no vary. If it only varies a little then it will have a flatter slope but non zero.

But I need to find time to do this.

[ricksastro]

OK...here's a bonanza of files.   I modded the c# program to output 48 bit PNG files directly from the SEEK, uncalibrated and unscaled...direct from the camera.   I just duplicated value for each of the RGB channels.   Not pretty, but it got the job done.   Each frame got an incremented number (whether cal or image), so frame 1 is always a calibrate image, frame 2 starts 3 or 4 real images, then a calibration and so on as the shutter runs.

I then used my astro program to convert to true 16 bit grayscale and I saved in both PNG format and FITS format.   I included the paired raw shutter closed calibration frame (ends in c) taken immediately before the image frame (ends in i).  I have placed the files here:

http://www.ricksastro.com/temp/seekraw/

There are 2 different scenes.   One is taken against flat notebook paper (flatpaper) at ambient 73F. One is an image of my router (router).

To give you an idea of what things look like, here is a screenshot of the router images and the process used in the Astro software (MaximDL).   I set the Black and Whitepoints to linearly stretch and show the real data ranges without altering the data.  The top left image is the raw calibration frame.  The top right is the actual image file...it's incredible that you really can't see barely a hint of the object, obfuscated by noise.

The bottom left is simply the image - cal + 3000 for each pixel.   That's where the magic happens and the veil of noise is lifted.  The 3000 is arbitrary just to prevent <0.   

The middle bottom is that image with a bad pixel map applied (created from a calibration frame to get rid of the dead (black) pixels that are in every raw frame).   Interestingly, I didn't find any fixed pattern hot (white) pixels.    When I isolated the few brightest pixels from several calibration frames, there was no repeating pattern. 

The right-bottom had a simple median neighbor filter applied.

I completely gave up hope of getting anything useful without subtracting the cal frame.    I'd be interested to understand how seek gets rid of most of the horizontal lines.   They appear to be pretty fixed, so maybe some Flat Field is stored off.

I'll try next to create a true flatfield calibration from the paper image and apply to the router image to see how that improves it.

I'm having a tough time analyzing the gradient source, since it's nearly impossible to glean anything from either the cal frame or the flat image frame in isolation.

As you can see on the Screen stretch dialog of the median image, the real data is only really from 2555-3158 (remember, I added 3000), which is only a range of 603 (of course the temp range is only about 25F in that scene).

[efahrenholz]

That image looks shaaaaarp right up to that median filter. I really think there could be a better way. Perhaps you could scan each pixel and compare it to the average of its neighbors. If its value differs by a specific amount (say 10), replace that pixel with the average and move to the next pixel. Individual bad pixels that aren't showing as dead pixels xould be calculated out that way. The only thing that can't be solved is blobs, but I think that's a fair compromise for sharpness.

[miguelvp]

So I don't know how your flat notebook paper and router images are, but if I subtract the flat paper from the router the bands go away, this is what I did on my version of the sgstairs program that is posted somewhere in this thread. Pretty much I have a button that takes a flat reference image that gets used as well as the flat field.

Anyways I'm getting some thresholding overflow or underflow from your images but the banding is gone, other that introduces some vertical banding now.

But the result is not great using some random flat field to get rid of the horizontal lines.
I think Seek knows what the lines are or they do some filtering to average the lines

[miguelvp]

So playing with your end result and doing a lot of filtering per pixel arithmetic you can clear up the bands and add detail as well, I over did it in this one and it shows more heat on the router that it should have.

[eneuro]

@ricksastro
Nice 14bit sensor data 
Did you modified its pixels eg, those output FITS files and 16bit channel RGB (48bit) converted to 16bit gray PNGs or only this JPEG with screen shot from your software has calibrated pixel data in a way you descibed in your post ? 

I'd be interested to understand how seek gets rid of most of the horizontal lines.   They appear to be pretty fixed, so maybe some Flat Field is stored off.

Maybe data from last 2 columns is used to modify somehow rows 

Opened in fv this FITS file and later will try to predict this yellow value based on pixel values in row from 1-106 columns 
The more data the better aproximation could be, while multidimensional aproximation could be used to find out if there is correlation with row pixel data in column 207-208 in those Seek Thermal USB raw data images.
If more such calibration 16bit gray images or FITS were provided, I could try to investigate and even teach neural network to try to predict those last two columns 207-208 

Another concern is those amount of conected dead pixels on this sensor 
I'll calulate later its percentage based on those few sample images, but I do not know why there are on some sensors more than one such 4 dead pixels groups close to "good black" hexagon patented ones?

It is interesting if Seek Thermal has quaility measures of limits of such connected dead sensor pixels count, while total percentage can be fine, but such blobs of dead pixels in this area including those hexagon pixels doesn't looks great and now if we want fill them with some data it is even more challenging if it were only one pixel and all around with real thermal image data
Now dead pixel area is 2 times bigger than their great 12um resolution...

Maybe Seek Thermal Team EEVBLOg forum member will tell us how many such dead pixels one could expect on decent quality thermal sensor 

BTW: It looks like those FITS files are flipped vertical so first row on PNG image is last on FITS etc 

[frenky]

If anyone wants to play with FITS files in C# this should be helpful.
Get DLL here: http://vo.iucaa.ernet.in/~voi/CSharpFITS.html

Add reference to this dll in VS project.

Use the code below to loop through all pixels...

Code: [Select]

using nom.tam.fits;
using System;

namespace ConsoleApplication1
{
    class Program
    {
        static void Main(string[] args)
        {
            Fits f = new Fits(@"C:\temp\thermo\router_image2i.fit");
            ImageHDU h= (ImageHDU) f.ReadHDU();
            System.Array img = (System.Array[])h.Kernel;

            int x = 0, y = 0;//for pixel position tracking

            foreach (short[] collection in img)
            {
                Console.WriteLine("LineNo: "+ y);
                foreach (UInt16 pixVal in collection)
                {
                    Console.Write(pixVal + ",");
                    x++;
                }
                y++;
                Console.WriteLine("");
            }
            Console.Read();
        }
    }
}


[ricksastro]

@ricksastro
Nice 14bit sensor data 
Did you modified its pixels eg, those output FITS files and 16bit channel RGB (48bit) converted to 16bit gray PNGs or only this JPEG with screen shot from your software has calibrated pixel data in a way you descibed in your post ? 

The output fits and png's are unmodified, unscaled, unbiased, unstretched...they are how they come from the camera.
For the JPG, MaximDL allows you to stretch the image visually without modifying the underlying data, since any mods to the data screws up calibration.   So the jpg screenshot shows them linearly stretched (black and whitepoint set to the edged of the data on the histogram), but the underlying files are not, even the filtered one.    Once you perform the subtraction calibration, then you need to add some sort of bias, otherwise you'll get negative values.  It's not like what I'm used to in light land where you take a covered dark and/or bias frame to isolate the noise sources for proper calibration...you can't "cover" thermal, so you have to deal with signal in your calibration frames.

BTW: It looks like those FITS files are flipped vertical so first row on PNG image is last on FITS etc 

When I view either file (PNG or FITS), they look and act identical, unflipped.   Must just be the way they store the FITS file or the way your library is handling them differently.    I just did a"save-as" of the png file to the FITS from MaximDL, so no process that would do any flipping.

[eneuro]

Once you perform the subtraction calibration, then you need to add some sort of bias, otherwise you'll get negative values.

I research test software I use libPNG to load 16bit gray PNG 208x156 images, and then convert it to double so no problem with negative values 
After image processing those images can be scaled to 0.0-1.0 and calculate index integer values needed for chosen LUT size (8bit-10bit-14bit-16bit):

Code: [Select]

index= round(val_double*LUT_size);

Next step is plot this iron LUT table which is 256 size and rescale to 1024, for 10bit LUT or even to 14bit 16384 long - probably it can be created in software using math functions for RGB channels based on desired LUT size-do not played with this iron LUT table channels so far, but probably they can be aproximated and recreated later in software  without any hardcoding in source files or imput LUT images

[ricksastro]

Once you perform the subtraction calibration, then you need to add some sort of bias, otherwise you'll get negative values.

I research test software I use libPNG to load 16bit gray PNG 208x156 images, and then convert it to double so no problem with negative values 

Not sure why converting double does anything for negative values.   When you mathematically subtract the calibration frame from the thermal frame, those areas in the thermal frame that are of a lower temperature than the calibration frame will mathematically be negative no matter the format unless you add in a bias to compensate.   You could convert to floating point and have the same issue

[eneuro]

When you mathematically subtract the calibration frame from the thermal frame, those areas in the thermal frame that are of a lower temperature than the calibration frame will mathematically be negative no matter the format unless you add in a bias to compensate.

I forgot about those calibration frames for the moment at all, while just processing visual web cam outputs in OpenCV

For the moment just ploted this iron256.lut.png and converted each RGB color channnel to text data to model in maxima and find its aproximations, to be able by passing real number in the range of 0.0-1.0 get LUT RGB channel colors, so later easy to resize to any size in own software

Does it makes any sense this Thermal Iron LUT plot?
It is interesting that its blue channel has such strange response to temperature 

Attached processed PNG with this iron LUT table as text data file with normalized temperature index in first column if someone wanted do the same dynamic scaling, resizing to 1024, etc..