Yet another cheap thermal imager incoming.. Seek Thermal

[miguelvp]

Thanks, still needs more work, can't get rid off all the horizontal banding on all the ranges.

But eventually since it seems to just be a ramp. And after that some ramp calibration to compensate for the sensor ramp deviation.

[eneuro]

A few observations :
The diode on the back is definitely sensing temp - shunting it causes an instant change in temp readings, but no change in the image, or any change after a shutter close, so presumably being used to calculate a delta between the sensor pixel value and actual temp.

By on the back do you mean still inside this dongle? This temp sensor Is visible in your Seek Thermal teardown videos?

Cooling the lens mount produces a massive gradient, with the bottom of the display showing cold. The lens is inverting, so this corresponds to the shutter opening side.

It was such massive cooling as in one of your teardowns with this magic spray or something softer? Sensor PCB was without its enclosure, I guess?


BTW: Maybe someone with highend other thermal camera could do simple experiment by pointing this device to flat metal copper sheet but not perpendicular to its surface, but also slightly turned so lets say 5-10  degrees between thermal camera axis and flat surface normal to see if we get any temp gradients due to this small flat surface rotation ?
I've got 1.3 DEG angle in approximated calibration frame example shown above, so now it is time to estimate possible temperature changes at the edges of such gradient.
To do this we need to findout proper way to estimate absolute temperatures based on raw sensor data available,
but also we need to know which edge on image affected by gradient has clossest to correct temperature readings-hotter or cooler edge?
If we knew this and it was consistent on other Seek Thermal dongless than we could try make this gradient correction.
For the moment I will probably use midpoint with option to overwritte temperature shift in user interface for better gradient compensation.

Thanks @mikeselectricstuff for those observations  notes

[miguelvp]

BTW I forgot to say that it will dump the first 20 frames including the first 6 system frames including frame 1 with ID 4 which I use instead of column 207, and metadata in the current working directory.
The raw data will be in a file named data.dat, the metadata in a file named data.txt with the frame IDs on it.

Raw data is 16bit unsigned totally unprocessed, 208x156 per frame.

Edit: Frame IDs link with visual pictures on how they look like:
https://www.eevblog.com/forum/testgear/yet-another-cheap-thermal-imager-incoming/msg545910/#msg545910

Edit2: Also I don't know what I did to get to
https://www.eevblog.com/forum/testgear/yet-another-cheap-thermal-imager-incoming/msg547638/#msg547638

didn't have source control then, but it looks like it was a better result, will try to get back to that.

[mikeselectricstuff]

If the image quality can be improved there is a massive CCTV market waiting for lower priced thermal security cameras.

I'm not convinced about that. Low-res thermal isn't good enough for identification, only detection. Even then it probably needs a better lens than we've see on F1/Seek
There may be some scope for automatic person detection using image processing, but I'm wondering how often this would actually be useful in a civillian security application.
 
I'm struggling to think of an example of a scenario where a cheap low-res security camera would significantly augment existing solutions.
 

[eneuro]

Just finished nonlinear approximation of Seek Thermal calibration frames and it looks like it can be done in realtime on decent modern CPU

seek_thermal: Approximation of calibration frame parameters: -0.002520609 -0.000198617 -0.000972260 +0.001552013 -0.003539129 +0.005632120 +0.008891812 +0.000917557 -0.000869635 +1.587426345  time: 0.016320916 s

10 approximation parameters are computed at time below 0.02s, so even for 10Hz output should be fine.

This is the same calibration frame as used with linear approximation-now with a little bit more nonlinearity it nicelly fits into this frame, however it could be more tricky to detect gradients now.

I will implement those two methods (nonlinear,linear) of gradient approximation and when approximation time will be too slow for nonlinear than it can be forced to linear in user options.
We'll see how it will perform on Android tablet, but thi sis not the main purpose of this custom Seek Thermal capure software-it will be used in thermal security system, so it doesn't matter if it will be slow on crappy phone, etc 

[blackboxdisease]

I have a strong belief on what is going on, it's just difficult for me to try put it into words.

The gradient is programmed. But the relationship to the color of gradient spectrum is established during first temperature reading. As an example, the pantry image where it was 46F at it's color purple/blue, was the same color of my first reading when the camera was in the freezer. Different temperatures, but same color. The gradient is dynamic in sense that, when the sensor detects higher radiation in the corner, the rest of the sensor compensates, by moving that blue color away from it's inital location. So what was once blue, is now white and what was once orange is now blue.

If radiation on the sensor exceeds what it first detects from inside the housing, there will be no gradient.

The sensor detects radiation from the chips, enough to over power the temp in the freezer because. The chips must be isolated from the sensor better.
If the sensor was placed over top the chips, centered on the pcb, I am almost certain there would be no linear gradient, at worst, a radial gradient.

I also noticed that how a traditional camera reacts to different light conditions by adjusting exposure, is the same that thermal camera does with the changing levels of radiation, creating an awful image.
The lens has a fixed aperture and I'm wondering if different apertures will create a sharper image and affect the range of gradient.

It's more than one issue.

This video shows how the color association changes. Keep your eye on the bottom left. It shouldn't change color at all as the freezer temp hasn't changed. The color only chages as my hand moves toward it. Re-associating the color to changing temperature introduced by my hand.

video:
http://youtu.be/e5M3v7k3qa8?list=UUnTjoJWDkUC85-eqBk2lLyQ

This video show's how the camera falsely associates the color to temperature. When hotter temps are detected, the color previously associated with a lower temp, changes as well as the gradient range. When there are very little differences between hot and cold temps, such as when the camera is pointed at the radiator, There is very minimal gradient, and a Gaussian blur is applied. When the camera is taken into a room that is 10c or so, the curve drops and all hell breaks loose and the tv is over saturated, showing white in area's that would make one assume that was a high temp, when really was only a few degree's above THIS rooms ambient temp.

Video: http://youtu.be/-GG8PaPEkF0?list=UUnTjoJWDkUC85-eqBk2lLyQ

Now if it had maintained the range of gradient and the colors associated with temp, the quality should have been expected to stay the same, regardless of ambient temperature. The tv would show a hue of orange perhaps, and the remainder of the items in the cold room, hue's of blue.

I don't know if any of this makes sense but these are my thoughts and if I had the technical ability I would tear the camera up and test these theories out.

[eneuro]

Low-res thermal isn't good enough for identification, only detection.

Yep, however if it detect suspicious heat sources it can guide visual hires camera, so operator can have fusion of thermal & visual systems and can toggle between them or thermal part can auto fire additional light source to help visual system record unusual events and catch in high intensity visual light flash

[frenky]

@miguelvp:
How did you connect the module to PC? Did you make your own diy usb cable?
What about the drivers?

[Fraser]

@mike,

Visbile light and Near IR illuminated CCTV is quite easy to beat 

Thermal CCTV is quite hard to beat by your average thug 

Its all about detection and NOT identification.

Aurora

[eneuro]

Different temperatures, but same color. The gradient is dynamic in sense that, when the sensor detects higher radiation in the corner, the rest of the sensor compensates, by moving that blue color away from it's inital location.

Probably you mean something like this in this approximated calibration sensor frame

I think user should be able to set in options palette color range and minimum maximum temperatures and it have to be fixed not dynamic on output images, while if it changes based on thermal scene temperatures one have no idea what absolute temperature might be 

Did anyone checked if ONLY 8bit (256) colors palette is used in Seek app?
I looked into app LUT folder but it looks like a garbage as wrote a few posts above...
They could be defined parametric in software, but who knows without source code.

Maybe that is why they try to fit it into thermal scene somehow when there is small difference in overall detected temperatures? 
It is time to see other Seek Thermal videos available on youtube to investigate this, while this is important how to implement those absolute temperatures in own application-I'll let user define fixed temperature range as mentioned above, than try to keep constant 16bit RGB thermal iron LUT colors for chosen temp ranges and palette part regardless what is present in the scene-saturate if above set min/max limits...

[efahrenholz]

@miguelvp,

I tried out your new version. For some reason my camera looks like crap. I get extremely noisy image. The external reference frame button doesn't seem to give the expected results. I don't get a smooth image after subtraction. Its got a round circular gradient going on.

I don't know how your images look so good. My sensor must be crap. Maybe lots of bad pixels or something. That would be my luck. Also, I have to rotate my tablet upside down to get proper orientation. Its like the image is flipped sideways?

[eneuro]

My sensor must be crap. Maybe lots of bad pixels or something.

Probably you must have option in this soft to output raw sensor data or recognized raw sensor frames as 16bit PNG images, so easy to look how those frames looks like even using imagej2 or other soft capable to make image processing of 16bit depth PNG images...

[casper.bang]

BTW I forgot to say that it will dump the first 20 frames including the first 6 system frames including frame 1 with ID 4 which I use instead of column 207, and metadata in the current working directory.
The raw data will be in a file named data.dat, the metadata in a file named data.txt with the frame IDs on it.

Raw data is 16bit unsigned totally unprocessed, 208x156 per frame.

Edit: Frame IDs link with visual pictures on how they look like:
https://www.eevblog.com/forum/testgear/yet-another-cheap-thermal-imager-incoming/msg545910/#msg545910

Edit2: Also I don't know what I did to get to
https://www.eevblog.com/forum/testgear/yet-another-cheap-thermal-imager-incoming/msg547638/#msg547638

didn't have source control then, but it looks like it was a better result, will try to get back to that.

@miguelvp, I found your executable a few pages back. Thought about putting the code on GitHub for all os us to play with?

[miguelvp]

Just to clarify that the code I posted is originally sgstair's code, without that framework and USB communication I wouldn't been able to slightly modify it and also adapted frenky's filtering code.
I even removed my own dead pixel interpolation which is was very similar to what frenky did but was targeted to just certain pixels while his works with those and others dead pixels.

@miguelvp:
How did you connect the module to PC? Did you make your own diy usb cable?
What about the drivers?

I followed sgstair's advice given here with his first release:
https://www.eevblog.com/forum/testgear/yet-another-cheap-thermal-imager-incoming/msg534497/#msg534497

And used

Zadig to install the winusb driver on the "iAP interface"

As for the cable I did build one myself
https://www.eevblog.com/forum/testgear/yet-another-cheap-thermal-imager-incoming/msg538269/#msg538269

But i also bought a micro b female to usb a male adaptor from ebay, cant find the vendor since I did ask my wife to get it for me but it's similar to this one and they offer 2 in here:
http://www.ebay.com/itm/2X-USB-2-0-A-type-male-to-Micro-USB-B-type-5p-female-Connector-Adapter-convertor-/151445719442

@miguelvp,

I tried out your new version. For some reason my camera looks like crap. I get extremely noisy image. The external reference frame button doesn't seem to give the expected results. I don't get a smooth image after subtraction. Its got a round circular gradient going on.

I don't know how your images look so good. My sensor must be crap. Maybe lots of bad pixels or something. That would be my luck. Also, I have to rotate my tablet upside down to get proper orientation. Its like the image is flipped sideways?

It's probably because that code is pretty much tailored to my camera since I don't have other samples to work with.

The program outputs the first 20 frames of raw data (under 2 seconds) into a data.dat and metadata in data.txt.

If point it to something that shows some heat before you launch the program and then you zip those files and post them (hopefully they'll be within the limit or I could open an ftp server if you prefer). I'll take a look to see if I can make the program more general.

@Marshall same with you, I can see what I can do so it gets rid off your three blocks that throw away the range.

Also the external reference image doesn't quite work as good as before since I added the Frame ID 4 image (first frame) but you can change the code not to use that frame. I'll add a button to enable disable the different approaches.

@miguelvp, I found your executable a few pages back. Thought about putting the code on GitHub for all os us to play with?

The code is sgstair's derived and he has a github. Link:
https://github.com/sgstair/winusbdotnet

If someone wants to fork it with my changes, that's fine with me, I'm not going to support it because I do have very limited time and it's going to become even more limited in the next 4 months.

[marshallh]

miguelvp:
http://retroactive.be/data.zip

[eneuro]

This is quite interesting comparision of 3 parameter linear approxiamtion of Seek  calibration frame

vs 10 parameters nonlinear one after rather short time after turning this device on, processed by my latest version of calibration frames calibration tools with support to choose how many parameters will define approximated calibration sensor data.


It will be interesting to see difference in output images when gradient corection will be made using linear vs nonlinear approximation which shows not one but a few temperature gradients close to sensor corners, so nonlinear approximation is more accurate of course.

BTW: Different temperature scale range on those images doesn't matter since it is only result of gnuplot autoscalling of discrete input text data file provided by my software after approximation for testing purposes-those peaks are also affected by discrete values sequence-approximation result is smooth

[miguelvp]

@eneuro I can't wait to see your code so we can integrate it with sgstair's framework.

@Marshall, thanks I'll take a look during lunch and after work.

[eneuro]

@eneuro I can't wait to see your code so we can integrate it with sgstair's framework.

For the moment I will release only custom made Iron LUTs on my OpenThermal Web Site while probably I have very different vision how this thermal imagining software should look like and no plans to support Window$ OS, only Linux...maybe Android.
When I will be able put my hands on this Seek Thermal dongle soon and ensure that software is well tested and ready to share than will commit it there http://openthermal.sourceforge.net , but probably Seek Thermal will provide his own developers tools meantime, so not such a big deal now and it is low priority task now in my schedule 

[miguelvp]

I asked for a Jetson dev kit for xmas and I'm planning to use OpenCV as well, but that's when the time comes for that.

And code is code so only the formulas is what is needed, myself I'm using VisiQuest that I bought a long time ago and it's no longer available to figure things out before placing it in code.
But if you don't feel like sharing that is fine as well.

[cynfab]

And while we are talking about code, here is my latest Python code.

It has several palette options thanks to: https://github.com/pklaus/python-colorscale

Runs at about 5 fps on my i7 desktop and about 1fps on a MinnowBoard Max under Linux Mint 17

   enjoy
     ...ken...

btw, the files are attached as .py.txt since the forum doesn't like the .py extension.

[blackboxdisease]

I re-watched mikeselectricstuff's gradient video.

when he removed the shutter, there was no gradient. He then partially reassembled everything, and I think shorted out the shutter so he could do it manually, and the gradient re-appeared, but he said the issue was the lens, even though the gradient had disappeared after he removed the shutter only.

Now, just over 3/4 past the video, and for a very short period, he is showing the top view of the sensor with the shutter attached. The shutter closes and opens again. For a split second, you see that the shutter doesn't cover the entire sensor, leaving part of it exposed. Noticing the unusually shaped shutter, I immediately asked myself, "why isn't the shutter cut to cover the entire sensor"? If part of it is exposed, then there could be a very good possibility it will detect radiation on that exposed part. So how about chopping that section of shutter and creating one that covers the entire sensor, using some epoxy or super glue to reattach? or even 3D printing out one?

[Spyke]

I re-watched mikeselectricstuff's gradient video.

when he removed the shutter, there was no gradient. He then partially reassembled everything, and I think shorted out the shutter so he could do it manually, and the gradient re-appeared, but he said the issue was the lens, even though the gradient had disappeared after he removed the shutter only.

Now, just over 3/4 past the video, and for a very short period, he is showing the top view of the sensor with the shutter attached. The shutter closes and opens again. For a split second, you see that the shutter doesn't cover the entire sensor, leaving part of it exposed. Noticing the unusually shaped shutter, I immediately asked myself, "why isn't the shutter cut to cover the entire sensor"? If part of it is exposed, then there could be a very good possibility it will detect radiation on that exposed part. So how about chopping that section of shutter and creating one that covers the entire sensor, using some epoxy or super glue to reattach? or even 3D printing out one?

The shutter usually smacks against the lens housing, so it likely does cover the entire sensor where it stops. You can see the glue where the lens housing should be and the shutter is over it.

[miguelvp]

It does cover the whole sensor, otherwise you'll see ghosting where it didn't cover it.

[marcan]

It only has to cover the active part of the sensor array anyway, not the entire top surface of the sensor.

[miguelvp]

miguelvp:
http://retroactive.be/data.zip

I made a new version of the program that will read the first 20 frames from a file as if they came from the camera as long as there is a data.dat in there.

So I can reproduce what your camera is sending so I can work with that data to try to make it better.