Yet another cheap thermal imager incoming.. Seek Thermal

[cynfab]

Here is a great way to get started investigating the Seek Thermal Camera.
1) Install ImageJ: http://fiji.sc/Fiji
2) Use the following Python code to capture as many raw frames as you choose.

Code: [Select]

# This is a quick python program to get and save the first few frames
# of raw data from the Seek Thermal camera.

# copy & paste this code into a file xxxx.py
# run the code like this "python xxxx.py 42"
# 42 being the number of frames to capture.




import usb.core
import usb.util
import sys
import Image

y = int(sys.argv[1])

# find our Seek Thermal device  289d:0010
dev = usb.core.find(idVendor=0x289d, idProduct=0x0010)

# was it found?
if dev is None:
    raise ValueError('Device not found')

# set the active configuration. With no arguments, the first
# configuration will be the active one
dev.set_configuration()

# get an endpoint instance
cfg = dev.get_active_configuration()
intf = cfg[(0,0)]


ep = usb.util.find_descriptor(
    intf,
    # match the first OUT endpoint
    custom_match = \
    lambda e: \
        usb.util.endpoint_direction(e.bEndpointAddress) == \
        usb.util.ENDPOINT_OUT)

assert ep is not None


#msg = '\x41\x41'

#print (msg)

# Deinit the device

msg= '\x00\x00'
assert dev.ctrl_transfer(0x41, 0x3C, 0, 0, msg) == len(msg)
assert dev.ctrl_transfer(0x41, 0x3C, 0, 0, msg) == len(msg)
assert dev.ctrl_transfer(0x41, 0x3C, 0, 0, msg) == len(msg)


# Setup device

#msg = x01
assert dev.ctrl_transfer(0x41, 0x54, 0, 0, 0x01)

#

msg = '\x00\x00'
assert dev.ctrl_transfer(0x41, 0x3C, 0, 0, msg) == len(msg)

ret1 = dev.ctrl_transfer(0xC1, 0x4E, 0, 0, 4)
ret2 = dev.ctrl_transfer(0xC1, 0x36, 0, 0, 12)

#print ret1
#print ret2
#

msg = '\x20\x00\x30\x00\x00\x00'
assert dev.ctrl_transfer(0x41, 0x56, 0, 0, msg) == len(msg)

ret3 = dev.ctrl_transfer(0xC1, 0x58, 0, 0, 0x40)
#print ret3

#

msg = '\x20\x00\x50\x00\x00\x00'
assert dev.ctrl_transfer(0x41, 0x56, 0, 0, msg) == len(msg)

ret4 = dev.ctrl_transfer(0xC1, 0x58, 0, 0, 0x40)
#print ret4

#

msg = '\x0C\x00\x70\x00\x00\x00'
assert dev.ctrl_transfer(0x41, 0x56, 0, 0, msg) == len(msg)

ret5 = dev.ctrl_transfer(0xC1, 0x58, 0, 0, 0x18)
#print ret5

#

msg = '\x06\x00\x08\x00\x00\x00'
assert dev.ctrl_transfer(0x41, 0x56, 0, 0, msg) == len(msg)

ret6 = dev.ctrl_transfer(0xC1, 0x58, 0, 0, 0x0C)
#print ret6

#

msg = '\x08\x00'
assert dev.ctrl_transfer(0x41, 0x3E, 0, 0, msg) == len(msg)

ret7 = dev.ctrl_transfer(0xC1, 0x3D, 0, 0, 2)
#print ret7

#

msg = '\x08\x00'
assert dev.ctrl_transfer(0x41, 0x3E, 0, 0, msg) == len(msg)
msg = '\x01\x00'
assert dev.ctrl_transfer(0x41, 0x3C, 0, 0, msg) == len(msg)

ret8 = dev.ctrl_transfer(0xC1, 0x3D, 0, 0, 2)
#print ret8

# Get and save the first 11 frames as 16bit .png files
# The file name is rawDataXX_YY.png where xx is the frame # and YY is the frame ID
# except for frame # 2 where the data in pixil 10 is not the frame # but part of the
# gradient

x = 0
while x < y:

    msg = '\xC0\x7E\x00\x00'
    assert dev.ctrl_transfer(0x41, 0x53, 0, 0, msg) == len(msg)

    ret9  = dev.read(0x81, 0x3F60, 1000)
    ret9 += dev.read(0x81, 0x3F60, 1000)
    ret9 += dev.read(0x81, 0x3F60, 1000)
    ret9 += dev.read(0x81, 0x3F60, 1000)

    status = ret9[20]

    img = Image.fromstring("I", (208,156), ret9, "raw", "I;16")

    x = x + 1

    img.save('rawData%d'%(x,)+'_%d'%(status,)+'.png')
#    img.show()

print 'Done'

3) Use the image processing abilities of ImageJ to add/subtract/offset frames to see what happens.

The above Python code saves the images as 16bit .png so all the data from the camera is there.


Thanks to eneuro for pointing me at ImageJ.

   ...ken...
 

[eneuro]

@barehill It is quite interesting that without any Seek developers official tools it is possible to play with this dongle and retrieve some raw sensor data 
Let Seek Thermal Team make best hardware they can and keep protocols simply to be able easy connect to this dongle via USB from any OS.
It looks like not a lot of code needed to output some thermal imaginery from this device, so it can be killer of Flir One of course, while they use... crappy manual shutter and it is not small USB dongle like this Seek device...

2) Use the following Python code to capture as many raw frames as you choose.

Code: [Select]

...
    msg = '\xC0\x7E\x00\x00'
    assert dev.ctrl_transfer(0x41, 0x53, 0, 0, msg) == len(msg)

    ret9  = dev.read(0x81, 0x3F60, 1000)
    ret9 += dev.read(0x81, 0x3F60, 1000)
    ret9 += dev.read(0x81, 0x3F60, 1000)
    ret9 += dev.read(0x81, 0x3F60, 1000)

    status = ret9[20]

    img = Image.fromstring("I", (208,156), ret9, "raw", "I;16")
...


I'm not python expert rather more familar with C/C++/Java/ILERPG , but when looked into similar example of reading mouse using pyusb there Read and Decode Data From Your Mouse Using This PyUSB Hack it looks like frames are taken there after setup above, but it means that this USB read gives you EXACTLY 1 frame from Seek sensor : dot[0], to dot[32448-1] than saved as 16bit PNG?

I've developed some pattern match tool in my sts2img software which reads raw byte by byte sensor frames and then detects its start, end and verify against hexagon pattern so I can extract each Seek sensor frame from bytes stream, but it looks like it is very simple when we have direct access to USB port   and this matching is not needed since I guess we get frame after frame, but correct me if I'm wrong

Anyway before using ImageJ2 when Seek sensor frame is detected that when calibration frame comes 0x1 than I iterate all hexagon patterns as well as other pixels close to 0x0000 value to output for imageprocessing tools 16bit images but with those hexagon pattern pixels already filled in, so there is no need to make blurs, medians etc on whole image before playing with those sensor output frames in such software like Imagej2 
So, I have two frames oryginal 16bit untouched and with preprocessed black hexagon pattern pixels and dead if any.
Such preprocessed calibration images difference I've shown a few posts above-there were no pixels close to 0x0000, but it looked like minimum on those calibration frames was about 5000 when filled pixels needed to fill.

[cynfab]

eneuro,

Yes frames are captured from the first to the last you enter on the command line.

They are saved in 16bit .png format. with ImageJ you can add 2000 to a data frame then subtract the cal frame and see the 16biit result.

Look at frames 1,3,4,5. they all look like cal frames except with the max data at different points. I think it is worth looking at them further, maybe they are factory cal data for that specific imager.

   ...ken...

[miguelvp]

Look at frames 1,3,4,5. they all look like cal frames except with the max data at different points. I think it is worth looking at them further, maybe they are factory cal data for that specific imager.

   ...ken...

The first 6 frames have the following frame ids on the first patent pixel:

4, 9, 8, 7, 10, 5

More info:
https://www.eevblog.com/forum/testgear/yet-another-cheap-thermal-imager-incoming/msg545910/#msg545910

[cynfab]

Well, the images posted at:

https://www.eevblog.com/forum/testgear/yet-another-cheap-thermal-imager-incoming/msg545910/#msg545910

are all 8/24bit rgb which do not contain all the data from the camera.

Attached are 16bit .png of frames 1-9 from my camera.

Frame 1 ID 4, Frame 5 ID 10, Frame 3 ID 8, Frame 4 ID 7 look to me like they might be sensor calibration data unique to each sensor, they "resemble" both the data frames ID 3 and the "calibration" frames ID 1 (for some definitions of " resemble")

Frame 2 ID 9 is a "gradient", the only pixels different from the gradient are # 1 and # 10 (not counting the big white patch (value 15464) in the middle.
Frame 6 ID 5 is half black and half something else, I couldn't begin to guess what it is for.
Frame 7 ID 1 is the first calibration frame, (assuming that is what ID 1 really means)
Frame 8 ID 3 is the first data frame, (there are a lot more of them than any other ID so it is logical to assume that's what they are)
Frame 9 ID 6 is the first "pre-calibration" frame, (since it occurs right before the ID 1 calibration frame (as to what is used for I'm clueless)

The Seek application produces images that are free of banding which can be clearly seen in my as well as other non-seek images.
Seek knows how to remove that banding, so I think that they may be doing something with frames 1,3,5,4. Why then would those frames be produced at the beginning of each camera initialization, and never produced again. (I should look to see if those frames are identical each time the camera is initialized.... maybe tomorrow)

Looking at Frames 5,1,3,4 I see max values of 170, 4258, 9606 and 15675. I see min values or 0 for all these frames, but since they include the "patent pixels" and maybe some dead pixels, that is understandable. I will look at removing the zero values to see if that casts any light on the range of these frames.

I think eneuro has already done this to some of the raw USB data dumps, so i may be repeating some research, but it may help my little pea brain comprehend what's going on.

   ...ken...

These images below may look black, but they aren't. View them with something that can read 16bit .png files.

[miguelvp]

I know they are 8bits it was more for visual inspection, but I did post raw data for those and eneuro has been using the raw data.

As for repeating research, that is a good thing, you might find a different angle than the rest.

[eneuro]

Frame 9 ID 6 is the first "pre-calibration" frame, (since it occurs right before the ID 1 calibration frame (as to what is used for I'm clueless)

As showed above the difference between frame 0x6 and 0x1 is very small and biggest are in those first and last lines.
That is why I missed a few (4) other frames in this short 20 frames 1MB USB dump, so extracted from bytes stream 16/20.

I think there is no sense cal those dot[10] values frame id, while we do not know this protocol and for example this value can be result of codind a few parameters (bit flags) so something like looks: 0x1 and 0x6 different can be the same shutter thermal image, while when we print it as bits, than:

Code: [Select]

0x1: 001
0x6: 110
0x3: 011
Ok, there is no clear bit connections with similar histogram frames 0x1 and 0x6, so maybe this is some kind of frame id, but who knows 
So, I do not give them any names for the moment, just looking into its histograms, but after filling hexagon and dead pixels first, and will concentrate on first and last row hexagon patent dots next week during my research, while it looks like something is going on in those rows.

I see min values or 0 for all these frames, but since they include the "patent pixels" and maybe some dead pixels, that is understandable. I will look at removing the zero values to see if that casts any light on the range of these frames.

When filled hexagon patent dots with its neighbours weighted average and shifting to fit into 208x156 output image from oryginal native 206x156 resolution as shown in posts above I see no lower values than about 5000 in frames 0x3 with scene thermal data, as well as frames 0x1 and 0x6 which looked like only small changes in non hexagon patent pixels (regular real sensor thermal ones) exists and its standard deviation was only 9 while average pixels value about 7500, so very small difference between those 0x1 0x6 frames .

NOTE: Those 4700 min values might look black there, that is because of imagej2 auto scale and normalize so output grey image might look like there are black pixels, but histogram proves there is no values smaller than 4766 

[eneuro]

I know they are 8bits it was more for visual inspection...

Isn't better make screen shot than everybody see that it is not raw sensor data? 

Anyway its cleary visible that even image difference between only two sequence frames: 0x3-0x1+6794 and shown below in imagej2 creates those strange gradients in the corners better visible with thermal LUT applied below.

There is no sensor data postprocessing, but hexagon pattern and dead pixels filling as descibed above, so we can see a lot of noise in result image sts.raw.0x3-0x1+6794.png , but when made mentioned median filter on this image and some bluring those strange gradients in corners looks like affects image quality, while noise is not a problem in those ONLY two frames there, but destruction of thermal image in the corners looks like it is real issue and while there this output image was normalized and LUt applied from minimum temperature to maximum (pixel values shown in its histogram above) than it is very visible as huge temperature differencem, while it can be  a few degrees, but something is going on in those corners and it doesn not look great, however for some applications like thermal objects deetction if it is in acceptable limits and we know it happends on this device this thing can be usabe.

Now, while we know that those corners gradient efect is such easy to catch and it clear corelation to approximated sensor calibration shutter frames, lets try next week make corrections, because from this router image we know that those temperatures in the corners should be close to "blue sky" colors around router in the centre, so we have some hints how to correct thise edges gradient effects in this case.
If it works, we'll try the same in different ambient temperatures within Seek Thermal operating range, for different object temperatures and we'll see if we can improove output thermal image reliability, while it looks like resolution itself is not so bad, and of course we have decent level of details even using only 2 thermal frames including shutter, so no moving average so far-it looks much better than any attempts to make super resolution using 80x60 Flir One Lepton sensor shown in other EEVblog threads 

[frenky]

I think that for now we should forget about cleaning the "router" images...

My approach will be to take one capture of painted aluminium plate at 0*C and also one at 100*C.
The alu plate will cover the entire view of sensor.

After that I'll try to figure out the algorithm that will make both of this captures noise and gradient free.
If I succeed all other thermal captures should also come out perfect...

[cynfab]

This may be of interest. On my camera, each time I start the camera frame 1 and 5 are the same as the last time. However, frames 3 and 4 are different each time.

in other words:

rawdata1_4(n) == rawdata1_4(n+1) and rawdata5_10(n) == rawdata5_10(n+1) but rawdata3_8(n) != rawdata3_8(n+1) and rawdata4_7(n) != rawdata4_7(n+1)

3 and 7 are very close on consecutive starts while 1 and 5 are bit identical.

   ...ken...

[eneuro]

My approach will be to take one capture of painted aluminium plate at 0*C and also one at 100*C.
The alu plate will cover the entire view of sensor.

Probably you have to keep those things at proper distance from Seek sensor to avoid reflections back of heat around sensor which was around 35*C, so maybe also try different angles.
It could be nice do the same with some higher quality equipment to compare if effects seen on Seek sensor are really affected by his hardware, not by thermal phenomena mentioned by @Aurora in this thread before 
Putting just Seek on table very close to any plate is a bad idea, but here we see that those corners are affected somehow by heat arorund it it looks very circular, so maybe on thermal scenes where ambient room temperature where this router was powered on was lets say 25*C and this is what we should get around router, but internal Seek Thermal temperature shown by @Aurora was about 36*C visible from top view using another thermal camera, so maybe lens construction itself while they are cooler there and will get hotter than air temperature, simply hot lens holder creates this circular aurora in the corners which should be easier visible if router was removed from the centre and we tried fit circle via corners.
It could also explain that lens positioning changes those gradients effects, but we have no sensor block temperatures itself, so difficult to dismiss theories around it including sensor temperature instability I've shown many posts above.

Anyway it could be nice make some 3D model of sensor block with proportions to die and expected focus area of lenses used with lens holder around it, moving shutter,  but it needs of course physical measurements of disassembled Seek dongle, but probably @Aurora already did this while he wants design his own version with this sensor...

[Fraser]

@eneuro,

Sadly I have placed the SEEK camera to one side while I concentrate on some more pressing projects like my newly acquired high resolution laboratory X-Ray machine    

I am waiting to see whether SEEK address the high noise content of images in their next release. If not, the SEEK will not meet my needs and so is not worth remodelling to take a better lens.

Aurora

[callipso]

Is.. Is that xray running windows? Hope you're wearing a dosimeter badge and lead lined clothes...

[callipso]

Also with all that gear... Is your employer hiring anytime soon? ;-)

[Fraser]

@Callipso,

I am a qualified radiographer so no worries about the safety side 

Yep its running Windows 2000 professional ..... and it seems to be stable 

This is my kit and not my employers .... I drive a desk in the office these days 

Aurora

[encryptededdy]

New Post on Seek's Facebook page: https://www.facebook.com/permalink.php?story_fbid=775531895852717&id=628554970550411

TL;DR

• Android is off backorder and orders are shipping as soon as they come in
• Amazon is receiving regular shipments so those orders will be processed quickly
• App update that promises to minimise gradient effect is in testing and they hope to have a posted before the end of the month
• Still working on Apple approval
• They hope that they will have a ETA for when this might be resolved in the next few weeks

[Fraser]

I am glad that I am an Android user  APPLE appear to be a real PITA over their approvals process and seem none too fussed about a prompt resolution for SEEK. How on earth did Mu get approval on their POS ? Maybe they didn't ?

[frenky]

I can confirm that amazon orders are shipped out immediately.
If I was in USA I would already have it in hand because the package has arrived to my virtual US address.
(And is now on it's way to Europe...)

[mikeselectricstuff]

How on earth did Mu get approval on their POS ? Maybe they didn't ?

Why would you think they got approval? - they never had a real product and never will.
I have no doubt that the issue is that Apple are being a bunch of dicks, but Seek aren't exactly handling things well, though they are probably to some extent being constrained by Apple's NDAs and similar bullshit.

[eneuro]

New Post on Seek's Facebook page:

Yep, but so many people have their bankrolls $200 less and still can only watch how we play with those images 

Scott Maggio Why are you still accepting Apple orders if you a "few weeks away" from knowing if you will even be approved?

Aprops thermal gradient:

"There have been reports from users about thermal gradient problems with the images in the app. This means that the temperature can vary across a uniform surface, which results in color differences. Some variation is expected in the camera, but we are also working on an app update that will further minimize the effect. That updated version is in testing right now and we hope to have it posted by the end of the month."

We'll see who will have better gradients correction-my open source version which is also tested or Seek update if they do it for Santa Claus really? 
OK, so no more hints for Seek team in this thread until my version will be ready next week and lets see how good this Seek update will be.
it is small challenge-They can be ahead while can modify easy Seek dongle firmware, so it will be interesting to compare how much its MPU firmware changed 

I suggest log as many raw data in current software version as possible, just to compare later if any significant changes in USB protocol were made 

[cynfab]

Here is a screenshot of  some startup frames from my camera. I had it pointing at a hot soldering gun when I started it up. The tip temp was about +300C.
I noticed a few things:
1) Frame 11_6 looks more like calibration frame 12_1 than a image frame.
2) The soldering gun is clearly seen on the uncorrected image frames 8_3, 9_3, 10_3, 13_3 and not on 11_6.

 Not shown are images 1_4, 3_8, 4_7, 5_10 and 7_1 none of which showed any hint of the soldering gun.

I took frame 13_3 and added 2000 to each pixel, then subtracted frame 12_1 from it giving the result on the bottom left.
The image on top left is frame 13_3 + 2000 - frame 11_6. Interesting to note the large number of white pixels vs few if any on the bottom image.

No conclusions, just interesting data.

   ...ken...

my images names are of the form rawdataXX_YY.png   where XX is the frame sequence # and YY is the Frame ID from Pixel 10.

[blackboxdisease]

I have something that may be helpful to someone.

included is the source. Decompiled, SEEK thermal app.

https://drive.google.com/folderview?id=0B58-Lp-QiJ7HRHVQRmxEWFJ5UTA&usp=sharing

Just quickly browsed and found some interesting directories.

/src/com/tyriansystems
/src/com/subi/usb
/src/org

[cynfab]

I've seen the smali decompiled sources before, and have also decompiled them myself using dex2jar and jd. The interesting code is buried in a native method inside libSeekware.so.
I'm sure there is lots to learn from the java sources too, any insights anyone has as to the purpose of any of the first frames will be welcomed by all interested in using the Seek Imager on something other than Android.

Here is a list of some of the interesting methods within libSeekware.so:

Code: [Select]

00007298 <Java_com_tyriansystems_Seekware_SeekwareNativeLib_getLibraryVersion>:
00007318 <Java_com_tyriansystems_Seekware_SeekwareNativeLib_requestShutterOperation>:
00007360 <Java_com_tyriansystems_Seekware_SeekwareNativeLib_CheckGainTable>:
0000737c <Java_com_tyriansystems_Seekware_SeekwareNativeLib_ColorizeInit>:
000073a0 <Java_com_tyriansystems_Seekware_SeekwareNativeLib_ColorizeProcess>:
00007480 <Java_com_tyriansystems_Seekware_SeekwareNativeLib_ColorizeDeInit>:
00007488 <Java_com_tyriansystems_Seekware_SeekwareNativeLib_applyThermographyLut>:
00007800 <Java_com_tyriansystems_Seekware_SeekwareNativeLib_applyLUT>:
00007948 <Java_com_tyriansystems_Seekware_SeekwareNativeLib_TemperatureInit>:
000079d4 <Java_com_tyriansystems_Seekware_SeekwareNativeLib_Temperature>:
000079f4 <Java_com_tyriansystems_Seekware_SeekwareNativeLib_TemperatureDeInit>:
000079fc <Java_com_tyriansystems_Seekware_SeekwareNativeLib_ImageProcessInit>:
00007a14 <Java_com_tyriansystems_Seekware_SeekwareNativeLib_ImageProcessDeInit>:
00007a18 <Java_com_tyriansystems_Seekware_SeekwareNativeLib_ImageProcessProcess>:
00007abc <Java_com_tyriansystems_Seekware_SeekwareNativeLib_BclaheInit>:
00007ae4 <Java_com_tyriansystems_Seekware_SeekwareNativeLib_BclaheProcess>:
00007b46 <Java_com_tyriansystems_Seekware_SeekwareNativeLib_BclaheDeInit>:
00007b4a <Java_com_tyriansystems_Seekware_SeekwareNativeLib_ThermographyInit>:
00007b5e <Java_com_tyriansystems_Seekware_SeekwareNativeLib_ThermographyDeInit>:
00007b62 <Java_com_tyriansystems_Seekware_SeekwareNativeLib_ThermographyCalculate>:
00007c4e <Java_com_tyriansystems_Seekware_SeekwareNativeLib_Golomb>:
00007d4c <Java_com_tyriansystems_Seekware_SeekwareNativeLib_RGBAToYUV420SemiPlanar>:
00007e26 <Java_com_tyriansystems_Seekware_SeekwareNativeLib_WriteBits>:


   ...ken...

[eneuro]

Decompiled, SEEK thermal app.

Decompiled or unzipped only? 
JAR (file format)

JAR files are fundamental archive files, built on the ZIP file format and have the .jar file extension.

my images names are of the form rawdataXX_YY.png   where XX is the frame sequence # and YY is the Frame ID from Pixel 10.

If you attached this hot iron raw sensor data  rawdataXX_1.png  rawdataXX_3.png  rawdataXX_6.png  we could see if this gradient issue is also present on your Seek dongle around hot iron in the corners, as well as since we know it was about 300*C iron and it looks like around 15000 in histograms make analysis for frames id 0x1 0x3 0x6 with hexagon patent and dead pixels removed including data from 207th column 

Than we could see what real histograms are when we have only real thermal object data like I used to do in my analysis.

BTW: If someone knows howto add custom LUT to imagej2 let us know-it could be nice include there my favourite Flir's like thermal Iron shown already in this thread, while this builtin to imagej2 is different one and I''d like to use something similar to Flir's just to be able compare output imaginery with similar  LUTs 

Update: I've did it just need to put *.lut file to imagej2   IMAGEJ2_PATH/imagej2/Fiji.app/luts directory like my attached iron256_by_eneuro.lut :

Code: [Select]

-rw-rw-r-- 1 501 501 2593 2014-11-23 15:10 /opt/imagej2/Fiji.app/luts/iron256_by_eneuro.lut

Than it is simply to apply new table to 8bit or 16bit gray thermal images in imagej2 while it is visible in LUTs list 

[cynfab]

eneuro,

 Here ya go...

Three consecutive frames, 11_6, 12_1 and 13_3. 16bit .png

As for the gradient, the camera had been on for several hours, not taking data continuously, but plugged in to my desktop, I would capture data every 5-10 minutes depending on the tweaks I was making to my Python code.

   enjoy
     ...ken...