Yet another cheap thermal imager incoming.. Seek Thermal

[miguelvp]

Seek comparison (yes, both on iPhone, so they do exist)

http://appleinsider.com/articles/14/11/23/review-flir-one-and-seek-bring-thermal-imaging-to-iphone

Video comparison from the article.

[Fraser]

@tomas123,

Totally agree on the lens being the route of the E4 image limitations.

For those unaware a major cost in a thermal camera was traditionally the Microbolometer and optical block. Whilst the microbolometer has been developed and the cost per unit lowered, the lens structure remained expensive. The reason was the material used and the manufacturing process. The material was a single crystal of Germanium that was cut to shape by a mono diamond cutter and then polished before AR coatings were added. Large lenses like those in my PM series cameras cost thousands of Dollars each. They are around 55mm optics diameter and multi element. A revolution in thermal camera optics occurred in 2011 when research into special glasses succeeded in producing a usable moulded chalcogenide lens that was cheap(er) to produce than Germanium. It has the added benefit of not changing its transmission rate with delta T.  The down side is that it is a compromise solution that is not, IMHO, capable of producing images of the clarity obtained with Germanium lenses. The chalcogenide material is a mixture of Germanium and other materials. Early images produced by these lenses showed clear evidence of distortion caused by the many micro fractures that were present within the material. The micro-fractures were the result of the prodiution process and I have not seen anything detailing if or how this issue has been addressed. Its a bit like comparing a pair of binoculars with plastic lenses with those of a Nikon with top quality glass optics. The cheaper lenses did open up the market to lower cost thermal imaging however and for that I am very grateful. With many thermal cameras offering 160x120 or 320x240 thermal resolution you have to consider how good the lens actually needs to be in order to produce an ACCEPTABLE image. We are basically talking about early web cam resolutions here and remember how poor the images were from this cameras due to resolution and lens limitations. The moulded lenses are still not pocket money cheap though and so manufacturers tend to limit their size to the minimum needed for a particular application. This is why the SEEK and E4 have very small moulded lenses and not decent sized optics or even Germanium lens elements.

Lightpath are a large producer of the chalcogenide moulded thermal glass lenses.

http://www.lightpath.com/infrared-optics.html

Another interesting page on moulded lenses for TIC's.

http://www.laserfocusworld.com/articles/print/volume-38/issue-7/features/optics/chalcogenide-glass-molds-thermal-imaging.html

http://arizona.openrepository.com/arizona/bitstream/10150/223357/1/azu_etd_12017_sip1_m.pdf

http://www.gizmag.com/cheap-infrared-lenses-fraunhofer/23659/

Aurora

[ebeall]

Sorry if this is a bit off-topic from where this has gone, but I just heard from a backer this afternoon that Mu has thrown in the towel, citing the seek thermal. I don't think they've quoted out Seek or they'd discover that Seek sells cores, not cameras, and that Seek would be happy to sell cores to them at a price that shouldn't be a problem for their target price. I think they were just pretty stressed and looking for a reason to leave.

I think it might be enlightening to find out what the plans and preparation were when they launched, and then their internal state over the last year and a half, hopefully they'll open up in a few months and do a proper retrospective post-mortem for everyone.

[Fraser]

@ebeal,

Please don't hold your breath on that one. Many of us had serious doubts regarding the clains made by John and the Mu team. I doubted that a thermal camera could be built down to the stated price claimed by Mu due to my experience with microbolometers and thermal lenses. I was proved wrong by the SEEK but it has issues as a result of the cost savings necessary for a $200 TIC module. The real issue was Mu (John) claiming that they were using an extant commonly available microbolometer. Later it was claimed that the microbolometer was military surplus stock from Missiles. That claim really made me laugh  . Unlike SEEK Mu were effectively claiming that they had discovered an untapped source of bargain priced microbolometers that no one had noticed before and that were covered by an NDA. I called bullsh*t on that claim as well. There was no evidence that such a cheap microbolomeyter existed and I even wondered whether they were trying to use a wide-band CCD chip as a thermal sensor as that responds to heat in excess of 400C.

Now SEEK were clever, they got into bed with some serious players in the thermal imaging and FPA fabrication world. Such players must have liked what SEEK proposed and likely saw the development opening up new markets. At that point Mu was a dead duck, even if it was viable, which I seriously doubt. Mu demonstrated something at a trade fair for machinery, not imaging or electronics, machinery for heavens sake ! We do not know what was inside their demo unit and I doubt you will ever see it again. It looked bulky and poorly designed. Quite a contrast to the obvious thought that has gone into the SEEK design, including its magnesium case material. Nope I expect the Mu to disappear into history with no light being shed on what, if anything, the design team produced by way of physical hardware. It isn't in their interests to say anything more. The project died with no published evidence to prove them to be wrong doers. Why risk placing evidence in the hands of potential plaintiffs ?

Aurora

[ebeall]

Whoa, thats pretty far out there (the commonly available microbolometer and the surplus missile claims), I hadn't followed the comments or updates after they got funded. I just remember when I first learned of the Mu campaign last March, I was pretty stressed out by it because I thought there was no chance for me (at that point I had just done my first design for the 32x31, which needed a lot more work), but I recovered when I calmly looked at their claims. I'd quoted out several of the lowest-cost arrays, and what they were claiming just didn't exist on the market at that point.

You make a good point that they probably won't ever open up. The manufacturing show demo may have been their proof of good faith in case of a future lawsuit but we may never know. Its fine that they spent larger investors' money to figure this out, but not cool that they spent backers money on themselves (the claim they "only" paid themselves McDonalds-equivalent pay). At first this afternoon I felt a little bad for them, but I really don't feel bad for the main guy - the way he presents _everything_ smacks of manipulation.

[blackboxdisease]

Not sure if this is what is expected. Dropbox contains the simple extracted apk which contains the libseekware.so which has also been decompiled (may not be of use), as well as the decompiled/disassebled apk file which includes all the java source.

https://www.dropbox.com/sh/7688u3m987max2q/AACBJ5hGIKm8H78m4S7dFAK7a?dl=0

[miguelvp]

So I did some experiment tonight about timing using the query performance counter.

The frame counter pixel 40 wrapped around to a total of 54027 captured frames by the firmware (not captured by the application, since the application only captured 20000 frames including pre-cal and calibration frames) in 2696.23041648431 seconds giving an average frame rate of 20.04 fps (20.037975860552493637834483136496) not all visible just captured by the firmware, not visual frame rate.

Visible frame rate will be the number of frames with ID 3  in that time so it's probably around 7 fps (20000/2696.23041648431 = 7.4177636591158101089582923858427 minus pre-cal and calibration frames).

Smallest frame time was 0.122480146923205 for 4 frames in frame 12877 (55864 in counter)
Largest frame time was 0.125570133280043 for 4 frames in frame 12658 (54928 in counter)

It went up to 5 frames per count never 8 like on the previous dump by someone else.
Largest frame time was 0.16178046177331 for 5 frames in frame 870

Also both the pre-cal frame and the cal frame took up to 0.1868 seconds

I'm attaching the file with frame count out of pixel 40, frame ID and delta time, all captured into memory and only written when it was all done.
Also there is a total time at the end after rolling over 16 bits for a total of 20000 frames that took 44 minutes and 56 seconds.

So the fastest conceivable frame rate looking at that router of mine is 32.66 fps, but overall it's only 20.04 fps max speed with the current firmware, since the pre-cal and cal times take forever (Frames ID 6 and 1)

The line number (if you have an editor that shows it) reflects the actual captured frame number (not the frame counter) for that ID.

The delta on the frame counter is how many images it took for that one frame to be captured.

Edit: warning long text file with the data attached.

Edit2: 387 is the total of pre-cal and cal frames (each for a total of 774 out of the 20000 frames captured - 5 pre-captured frames, and it should be 6 but there is an extra cal frame, so 7.13 fps average displayed frames)

[eneuro]

Video comparison from the article.

Probably different thermal LUTs used there or on Flir One the same trick was made which we saw above when very hot spots were present in Flir E4+ image-on Flir One they saturated so even if the same LUTs were used we'll get very different RGB output images recorded in this video 
Anyway we now know how Flir deals with hot spots or maybe those spots are above its maximum limits.

When we make closer look into this magic MSX on Flir One it looks that edge detection is very bad on Flir One and in more complicated scenes like this below Flir's MSX simply damages thermal image by very noisy visual context while they are not able to detect contoures and skip another unwanted visual noise

They made closer look test in this article (image above is taken from this comparision) and didn't notice that while IR & visual camera are shifted those IR & visual pixels do not fit no more at such close distance and this image "enhanced" by Flir's MSX is a garbage 
They have very limited processing power in Flir One, so of course can not do more advanced image processing, so probably that is why they also limited emissivity setting to four (4) options, than tested if output image is not complete garbage, but it is when there are many small details in visual image.

Of course there is a lot of filtering and this low res thermal 80x60 sensor output looks smooth, so when probably used in this article ancient first versions of Seek software (or newer versions are not upgraded for iPhone), Seek output looks very bad in the therms of nice image-a lot of noise present-but what we can expect when people writting this article never saw Seek frame raw data and talking about 32.1k thermal pixels which is only 206x156 result, while 2.1k of those pixels are... black hexagon pixels and we can find also a few dead pixels too in those Seek calibration frames.

Edit: warning long text file with the data attached.

Compression turns this long file into 134KB only 

Code: [Select]

$ gzip -9 dot40_20000frames_2696s.txt
$ ls -n dot40_20000frames_2696s.txt.gz
-rw-rw-r-- 1 501 501 37021 2014-11-25 11:21 dot40_20000frames_2696s.txt.gz

Did you examined Seek PCB firmware and have conclusion that it can be >30fps before it is send via USB?
This frame_dot[40] word value can be simply timestamp not any frame count, so since we do not know what is going on in Seek firmware difficult to say anything, but 20000 catched from USB in 0.75 hour gives us: 7.4 fps, but there are not only thermal scene frames there, so it will be interesting to make histogram of frame IDs which were present in this what you were able to capture and get statistics-probablity of capturing each frame ID 

Update: Histogram of frames IDs provided by @miguelvp below:

Code: [Select]

0x1 ( 1)    1.9% 388
0x3 ( 3)   96.1% 19219
0x4 ( 4)    0.0% 1
0x5 ( 5)    0.0% 1
0x6 ( 6)    1.9% 387
0x7 ( 7)    0.0% 1
0x8 ( 8)    0.0% 1
0x9 ( 9)    0.0% 1
0xa (10)    0.0% 1
So, we had in thi scase 2% of shutter thermal images in frames 0x1 and 0x6, so about 96% frames was thermal scene image frames, but probably depending on temperature changes inside this Seek Thermal dongle maybe there will be more frequent shutter calibration.
However it looks like number of Seek usb frames per time is estimate FPS of different thermal images available-in this case: 20000*0.96/2696 gives us estimate 7.1fps < 9fps ONLY available for any averaging etc? 

But ok in one of my custom applications with desired 1Hz output 4-8 image frames weighted averaging should be around needed 1Hz so its fine   

[miguelvp]

Did you examined Seek PCB firmware and have conclusion that it can be >30fps before it is send via USB?

No need to look at it, a simple image capture of a fast moving object show all the frames combined, search for Mike's video.

This frame_dot[40] word value can be simply timestamp not any frame count

It could be used as a time stamp but not a very accurate one (1/4th of a millisecond variance), all indications are that it is a frame count.

so since we do not know what is going on in Seek firmware difficult to say anything, but 20000 catched from USB in 0.75 hour gives us: 7.4 fps, but there are not only thermal scene frames there, so it will be interesting to make histogram of frame IDs which were present in this what you were able to capture and get statistics-probablity of capturing each frame ID 

...

There was no need for any of that, as I mentioned before:

Edit2: 387 is the total of pre-cal and cal frames (each for a total of 774 out of the 20000 frames captured - 5 pre-captured frames, and it should be 6 but there is an extra cal frame, so 7.13 fps average displayed frames)

Well I guess  7.128100  if you want to be more precise. (from 7.1280999882273377242034710681756)

[eneuro]

most of blurring comes from the lense
see my images here
https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg343791/#msg343791

Unfortunate those images attached in this thread are 800x600 8-bit grayscale, while it could be nice to work on 16bit 320x240 oryginal ones from Flir E4+: FLIR0238.jpg  and FlirE40: IR_0480.jpg

Code: [Select]

e40-gray.png: PNG image, 800 x 600, 8-bit grayscale, non-interlaced
e4-gray.png:  PNG image, 800 x 600, 8-bit grayscale, non-interlaced
I have latest Image-ExifTool-9.76 installed, so no problem to extract 16bit PNGs even from oryginal Flir's .JPG, so could you provide those oryginal images or 16bit ones for deeper analysis, please?

I'm very suspicious if those images are not additionaly smoothed, blured, etc.  in Flir's firmware regardless of lens used and I'd like to try enhance its quality as well as Seek Thermal blured images in my software by using methods similar to those in aerial photography:

Right is deblurred left image. so it looks like image magic but it is filtered 

[tomas123]

Unfortunate those images attached in this thread are 800x600 8-bit grayscale, while it could be nice to work on 16bit 320x240 oryginal ones from Flir E4+: FLIR0238.jpg  and FlirE40: IR_0480.jpg

sorry, I don't publish original images with camera s/n

but I described all steps

Code: [Select]

exiftool -b -RawThermalImage FLIR0238.jpg | convert - gray:- | convert -depth 16 -endian msb -size 320x240 gray:- -auto-level -resize 800x -depth 8 e4-gray.png
convert e4-gray.png iron.png -clut e4-iron.png

exiftool -b -RawThermalImage IR_0480.jpg | convert - -auto-level -resize 800x -depth 8 e40-gray.png
convert e40-gray.png iron.png -clut e40-iron.png


[eneuro]

but I described all steps

I know those exiftool steps

No problem, can work on Seek thermal images and this provided by @cynafab from Flir E4+

[blackboxdisease]

interesting from seekwaredevice.java

        numRows = 156;
        numColumns = 208;
        rawBitDepth = 16;
        normalizedBitDepth = 16;
        displayBitDepth = 8;
        frameRate = 30;
        name = "SeekThermal Camera";
        chipID = "#0000001";


seekwaredevice.java seems to contain everything correlating with requesting image, requesting the shutter action, FPS.


imports the following:



import android.hardware.usb.UsbDevice;
import android.view.Surface;
import com.subi.usb.CustomRequests;
import com.subi.usb.SUBIDevInfo;
import com.subi.usb.utils.Utils;
import com.tyriansystems.Seekware.enums.SeekwareOperationMode;
import com.tyriansystems.Seekware.enums.SeekwareResult;
import com.tyriansystems.Seekware.enums.SeekwareVideoStreamType;
import com.tyriansystems.Seekware.interfaces.SeekwareDisplayVideoCallback;
import com.tyriansystems.Seekware.interfaces.SeekwareNormalizedVideoCallback;
import com.tyriansystems.Seekware.interfaces.SeekwareProgramFirmwareCallback;
import com.tyriansystems.Seekware.interfaces.SeekwareThermalVideoCallback;
import com.tyriansystems.Seekware.interfaces.SeekwareVerifyFirmwareCallback;
import java.io.InputStream;
import java.io.UnsupportedEncodingException;
import java.nio.ByteBuffer;
import java.nio.FloatBuffer;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.Iterator;
import java.util.List;


Some interesting Device settings:



public static final int CHIP_ID_SIZE_IN_BYTES = 12;
    public static final int COMMAND_ARRAY_INFO_SIZE_IN_BYTES = 4;
    public static final int COMMAND_ARRAY_SIZE_IN_BYTES = 64;
    public static final int COMMAND_ARRAY_SIZE_IN_WORDS = 32;
    private static final byte CR = 13;
    public static final int ERROR_SIZE_IN_BYTES = 4;
    public static final int FIRMWARE_INFO_SIZE_IN_BYTES = 4;
    public static final int IMAGE_BUFFER_FRAGMENT_SIZE_FOR_SURE = 16224;
    public static final int IMAGE_BUFFER_SIZE_IN_BYTES = 64896;
    public static final int IMAGE_BUFFER_SIZE_IN_WORDS = 32448;
    public static final int IMAGE_DELAY_VALUE_IN_MILLISECONDS = 0;
    public static final int IMAGE_INFO_SIZE_IN_BYTES = 4;
    private static final byte LF = 10;
    public static final int MAX_FLASHABLE_AMOUNT_OF_BYTES = 256;
    public static final int MEMORY_FLASHING_TIME = 100;
    private static final int NEW_IMAGE_MEMORY_LOCATION = 0;
    private static final int NEW_IMAGE_SETTINGS_MEMORY_LOCATION = 1;
    public static final int ONE_BYTE_DATA_COMMAND_SIZE_IN_BYTES = 1;
    public static final int ONE_WORD_DATA_COMMAND_SIZE_IN_BYTES = 2;
    public static final int RDAC_ARRAY_SIZE_IN_WORDS = 32448;
    private static final String TAG = "CustomRequests";
    public static final int TOGGLE_SHUTTER_SIZE_IN_BYTES = 4;
    public static final int VDAC_ARRAY_SIZE_IN_WORDS = 156;
    private final int TIME_OUT_IN_MILLISECONDS = 1000;
    private final int URB_SUBI_BEGIN_MEMORY_WRITE = 82;
    private final int URB_SUBI_COMPLETE_MEMORY_WRITE = 81;
    private final int URB_SUBI_GET_BIT_DATA = 59;
    private final int URB_SUBI_GET_CURRENT_COMMAND_ARRAY = 68;
    private final int URB_SUBI_GET_DATA_PAGE = 65;
    private final int URB_SUBI_GET_DEFAULT_COMMAND_ARRAY = 71;
    private final int URB_SUBI_GET_ERROR_CODE = 53;
    private final int URB_SUBI_GET_FACTORY_SETTINGS = 88;
    private final int URB_SUBI_GET_FIRMWARE_INFO = 78;
    private final int URB_SUBI_GET_IMAGE_PROCESSING_MODE = 63;
    private final int URB_SUBI_GET_OPERATION_MODE = 61;
    private final int URB_SUBI_GET_RDAC_ARRAY = 77;
    private final int URB_SUBI_GET_SHUTTER_POLARITY = 57;
    private final int URB_SUBI_GET_VDAC_ARRAY = 74;
    private final int URB_SUBI_READ_CHIP_ID = 54;
    private final int URB_SUBI_RESET_DEVICE = 89;
    private final int URB_SUBI_SET_BIT_DATA_OFFSET = 58;
    private final int URB_SUBI_SET_CURRENT_COMMAND_ARRAY = 67;
    private final int URB_SUBI_SET_CURRENT_COMMAND_ARRAY_SIZE = 66;
    private final int URB_SUBI_SET_DATA_PAGE = 64;
    private final int URB_SUBI_SET_DEFAULT_COMMAND_ARRAY = 70;
    private final int URB_SUBI_SET_DEFAULT_COMMAND_ARRAY_SIZE = 69;
    private final int URB_SUBI_SET_FACTORY_SETTINGS = 87;
    private final int URB_SUBI_SET_FACTORY_SETTINGS_FEATURES = 86;
    private final int URB_SUBI_SET_FIRMWARE_INFO_FEATURES = 85;
    private final int URB_SUBI_SET_IMAGE_PROCESSING_MODE = 62;
    private final int URB_SUBI_SET_OPERATION_MODE = 60;
    private final int URB_SUBI_SET_RDAC_ARRAY = 76;
    private final int URB_SUBI_SET_RDAC_ARRAY_OFFSET_AND_ITEMS = 75;
    private final int URB_SUBI_SET_SHUTTER_POLARITY = 56;
    private final int URB_SUBI_SET_VDAC_ARRAY = 73;
    private final int URB_SUBI_SET_VDAC_ARRAY_OFFSET_AND_ITEMS = 72;
    private final int URB_SUBI_START_GET_IMAGE_TRANSFER = 83;
    private final int URB_SUBI_TARGET_PLATFORM = 84;
    private final int URB_SUBI_TOGGLE_SHUTTER = 55;
    private final int URB_SUBI_UPLOAD_FIRMWARE_ROW_SIZE = 79;
    private final int URB_SUBI_WRITE_MEMORY_DATA = 80;
    private final int USB_TYPE_VENDOR_TYRIAN = 65;


 I read through too much code. my head hurts. Don't know where those numbers are actually referenced. Source is missing /com/android/os but it looks like if you really want to talk to the camera, at least using java, these are the customrequests to pass to it. Maybe someone else can take over.

[cynfab]

I've been looking at the "sources" as well and have added a bit of documentation to my Python program detailing what the startup sequence is.

Read the comments imbedded in the program. Comments welcome
There is a lot of useful info in the JAD or jd decompiled sources, each are different and provide a different view of what the original .java files were.
I haven't dug into either the disassembled libseekware.so or what I got using objdump. Understanding armv7a assembly of a C++ program is not easy.

The camera initilization code corresponds to
Seek code at line 1017 of SeekwareLibrary.java:

    protected void finalizeActivation(SeekwareDevice seekwaredevice)

Code: [Select]

# You will need to have python 2.7 (3+ may work, not tried)
# and PyUSB 1.0 (needs to be gotten as source from Github and installed as root)
# and PIL (Pillow fork, often in debian distros)
# and numpy (often in debian base distros)
# and scipy
# and Tkinter
# and ImageTk (part of Pillow but a seperate module)
# and maybe some other stuff


# You will probably have to run this as root unless you get your udev/mdev rules
# set up to allow the Seek device to be used by other than root.

# Many thanks to the folks at eevblog, especially (in no particular order)
#   miguelvp, marshallh, mikeselectricstuff, sgstair, Fry-kun and many others
#     for the inspiration to figure this out
# This is not a finished product and you can use it if you like. Don't be
# suprised if there are bugs as I am NOT a programmer..... ;>))
# There is also a lot of test code sprinkled about which probably doesn't work.
# Updated the USB send/receive mseeage code to use Fry-Kun's (sort of). And add
# a bit of error checking.

# There are also the beginnings of some documentation on the Seek init sequence.


import usb.core
import usb.util
import sys
from PIL import Image, ImageTk
import numpy
import colorscale # used to colorizing the image, be sure that colorscale.py is in the
# current directory
# Original colorscale.py from https://github.com/pklaus/python-colorscale
from scipy.misc import toimage
from scipy import ndimage
import Image
from numpy import array
import Tkinter

class App(Tkinter.Tk):
    def __init__(self,parent):
        Tkinter.Tk.__init__(self,parent)
        self.parent = parent
        self.initialize()

# defs

    def usbinit(self):
# find our Seek Thermal device  289d:0010
dev = usb.core.find(idVendor=0x289d, idProduct=0x0010)

# was it found?
if dev is None:
        raise ValueError('Device not found')

# set the active configuration. With no arguments, the first
# configuration will be the active one
dev.set_configuration()

# get an endpoint instance
cfg = dev.get_active_configuration()
intf = cfg[(0,0)]

ep = usb.util.find_descriptor(
    intf,
    # match the first OUT endpoint
        custom_match = \
        lambda e: \
usb.util.endpoint_direction(e.bEndpointAddress) == \
    usb.util.ENDPOINT_OUT)

assert ep is not None

return dev
#

# send_msg sends a message that does not need or get an answer
    def send_msg(self,dev,bmRequestType, bRequest, wValue=0, wIndex=0, data_or_wLength=None, timeout=None):
assert (dev.ctrl_transfer(bmRequestType, bRequest, wValue, wIndex, data_or_wLength, timeout) == len(data_or_wLength))

# alias method to make code easier to read
# receive msg actually sends a message as well.
    def receive_msg(self,dev,bmRequestType, bRequest, wValue=0, wIndex=0, data_or_wLength=None, timeout=None):
zz = dev.ctrl_transfer(bmRequestType, bRequest, wValue, wIndex, data_or_wLength, timeout) # == len(data_or_wLength))
return zz


#  Some day we will figure out what all this init stuff is and
#  what the returned values mean.

# Here are what the commands mean, still no clue about the passed params or return values:

#   ("WINDOWS_TARGET", 0, 0);
#   ("ANDROID_TARGET", 1, 1);
#   ("MACOS_TARGET", 2, 2);
#   ("IOS_TARGET", 3, 3);

#  There seem to be 2 SeekOperationMode(s) 0 = Sleep, 1 = Run


#    BEGIN_MEMORY_WRITE = 82;
#    COMPLETE_MEMORY_WRITE = 81;
#    GET_BIT_DATA = 59;
#    GET_CURRENT_COMMAND_ARRAY = 68;
#    GET_DATA_PAGE = 65;
#    GET_DEFAULT_COMMAND_ARRAY = 71;
#    GET_ERROR_CODE = 53;
#  * GET_FACTORY_SETTINGS = 88;
#  * GET_FIRMWARE_INFO = 78;
#    GET_IMAGE_PROCESSING_MODE = 63;
#  * GET_OPERATION_MODE = 61;
#    GET_RDAC_ARRAY = 77;
#    GET_SHUTTER_POLARITY = 57;
#    GET_VDAC_ARRAY = 74;
#  * READ_CHIP_ID = 54;
#    RESET_DEVICE = 89;
#    SET_BIT_DATA_OFFSET = 58;
#    SET_CURRENT_COMMAND_ARRAY = 67;
#    SET_CURRENT_COMMAND_ARRAY_SIZE = 66;
#    SET_DATA_PAGE = 64;
#    SET_DEFAULT_COMMAND_ARRAY = 70;
#    SET_DEFAULT_COMMAND_ARRAY_SIZE = 69;
#    SET_FACTORY_SETTINGS = 87;
#  * SET_FACTORY_SETTINGS_FEATURES = 86;
#    SET_FIRMWARE_INFO_FEATURES = 85;
#  * SET_IMAGE_PROCESSING_MODE = 62;
#  * SET_OPERATION_MODE = 60;
#    SET_RDAC_ARRAY = 76;
#    SET_RDAC_ARRAY_OFFSET_AND_ITEMS = 75;
#    SET_SHUTTER_POLARITY = 56;
#    SET_VDAC_ARRAY = 73;
#    SET_VDAC_ARRAY_OFFSET_AND_ITEMS = 72;
#  * START_GET_IMAGE_TRANSFER = 83;
#  * TARGET_PLATFORM = 84;
#    TOGGLE_SHUTTER = 55;
#    UPLOAD_FIRMWARE_ROW_SIZE = 79;
#    WRITE_MEMORY_DATA = 80;


#  Only a few of the above (*) seem to be used in the normal startup sequence



# De-init the device
    def deinit(self,dev):
msg = '\x00\x00'
        for i in range(3):
    self.send_msg(dev,0x41, 0x3C, 0, 0, msg)           # 0x3c = 60  Set Operation Mode 0x0000 (Sleep)

# Camera initilization
    def camerainit(self,dev):

try:
    msg = '\x01'
    self.send_msg(dev,0x41, 0x54, 0, 0, msg)              # 0x54 = 84 Target Platform 0x01 = Android
except Exception as e:
    self.deinit(dev)
    msg = '\x01'
    self.send_msg(dev,0x41, 0x54, 0, 0, msg)              # 0x54 = 84 Target Platform 0x01 = Android


self.send_msg(dev,0x41, 0x3C, 0, 0, '\x00\x00')              # 0x3c = 60 Set operation mode    0x0000  (Sleep)
ret1 = self.receive_msg(dev,0xC1, 0x4E, 0, 0, 4)             # 0x4E = 78 Get Firmware Info
#print ret1
#array('B', [1, 3, 0, 0])                                                    # version 1.3.0.0 firmware. This was captured a while ago and may not represent the current  version.

ret2 = self.receive_msg(dev,0xC1, 0x36, 0, 0, 12)            # 0x36 = 54 Read Chip ID
#print ret2
#array('B', [20, 0, 12, 0, 86, 0, 248, 0, 199, 0, 69, 0])

self.send_msg(dev,0x41, 0x56, 0, 0, '\x20\x00\x30\x00\x00\x00')                  # 0x56 = 86 Set Factory Settings Features
ret3 = self.receive_msg(dev,0xC1, 0x58, 0, 0, 0x40)                              # 0x58 = 88 Get Factory Settings
#print ret3
#array('B', [2, 0, 0, 0, 0, 112, 91, 69, 0, 0, 140, 65, 0, 0, 192, 65, 79, 30, 86, 62, 160, 137, 64, 63, 234, 149, 178, 60, 0, 0, 0, 0, 0, 0, 0, 0, 72, 97, 41, 66, 124, 13, 1, 61, 206, 70, 240, 181, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 20, 66, 0, 0, 2, 67])

self.send_msg(dev,0x41, 0x56, 0, 0, '\x20\x00\x50\x00\x00\x00')                  # 0x56 = 86 Set Factory Settings Features
ret4 = self.receive_msg(dev,0xC1, 0x58, 0, 0, 0x40)                              # 0x58 = 88 Get Factory Settings
#print ret4
#array('B', [0, 0, 0, 0, 0, 0, 0, 0, 255, 255, 255, 255, 255, 255, 255, 255, 161, 248, 65, 63, 40, 127, 119, 60, 44, 101, 55, 193, 240, 133, 129, 63, 244, 253, 96, 66, 40, 15, 155, 63, 43, 127, 103, 186, 9, 144, 186, 52, 0, 0, 0, 0, 0, 0, 2, 67, 0, 0, 150, 67, 0, 0, 0, 0])

self.send_msg(dev,0x41, 0x56, 0, 0, '\x0C\x00\x70\x00\x00\x00')                  # 0x56 = 86 Set Factory Settings Features
ret5 = self.receive_msg(dev,0xC1, 0x58, 0, 0, 0x18)                              # 0x58 = 88 Get Factory Settings
#print ret5
#array('B', [0, 0, 0, 0, 255, 255, 255, 255, 190, 193, 249, 65, 205, 204, 250, 65, 48, 42, 177, 191, 200, 152, 147, 63])

self.send_msg(dev,0x41, 0x56, 0, 0, '\x06\x00\x08\x00\x00\x00')                  # 0x56 = 86 Set Factory Settings Features   
ret6 = self.receive_msg(dev,0xC1, 0x58, 0, 0, 0x0C)                              # 0x58 = 88 Get Factory Settings
#print ret6
#array('B', [49, 52, 48, 99, 49, 48, 69, 52, 50, 78, 55, 49])

self.send_msg(dev,0x41, 0x3E, 0, 0, '\x08\x00')                                  # 0x3E = 62 Set Image Processing Mode 0x0008
ret7 = self.receive_msg(dev,0xC1, 0x3D, 0, 0, 2)                                 # 0x3D = 61 Get Operation Mode
#print ret7
#array('B', [0, 0])

self.send_msg(dev,0x41, 0x3E, 0, 0, '\x08\x00')                                  # 0x3E = 62 Set Image Processing Mode  0x0008
self.send_msg(dev,0x41, 0x3C, 0, 0, '\x01\x00')                                  # 0x3c = 60 Set Operation Mode         0x0001  (Run)
ret8 = self.receive_msg(dev,0xC1, 0x3D, 0, 0, 2)                                 # 0x3D = 61 Get Operation Mode
#print ret8
#array('B', [1, 0])

# 11/25/2014-21:10 MDT no documentation of Seekware stuff below here... This will change.

    ...ken...

The program above is not complete and will not run.

[miguelvp]

Thank you blackboxdisease and cynfab

BTW the device settings are in CustomRequests.java, if someone is looking for them

[eneuro]

I've been looking at the "sources" as well and have added a bit of documentation to my Python program detailing what the startup sequence is.

Code: [Select]

...
self.send_msg(dev,0x41, 0x3E, 0, 0, '\x08\x00')                                 
# 0x3E = 62 Set Image Processing Mode  0x0008
self.send_msg(dev,0x41, 0x3C, 0, 0, '\x01\x00')                                 
# 0x3c = 60 Set Operation Mode         0x0001  (Run)
ret8 = self.receive_msg(dev,0xC1, 0x3D, 0, 0, 2)                                 
# 0x3D = 61 Get Operation Mode
...


I looks like parameters are passed in Little Endian 

Code: [Select]

public ShutterArg(short word0, short word1)
        {
            this$0 = RequestSequence.this;
            super();
            response = new byte[4];
            ByteBuffer bytebuffer = ByteBuffer.wrap(response).order(ByteOrder.LITTLE_ENDIAN);
            bytebuffer.putShort(word0);
            bytebuffer.putShort(word1);
        }

 public SingleCommandArg(byte byte0, byte byte1)
        {
            this$0 = RequestSequence.this;
            super();
            command = ByteBuffer.allocate(2).order(ByteOrder.LITTLE_ENDIAN);
            command.put(byte0);
            command.put(byte1);
        }

So, in the python code where frames are captured we have frame pixels count:

0x7ec0  32448=208*156   
0x41 65
private final int USB_TYPE_VENDOR_TYRIAN = 65;

0x53 83
private final int URB_SUBI_START_GET_IMAGE_TRANSFER = 83;

   msg = '\xC0\x7E\x00\x00'
  assert dev.ctrl_transfer(0x41, 0x53, 0, 0, msg) == len(msg)

However in java code they have something like this:

Code: [Select]

int j=usbdeviceconnection.controlTransfer(65, 83, 0, 0, abyte0, abyte0.length, 1000);
In your code you have something like this:

Code: [Select]

# For some reason, we need to read the result in 4 blocks, setting the
# read length to 4*0x3F60 returns only 0x3F60 bytes. Why?? maybe it is a
# PyUSB thing, since it seems to work elsewhere.

        data  = dev.read(0x81, 0x3F60, 1000)
        data += dev.read(0x81, 0x3F60, 1000)
        data += dev.read(0x81, 0x3F60, 1000)
        data += dev.read(0x81, 0x3F60, 1000)

        return data
Since 0x3F60 = 16224 = 1/2  32448=280*156, while we have 16bit raw data requested 32448 two byte words is 64896 bytes per frame which is exactly 4*0x3F60 
Not sure what 1000 means in this java code, but it looks like this what is done in python code makes now sense 

The most interesting thing is now if we could tweak frame rate at which we are able to capture thermal images from Seek camera, while quick review fo java code and no direct calls to set this frame rate, but maybe someone will find this   
Maybe there is some image processing mode different than 0x8 used in Android Seek dongle, so probably it should be safe to try different image processing starting from 0x0 to... maybe even 0xff 

Code: [Select]

private final int URB_SUBI_GET_IMAGE_PROCESSING_MODE = 63;
private final int URB_SUBI_SET_IMAGE_PROCESSING_MODE = 62;
private final int URB_SUBI_SET_OPERATION_MODE = 60;
Than try to catch frames in different mode and see if we are able to get something at higher frame rate 

For the moment optimizing my gradient correction routines for end of month demo, so if someone have time to add small loop to python code to try different image processing modes it could be fun what we get 
Maybe operation mode should be altered too, so two loops might be better-operating image processing modes from 0x0..0x8 first and operaions modes from 0x0...0x8 too 

I can't see for the moment another way to change this frame rate-maybe they locked it in hardware in those image processing and operation modes...

[eneuro]

@marshallh @miguelvp @cynfab @rickastro
After running automated tools for gradients detection on Seek sensor shutter calibration frames its output looks very interesting and maybe those efects depend on how long time this dongle was running, so this sensor response  in shutter calibration frames 0x1 was so different accross those your devices? 

NOTE: This is not  gradients visible on output thermal image, just approximation of sensor shutter frames with 6 parameters polynominals, so I'm able to compute exactly place where those gradients are zero which means maximum (or minimum) marked with black cross on normalized shutter calibration frames surfaces.

It will be very interesting to see how this shutter surface changes in time and animate this from Seek device in ambient room temperature within a few working hours  where it looks like inside around sensor was easy 36*C at 23*C room temp

[miguelvp]

I took my Seek to the optometrist (my ZnSe 4" Focal length lens showed up today).
And my seek can read the label in my router. Edit: (The red Z doesn't show neither does a big red V over the whole verizon logo) the letters showing are white/silver and they probably dissipate heat quicker.

I don't have a good adapter for the lens yet, the 3d printed one a friend made me was too small because his printer wasn't calibrated.

[Yushir0]

Fyi,  seek appears to have pushed 1.6.0 to the android market which says it addresses the thermal gradient problem.  DO NOT UPGRADE!  Not only does the gradient appear to be unchanged,  now all of the measurements are off by a huge amount!  The device is completely unusable for measuring temperature on this new version.  Example, a surface with a uniform temperature that I know for sure is around 30 degrees Celsius is coming up as around 84 degrees now.

[Galaxyrise]

around 30 degrees Celsius is coming up as around 84 degrees now.

84F is around 30C... coincidence?

[eneuro]

Not only does the gradient appear to be unchanged,  now all of the measurements are off by a huge amount!

One image or better video is worth thousands of words 
Maybe after installing new app settings were reset to default one, so probably Fahrenheit is default temperature while target market was US.

F= 1.8*C +32

30*C ~ 86*F

Lets download this new android 1.6.0 version and check it out if Seek firmware maybe changed somehow too
It is time to dissassemble this MPU *.bin file and try to look around what is going on there.
Did someone tried to assign this MPU pins to PCB circuit traces?
Probably @Aurora with his huge X-ray gun have nice photos of this dongle while can see not only the heat

Update: Just downloaded Seek apk from http://apk.dwntalk.com/wp-content/themes/iGoogler/up/com.tyriansystems.SeekThermalSeek-Thermal-1.6.0.apk -do you have the same MD5s of this apk?

Code: [Select]

$ md5sum com.tyriansystems.SeekThermalSeek-Thermal-1.6.0-20141127v102900.apk
4f46662903727d09079da60ea27a92c0  com.tyriansystems.SeekThermalSeek-Thermal-1.6.0-20141127v102900.apk

[MLutz]

same result here, temps are WAY off now...
pics attached!

btw, for my fellows in europe, here you can grab the apk:
http://apk-dl.com/  (i know this has been posted before)

and, because its my first post here:
i've read this forum since weeks, found it because Aurora posted on FB seek site, i have to say that you are brilliant guys here!!!

br,
markus

[eneuro]

btw, for my fellows in europe, here you can grap the apk:
http://apk-dl.com/

Could you run md5sum on this downloaded *.apk file?
We could see if there is exactly the same *.apk file from this link provided by you there and another one I've found and included above.
So, I've *.apk as shown above:

4f46662903727d09079da60ea27a92c0  com.tyriansystems.SeekThermalSeek-Thermal-1.6.0-20141127v102900.apk

Computed also md5sum of Seek firmware files in previous version:

398486dac94b80b0bb77efa8ec78ab41  com.tyriansystems.SeekThermal_1.4.0.2-20141113.apk

It is interesting that there is NO CHANGE in Seek firmware, so they made only changes in Android app 
Latest MPU Seek MPU firmware:

121abfd4e5fcadc246c1dfe9448dd000  com.tyriansystems.SeekThermalSeek-Thermal-1.6.0-20141127v102900/com/tyriansystems/Seekware/SUBI_LPC43XX_LPCOpen_1.2.0.0.bin
94ac6ed421559d04c6a36f65f4e67c8e  com.tyriansystems.SeekThermalSeek-Thermal-1.6.0-20141127v102900/com/tyriansystems/Seekware/SUBI_LPC43XX_LPCOpen_1.3.0.0.bin

Previous:
 

121abfd4e5fcadc246c1dfe9448dd000  com.tyriansystems.SeekThermal_1.4.0.2-20141113/com/tyriansystems/Seekware/SUBI_LPC43XX_LPCOpen_1.2.0.0.bin
94ac6ed421559d04c6a36f65f4e67c8e  com.tyriansystems.SeekThermal_1.4.0.2-20141113/com/tyriansystems/Seekware/SUBI_LPC43XX_LPCOpen_1.3.0.0.bin

It looks like is the same, but what the hell there is two versions of this firmware *.bin files and which is written to Seek dongle? 

-rw-rw-r-- 1 501 501 47768 2014-11-25 12:38 SUBI_LPC43XX_LPCOpen_1.2.0.0.bin
-rw-rw-r-- 1 501 501 47768 2014-11-25 12:38 SUBI_LPC43XX_LPCOpen_1.3.0.0.bin

Maybe close look to decompiled Android java code will tell us which version of firmware is applied for PCB MPU - it's size is the same

Update: quick binary comparision of Seek firmware files marked as 1.2.0 vs 1.3.0 might suggest that they maybe use even the same firmware for iOS while there are messages in those *.bin file like this:

Code: [Select]

Target platform: Guessing IOS
So, OK lets assume 1.3.0 firmware version and see if it is java code uploaded or somehow different using iAP ( in-application programming) and system tools 
Any reason they left those two files in the same directory from first apk versions? 

update2: libSeekware is about 10KB bigger in latest 1.6.0

Code: [Select]

-rw-rw-r-- 1 501 501 385144 2014-11-25 12:30 libSeekware-1.6.0.so
-rw-rw-r-- 1 501 501 343456 2014-11-25 12:30 libSeekware-1.6.0-v7a.so

-rw-r--r-- 1 501 501 375728 2014-10-30 15:39 libSeekware-1.4.0.2.so
-rw-r--r-- 1 501 501 329892 2014-10-30 15:39 libSeekware-1.4.0.2-v7a.so

-rw-rw-r-- 1 501 501 372808 2014-09-30 15:12 libSeekware-1.0.0.so
-rw-rw-r-- 1 501 501 326028 2014-09-30 15:12 libSeekware-1.0.0-v7a.so
BTW: This 1.6.0 apk is 2 days long (2014 Nov 25 dates inside).

[MLutz]

here is the same pic as before with °F, sorry, i forgot that...

md5 attached:
# MD5 checksums generated by MD5summer (http://www.md5summer.org)
# Generated 27.11.2014 11:19:18

4f46662903727d09079da60ea27a92c0 *com.tyriansystems.SeekThermal_1.6.0.apk

[eneuro]

here is the same pic as before with °F, sorry, i forgot that...

Yep, it will be fun to see what they did wrong in his new app if we'll have decompiled sources if possible of this new "sick" version.
Anyway emissivity settings are the same as in previous 1.4.0 version and correct for skin?
Did you tried install previous version and set the same emissivity, temp units and compare?
I can't believe they did not noticed something like this during testing 

BTW: md5sums are fine, so we have the same versions of "Sick" Thermal 1.6.0 apk