Seek Thermal camera with Raspberry Pi

[antolanca]

Hello everyone.
I wanted to buy a FLIR Lepton 3.5 for a project I'm working on but I've found out on this forum that you can use a Seek thermal camera with the Raspberry Pi as well.
For me, the FLIR would cost close to £300 while the Seek Compact would be half of that.
However the Lepton is pretty much plug and play but has less resolution at 160x120 instead of having to install and configure the Seek with a resolution of 206x156.

My questions are:

How easy is it to setup the Seek camera?
What results should I expect from it? Would they be the same as if it was connected to a mobile phone?

By the way, my knowledge of programming is very low but I am willing to give it a go if needed.

Here's the post I was referring to :

https://www.eevblog.com/forum/thermal-imaging/running-seek-thermal-cameras-(-others)-on-a-raspberry-pi/

Thank you in advance.

[Uho]

Why is it so expensive for flir? I sell mine for $ 56 with delivery. My ad  https://www.eevblog.com/forum/buysellwanted/i-want-to-change-flir-lepton-to-ssd-or-sell-it/msg2801812/#msg2801812

[Hydron]

Are you unable to buy from digikey? The lepton 3.5 is 199 USD which is ~183 GBP with 20% VAT included. Or is digikey fussy about shipping one to the UK?

[antolanca]

It's from Digikey including a PT Mini board, a Lepton 3.5, VAT & shipping from the US.
But since the Seek looks better resolution wise and is cheaper I'm considering it.

[IwuzBornanerd]

@antolanca

I am probably not the best one to answer the "How easy is it" question since I wrote the code so things are more obvious to me, but you can certainly download the zip file & look at the Readme.html file to see what is involved.  You could also try the code from git-hub that I linked in my thread.  You do need to make sure a bunch of "packages" are installed.  Here is an excerpt from the readme file:

Code: [Select]

To use this program you will need to have the following "packages" installed on your system:
 
 python 2.7 (3+ may work)
 and pyusb-1.0.0b2; even the version in Raspbian Stretch was not good enough
 and python-imaging (PIL) 1.1.6 or better
 and python-imaging-tk (python-pil.imagetk in Stretch?)
 and python-numpy
 and python-scipy
 and ImageMagick(? I think that was just for Fry-kun's original code)

ALSO now requires
 python-opencv and its openCV 2 dependencies
 Rsaspbian Jessie needed to have libgl1-mesa-dri installed
!!!!!Python PIL in Raspbian Stretch uses "Image.frombytes" instead of "Image.fromstring", which makes more sense, BUT
  I am leaving it as fromstring in this code since I am still using "Jessie" and maybe others are too.
  There are 3 places that you need to search for & change it in the code if you use Stretch (a 4th is commented out).

After installing all the required stuff, change to the directory containing the program files and compile the pixelmath module
by running 2 commands:

 python setupA.py build
 sudo python setupA.py install

Then run the program with the command: [sudo (if needed)] python Rpi2Seek.py
To avoid needing "sudo", add a file containing the following line:
   SUBSYSTEM=="usb", ATTRS{idVendor}=="289d", ATTRS{idProduct}=="0010", GROUP="plugdev"
   to the directory /etc/udev/rules.d
   My laptop somehow generated this file with the filename 50-seekthermal-usb.rules
As for the other question, the image quality with my Python code will be about the same (possibly better) as with the Seek app but the temperatures may/will be off.  Particularly since Seek changed the "frame 9" data set stored on the camera which I use as a lookup table for determining the temperatures.  As we found in @eecs's thread ( https://www.eevblog.com/forum/thermal-imaging/seek-compact-xr-lemon-or-not/ ) this needs further work & I have not found time & motivation to do that and don't have a new camera to test it with.

Amazon US currently is selling the compact & XR for less than $200 new in the micro USB version & if you intend to use it on a Pi or PC & not (or seldom) with a phone you won't be concerned about it NOT having USB-C.

Along with the low pixel count the "FLIRonie" also has a much wider field of view than any Seek and is fixed-focus so depending on just what you are using it for that could also be a factor in the decision of which one to buy.

What are you referring to that makes the FLIRone plug & play?  The vloopback driver??

[antolanca]

The FLIR Lepton I'm referring is with the PT Mini board from Groupgets not the FLIROne for mobile phone. and it's their driver I'd use.

https://store.groupgets.com/products/purethermal-mini-flir-lepton-smart-i-o-module

However, this is what I'm building. A set of Night/Thermal Vision goggles based on the Predator/Splinter Cell franchises.
I understand that the narrow field of vision on the Seek will make navigating tricky but I could later find a way to add a wider fov lens (I'm basing this assumption on the macro lens mod for the Seek. If it does have a lens on it then I might be able to change it to a wider fov or maybe even using mirrors) or I'll develop a technique to better use it as is such as scan in Thermal, navigate in Night mode.
Either way, the idea of the project is based on the kiss principle. It has to be easy to replicate.

[Vipitis]

the image latency and slow framerate of the Lepton will make navigating with it very difficult (ignoring the limited resolution and unfocused mess). You also want to think if you might want a shutterless version for not having blackouts every few minutes for a whole second. My Lepton shutter broke today and needs a friendly slap to open again.
Seek will give you more resolution and framerate  (depending on model you are getting). The fov is narrower but therefore gives you a better spacial resolution. you could look for a wide angle auxiliary adapter - but that will not be easy or cheap to find correctly.
You should consider making in picture in picture, especially when you have other wide angle camera systems mounted next to it.

[antolanca]

You mean video overlay? If I can find a way to do it then yes, it's the best way. I know I can overlay them via hardware but am not talented enough to do it by software. Unless someone has done the software for it using the Pi camera and the Seek.

[LesioQ]

Wow ! Cool project !
SeekThermal's fastframe mode would find good use in Your application,
provided You succeed digging into this esoteric field, which I failed   
 

[Fraser]

If making a thermal camera headset and intending to move around with it as a form of night vision..... remember to take anti sea sickness medication first !

A good friend of mine who designs thermal imaging systems has told me of the difficulties in using head mounted thermal imaging cameras when moving around. Such systems have been investigated for fire fighting and military deployments. The significant challenge is image latency. There is likely information to detail the issue on the internet, but in précis, if the human inertial guidance system (inner ear) is not synchronised closely with the optical inputs from the eyes, you get motion sickness very quickly ! If you are static then the effects are less of an issue. As soon as you start moving the latency in the thermal image will upset your brains navigation centres and you will vomit 

The SEEK and LEPTON cores very likely have too much latency in their image processing and adding a R-Pi to the equation will further increase the image latency. You are almost guaranteed to vomit when using such for a navigation headset. Ultra low image latency is the solution but you will not find that in cheap thermal imaging cores or processing electronics.

Just thought you should know 

Fraser

[Fraser]

http://www.chioka.in/what-is-motion-to-photon-latency/

https://en.m.wikipedia.org/wiki/Virtual_reality_sickness

https://uxplanet.org/motion-sickness-in-vr-3fa8a78216e2?gi=4f1da2da27b1

https://www.harmony.co.uk/vr-motion-sickness/


Fraser

[Fraser]

A paper on the issues .....

https://tigerprints.clemson.edu/cgi/viewcontent.cgi?article=2689&context=all_dissertations

Fraser

[Ultrapurple]

I (and others) have used the 9 and 25Hz Therm-App 384x288 cameras for night vision goggles using a cheap cellphone VR head mount. The driver software (eg ThermViewer) has to support side-by-side displays on the smartphone for this to work properly.

My experience was OK on the latency side but it was hard to work out what focal length lens felt most comfortable. A wide angle (6.8mm) was too wide; the standard 19mm feels a bit too narrow. The 13mm lens is better but I'm still not happy with the compromise.

Latency is quite small on this sort of setup, though not negligible.

There are some very nice single eye OLED eyepieces available on eBay and elsewhere that would work well with a camera giving analogue output.

Whether you'd want one or two cameras is an open question. Very few military night sights (image intensifier or thermal) appear to have two   sensors, so I guess they've decided it's not worth it. And they have the resources to test that sort of thing...

[antolanca]

My experience was OK on the latency side but it was hard to work out what focal length lens felt most comfortable. A wide angle (6.8mm) was too wide; the standard 19mm feels a bit too narrow. The 13mm lens is better but I'm still not happy with the compromise.

With the night vision, I am quite happy with the fisheye lens I have and can navigate with some ease and enough focal length to see 10m in front of me though I do need an IR Torch.
I'd like to have a look to these eye pieces you mentioned. Could you link one for me please?

If making a thermal camera headset and intending to move around with it as a form of night vision..... remember to take anti sea sickness medication first !

Luckily I'm a sailor so don't get seasick easily.  .

A good friend of mine who designs thermal imaging systems has told me of the difficulties in using head mounted thermal imaging cameras when moving around. Such systems have been investigated for fire fighting and military deployments. The significant challenge is image latency. There is likely information to detail the issue on the internet, but in précis, if the human inertial guidance system (inner ear) is not synchronised closely with the optical inputs from the eyes, you get motion sickness very quickly ! If you are static then the effects are less of an issue. As soon as you start moving the latency in the thermal image will upset your brains navigation centres and you will vomit 

This further reinforces the idea of developing a technique to scan with thermal and navigate with night vision. I can get up to 60fps at 720p on the Raspberry Pi camera but only 9 with the Seek.

The SEEK and LEPTON cores very likely have too much latency in their image processing and adding a R-Pi to the equation will further increase the image latency. You are almost guaranteed to vomit when using such for a navigation headset. Ultra low image latency is the solution but you will not find that in cheap thermal imaging cores or processing electronics.

Just thought you should know 

Fraser

I did know but it's good to be reminded. Thank you.

Wow ! Cool project !
SeekThermal's fastframe mode would find good use in Your application,
provided You succeed digging into this esoteric field, which I failed   
 

Thanks for making me aware of it.

Even i I can't get fastframe to work, I'll still be putting instructions on how to build the system online (for both the Seek and the MLX90640 thermal cameras) as I believe it would be a great tool for people to showcase as a science project, to use in airsoft or even to locate and view wild animals at night. I have contacted my son's school about showcasing it to the students as a way to get them interested in STEM (just in case it means Science, Technology, Engineering and M

[Ultrapurple]

One example of the eyepiece I mentioned is - eBay auction: #263668963160 I am sure there are many others. There is also a 1024x768 version available, but it's rather more expensive.



There is no comparison between the Raspberry Pi visible light camera and a thermal camera. A huge amount of correction and processing is needed to retrieve a usable image from the datastream emanating from any thermal sensor - that's why they usually have heavy-duty number crunching on the driver board. Back a while someone managed to get the ThermApp camera running on a Raspberry Pi but its speed was measured in seconds per frame, not frames per second. The author said his proof-of-concept code was inefficient and could be significantly optimised, though I don't know whether that has happened.

If 'all' you're trying to do is demonstrate a head mounted thermal imager then the dongle and phone route is probably your fastest - and possibly cheapest - option. Next best would be an old firefighting or similar core driving an eyepiece, though the more modern cores will be much smaller and lighter and likely have much lower power consumption.

[RoshV]

Hi sorry for reviving this older thread. I am new to SEEK thermal.

I am trying to run your python code on RPI4, Raspbian buster, with python 3.7 and SEEK thermal compact XR

I get this error:

 File "seek.py", line 150, in <module>
    disp_img = get_image()
  File "seek.py", line 104, in get_image
    send_msg(0x41, 0x53, 0, 0, '\xC0\x7E\x00\x00')
  File "seek.py", line 32, in send_msg
    assert (dev.ctrl_transfer(bmRequestType, bRequest, wValue, wIndex, data_or_wLength, timeout) == len(data_or_wLength))
  File "/home/pi/codes/lib/python3.7/site-packages/usb/core.py", line 1043, in ctrl_transfer
    self.__get_timeout(timeout))
  File "/home/pi/codes/lib/python3.7/site-packages/usb/backend/libusb1.py", line 883, in ctrl_transfer
    timeout))
  File "/home/pi/codes/lib/python3.7/site-packages/usb/backend/libusb1.py", line 595, in _check
    raise USBError(_strerror(ret), ret, _libusb_errno[ret])
usb.core.USBError: [Errno 32] Pipe error

The C++ version from https://github.com/maartenvds/libseek-thermal works well.

Many thanks in advance for your help.

[IwuzBornanerd]

"seek.py" is very old code isn't it?  But I don't know why that line would give you an error unless you did not yet have the UDEV rule set as specified on the C++ version you linked.  But why do you want slow Python code when the C++ works?  I have run my Python code on Raspian Jessie, but I don't think I tried on Buster, since I am developing C/C++ code now.  You can try my code, Rpi2SeekRevA from this thread, but Seek has made changes that will cause the temperatures to be off:

https://www.eevblog.com/forum/thermal-imaging/running-seek-thermal-cameras-(-others)-on-a-raspberry-pi/

My Python code has more features than the old seek.py did, but of course I don't know if that is the same seek.py that you have.

[RoshV]

thank you for coming back.

"usb.core.USBError: [Errno 32] Pipe error" occurs at "send_msg(0x41, 0x53, 0, 0, '\xC0\x7E\x00\x00')" after the UDEV rules are set, before that "seek.py" was throwing "no persmission" error. The permission worked on with c++ code from github.

I tried to run Rpi2SeekRevA but Import pixelmathA throws error. The only thing I changed was path to python.h (/usr/include/python3.7/Python.h) in pixelmathA.c, built and installed stepup.py using python3:
$python3 setupA.py install
running install
running build
running build_ext
running install_lib
copying build/lib.linux-armv7l-3.7/pixelmathA.cpython-37m-arm-linux-gnueabihf.so -> /home/pi/codes/lib/python3.7/site-packages
running install_egg_info
Removing /home/pi/codes/virtualenvironment/seek/lib/python3.7/site-packages/Pixelmath_rev_A-1.1.egg-info
Writing /home/pi/codes/virtualenvironment/seek/lib/python3.7/site-packages/Pixelmath_rev_A-1.1.egg-info

(seek) pi@test2:~/codes/Rpi2SeekRevA $ python3 Rpi2SeekA.py
Traceback (most recent call last):
  File "Rpi2SeekA.py", line 49, in <module>
    import pixelmathA
ImportError: /home/pi/codes/virtualenvironment/seek/lib/python3.7/site-packages/pixelmathA.cpython-37m-arm-linux-gnueabihf.so: undefined symbol: Py_InitModule

I don't really know c++, and most of the thing I have is in python, so a python wrapper for SEEK XR would be great. The one I found at https://github.com/hntuan94/libPySeek is for SEEK thermal Pro and doesn't compile: returns deprecated numpy warning and eventually ...python3.7m/numpy/__multiarray_api.h:1541:35: error: return-statement with a value.

My system: RPI4, SEEK thermal compact XR, Python 3.7

[IwuzBornanerd]

This link https://stackoverflow.com/questions/28305731/compiler-cant-find-py-initmodule-is-it-deprecated-and-if-so-what-should-i says (about someone else's code with the same error):

The code you have would work fine in Python 2.x, but Py_InitModule is no longer used in Python 3.x. Nowadays, you create a PyModuleDef structure and then pass a reference to it to PyModule_Create.

Do you really need to use Python3.7?  I have a Raspian Buster SD card here with both 3.7 & 2.7 on it & I did not put them there.  So could you try it with Python 2.7?

If you are familiar with Python you would be better than I at making the change described on the above link.

[RoshV]

I configured an ubuntu laptop to try your code with python 2.7 because I was getting resource busy error on RPI. Unfortunately, its the same on my laptop as well:

  File "Rpi2SeekA.py", line 73, in <module>
    dev.set_configuration()
  File "/home/rosh/anaconda2/lib/python2.7/site-packages/usb/core.py", line 869, in set_configuration
    self._ctx.managed_set_configuration(self, configuration)
  File "/home/rosh/anaconda2/lib/python2.7/site-packages/usb/core.py", line 102, in wrapper
    return f(self, *args, **kwargs)
  File "/home/rosh/anaconda2/lib/python2.7/site-packages/usb/core.py", line 148, in managed_set_configuration
    self.backend.set_configuration(self.handle, cfg.bConfigurationValue)
  File "/home/rosh/anaconda2/lib/python2.7/site-packages/usb/backend/libusb1.py", line 794, in set_configuration
    _check(self.lib.libusb_set_configuration(dev_handle.handle, config_value))
  File "/home/rosh/anaconda2/lib/python2.7/site-packages/usb/backend/libusb1.py", line 595, in _check
    raise USBError(_strerror(ret), ret, _libusb_errno[ret])
usb.core.USBError: [Errno 16] Resource busy

any leads?

[IwuzBornanerd]

Have you ever felt like bad luck follows you wherever you go? 

Well, Googling the error, I find that it is a common error but it makes no sense to me why the "resource" should be busy when there is nothing else on the machine that would try to talk to it.  This page has a solution or 2 https://stackoverflow.com/questions/29345325/raspberry-pyusb-gets-resource-busy

I added the dev.reset() command  on a line immediately before the dev.set_configuration() command & the program ran fine here so if it does not fix your error at least it "should" not cause other problems.   I have never had that busy error so I don't know how to duplicate it to see if the reset fixes it.

The other solution on that page with dev.detach_kernel_driver(0) looks like it needs re-attach somewhere else in the code, and they have it in the example, but I don't know where it would go in my code.

[nvsshankar]

Hi,

I am too trying your RpiSeekA.py
I am too getting an error

Traceback (most recent call last):
  File "/home/pi/Software/Seek Pi Python Codes/t4.py", line 4, in <module>
    dev.ctrl_transfer(0x41, 0x53, 0, 0, len('\xC0\x7E\x00\x00'))
  File "/usr/lib/python3/dist-packages/usb/core.py", line 1043, in ctrl_transfer
    self.__get_timeout(timeout))
  File "/usr/lib/python3/dist-packages/usb/backend/libusb1.py", line 883, in ctrl_transfer
    timeout))
  File "/usr/lib/python3/dist-packages/usb/backend/libusb1.py", line 595, in _check
    raise USBError(_strerror(ret), ret, _libusb_errno[ret])
usb.core.USBError: [Errno 32] Pipe error

All the configurations have been done as indicated in the file.
How to solve this?
Thanks in advance.

[IwuzBornanerd]

@nvsshankar;

I'm sorry I don't have a good answer for you.  But I see that your error messages are showing Python 3 libraries.  Other users have found that this code does not work with Python 3  without major changes.