« previous next »
Pages: [1] 2 3 4 Next  All   Go Down

Author Topic: Flir Ex: Realtime raw radiometric data streaming via UVC  (Read 82010 times)

0 Members and 1 Guest are viewing this topic.

Offline tmbincTopic starter

  • Frequent Contributor
  • **
  • Posts: 253
Flir Ex: Realtime raw radiometric data streaming via UVC
« on: March 05, 2015, 11:20:56 pm »
[Apologize if this is common knowledge and I just missed the memo. As everyone knows, the main thread is _huge_ and there was some talk about this, but I didn't find the final verdict of whether it's possible or not.]

I was trying to catch a fox that ate our pet rabbits from our garden on camera (people say they return to the scene of the crime). I wasn't successful (yet!), but I learned to hate the FLIR UVC implementation since it's randomly stops working during the night.

In an attempt to rectify the situation, I looked at the UVC implementation on the camera, and figured out that there's a dormant function to stream 9Hz raw radiometric data instead of just capturing the screen. With Windows and the regular UVC drivers, the best I could capture was the screen with 3.3Hz (for whatever reason; 15Hz should be possible) at (effectively) 8-bit over a pre-selected temp range.

The catch is that the UVC driver on the Flir explicitely disables radiometric streaming for the Z3 (=Flir Ex). It enables it for other cameras, though - i think the Flir Ex0 suports it out-of-the-box. The UVC stack is implemented in usbfnvideo.dll, which unfortunately is part of nk.bin, so it can't be easily patched (at least, I can't with my limited WinCE knowledge and my unwillingness to brick the E4).

The great thing is that FLIR sucks at implementing device restrictions; they have a dedicated grabber thread for the Flir Ex-series, and it _does_ indeed implement grabbing the raw data (via FVD1: instead of capturing the screen), and they _let_ you select capturing that (via setting bFormatIndex in the stream control), but they don't announce the ability for that in the UVC descriptors, so any "well-behaving" UVC client will not allow you to select it.

I've hacked libuvc (which is based on libusb) and - tada, it just worked. I patched libuvc pretty crudely so I'm sure there's a better way to do it. I've attached the patch and the crude sample (which just dumps out raw data into a file).

Let me know if you find this useful.
Logged
 

Offline OrBy

  • Regular Contributor
  • *
  • Posts: 220
Re: Flir Ex: Realtime raw radiometric data streaming via UVC
« Reply #1 on: March 06, 2015, 01:14:13 am »
First time I have seen any info on this myself - nice work!
Would love to see what the output can turn out like.

Logged
 

Offline tmbincTopic starter

  • Frequent Contributor
  • **
  • Posts: 253
Re: Flir Ex: Realtime raw radiometric data streaming via UVC
« Reply #2 on: March 10, 2015, 07:56:12 pm »
A few hundred lines of code later (simply record all frames with a max temp of higher than some threshold, re-mapping to 8-bit, then encoding as mp4), I got a good picture of the beast:



Mission accomplished.
Logged
 

Offline Bud

  • Super Contributor
  • ***
  • Posts: 7145
  • Country: ca
Re: Flir Ex: Realtime raw radiometric data streaming via UVC
« Reply #3 on: March 11, 2015, 01:55:43 am »
Hi tmbinc, if you could post a compiled executable and a patched dll that would help us try.  ;)
Logged
Facebook-free life and Rigol-free shack.
 

Offline tomas123

  • Frequent Contributor
  • **
  • Posts: 832
  • Country: de
Re: Flir Ex: Realtime raw radiometric data streaming via UVC
« Reply #4 on: August 12, 2015, 06:31:02 pm »
edit 08.09.2015
----------------------link list for this thread: ----------------------

You need a E4 firmware 2.3.0!!  FW 1.19.8  doesn't work (see post below)

source code for above video fuchsjagd
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg736344/#msg736344

calculate temperature from 16 bit RAW values
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg748218/#msg748218
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg744673/#msg744673

rebuild a radiometric flir jpeg from a single RAW frame
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg732177/#msg732177

use ImageJ for reading a RAW video file
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg732349/#msg732349

stream RAW thermal 16-bit video and display a rescaled 8-bit video in a OpenCV window:
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg749464/#msg749464
source code:  https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg748654/#msg748654


stream a  8-bit video to V4L2 loopback devices ( /dev/video0 ):
https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg756530/#msg756530

an older post for streaming the E4 screen video to a raspberry pi (UVC, no RAW video)
https://www.eevblog.com/forum/testgear/capturing-video-from-flir-e4-to-portable-device/msg368083/#msg368083

---------------------- end link list  ----------------------

only for information:

I'm played around with the uvc patch, but it doesn't work with my E4 (RAW Video).

E4
Modell: E4 1.1
Software: 1.19.8 

first compile patched UVC library
Code: [Select]
$ uname -a
Linux ubu64 3.13.0-32-generic #57-Ubuntu SMP Tue Jul 15 03:51:08 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

$ apt-get install CMake

$ apt --installed list| grep -i usb
libusb-0.1-4/trusty,now 2:0.1.12-23.3ubuntu1 amd64 [installed]
libusb-1.0-0/trusty,now 2:1.0.17-1ubuntu2 amd64 [installed]

$ sudo apt-get update && sudo apt-get install build-essential
$ sudo apt-get install  libjpeg-dev

$ sudo apt-get install libusb-1.0-0-dev

$ git clone [url]https://github.com/ktossell/libuvc[/url]

// don't forget to apply the patch from tmbinc in /src/
$ patch < libuvc.patch
patching file stream.c

$ cd libuvc
$ mkdir build
$ cd build
$ cmake ..
-- The C compiler identification is GNU 4.8.4
-- The CXX compiler identification is GNU 4.8.4
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- No build type selected, default to Release
-- No target type selected, default to shared library
-- Building libuvc with JPEG support.
-- Configuring done
-- Generating done
-- Build files have been written to: ../flir/libuvc/build


$ make
Scanning dependencies of target uvc
[ 11%] Building C object CMakeFiles/uvc.dir/src/ctrl.c.o
[ 22%] Building C object CMakeFiles/uvc.dir/src/ctrl-gen.c.o
[ 33%] Building C object CMakeFiles/uvc.dir/src/device.c.o
[ 44%] Building C object CMakeFiles/uvc.dir/src/diag.c.o
[ 55%] Building C object CMakeFiles/uvc.dir/src/frame.c.o
[ 66%] Building C object CMakeFiles/uvc.dir/src/init.c.o
[ 77%] Building C object CMakeFiles/uvc.dir/src/stream.c.o
[ 88%] Building C object CMakeFiles/uvc.dir/src/misc.c.o
[100%] Building C object CMakeFiles/uvc.dir/src/frame-mjpeg.c.o
Linking C shared library libuvc.so
[100%] Built target uvc

$ sudo make install
[100%] Built target uvc
Install the project...
-- Install configuration: "Release"
-- Installing: /usr/local/lib/libuvc.so
-- Installing: /usr/local/include/libuvc/libuvc.h
-- Installing: /usr/local/include/libuvc/libuvc_config.h
-- Installing: /usr/local/lib/cmake/libuvc/libuvcConfig.cmake
-- Installing: /usr/local/lib/cmake/libuvc/libuvcConfigVersion.cmake
-- Installing: /usr/local/lib/cmake/libuvc/libuvcTargets.cmake
-- Installing: /usr/local/lib/cmake/libuvc/libuvcTargets-release.cmake

renew library cache and compile flir.c tool from user tmbinc  :-+
Code: [Select]
$ sudo ldconfig
$ /usr/bin/cc  -o flir ../src/flir.c -luvc
check Flir E4 USB announcements
Code: [Select]
$ lsusb -v
Bus 001 Device 006: ID 09cb:1007
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass          239 Miscellaneous Device
  bDeviceSubClass         2 ?
  bDeviceProtocol         1 Interface Association
  bMaxPacketSize0        64
  idVendor           0x09cb
  idProduct          0x1007
  bcdDevice            0.00
  iManufacturer           7 FLIR Systems
  iProduct                8 FLIR Ex-Series
  iSerial                 0
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength          241
    bNumInterfaces          3
    bConfigurationValue     1
    iConfiguration          0
    bmAttributes         0x80
      (Bus Powered)
    MaxPower              500mA
    Interface Association:
      bLength                 8
      bDescriptorType        11
      bFirstInterface         0
      bInterfaceCount         2
      bFunctionClass         14 Video
      bFunctionSubClass       3 Video Interface Collection
      bFunctionProtocol       0
      iFunction              10 FLIR USB Video
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass        14 Video
      bInterfaceSubClass      1 Video Control
      bInterfaceProtocol      0
      iInterface              2 FLIR Ex-Series
      VideoControl Interface Descriptor:
        bLength                13
        bDescriptorType        36
        bDescriptorSubtype      1 (HEADER)
        bcdUVC               1.00
        wTotalLength           85
        dwClockFrequency        0.001000MHz
        bInCollection           1
        baInterfaceNr( 0)       1
      VideoControl Interface Descriptor:
        bLength                18
        bDescriptorType        36
        bDescriptorSubtype      2 (INPUT_TERMINAL)
        bTerminalID             1
        wTerminalType      0x0201 Camera Sensor
        bAssocTerminal          0
        iTerminal               0
        wObjectiveFocalLengthMin      0
        wObjectiveFocalLengthMax      0
        wOcularFocalLength            0
        bControlSize                  3
        bmControls           0x00020260
          Focus (Absolute)
          Focus (Relative)
          Zoom (Absolute)
          Focus, Auto
      VideoControl Interface Descriptor:
        bLength                 9
        bDescriptorType        36
        bDescriptorSubtype      3 (OUTPUT_TERMINAL)
        bTerminalID             3
        wTerminalType      0x0101 USB Streaming
        bAssocTerminal          0
        bSourceID               6
        iTerminal               0
      VideoControl Interface Descriptor:
        bLength                 7
        bDescriptorType        36
        bDescriptorSubtype      4 (SELECTOR_UNIT)
        bUnitID                 4
        bNrInPins               1
        baSource( 0)            1
        iSelector               0
      VideoControl Interface Descriptor:
        bLength                11
        bDescriptorType        36
        bDescriptorSubtype      5 (PROCESSING_UNIT)
      Warning: Descriptor too short
        bUnitID                 5
        bSourceID               4
        wMaxMultiplier          0
        bControlSize            2
        bmControls     0x00000217
          Brightness
          Contrast
          Hue
          Sharpness
          Gain
        iProcessing             0
        bmVideoStandards     0x1b
          None
          NTSC - 525/60
          SECAM - 625/50
          NTSC - 625/50
      VideoControl Interface Descriptor:
        bLength                27
        bDescriptorType        36
        bDescriptorSubtype      6 (EXTENSION_UNIT)
        bUnitID                 6
        guidExtensionCode         {d41f59fa-5094-463a-b3bb-e7858a831fa3}
        bNumControl             4
        bNrPins                 1
        baSourceID( 0)          5
        bControlSize            2
        bmControls( 0)       0x0f
        bmControls( 1)       0x00
        iExtension              0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0008  1x 8 bytes
        bInterval              16
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        1
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass        14 Video
      bInterfaceSubClass      2 Video Streaming
      bInterfaceProtocol      0
      iInterface              0
      VideoStreaming Interface Descriptor:
        bLength                            14
        bDescriptorType                    36
        bDescriptorSubtype                  1 (INPUT_HEADER)
        bNumFormats                         1
        wTotalLength                       79
        bEndPointAddress                  130
        bmInfo                              0
        bTerminalLink                       3
        bStillCaptureMethod                 0
        bTriggerSupport                     0
        bTriggerUsage                       0
        bControlSize                        1
        bmaControls( 0)                    27
      VideoStreaming Interface Descriptor:
        bLength                            27
        bDescriptorType                    36
        bDescriptorSubtype                  4 (FORMAT_UNCOMPRESSED)
        bFormatIndex                        1
        bNumFrameDescriptors                1
        guidFormat                            {59555932-0000-1000-8000-00aa00389b71}
        bBitsPerPixel                      16
        bDefaultFrameIndex                  1
        bAspectRatioX                       0
        bAspectRatioY                       0
        bmInterlaceFlags                 0x00
          Interlaced stream or variable: No
          Fields per frame: 2 fields
          Field 1 first: No
          Field pattern: Field 1 only
          bCopyProtect                      0
      VideoStreaming Interface Descriptor:
        bLength                            38
        bDescriptorType                    36
        bDescriptorSubtype                  5 (FRAME_UNCOMPRESSED)
        bFrameIndex                         1
        bmCapabilities                   0x03
          Still image supported
          Fixed frame-rate
        wWidth                            320
        wHeight                           240
        dwMinBitRate                   912384
        dwMaxBitRate                   912384
        dwMaxVideoFrameBufferSize      153600
        dwDefaultFrameInterval        2666664
        bFrameIntervalType                  3
        dwFrameInterval( 0)            666666
        dwFrameInterval( 1)           1333332
        dwFrameInterval( 2)           2666664
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x82  EP 2 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        2
      bAlternateSetting       0
      bNumEndpoints           2
      bInterfaceClass         8 Mass Storage
      bInterfaceSubClass      6 SCSI
      bInterfaceProtocol     80 Bulk-Only
      iInterface              0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x83  EP 3 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x04  EP 4 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
Device Qualifier (for other device speed):
  bLength                10
  bDescriptorType         6
  bcdUSB               2.00
  bDeviceClass          239 Miscellaneous Device
  bDeviceSubClass         2 ?
  bDeviceProtocol         1 Interface Association
  bMaxPacketSize0        64
  bNumConfigurations      1
Device Status:     0x0000
  (Bus Powered)
now grep a stream
Code: [Select]
$ sudo libuvc/flir e4-1.raw
UVC initialized
Device found
Device opened
DEVICE CONFIGURATION (09cb:1007/[none]) ---
Status: idle
VideoControl:
     bcdUVC: 0x0100
VideoStreaming(1):
     bEndpointAddress: 130
     Formats:
     UncompressedFormat(1)
            bits per pixel: 16
            GUID: 5955593200001000800000aa00389b71 (YUY2)
            default frame: 1
            aspect ratio: 0x0
            interlace flags: 00
            copy protect: 00
               FrameDescriptor(1)
                 capabilities: 03
                 size: 320x240
                 bit rate: 912384-912384
                 max frame size: 153600
                 default interval: 1/3
                 interval[0]: 1/15
                 interval[1]: 1/7
                 interval[2]: 1/3
END DEVICE CONFIGURATION
bmHint: 0000
bFormatIndex: 1
bFrameIndex: 1
dwFrameInterval: 666666
wKeyFrameRate: 0
wPFrameRate: 0
wCompQuality: 0
wCompWindowSize: 0
wDelay: 1
dwMaxVideoFrameSize: 153600
dwMaxPayloadTransferSize: 157696
bInterfaceNumber: 1
Streaming...
got frame 1439397994:439760, 153600 bytes
got frame 1439397994:533467, 153600 bytes
got frame 1439397994:629556, 153600 bytes
got frame 1439397994:726339, 153600 bytes
got frame 1439397994:820088, 153600 bytes
got frame 1439397994:916235, 153600 bytes
got frame 1439397995:22432, 153600 bytes
got frame 1439397995:115699, 153600 bytes
got frame 1439397995:209889, 153600 bytes
got frame 1439397995:304196, 153600 bytes

split the stream and convert a single frame
Code: [Select]
$ split -b 153600 e4-1.raw
$ convert -size 320x240  yuv:xaa -separate 4.png
the result is the known screen image and not a RAW image:(
« Last Edit: September 16, 2015, 10:52:10 pm by tomas123 »
Logged
 
The following users thanked this post: eg14

Offline tomas123

  • Frequent Contributor
  • **
  • Posts: 832
  • Country: de
Re: Flir Ex: Realtime raw radiometric data streaming via UVC
« Reply #5 on: August 12, 2015, 08:07:38 pm »
the same process with an E40 works fine

E40
Modell E40 0.10
Software: 2.23.14

USB annoucement
Code: [Select]
$ lsusb -v

Bus 001 Device 008: ID 09cb:1005
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass          239 Miscellaneous Device
  bDeviceSubClass         2 ?
  bDeviceProtocol         1 Interface Association
  bMaxPacketSize0        64
  idVendor           0x09cb
  idProduct          0x1005
  bcdDevice            0.00
  iManufacturer           6 FLIR Systems
  iProduct                7 FLIR Exx Series
  iSerial                 0
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength          345
    bNumInterfaces          4
    bConfigurationValue     1
    iConfiguration          0
    bmAttributes         0xc0
      Self Powered
    MaxPower                0mA
    Interface Association:
      bLength                 8
      bDescriptorType        11
      bFirstInterface         0
      bInterfaceCount         2
      bFunctionClass          2 Communications
      bFunctionSubClass       2 Abstract (modem)
      bFunctionProtocol     255 Vendor Specific (MSFT RNDIS?)
      iFunction               9 FLIR Network Camera
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass         2 Communications
      bInterfaceSubClass      2 Abstract (modem)
      bInterfaceProtocol    255 Vendor Specific (MSFT RNDIS?)
      iInterface              0
      CDC Header:
        bcdCDC               1.10
      CDC Call Management:
        bmCapabilities       0x00
        bDataInterface          1
      CDC ACM:
        bmCapabilities       0x00
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0008  1x 8 bytes
        bInterval               4
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        1
      bAlternateSetting       0
      bNumEndpoints           2
      bInterfaceClass        10 CDC Data
      bInterfaceSubClass      0 Unused
      bInterfaceProtocol      0
      iInterface              0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x82  EP 2 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x03  EP 3 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
    Interface Association:
      bLength                 8
      bDescriptorType        11
      bFirstInterface         2
      bInterfaceCount         2
      bFunctionClass         14 Video
      bFunctionSubClass       3 Video Interface Collection
      bFunctionProtocol       0
      iFunction              10 FLIR USB Video
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        2
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass        14 Video
      bInterfaceSubClass      1 Video Control
      bInterfaceProtocol      0
      iInterface              4 FLIR Exx Series
      VideoControl Interface Descriptor:
        bLength                13
        bDescriptorType        36
        bDescriptorSubtype      1 (HEADER)
        bcdUVC               1.00
        wTotalLength           85
        dwClockFrequency        0.001000MHz
        bInCollection           1
        baInterfaceNr( 0)       3
      VideoControl Interface Descriptor:
        bLength                18
        bDescriptorType        36
        bDescriptorSubtype      2 (INPUT_TERMINAL)
        bTerminalID             1
        wTerminalType      0x0201 Camera Sensor
        bAssocTerminal          0
        iTerminal               0
        wObjectiveFocalLengthMin      0
        wObjectiveFocalLengthMax      0
        wOcularFocalLength            0
        bControlSize                  3
        bmControls           0x00020260
          Focus (Absolute)
          Focus (Relative)
          Zoom (Absolute)
          Focus, Auto
      VideoControl Interface Descriptor:
        bLength                 9
        bDescriptorType        36
        bDescriptorSubtype      3 (OUTPUT_TERMINAL)
        bTerminalID             3
        wTerminalType      0x0101 USB Streaming
        bAssocTerminal          0
        bSourceID               6
        iTerminal               0
      VideoControl Interface Descriptor:
        bLength                 7
        bDescriptorType        36
        bDescriptorSubtype      4 (SELECTOR_UNIT)
        bUnitID                 4
        bNrInPins               1
        baSource( 0)            1
        iSelector               0
      VideoControl Interface Descriptor:
        bLength                11
        bDescriptorType        36
        bDescriptorSubtype      5 (PROCESSING_UNIT)
      Warning: Descriptor too short
        bUnitID                 5
        bSourceID               4
        wMaxMultiplier          0
        bControlSize            2
        bmControls     0x00000217
          Brightness
          Contrast
          Hue
          Sharpness
          Gain
        iProcessing             0
        bmVideoStandards     0x1b
          None
          NTSC - 525/60
          SECAM - 625/50
          NTSC - 625/50
      VideoControl Interface Descriptor:
        bLength                27
        bDescriptorType        36
        bDescriptorSubtype      6 (EXTENSION_UNIT)
        bUnitID                 6
        guidExtensionCode         {d41f59fa-5094-463a-b3bb-e7858a831fa3}
        bNumControl             4
        bNrPins                 1
        baSourceID( 0)          5
        bControlSize            2
        bmControls( 0)       0x0f
        bmControls( 1)       0x00
        iExtension              0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x84  EP 4 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0008  1x 8 bytes
        bInterval              16
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        3
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass        14 Video
      bInterfaceSubClass      2 Video Streaming
      bInterfaceProtocol      0
      iInterface              0
      VideoStreaming Interface Descriptor:
        bLength                            15
        bDescriptorType                    36
        bDescriptorSubtype                  1 (INPUT_HEADER)
        bNumFormats                         2
        wTotalLength                      145
        bEndPointAddress                  133
        bmInfo                              0
        bTerminalLink                       3
        bStillCaptureMethod                 0
        bTriggerSupport                     0
        bTriggerUsage                       0
        bControlSize                        1
        bmaControls( 0)                    27
        bmaControls( 1)                    27
      VideoStreaming Interface Descriptor:
        bLength                            27
        bDescriptorType                    36
        bDescriptorSubtype                  4 (FORMAT_UNCOMPRESSED)
        bFormatIndex                        1
        bNumFrameDescriptors                1
        guidFormat                            {59555932-0000-1000-8000-00aa00389b71}
        bBitsPerPixel                      16
        bDefaultFrameIndex                  1
        bAspectRatioX                       0
        bAspectRatioY                       0
        bmInterlaceFlags                 0x00
          Interlaced stream or variable: No
          Fields per frame: 2 fields
          Field 1 first: No
          Field pattern: Field 1 only
          bCopyProtect                      0
      VideoStreaming Interface Descriptor:
        bLength                            38
        bDescriptorType                    36
        bDescriptorSubtype                  5 (FRAME_UNCOMPRESSED)
        bFrameIndex                         1
        bmCapabilities                   0x03
          Still image supported
          Fixed frame-rate
        wWidth                            320
        wHeight                           240
        dwMinBitRate                   912384
        dwMaxBitRate                   912384
        dwMaxVideoFrameBufferSize      153600
        dwDefaultFrameInterval        1333332
        bFrameIntervalType                  3
        dwFrameInterval( 0)            333333
        dwFrameInterval( 1)            666666
        dwFrameInterval( 2)           1333332
      VideoStreaming Interface Descriptor:
        bLength                            27
        bDescriptorType                    36
        bDescriptorSubtype                  4 (FORMAT_UNCOMPRESSED)
        bFormatIndex                        2
        bNumFrameDescriptors                1
        guidFormat                            {46374d30-0000-1000-8000-00aa00389b71}
        bBitsPerPixel                      16
        bDefaultFrameIndex                  1
        bAspectRatioX                       0
        bAspectRatioY                       0
        bmInterlaceFlags                 0x00
          Interlaced stream or variable: No
          Fields per frame: 2 fields
          Field 1 first: No
          Field pattern: Field 1 only
          bCopyProtect                      0
      VideoStreaming Interface Descriptor:
        bLength                            38
        bDescriptorType                    36
        bDescriptorSubtype                  5 (FRAME_UNCOMPRESSED)
        bFrameIndex                         1
        bmCapabilities                   0x03
          Still image supported
          Fixed frame-rate
        wWidth                            320
        wHeight                           246
        dwMinBitRate                   912384
        dwMaxBitRate                   912384
        dwMaxVideoFrameBufferSize      157440
        dwDefaultFrameInterval        1333332
        bFrameIntervalType                  3
        dwFrameInterval( 0)            333333
        dwFrameInterval( 1)            666666
        dwFrameInterval( 2)           1333332
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x85  EP 5 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
Device Qualifier (for other device speed):
  bLength                10
  bDescriptorType         6
  bcdUSB               2.00
  bDeviceClass          239 Miscellaneous Device
  bDeviceSubClass         2 ?
  bDeviceProtocol         1 Interface Association
  bMaxPacketSize0        64
  bNumConfigurations      1
Device Status:     0x0001
  Self Powered
grep a stream
Code: [Select]
$ sudo libuvc/flir e40.raw
UVC initialized
Device found
Device opened
DEVICE CONFIGURATION (09cb:1004/0a7f4000-c632-0000-0000-000000000000) ---
Status: idle
VideoControl:
     bcdUVC: 0x0100
VideoStreaming(1):
     bEndpointAddress: 130
     Formats:
     UncompressedFormat(1)
            bits per pixel: 16
            GUID: 5955593200001000800000aa00389b71 (YUY2)
            default frame: 1
            aspect ratio: 0x0
            interlace flags: 00
            copy protect: 00
               FrameDescriptor(1)
                 capabilities: 03
                 size: 320x240
                 bit rate: 912384-912384
                 max frame size: 153600
                 default interval: 1/7
                 interval[0]: 1/30
                 interval[1]: 1/15
                 interval[2]: 1/7
     UncompressedFormat(2)
            bits per pixel: 16
            GUID: 46374d3000001000800000aa00389b71 (F7M0)
            default frame: 1
            aspect ratio: 0x0
            interlace flags: 00
            copy protect: 00
               FrameDescriptor(1)
                 capabilities: 03
                 size: 320x246
                 bit rate: 912384-912384
                 max frame size: 157440
                 default interval: 1/7
                 interval[0]: 1/30
                 interval[1]: 1/15
                 interval[2]: 1/7
END DEVICE CONFIGURATION
bmHint: 0000
bFormatIndex: 1
bFrameIndex: 1
dwFrameInterval: 666666
wKeyFrameRate: 0
wPFrameRate: 0
wCompQuality: 0
wCompWindowSize: 0
wDelay: 1
dwMaxVideoFrameSize: 157440
dwMaxPayloadTransferSize: 157696
bInterfaceNumber: 1
Streaming...
got frame 1439394751:346415, 157440 bytes
got frame 1439394751:411283, 157440 bytes
got frame 1439394751:479331, 157440 bytes
got frame 1439394751:546262, 157440 bytes
got frame 1439394751:610415, 157440 bytes
got frame 1439394751:679598, 157440 bytes
got frame 1439394751:741424, 157440 bytes
got frame 1439394751:810494, 157440 bytes
got frame 1439394751:874681, 157440 bytes
got frame 1439394751:938867, 157440 bytes
....
the streams contains RAW FFF frames
Code: [Select]
$ hexdump -C  e40.raw
00000000  f4 48 f6 48 f7 48 03 49  04 49 02 49 01 49 00 49  |.H.H.H.I.I.I.I.I|
00000010  14 49 fc 48 ff 48 04 49  fe 48 0a 49 fd 48 0d 49  |.I.H.H.I.H.I.H.I|

...
000257f0  09 49 19 49 18 49 0e 49  0e 49 0c 49 13 49 32 49  |.I.I.I.I.I.I.I2I|
00025800  46 46 46 00 55 56 43 00  00 00 00 00 00 00 00 00  |FFF.UVC.........|
00025810  00 00 00 00 64 00 00 00  40 00 00 00 02 00 00 00  |....d...@.......|
split the stream and convert a frame
Code: [Select]
$ split -b 157440 e40.raw
$ convert -depth 16 -size 320x240 -auto-level gray:xaa 8.png
it's a RAW image  :-+


check frame rate with exiftool (15 fps)
Code: [Select]
$ dd if=e40.raw of=e40allframe.ff bs=153600 skip=1

$ exiftool -Date* x*
======== xaa
Date/Time Original              : 2015:08:12 17:39:48.413+01:00
======== xab
Date/Time Original              : 2015:08:12 17:39:48.479+01:00
======== xac
Date/Time Original              : 2015:08:12 17:39:48.545+01:00
======== xad
Date/Time Original              : 2015:08:12 17:39:48.611+01:00
======== xae
Date/Time Original              : 2015:08:12 17:39:48.677+01:00
======== xaf
Date/Time Original              : 2015:08:12 17:39:48.743+01:00
======== xag
Date/Time Original              : 2015:08:12 17:39:48.809+01:00
======== xah
Date/Time Original              : 2015:08:12 17:39:48.875+01:00
======== xai
Date/Time Original              : 2015:08:12 17:39:48.941+01:00
======== xaj
Date/Time Original              : 2015:08:12 17:39:49.007+01:00
======== xak
Date/Time Original              : 2015:08:12 17:39:49.073+01:00
======== xal
Date/Time Original              : 2015:08:12 17:39:49.139+01:00
======== xam
Date/Time Original              : 2015:08:12 17:39:49.205+01:00
======== xan
Date/Time Original              : 2015:08:12 17:39:49.271+01:00
======== xao
Date/Time Original              : 2015:08:12 17:39:49.337+01:00
======== xap
Date/Time Original              : 2015:08:12 17:39:49.403+01:00
or all values:
Code: [Select]
$ exiftool xaa
ExifTool Version Number         : 9.54
File Name                       : xaa
Directory                       : .
File Size                       : 154 kB
File Modification Date/Time     : 2015:08:12 18:16:45+02:00
File Access Date/Time           : 2015:08:12 18:27:35+02:00
File Inode Change Date/Time     : 2015:08:12 18:16:45+02:00
File Permissions                : rw-r--r--
File Type                       : FLIR
MIME Type                       : application/unknown
Emissivity                      : 0.95
Object Distance                 : 1.00 m
Reflected Apparent Temperature  : 20.0 C
Atmospheric Temperature         : -4.0 C
IR Window Temperature           : 20.0 C
IR Window Transmission          : 1.00
Relative Humidity               : 50.0 %
Planck R1                       : 14345.714
Planck B                        : 1347.7
Planck F                        : 1
Atmospheric Trans Alpha 1       : 0.006569
Atmospheric Trans Alpha 2       : 0.012620
Atmospheric Trans Beta 1        : -0.002276
Atmospheric Trans Beta 2        : -0.006670
Atmospheric Trans X             : 1.900000
Camera Temperature Range Max    : 120.0 C
Camera Temperature Range Min    : -20.0 C
Camera Model                    : FLIR E40
Camera Part Number              : 49001-2001
Camera Serial Number            : 490337633
Camera Software                 : 20.0.0
Lens Model                      : FOL18
Lens Part Number                :
Lens Serial Number              :
Field Of View                   : 25.0 deg
Filter Model                    :
Filter Part Number              :
Filter Serial Number            :
Planck O                        : -5711
Planck R2                       : 0.01234
Raw Value Median                : 18457
Raw Value Range                 : 780
Date/Time Original              : 2015:08:12 17:39:48.413+01:00
Focus Step Count                : 1526
Focus Distance                  : 5.3 m
« Last Edit: August 12, 2015, 08:13:41 pm by tomas123 »
Logged
 

Offline tmbincTopic starter

  • Frequent Contributor
  • **
  • Posts: 253
Re: Flir Ex: Realtime raw radiometric data streaming via UVC
« Reply #6 on: August 12, 2015, 09:21:31 pm »
Meh. Okay.

I'm on 2.3, so maybe it was added later. Could someone with a Flir Ex with 2.3 firmware give it a try? I wouldn't know what else it could be.
Logged
 

Offline tomas123

  • Frequent Contributor
  • **
  • Posts: 832
  • Country: de
Re: Flir Ex: Realtime raw radiometric data streaming via UVC
« Reply #7 on: August 12, 2015, 10:54:06 pm »
Do you see differences in the usb announcements "VideoControl Interface Descriptor" between my firmware 1.19.8  and your firmware 2.3?
Code: [Select]
$ lsusb -v
Bus 001 Device 006: ID 09cb:1007
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass          239 Miscellaneous Device
  bDeviceSubClass         2 ?
  bDeviceProtocol         1 Interface Association
  bMaxPacketSize0        64
  idVendor           0x09cb
  idProduct          0x1007
  bcdDevice            0.00
  iManufacturer           7 FLIR Systems
  iProduct                8 FLIR Ex-Series
  iSerial                 0
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength          241
    bNumInterfaces          3
    bConfigurationValue     1
    iConfiguration          0
    bmAttributes         0x80
      (Bus Powered)
    MaxPower              500mA
    Interface Association:
      bLength                 8
      bDescriptorType        11
      bFirstInterface         0
      bInterfaceCount         2
      bFunctionClass         14 Video
      bFunctionSubClass       3 Video Interface Collection
      bFunctionProtocol       0
      iFunction              10 FLIR USB Video
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass        14 Video
      bInterfaceSubClass      1 Video Control
      bInterfaceProtocol      0
      iInterface              2 FLIR Ex-Series
      VideoControl Interface Descriptor:
        bLength                13
        bDescriptorType        36
        bDescriptorSubtype      1 (HEADER)
        bcdUVC               1.00
        wTotalLength           85
        dwClockFrequency        0.001000MHz
        bInCollection           1
        baInterfaceNr( 0)       1
      VideoControl Interface Descriptor:
        bLength                18
        bDescriptorType        36
        bDescriptorSubtype      2 (INPUT_TERMINAL)
        bTerminalID             1
        wTerminalType      0x0201 Camera Sensor
        bAssocTerminal          0
        iTerminal               0
        wObjectiveFocalLengthMin      0
        wObjectiveFocalLengthMax      0
        wOcularFocalLength            0
        bControlSize                  3
        bmControls           0x00020260
          Focus (Absolute)
          Focus (Relative)
          Zoom (Absolute)
          Focus, Auto
      VideoControl Interface Descriptor:
        bLength                 9
        bDescriptorType        36
        bDescriptorSubtype      3 (OUTPUT_TERMINAL)
        bTerminalID             3
        wTerminalType      0x0101 USB Streaming
        bAssocTerminal          0
        bSourceID               6
        iTerminal               0
      VideoControl Interface Descriptor:
        bLength                 7
        bDescriptorType        36
        bDescriptorSubtype      4 (SELECTOR_UNIT)
        bUnitID                 4
        bNrInPins               1
        baSource( 0)            1
        iSelector               0
      VideoControl Interface Descriptor:
        bLength                11
        bDescriptorType        36
        bDescriptorSubtype      5 (PROCESSING_UNIT)
      Warning: Descriptor too short
        bUnitID                 5
        bSourceID               4
        wMaxMultiplier          0
        bControlSize            2
        bmControls     0x00000217
          Brightness
          Contrast
          Hue
          Sharpness
          Gain
        iProcessing             0
        bmVideoStandards     0x1b
          None
          NTSC - 525/60
          SECAM - 625/50
          NTSC - 625/50
      VideoControl Interface Descriptor:
        bLength                27
        bDescriptorType        36
        bDescriptorSubtype      6 (EXTENSION_UNIT)
        bUnitID                 6
        guidExtensionCode         {d41f59fa-5094-463a-b3bb-e7858a831fa3}
        bNumControl             4
        bNrPins                 1
        baSourceID( 0)          5
        bControlSize            2
        bmControls( 0)       0x0f
        bmControls( 1)       0x00
        iExtension              0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0008  1x 8 bytes
        bInterval              16
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        1
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass        14 Video
      bInterfaceSubClass      2 Video Streaming
      bInterfaceProtocol      0
      iInterface              0
      VideoStreaming Interface Descriptor:
        bLength                            14
        bDescriptorType                    36
        bDescriptorSubtype                  1 (INPUT_HEADER)
        bNumFormats                         1
        wTotalLength                       79
        bEndPointAddress                  130
        bmInfo                              0
        bTerminalLink                       3
        bStillCaptureMethod                 0
        bTriggerSupport                     0
        bTriggerUsage                       0
        bControlSize                        1
        bmaControls( 0)                    27
      VideoStreaming Interface Descriptor:
        bLength                            27
        bDescriptorType                    36
        bDescriptorSubtype                  4 (FORMAT_UNCOMPRESSED)
        bFormatIndex                        1
        bNumFrameDescriptors                1
        guidFormat                            {59555932-0000-1000-8000-00aa00389b71}
        bBitsPerPixel                      16
        bDefaultFrameIndex                  1
        bAspectRatioX                       0
        bAspectRatioY                       0
        bmInterlaceFlags                 0x00
          Interlaced stream or variable: No
          Fields per frame: 2 fields
          Field 1 first: No
          Field pattern: Field 1 only
          bCopyProtect                      0
      VideoStreaming Interface Descriptor:
        bLength                            38
        bDescriptorType                    36
        bDescriptorSubtype                  5 (FRAME_UNCOMPRESSED)
        bFrameIndex                         1
        bmCapabilities                   0x03
          Still image supported
          Fixed frame-rate
        wWidth                            320
        wHeight                           240
        dwMinBitRate                   912384
        dwMaxBitRate                   912384
        dwMaxVideoFrameBufferSize      153600
        dwDefaultFrameInterval        2666664
        bFrameIntervalType                  3
        dwFrameInterval( 0)            666666
        dwFrameInterval( 1)           1333332
        dwFrameInterval( 2)           2666664
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x82  EP 2 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        2
      bAlternateSetting       0
      bNumEndpoints           2
      bInterfaceClass         8 Mass Storage
      bInterfaceSubClass      6 SCSI
      bInterfaceProtocol     80 Bulk-Only
      iInterface              0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x83  EP 3 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x04  EP 4 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               0
Device Qualifier (for other device speed):
  bLength                10
  bDescriptorType         6
  bcdUSB               2.00
  bDeviceClass          239 Miscellaneous Device
  bDeviceSubClass         2 ?
  bDeviceProtocol         1 Interface Association
  bMaxPacketSize0        64
  bNumConfigurations      1
Device Status:     0x0000
  (Bus Powered)

Quote from: tmbinc on March 10, 2015, 07:56:12 pm
A few hundred lines of code later (simply record all frames with a max temp of higher than some threshold, re-mapping to 8-bit, then encoding as mp4)

I'm very interested in your Fuchsjagd code snippets  :)
« Last Edit: August 12, 2015, 10:56:40 pm by tomas123 »
Logged
 

Offline OrBy

  • Regular Contributor
  • *
  • Posts: 220
Re: Flir Ex: Realtime raw radiometric data streaming via UVC
« Reply #8 on: August 13, 2015, 03:27:23 am »
Since mine is 2.3.0 I spun up a Ubuntu 14.04.3 VM (First time every trying my hand at compiling anything) and got as far at the line listed above before I for the following error:

Code: [Select]
orby@ubuntu:~/libuvc/build$ /usr/bin/cc  -o flir ../src/flir.c -luvc
../src/flir.c: In function ‘cb’:
../src/flir.c:30:3: warning: format ‘%d’ expects argument of type ‘int’, but argument 2 has type ‘__time_t’ [-Wformat=]
   printf("got frame %d:%d, %d bytes\n", tv.tv_sec, tv.tv_usec, frame->data_bytes);
   ^
../src/flir.c:30:3: warning: format ‘%d’ expects argument of type ‘int’, but argument 3 has type ‘__suseconds_t’ [-Wformat=]
../src/flir.c:30:3: warning: format ‘%d’ expects argument of type ‘int’, but argument 4 has type ‘size_t’ [-Wformat=]

Everything seemed to go word for word as outlined in tomas123 detailed instructions till that point.

I have no issues saying that I am in way over my head so unless anyone has any suggestions I don't think I can test further.

I would love to get a libuvc with this patch with this working on windows but my limited looking into it seems to indicate that the USB driver wrangling situation to accomplish this would be even more complex.

Logged
 

Offline Bud

  • Super Contributor
  • ***
  • Posts: 7145
  • Country: ca
Re: Flir Ex: Realtime raw radiometric data streaming via UVC
« Reply #9 on: August 13, 2015, 05:27:44 am »
Quote from: tomas123 on August 12, 2015, 06:31:02 pm
the result is the known screen image and not a RAW image:(

Same here, with FW 2.3.0

Code: [Select]
root@Ubuntu32:/home/some/Temp/libuvc# ./myflir test6.raw
UVC initialized
Device found
Device opened
DEVICE CONFIGURATION (09cb:1007/[none]) ---
Status: idle
VideoControl:
bcdUVC: 0x0100
VideoStreaming(1):
bEndpointAddress: 130
Formats:
UncompressedFormat(1)
  bits per pixel: 16
  GUID: 5955593200001000800000aa00389b71 (YUY2)
  default frame: 1
  aspect ratio: 0x0
  interlace flags: 00
  copy protect: 00
FrameDescriptor(1)
  capabilities: 03
  size: 320x240
  bit rate: 912384-912384
  max frame size: 153600
  default interval: 1/3
  interval[0]: 1/15
  interval[1]: 1/7
  interval[2]: 1/3
END DEVICE CONFIGURATION
bmHint: 0000
bFormatIndex: 1
bFrameIndex: 1
dwFrameInterval: 666666
wKeyFrameRate: 0
wPFrameRate: 0
wCompQuality: 0
wCompWindowSize: 0
wDelay: 1
dwMaxVideoFrameSize: 153600
dwMaxPayloadTransferSize: 157696
bInterfaceNumber: 1
Streaming...
got frame 1439442632:196459, 153600 bytes
got frame 1439442632:290426, 153600 bytes
got frame 1439442632:384196, 153600 bytes
got frame 1439442632:478789, 153600 bytes
got frame 1439442632:573349, 153600 bytes

Done streaming.
Device closed
UVC exited


An extracted frame then is just a screen capture.
Logged
Facebook-free life and Rigol-free shack.
 

Offline tomas123

  • Frequent Contributor
  • **
  • Posts: 832
  • Country: de
Re: Flir Ex: Realtime raw radiometric data streaming via UVC
« Reply #10 on: August 13, 2015, 09:24:48 am »
Quote from: OrBy on August 13, 2015, 03:27:23 am
Since mine is 2.3.0 I spun up a Ubuntu 14.04.3 VM (First time every trying my hand at compiling anything) and got as far at the line listed above before I for the following error:

Code: [Select]
orby@ubuntu:~/libuvc/build$ /usr/bin/cc  -o flir ../src/flir.c -luvc
../src/flir.c: In function ‘cb’:
../src/flir.c:30:3: warning: format ‘%d’ expects argument of type ‘int’, but argument 2 has type ‘__time_t’ [-Wformat=]
   printf("got frame %d:%d, %d bytes\n", tv.tv_sec, tv.tv_usec, frame->data_bytes);
   ^
../src/flir.c:30:3: warning: format ‘%d’ expects argument of type ‘int’, but argument 3 has type ‘__suseconds_t’ [-Wformat=]
../src/flir.c:30:3: warning: format ‘%d’ expects argument of type ‘int’, but argument 4 has type ‘size_t’ [-Wformat=]

Everything seemed to go word for word as outlined in tomas123 detailed instructions till that point.

I have no issues saying that I am in way over my head so unless anyone has any suggestions I don't think I can test further.

I would love to get a libuvc with this patch with this working on windows but my limited looking into it seems to indicate that the USB driver wrangling situation to accomplish this would be even more complex.

This is only a compiler warning, not an error. I got the excact same warning.
Simple use the successful compiled flir binary.
« Last Edit: August 13, 2015, 09:27:23 am by tomas123 »
Logged
 

Offline tmbincTopic starter

  • Frequent Contributor
  • **
  • Posts: 253
Re: Flir Ex: Realtime raw radiometric data streaming via UVC
« Reply #11 on: August 13, 2015, 09:39:12 am »
I don't have my E4 here right now. I'll investigate. Sorry for wasting everyone's time. (It definitely worked for me, as you can see in the youtube video)
Logged
 

Offline tomas123

  • Frequent Contributor
  • **
  • Posts: 832
  • Country: de
Re: Flir Ex: Realtime raw radiometric data streaming via UVC
« Reply #12 on: August 13, 2015, 09:56:34 am »
Sorry, but I'm don't understand the "bFormatIndex - 1" hack.
Can you give us some more informations about the patch?

Which stream from the lsusb tree (see above) we are grabbing?

Thanks for help!

Code: [Select]
diff --git a/src/stream.c b/src/stream.c
index 571be16..aabb6ff 100644
--- a/src/stream.c
+++ b/src/stream.c
@@ -183,6 +183,7 @@ uvc_error_t uvc_query_stream_ctrl(
   /* prepare for a SET transfer */
   if (req == UVC_SET_CUR) {
     SHORT_TO_SW(ctrl->bmHint, buf);
+    ctrl->bFormatIndex = 2;
     buf[2] = ctrl->bFormatIndex;
     buf[3] = ctrl->bFrameIndex;
     INT_TO_DW(ctrl->dwFrameInterval, buf + 4);
@@ -222,6 +223,7 @@ uvc_error_t uvc_query_stream_ctrl(
   if (req != UVC_SET_CUR) {
     ctrl->bmHint = SW_TO_SHORT(buf);
     ctrl->bFormatIndex = buf[2];
+    ctrl->bFormatIndex = 1;
     ctrl->bFrameIndex = buf[3];
     ctrl->dwFrameInterval = DW_TO_INT(buf + 4);
     ctrl->wKeyFrameRate = SW_TO_SHORT(buf + 8);
@@ -828,7 +830,7 @@ uvc_error_t uvc_stream_start(
   strmh->pts = 0;
   strmh->last_scr = 0;
 
-  frame_desc = uvc_find_frame_desc_stream(strmh, ctrl->bFormatIndex, ctrl->bFrameIndex);
+  frame_desc = uvc_find_frame_desc_stream(strmh, ctrl->bFormatIndex  - 1, ctrl->bFrameIndex);
   if (!frame_desc) {
     ret = UVC_ERROR_INVALID_PARAM;
     goto fail;
@@ -1037,7 +1039,7 @@ void _uvc_populate_frame(uvc_stream_handle_t *strmh) {
    * is going to be reopen_on_change anyway
    */
 
-  frame_desc = uvc_find_frame_desc(strmh->devh, strmh->cur_ctrl.bFormatIndex,
+  frame_desc = uvc_find_frame_desc(strmh->devh, strmh->cur_ctrl.bFormatIndex - 1,
     strmh->cur_ctrl.bFrameIndex);
 
   frame->frame_format = strmh->frame_format;
« Last Edit: August 13, 2015, 10:10:25 am by tomas123 »
Logged
 

Offline tomas123

  • Frequent Contributor
  • **
  • Posts: 832
  • Country: de
Re: Flir Ex: Realtime raw radiometric data streaming via UVC
« Reply #13 on: August 13, 2015, 10:09:17 am »
I think, the RAW Video of E40 is
Code: [Select]
bFormatIndex= 2
dwMaxVideoFrameBufferSize = 157440
but the flir.c log:
Code: [Select]
bFormatIndex: 1
bFrameIndex: 1
dwMaxVideoFrameSize: 157440
...very magic

compress output from lsusb -v

E40

      VideoStreaming Interface Descriptor:
        bFormatIndex                        1
        bBitsPerPixel                      16
        wWidth                            320
        wHeight                           240
        dwMaxVideoFrameBufferSize      153600 // = 320x240x2 Byte ( Screen Shot Video)

      
     VideoStreaming Interface Descriptor:
        bFormatIndex                        2
        bBitsPerPixel                      16
        wWidth                            320
        wHeight                           246    // 6 extra lines for the FFF Header (see exiftool output above)
        dwMaxVideoFrameBufferSize      157440 // 157440 is same size as the output from the RAW Video grab tool
 
 

 E4
 
      VideoStreaming Interface Descriptor:
        bFormatIndex                        1
        bBitsPerPixel                      16
        wWidth                            320
        wHeight                           240
        dwMaxVideoFrameBufferSize      153600 // = 320x240x2 Byte ( Screen Shot Video)
« Last Edit: August 13, 2015, 12:33:56 pm by tomas123 »
Logged
 

Offline OrBy

  • Regular Contributor
  • *
  • Posts: 220
Re: Flir Ex: Realtime raw radiometric data streaming via UVC
« Reply #14 on: August 13, 2015, 01:19:35 pm »
Quote from: tomas123 on August 13, 2015, 09:24:48 am
This is only a compiler warning, not an error. I got the excact same warning.
Simple use the successful compiled flir binary.

I will spin that VM back up and test it out when I get off work later today then.

Quote from: tmbinc on August 13, 2015, 09:39:12 am
I don't have my E4 here right now. I'll investigate. Sorry for wasting everyone's time. (It definitely worked for me, as you can see in the youtube video)

I don't think anyone thinks you have wasted their time. (I don't) There could be different configurations or variables that could be at play here.

What USB mode is your camera in when capturing? (just UVC or one of the mixed modes?)
Logged
 

Offline tomas123

  • Frequent Contributor
  • **
  • Posts: 832
  • Country: de
Re: Flir Ex: Realtime raw radiometric data streaming via UVC
« Reply #15 on: August 13, 2015, 09:48:57 pm »
Quote from: OrBy on August 13, 2015, 01:19:35 pm
What USB mode is your camera in when capturing? (just UVC or one of the mixed modes?)
I checked all UVC variants without success.
Logged
 

Offline OrBy

  • Regular Contributor
  • *
  • Posts: 220
Re: Flir Ex: Realtime raw radiometric data streaming via UVC
« Reply #16 on: August 13, 2015, 11:23:49 pm »
Just took it for a little spin but getting some odd results:

Code: [Select]
orby@ubuntu:~$ sudo libuvc/build/flir e4-1.raw
UVC initialized
Device found
Device opened
DEVICE CONFIGURATION (09cb:1007/[none]) ---
Status: idle
VideoControl:
bcdUVC: 0x0100
VideoStreaming(1):
bEndpointAddress: 130
Formats:
UncompressedFormat(1)
  bits per pixel: 16
  GUID: 5955593200001000800000aa00389b71 (YUY2)
  default frame: 1
  aspect ratio: 0x0
  interlace flags: 00
  copy protect: 00
FrameDescriptor(1)
  capabilities: 03
  size: 320x240
  bit rate: 912384-912384
  max frame size: 153600
  default interval: 1/3
  interval[0]: 1/15
  interval[1]: 1/7
  interval[2]: 1/3
END DEVICE CONFIGURATION
bmHint: 0000
bFormatIndex: 1
bFrameIndex: 1
dwFrameInterval: 666666
wKeyFrameRate: 0
wPFrameRate: 0
wCompQuality: 0
wCompWindowSize: 0
wDelay: 1
dwMaxVideoFrameSize: 153600
dwMaxPayloadTransferSize: 157696
bInterfaceNumber: 1
Streaming...
got frame 1439507727:121939, 153600 bytes
got frame 1439507727:122047, 153600 bytes
orby@ubuntu:~$ split -b 153600 e4-1.raw
orby@ubuntu:~$ convert -size 320x240  yuv:xaa -separate 4.png
convert.im6: unexpected end-of-file `xaa': No such file or directory @ error/yuv.c/ReadYUVImage/428.

It seems like it may be the raw feed at least for me.
Attached are the 2 png's it spat out and the zip has the raw and a xaa file it in it.

Logged
 

Offline Bud

  • Super Contributor
  • ***
  • Posts: 7145
  • Country: ca
Re: Flir Ex: Realtime raw radiometric data streaming via UVC
« Reply #17 on: August 14, 2015, 03:20:37 am »
Quote from: tmbinc on August 12, 2015, 09:21:31 pm
Meh. Okay.

I'm on 2.3, so maybe it was added later. Could someone with a Flir Ex with 2.3 firmware give it a try? I wouldn't know what else it could be.

Can you share your patched stream.c ?
Logged
Facebook-free life and Rigol-free shack.
 

Offline Bud

  • Super Contributor
  • ***
  • Posts: 7145
  • Country: ca
Re: Flir Ex: Realtime raw radiometric data streaming via UVC
« Reply #18 on: August 14, 2015, 03:24:34 am »
@OrBy

I am getting same error and same result (Ubuntu 11.10 on a VM). But the png produced is not radiometric. Yours does not seem to be it either, it does not contain radiometric metadata.
Logged
Facebook-free life and Rigol-free shack.
 

Offline tomas123

  • Frequent Contributor
  • **
  • Posts: 832
  • Country: de
Re: Flir Ex: Realtime raw radiometric data streaming via UVC
« Reply #19 on: August 14, 2015, 08:03:25 am »
@Orby

your png-image is radiometric with swapped bytes
see my tutorial here: https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg348398/#msg348398

Code: [Select]
>identify 4-0.png
4-0.png PNG 320x240 320x240+0+0 16-bit sRGB 67.3KB 0.000u 0:00.001
>convert -define png:swap-bytes=on 4-0.png -auto-level raw.png
better use the splitted part
Code: [Select]
convert -depth 16 -size 320x240 -auto-level gray:xaa raw.png
@Orby + @Bud
You have both the firmware 2.3.0.
Find the difference between the configuration  ;)

check "lsusb -v"

our logs from flir.c are 100% identical :-[
Code: [Select]
UVC initialized
Device found
Device opened
DEVICE CONFIGURATION (09cb:1007/[none]) ---
Status: idle
VideoControl:
bcdUVC: 0x0100
VideoStreaming(1):
bEndpointAddress: 130
Formats:
UncompressedFormat(1)
  bits per pixel: 16
  GUID: 5955593200001000800000aa00389b71 (YUY2)
  default frame: 1
  aspect ratio: 0x0
  interlace flags: 00
  copy protect: 00
FrameDescriptor(1)
  capabilities: 03
  size: 320x240
  bit rate: 912384-912384
  max frame size: 153600
  default interval: 1/3
  interval[0]: 1/15
  interval[1]: 1/7
  interval[2]: 1/3
END DEVICE CONFIGURATION
bmHint: 0000
bFormatIndex: 1
bFrameIndex: 1
dwFrameInterval: 666666
wKeyFrameRate: 0
wPFrameRate: 0
wCompQuality: 0
wCompWindowSize: 0
wDelay: 1
dwMaxVideoFrameSize: 153600
dwMaxPayloadTransferSize: 157696
bInterfaceNumber: 1
Streaming...
A interesting difference to my E40 stream:
The E4 RAW stream contains no FFF Header:
Image Size: 320x240x2 Byte =  frame size =  153600 Byte

The E40 RAW Stream contains full FFF Imgages
frame size 157440 byte
see sample in my post https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg729688/#msg729688
« Last Edit: August 14, 2015, 08:44:30 am by tomas123 »
Logged
 

Offline OrBy

  • Regular Contributor
  • *
  • Posts: 220
Re: Flir Ex: Realtime raw radiometric data streaming via UVC
« Reply #20 on: August 14, 2015, 01:16:22 pm »
Quote from: tomas123 on August 14, 2015, 08:03:25 am
@Orby

your png-image is radiometric with swapped bytes
see my tutorial here: https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg348398/#msg348398

Code: [Select]
>identify 4-0.png
4-0.png PNG 320x240 320x240+0+0 16-bit sRGB 67.3KB 0.000u 0:00.001
>convert -define png:swap-bytes=on 4-0.png -auto-level raw.png
better use the splitted part
Code: [Select]
convert -depth 16 -size 320x240 -auto-level gray:xaa raw.png
@Orby + @Bud
You have both the firmware 2.3.0.
Find the difference between the configuration  ;)

check "lsusb -v"

our logs from flir.c are 100% identical :-[
Code: [Select]
UVC initialized
Device found
Device opened
DEVICE CONFIGURATION (09cb:1007/[none]) ---
Status: idle
VideoControl:
bcdUVC: 0x0100
VideoStreaming(1):
bEndpointAddress: 130
Formats:
UncompressedFormat(1)
  bits per pixel: 16
  GUID: 5955593200001000800000aa00389b71 (YUY2)
  default frame: 1
  aspect ratio: 0x0
  interlace flags: 00
  copy protect: 00
FrameDescriptor(1)
  capabilities: 03
  size: 320x240
  bit rate: 912384-912384
  max frame size: 153600
  default interval: 1/3
  interval[0]: 1/15
  interval[1]: 1/7
  interval[2]: 1/3
END DEVICE CONFIGURATION
bmHint: 0000
bFormatIndex: 1
bFrameIndex: 1
dwFrameInterval: 666666
wKeyFrameRate: 0
wPFrameRate: 0
wCompQuality: 0
wCompWindowSize: 0
wDelay: 1
dwMaxVideoFrameSize: 153600
dwMaxPayloadTransferSize: 157696
bInterfaceNumber: 1
Streaming...
A interesting difference to my E40 stream:
The E4 RAW stream contains no FFF Header:
Image Size: 320x240x2 Byte =  frame size =  153600 Byte

The E40 RAW Stream contains full FFF Imgages
frame size 157440 byte
see sample in my post https://www.eevblog.com/forum/testgear/flir-ex-realtime-raw-radiometric-data-streaming-via-uvc/msg729688/#msg729688

Yes I kinda thought that it had the look like it was backwards and that it needed just a bit more processing to look right. I ran out of time to play last night but I was going to try messing with it a little more when time permitted.

It's interesting how the E4 is missing the header info - guess they cant make it too easy!

I will post up my lsusb -v log when I get some time back on my PC.
Logged
 

Offline tomas123

  • Frequent Contributor
  • **
  • Posts: 832
  • Country: de
Re: Flir Ex: Realtime raw radiometric data streaming via UVC
« Reply #21 on: August 14, 2015, 01:26:23 pm »
The only missing header infos are the time stamps.

All other Flir values to calculate the real temperature of a pixel are (calibrated) constants.
You can get them from every radiometric jpg of the same (!) camera.

I hope tmbinc give us some explanations to the bFormatIndex-hack
and some code snippets for our own Fuchsjagd   ;)
« Last Edit: August 14, 2015, 01:32:11 pm by tomas123 »
Logged
 

Offline Bud

  • Super Contributor
  • ***
  • Posts: 7145
  • Country: ca
Re: Flir Ex: Realtime raw radiometric data streaming via UVC
« Reply #22 on: August 14, 2015, 01:31:51 pm »
By non-radiometric i mean no temperature scaling information is in the file. I do not have access to my computer at the moment but if you load the image in Flir Tools i think temperature measurements will be incorrect and all the way between -40 +300 or something. To get it right i think the image levels need to be scaled against a known reference, same as you do when creating panoramas after stitching step.
Anyway, try loading it in Flir Tools to see if it can be correctly measured.
Logged
Facebook-free life and Rigol-free shack.
 

Offline tomas123

  • Frequent Contributor
  • **
  • Posts: 832
  • Country: de
Re: Flir Ex: Realtime raw radiometric data streaming via UVC
« Reply #23 on: August 14, 2015, 01:35:34 pm »
@Bud
Currently you have not radiometric raw values in your stream.  Therefore Flir Tools can't read anything.

You need a RAW stream, like Orby E4.
Then you can inject a extracted 16 bit raw png in a radiometric jpg (from the same E4!) and the temperature measurement of Flir Tools works fine

simple use my (panorama) script splitjpg.php for injecting
https://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg348715/#msg348715

// no -auto-level  ;)
Code: [Select]
$ convert -depth 16 -size 320x240  gray:xaa raw.png
$ php splitjpg.php -i IR_0xxx.jpg -r raw.png -o sampleLoad the sample.jpg in Flir Tools
« Last Edit: August 14, 2015, 02:42:46 pm by tomas123 »
Logged
 

Offline Bud

  • Super Contributor
  • ***
  • Posts: 7145
  • Country: ca
Re: Flir Ex: Realtime raw radiometric data streaming via UVC
« Reply #24 on: August 14, 2015, 03:32:50 pm »
Yes i have already tried that with my non- radiometric streamed images and if i use your splitjpg script and a reference radiometric jpg, i get a proper thermal image in Flir Tools but temperature readings are still somewhat off, probably because the reference jpg was not from the same scene. I will try tonight without the scaling step.

Having said that, i begin to believe that may not be difference between the E4 hacked radiometric streamed images vs screen capture streamed ones, providing i turn measurements and everything else off on the camera, and also turn MSX off so the screen only displays the thermal image. I can do this on my doctored E4 2.3.0.

 I will still try to get the camera stream same way as OrBy's so i can post processes images received using the two methods and compare the end result.
Logged
Facebook-free life and Rigol-free shack.
 
Pages: [1] 2 3 4 Next  All   Go Up
« previous next »

Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf

 


EEVblog Main Site EEVblog on Youtube EEVblog on Twitter EEVblog on Facebook EEVblog on Odysee