Hi,
hi Dave, hi mike, your channels are awesome!!
This seems to be a thread about mikes last FLIR E4 teardown - bet the folks at FLIR HQ are now really fearfully subscribing :-)
I had some thoughts yesterday:
Well, I wonder wether FLIR would sell their sensor assembly as replacement part - and if they do, how much it would cost. Getting a germanium lens from somewhere shouldn't be that much of a problem...
I have recently done some FPGA projects in VHDL at University and I bet its not very difficult to build something simple and cheap on a Spartan 6 to translate the sensor output to a visible image. VGA on an FPGA is more or less trivial and all the image stuff cant be that hard! Its just 320x240 @ 60 fps... Thats not particularily much data - especially since todays FPGAs contain enough S-Ram to allow for a full frame Buffer.
VGA Output
Say we have 256 colors on our output - this is one byte per pixel. One Frame contains 76k Pixels. So we need 76 kilobyte of Ram that is fast enough just for the output buffer.
Now lets see the input:
We have the AD on the chip which is providing around 16 bits of data per Pixel. So we get double the output data as raw in from the sensor, which is 150 kilobyte of raw AD-output per frame. The fact that its in two lanes which somehow each only cover 160 of the 320 total lines is interesting too, but not a big problem.
So our fpga needs at least around 300 kilobytes of Sram or dual port ram.
Now lets see how fast we need this ram to be:
We get 320*240*60 pixels per second which is a pixel clock of 5 megahertz. Thats a needed ram access time of 200 nanoseconds if the fpga has got a ram bus width of at least our needed 16 bits - no problem for sram which goes down to 50 nanoseconds without any problems...
Ok, now lets see how i would build our machine. Please corret me if i am wrong on something:
First we pick an FPGA with a little more than double the required ram space to make things a bit easier.
First we divide our ram into pieces:
ri1: 300 kb of raw input frame buffer 1
ri2: 300 kb or raw input frame buffer 2
ro1: 75 kb for our output frame buffer 1
ro2: 75 kb for our output frame buffer 2
sh0: 300 kb of raw input frame buffer for shutter images
We use three general purpose io pins on our fpga - one to sync on the clock and two for the two data lines which come out of the sensor.
We get our data from the sensor in words, which are grouped into lines, which are again grouped into half frames.
We need to load each word into a little buffer made of flip-flops. When a buffer is full, we write it out to ram and fill a second buffer while the first one writing out. We always use one word bufer to fill it with ad data while we push the others contents to ram. This allows us for pushing the data out into ram continuously and letting it sit there for a bit.
When ri1 is full after the sensor transmitted a frame and we pused it into our ram buffer pixel by pixel, we continue to fill ri2 while we work with the data in ri1. When we are finished with working with the data in ri1 (translating it into a output frame), we can start to overwrite it with new data, while now working with ri2 and so on...
fill ri1
work with ri1 and fill ri2
work with ri2 and fill ri1 (overwerwrite ri1)
work with ri1 and fill ri2 again (overwrite)....
We have always got one input ram to work with (translate) and one to write into.
Now lets see how we work with data:
Our goal is to produce a useful output image for vga which has some sort of intensity information for each pixel. We said we use 256 levels of intensity for the output (colors or just whatever you want).
Lets use the same technique as with the raw input buffer rams.
we alwas fill one output frame buffer with data from one input frame buffer (by translating pixel by pixel). The other output frame buffer is being red by our VGA code at the same time (for screen display or whatever) and the other input buffer is being filled with raw data from the sensor a the same time. But we dont care about this until we are finished with translating the frame and have to switch buffers.
Read from ri1 (while our raw input parser writes into ri2) -> translate -> write into ro1 (while our output core shows pixels in ro2 to the screen)
Read from ri2 (while our raw input parser writes into ri1) -> translate -> write into ro2 (while our output core shows pixels in ro1 to the screen)
and so on...
We have a continuous flow of data through our machine.
Now lets look at our translation:
Each pixel in the current read-scheduled input buffer (the one which is not being overwritten with sensor data right now) has an according pixel in our write scheduled output buffer (the one which is not being red out to the screen by our VGA core right now).
Of course we could just scale down our 16 bit value to 8 bit and display it. But this would not result in a good image because our eyes cant see this fine kinds of gradients and our displays probably cant display them.
The information in an average input pixel contains some noise (from surrounding temperature,...) which is completely meaningless, then some valuable data which tells us important stuff about the objects we are looking at and then some range which never gets triggered, because we dont have a hot enough objects in the scene. We need to find this narrow band of meaningful variation in the values of your images.
So our 16 bit values of decimal meanings between 0 and 65k each contain information of differnt kinds:
Fact: value is >= 65k - never triggered, tells us nothing
Fact: value is >= 60k - never triggered, tells us nothing
Fact: value is >= 55k - never triggered, tells us nothing
Fact: value is >= 50k - never triggered, tells us nothing
Fact: value is >= 45k - never triggered, tells us nothing
Fact: value is >= 40k - valuable info
Fact: value is >= 35k - valuable info
Fact: value is >= 30k - valuable info
Fact: value is >= 25k - valuable info
Fact: value is >= 20k - almost certainly due to noise, tells us nothing
Fact: value is >= 15k - almost certainly due to noise, tells us nothing
Fact: value is >= 10k - almost certainly due to noise, tells us nothing
Fact: value is >= 5k - almost certainly due to noise, tells us nothing
Fact: value is >= 0 - almost certainly due to noise, tells us nothing
Lets find out what the range carrying our valuable info is (in this case the variation of the values between 25k and 40k, since almost all pixels are bigger than 20k, and smaller than 45k. The fact that they are bigger than 25k and smaller than 45k is predicatable and thus carries no information). We need to find the low and high treasholds of valuable information for each frame or pixel.
The upper treshhold is no problem: We just reserve some flip flops for values of some statistics work in the fpga and look at the data to find out, what is the maximum of the last few frames for most of the pixels. This kinds of calculations are wehre fpgas get into their really strong side, so no problem for us.
Finding the noise is no problem too: We just take a reference picture while our shutter is in front of the sensor and wirte it into sh0, since the variation in noise is big from pixel to pixel. Maybe a general treshold for a whole frame would be sufficient too, but i am not sure about this and ram is so cehap, that we can afford it without a problem.
The rest is simple. We pull a pixel from our input buffer and subtract the noise which we know from sh0.
Then we adjust our scaling from 65k to 256 by using our upper treshold and scale.
This will give us a good image. Not an awesomely perfect image, but a good one and 60Hz and 320x240.
We can add stuff like a thermister for absolute readings and more stuff, if we want of course.
The question is now, how much FLIR charges for a replacement sensor assembly and lens. An FPGA board for easy use is probably ceaper than 100 USD and when we have the code once, we can upload it to Github and share :-) ?
Of course you can also just use half the ram and not my approach of always saving one part of it for writing and one for reading, but the cool thing is, that this design will later be easily upgradable for a shutter wheel that constantly calibrates the sensor instead of an intruding calibration phase which stopps your image. The price of using a shutter wheel will probably be dropping the usable frame rate from 60 to 30 hertz... But yeah, still better than stopping the picutre for more than a second! Plus: You can easily add the visible light enhancements if you start out with a design which simplifies timing and racing conditions at the price of ram.
So: Sorry for the wall of text and maybe partially difficult to understand post... Just wanted to share some thoughts - who knows, maybe this helps someone to get a few ideas?
Any thoughts?
Any obvious missassumptions on my side?
Has anyone here ever bought replacement parts from FLIR?
Maybe I should give them a call and ask for replacement sensors?
Home
Help
Search
Login
Register
