Why another one Thermal Camera?
Aren’t there enough designs out there already? Yes…but, for starters, I like to have a wintertime project to help the time go by when it is cold and dark outside here in Northern California. Then again there is always something fun in bringing a project to life and I always learn useful things that I can put to good use in commercial projects later and finally having a thermal camera in the Lab is just the ticket to finding all the hot spots on a PCB, looking at the heat generated by electrical appliances and even tracing electrical wiring and hot water pipes in the walls of my house.
I won’t describe in detail every step on how I designed and implemented this project, but I will list the issues that I found and that I have not seen addressed anywhere else and just to share how I went about bringing the project to life.
Processor
The processor that I chose for this project is the Microchip PIC32MZ2048EFH064. This processor is fast with a core frequency of up to 252 MHz, has a built in Floating Point Math Unit (FPU), lots of memory: 2048k bytes of ROM and 512k bytes of RAM, available in a small pin count, easy to solder TQFP-64 package and is well suited for this project.
I have for years used a hand optimized bit-bang I2C driver that I wrote and this allowed me to tune the speed as required and it was easy to get 1 MHz operation for the Melexis sensor. The SPI used for the display uses the PIC32 built in SPI Peripheral.
All in all the processor uses 10x less power, has more memory and 10x the processing power of an Intel 386 chip, all at 30x less cost.
Melexis MLX90640 Temperature Sensor
This IR sensor [1] is really quite something. It may only have 32 x 24 pixel resolution but at better than 0.1 Degree C resolution it is very, very sensitive which makes for a very detailed display. The sensor works by having a set of calibration constants stored in its EEPROM that can be read out and applied to the sensor ADC readings to get at a calibrated Deg C reading. This is easy to say but, the math involved takes some 20 pages of the data sheet to describe! Thankfully Melexis provides a very easy to use C based driver on GitHub [2]. The driver dropped right into my Microchip MPLAB-X / XC32 development environment and compiled straight away with only a few minor changes, mostly to the I2C interface as I needed to conform the Melexis driver model to the way my I2C driver operates.
The driver as written uses floating point math so a fairly powerful processor is required. Also of note is the driver itself uses some 17k bytes of memory minimum to store all the calibration constants and sensor data. With the processor running at 200 MHz the calculation time of the Melexis driver data to get the temperature array was measured at around 6 milliseconds.
The interface to the sensor is via I2C and it takes a lot of time to get the entire Sensor array data out as the protocol is not very efficient. The Sensor updates it’s full data buffer fully only on every other sampling cycle – that is it operates in ‘frames’ where each frame is one-half of the Sensor arrays data points, and it takes two updates to update the entire Sensor data array. You must read the entire Sensor array of data each time, so one half of the data read on each frame is old data, this is very inefficient. The I2C interface does run at a fast 1 MHz, but even at this fast rate it takes 15 milliseconds to transfer a frame out, and twice that to get the full data array of updated values.
The sensor data acquisition time is adjustable in software with the usual trade-off: Faster Data Acquisition yields a higher noise result. I found a good compromise by setting the sensor to update rate to 4 Hz or 0.25 Seconds. The complete sensor data array update takes twice this time or 0.5 Seconds. This was about as slow as I could go and still have a decent frame update rate on the display so that it would appear to be real time to the user.
Figure 1 – A small portion of the math required to calculate the temperature as seen by the Sensor – this goes on for some 20 pages! Writing a driver like this by hand would drive anyone insane! Fortunately for us, Melexis provides a very well written driver on GitHub which takes care of all this math.
Display
I chose to use the Adafruit TFT displays with breakout boards since they are complete solutions and have very well thought out driver support which I had already used in previous projects. This led me to the Adafruit 2.8 inch display with Capacitive Touch [3]. This display has a native resolution of 320 x 280 pixels, which makes up-interpolating from the sensors 32 x 28 resolution easy, as it is an even number with the same aspect ratio as the sensor. The Adafruit driver is SPI based, and is written for Arduino and hence it is in C++, so this always requires some ‘flattening’ of the C++ classes to get it to be C compatible and to translate the SPI commands from Arduino to the PIC32 processor that I was going to use. Since the driver is intended for an Arduino type of device it is very heavily optimized for size and versatility, not speed. It is a wonderful driver however, as it is well thought out and very easy to use.
To draw a color point or color area on the display, first you need to define a window with four, 16 bit points, then fill the area with a 16 bit color number. This is simple for the user, but as will be seen, it takes some time to actually write large amounts of little color areas to the display.
I setup some test code to display various pixel resolutions on the display, starting with the Sensor pixel resolution of 32 x 24 blocks displayed, well that looks horrible as you might imagine and 320 x 240 pixels displayed looks a bit too fine and took forever to write to the display. I finally settled on a final resolution of each ‘Temperature Pixel’ being a block of 2 x 2 ‘Display Pixels’, so the final display resolution would be 160 x 120 ‘Temperature Pixels’.
I measured the display update speed on my breadboard and found that it takes a very long time to update even the 160 x 120 pixels in the display this way – over a second!
This was clearly not going to work very well as a reasonably fast update rate is needed for a handheld instrument.
The first lazy speed up thing I tried was to up the SPI clock rate from the specified 10 MHz maximum, first to 20 MHz with no issues, then 33 MHz and finally to 50 MHz also with no issues. The driver IC data sheet clearly indicates that 10 MHz is the maximum clock rate, but I found no issues at 50 MHz. I even tested this speed with other Adafruit LCD’s that I have here in the lab, specifically the Adafruit 1.8, 2.2 and 3.2 inch versions all with no issues. So I decided that this was OK, as this is a personal project and I am not going to be making more than a few of them.
Note however that it is a very, very, very, very bad practice to use any part beyond its specified maximum ratings in a commercial / production environment and I would never do that without first having a discussion with the parts manufacturer as to the ramifications of such out of specifications usage!
Now the display time for a complete update was around 226 milliseconds. Before this SPI speed improvement I was considering rewriting the SPI driver or even writing a parallel driver as the Adafruit 2.8 inch display also supports a parallel interface. However, the SPI speed improvement was enough, thus saving me a lot of development time.
Even at 226 milliseconds, the display update is still visible to the naked eye, so I decided to interlace the display update. That is, on one Sensor frame, I would update the ‘even’ rows on the display and on the next Sensor frame I would update ‘odd’ rows. This then makes the display look much more instantaneous and each display write is now on the order of 113 milliseconds and you can’t see any obvious flickering.
The Adafruit display has capacitive touch that uses a I2C interface with the well known FT6206 Touch Controller IC. This IC is easy to use and control and has an interrupt output to allow quick processor attention to any display touches.
One final note: In this application, a 2.2 inch display would have been plenty big enough, but those displays are not available with capacitive touch, so I was stuck with the bigger display as I really wanted a capacitive touch display as they always have much better image quality.
Sensor Data Processing
After all the Sensor reading and calculations, we end up with an array of 768 floats representing the temperature at each of the Sensors pixels is. The task now is to manipulate that temperature data into something useful for the display.
The first thing I did was to get the minimum and maximum temperature values of what the sensor saw in the current scene, then convert the rest of the values to fixed point values with 0.01 degree resolution. The fixed point notation easily fits into int16_t data types, doing this conversion makes all the subsequent math much faster and smaller.
For the min and max temperature values, I used a 4 frame moving average of the temperature reading, so that the display will appear more stable, and although the display will start to range immediately when pointing at a different temperature scene, the display will take about a second of looking at the same temperatures to fully stabilize on a new average. I also capture and average the center pixel so that I can display the center ‘spot’ temperature on the display.
The next processing step was to resample the data so that the final resolution is 160 x 120 temperature values for display purposes. There are many ways of doing this and many people have used Bicubic interpolation. Bicubic interpolation is the best way to up-sample a photographic picture, but I think Bicubic is not the right way to go for an application like this. The reason being that Bicubic interpolation actually overshoots the edges across areas of fast changing temperature, and while this makes a photographic picture look better, in a thermal camera it actually makes the edges of a hot object appear hotter than they are. So I chose to implement the simpler Bilinear interpolation method. You can find many fine tutorials on the web for how to do Bilinear interpolation, but basically you first interpolate the rows between the known Sensor data points, then once this data is found you can interpolate the columns between the now interpolated rows.
I chose to hard code the up-sampling algorithm to maximize speed and coding time. I know my input size is 32 x 24 and my output size is always 160 x 120. Since the interpolation is working with an array of int16_t values instead of 4 byte floats, the resulting routine is fast and small.
After up-sampling the data array the values are then mapped into a 0-255 uint8_t array. The basic step is a Histogram Expansion where all the temperature values from min to max are expanded or compacted to fit in a 0-255 value array of uint8_t’s.
At this point we no longer care about the absolute temperature value of any pixel because all the pixel temperature values have been forced into an an array of uint8_t’s between the min and max temperature values of the current scene. This makes sure that the full range of color values will be used for any scene that is viewed. The expansion can be thought of as a display automatic gain control and since I averaged the min and max temperatures of the hottest and coldest pixel in any scene over an approximately 1 second time span, this then sets the attack and decay times of the AGC.
This is a good spot in the code to do any further contrast or brightness adjustments, these are user adjustments that the operator can make through interaction with the touch screen interface as will be described later.
The final step is to map the resulting array of unit8_t’s into a color map of 255, 16 bit color values. I found an excellent resource of constant luminance color maps [4] and chose three for this project, a ‘Hot Iron’ color map, because I like the way these look as they resemble the color of Iron as the temperature raises, the classic ‘Rainbow’ color map as this is just the classic style and a ‘Black and White’ color map, which looks like the IR images that you see on thermal surveillance cameras. Each color map can be selected and changed on the fly at runtime. These color maps were encoded into 256, RGB565 values and put into ENUM’s and then it is a simple matter to look up the color number from the ENUM when displaying the specific pixel color.
While displaying each pixels color temperature, on the right edge of the display I put the maximum, center point and minimum temperature average with a 1 decimal point resolution. Since these values are the average of 4 frames of data, the values don’t flicker much and are very readable.
To minimize the flicker of the text on the display, on each display interlace, when the row has just passed the now overwritten and cleared text, I go and rewrite the text in that position, then draw more rows and when just under the next text position, go and rewrite that value. This was accomplished with a row counter and values hardcoded to write the text back on the screen when that row is reached.
Some thermal cameras display a movable box or spot indicator that follows the hottest object in the scene. I find that this isn’t really the way I use a thermal camera, I don’t always want to measure the hottest thing, most of the time I want to selectively measure certain items in the scene that aren’t always the hottest or coolest objects. With the color display it is easy to pick out the object to measure and then to put the ‘spot’ over it for a temperature readout. I opted for a ‘spot temperature’ reading from the center of the scene, and I denote the spot by drawing a small white box at the location of the center pixel.
Touchscreen Interface
A few adjustments are useful in real time. Since there is an effective Automatic Gain Control happening in real time because I am constantly averaging the minimum and maximum temperature in a scene and expanding or compressing the rest of the data in this range this takes care of most scenes automatically, but there are times where the ‘Brightness’ needs to be increased or decreased. This is accomplished buy swiping up and down on the left side of the screen like a volume control, except instead of controlling the ‘volume’ the offset in values is changed effectively changing the displayed image brightness. This can sometimes help to bring out desired detail in really hot or cold scenes.
The brightness, if you get too far off in adjustment can be set back to ‘normal’ by tapping the center of the screen and it will revert to the no adjustment value.
The other adjustment is a ‘Contrast’ control, many scenes aren’t evenly distributed in temperature and sometimes you want extra detail in the hot or cold portion of the scene. As shown in Figure 3 the contrast is just a math mapping function that the data is run through. I found that three fixed contrasts were easy to implement and really do a good job of enhancing the lower or upper temperature ranges.
Hardware Design Details
All of the main components run on 3.3 Volts, which was supplied by a Texas Instrruments TPS77301 Low Dropout Regulator, this regulator has the added feature of having a delayed ‘Power Good’ signal which is ideal for use as a Microprocessor Power On Reset. The Melexis Sensor has its own dedicated, low noise 3.3 Volt power regulator since it has almost no power supply rejection ratio and any low frequency noise can and will show up as ‘pattern noise’ on the sensor output data and subsequently the display. To reduce the chance of any pattern noise showing up, the Thermal Sensors I2C bus pullup resistors are tied to the Microprocessors +3.3 Volt supply as an extra precaution to keep any digital I2C bus noise from contaminating the sensor voltage rail.
Common practice is to power devices like this from a rechargeable Li-Ion battery, but I find that in instruments like this that I don’t use daily and perhaps not even every month, invariably when I do want to use the Camera it has been sitting around long enough so that the battery is dead and needs recharging. I want an ‘Instant On’ kind of a device for Lab use and if the battery is dead, I would rather change it to a fresh one right away so that I can keep working instead of waiting even 20 minutes to get the minimum charge back into the device.
The PCB is a simple two layer design. The wonderful thing about modern 32 Bit processors is that they need very little in the way of extra support circuitry. Altium Designer was used to layout the circuit and render the 3D model of the board. This model was then imported directly to Design Spark Mechanical so that the housing could be designed around it.
The simplest power approach then is to use a ubiquitous 9V Alkaline cell, they are readily available anywhere, have around 300 to 550 mAH of capacity and long shelf life. Running the 9 volts directly into the 3.3 volt regulators would be extremely power inefficient, so a small premade buck regulator PCB was purchased in a 10 pack off eBay for $5.00. These little boards work fine and have a trimpot adjustable output voltage. I set the Buck Regulator output to 3.7 Volts, then when the 9V battery is attached the current drain is much less and the overall efficiency of the design is markedly increased, lowering the generated heat and increasing the battery runtime immensely. A 9 Volt battery has a discharge curve over it’s life from 9 Volts down to 4 Volts and this works out well with the overall power system design. If the battery is drained to 4 Volts or below there won’t be enough ‘Oomph’ left to start the circuit and the camera will fail to start when the ON/OFF button is pressed. That is what I call: “Positive indication of a dead battery”, and no circuitry or software was required to implement this feature!
The actual current drain of the completed design was measured as,
@9V Input = 90 mA
@4V Input = 205 mA
This current draw gives about 2 to 4 hours life out of a good consumer 9V Alkaline battery and the efficiency keeps the PCB temperature rise very low.
The majority of the current drain is attributed to the LCD Display backlight first, and then the PIC32 second. The backlight could be PWM modulated for lower brightness but this isn’t a super bright display anyway, so this was deemed not worthwhile. The display is fine inside and outside in shade, but not bright enough to be used in direct sunlight.
Upstream of the power supplies I used one of those very clever Push Button / Power and Function IC’s made by Linear Technology as a main power switch. The LTC2955 Push Button Controller operates by watching the state of a push button. When the push button is pressed, the IC causes a small PMOS FET to turn on and supply power to the downstream circuits. The IC also has an adjustable power off timer, so that when the push button is held for a certain period of time the IC will switch the PMOS FET off. I set this delay to about 2.5 seconds. The operation is then: Quick Push on the power button and the camera turn on, hold the button for 2.5 Seconds and the Camera turns off. The IC also has a ‘KILL’ input that when pulled low will turn the power off. I tied this to an open drain output on the Microprocessor. The Microprocessor can then remember the last time the user interacted with any control input and if the unit has not been touched in sufficient time the Camera will turn itself off. This delay is adjustable in software and I set the delay to about 3 minutes.
While in operation, if the push button is pressed for a short time interval an ‘Interrupt’ pin will go low indicating to the Microprocessor that a ‘Push’ happened. I read this pin on the Microprocessor and use this interrupt to cycle through the various color maps that are programmed in the Camera, so that, when in operation then, pressing the power button for a short interval will cycle the color map on the display.
Emissivity
If you have ever used a Thermal Camera before you will have quickly learned that ‘Shinny’ objects don’t read the proper temperature and actually reflect the temperature of other objects around them like a mirror. Even objects at the same temperature with different surface finishes will read different temperatures. With Electronics work, most of the interesting parts on a PCB, like the IC’s have the same black matte finish, so they mostly at least read relative temperatures OK.
This apparent temperature reading difference due to surface finish is call the ‘Emissivity’ of the object and you can find a list of common calibration factors for different materials in many textbooks. Usually a Thermal Camera will have some menu arrangement to set the Emissivity of the object being measured and a calculation is done to correct for that, but in practice I have never used those functions. If I have ever wanted to get really good relative temperatures, I have just spray painted the entire assembly with flat black spray paint and be done with it. Hence I have not implemented any such Emissivity correction in this design.
It is good to remember that IR measurements are subject to a host of possible errors besides just the objects Emissivity and that the real power is to be able to see the relative temperatures of a ‘scene’ all at once with a picture like image. If more precise measurements are needed then thermocouples should be attached to the objects in question as this is much more accurate than any IR approach. It should also be noted that if you are in need of a higher resolution IR Camera that has NIST Calibration traceability, then a commercial unit is what you should buy. They aren't expensive and even cost less than a good Laptop now. Keysight makes a number of TrueIR cameras that are nice and small [5].
Mechanics
Once the housing was roughed out, a simple 2 layer PCB was designed to fit, and then the housing was re-iterated so that everything finally came together. The PCB and display are electrically attached together with a 1.5 inch long, premade flexible jumper cable. The Thermal Sensor is mounted on the backside of the PCB so that it faces out the backside of the unit on the opposite side of the display. The top panel was milled from 62 mil thick aluminum, then engraved with the lettering and ink wiped for contrast. The top panel has the screw mounts for the display, and is glued to the top of the housing with a thin layer of Silicone adhesive. The PCB screws to the main body and finally the housing bottom screws into the main housing body making the assembly complete.
Figure 5 – The component parts were modeled and designed with Design Spark Mechanical software and processed for final 3D printing. Four separate 3D printed parts go into the making of the plastic housing.
Figure 6 - Detailed interior view showing how the Display, PCB and Battery fit in the design.
Application
Excellent dynamic range is achieved as this picture of measuring the Gas Flame from a Kitchen stove demonstrates. Here the display spans 28 to 285 Degrees C.
An IR Camera is useful around the lab too. Here the insides of my Keysight 34401 DMM are measured, showing two warm areas (left image). Getting a closer look at the warm area in the upper right of the DMM reveals the temperature of two voltage regulators in detail (right image).
The first thing I do after applying power to a new PCB is to look for smoke, when that test passes, the second thing I do is to reach for the IR Camera and look at the PCB for hot spots. Here an ordinary looking, and properly functioning PCB shows that the hottest item on the board is the voltage regulator at the lower center of the board between the two Tantalum Capacitors.The regulators temperature is approximately 46.8 Degrees.
Downloads
The project files can be downloaded from GitHub at the following location,
https://github.com/Hagtronics/YetAnotherThermalCamera
References
[1] https://www.melexis.com/en/product/MLX90640/Far-Infrared-Thermal-Sensor-Array
[2] https://github.com/melexis/mlx90640-library
[3] Adaftuit display 2.8" TFT LCD with Cap Touch, PRODUCT ID: 2090
https://www.adafruit.com/product/2090
[4] https://peterkovesi.com/projects/colourmaps/
[5] Keysight TrueIR product page,
https://www.keysight.com/en/pc-2386399/trueir-series-thermal-imagers
[6] https://www.rs-online.com/designspark/mechanical-software
Article By: Steve Hageman / www.AnalogHome.com
We design custom: Analog, RF and Embedded systems for a wide variety of industrial and commercial clients. Please feel free to contact us if we can help on your next project.
Note: This Blog does not use cookies (other than the edible ones).
Great post! I found the information very helpful. Thanks for sharing!
ReplyDelete