Embedded Computing
  • About
  • Blog
  • Hardware
    • Which Platform?
    • Controller Platforms >
      • Adafruit Platform
      • Arduino Plaform
      • BBC micro:bit
      • Espressif Platform
      • iLabs Platform
      • Raspberry Pi Platform (MCU)
      • Seeed Platform
      • Silicon Labs Platform
      • Teensy Plaform
    • Computer Platforms >
      • BeagleBone Platform
      • Raspberry Pi Platform (SBC)
      • UDOO Platform
    • Peripherals >
      • Shields
      • Grove System
      • Sensors
      • Actuators
    • Displays >
      • E-Paper Displays
      • Reflective Displays
      • TFT Displays
      • LCD Displays
    • Legacy Platforms >
      • chipKIT Plaform
      • 4D Systems Platform
      • Intel Platform
      • LaunchPad Plaform
      • BoosterPacks for LaunchPads
      • LightBlue Bean
      • Maple Plaform
      • Mediatek Platform
      • Microsoft Azure IoT DevKit
      • Particle Platform
  • Software
    • Exploring RTOS with Galaxia >
      • Event Library
      • Semaphore Library
      • Mailbox Library
      • Timer Library
      • Clock Library
      • SWI Library
      • Task Library
    • Ultra-Low Power with EnergyTrace >
      • Ultra-Low Power with MSP430
      • Ultra-Low Power with Energia MT and Galaxia
    • Using Integers Instead of Reals
    • Going Python?
  • IoT
    • IoT Platforms: Which Hardware? >
      • Matter with Silicon Labs MG24
    • IoT Services: Which Solution? >
      • Recommended IoT Solutions
      • Platform-Specific IoT Solutions
      • Other IoT Solutions
      • Not tested IoT Solutions
      • Notification Solutions
    • Get Date and Time from Internet with NTP
    • Fast and Easy WiFi Connection with QR-Code
  • Tools
    • How to Start?
    • Reference >
      • Asking for Help
      • Boards Pins Maps
      • Ruler
      • Boards and Plugs
      • I²C Logic Level Converter
      • Standards for Connectors
    • Training >
      • Texas Instruments Workshops
      • Embedded Systems: Shape The World — MOOC edX UTAustinX UT.6.02x
      • Embedded Systems - Shape The World: Microcontroller Input/Output — MOOC edX UTAustinX UT.6.10x
      • Embedded Systems - Shape The World: Multi-Threaded Interfacing — MOOC edX UTAustinX UT.6.20x
      • Real-Time Bluetooth Networks: Shape the World — MOOC edX UTAustinX UT.RTBN.12.01x
      • Systems Thinking with Texas Instruments Robotics System Learning Kit
    • Books >
      • Getting Started with the MSP430 LaunchPad
      • Getting Started with Arduino
      • Arduino Cookbook
    • IDE >
      • The Battle of IDEs
      • More Options
      • Assessing the Next Generation of IDEs
      • Tools for Documentation
    • Equipment >
      • Saleae Logic Analyser
      • Rigol DS1102E Oscilloscope
      • XDS110 Debug Probe with EnergyTrace​
      • Segger J-Link Programmer-Debugger
      • Nordic Power Profiler Kit II
  • Projects
    • Libraries >
      • Master I²C Software Library
      • Date and Time Library
      • highView Library Suite
      • Others Libraries
    • smartDevices >
      • I²C smartColours Smart Sensor
      • I²C smartRFID Smart Sensor
      • I²C smartLED Display
      • I²C smartControls Smart Device
      • I²C smartWiFi Smart Device
      • I²C smartBLE Smart Device
      • I²C smartNode Smart Device
    • IoT Projects >
      • Remote E-Paper Weather and Message Board
      • Typie-Walkie with LoRa and E-Paper Screen
      • Typie-Walkie with E-Paper Screen
      • Remote e-Paper Pictures Panel
      • Remote e-Paper Messages Panel
      • Industrial IoT Project
      • Remote Contactless Temperature Monitor
      • Using Node-RED for IIoT
      • Low Power Home Network Weather Monitoring
      • Updated Low Power Home Network Weather Monitoring
      • Weather and Security Station with Blynk
      • SensorTag to Blynk Using Node-RED
      • Pervasive Reporting
    • AI Projects >
      • Colour Recognition with Neural Network
    • Other Projects >
      • Air Quality Monitoring
      • Driving a Large E-Paper Display with a Compact Xiao RP2040
      • Low-Power E-Paper Weather Station
      • Portable Particulate​ Matter Monitor
      • FRAM-based E-Paper Screen Controller
      • General Purpose 3.5" Screen
      • Colour Recognition with Neural Network
      • A Low Power Weather Station
      • Digital Volt-Amp-Watt Meter
      • Mobile Measurement with LCD Display
      • Screen with SRAM for GUI
      • Volt-Amp-Watt-Meter for Grove
      • Multi-Touch Project with CapTIvate

Remote Contactless Temperature Monitor​

How to check the temperature with no contact in COVID times? The project combines PIR detection, thermal camera and hand-gesture recognition for perfect safety. An IoT extension allows to operate it remotely.
​
WARNING — This project is for EDUCATIONAL PURPOSE ONLY and SHALL NOT be used for effective fever detection. Contact your local health authority for appropriate tools and procedures.

​The hardware was sampled by SeeedStudio.
  • Wio Terminal: ATSAMD51 Core with Realtek RTL8720DN BLE 5.0 and Wi-Fi 2.4G/5G Development Board
  • Grove - I2C Hub 6 Ports
  • Grove - AMG8833 8x8 Infrared Thermal Temperature Sensor Array
  • Grove - Mini PIR Motion Sensor
  • Grove - PAJ7620U2 Gesture Sensor for Arduino​
The project was originally posted on Hackster.io.
Picture
Picture

How it works

The contactless temperature monitor is built around a finite-state machine and includes seven phases.
​

  • ​Phase 0: Initialisation
​The system initialises the sensors and checks they are operational.
  • Phase 1: Sleep
The system is in low-power mode.
  • Phase 2: Wake-up
The PIR sensor wakes up the system when it detects a person.
  • Phase 3: Take a picture
The IR camera takes a picture and displays it on the screen.
  • Phase 4: Result
If no temperature is detected, the person receives a green light.
After a while, the system goes back to sleep.
  • Phase 5: Question and action
Otherwise, a red light shows up and the person is given two options: either take another picture or call for assistance. The options are displayed on the screen with the corresponding hand gestures: from right to left to take another picture, from left to right to call for assistance.
  • Phase 6: Call for assistance
The call for assistance is performed through WiFi. The system then returns to low-power mode.
Picture

Prepare the hardware

The core of the system is the Wio Terminal. It is ideally suited as it features on a compact white case a powerful MCU, a colour display, and two Grove connectors to attach the sensors.

All the sensors use the Grove connector. It provides a quick, easy and secure way to build whole applications. The IR camera and the hand-gesture sensor go to the I²C Grove connector through the I²C hub. The PIR sensor goes to the GPIO Grove connector.
​
The Grove connectors are fool-proof and allow to build a circuit in a couple of minutes.
​
If you're using the Wio Terminal for the first time, you may need to go through the procedures Get Started with Wio Terminal and Update the Wireless Core Firmware.
  • Connect the Grove I²C hub to the Wio Terminal I²C connector.
  • Connect the thermal camera and the gesture sensor to Grove I²C hub.
  • Connect the IR sensor to the Wio Terminal GPIO connector.
  • Finally, connect the Wio Terminal to the computer.
Picture

Prepare the software

Although the project relies on the Arduino SDK, I didn't use the Arduino IDE to develop the project but embedXcode, embedded computing on Xcode, for a better productivity.

Each Grove sensor comes with its library for the Arduino SDK, and so do the screen and the WiFi radio.

The project includes five sub-systems: PIR sensor, LCD display, thermal camera, hand-gesture sensor, optional WiFi for MQTT.

Each sub-system was first tested separately, then integrated to the main project and fully validated before proceeding with the next one.
  • Download the Arduino project from the GitHub repository.
  • Download and install the libraries for each sub-system:
  • Screen: Seeed_Arduino_LCD, Adafruit Zero DMA,
  • Thermal camera: Seeed-AMG8833,
  • Gesture sensor: Gesture-PAJ7620,
  • WiFi for MQTT: Seeed-Arduino-FreeRTOS, Seeed_Arduino_atUnified, Seeed-Studio/esp-at-lib, Seeed_Arduino_mbedtls and PubSubClient; optionally Seeed_Arduino_atWiFiClientSecure.
If needed, refer to the procedure How to install library.

​Each sub-system comes with a detailed wiki page: Wio Terminal, Wio Terminal WiFi, Wio Terminal screen, thermal camera, gesture sensor.

​The code is very basic as the project is built around a finite-state machine on the loop() function. A C++ library wraps the Gesture-PAJ7620 code as an object for better consistency. The Free_Fonts.h file lists the fonts for the 320x240 screen.

The excellent PubSubClient library provides the MQTT features. The separate file Credentials.h includes the credentials for the IoT extension.
Picture

How to use it ​

  • Phase 0: Initialisation
The system initialises the sensors and checks they are operational.
​

  • Phase 1: Sleep
The system is in low-power mode. The screen turns off.

Picture
  • Phase 2: Wake-up
The PIR sensor wakes up the system when it detects a person.
The screen displays the warning message.
Picture
  • Phase 3: Take a picture
The IR camera takes pictures and displays them on the screen.
​

On the right side, the screen displays the scale of colours, with the minimum and maximum temperatures in °C.​

​The picture contains 8 rows of 8 columns, or 
64 pixels. A picture is considered as valid is more than three pixels have temperatures higher than 30.0 °C. For a valid picture, the test can be negative or positive based on the pixels with temperatures higher than 37,3 °C.
Picture
  • Phase 4: Result
If no temperature is detected, the person receives a green light.

​The test is considered as negative if the picture meets two criteria:
  1. The picture is valid if more than three pixels have temperatures higher than 30.0 °C; and
  2. The picture has less than two pixels with temperature higher than 37.3 °C.


After a while, the system goes back to sleep.​
Picture
  • Phase 5: Question and action
Otherwise, there are two cases: either the previous step is inconclusive, or some fever has been detected.

​The 
test is considered as inconclusive if
  • The picture has less than three pixels with temperatures higher than 30.0 °C.

If the previous step is inconclusive, the person is given two options: either take another picture or call for assistance.

Picture
If some fever has been detected, the person is also given the same two options: either take another picture or call for assistance.

​
The test is considered as positive if the picture meets two criteria: 
  1. The picture is valid if three pixels have temperatures higher than 30.0 °C; and
  2. The picture has more than two pixels with temperature higher than 37.3 °C.

The options are displayed on the screen with the corresponding hand gestures: from right to left to take another picture, from left to right to call for assistance.
Picture
Moving the hand to the left takes another picture.
​
Picture
Moving the hand to the right calls the assistance.
​
Picture
  • Phase 6: Call for assistance
The call for assistance is performed through WiFi.

​
The system then returns to low-power mode.
Picture

Enable the IoT extension

The IoT extension relies on MQTT to connect to computer with a Node-RED server.
​
On a computer, for example a single board computer:
  • Follow the procedure Getting Started to install Node-RED locally, on a Raspberry Pi or on BeagleBone, on a Docker container, on an Android device.
  • Follow the procedure Binary Installation to install the MQTT broker. Binaries are available for Windows, macOS, various flavours of Linux and Raspberry Pi.
  • Launch MQTT and Node-RED.
  • Install the required node node-red-dashboard.
  • Download the Node-RED flow from the GitHub repository and import it, then  press Deploy.​
  • Connect the computer to the WiFi LAN and note the IP address of the computer, 192.168.1.10 on the example.
​
Picture
​On the Arduino IDE,
  • Open the project.
  • Ensure the PubSubClient Library: Arduino Client for MQTT is already installed.
For more informations on MQTT, please refer to the excellent project MQTT on Wio Terminal by Salman Faris.
  • Edit the credentials file Credentials.h.
  • Check OPTION_WIFI is set to 1 to activate WiFi.

    
  • Set the computer IP address brokerIP, 192.168.1.10 on the example.





    
  • Enter ssidWiFi and  passwordWiFi to match with your local configuration.
  • Build and upload the project.
​

    
On Node-RED,
  • Call the menu View > Dashboard to display the dashboard.
Picture
  • Click on OK to accept the warning.

​
Picture
The dashboard displays two thermal pictures.

On the left, the scale is dynamic with the lowest and highest temperatures to define the range. Here, the highest temperatures is 33.0 °C, which is safe.


On the right, the scale is absolute from 33.0 to 42.5 °C. The threshold is set at 37.5 °C: lime is just below 37.5 °C and yellow just above.
Picture
Here, a cup of tea simulates a positive case.
Picture
​The two thermal pictures with different ranges and scales help the staff decide what to do.
​
Finally, two more fields give the result and recommended action. When action displays Call for assistance, the person is waiting for the staff.
Picture

Improve the local display

By default, the image displayed has the same resolution as the thermal sensor, or 8x8 resolution.

​The initial image is resampled through a bi-cubic interpolation to provide a nicer picture, here in 16x16 resolution.

The FPU of the ATSAMD51 manages all the computations required by the interpolation effortlessly.

Conclusion

The Wio Terminal handles it all: data acquisition and processing, display and WiFi communication with standard MQTT protocol. The Grove connectors are safe and quick to use to connect all the sensors, either I²C or digital.

Below are some ideas to refine the project and add new features.
Picture

Going further

The selected IR camera provides a 8x8 matrix with ±2.5°C accuracy. Better resolution and accuracy can be achieved with other Grove thermal cameras like the Grove Thermal Imaging Camera - MLX90621 BAA 16x4 IR Array with 25° FOV (±1°C ±3% accuracy), the Grove - Thermal Imaging Camera - MLX90641 BCA 16x12 IR Array with 110° FOV (±1.5℃ accuracy), or the Grove - Thermal Imaging Camera - MLX90640 32x24 IR Array with 110° FOV (±1.5℃ accuracy).
The entire system can run on batteries with the Wio Terminal Battery Chassis - Built-in 650mAH Lithium Battery with 6 Grove Interfaces.
​
On the IoT side, the project relies on MQTT and Node-RED with a local router. Other options include adding messages sent from the dashboard to the Wio Terminal screen, using an external MQTT broker like HiveMQ, turning WiFi as an access point to provide a private LAN, relying on other protocols like CoAPand on a third-party service like IFTTT or Blynk.

Also to be considered, an OTA update of the firmware for easier maintenance.
Picture
Picture
Picture
Picture
The same thermal picture, clockwise from top-left: original image 8x8, oversampled bi-cubic images 16x16, 24x24 and 32x32

Links

  • Wio Terminal: ATSAMD51 Core with Realtek RTL8720DN BLE 5.0 and Wi-Fi 2.4G/5G Development Board
  • Grove - I2C Hub 6 Ports
  • Grove - AMG8833 8x8 Infrared Thermal Temperature Sensor Array
  • Grove - Mini PIR Motion Sensor
  • Grove - PAJ7620U2 Gesture Sensor for Arduino​
  • GitHub repository with the Arduino and Node-RED projects
Picture
Picture

Posted: 24 Aug 2020
Updated:

Powered by Create your own unique website with customizable templates.