Babysafe is an IoT solution designed to prevent adults from forgetting their babies in the rear seat of their cars.
In Israel, from the start of 2008 and up until mid-2013 the total number of cases where people forgot their babies in the car was 205. This could be easily prevented with today’s technology. Babysafe is our way of doing that.
A Raspberry Pi 3 with a thermal camera (MLX90640) connected to it via I2C is installed in the car. The Raspberry Pi is connected via Bluetooth to an OBDII unit in the car. The Raspberry Pi reads the RPM from the car to know if the car is on or off. If the car is off it looks for a baby in the back seat via the thermal camera. If a baby is detected for a long period of time after the car is turned off, a message is sent to the cloud.
In the cloud we have an IoT Hub which receives the messages from the Raspberry Pi. When a message is received, the IoT Hub forwards it to a logic app in the cloud which reads the device id the message came from and locates the owner of this device to send them a notification to the mobile app.
The mobile app is used to connect the device id with the owner. The registration is sent to a database in the cloud. The app receives push notification when the owner forgets their baby in the car.
We have a Raspberry Pi 3 connected to the internet via WIFI. To the Raspberry Pi we attached a Qwiic IR Array MLX90640 thermal camera to the I2C bus and connected the Raspberry Pi to the On-Board Diagnostics unit in the car via Bluetooth.
On the Raspberry Pi we installed Raspbian OS which is Linux based. We installed .NET Core 2 runtime for Linux, so we’ll be able to run .NET Core applications.
The application was developed in C# on Windows, compiled and published for Linux and ARM architecture. Then we use the .NET Core runtime we previously installed on the Raspberry Pi to run the application.
The application starts with the car engine. It keeps checking the car status every two minutes and sends a message to the cloud with the status. If at some point, we recognize that the car is off we immediately look for the baby. If no baby is detected, we send the data to the cloud and shutdown. Otherwise we wait another two minutes to give the parent some time between turning off their car and getting the baby and then we look again for it. After that we send a message to the cloud indicating the situation and we shutdown.
In the cloud we have an IoT Hub configured to receive and send messages from and to devices. We used the SDK provided by Microsoft to create an instance of the hub in our code and used it to send the messages to the cloud.
Each message is a json with two fields and an additional property. The two fields are the car status which can be on or off and a presence field which can be true or false depending on if there’s a baby in the car or not, or null if this information is not available.
The additional property indicates that there’s a baby forgotten in the car. This property is sent only if the car is off and the baby was and still in the car.
The camera is used to detect the baby presence in the car. We took a C++ application that is built over the C++ driver provided by the makers of the camera and changed it to suit our need. The result was an application that initializes the camera and captures an image. The image is stored in a text file with specific format.
In the main application we created an image class. The class holds a matrix of pixels, each pixel is represented by the RGB color. Then we developed a helper class that can take the output of the C++ application and create an object of type image. In the image class we wrote a function that can recognize a presence in a given image. It counts the number of adjacent red (hue between 0 and 60) pixels in the image and if it exceeds a specific threshold (which was set by trying a lot of images and situations) then it positively identifies a presence in the image. We also added a functionality for the helper class to create jpeg images from the image object. This was used for testing and to help us understand what we see.
The main application holds a process that runs the C++ application. Each time we need to take an image the process is called and with the helper class we create the image object, then we ask the object if there’s a presence in the image.
The On-Board Diagnostics unit is plugged in the car. It communicates with the Raspberry Pi over Bluetooth. We used a Python application that can read and write data to the OBD to read the RPM of the engine.
When the car is off the RPM is zero and when the engine is running, and the car is parked it indicates around 700 revolutions per minute. We set the threshold to zero. When the Python application runs it initializes the connection with the OBD and then sends the command to read the RPM.
In the main application we hold a process that can run the Python application and each time we need to read the RPM we run the process and look for the result in the output. We parse the output and return car status on if the result is greater than zero and off otherwise.
We have six resources in the cloud: Azure IoT Hub, Azure Service Bus, Azure Logic App, Azure Function, Azure Database and Firebase Cloud Messaging service.
The IoT Hub was used to read messages sent from the Raspberry Pi to the cloud. It can filter the messages received by specific properties. We used this ability to filter messages that need action from our side, these are the messages which said that someone left her or his baby in the car. Said messages were inserted to a queue for further processing.
Azure Service Bus holds a queue that stores messages from the IoT Hub for further processing.
We created a logic app that is triggered by items inserted to a queue and configured it to listen to the queue we used in the IoT Hub to store the messages. When a message is inserted, the logic app flow is started.
The flow consists of steps. The first step is to parse the message receive and extract the device id that sent it. Then we send the device id to function in the Azure Database that return the FCM (Firebase Cloud Messaging) token saved for this device. Then we create an HTTP Rest call to the FCM service with said token and this sends a notification to the owner of this device.
The Azure Database was used to store FCM tokens of each device we have. In the database we have a stored procedure the takes a device id as input and finds a record in the database with this device id. This was used by the logic app as we explained earlier to find the token of the owner of this device.
The Azure Functions we developed was used in the mobile app. We send an HTTP Rest call with the device id and the token. The function connects to the Azure Database and inserts or updates the token for the specified device id.
The Firebase Cloud Messaging or FCM is a service that can send push notifications to mobile apps. It’s owned by Google. It can send a broadcast or a specific user notification. Each user is assigned an FCM token that is used to send them specific notifications. The token is assigned when the user connects to the service and is refreshed periodically.
We developed a mobile app that can receive push notifications. The app was developed for Android in C# and Xamarin. The notifications were sent via the Firebase Cloud Messaging service by Google. The app has a login screen which is used to register the device id with token generated by the FCM service for the user. The user is required to sign in with device id as username.
When the user logs in, a Rest call is made to the Azure Function we have with the device id and the FCM token to store them in the Azure Database. Each time the token is refreshed the same call is made again.
