Python, OpenCV, DeepFace, Raspberry Pi, Arduino
Machine Learning Object Detection Camera
The IdeaI recently began toying around with Raspberry Pis and Arduinos, and wanted to test out some ideas I had in the world of Machine Learning and Python Camera and Video libraries. This was my first foray into single board computers and microcontrollers, but I have a decent proficiency in Python, and have tooled around with deep learning models/frameworks in various applications.
My hardware initially consisted of a Raspberry Pi 5, and a Raspberry Pi Camera Module 2. I was ssh’ing into my Pi, so that I didn’t have to set up a whole monitor and mouse/keyboard. My initial goal of the project was for the device to have the ability to capture video, and from that video it could pull data like human detection, and more specifically face detection. From my research I saw that OpenCV might be the best application of Python to read data live from a video. I was sort of successful in this goal, but I soon realized that variables like available light and the angle of capture affected my results. I knew my device would be limited in it’s application to well lit areas, and the device needed to be eye level to accurately capture the data from the video. For clarity sake, I was using a Pi Camera Module 2 “Wide”, so I knew that the subject of the video had to be relatively close to the device as well for accurate data to be captured.
Automate ItNow that I had gotten the bones of the device running, I wanted to find a way to automate the device and to store the data captured. I wasn’t satisfied with only capturing faces, I wanted to capture biometric data from the faces detected. I also knew that it wouldn’t be efficient to have a device running a live video feed constantly, so I needed to set up a sort of wake/sleep detection cycle. Enter the Arduino.
With the addition of the Arduino, I was first able to setup a sleep/wake cycle using an ultrasonic motion sensor. This allowed the device to sleep and conserve power, only being awoken when motion was detected within a certain distance of the device. The Arduino would then send a boot request to the Pi, which would start running my program and activate the camera. Once the camera was running, it would attempt to detect human faces in view. If a face was detected it would take a picture of the face. I would then use DeepFace to pull the biometric data. In this first application I wanted to detect emotions in the face detected. DeepFace would run the image past its own archive of stored emotions, I chose a pre-trained model, as I didn’t want to waste the computing power and time training my own model. In the future I would like to train the model based on my own data. I built in a sort of fail-safe, so that if no face was detected within a set time frame, the device would go back to sleep. Waiting to be reawakened by the Arduino.
I stored the data and captured photos locally on the Pi, so that I could check the accuracy of the collected data. DeepFace also output an accuracy percentile, where it would tell me that it was “XX%” sure on a 1-100 scale that the emotion detected was accurate.
I tested other applications of this device, trying to test the extent of the biometric data that I could collect. I was able to pull things like estimate age, skin color, gender (in a binary sense) as well.
What did I learnFirst and foremost I learned that it was scary simple than I had expected to build a biometric capturing device, and all for under $100. My intention in building a device like this was to more a theoretical and explorative one. I was thinking about a fine-art application, where a gallery or space was filled with devices like these, all silently collecting data on the participants of the space. I wanted to tow the line between the transparent world of viewership and the opaque world of data collection, where devices all around us, from our mobile phones to police and security cameras silently learn about who we are and what makes us individuals.
The larger application of a device like this skews more in the direction of big tech and government authoritarianism. Totalitarian countries are already employing technology like this on their citizens, and tech companies are employing this sort of technology in your smart devices. But those smart devices and IOT objects we are increasingly adding to our homes, cars, grocery stores, and baby cribs are employing this same technology, to better serve the end-user.
All that to say, this was a super fun and interesting first venture into electrical engineering and computer hardware, and I have since bought more microcomputers and microprocessors, in hopes of automating more practical parts of my life.