Stop the Milk Thief

picture of the shameless milkman
Output from the raspberry pi that runs the fridge
notifications view
Application first view

Inspiration

You know that one annoying person who always takes your milk from the fridge? What if you could catch them in the act, and publicly name and shame them?

What it does

Our smart fridge uses facial recognition to tell if the person removing an item is the same one who put it in. If not, the owner receives a notification on their phone, saying who stole from them and what they took.

How We built it

We have sensors from Reply to detect when the fridge is opened/closed and when an object is put down or picked up.

We use Microsoft Azure to identify faces and objects.

The sensors are connected to an Arduino, which is connected to a raspberry pi. The pi controls the camera and takes pictures, then sends them to our trained Azure models to pick out people and fridge objects such as orange juice

Notifications are managed by Firebase and then we made an Android app to show the milk thief to people.

Challenges We ran into

The pressure sensor kept giving us random readings We lost the pressure sensor (and found it) Running docker in the raspberry pi caused a segfault

Accomplishments that We're proud of

What We learned

First time programming on Arduino and raspberry pi

How to train and deploy an image recognition model on Microsoft Azure

What's next for Stop the Milk Thief

Built With

Submitted to

Hack Cambridge 101

Created by

I made the face recognition model and the program that checks if you are allowed to take an item out

Simon Crane
I worked largely on the Android application, interacting with Firebase and Google Cloud storage to receive push notifications and share images of culprits. I also worked on the object recognition API using Microsoft Azure's Custom Vision AI, and interfaced it with our RaspberryPi.

Chris Underhill
I worked on Android studio to help create the phone app and contributed on it's connection with Firebase and the hardware. I was by far the least experienced member of the group so this was an immense learning experience for me as everything I used was new for me.

Savvas Shiakas
Worked on the electronics and hardware part of the project. This was the first time that I used an Arduino and the raspberry pi towards an actual goal.

SinisterPotato