STEM-In-Action Spring Scoop: Cloudy with a Chance of Robotics
Our STEM-In-Action Grant winning teams have been hard at work for the past year advancing their eCYBERMISSION projects to make a difference in their communities. If you're new to the eCYBERMISSION blog, the Army Educational Outreach Program (AEOP) awards STEM-In-Action Grants of up to $5,000 to eCYBERMISSION teams that wish to further develop and implement their projects in their communities. We're checking back in with this year's STEM-In-Action Grant teams to get the inside scoop on their project process. Today, say hello to Idaho team, Cloudy with a Chance of Robotics.
___________________________
Welcome back! We are team Cloudy With a Chance of Robotics, from Eagle, Idaho! If you’re just tuning in, our team consists of Kavya Bansal, Kashvi Bansal, and Chinmay Tiwari. We are in different grades and go to different schools, but we all love STEM and helping our community. Our goal with the STEM-In-Action Grant is to help the Idaho Foodbank with efficiently managing their warehouse space.
The Foodbank needs a way to do a quick and comprehensive inventory assessment and find out which bins are available and which are occupied. However, warehouse management is currently done manually. We want to help using the power of robotics and artificial intelligence (AI). Our proposed solution involves capturing footage of the storage bins in the warehouse using a contemporary drone and processing this video using computer vision and AI. We believe we can extract the data from the video and articulate the information similar to how a warehouse employee might do manually. This could mean knowing which bins are occupied and which items are stored in them. This will significantly increase the efficiency of their inventorying process from hours to minutes.
We have made a lot of progress towards achieving this since our last blog post! Two more visits to the Foodbank and a ton of work at home have given us a lot of learning about drone hardware, flying skills and repair, and learning to train the AI model.
The goal of our second visit was mostly to get a lot of pictures. Training an AI model takes a lot of data, like how a child needs a lot of practice before they can read. Our little baby AI model was in need of some examples of the Foodbank bins in four different states: both full, both empty, left full, and right full (because we’re optimistic people). So, we brought two drones to the Foodbank and prepared to take some photo and video.
While we were there, disaster struck. A drone, while gliding along an aisle, crashed into one of the racks and fell to the ground, critically wounded. We had broken one of the critical components that supports the propeller. On the bright side, we learned how to fix the arm of a drone.
Meanwhile, we started working with the pictures that we had gotten out of that visit. We labeled them and trained a model, but it wasn’t very good. Our baby was struggling. We had less than a hundred pictures, which was grossly short of what we needed. It seemed to just flip a coin between both full and both empty, and it always had low confidence. However, there was a simple solution that we knew would help — more data and more training, and we decided to soldier on!
Our next visit to the Foodbank happened early in January, because we had to wait for the inventory to cycle through to get new pictures. Our model now consists of around 400 pictures of the Foodbank bins in four different states: both full, both empty, left full, and right full (because we’re optimistic people). We sorted all of our pictures into these states and used that to retrain the model, which has given us a much more successful version of the model than our initial one. This model has on average a higher confidence than the first one, and it is usually right about the states of both full and both empty.
We aim to have a model at 90% or greater accuracy and confidence. Right now, we are almost at that goal for the both full and both empty states on average, because the majority of our training data falls into those categories. Once the Foodbank implements their plan for use of barcodes for inventory, we hope to integrate that with our system of counting bin availability for a complete and comprehensive warehouse management system.
We gained some publicity before we started our STEM-In-Action-Grant project. We were interviewed by KTVB and the Idaho Statesman, to whom we mentioned our plans about warehouse management. Now that we have made substantial progress, we plan to contact them again for a follow up interview about our current project.
So far, this project hasn’t been without its trials, but it has been very rewarding to see our model accuracy improve over the three versions so far. We believe we are on the path to success and really help out the Foodbank staff fulfill their mission to the community.
___________________________
Perseverance in the face of adversity is one of the toughest things to overcome, but Cloudy with a Chance of Robotics are proving that they're unshakeable. It's awesome to see their model progress from its early stages to them nearly hitting their accuracy and confidence goal. We're certain this won't be the last we see of this hardworking team!
Faith Benner
Senior Communications and Marketing Specialist
We have made a lot of progress towards achieving this since our last blog post! Two more visits to the Foodbank and a ton of work at home have given us a lot of learning about drone hardware, flying skills and repair, and learning to train the AI model.
The goal of our second visit was mostly to get a lot of pictures. Training an AI model takes a lot of data, like how a child needs a lot of practice before they can read. Our little baby AI model was in need of some examples of the Foodbank bins in four different states: both full, both empty, left full, and right full (because we’re optimistic people). So, we brought two drones to the Foodbank and prepared to take some photo and video.
While we were there, disaster struck. A drone, while gliding along an aisle, crashed into one of the racks and fell to the ground, critically wounded. We had broken one of the critical components that supports the propeller. On the bright side, we learned how to fix the arm of a drone.
Meanwhile, we started working with the pictures that we had gotten out of that visit. We labeled them and trained a model, but it wasn’t very good. Our baby was struggling. We had less than a hundred pictures, which was grossly short of what we needed. It seemed to just flip a coin between both full and both empty, and it always had low confidence. However, there was a simple solution that we knew would help — more data and more training, and we decided to soldier on!
Our next visit to the Foodbank happened early in January, because we had to wait for the inventory to cycle through to get new pictures. Our model now consists of around 400 pictures of the Foodbank bins in four different states: both full, both empty, left full, and right full (because we’re optimistic people). We sorted all of our pictures into these states and used that to retrain the model, which has given us a much more successful version of the model than our initial one. This model has on average a higher confidence than the first one, and it is usually right about the states of both full and both empty.
We aim to have a model at 90% or greater accuracy and confidence. Right now, we are almost at that goal for the both full and both empty states on average, because the majority of our training data falls into those categories. Once the Foodbank implements their plan for use of barcodes for inventory, we hope to integrate that with our system of counting bin availability for a complete and comprehensive warehouse management system.
We gained some publicity before we started our STEM-In-Action-Grant project. We were interviewed by KTVB and the Idaho Statesman, to whom we mentioned our plans about warehouse management. Now that we have made substantial progress, we plan to contact them again for a follow up interview about our current project.
So far, this project hasn’t been without its trials, but it has been very rewarding to see our model accuracy improve over the three versions so far. We believe we are on the path to success and really help out the Foodbank staff fulfill their mission to the community.
___________________________
Perseverance in the face of adversity is one of the toughest things to overcome, but Cloudy with a Chance of Robotics are proving that they're unshakeable. It's awesome to see their model progress from its early stages to them nearly hitting their accuracy and confidence goal. We're certain this won't be the last we see of this hardworking team!
Faith Benner
Senior Communications and Marketing Specialist
Comments
Post a Comment
We welcome your comments and expect that our conversation will follow the general rules of respectful civil discourse. This is a moderated blog, and we will only post comments from bloggers over 13 years of age that relate to eCYBERMISSION. We will review comments for posting within one business day. Bloggers are fully responsible for everything that they submit in their comments, and all posted comments are in the public domain. We do not discriminate against any views, but we reserve the right not to post comments.