Have you ever received a small item from Amazon in a huge box? Well, you are not alone. Amazon customers have been scratching their heads on why a pack of batteries gets delivered in a massive box. It is such a famous problem that there are subreddits to discuss the issue and people occasionally ask: why does Amazon ship tiny items in giant boxes?
For those concerned, the good news is that Amazon is acutely aware of this issue and is working on a fix. This problem of small items in giant boxes arises because Amazon has an ever-changing catalog with hundreds of millions of products. As a result, finding the right amount of packaging remains a challenge. However, with its scale, if any company could fix this issue then it is Amazon. Here is how it plans to make packaging smarter for its customers.
Deep Learning will unpack the packaging issue
Amazon acknowledges that its scale is a major hindrance for using manual inspection to choose packaging for each and every item. An unofficial figure claims Amazon ships approximately 1.6M packages per day and processes over 18 orders per second. As a result, Amazon cannot apply general packaging rules or run-of-the-mill logic for its packaging woes.
It has shifted focus to machine learning, particularly deep learning, to create a “cutting-edge-smart automated mechanism that can adapt on the fly to changing circumstances.” It is combining deep learning with natural language processing and computer vision to hone in on the right amount of packaging. Amazon says some of these practices have already resulted in the reduction of per-shipment packaging by 36 per cent over the past six years.
“When I started at Amazon in 2017, we had a lot of physical testing of products going on, but not a scalable mechanism that could assess hundreds of millions of products to identify the optimal packaging type for each product,” research science manager Matthew Bales, says in a blog post.
While it all started with statistical tests, Amazon wanted to find a way to see how a package would perform in a less-protective environment. “We wanted the capability to predict how a product would fare in a less-protective, lighter, and more sustainable package type. And once you’re in that predictive space, you need machine learning,” Bales explains.
ML model to predict package type
Bales, who is a physicist and leads machine learning within Amazon’s Customer Packaging Experience team, worked with his colleagues to build an ML model largely based on the text-based data that customers find on the Amazon Store. The idea of this ML model is to predict whether a given product could be shipped safely in a particular package type. The ML model takes metadata such as item name, description, price, package dimensions, to name a few.
This ML model was then trained on millions of examples of products successfully delivered by the e-commerce giant in various packaging types. Bales and his team also trained the model on examples of products that arrived damaged in given packaging types. One of the advantages Amazon has is the real-time feedback received from customers when a product is not sufficiently protected by its packaging.
“Customer feedback is paramount,” says Bales. “It powers all of our statistical testing.”
Based on the training data, the model learned that certain keywords played an important role while making decisions about packaging. Amazon says keywords such as “ceramic”, “grocery”, “mug” and “glass” indicated that a paddled mailer would not be the right packaging. Similarly, keywords such as “multipack” and “bag” indicated that mailers were the right choice.
The ML model is essentially trying to correct for redundancies like when Amazon offers extra protective packaging even if the product has some form of protective packaging. “The portion of the model that’s learning from the Amazon Store has learned really well what the product is, and about its dimensions,” Bales says about the model.
Computer Vision to fill the gap
The ability of the ML model to automatically learn what a product is, only solves half the problem. Another challenge that Amazon needs to solve to overcome this packaging menace is to understand how the vendor packaged the product before sending it to a fulfillment centre. In order to identify product packaging at scale, Amazon deployed computer vision across its fulfillment centres.
The ML team understood early enough that product images on the Amazon Store alone were not helpful when selecting the right packaging. In order to address this, Bales and his team turned to Amazon’s own image data. The products are delivered to Amazon’s fulfillment centres by sending them through a conveyor belt. These conveyor belts take the products through special computer-vision tunnels that are equipped with cameras to capture images of the products from multiple angles.
Based on this data, Prasanth Meiyappan, an applied scientist at Amazon, built a multimodal approach. This approach expands the training of the ML model to include these standard product images captured by cameras in computer-vision tunnels in addition to the text classifiers from the catalogue.
“Our model detects the packaging edges to determine shape, identifies a perforation, a bag around the product, or light shining through a glass bottle,” Meiyappan explains.
The multimodal approach not only works but is so intelligent that it is difficult for humans to discern how the ML model makes its judgement about what it detects in images. The product features identified and weighted by the model tend to be complex. However, Bales adds that “the packaging decisions generated by the model are empirically accurate.”
“When the model is certain of the best package type for a given product, we allow it to auto-certify it for that pack type,” says Bales. “When the model is less certain, it flags a product and its packaging for testing by a human.”
Amazon is currently applying this multimodal ML model to product lines across North America and Europe, automatically reducing waste at a growing scale.
While it sounds like a major win, Amazon did come across a challenge that is common in the ML domain: class imbalance. For an ML model to learn effectively, it needs to be served with many examples of failures as successes. This allows the model to differentiate effectively between the two. However, Amazon had a problem with the training data since as little as 1 per cent of product/package pairing turned out to be unsuitable in some way for the product within.
“Prior to implementing ML, we’ve shipped some products in envelopes and mailers for some time. So, we had loads of examples of things that were good in mailers, but didn’t have a lot of examples of things that were bad in mailers. ML models have problems with this kind of overwhelming imbalance,” Bales explains.
“The machine learning literature to do with packaging is pretty sparse. Not many people deal with the kind of datasets we are dealing with in the packaging domain. How effective a technique is in dealing with dataset imbalance is both domain and dataset specific,” Meiyappan adds.
In order to solve this problem, Bales and his team relied on an experimental approach that involved four data-based and two algorithm-based solutions. The winner, according to Amazon, produced a marked improvement in model accuracy. The winner, a data-based approach called two-phase learning with random under-sampling, focuses the model on the minority class in the first phase of training.
In the second phase of training, the model focuses on all of the data. “In our position paper we share that knowledge with the ML community, so that anyone who encounters a similar problem might choose to try this approach for themselves, to see if it also works in their problem space,” Bales said about the approach.
What’s next for Amazon’s ML model
Bales and his team are now focussing on expanding the use of this tool by training the model to understand all Amazon’s customer languages. The Amazon team also plans to incorporate the unique aspects of fulfillment in each country.
While ML and artificial intelligence can aid in eliminating waste, Amazon is also mindful of reducing packaging waste through the e-commerce supply chain. Amazon aims to deliver 50 per cent of its shipments with net-zero carbon by 2030 as detailed in its Shipment Zero goal. This is only possible without added Amazon packaging or by shipping items in carbon-neutral packaging.
Amazon is also incentivising its vendors to create optimised e-commerce packaging themselves, saving space and materials without compromising the protection of the product. With its goal to reach net-zero carbon by 2040, Amazon’s Climate Pledge needs to tackle the packaging issue around the world first and the ML model seems like a promising first step.