Authored by Ionut Preda and originally published on 09.11.2022 in Mindcraft Stories, a publication on emerging technology
If you've ever been annoyed about a supermarket purchase that cost more at the checkout than the shelf price tag suggested, you've probably thought, irritated, that the retailer was trying to trick you. In reality, store price mismatches are often caused by errors that inevitably appear at large stores. Prices can change as often as several times per day, and with thousands of price tags that must be updated manually, the stores simply don’t have enough employees to update prices immediately.
Adapta Robotics, a Bucharest-based robotics startup, offers an almost automated solution to this problem. They created ERIS (Effective Retail Intelligent Scanner), called MARCEL in Carrefour stores, a robot that automatically scans all the labels on a shelf, reads and compares the prices with the correct ones, and prints updated labels. At the same time, the robot also analyzes how fully the store shelves are stocked and, in the future, could even check the accuracy of the placement of the products themselves. ERIS was officially launched on November 3rd, 2022, and the company has already partnered with one of the leading European retail giants, Carrefour, to use the robot in their stores.
ERIS is not the first successful product for the Bucharest-based company. Adapta Robotics has previously launched MATT, a robot that tests physical inputs for device screens in many industries, from smart washing machines to instrument panels on airplanes, on the international market.
Founded by the members of a robotics team created by students from the Polytechnic University of Bucharest, with awards at dozens of professional competitions, Adapta Robotics got its start as a division of the software development company rinf.tech, and later turned into an independent company. I spoke with the three founders of the company – Diana Baicu, Mihai Crăciunescu, and Cristian Dobre – about how they turned their passion for robotics into a successful business, how the robots they invented work and about the training and development process of using them.
Co-founders of Adapta Robotics, from left to right: Cristian Dobre, Diana Baicu, and Mihai Crăciunescu.
PHOTO: Andrei Cojan/Adapta Robotics
Mihai Crăciunescu: Our team started many years ago at the Bucharest Polytechnic when we were building robots to participate in various competitions. After a while, when we won almost all the competitions we went to, national or international, we decided to take the step to create a company.
Cristian Dobre: Robotics is a very expensive hobby, and for this reason, we had to work hard. After a few summer jobs, I ended up at rinf.tech. They wanted to build a robot to test mobile devices, but they didn't have the know-how. I joined their team, and they helped us apply for a European funding project to develop such a robot, which we called MATT.
Diana Baicu: MATT was the first prototype we made when we were a division within rinf.tech, this was around 2015. Since then, and especially in the last two years, we started to separate from them and create an identity of our own as a new robotics startup. In the meantime, we also started working on developing ERIS.
Andrei Cojan/Adapta Robotics
M.C.: The shorter answer is that we talked with a retailer from Romania, who presented us with some problems they faced in the stores, and we saw it as a kind of technical challenge.
Retailers can pay serious fines if the prices on the shelf are not in line with the actual ones. And the mislabeling part isn't bad faith on the part of a retailer, it's an error. To stay competitive, retailers issue daily price changes or temporary promotions on certain products. Logistically, at a store with 10,000 labels, making sure every label is updated every time is a problem. Human error will still occur somewhere.
As engineers who have been developing robots for so many years, so obviously, our solution was a robot.
D.B.: From discussions with various actors in the retail environment, we realized that most solutions in this field propose autonomous platforms. Which, of course, from a technical point of view sounds very good, but it is not necessarily the most efficient solution. This is because the problems identified by that robot must still be solved by humans.
With ERIS, we decided from the beginning that it must be operated by a human. Thus, we make the pipeline more efficient, as the operator can solve the problems that arise on the spot.
For example, when ERIS starts scanning after being placed in front of a shelf, there is a distance it must maintain from it. All of this information is specified by the robot, which tells the operator how close or far away to stand, as well as the speed at which it can scan the tags.
We started with one prototype, of course, which we tested in a pilot project in stores. Now we've done a redesign based on everything I've learned in about a year and a half since ERIS has been running.
C.D.: ERIS is a tool. You can think of it as a mega scanner. Instead of looking at every item on a shelf and every barcode to check every price, as happens when you check manually, it scans the entire surface, and the operator just has to push it past the shelf.
ERIS is equipped with sixteen cameras. Eight of these are distance-measuring cameras — that is, instead of recording color, the camera will return a black-and-white image; white means that the product is very close, and black means it is very far.
After taking in the data, the robot analyzes each image captured with conventional cameras and uses a software detection model to locate the regions where the price tags are located.
M.C.: The robot takes images from all the cameras simultaneously and builds a single large vertical image, almost two meters high, as long as the shelf. From this, it identifies price label locations.
Once it has discovered the areas with price tag, they are cropped out of the big picture and sent to two neural networks: one that identifies the barcode and another that reads the price. The latter is developed with a machine learning model created by us. Once it has all of this information, it can query the current product price against the retailer's updated database.
There is also a separate module that identifies shelf occupancy to alert the retailer if the shelf is empty or out of stock.
The accuracy of the entire system is over 98%, in some cases even 100%, because the entire flow has been optimized to work as well as possible in the real store environment.
D.B.: If the operator identifies a wrong label, he can print it on the spot and replace it because there are two printers on the robot. Alternatively, it also has the option to print several corrected labels at once after it has finished scanning. In this case, the robot has a location finder feature that can show the operator where the wrong label is.
ERIS1 in Store. Adapta Robotics
M.C.: The biggest challenge we had was the datasets, that is, gathering enough relevant information that is labeled correctly so that you can train a neural network with the accuracy I mentioned earlier. In the beginning, we took many pictures of store shelves, which we had to manually annotate or use online annotation services.
The volume of data required was enormous, and the quality of the results returned by online services was very poor. That is, out of 1,000 pictures, maybe 200 were properly annotated.
We invested a lot of time and resources in a simulator, where we simulated the store with products, with real data from the store, through which we were able to train our networks with high accuracy.
D.B.: We made a kind of game using Unreal Engine, a virtual store, to generate our data so that we could get a large amount of data with minimal annotation effort. At the same time, this also allowed us to more easily configure the models in case of later changes. If we change the retailer, for example, the solution works, by and large, but small adjustments still need to be made.
C.D.: The colleague who worked on this area made a very cool algorithm that generates the shelves programmatically. Each time you started the simulator, their arrangement in the store remained the same, but the labels and shelf occupancy differed.
A very big advantage is that in a game environment, you can also acquire data about the distance to objects and test that part. Such data can be obtained either if you go with the camera that measures the distance and scan the whole store or, much easier, if you load the algorithms into such a simulator. With Unreal Engine, it's very simple to calculate the distance from the camera to a product on the shelf.
M.C.: Each retailer has labels with a different format and maybe even a different font. And we need to make sure that ERIS can recognize the price regardless of these variables. They sometimes exist even within the same store. For example, they are yellow or electronic tags, with a promotion or without a promotion - you can get up to ten different kinds. The solution must be robust to such changes.
ERIS (left) and two versions of the MATT robot. Andrei Cojan/Adapta Robotics
D.B.: We are at the point where we have already seen that the robot is more efficient than a human operator. By our calculations, comparing how long it would take an experienced operator to scan a shelf of around 200-300 tags and how long it would take ERIS to do the same, it came out to an 85% improvement. Taking into account the label change part, the efficiency percentage is 45-50%. Our plan is to bring this percentage somewhere to 65-70%.
D.B.: We already have the second version, which we have already developed and are planning to introduce on the market. We optimized both the hardware side and costs, improved performance, and made it more reliable and easier to use. This will also increase the number of imaging cameras to ten.
C.D: ERIS is already very efficient, and most data processing is done directly on the robot. To give you an idea, in one minute, around 60 GB of data are transmitted and processed on ERIS. We envision, in the future, that this data will be aggregated and transmitted to the retail and logistics systems so that it can be used there as well.
D.B: That would mean integration with other store systems to know, for example, the historical inventory stock patterns. We have a lot of plans on that side, including an analytics platform based on all the information that ERIS gets to actually help the retailer understand everything that's going on in the store.
M.C.: Another functionality we have on the radar is a planogram, through which you can check if a product is positioned where it should be on the shelf. For example, product manufacturers pay a premium to have their products on the top shelf, third shelf, etc., usually to be more visible, and the retailer has to ensure that those products are there. That's where you get into a detection area of the product itself, which is more complicated than the tag, but that's a feature we're discussing for the future.
MATT testing the tablet touchscreen. Adapta Robotics
M.C.: MATT was designed as a robot to automate testing on phones, around 2015. Now you can automate pretty much everything that happens on a phone with emulation software, so MATT is not that relevant for smartphones anymore.
It's just that other industries have emerged that require testing for touchscreens, such as automotive. There are other situations where you need to integrate a phone with a device and make sure that the Bluetooth connection works or that if you press a button on the phone, something happens on the other device, like in the case of appliances that you control through the phone.
I saw that MATT is perfect for this, so I put it on input testing, and it became kind of a benchmark in this industry. Most of our customers for MATT are in the United States but also in the Netherlands, Germany, France, Spain, and even Lithuania.
Andrei Cojan/Adapta Robotics
M.C: We are actively scaling the use of ERIS, primarily in Romania, because we are a Romanian startup. But our targets are abroad because that's where the impact will be much more relevant. In Romania, labor in stores is also cheaper. And as there is always the question of return on investment for the retailer, in countries like Sweden, Norway, Great Britain, Germany, or France, the investment will be much more attractive for the retailer.
The vast majority of our competitors are well-known or are credibly well-funded companies. We have to focus on quality, that is, that robot does what we promise it does.
Many of the initiatives in this field fail because they have a lot of nice words in the presentation, but when they get to launch a robot, it does not perform the basic functionalities or performs them very poorly. To get a "piece of the pie", we focus on quality: the AI and the robot must work at very good percentages so that we can deliver a reliable solution that actually improves store operations.
This Q&A interview session with Adapta Robotics happened at gotech.world 2022 Expo, where the company introduced the second iteration of ERIS, the retail robot.
Ionuț Preda is an editor with several years of experience in mainstream media. He is curious about the application of technologies to the real world and the evolution of ideas throughout history.
Mindcraft Stories is a Romanian language media platform created by BRD (a Romanian bank based in Bucharest) for editorial projects on science and technology. They are involved in the tech world to support teams, competitions and robotics labs, research, entrepreneurial innovation, and the Romanian community of IT & Communications companies and startups. You can follow their latest updates on Facebook and Instagram.
Copyright © 2024 Adapta Robotics.All rights reserved.
