Questionmark is a Dutch non-profit that’s changing how shoppers choose which products to buy in the supermarket. When consumers use the Questionmark app to scan a product barcode, they are able to compare it against similar options with regards to sustainability, healthiness, plus the human welfare and environmental impact of the item.
The organisation hopes that providing consumers with clear and honest information about their food purchases will help them to make better choices and, as a result, incentivize companies to sell better products.
“Direct change needs to happen within food companies and their supply chains, but for that, change also needs to take place in consumers’ buying trends and demand for healthy and sustainable products,” explains Willem van Engen-Cocquyt, lead developer at Questionmark.
Questionmark gathers all the data that they can about food products, but sometimes important details such as Fairtrade and EKO are missing from the labels. Manually checking for incomplete information is time consuming, so the development team wanted to create a machine learning solution to automate the process. They turned to Xomnia for help with the project.
“We wanted to collaborate and implement something we didn't have time for ourselves, and learn how an experienced AI company would tackle this issue. Xomnia knew where to start.”
Finding a better answer to the question
Three junior data scientists from Xomnia’s junior development program teamed up with the Questionmark team for several weeks on a pro-bono basis. They started to classify the images of quality marks, but soon determined that method was too limited. That’s when Xomnia junior data scientist Andrea Kuijt says they decided to further optimise the classification process by training a model to detect objects.
“When humans look at an image, they instantly know what and where objects are. Object detection algorithms are able to do the same. Therefore, we applied a neural network called YOLOv3, that is able to predict class probabilities directly from full images in one evaluation. This allowed detecting quality marks on products in images.”
The team’s biggest challenge was collecting enough viable data for the model. The junior data scientists emphasise that quality and quantity are essential to the success of this kind of project.
“It is quite easy to implement and this solution can be applied for many different use cases such as facial or traffic light detection. But you must ensure you have got a lot of annotated images available to train the model with.”
According to junior data scientist Jesse Haenen, their solution proves how much more accessible artificial intelligence is becoming - even for companies without huge budgets.
“Most state-of-the-art models, such as YOLOv3, can be implemented with very few lines of code. Of course, without careful study your model now remains somewhat of a black box, but it does signify an exciting change in the field of artificial intelligence where some of the most powerful models can be implemented with basic knowledge.”
Artificial intelligence for good, food for the future
Object detection not only makes the identification process faster and more complete, it also provides a valuable tool to help Questionmark continue to improve their database.
“With object detection we can see if, where, and which quality mark was present on a product. This helps Questionmark label their data so they can more easily add products and product information to their app,” says Judith Klepper, a junior data scientist at Xomnia. “This project also provided the organisation with training and guidance to apply to future challenges.”
Questionmark will soon integrate the solution into their data pipeline and Willem says it already looks to be a great fit for their needs.
“We will likely expand the number of certifications recognized, based on documentation provided by the junior data scientists. Next to that, we will work on parsing and recognizing ingredients from the package label, which will help us, for example, to monitor and support the transition to more sustainable protein production and consumption.”
In the next year, Questionmark plans to release a comparison of supermarkets, which will also incorporate information from product certifications.
For the Xomnia junior data scientists, the project provided the opportunity to implement a creative artificial intelligence solution that can make a positive impact.
“We found it fascinating to dive into the promising opportunities of artificial intelligence while contributing to a more transparent food industry and to a more sustainable world. It made the project interesting, both technically as well as non-technically.”
We at Xomnia believe that artificial intelligence is key to a sustainable future. Are you looking for a data-driven approach to your sustainability project? Contact us to talk about how we can collaborate and use artificial intelligence for good.