Product detection solution for retail company

01 The Challenge

The Client approached Altamira with the need to automate routine retail workflow and optimise inventory management. The Client’s team was open to the adoption of new technologies, so for this purpose, we envisioned an AI-powered solution by designing and integrating a product recognition system.

However, one of the primary challenges encountered in this project was the poor data quality. This issue manifested in various forms, such as incomplete datasets, inaccuracies, and a lack of standardisation across the data collected, which hindered the analysis and modelling phases. It was especially detrimental as AI systems heavily rely on high-quality data to learn, make predictions, and provide insights.

As a result, our team of experts undertook a multifaceted approach. We audited the existing datasets and pinpointed the most critical areas needing attention. Additionally, our experts refined the existing data annotations that lacked clarity, ensuring that each entry was accurately labelled.

Also, Altamira team introduced an iterative review process, where data quality was continually assessed at various stages of the project. This ongoing validation allowed for the early detection and rectification of emerging data issues, resulting in sustained improvements in data quality.

02 The Solution

Use cases

Mobile application for shelf scanning

We presented the Client with a ready-to-scale smart data capture and retail shelf analytics solution. So far, users utilize their mobile devices to capture images of the shelves, which are then processed for product identification. The delivered app interface is easy to navigate, making it accessible to staff with varying levels of technical proficiency.

Mobile application flow

• The user opens the mobile application
• The application opens the camera
• The application captures an image in the background (while the user can continue moving the phone)
• The application shows bounding boxes for products in (almost) real time
• The user scans the whole shelf and, once ready, hits the button “Analyse”
• The application sends the bounding boxes to the Cloud to recognize products (via an API call)
• The Cloud analyses the picture and returns product names
• The application presents the list – counts and names of front-facing products

Product management platform

At the core of the solution is the product management platform, a large database containing detailed information about various products. This platform serves as a reference point for the mobile app. When the app scans a specific product, the system matches it with the corresponding product in this database. The platform is continually updated to include the latest products and variations.

AI Integration: Detector and Matching

Product shape detection

Product shape detection employs sophisticated algorithms to identify and delineate the bounding boxes of products in the images captured by the mobile app. This detection process is important to accurately locate and identify different products on crowded shelves.

Feature matching

This process entails calculating the unique “features” of the recognized product bounding boxes. The calculated features are then compared against the features of products stored in the product management platform. The AI system is designed to perform this comparison with high precision, enabling it to match the product captured in the image with its corresponding entry in the database.

By combining these two AI-driven processes within the PoC development, the solution efficiently identifies front-facing products. The integration of the mobile app with the product management platform ensures accurate product recognition, greatly improving inventory management and operational efficiency.

03 The Result

Design

The delivered software design effectively integrates real-time image processing with AI-based object detection and inventory analysis, providing a user-friendly interface for efficient shelf scanning and product recognition.

The application is actively scanning a shelf stocked with various products. The screen shows the camera’s perspective, with no additional visual markers or information overlay, giving a clear view of the items on the shelf.

As the user continues to hold the device, the screen displays multiple red bounding boxes overlaid on each product on the shelf labelled with the word “object,” followed by a numerical confidence score. AI-powered detection feature identifies potential products and assesses the likelihood of each being a correct match.

In the final stage, the application presents a processed list of the recognized products. The list is displayed in an overlay on the lower portion of the screen, allowing the user to see both the physical shelf and the digital output simultaneously.

Features

• Real-time image capture: The application continuously captures images of the shelf as the user moves the device without requiring manual input for each photo
• Automated object recognition: Powered by AI, the app detects objects in real-time, displaying bounding boxes around each product on the shelf
• Confidence scoring: Each detected object is accompanied by a confidence score, indicating the AI’s certainty in the product’s identification
• Instant analytical feedback: Upon user command, the application instantly processes and displays the recognized items with their quantities on the screen
• Cloud integration: The application communicates with a cloud-based system to match detected items with a product database, ensuring up-to-date information and storage
• Inventory list: After analysis, it presents a summarised inventory list that includes product names, counts, and other relevant details like volume or type
• User interaction: The application includes an “Analyse” button, which users press when ready to finalise the scanning process and view the inventory count
• API communication: For product recognition, the application uses an API call to send the data to the cloud for analysis
• Dynamic result display: The final inventory count overlaps the live camera feed, allowing users to cross-reference the digital data with the physical products in view