Artificial Intelligence in detecting Product Misplacements

AI Case Study: Product Misplacements and Virtual Areas

Detecting product misplacement is a common task in retail environment, a customer takes a product and then leaves it on another shelf or a staff member places products in the wrong place during stock-take or stock replenishment. The ability to detect such events will lead to significant savings and will improve customer experience.

We have developed an AI model that is capable of recognising, identifying and analysing products located in a retail shelf landscape, in single image or video stream. In addition to the detection of product misplacements we have added an ability of AI model to learn and recognise boundaries of ‘product placement’ areas, see ‘Chocolates Only’ area.

The below video demonstrates how AI model responds to product movement and product placement area resizing events in real time. Product misplacement alert is automatically generated when product is detected as being outside of the dedicated area or product was detected inside prohibited area.

Skip to relevant parts of the video by selecting below.

0:00 Start of the recording.
0:02 Commenced defining 'Chocolates Only' virtual area by placing pre-trained corner markers.
0:15 'Chocolates Only' virtual area is detected by AI model.
0:22 Caramello Koala #1 is added to 'Chocolates Only' area.
0:26 Freddo #1 is added to 'Chocolates Only' area.
0:30 Caramello Koala #3 is added to 'Chocolates Only' area.
0:36 Caramello Koala #1 is moved out of 'Chocolates Only' area. Misplaced Product alert is generated.
0:44 Caramello Koala #2 is moved out of 'Chocolates Only' area. Second Misplaced Product alert is generated.
0:53 Caramello Koala #2 is placed back to 'Chocolates Only' area.
0:54 Commenced resizing of 'Chocolates Only' area.
1:03 Resizing of 'Chocolates Only' area is completed. Count of misplaced products is increased.
1:07 Commenced resizing of 'Chocolates Only' area.
1:13 Resizing of 'Chocolates Only' area is completed. Count of misplaced products is decreased.
1:24 Soup #1 is added to 'Chocolates Only' area. Misplaced Product alert is generated.
1:27 Soup #2 is added to 'Chocolates Only' area. Second Misplaced Product alert is generated.
1:33 Soup #2 is moved out of 'Chocolates Only' area. Count of misplaced products is decreased.
1:40 Commenced resizing of 'Chocolates Only' area.
1:47 Resizing of 'Chocolates Only' area is completed. Soup #1 is now detected as lying outside of prohibited area.
1:57 Caramello Koala #2 is placed to 'Chocolates Only' area. Misplaced Products alerts are cleared.
2:04 End of recording. All products are located in relevant areas.


This AI case study has been conducted in Australia, Melbourne by FifthOcean and Nola, a foot traffic counter & visitor analytics solution for retailers, venues & events.