XJTLU creates algorithm that models human visual perception

As bricks-and-mortar shops shuttered amid the global pandemic, markets around the world have turned to online shopping in numbers never seen before. While the shift to online was already underway before 2020, the pandemic accelerated the move, leaving traditional retailers and big tech companies alike scrambling to provide a seamless online shopping experience.

One option for improving online shopping is the use of reverse image search to find what you need based on a photo rather than text-based search. It’s a feature already popular in China’s ecommerce market but far less common in other parts of the globe – and part of the reason is the complicated tech behind it, according to researchers from Xi’an Jiaotong-Liverpool University.

The group of PhD students, based in the University’s School of Advanced Technology, have developed a new algorithm that makes it easier for software to determine the main object in an image, improving both the speed and accuracy of image search functionality.

According to PhD student Siyue Yu, the algorithm empowers computers “see” more like a real person, helping it distinguish between the main object of an image and the background or other objects more accurately.

“Our method uses a type of weakly supervised salient object detection, which is common technique in image processing and computer vision algorithms,” she says.

“Weakly supervised refers to object detection that requires less human work to ‘train’ the computer to identify the main object in an image.The

Yu’s supervisor Dr Jimin Xiao says the drawback of the weakly supervised training method is that it can only give limited information and therefore produces inaccurate results when used in isolation.

“Weakly supervised object detection doesn’t perform that well, giving users a subpar experience when they attempt the reverse image
search function in online shopping, for example,” he says.

“Because it can’t cover the whole object area in an image or show a clear boundary of the object, often other methods are needed to enhance the boundary so the computer can accurately predict the shape of the foreground.

“This comes at a significant cost as it requires humans to spend far more time ‘training’ computer software to get optimal image search results. Image accuracy therefore requires more supervision, which is expensive and labour-intensive.

“What we propose with our new technique is a one-stage training algorithm that requires no pre-or-post-processing steps or extra supervision. This makes it a relatively easily technique for a computer to ‘learn’.

“We hope this will mean that fast and accurate image detection will be more accessible to users in places like online shopping platforms.”

Comparison of object detecting results generated by Yu’s method (3rd column from the left) compared to other methods

Yu says their new approach moves away from previous research projects, which rely on edge detection to capture object boundaries – a fully
supervised method that requires both extra data information at the training stage and a manual process to mark every pixel point along the boundary.

“By contrast, our weakly supervised method can recover missing structure information of an object based on the same colors or positions. It is cost effective without having to pay for manual marking and it also generates more accurate results,” she says.

“This method can also work as the first step in other technologies for recognising images and separating the main object from everything else. This technology helps capture essential image information more efficiently. It also has wide applications in image editing, image retrieval, object detection and robotics.”

The research was presented at the 35th AAAI Conference on Artificial Intelligence. The ongoing research project has received funding from National Natural Science Foundation of China.

By Huatian Jin, translated by Ke Tang

Photo by Siyue Yu