New algorithm helps make better decisions

Do you often feel overwhelmed when faced with many choices? Whether it’s deciding between buying a bubble tea or a book, or even selecting a new drug for clinical trials, these dilemmas point to a common mathematical problem: the “multi-armed bandit problem”.

In a recent study, Dr Keqin Liu from Xi’an Jiaotong-Liverpool University’s (XJTLU) Department of Financial and Actuarial Mathematics has found an efficient new way to solve this classic challenge. His research has been published in the journal Management Science.

The core problem: How to choose?

Imagine you’re at a row of slot machines and don’t know which one will pay out. If you keep playing the machine that’s currently giving you the most money (exploitation), you might miss out on other, better machines. But if you constantly try new machines (exploration), you might reduce your immediate winnings. This balancing act between “exploitation” and “exploration” is common in many fields, such as medical trials, advertising, and drone scheduling.

Dr Liu explains: “People often struggle with this balance when making decisions. Doctors must choose between proven treatments and new drugs; e-commerce platforms weigh promoting bestsellers against testing new products. Our goal is to find the best strategies to maximise long-term returns.”

No more “choice overload”

Traditional methods for calculating this kind of problem face a problem called the “curse of dimensionality” when there are too many choices. This means that the more options there are, the more complex calculations become, growing exponentially. For example, just ten choices could lead to millions of combinations, which can overwhelm standard algorithms.

Dr Liu’s key innovation is the concept of “relaxed indexability.” This method simplifies the problem by relaxing strict strategy rules. He uses an analogy: “It’s like organising a wardrobe: we don’t calculate every possible outfit combination. Instead, we categorise by season and function, so we can quickly find the best matches.” By dynamically adjusting priorities, his method breaks down complex problems into smaller, easier-to-handle parts, significantly reducing computation costs.

Dr Liu says: “This research isn’t just theoretical; it offers a mathematical solution for real-life decision making scenarios.” His algorithm has already shown promise in areas like dynamic pricing and drone control, and it has the potential for broader use in intelligent decision-making systems. He says: “Whether in personal life or business management, the logic for optimising decisions is the same: finding a balance between what is known and what is unknown.”

About the researcher

Dr Liu has published over 40 papers in top journals and has more than 2,600 citations of his work in connecting mathematics and artificial intelligence. In January 2025, he also received the 2024 SIP International Leading Talent Award (Youth category). This study, “Relaxed Indexability and Index Policy for Partially Observable Restless Bandits”, is published in Management Science, a journal ranked among the top in the field of management and operations research.

By Qinru Liu

Edited by Catherine Diamond