Throwing down the scientific gauntlet to assess methods for anomalous diffusion

Almost 80 years after Scottish botanist Robert Brown described the continuous random motion of microscopic particles in a fluid, Albert Einstein provided a theoretical foundation for this observation. Since then, scientists have discovered systems that deviate substantially from the laws of Brownian motion. Such deviations are referred to as anomalous diffusion and occur in a wide range of systems, ranging from the transport of molecules in the nucleus to animal foraging strategies and stock market fluctuations.

Over time, scientists developed various methods for understanding anomalous diffusion using classical statistics. However, advances in machine learning recently gave birth to more sophisticated tools, with data-based methods that characterize anomalous diffusion from single trajectories.

Now, researchers supported by the EU-funded NOQIA, OPTOlogic and ComplexSwimmers projects have carried out the first objective comparison of traditional and novel methods used to decode anomalous diffusion in various realistic conditions. The results were published in the journal Nature Communications.

The challenge

To compare the different methods, the research team gathered the scientific community and organized an open competition called the Anomalous Diffusion (AnDi) challenge. Participating teams applied their algorithms to a common data set, including diverse conditions.

The AnDi challenge consisted of three tasks. The first task entailed inferring the anomalous diffusion exponent, the second required participants to classify the underlying diffusion model and the final one involved trajectory segmentation. Each task was then divided into three subtasks corresponding to the trajectory dimensions: 1D, 2D and 3D.

According to the study, although no particular method performed the best across all scenarios, machine learning-based approaches surpassed conventional methods for all tasks. “For each dimension, we could identify a group of methods with comparable performance that greatly improve the precision of the anomalous diffusion exponent with respect to the baseline provided by the classical estimation of the MSD,” the study authors write. MSD stands for mean squared displacement. “These approaches were all based on machine learning, so we can infer that machine-learning-based methods can go beyond classical statistics, probably because they can extract from the trajectories of complex models some information that is not easily assessed by classical statistics.”

The study’s achievements highlight the importance of community-based efforts in the quest to advance science. In the hope that their effort will be extended to reach a larger consensus, the research team has built an interactive tool for storing data sets and the results of the challenge. Scientists can benchmark new methods according to the challenge’s rules and compare scores with those of other challenge participants. The AnDi challenge will remain open permanently, with continuous updates of performance improvements on demand.

Senior author Dr. Carlo Manzo commented on the study’s findings in a news release: “The results of this study further highlight the central role that anomalous diffusion has in defining biological functions at multiple scales while revealing insight into the current state of the field and providing a benchmark for future developers.” Dr. Manzo is a visiting scientist at NOQIA (NOvel Quantum simulators—connectIng Areas) and OPTOlogic (Optical Topologic Logic) projects coordinator Institute of Photonic Sciences, Barcelona. The ComplexSwimmers (Biocompatible and Interactive Artificial Micro- and Nanoswimmers and Their Applications) project is coordinated by the University of Gothenburg in Sweden.