Nico Wilke, a professor in the Department of Mechanical and Aeronautical Engineering in the University of Pretoria’s Faculty of Engineering, Built Environment and Information Technology, delivered his inaugural address on 8 November 2022. The topic of his address was: “When nothing seems to go right, why not go left? In search of alternative perspectives and paradigms”.

His research interests include state-of-the-art digital twin deployment based on data-driven generative modelling efficiently trained using gradient-only optimisation, and physics-based generative modelling using efficiently solved particle-based solvers on graphical processing units.

Prof Wilke aligned his perspective on research with that of the Hungarian biochemist and Nobel Prize winner, Albert Szent-Gyorgyi, who stated: “Research is to see what everybody else has seen, and to think what nobody else has thought”. Prof Wilke offered a classical viewpoint of three categories of problems researchers typically encounter: problems with many solutions; problems with only one solution; and problems with no solution. He presented an alternative viewpoint, stating that a problem’s category is not necessarily fixed. This epiphany inspires a researcher to search for alternative perspectives and paradigms.

Prof Wilke professed five lessons learnt in his search for alternative perspectives and paradigms, and how these alternatives have impacted research.

He started his presentation by exploring how one can learn from one’s mistakes. He discovered this when he was confronted, for the first time, by two unwitting inconsistent algorithmic formulations prevalent in particle swarm optimisation research. Instead of ignoring the existence of the two formulations, he realised that the fundamental differences between them resulted in the identification of a new class of particle swarm optimisation optimisers.

In computational mechanics, Prof Wilke compared the maturity of mesh-based and particle-based solvers with the industry’s modelling needs. Evidently, the second-most processed substance by mass on the planet after water is granular materials, and yet the computational maturity of modelling granular materials up to 2014 was largely limited to two-dimensional simulations of a few thousand spherical particles. By embracing new computing technology offered by Nvidia graphical processing units, a paradigm shift resulted in discrete element modelling that enabled three-dimensional simulations using tens of millions of faceted particles. In 2022, all major commercial discrete element modelling software now support graphical processing compute and faceted particle shapes.

Quoting the British statistician, George Box, who stated that “all models are wrong, but only some are useful”, Prof Wilke reflected on the sequence of events that are required to make “models useful”. He remarked that one of the essential ingredients – parameter identifiability with subsequent parameter identification from experimental data – is often neglected. The implication of neglecting this is that models cannot be used to predict the unseen, but only to reproduce what has been seen. Prof Wilke illustrated how vices of laziness and frugality inspired virtual calibration by only using what is available, and essentially developing a shortcut to properly quantify the usefulness of models, as well as guiding what additional experimental data is required to make them more useful. He remarked how some vices, such as frugality, are steadily becoming traits as we are increasingly judged not only on what we have achieved but on what resources we required to get to those accomplishments.

In highlighting the computational cost of up to millions of dollars to train machine and deep learning models, he contrasted it to the elementary nature of choosing a priori-selected learning rates or learning rate schedules. The choice of learning rate plays a dominant role in the cost of training. A potentially more efficient approach that adaptively resolves learning rates is line searches, which have gained almost no traction in machine learning training. The main reason is that, during training, only a few samples are used to evaluate the loss and gradient functions. To reduce the bias, the samples are changed every time the loss and gradient functions are evaluated. The consequence is that the loss function is point-wise discontinuous, negating the developed optimisation theory and approaches of Kepler, De Fermat, Newton, Leibnitz, Euler, Lagrange, Gauss, Legendre, Cauchy and Dantzig. Prof Wilke demonstrated how a simple and elegant approach that only considers gradients enable the utilisation of line searches by resolving non-negative gradient projection points as opposed to minimising along a search direction. Gradient-only optimisation essentially enables the potential of line searches to be investigated and is showing significant promise in reducing the computational cost of machine and deep learning training.

Drawing from his research on gradient-only optimisation, Prof Wilke illustrated how gradients are transforming other domains, with specific reference to the development of deep digital twins from sensor data and high-dimensional surrogates. He highlighted the importance of the generalisation of concepts and ideas to harness commonalities in transdisciplinary research that can lead to the transfer of perspectives to circumvent fundamental stumbling blocks across fields and disciplines.

He concluded his address by stating that academia needs to value ideas and thinking again, as it is only by continuously considering existing knowledge from new perspectives that researchers can ensure sustainable growth in science.