What is Physics Enhanced Machine Learning (PEML)?

Physics Enhanced Machine Learning is the only way to create an expert system for the task of detailed, workable and realizable engineering design, based on CAE and simulation. Whilst at the same time restricting the upfront training effort to something manageable that does not require a super-computer and tens thousands of individual simulation datapoints.

• The Problem with Traditional AI: Standard Artificial Neural Networks (ANNs) are not ideal for complex turbomachinery design because they require an enormous amount of training data (tens of thousands of CFD simulations) and can "hallucinate" physically impossible solutions, such as mass spontaneously appearing.
• The Flaws of PINNs: Physics Informed Neural Networks (PINNs) fix the hallucination problem by embedding physics equations into the network, but they are notoriously difficult to train, struggle to capture sharp flow changes (spectral bias), and fail to generalize well to new scenarios.
• The Solution - PEML: Physics Enhanced Machine Learning (PEML), developed by Advanced Design Technology (ADT), is the ideal solution for creating expert engineering design systems. Instead of relying on vast datasets, it uses 3D inverse design where the inputs control the physics of the flow (like blade loading), and the system outputs the corresponding blade shapes.
• Key Benefits of PEML:
  • Physics-Bound: The system cannot hallucinate because the relationship between the inputs and outputs is strictly bound by physics, meaning every output candidate is valid.
  • Small Data Overhead: It only requires 10-15 input parameters and a highly targeted training dataset of fewer than 70 candidate designs (small input tensor)
  • Fast and Optimal: Using the small dataset, PEML constructs a simple N-dimensional surrogate model that can be incredibly quickly searched to find optimal 3D turbomachinery designs.

• Why not use an Artificial Neural Network (ANN)?

• How to reduce the data bottleneck?

• Why not use Physics Informed Neural Networks?

• Why Physics-Enhanced Machine Learning is the Solution

Why not use an Artificial Neural Network (ANN)?

Artificial Neural Networks (ANN) are the architecture that underpins most AI models, including Large Language Models (LLMs) like Claude and Gemini.

This type of AI seems to have the answer to many problems, but can it be applied to the problem of designing something as complex as a turbomachine? In theory, instead of using the internet and all it’s text and image content as training data, one could feed in CFD simulations, and according to the architecture of an Artificial Neural Network (ANN), with enough training data the network should be able to predict the performance of any given blade geometry under any given operating conditions.

There are immediately two intractable issues with applying this in principle. Firstly the enormous amount of training data (in the form of CFD simulations) that need to be generated in order to train the model. Thinking about the number of geometric parameters alone, and the curse of factorial dimensionality that turns the design space into a multi-dimensional giant, plus multiple operating points, plus the non-linearity of fluid flow means that many thousands of training points need to be created. Which does not look like a good value proposition when compared to finding an improved design through simple trial and error. Secondly the model is still predicting the performance of a given blade shape. So the shape is the input and the required performance is the output. This is the wrong way around. We should be able to define the performance characteristic that we want, and the system creates the blade shape that delivers that.

ANNs also have no guardrails against ‘hallucination’. Just as LLMs can create images of hands with 6 fingers, even a well trained ANN does not ‘know’ that flow cannot leave through a wall, that a boundary layer must develop according to the law of physics, or mass/energy cannot spontaneously appear/disappear within a fluid continuum. Thus such impossible solutions can be manifest, and the model may think that somehow this is the ‘best’ solution. 

 

Figure 1: ‘Traditional’ ANN architecture relies on tens of thousands of training datapoints, but can still ‘hallucinate’ complex flow physics

How to reduce the data bottleneck?

In their 2021 paper ‘Physics informed machine learning’, Karinadakis et al posited a simple framework where adding physics into the model should be able to reduce the amount of training data required.

 

Picture 2 : Ref: Karinadakis et al, Physics informed machine learning, Nature Reviews Physics, Vol 3, P423, 2021

There are two ways to broadly achieve this :

1. Adding the physics into the ANN, so called Physics Informed Neural network (PINN)
2. Adding the physics in the parametrization of the input data - so called Physics Enhanced Machine Learning. (PEML)

Why not use Physics Informed Neural Networks?

Physics Informed Neural Networks (PINN) seem to have a solution to this problem, by embedding the governing equations into the nodal weights at randomly scattered ‘co-location points’ thought the fluid domain, PINNs do not suffer from hallucinations or impossible physics that arise in ANNs. However PINNs do suffer from a phenomenon called spectral bias. By their mathematical nature, PINNs prefer to learn smooth, low-frequency patterns first and struggle to capture sharp, high-frequency changes seen in turbulent boundary layers and transonic flows. 

PINNs are notoriously unstable and difficult to train as different loss functions  (L_data, L_physics, L_boundary, etc) pull the optimizer for hyperparameters to different directions. This creates a phenomenon called gradient pathology. The network gets confused because satisfying the boundary condition might temporarily make the physics error worse, or vice versa. The loss landscape becomes incredibly jagged and stiff, causing the training process to get stuck in "local minima" where the network just stops learning before it finds the correct physical solution.

Trained PINNs are also incredibly specific and do not generalise well. A standard PINN is not a solver; it is an approximator tailored to a specific setup. If you train a PINN to predict the flow through a blade passage at a given speed, it essentially "memorizes" the solution for that exact scenario. If you want to know what happens if the fluid speed increases by 10%, or if you change the cylinder to an oval, you cannot just adjust a setting. You often have to retrain the entire neural network from scratch.

Researchers are aware of these shortcomings and are actively developing solutions. Techniques like Fourier Features (to force the network to see high frequencies), dynamic loss weighting (to balance the competing equations), and next-generation architectures like Neural Operators (which learn to map entire families of equations rather than single instances) are pushing PINNs closer to real-world viability, but they are not there yet.

 

Figure 3: PINN architecture seems to have a solution to the hallucination problem, but is not good at resolving large gradients in a flow field

Why Physics-Enhanced Machine Learning is the Solution

The real solution to creating expert engineering design systems is Physics Enhanced Machine Learning (PEML). This is a system developed by Advanced Design Technology (ADT) where the inputs to the problem are controls on the physics of the flow in the blade passage. The outputs are blade shapes. Thus what we really want from the design (flow, performance, speed etc) is a controlled input value, the solutions (the blade geometry) just attempt to answer that requirement in different ways. The system can’t hallucinate because the process of getting outputs from the input is physics-bound, all variations on the input recipe result in a valid output candidate and the system just needs to find the most efficient or multi-objectively optimal (‘Pareto-optimal’).

ADT’s Physics Enhanced Machine Learning leverages the power of 3D inverse design (see See our previous blog: "Blade Loading, Inverse Design and Shape Parameterisation”) which means that input parameters do not encode geometry variability (i.e. streamwise blade angle distribution at spanwise sections), but rather encode aerodynamic variation for a given performance target (e.g. pump head or compressor pressure ratio). This means that just a handful of parameters are required to fully describe the blade design space (basically how much work is being done by the blade, and where in the blade passage is that work being done.) Typically 10-15 input parameters is sufficient to fully capture enough variation to find optimal solutions.

Crucially, Physics Enhanced Machine Learning does not need a large training dataset. Because of the efficiency of the 3D Inverse Design parameters and the fact that the system, as it selects training point candidates, is ‘physics-aware’ typically less that 70 candidate designs are required. Once the training data is assembled, the system constructs a surrogate model - which is essentially just an N-dimensional surface fit over the data. Finally the surrogate model is searched (this is incredibly quick, as we are using algebraic surrogate data) for the global optima. A more detailed description of the method - known as the Reactive Response Surface (RRS) is here : “What is Reactive Response Surface”.

 

Figure 4: PEML requires a small training dataset and cannot hallucinate

Thus the model itself is simple (an n-dimensional response surface surrogate model, fitted to a highly selective and targeted set of datapoints). But the physics encoded in the input tensor represents all the complexity of a 3D turbomachinery flow field. This is what gives Physics Enhanced Machine Learning its unrivalled power to create expert systems for turbomachinery design.

See examples of PEML in action here:

https://blog.adtechnology.com/machine-learning-wastewater-pump-design

https://blog.adtechnology.com/machine-learning-centrifugal-compressor-design

https://blog.adtechnology.com/machine-learning-hydraulic-turbine-francis-runner-design-optimization