One-shot, Offline and Production-Scalable PID Optimisation with Deep Reinforcement Learning

Voices Powered byElevenlabs logo
Connected to paperThis paper is a preprint and has not been certified by peer review

One-shot, Offline and Production-Scalable PID Optimisation with Deep Reinforcement Learning


Zacharaya Shabka, Michael Enrico, Nick Parsons, Georgios Zervas


Proportional-integral-derivative (PID) control underlies more than $97\%$ of automated industrial processes. Controlling these processes effectively with respect to some specified set of performance goals requires finding an optimal set of PID parameters to moderate the PID loop. Tuning these parameters is a long and exhaustive process. A method (patent pending) based on deep reinforcement learning is presented that learns a relationship between generic system properties (e.g. resonance frequency), a multi-objective performance goal and optimal PID parameter values. Performance is demonstrated in the context of a real optical switching product of the foremost manufacturer of such devices globally. Switching is handled by piezoelectric actuators where switching time and optical loss are derived from the speed and stability of actuator-control processes respectively. The method achieves a $5\times$ improvement in the number of actuators that fall within the most challenging target switching speed, $\geq 20\%$ improvement in mean switching speed at the same optical loss and $\geq 75\%$ reduction in performance inconsistency when temperature varies between 5 and 73 degrees celcius. Furthermore, once trained (which takes $\mathcal{O}(hours)$), the model generates actuator-unique PID parameters in a one-shot inference process that takes $\mathcal{O}(ms)$ in comparison to up to $\mathcal{O}(week)$ required for conventional tuning methods, therefore accomplishing these performance improvements whilst achieving up to a $10^6\times$ speed-up. After training, the method can be applied entirely offline, incurring effectively zero optimisation-overhead in production.

Follow Us on

1 comment


Possibly useful background for non-expert audience deciphering the acronym in the title: The proportional-integral-derivative (PID) controller is a widely used control algorithm in industrial process control and automation systems. It combines three essential components - proportional (P), integral (I), and derivative (D) control actions - to provide an effective way to maintain the desired output of a system in response to changes in input or external disturbances. The proportional component adjusts the control output based on the present error between the desired setpoint and the actual process variable. The integral component accumulates past errors over time and adjusts the control output accordingly, helping to eliminate steady-state error. Lastly, the derivative component anticipates future error based on the rate of change of the error, improving the system's ability to react to sudden changes. By tuning the three PID parameters, the controller can be optimized for various applications, providing precise and stable control of dynamic systems.
ScienceCast Board

Add comment