Optimization and Control (math.OC)

Abstract: A broad class of optimization problems can be cast in composite form, that is, considering the minimization of the composition of a lower semicontinuous function with a differentiable mapping. This paper discusses the versatile template of composite optimization without any convexity assumptions. First- and second-order optimality conditions are discussed, advancing the variational analysis of compositions. We highlight the difficulties that stem from the lack of convexity when dealing with necessary conditions in a Lagrangian framework and when considering error bounds. Building upon these characterizations, a local convergence analysis is delineated for a recently developed augmented Lagrangian method, deriving rates of convergence in the fully nonconvex setting.

Abstract: In this article, the feasibility of using optimal control theory will be studied to develop control theoretic methods for personalized treatment of HCV patients. The mathematical model for HCV progression includes compartments for healthy hepatocytes, infected hepatocytes, infectious virions and noninfectious virions. Methodologies have been used from optimal control theory to design and synthesize an open-loop control based treatment regimen for HCV dynamics.

Abstract: This paper introduces PROMISE ($\textbf{Pr}$econditioned Stochastic $\textbf{O}$ptimization $\textbf{M}$ethods by $\textbf{I}$ncorporating $\textbf{S}$calable Curvature $\textbf{E}$stimates), a suite of sketching-based preconditioned stochastic gradient algorithms for solving large-scale convex optimization problems arising in machine learning. PROMISE includes preconditioned versions of SVRG, SAGA, and Katyusha; each algorithm comes with a strong theoretical analysis and effective default hyperparameter values. In contrast, traditional stochastic gradient methods require careful hyperparameter tuning to succeed, and degrade in the presence of ill-conditioning, a ubiquitous phenomenon in machine learning. Empirically, we verify the superiority of the proposed algorithms by showing that, using default hyperparameter values, they outperform or match popular tuned stochastic gradient optimizers on a test bed of $51$ ridge and logistic regression problems assembled from benchmark machine learning repositories. On the theoretical side, this paper introduces the notion of quadratic regularity in order to establish linear convergence of all proposed methods even when the preconditioner is updated infrequently. The speed of linear convergence is determined by the quadratic regularity ratio, which often provides a tighter bound on the convergence rate compared to the condition number, both in theory and in practice, and explains the fast global linear convergence of the proposed methods.

Abstract: We explore the class of trilevel equilibrium problems with a focus on energy-environmental applications. In particular, we apply this trilevel framework to a power market model, exploring the possibilities of an international policymaker in reducing emissions of the system. We present two alternative solution methods for such problems and a comparison of the resulting model sizes. The first method is based on a reformulation of the bottom-level solution set, and the second one uses strong duality. The first approach results in optimality conditions that are both necessary and sufficient, while the second one results in a model with fewer constraints but only sufficient optimality conditions. Using the proposed methods, we are able to obtain globally optimal solutions for a realistic five-node case study representing the Nordic countries and assess the impact of a carbon tax on the electricity production portfolio.

Abstract: Stochastic optimization methods have been hugely successful in making large-scale optimization problems feasible when computing the full gradient is computationally prohibitive. Using the theory of modified equations for numerical integrators, we propose a class of stochastic differential equations that approximate the dynamics of general stochastic optimization methods more closely than the original gradient flow. Analyzing a modified stochastic differential equation can reveal qualitative insights about the associated optimization method. Here, we study mean-square stability of the modified equation in the case of stochastic coordinate descent.

Abstract: We introduce a finite dimensional version of backstepping controller design for stabilizing solutions of PDEs from boundary. Our controller uses only a finite number of Fourier modes of the state of solution, as opposed to the classical backstepping controller which uses all (infinitely many) modes. We apply our method to the reaction-diffusion equation, which serves only as a canonical example but the method is applicable also to other PDEs whose solutions can be decomposed into a slow finite-dimensional part and a fast tail, where the former dominates the evolution in large time. One of the main goals is to estimate the sufficient number of modes needed to stabilize the plant at a prescribed rate. In addition, we find the minimal number of modes that guarantee the stabilization at a certain (unprescribed) decay rate. Theoretical findings are supported with numerical solutions.

Abstract: How can one lift a functional defined on maps from a space X to a space Y into a functional defined on maps from X into P(Y) the space of probability distributions over Y? Looking at measure-valued maps can be interpreted as knowing a classical map with uncertainty, and from an optimization point of view the main gain is the convexification of Y into P(Y). We will explain why trying to single out the largest convex lifting amounts to solve an optimal transport problem with an infinity of marginals which can be interesting by itself. Moreover we will show that, to recover previously proposed liftings for functionals depending on the Jacobian of the map, one needs to add a restriction of additivity to the lifted functional.

Abstract: We present a semi-real-time algorithm for minimal-time optimal path planning based on optimal control theory, dynamic programming, and Hamilton-Jacobi (HJ) equations. Partial differential equation (PDE) based optimal path planning methods are well-established in the literature, and provide an interpretable alternative to black-box machine learning algorithms. However, due to the computational burden of grid-based PDE solvers, many previous methods do not scale well to high dimensional problems and are not applicable in real-time scenarios even for low dimensional problems. We present a semi-real-time algorithm for optimal path planning in the HJ formulation, using grid-free numerical methods based on Hopf-Lax formulas. In doing so, we retain the intepretablity of PDE based path planning, but because the numerical method is grid-free, it is efficient and does not suffer from the curse of dimensionality, and thus can be applied in semi-real-time and account for realistic concerns like obstacle discovery. This represents a significant step in averting the tradeoff between interpretability and efficiency. We present the algorithm with application to synthetic examples of isotropic motion planning in two-dimensions, though with slight adjustments, it could be applied to many other problems.

Abstract: In this work, we develop first-order (Hessian-free) and zero-order (derivative-free) implementations of the Cubically regularized Newton method for solving general non-convex optimization problems. For that, we employ finite difference approximations of the derivatives. We use a special adaptive search procedure in our algorithms, which simultaneously fits both the regularization constant and the parameters of the finite difference approximations. It makes our schemes free from the need to know the actual Lipschitz constants. Additionally, we equip our algorithms with the lazy Hessian update that reuse a previously computed Hessian approximation matrix for several iterations. Specifically, we prove the global complexity bound of $\mathcal{O}( n^{1/2} \epsilon^{-3/2})$ function and gradient evaluations for our new Hessian-free method, and a bound of $\mathcal{O}( n^{3/2} \epsilon^{-3/2} )$ function evaluations for the derivative-free method, where $n$ is the dimension of the problem and $\epsilon$ is the desired accuracy for the gradient norm. These complexity bounds significantly improve the previously known ones in terms of the joint dependence on $n$ and $\epsilon$, for the first-order and zeroth-order non-convex optimization.

Abstract: The phase-field method is based on the energy minimization principle which is a geometric method for modeling diffusive cracks that are popularly implemented with irreversibility based on Griffith's criterion. This method requires a length-scale parameter that smooths the sharp discontinuity, which influences the diffuse band and results in mesh-sensitive fracture propagation results. Recently, a novel approach based on the optimization on Riemannian shape spaces has been proposed, where the crack path is realized by techniques from shape optimization. This approach requires the shape derivative, which is derived in a continuous sense and used for a gradient-based algorithm to minimize the energy of the system. Due to the continuous derivation of the shape derivative, this approach yields mesh-independent results. In this paper, the novel approach based on shape optimization is presented, followed by an assessment of the predicted crack path in anisotropic brittle material using numerical calculations from a phase-field model.