Optimization and Control (math.OC)
Mon, 29 May 2023
1.Adaptive Localized Cayley Parametrization for Optimization over Stiefel Manifold
Authors:Keita Kume, Isao Yamada
Abstract: We present an adaptive parametrization strategy for optimization problems over the Stiefel manifold by using generalized Cayley transforms to utilize powerful Euclidean optimization algorithms efficiently. The generalized Cayley transform can translate an open dense subset of the Stiefel manifold into a vector space, and the open dense subset is determined according to a tunable parameter called a center point. With the generalized Cayley transform, we recently proposed the naive Cayley parametrization, which reformulates the optimization problem over the Stiefel manifold as that over the vector space. Although this reformulation enables us to transplant powerful Euclidean optimization algorithms, their convergences may become slow by a poor choice of center points. To avoid such a slow convergence, in this paper, we propose to estimate adaptively 'good' center points so that the reformulated problem can be solved faster. We also present a unified convergence analysis, regarding the gradient, in cases where fairly standard Euclidean optimization algorithms are employed in the proposed adaptive parametrization strategy. Numerical experiments demonstrate that (i) the proposed strategy succeeds in escaping from the slow convergence observed in the naive Cayley parametrization strategy; (ii) the proposed strategy outperforms the standard strategy which employs a retraction.
2.Communication Efficient Distributed Newton Method with Fast Convergence Rates
Authors:Chengchang Liu, Lesi Chen, Luo Luo, John C. S. Lui
Abstract: We propose a communication and computation efficient second-order method for distributed optimization. For each iteration, our method only requires $\mathcal{O}(d)$ communication complexity, where $d$ is the problem dimension. We also provide theoretical analysis to show the proposed method has the similar convergence rate as the classical second-order optimization algorithms. Concretely, our method can find~$\big(\epsilon, \sqrt{dL\epsilon}\,\big)$-second-order stationary points for nonconvex problem by $\mathcal{O}\big(\sqrt{dL}\,\epsilon^{-3/2}\big)$ iterations, where $L$ is the Lipschitz constant of Hessian. Moreover, it enjoys a local superlinear convergence under the strongly-convex assumption. Experiments on both convex and nonconvex problems show that our proposed method performs significantly better than baselines.
3.A Parameter-Free Conditional Gradient Method for Composite Minimization under Hölder Condition
Authors:Masaru Ito, Zhaosong Lu, Chuan He
Abstract: In this paper we consider a composite optimization problem that minimizes the sum of a weakly smooth function and a convex function with either a bounded domain or a uniformly convex structure. In particular, we first present a parameter-dependent conditional gradient method for this problem, whose step sizes require prior knowledge of the parameters associated with the H\"older continuity of the gradient of the weakly smooth function, and establish its rate of convergence. Given that these parameters could be unknown or known but possibly conservative, such a method may suffer from implementation issue or slow convergence. We therefore propose a parameter-free conditional gradient method whose step size is determined by using a constructive local quadratic upper approximation and an adaptive line search scheme, without using any problem parameter. We show that this method achieves the same rate of convergence as the parameter-dependent conditional gradient method. Preliminary experiments are also conducted and illustrate the superior performance of the parameter-free conditional gradient method over the methods with some other step size rules.
4.Necessary and sufficient conditions for unique solvability of absolute value equations: A Survey
Authors:Shubham Kumar, Deepmala
Abstract: In this survey paper, we focus on the necessary and sufficient conditions for the unique solvability and unsolvability of the absolute value equations (AVEs) during the last twenty years (2004 to 2023). We discussed unique solvability conditions for various types of AVEs like standard absolute value equation (AVE), Generalized AVE (GAVE), New generalized AVE (NGAVE), Triple AVE (TAVE) and a class of NGAVE based on interval matrix, P-matrix, singular value conditions, spectral radius and $\mathcal{W}$-property. Based on the unique solution of AVEs, we also discussed unique solvability conditions for linear complementarity problems (LCP) and horizontal linear complementarity problems (HLCP).