Graduate Seminar with Keivan Ebrahimi: Saddle Dynamics for Solving Robust Optimization and Adversarially Robust Learning Problems

When

April 10, 2019    
1:10 pm - 2:00 pm

Where

3043 ECpE Building Addition
Coover Hall, Ames, Iowa, 50011

Event Type

Speaker: Keivan Ebrahimi, ECpE Graduate Student

Advisor: Umesh Vaidya

Title: Saddle Dynamics for Solving Robust Optimization and Adversarially Robust Learning Problems

Abstract: We propose a dynamical system-based approach for solving robust optimization problems with applications in adversarially robust machine learning e.g. image classification in the presence of worst-case perturbations. The well-known continuous-time dynamical system for solving deterministic optimization problems arises in the form of primal-dual gradient dynamics where the vector field is derived as the gradient of the Lagrangian. The new continuous-time dynamical system and its discrete-time counterpart we introduce for solving robust optimization problems differ from the primal-dual dynamics in the sense that the vector field is not derived as the gradient of the Lagrangian function. We call this new dynamical system as saddle point dynamics. In the saddle point dynamics, the uncertain variable arises as a dynamical state. For a general robust convex optimization problem, we show that the optimal solution can be recovered as a globally asymptotically stable equilibrium point of the saddle point dynamical system. One of the distinguishing features of the proposed subgradient algorithm is that it can be implemented in a distributed manner. Under the assumption that the cost function is convex and uncertainties enter concavely in the problem, we prove that the discrete-time subgradient algorithm converges asymptotically to the robust optimal solution under diminishing step size. Furthermore, under the assumption of constant step-size, we show that the subgradient algorithm will converge to the arbitrary small neighborhood of robust optimal solution at a linear rate. Simulation results like a robust image classification problem with adversarial perturbations are presented to demonstrate the capability of this new dynamical system to solve various robust optimization e.g. robust learning problems. We also compare our proposed approach with existing methods based on robust counterpart and scenario-based random sampling.

Loading...