报告题目: Optimal Control of Entropic-Stochastic Gradient Descent: a PDE approach
报告人：Hicham Kouhkouh（University of Graz）
邀请人&主持人：黄辉
报告时间与地点：6月23日 15:00-16:00， 数学学院425
报告摘要: Entropic variants of stochastic gradient descent have been proposed as effective methods for exploring complex optimization landscapes arising in the training of deep neural networks.
In this talk, I will present a PDE and stochastic-control framework for the analysis of a class of singularly perturbed stochastic systems that includes the entropic-stochastic gradient descent model introduced by Chaudhari et al. (Res. Math. Sci. 2018). The dynamics exhibit a two-scale structure, with fast and slow variables interacting through controlled stochastic differential equations. Such systems naturally lead to Hamilton-Jacobi-Bellman (HJB) equations with unbounded data, raising significant analytical challenges.
The main results concern the rigorous derivation of effective dynamics in the singular perturbation limit. Using techniques from stochastic control, viscosity solutions, and homogenization theory, we establish the convergence of the associated value functions to the solution of an effective HJB equation and characterize the corresponding effective Hamiltonian. Furthermore, the limiting value function is shown to admit a control-theoretic interpretation as the value function of an effective optimization problem with extended controls. We also discuss the convergence of trajectories and the implications of the resulting effective model for entropic gradient-based optimization algorithms.
This talk is based on two joint works with Martino Bardi from University of Padova, in (ESAIM Control Optim. Calc. Var. 2023) and in (SIAM J. Control Optim. 2024).
报告人介绍：Hicham Kouhkouh received his Ph.D. degree in Mathematics from the University of Padova, Italy, in 2022. He holds an M.Sc. degree in Mathematics with a specialization in Optimization from Paris-Sud University (now part of Université Paris-Saclay), and an engineering degree from ENSTA Paris, both obtained in 2018. From 2022 to 2023, he was a postdoctoral researcher at RWTH Aachen University, Germany. He is currently a University Assistant in the Department of Mathematics and Scientific Computing at the University of Graz, Austria. His research interests include optimal control theory, optimization, and applied mathematics.
