報告人：張進 副教授 南方科技大學數學系/深圳國家應用數學中心
報告時間：2024年10月25日（周五）下午 2:00-3:00
報告地點：bet356手机版唯一官网九龍湖校區計算機樓513室
報告摘要：Gradient methods have become mainstream techniques for Bi-Level Optimization (BLO) in learning fields. The validity of existing works heavily rely on either a restrictive Lower-Level Strong Convexity (LLSC) condition or on solving a series of approximation subproblems with high accuracy or both. In this work, by averaging the upper and lower level objectives, we propose a single loop Bi-level Averaged Method of Multipliers (slBAMM) for BLO that is simple yet efficient for large-scale BLO and gets rid of the limited LLSC restriction. We further provide non-asymptotic convergence analysis of sl-BAMM towards KKT stationary points, and the comparative advantage of our analysis lies in the absence of strong gradient boundedness assumption, which is always required by others. Thus our theory safely captures a wider variety of applications in deep learning, especially where the upper-level objective is quadratic w.r.t. the lower-level variable. Experimental results demonstrate the superiority of our method.
報告人簡介：張進，南方科技大學數學系/深圳國家應用數學中心副教授，博士畢業于加拿大維多利亞大學，緻力于最優化理論和應用研究，代表性成果發表在《Math. Program.》、《SIAM J. Optim.》、《Math. Oper. Res.》、《SIAM J. Numer. Anal.》、《J. Mach. Learn. Res.》、《IEEE TPAMI》，以及ICML、NeurIPS、ICLR等有重要影響力的最優化、計算數學、機器學習期刊與會議上。研究成果獲得中國運籌學會青年科技獎、廣東省青年科技創新獎，主持國家自然科學基金優青、天元重點、面上項目、廣東省自然科學基金傑青項目、深圳市科技創新培養人才優青項目、以及科技部重點研發計劃“數學與應用數學”專項課題。
