Book

• Introduction to Derivative Free Optimization with A. R. Conn and L. N. Vicente. Available from SIAM Series on Mathematical Programming. December 2008 / approx. xii + 277 pages / Softcover / ISBN: 978-0-898716-68-0. Errata

Selected Papers

• Stochastic ISTA/FISTA Adaptive Step Search Algorithms for Convex Composite Optimization, with Trang H. Tran and Lam M. Nguyen, 2024.
• Stochastic Adaptive Regularization Method with Cubics: A High Probability Complexity Bound, with Miaolan Xie, Winter Simulation Conference 2023.
• Sample Complexity Analysis for Adaptive Optimization Algorithms with Stochastic Oracles, with Billy Jin and Miaolan Xie, to appear in Mathematical Programming.
• Finite Difference Gradient Approximation: To Randomize or Not? INFORMS Journal on Computing, 2022.
• Nesterov Accelerated Shuffling Gradient Method for Convex Optimization. with Trang H. Tran and Lam M. Nguyen, ICML 2022: 21703-21732.
  
• First- and Second-Order High Probability Complexity Bounds for Trust-Region Methods with Noisy Oracles,  with Albert S Berahas and Liyuan Cao, to appear in Mathematical Programming.
• Global Convergence Rate Analysis of a Generic Line Search Algorithm with Noise. with Albert S Berahas and Liyuan Cao, SIAM J. Optim. 31(2): 1489-1518 (2021)
• High Probability Complexity Bounds for Adaptive Step Search Based on Stochastic Oracles with Billy Jin and Miaolan Xie, to appear in SIAM J. Optim, (conference version NeurIPS 2021: 9193-9203)
• A Theoretical and Empirical Comparison of Gradient Approximations in Derivative-Free Optimization, with Albert S Berahas,  Liyuan Cao, and Krzysztof Choromanski, Found. Comput. Math. 22(2): 507-560 (2022).
• Inexact SARAH Algorithm for Stochastic Optimization, with Lam M. Nguyen and  Martin Takáč, Optim. Methods Softw. 36(1): 237-258 (2021)
• Optimal Generalized Decision Trees via Integer Programming, with O. Gunluk, M. Menickelly and J. Kalagnanam, J. Glob. Optim. 81(1): 233-260 (2021)

• A Stochastic Line Search Method with Expected Complexity Analysis, with Courtney Paquette, SIAM J. Optim. 30(1): 349-376 (2020)

• Adaptive Stochastic Optimization: A Framework for Analyzing Stochastic Optimization Algorithms, with Frank Curtis, IEEE Signal Process. Mag. 37(5): 32-42 (2020)
• Novel and Efficient Approximations for Zero-One Loss of Linear Classifiers with Hiva Ghanbari, and Minhan Li, 2019.
• New Convergence Aspects of Stochastic Gradient Algorithms, with Lam M. Nguyen, Phuong Ha Nguyen, Peter Richtárik,  Martin Takác and Marten van Dijk, J. Mach. Learn. Res. 20: 176:1-176:49 (2019)
  
• A Stochastic Trust Region Algorithm Based on Careful Step Normalization, with Frank Curtis and Rui Shi, INFORMS Journal on Optimization. 1(3), 200–220 (2019)

• Proximal Quasi-Newton Methods for Convex Optimization, with H. Ghanbari, Comput. Optim. Appl. 69(3): 597-627 (2018).
  

• Convergence Rate Analysis of a Stochastic Trust Region Method via Submartingales with Jose Blanchet, Coralia Cartis and Matt Menickelly,  2018.
• Stochastic Optimization Using a Trust-Region Method and Random Models, with R. Chen and M. Menickelly, Math. Program. 169(2): 447-487 (2018)
• Global convergence rate analysis of unconstrained optimization methods based on probabilistic models, with C. Cartis, Math. Program. 169(2): 337-375 (2018)

• Optimization Methods for Supervised Machine Learning: From Linear Models to Deep Learning, with  Frank E. Curtis, Informs TutORials, pages 89–114, (2017).

• Stochastic Recursive Gradient Algorithm for Nonconvex Optimization, with L. Nguyen, Jie Liu and M. Takac, Technical Report, (2017).

• Alternating direction methods for non convex optimization with applications to second-order least-squares and risk parity portfolio selection, with Xi Bai, 2015, technical report.
• Practical Inexact Proximal Quasi-Newton Method with Global Complexity Analysis, with Xiaocheng Tang, Mathematical Programming, 2016, 160(1-2) pp 495–529
• Least-squares approach to risk parity in portfolio selection, with X. Bai and R. Tutuncu,  Quantitative Finance, 2016, 16(3), pp 357-376.
• Aligning ligand binding cavities by optimizing superposed volume, with B. Chen and R. Chen, in BIBM 2012.
• Convergence of trust-region methods based on probabilistic models, with A. Bandeira and L.N. Vicente, SIOPT, 14(3), (2014), pp. 1238-1264.
• Fast first-order methods  for composite convex optimization with backtracking, with D. Goldfarb, FOCM, 2014, 14: 389-417.
• Efficient Block-coordinate Descent Algorithms for the Group Lasso.  with Z. Qin,  and D. Goldfarb. Math Prog. Comp., 2013, Volume 5, Issue 2, pp 143-169.
• Computation of sparse low degree interpolating polynomials and their application to derivative-free optimization, with A. Bandeira and L.N. Vicente, Math. Prog., Series B, (2012), 134, pp 223-257.
• On partially sparse recovery, with A. Bandeira and L.N. Vicente, 2011
• Fast alternating linearization methods for minimizing the sum of two convex functions, with D. Goldfarb and S. Ma, 2013, Math. Prog. Series A, 141: pp 349-382.
• Sparse Inverse Covariance Selection via Alternating Linearization Methods, with D. Goldfarb and S. Ma, NIPS 2010
• SINCO – a greedy coordinate ascent method for sparse inverse covariance selection problem, with I Rish, 2009
• Optimization Methods for Sparse Inverse Covariance Selection Problem. with S. Ma, In S. Sra, S. Nowozin, and S. J. Wright editors: Optimization for Machine Learning, MIT Press, 2010
• Row by row method for semidefinite programming, with Z. Wen, D. Goldfarb and S. Ma, submitted, 2009
• A Derivative-Free Algorithm for the Least-square minimization, with H. Zhang and A.R. Conn, submitted, 2009
• Self-correcting geometry in model-based algorithms for derivative-free unconstrained optimization with Ph. L. Toint, to appear, 2009.
• A MAP approach to learning sparse gaussian markov networks. with N. Bani Asadi, I. Rish, D. Kanevsky, B. Ramabhadran,  ICCASP 2009.
• Global Convergence of General Derivative-Free Trust-Region Algorithms to First and Second Order Critical Points ,  with A.R. Conn and L.N. Vicente, SIAM J. on Optimization, (2009).
• Geometry of Sample Sets in Derivative Free Optimization: polynomial regression and incomplete interpolation., with A.R. Conn and L.N. Vicente, IMA Journal of Numerical Analysis 2008 28(4):721-748.
• Geometry of Interpolation Sets in Derivative Free Optimization., with A.R. Conn and L.N. Vicente, Mathematical Programming, 111 (2008), 141-172.
• Product-Form LDL^T Factorizations in Interior-Point Methods for Convex Quadratic , with D. Goldfarb, IMA Journal of Numerical Analysis 2008 28(4):806-826.
• IBM Research TRECVID-2006 Video Retrieval System with M. Campbell, S. Ebadollahi, D. Joshi, M. Naphade, A. Natsev, J. Seidl, J. R. Smith, J. Tesic and L. Xie.
• Detecting Generic Visual Events with Temporal Cues. with Lexing Xie, Dong Xu, Shahram Ebadollahi, Shih-Fu Chang, John R. Smith. In Proc. 40th Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, October 2006.
• An Efficient Implementation of an Active Set Method for SVM, J. of Machine Learning Research 7 (2006) 2237-2257.
• (Conference version: Incas: An incremental active set method for SVM, with Shai Fine (2002) ).
• Product-form Cholesky factorization in interior point methods for second-order cone programming, with D. Goldfarb, Mathematical Programming, v. 103 (2005), pp. 153-179.
• A product-form Cholesky factorization method for handling dense columns in interior point methods for linear programming, with D. Goldfarb, Mathematical Programming, v. 99 (2004), pp. 1-34.
• Incremental learning and selective sampling via parametric optimization framework for SVM, with S. Fine, in “Advances in Neural Information Processing Systems” 14, MIT Press, (2002) 705-711.
• Efficient SVM training using low-rank Kernel representations, with S. Fine, J. of Machine Learning Research, Special issue on Kernel methods, 2(2001) 243-264.
• Manual for FORTRAN software package DFO v1.2, (2001).
• Parametric linear semidefinite programming, , In “Handbook on Semidefinite Programming”, eds. H. Wolkowicz, R. Saigal, L. Vandenberghe, pp 92–110. Kluwer, 2000.
• On parametric semidefinite programming, with D. Goldfarb, Applied Numerical Mathematics, v. 29(3), pp. 361-377, 1999.
• Modified barrier-penalty functions for constrained minimization problems, with Goldfarb D., Polyak R. and Yusefovich B. Computational Optimization and Applications, v. 14(1), pp. 55-74, 1999.
• A derivative free optimization algorithm in practice, with Conn A. R. and Toint Ph. L., Proceedings of 7th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, St. Louis, MO, 1998.
• Interior point trajectories in semidefinite programming, with D. Goldfarb, SIAM J. on Optimization v. 8(4), pp. 871-886, 1998.
• Recent progress in unconstrained nonlinear optimization without derivatives, with A.R. Conn and Ph. L. Toint, Mathematical Programming v. 79 (1997) 397-414.
• On the convergence of derivative-free methods for unconstrained optimization, with A.R. Conn and Ph. L. Toint, “Approximation Theory and Optimization: Tributes to M. J. D. Powell”, eds. A. Iserles and M. Buhmann, pp 83–108, 1997.
• Issues related to interior point methods for linear and semidefinite programming., PhD Thesis, Dept. of IEOR, Columbia University, 1997.
• Extension of Karmarkar’s algorithm to convex quadratically constrained quadratic , with A. Nemirovskii, Mathematical Programming, v. 72 (1996) 273-289.