Stochastic Optimization for Large-scale Machine Learning

Chapter 7 Stochastic Trust Region Inexact Newton Method

DOI: 10.1201/9781003240167-7

Nowadays stochastic approximation methods are one of the major research direction to deal with the large-scale machine learning problems. From stochastic first order methods, now the focus is shifting to stochastic second order methods due to their faster convergence and availability of computing resources. In this chapter, we have proposed a novel Stochastic Trust RegiOn Inexact Newton method, called as STRON, which uses conjugate gradient (CG) to inexactly solve trust region subproblem. The method uses progressive subsampling in the calculation of gradient and Hessian values to take the advantage of both, stochastic and full-batch regimes. We have extended ...