Header logo is

Fast Newton-type Methods for the Least Squares Nonnegative Matrix Approximation Problem

2007

Conference Paper

ei


Nonnegative Matrix Approximation is an effective matrix decomposition technique that has proven to be useful for a wide variety of applications ranging from document analysis and image processing to bioinformatics. There exist a few algorithms for nonnegative matrix approximation (NNMA), for example, Lee & Seung’s multiplicative updates, alternating least squares, and certain gradient descent based procedures. All of these procedures suffer from either slow convergence, numerical instabilities, or at worst, theoretical unsoundness. In this paper we present new and improved algorithms for the least-squares NNMA problem, which are not only theoretically well-founded, but also overcome many of the deficiencies of other methods. In particular, we use non-diagonal gradient scaling to obtain rapid convergence. Our methods provide numerical results superior to both Lee & Seung’s method as well to the alternating least squares (ALS) heuristic, which is known to work well in some situations but has no theoretical guarantees (Berry et al. 2006). Our approach extends naturally to include regularization and box-constraints, without sacrificing convergence guarantees. We present experimental results on both synthetic and realworld datasets to demonstrate the superiority of our methods, in terms of better approximations as well as efficiency.

Author(s): Kim, D. and Sra, S. and Dhillon, I.
Book Title: SDM 2007
Journal: Proceedings of the SIAM International Conference on Data Mining (SDM 2007)
Pages: 343-354
Year: 2007
Month: April
Day: 0
Editors: Apte, C.
Publisher: Society for Industrial and Applied Mathematics

Department(s): Empirical Inference
Bibtex Type: Conference Paper (inproceedings)

Event Name: SIAM International Conference on Data Mining
Event Place: Minneapolis, MN, USA

Address: Pittsburgh, PA, USA
Digital: 0
Institution: Society for Industrial and Applied Mathematics
Language: en
Organization: Max-Planck-Gesellschaft
School: Biologische Kybernetik

Links: PDF
Web

BibTex

@inproceedings{5219,
  title = {Fast Newton-type Methods for the Least Squares Nonnegative Matrix Approximation Problem},
  author = {Kim, D. and Sra, S. and Dhillon, I.},
  journal = {Proceedings of the SIAM International Conference on Data Mining (SDM 2007)},
  booktitle = {SDM 2007},
  pages = {343-354},
  editors = {Apte, C. },
  publisher = {Society for Industrial and Applied Mathematics},
  organization = {Max-Planck-Gesellschaft},
  institution = {Society for Industrial and Applied Mathematics},
  school = {Biologische Kybernetik},
  address = {Pittsburgh, PA, USA},
  month = apr,
  year = {2007},
  month_numeric = {4}
}