An implementation of local coordinate coding (lcc) that codes data which approximately lives on a manifold using a variation of l1-norm regularized sparse coding; in lcc, the penalty on the absolute value of each point's coefficient for each atom is weighted by the squared distance of that point to that atom.
LocalCoordinateCoding (const arma::mat &data, const size_t atoms, const double lambda)
Set the parameters to LocalCoordinateCoding. const arma::mat & Codes () const
Accessor the codes. arma::mat & Codes ()
Modify the codes. const arma::mat & Data () const
Access the data. const arma::mat & Dictionary () const
Accessor for dictionary. arma::mat & Dictionary ()
Mutator for dictionary. void Encode (const size_t maxIterations=0, const double objTolerance=0.01)
Run local coordinate coding. double Objective (arma::uvec adjacencies) const
Compute objective function given the list of adjacencies. void OptimizeCode ()
Code each point via distance-weighted LARS. void OptimizeDictionary (arma::uvec adjacencies)
Learn dictionary by solving linear system. std::string ToString () const
size_t atoms
Number of atoms in dictionary. arma::mat codes
Codes (columns are points). const arma::mat & data
Data matrix (columns are points). arma::mat dictionary
Dictionary (columns are atoms). double lambda
l1 regularization term.
An implementation of Local Coordinate Coding (LCC) that codes data which approximately lives on a manifold using a variation of l1-norm regularized sparse coding; in LCC, the penalty on the absolute value of each point's coefficient for each atom is weighted by the squared distance of that point to that atom.
Let d be the number of dimensions in the original space, m the number of training points, and k the number of atoms in the dictionary (the dimension of the learned feature space). The training data X is a d-by-m matrix where each column is a point and each row is a dimension. The dictionary D is a d-by-k matrix, and the sparse codes matrix Z is a k-by-m matrix. This program seeks to minimize the objective: min_{D,Z} ||X - D Z||_{Fro}^2
lambda sum_{i=1}^m sum_{j=1}^k dist(X_i,D_j)^2 Z_i^j where lambda > 0.
This problem is solved by an algorithm that alternates between a dictionary learning step and a sparse coding step. The dictionary learning step updates the dictionary D by solving a linear system (note that the objective is a positive definite quadratic program). The sparse coding step involves solving a large number of weighted l1-norm regularized linear regression problems problems; this can be done efficiently using LARS, an algorithm that can solve the LASSO (paper below).
The papers are listed below.
@incollection{NIPS2009_0719,
title = {Nonlinear Learning using Local Coordinate Coding},
author = {Kai Yu and Tong Zhang and Yihong Gong},
booktitle = {Advances in Neural Information Processing Systems 22},
editor = {Y. Bengio and D. Schuurmans and J. Lafferty and C. K. I. Williams
and A. Culotta},
pages = {2223--2231},
year = {2009}
}
@article{efron2004least,
title={Least angle regression},
author={Efron, B. and Hastie, T. and Johnstone, I. and Tibshirani, R.},
journal={The Annals of statistics},
volume={32},
number={2},
pages={407--499},
year={2004},
publisher={Institute of Mathematical Statistics}
}
Definition at line 91 of file lcc.hpp.
Set the parameters to LocalCoordinateCoding.
Parameters:
data Data matrix.
atoms Number of atoms in dictionary.
lambda Regularization parameter for weighted l1-norm penalty.
Accessor the codes.
Definition at line 144 of file lcc.hpp.
References mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::codes.
Modify the codes.
Definition at line 146 of file lcc.hpp.
References mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::codes.
Access the data.
Definition at line 136 of file lcc.hpp.
References mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::data.
Accessor for dictionary.
Definition at line 139 of file lcc.hpp.
References mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::dictionary.
Mutator for dictionary.
Definition at line 141 of file lcc.hpp.
References mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::dictionary.
Run local coordinate coding.
Parameters:
nIterations Maximum number of iterations to run algorithm.
objTolerance Tolerance of objective function. When the objective function changes by a value lower than this tolerance, the optimization terminates.
Compute objective function given the list of adjacencies.
Code each point via distance-weighted LARS.
Learn dictionary by solving linear system.
Parameters:
adjacencies Indices of entries (unrolled column by column) of the coding matrix Z that are non-zero (the adjacency matrix for the bipartite graph of points and atoms)
Number of atoms in dictionary.
Definition at line 153 of file lcc.hpp.
Codes (columns are points).
Definition at line 162 of file lcc.hpp.
Referenced by mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::Codes().
Data matrix (columns are points).
Definition at line 156 of file lcc.hpp.
Referenced by mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::Data().
Dictionary (columns are atoms).
Definition at line 159 of file lcc.hpp.
Referenced by mlpack::lcc::LocalCoordinateCoding< DictionaryInitializer >::Dictionary().
l1 regularization term.
Definition at line 165 of file lcc.hpp.
Generated automatically by Doxygen for MLPACK from the source code.