FOCUSS-based dictionary learning algorithms

Algorithms for data-driven learning of domain-specific over complete dictionaries are developed to obtain maximum likelihood and maximum a posteriori dictionary estimates based on the use of Bayesian models with concave/Schur- concave negative log-priors. Such priors are appropriate for obtaining sparse representations of environmental signals within an appropriately chosen dictionary. The elements of the dictionary can be interpreted as 'concepts,' features or 'words' capable of succinct expression of events encountered in the environment. This is a generalization of vector quantization in that one is interested in a description involving a few dictionary entries, but not necessarily as succinct as one entry. To learn an environmentally-adapted dictionary capable of concise expression of signals generated by the environment, we develop algorithms that iterate between a representative set of sparse representations found by variants of FOCUSS, an affine scaling transformation (ACT)-like sparse signal representation algorithm recently developed at UCSD, and an update of the dictionary using these sparse representations.