We claim and present arguments to the effect that a large class of manifold learning algorithms that are essentially local and can be framed as kernel learning algorithms will suffer from the curse of dimensionality, at the dimension of the true underlying manifold. This observation suggests to explore non-local manifold learning algorithms which attempt to discover shared structure in the tangent planes at different positions. A criterion for such an algorithm is proposed and experiments estimating a tangent plane prediction function are presented, showing its advantages with respect to local manifold learning algorithms: it is able to generalize very far from training data (on learning handwritten character image rotations), where a local non-parametric method fails.
View PDFacceptance rate: 207/822, 25%
Reference:
Non-Local Manifold Tangent Learning (Yoshua Bengio, Martin Monperrus), In Advances in Neural Information Processing Systems (Lawrence K. Saul, Yair Weiss, Léon Bottou, eds.), MIT Press, volume 17, 2004. (acceptance rate: 207/822, 25%)
Bibtex Entry:
@INPROCEEDINGS{monperrus04,
author = {Yoshua Bengio and Martin Monperrus},
title = {Non-Local Manifold Tangent Learning},
booktitle = {Advances in Neural Information Processing Systems},
year = {2004},
editor = {Lawrence K. Saul and Yair Weiss and Léon Bottou},
volume = {17},
pages = {129-136},
publisher = {MIT Press},
abstract = {We claim and present arguments to the effect that a large class of
manifold learning algorithms that are essentially local and can be
framed as kernel learning algorithms will suffer from the curse of
dimensionality, at the dimension of the true underlying manifold.
This observation suggests to explore non-local manifold learning
algorithms which attempt to discover shared structure in the tangent
planes at different positions. A criterion for such an algorithm
is proposed and experiments estimating a tangent plane prediction
function are presented, showing its advantages with respect to local
manifold learning algorithms: it is able to generalize very far from
training data (on learning handwritten character image rotations),
where a local non-parametric method fails.},
comment = {acceptance rate: 207/822, 25%},
url = {http://www.monperrus.net/martin/Non_Local_Manifold_Tangent_Learning.pdf},
x-abbrv = {NIPS}
}Powered by bibtexbrowser