Difference between revisions of "MLReadingGroup"

Revision as of 18:58, 17 November 2006

Machine Learning Reading Group

Welcome to the wiki of the machine learning reading group.

Mailing list

We maintain an announcement/discussion list for the reading group. You may sign up for the list here.

Schedule

Our weekly meetings are Fridays, 3pm to 5pm, in CS 402.

Scheduled readings:

• 5 October 2006 THURSDAY 4:30PM this week
  • Leader: Zafer
  • Paper: Horst, R., Thoai, N. V. 1999. DC Programming: Overview. J. Optim. Theory Appl.

  (If you need a review: Hindi, H. 2004. A Tutorial on Convex Optimization. American Control Conference.)

  (Application for the interested: Argyriou A. et al. 2006. A DC-Programming Algorithm for Kernel Selection. ICML.)

  And another application Ellis, S. and Nayakkankuppam, V. Phylogenetic Analysis Via DC Programming .

• 13 October 2006
  • Leader: Miro
  • Papers:
    • Robert Tibshirani. Regression shrinkage and selection via the Lasso. J. R. Statist. Soc. B, 1995.
    • Su-In Lee, Honglak Lee, Pieter Abbeel and Andrew Y. Ng. Efficient L1 regularized logistic regression. In Proceedings of the Twenty-First National Conference on Artificial Intelligence (AAAI-06), 2006.
  • Paper:
• 20 October 2006
  • Leader: Florian
  • Paper: N Meinshausen and P Bühlmann, High Dimensional Graphs and Variable Selection With the Lasso, Annals of Statistics 34(3), 1436-1462
  • Background reading: D Heckerman, DM Chickering, C Meek, R Rounthwaite, C Kadie, Dependency Networks for Inference, Collaborative Filtering, and Data Visualization, JMLR, 1(Oct):49-75, 2000
• 27 October 2006
  • Leader:
  • Paper:
• 3 November 2006
  • Leader:
  • Paper:
• 10 November 2006
  • "Leader": Jordan
  • Paper: Understanding the Yarowsky Algorithm Abney, S. 2004. Understanding the Yarowsky Algorithm. Comput. Linguist. 30, 3 (Sep. 2004), 365-395.
  • Background: The original paper Yarowsky, D. 1995. Unsupervised word sense disambiguation rivaling supervised methods. In Proceedings of the 33rd Annual Meeting on Association For Computational Linguistics (Cambridge, Massachusetts, June 26 - 30, 1995). Annual Meeting of the ACL. Association for Computational Linguistics, Morristown, NJ, 189-196.
  • Background: A good overview of the problem area Ide, N. and Véronis, J. 1998. Introduction to the special issue on word sense disambiguation: the state of the art. Comput. Linguist. 24, 1 (Mar. 1998), 2-40.
• 17 November 2006
  • Boss: Berk Kapicioglu
  • Paper: Weighted One-Against-All
  • Nostalgia: Rifkin and Klautau. "In Defense of One-Vs-All Classification." Journal of Machine Learning Research, Volume 5, pp. 101-141, 2004.
• 1 December 2006
  • Leader: Jonathan Chang
  • Paper: Snow, Jurafsky, and Ng. "Semantic Taxonomy Induction from Heterogenous Evidence." Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the ACL, pages 801-808, July 2006.
  • Some useful (perhaps) background on the technique: Snow, Jurafsky, and Ng. "Learning Syntactic Patterns for Automatic Hypernym Discovery."
  • Background on what they're trying to do: Miller. "WordNet: A Lexical Database for English." Communications of the ACM, Volume 38, Issue 11 (November 1995), Pages 39-41.

Proposed Topics and Papers

Please add further topics, suggest papers for particular topics, etc. here.

• Lasso
  • The original paper: Robert Tibshirani. Regression shrinkage and selection via the Lasso. J. R. Statist. Soc. B 58, 1995.
  • The Lasso Page
  • Generalization properties:
    • Sara A. van de Geer. High-dimensional generalized linear models and the Lasso. Tech. Report, June 2006.
    • Keith Knight and Wenjiang Fu. Asymptotics for lasso-type estimators. Ann. Statist. 28, 2000.
  • Sparse approximation:
    • David Donoho. For most large underdetermined systems of linear equations, the minimal l1-norm near-solution approximates the sparsest near-solution, Tech. Report, August 2004.
    • David Donoho. For most large underdetermined systems of linear equations, the minimal l1-norm solution is also the sparsest solution, Tech. Report, September 2004.
  • Model selection: P. Buhlmann and B. Yu. Boosting, Model Selection, Lasso and Nonnegative Garotte. Tech. Report, 2005.
  • Relatives:
    • RODEO
    • Least angle regression
    • Hui Zou and Trevor Hastie. Regularization and Variable Selection via the Elastic Net. J. R. Statist. Soc. B, 2005 + Addendum
    • Su-In Lee, Honglak Lee, Pieter Abbeel and Andrew Y. Ng. Efficient L1 regularized logistic regression. In Proceedings of the Twenty-First National Conference on Artificial Intelligence (AAAI-06), 2006.
• Optimization
• Language Applications
• Music
  • Smith, J. 2005. Music and machine learning are great. ISMIR 2006 Proceedings.
    • I've put the link above to give an example of how to link to a paper. It will soon be replaced by a link or description of one or more real papers on machine learning and music. --RebeccaFiebrink 20:50, 28 September 2006 (EDT)
• Markov Decision Processes (MDPs) and Reinforcement Learning
  • State-space abstraction/aggregation in MDPs
  • The E^3 algorithm (Kearns and Singh)
  • Inverse reinforcement learning
• Active Learning
• Deep Neural Networks
• Factor Graphs
  Algorithms that must deal with complicated global functions of many variables often exploit the manner in which the given functions factor as a product of "local" functions, each of which depends on a subset of the variables.
  • Tutorial: Factor Graphs and the Sum-Product Algorithm, F.R. Kschischang, B. Frey, H-A Loelinger, IEEE Transactions on Information Theory, Vol 47, No 2, Feb 2001.
  • Application: Physical Network Models, C-H Yeang, T Ideker, T Jaakkola, Journal of Computational Biology, Vol 11, No 2-3, 2004
• Cross-validation
  • Kohavi, R. 1995. "A study of cross-validation and bootstrap for accuracy estimation and model selection." Proceedings of the International Joint Conferences on Artificial Intelligence (IJCAI).
    • "Do 10-fold CV instead of leave-one-out," at least when doing model selection
  • Rivals, I., and L. Personnaz. 1999. "On cross-validation for model selection." Neural Computation 11 (4).
    • "Do statistical tests instead of leave-one-out," at least when doing model selection
  • Elisseeff, A., and M. Pontil. 2003. "Leave-one-out error and stability of learning algorithms with applications." Advances in Learning Theory: Methods, Models and Applications, NATO Science Series III: Computer and Systems Sciences, Vol. 190, J. Suykens et al. Eds.
    • Reasons about sufficient conditions for LOO-CV error to approach generalization error for fixed algorithms (including kernel methods). A more theoretical (and maybe less practical) paper.
    • see also Evgeniou, T., M. Pontil, and A. Elisseeff. 2004. "Leave-one-out error, stability, and generalization of voting combinations of classifiers." Machine Learning 55:(1): 71-97. (I haven't read this but it's very related.)

Participants

(Participants, please add your name to the list below.)

Faculty

• David Blei
• Rob Schapire

PostDocs

• Florian Markowetz, LSI

Students

• Zafer Barutcuoglu, CS
• Jordan Boyd-Graber, CS
• Joseph Calandrino, CS
• Jonathan Chang, EE
• Miroslav Dudik, CS
• Rebecca Fiebrink, CS
• Berk Kapicioglu, CS
• Umar Syed, CS