Difference between revisions of "MLReadingGroup"

From CSWiki
Jump to: navigation, search
m (→‎Schedule)
m (Reverted edit of Monicag, changed back to last version by Zbarutcu)
 
(88 intermediate revisions by 21 users not shown)
Line 9: Line 9:
 
We maintain an announcement/discussion list for the reading group. You may sign up for the list [https://lists.cs.princeton.edu/mailman/listinfo/ml-reading/ here].
 
We maintain an announcement/discussion list for the reading group. You may sign up for the list [https://lists.cs.princeton.edu/mailman/listinfo/ml-reading/ here].
  
−
== Schedule ==
+
==Schedule (Fall 2008) ==
 +
Our weekly meetings are '''Mo 1:00-5:00pm''' on the 3rd floor of the CS building (CS 302).
 +
 
 +
* Graphical Models, exponential families, and variational inference
 +
 
 +
 
 +
==Schedule (Spring 2008) ==
 +
Our weekly meetings are '''Tue 1:00-2:30pm''' in the AI lab on the 4th floor of the CS building (CS 431).
 +
 
 +
 
 +
Schedule of topics:
 +
 
 +
* 13 May 2008
 +
** Topic: Maximum Entropy Discrimination
 +
** Leader: Chong Wang
 +
** Main Paper: [http://people.csail.mit.edu/tommi/papers/JaaMeiJeb-nips99.ps Tommi Jaakkola, Marina Meila, and Tony Jebara, Maximum Entropy Discrimination, In ''NIPS'' 1999.]
 +
** Long Version: [http://people.csail.mit.edu/tommi/papers/maxent.ps Tommi Jaakkola, Marina Meila, and Tony Jebara, Maximum Entropy Discrimination, Technical Report AITR-1668, MIT, 1999]
 +
 
 +
* 6 May 2008
 +
** Topic: Feature selection for relational data
 +
** Leader: Jonathan Chang
 +
** Main Paper: [http://citeseer.ist.psu.edu/635777.html Jensen, Neville, and Hay (2003), Avoiding Bias when Aggregating Relational Data with Degree Disparity]
 +
** Background: [http://citeseer.ist.psu.edu/jensen02linkage.html Jensen and Neville (2002), Linkage and Autocorrelation Cause Feature Selection Bias in Relational Learning]
 +
 
 +
* 22 April 2008
 +
** Topic: Game theory
 +
** Leader: Indraneel Mukherjee
 +
** Main Paper: [http://projecteuclid.org/DPubS/Repository/1.0/Disseminate?view=body&id=pdf_1&handle=euclid.pjm/1103044235 An Analog of the Minimax Theorem for Vector Payoffs]
 +
 
 +
* 15 April 2008
 +
** Topic: Conditional Random Fields
 +
** Leader: Berk Kapicioglu
 +
** Main Paper: [http://www.seas.upenn.edu/~strctlrn/bib/PDF/crf.pdf J. Lafferty, A. McCallum, and F. Pereira (2001), Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data]
 +
 
 +
* 8 April 2008
 +
** Topic: Online Feature Selection
 +
** Leader: Melissa Carroll
 +
** Main Paper: [http://jmlr.csail.mit.edu/papers/volume7/zhou06a/zhou06a.pdf Jing Zhou, Dean P. Foster, Robert A. Stine, and Lyle H. Ungar (2006), Streamwise Feature Selection]
 +
** Background Paper: [http://www-stat.wharton.upenn.edu/~stine/research/smr.pdf Robert A. Stine (2003), Model Selection using Information Theory and the MDL Principle]
 +
 
 +
* 1 April 2008
 +
** Topic: Reinforcement learning and online learning
 +
** Leader: Umar Syed
 +
** Main Paper: [http://books.nips.cc/papers/files/nips20/NIPS2007_0631.pdf Alexander Strehl and Michael Littman (2008), Online Linear Regression and Its Application to Reinforcement Learning]
 +
** Background Paper: [http://citeseer.ist.psu.edu/638941.html Peter Auer (2002), Using Confidence Bounds for Exploitation-Exploration Trade-offs]
 +
** Background Paper: [http://citeseer.ist.psu.edu/443693.html Ronen I. Brafman and Moshe Tennenholtz (2002), R-max – A General Polynomial Time Algorithm for Near-Optimal Reinforcement Learning]
 +
 
 +
* 25 March 2008
 +
** Topic: Network/Relational Learning
 +
** Leader: Jonathan Chang 
 +
** Main Paper: [http://arxiv.org/abs/0803.1628v1 Janne Sinkkonen, Janne Aukia, Samuel Kaski.  Component models for large networks]
 +
** Background Paper: [http://citeseer.ist.psu.edu/cohn01missing.html D Cohn, T Hofmann.  The Missing Link-A Probabilistic Model of Document Content and Hypertext Connectivity]
 +
** Background Paper: [http://arxiv.org/abs/0705.4485 Edoardo M Airoldi, David M Blei, Stephen E Fienberg, Eric P Xing.  Mixed membership stochastic blockmodels]
 +
 
 +
* 11 March 2008
 +
** Topic: Online learning with experts
 +
** Leader: Indraneel Mukherjee
 +
** Paper: [http://nagoya.uchicago.edu/~jabernethy/Binning.pdf Jacob Abernethy, John Langford, Manfred Warmuth. The Binning Algorithm ]
 +
 
 +
* 04 March 2008
 +
** Topic: Incorporating domain knowledge into POS tagging
 +
** Leader: Jordan Boyd-Graber
 +
** Paper: [http://books.nips.cc/papers/files/nips20/NIPS2007_0964.pdf Toutanova, Kristina and Johnson, Mark. A Bayesian LDA-based model for semi-supervised part-of-speech tagging.  (2007)]
 +
** Paper: [http://portal.acm.org/citation.cfm?id=1219884 Smith, Noah and Eisner, Jason.  Contrastive Estimation: Training Log-Linear Models on Unlabeled Data.  (2005)]
 +
 
 +
* 26 February 2008
 +
** Leader: Berk Kapicioglu
 +
** Paper: [http://citeseer.ist.psu.edu/celeux95stochastic.html Gilles Celeux, Didier Chauveau, Jean Diebolt. "On Stochastic Versions of the EM Algorithm.", 1995]
 +
 
 +
==Schedule (Fall 2007) ==
 +
 
 +
Our weekly meetings are '''Wed 4:00-5:30pm''' in the AI lab on the 4th floor of the CS building (CS 431).
 +
 
 +
Schedule of topics:
 +
* 12 December 2007
 +
** Leader: Zafer Barutcuoglu
 +
** Paper: [http://www.cs.utoronto.ca/~hinton/absps/fastnc.pdf G.E. Hinton, S. Osindero, Y.W. Teh. "A Fast Learning Algorithm for Deep Belief Nets." Neural Computation, 2006.]
 +
** More empirical results: [http://www-etud.iro.umontreal.ca/~larocheh/publications/greedy-deep-nets-nips-06.pdf Bengio et al. "Greedy Layer-wise Training of Deep Networks." NIPS, 2006.]
 +
* 28 November 2007
 +
** Leader: Umar Syed
 +
** Paper: [http://www.cs.princeton.edu/~usyed/SyedSchapireNIPS2007.pdf Umar Syed and Robert E. Schapire. "A game-theoretic approach to apprenticeship learning", NIPS (2008)].
 +
** Background reading: The work is based on [http://www.cs.princeton.edu/~schapire/uncompress-papers.cgi/FreundSc96b.ps Yoav Freund and Robert E. Schapire, "Game theory, on-line prediction, and boosting", COLT (1996)] (see Section 2 and the Appendix).
 +
* 14 November 2007
 +
** Leader: Melissa Carroll
 +
** Paper: [http://www-stat.stanford.edu/~hastie/Papers/B67.2%20(2005)%20301-320%20Zou%20&%20Hastie.pdf Hui Zou and Trevor Hastie. "Regularization and variable selection via the elastic net." (2005) J. R. Statist. Soc. B, 67(2), pp. 301–320.]
 +
** Background reading: It may be helpful to read up on or review LASSO and LARS.  See [http://www-stat.stanford.edu/~tibs/lasso.html the LASSO Page]
 +
 
 +
* 24 October 2007
 +
** Patrón: Berk Kapicioglu
 +
** Paper: [http://www.research.microsoft.com/~joshuago/exponentialprior-final.pdf Joshua Goodman. "Exponential Priors for Maximum Entropy Models." North American ACL 2004.]
 +
 
 +
Additional topics:
 +
* relational network models
 +
* DP + parse trees
 +
* online learning
 +
* semi-supervised learning
 +
* stochastic gradient
 +
* convex optimizing
 +
* parallel learning
 +
* game theory
 +
 
 +
==Schedule (Spring 2007) ==
 +
 
 +
Our weekly meetings are '''Thu 1:30-3:00pm''' in the AI lab on the 4th floor of the CS building (CS 431).
 +
 
 +
Schedule of topics:
 +
* 29 March 2007
 +
** Leader: Indraneel Mukherjee
 +
** Paper: [http://www.cs.berkeley.edu/~feisha/pubs/nips2006.pdf Large Margin Hidden Markov Models for Automatic Speech Recognition]
 +
* 15 March 2007
 +
** Leader: Jordan Boyd-Graber
 +
** Paper 1: [http://www.cs.cornell.edu/home/llee/papers/textstruct.pdf Catching the Drift: Probabilistic Content Models, with Applications to Generation and Summarization (2004)]
 +
** Paper 2: [http://www.stanford.edu/~mpurver/papers/purver-et-al06acl.pdf Unsupervised topic modelling for multi-party spoken discourse (2006)]
 +
* 8 March 2007
 +
** Leader: Umar Syed
 +
** Paper: [http://ai.stanford.edu/~ang/papers/icml04-apprentice.pdf  Pieter Abbeel and Andrew Y. Ng. "Apprenticeship learning via inverse reinforcement learning." ICML 2004.]
 +
* 1 March 2007
 +
** Leader: Miro Dudik
 +
** Paper: [http://www.cs.princeton.edu/~mdudik/ShalevShwartzSi06.pdf Shai Shalev-Shwartz and Yoram Singer. "Convex Repeated Games and Fenchel Duality."]
 +
** See also [http://www.cs.huji.ac.il/~shais/papers/ShalevSi06_fench_tech.pdf a more recent version from NIPS 2006]. It contains more references and the math is slightly different; e.g., it introduces strong convexity relative to a norm. I find it a little bit too condensed and more difficult to read.
 +
* 22 February 2007
 +
** Leader: Melissa Carroll
 +
** Paper: [http://www.icml2006.org/icml_documents/camera-ready/055_Hidden_Process_Model.pdf R.A. Hutchinson, T. Mitchell, I. Rustandi. "Hidden Process Models." ICML 2006.]
 +
** Background on fMRI classification application: [http://www.cs.cmu.edu/afs/cs/project/theo-73/www/papers/mlj04-final-published.pdf T. Mitchell, R. Hutchinson, R. Niculescu, F. Pereira, X. Wang. "Learning to Decode Cognitive States from Brain Images." Machine Learning, 57, 145–175, 2004.]
 +
* 15 February 2007
 +
** Leader: Edo Airoldi
 +
** Paper:  [http://research.microsoft.com/~cmbishop/downloads/Bishop-CVPR-06.pdf J. Lasserre, C. M. Bishop, and T. Minka. "Principled hybrids of generative and discriminative models." CVPR 2006.]
 +
** Notes:  [ftp://ftp.research.microsoft.com/pub/tr/TR-2005-144.pdf T. Minka. "Discriminative models, not discriminative training." MSR-TR-144 2005.]
 +
* 8 February 2007
 +
** Leader: Joe Calandrino
 +
** Paper:  [http://www.cs.bgu.ac.il/~kobbi/papers/psd.pdf I. Dinur, K. Nissim. "Revealing Information while Preserving Privacy." PODS 2003.]
 +
* inverse RL (Umar)
 +
* disc/gen approaches (Bishop...)
 +
* active learning (BK)
 +
* hidden process models (MC)
 +
* Gaussian processes (Z)
 +
* Dirichlet processes
 +
* Semisupervised learning (Florian)
 +
** [http://www.e-publications.org/ims/submission/index.php/STS/user/submissionFile/45?confirm=ab8bceff <b>The use of unlabelled data in predictive modelling</b>, Liang F, Mukherjee S and West M, Statistical Science, to appear.]
 +
** some paper by Lafferty and Wasserman ?
 +
* Quantam neural networks (Vaneet)
 +
* Manifold learning (Z)
 +
* On-line learning (Berk)
 +
* Music stuff/transcription (R)
 +
* Variational methods (JC)
 +
* Random projections (Charikar)
 +
 
 +
== Schedule (Fall 2006) ==
 
Our weekly meetings are <b>Fridays, 3pm to 5pm, in CS 402</b>.
 
Our weekly meetings are <b>Fridays, 3pm to 5pm, in CS 402</b>.
  
Line 50: Line 197:
 
** Some useful (perhaps) background on the technique: [http://ai.stanford.edu/~ang/papers/nips04-hypernym.pdf Snow, Jurafsky, and Ng. "Learning Syntactic Patterns for Automatic Hypernym Discovery."]
 
** Some useful (perhaps) background on the technique: [http://ai.stanford.edu/~ang/papers/nips04-hypernym.pdf Snow, Jurafsky, and Ng. "Learning Syntactic Patterns for Automatic Hypernym Discovery."]
 
** Background on what they're trying to do: [http://portal.acm.org/citation.cfm?doid=219717.219748 Miller.  "WordNet: A Lexical Database for English." Communications of the ACM, Volume 38, Issue 11 (November 1995), Pages 39-41.]
 
** Background on what they're trying to do: [http://portal.acm.org/citation.cfm?doid=219717.219748 Miller.  "WordNet: A Lexical Database for English." Communications of the ACM, Volume 38, Issue 11 (November 1995), Pages 39-41.]
 +
* 15 December 2006
 +
** Leader: Rebecca Fiebrink
 +
** Paper:  [http://citeseer.ist.psu.edu/cache/papers/cs/2737/http:zSzzSzwizpak.iaf.uiowa.eduzSz%7EsgiserviceszSzVarsityzSzsilicon_campzSzMinesetzSztechzSzaccEst.pdf/kohavi95study.pdf Kohavi, R. 1995. "A study of cross-validation and bootstrap for accuracy estimation and model selection." Proceedings of the International Joint Conferences on Artificial Intelligence (IJCAI).]
  
 
== Proposed Topics and Papers ==
 
== Proposed Topics and Papers ==
Line 72: Line 222:
 
* Language Applications
 
* Language Applications
 
* Music
 
* Music
−
** [http://www.a.fake.paper/paper.pdf Smith, J. 2005. Music and machine learning are great. <i>ISMIR 2006 Proceedings</i>.]
+
** [http://scholar.google.com/scholar?q=%22a+generative+model+for+music+transcription%22&hl=en&lr=&btnG=Search Cemgil, A. T., H. J. Kappen, and D. Barber. 2005. A generative model for music transcription. <i>IEEE Transactions on Speech and Audio Processing</i>.]
−
*** I've put the link above to give an example of how to link to a paper. It will soon be replaced by a link or description of one or more real papers on machine learning and music. --[[User:RebeccaFiebrink|RebeccaFiebrink]] 20:50, 28 September 2006 (EDT)
 
 
* Markov Decision Processes (MDPs) and Reinforcement Learning
 
* Markov Decision Processes (MDPs) and Reinforcement Learning
 
** State-space abstraction/aggregation in MDPs
 
** State-space abstraction/aggregation in MDPs
Line 100: Line 249:
  
 
=== PostDocs ===
 
=== PostDocs ===
 +
* Edo Airoldi, LSI & CS
 
* Florian Markowetz, LSI
 
* Florian Markowetz, LSI
  
 
=== Students ===
 
=== Students ===
−
* Zafer Barutcuoglu, CS
+
* Indraneel Mukherjee, CS
 
* Jordan Boyd-Graber, CS
 
* Jordan Boyd-Graber, CS
 
* Joseph Calandrino, CS
 
* Joseph Calandrino, CS
 +
* Melissa Carroll, CS
 
* Jonathan Chang, EE
 
* Jonathan Chang, EE
 
* Miroslav Dudik, CS
 
* Miroslav Dudik, CS
Line 111: Line 262:
 
* Berk Kapicioglu, CS
 
* Berk Kapicioglu, CS
 
* Umar Syed, CS
 
* Umar Syed, CS
 +
* Chong Wang, CS
 +
* Sina Jafarpour, CS
 +
* Sean Gerrish, CS
 +
* Richard Socher, CS

Latest revision as of 09:18, 3 February 2009

Machine Learning Reading Group

Welcome to the wiki of the machine learning reading group.

Mailing list

We maintain an announcement/discussion list for the reading group. You may sign up for the list here.

Schedule (Fall 2008)

Our weekly meetings are Mo 1:00-5:00pm on the 3rd floor of the CS building (CS 302).

  • Graphical Models, exponential families, and variational inference


Schedule (Spring 2008)

Our weekly meetings are Tue 1:00-2:30pm in the AI lab on the 4th floor of the CS building (CS 431).


Schedule of topics:

Schedule (Fall 2007)

Our weekly meetings are Wed 4:00-5:30pm in the AI lab on the 4th floor of the CS building (CS 431).

Schedule of topics:

Additional topics:

  • relational network models
  • DP + parse trees
  • online learning
  • semi-supervised learning
  • stochastic gradient
  • convex optimizing
  • parallel learning
  • game theory

Schedule (Spring 2007)

Our weekly meetings are Thu 1:30-3:00pm in the AI lab on the 4th floor of the CS building (CS 431).

Schedule of topics:

Schedule (Fall 2006)

Our weekly meetings are Fridays, 3pm to 5pm, in CS 402.

Scheduled readings:

Proposed Topics and Papers

Please add further topics, suggest papers for particular topics, etc. here.

Participants

(Participants, please add your name to the list below.)

Faculty

  • David Blei
  • Rob Schapire

PostDocs

  • Edo Airoldi, LSI & CS
  • Florian Markowetz, LSI

Students

  • Indraneel Mukherjee, CS
  • Jordan Boyd-Graber, CS
  • Joseph Calandrino, CS
  • Melissa Carroll, CS
  • Jonathan Chang, EE
  • Miroslav Dudik, CS
  • Rebecca Fiebrink, CS
  • Berk Kapicioglu, CS
  • Umar Syed, CS
  • Chong Wang, CS
  • Sina Jafarpour, CS
  • Sean Gerrish, CS
  • Richard Socher, CS
Retrieved from "http://wiki.cs.princeton.edu/index.php?title=MLReadingGroup&oldid=9506"