From CSWiki
Revision as of 10:33, 29 March 2008 by RebeccaFiebrink (talk | contribs) (Abstract)

Jump to: navigation, search

Our fancy ISMIR paper outline.

Notes for us

6 page limit.

Title: Support for MIR prototyping and real-time applications in the ChucK programming language

Key points:

  • There is no general purpose language for MIR prototyping that both gives access to analysis building blocks and allows for low-level coding
  • This sort of framework can be really useful for fast prototyping, flexible coding, and education
  • There is furthermore no such framework that does this that combines MIR and performance / synthesis; ChucK does this.
  • We're going to show what is available in the language, and examples of how to work with it to accomplish MIR tasks


(re + ge)

In this paper, we discuss the recent additions of audio analysis and machine learning infrastructure to the ChucK music programming language that make it a suitable and unique tool both for music information retrieval system prototyping and for applying music information retrieval algorithms in real-time music performance contexts. From its inception, ChucK has offered both high-level control over building block components as well as fine-grained, sample-level manipulation; the new analysis and learning capabilities of the language were designed to preserve this breadth of control options, allowing the programmer to experiment with new features, signal processing techniques, and learning algorithms with ease and flexibility. Additionally, these new capabilities are tightly integrated into ChucK's synthesis framework, making it trivial to use the results of analysis and learning tasks to drive music creation and interaction in real time.

Designed as a language for composers and performers, ChucK offers programmers high-level control over building block components as well as fine-grained, sample-level control of audio data.

Something about bridging gap between MIR and performance

In this paper, we discuss recent additions to the ChucK music programming language that make it a suitable and unique tool for music information retrieval prototyping tasks and for applying music information retrieval algorithms to music in live performance settings. Fills a niche w/ low & highlevel control => prototyping. Also, as a language originally designed for the live performer, ChucK

the new analysis and learning capabilities of ChucK are exposed at both these levels, and tightly integrated with its synthesis and control capabilities to facilitate analysis and learning in real-time contexts. We discuss the language capabilities in some detail, outline examples of how an MIR researcher can use ChucK for prototyping and applying music analysis in real time, and present a toolkit written in ChucK to facilitate these tasks.


(re, ge)

Background & Motivation

(re, ge)

  • MIR & performance
    • How peformers/composers have used MIR-like algorithms, & what tools they use
    • Why performance should get more attention from MIR!!!
      • What is MIR, really?
      • What good are MIR tools doing for the world?
      • How is an MIR researcher supposed to evaluate and improve his/her work?
  • MIR prototyping
    • Define prototyping for our purposes; why is it important? (+ side benefits)
    • What tools exist
  • ChucK (ge)
    • A short history
    • Examples of how to use the language (esp. UGens & time control)
    • OTF & timeliness make suited for prototyping, but not originally for analysis
  • In summary, our goal was to modify ChucK & build tools in it to foster a tighter connection between MIR and performance, and to provide fast prototyping capabability in a new framework for MIRers

Recent additions to ChucK to allow for prototyping and realtime MIR

Unit analyzers


  • Say what they are, ...
  • Available features!

Learning framework

(re) Reference ICMC paper and make note of cool areas this opens up, but which we don't have time to discuss here (side benefit: make it clear that this isn't a copy of the ICMC paper)

introduce some "working pipelines"

intent: give reader a feel for "if I have an idea and want to code it quickly, how would this work in chucK?" (and why would I not use matlab, m2k, etc.) ge? the working pipeline to my brain is busted

case studies

  • intent: show that chuck indeed works for MIR tasks, that it is indeed a concise and nice syntax, flexible, powerful, etc.
  • case study 1: bergstra artist/genre ID (shows above specifically) (re)
  • case study 2: examples from Stanford class (shows some real uses by performers) (ge)
  • case study 3: AYYYYYYYGY

related work

For each, say what it does, why it is good, but why ours is different

  • CLAM
  • Marsyas
  • Matlab real-time stuff
  • Supercollider & Max/MSP externals

conclusion & map out future work

Discussion /take-home points

  • ChucK has expanded to a new frontier
  • We will be continuing to work on it, driven by our own applications as well as requests from the ChucK user community


  • Future work
    • file I/O is coming soon
    • online repository of templates and examples (like SMELT) that can get people started
    • modeling (HMMs)