(Apologies for cross-postings)

MIREX 2010 Call for Evaluation Task Proposals

The International Music Information Retrieval Systems Evaluation 
Laboratory (IMIRSEL) team at the University of Illinois at 
Urbana-Champaign, is soliciting proposals for evaluation tasks to be 
performed as part of the 2010 Music Information Retrieval Evaluation 
eXchange (MIREX 2010). Potential participants are invited to submit 
proposals for novel evaluation tasks, the re-running of past evaluation 
tasks and additional datasets and evaluation metrics to used to run tasks.

Since 2005, MIREX has conducted 760 evaluation runs over a wide range of 
MIR tasks. Evaluation tasks from past MIREX evaluations include:

    - Audio Test/Train Tasks
       - Audio Artist Identification
       - Audio Genre Classification
       - Audio Music Mood Classification
       - Audio Classical Composer Identification
    - Audio Onset Detection
    - Audio Key Detection
    - Audio Tag Classification
    - Audio Cover Song Identification
    - Real-time Audio to Score Alignment (a.k.a Score Following)
    - Query-by-Singing/Humming
    - Multiple Fundamental Frequency Estimation & Tracking
    - Audio Chord Estimation
    - Audio Melody Extraction
    - Query-by-Tapping
    - Audio Beat Tracking
    - Audio Music Similarity and Retrieval
    - Structural Segmentation


So that the MIREX team can gauge the level of interest in each task an
online poll has been setup. If you are considering participating in 
MIREX 2010 please indicate your likelihood of participation at:
https://spreadsheets.google.com/viewform?formkey=dHNtUjkxQ1M2Q0dQa293cC02TnM2OFE6MA


Please use the MIREX organisation mailing list (EvalFest) for discussion 
of MIREX task proposals and other MIREX related issues:
https://mail.lis.illinois.edu/mailman/listinfo/evalfest

The MIREX 2010 wiki will be used to embody and disseminate task 
proposals and can be found at:
http://www.music-ir.org/mirex/2010/index.php/Main_Page

Where possible, task related discussions, which should be conducted on 
the MIREX organization mailing list (EvalFest), should be summarized on 
the appropriate MIREX 2010 wiki pages.

Where possible, definitions or example code for new evaluation metrics 
or tasks should be provided to the IMIRSEL team who will embody them in
software as part of the NEMA analytics framework, which will be released 
to the community at or before ISMIR 2010 - providing a standardised set 
of interfaces and output to disciplined evaluation procedures for a 
great many MIR tasks.

Projected Target Dates:

    - 1st May 2010: MIREX submission system opens (somewhat hopeful 		 
       target date)
    - 15th June - 1st July 2010: Rolling MIREX submission system closures
         (dates to be announced)
    - 15th July 2010: MIREX results posting begins
    - 1st August 2010: All MIREX results posted (somewhat hopeful target
         date)
    - 2-6th August 2010: USMIR Summer School
         (http://ismir2010.ismir.net/usmir-summer-school/)
    - 9-13th August 2010: ISMIR 2010 Conference
          (http://ismir2010.ismir.net/)

Finally, a web-accessible submission and execution system is to be 
provided for MIREX by the NEMA project (http://nema.lis.illinois.edu/). 
This prototype "MIREX DIY" system will allow users to very simply 
submit, configure, test, execute and debug their own submissions and to 
inspect or publish their results. As this system is still in its infancy 
it may (initially) only be available for selected tasks and participants 
at MIREX 2010. Hence, we would like to invite institutions likely to 
produce multiple submissions to MIREX 2010 to contact the IMIRSEL team 
at [log in to unmask] and to nominate an individual from their 
institution to be tutored on the use of the NEMA service and to manage 
their institutions submissions.

Looking forward to the best MIREX yet.

Cheers,
J. Stephen Downie on behalf of the MIREX team.

-- 
**********************************************************
    "Research funding makes the world a better place"
**********************************************************
J. Stephen Downie, PhD
Associate Professor,
Graduate School of Library and Information Science; and,
Center Affiliate, National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign
[Vox/Voicemail] (217) 649-3839
NEMA Project Home: http://nema.lis.uiuc.edu