タスク参加募集 NTCIR-10 INTENT-2

[Makoto Kato <kato [at] dl.kuis.kyoto-u.ac.jp>]

日本データベース学会の皆様，
（重複して受け取られた場合にはご容赦ください）

京都大学の加藤と申します．
曖昧なクエリや不明快なクエリに秘められた検索意図を扱うタスクのご案内で
す。奮ってご参加ください．


###################################################
          CALL FOR TASK PARTICIPATION
            NTCIR-10 INTENT-2 Task:
English/Chinese/Japanese Subtopic Mining subtasks
                     and
Chinese/Japanese Document Ranking subtasks

http://research.microsoft.com/en-us/people/tesakai/intent2.aspx

Topics released: May 2012
Evaluation results released: Jan 2013
Final NTCIR-10 workshop: June 18-21 2013
###################################################


### OVERVIEW ###

The NTCIR INTENT task addresses the problem of
handling ambiguous and/or underspecified queries in Web search.
Does "office" mean a Microsoft product or a workplace? (ambiguity)
Does "harry potter" mean the books, the films or the character? (lack of
specificity)
Unlike traditional information retrieval where only relevance plays the
central role,
INTENT considers multiple possible intents for a given query and
aims to balance relevance and diversity.

The first round of INTENT at NTCIR-9 was concluded successfully,
thanks to great research contributions from
17 research groups from seven countries/regions.
At INTENT-1, it has been found that some successful systems
were relevance-oriented, while others were diversity-oriented.

NTCIR-10 will host the second round of INTENT (INTENT-2)
to further clarify and hopefully solve the challenges of
handling multiple intents.


### TASKS ###

SUBTOPIC MINING subtask:

Given a query, systems must return a ranked list of
"subtopic strings" that represent diverse intents.

In addition to Chinese and Japanese,
this time we are running an English Subtopic Mining subtask
as well. For this, we will syncronise with the TREC 2012 diversity task
and share the topic set.

DOCUMENT RANKNIG subtask:

Given a query, systems must return a selectively diversified
web search results. Unlike the TREC diversity task,
we take the importance of intents and
per-intent graded relevance into account when evaluating the
participating systems.

This time, we will include some "one item search" queries
in the topic set, which probably do not require diversification.
Can systems selectively and effectively diversify?


In addition to topics, the organisers will provide
query suggestion data and baseline web search results
to enhance repeatability and fairness, and to
encourage newcomers to participate.


### IMPORTANT DATES ###

May 2012     topics, query suggestions and baseline runs released
July 2012    all submissions due
Aug-Dec 2012 identify intents from subtopics -> per-intent relevance
assessments
Jan 2013     evaluation results released
March 2013?  draft papers due
May 2013?    camera ready papers due
June 18-21 2013    NTCIR-10


### HOW TO PARTICIPATE ###

Task registration page will be available soon on the NTCIR-10 website
(see below).

Participants who plan to index the document corpora
need to obtain the SogouT data for Chinese, ClueWeb09-JA for Japanese,
and ClueWeb09 for English.
How to obtain the data is described in
the INTENT-1 homepage (see below).

We encourage new participants to have a look at the INTENT-1 overview paper
to familiarise themselves with the task:
http://research.nii.ac.jp/ntcir/workshop/OnlineProceedings9/NTCIR/01-NTCIR9-OV-INTENT-SongR.pdf

Please note:
task participants must write a paper for the NTCIR-10 meeting (unrefereed)
and attend the meeting at NII, Tokyo in June 2013.


### ORGANISERS ###

Tetsuya Sakai, MSRA
Min Zhang, Tsinghua University
Ruihua Song, MSRA
Yiqun Liu, Tsinghua University
Zhicheng Dou, MSRA
Makoto Kato, Kyoto University
Mayu Iwata, Osaka University
Takehiro Yamamoto, Kyoto University

## QUESTIONS? PLEASE CONTACT ##

ntcadm-intent at
nii dot ac dot jp

## LINKS ##

INTENT-2 homepage:
http://research.microsoft.com/en-us/people/tesakai/intent2.aspx

INTENT-1 homepage:
http://www.thuir.org/intent/ntcir9/

NTCIR-10:
http://research.nii.ac.jp/ntcir/ntcir-10/index.html

NTCIR online proceedings:
http://research.nii.ac.jp/ntcir/publication1-en.html