演讲人: Prof. Tetsuya Sakai /酒井 哲也

讲座时间: 10月16日（周四）下午16:00——17:00

讲座地点: 信息楼四层417会议室

讲座内容: ABSTRACT: Evaluation conferences such as TREC (Text Retrieval Conference) and NTCIR (NII Testbeds and Community for Information access Research) build test collections every year, to enable fair comparisons across different research groups. However, the design principle they use tend to be ad hoc - for example, they typically decide to build n=50 topics, and then let the pool depth be pd=100 so that the test collections will have "reasonable" numbers of relevant documents sampled from the large target corpora. In this talk, I will show that the topic set size n can be determined based on statistical requirements, such as the maximum length of confidence intervals, and the minimum performance difference under given probabilities of Type I and Type II errors. The pool depth can also be determined based on the available budget. I will show that test collections need to be designed with specific evaluation measures in mind, and that this statistical approach can cut down assessment costs dramatically. BIO: TetsuyaSakai, an associate professor at the Department of Computer Science, Waseda University, Tokyo, joined the Toshiba Corporation in 1993 and obtained his Ph.D from Waseda in 2000. From 2000 to 2001, he was supervised by the late Karen Sparck Jones at the University of Cambridge Computer Laboratory. From 2009 to 2013, he was a lead researcher at Microsoft Research Asia. He served as a program co-chair for SIGIR 2013 and OAIR (RIAO) 2013, and as a general chair of AIRS 2009. In 2012, he served as the first chair of the Asia Information Retrieval Societies (AIRS) steering committee. Currently, he serves as the Editor-in-Chief of Information Retrieval (Springer) for Asia/Australasia, and is on the editorial board of Information Processing and Management (Elsevier). He is also a general co-chair of NTCIR. He will serve as a general chair of SIGIR 2017.

Prof. Tetsuya Sakai: http://www.f.waseda.jp/tetsuya/sakai.html