What question did this study set out to answer?

This task aims to evaluate document relevance while ensuring group fairness across different content types.

April 1, 2026Open Access

Overview of the NTCIR-18 FairWeb-2 Task

Key Points

This task aims to evaluate document relevance while ensuring group fairness across different content types.
Designed two subtasks: Web Search Subtask and Conversational Search Subtask.
Created three search topic types: researchers, movies, and YouTube contents.
Defined attribute sets for addressing group fairness in search results.
Received 23 runs from five teams for the Web Search Subtask.
Collected four runs from two teams for the Conversational Search Subtask.
Included submissions from the organizers in both subtasks.

Abstract

This paper provides an overview of the NTCIR-18 FairWeb-2 Task. Our task considers not only document relevance but also group fairness. We designed two subtasks: the Web Search Subtask, and the Conversational Search Subtask. We designed three types of search topics for this task: researchers (R), movies (M), and Youtube contents (Y). For each topic type, attribute sets are defined for considering group fairness. For the Web Search Subtask, we received 23 runs from five teams, including six runs from the organisers team. For the Conversational Search Subtask, we received four runs from two teams, including one run from the organisers team. In this paper, we describe the task, the test collection construction and the official evalution results of the submitted runs.

AI에게 질문

Bookmark

View Full Paper

Cite This Study

Tao et al. (Fri,) studied this question.

synapsesocial.com/papers/69cd79915652765b073a66e4 https://doi.org/https://doi.org/10.20736/0002002030

AI에게 질문

Bookmark

View Full Paper