UNIverse - Public Research Portal
Project cover

DRES - Distributed Retrieval Evaluation Server

Research Project
 | 
01.01.2021
 - 31.12.2024

Evaluation campaigns for interactive multimedia retrieval, such as the Video Browser Shodown (VBS) or the Lifelog Search Challenge (LSC), so far imposed constraints on both simultaneity and locality of all participants, requiring them to solve the same tasks in the same place, at the same time and under the same conditions. These constraints are in contrast to other evaluation campaigns that do not focus on interactivity, where participants can process the tasks in any place at any time. In this work, we are designing and implementing an evaluation scheme for interactive retrieval evaluation that relaxes both simultaneity and locality constraints, enabling participation from any place at any time within a predefined time frame. This scheme, as implemented in the Distributed Retrieval Evaluation Server (DRES), enables novel ways of conducting interactive retrieval evaluation and bridged the gap between interactive campaigns and non-interactive ones.

Members (3)

Profile Photo

Heiko Schuldt

Principal Investigator
MALE avatar

Ralph Gasser

Project Member
MALE avatar

Loris Sauter

Project Member