Simulating Interactive Information Retrieval
Simulation provides a powerful and cost-effective approach to explore and evaluate how interactions between a searcher and system influence search behaviour and performance. With a growing interest in simulation and an increasing number of papers using such an approach, there is a need for a flexible framework for simulation. Thus, we present SimIIR, an open-source toolkit for building and conducting Interactive Information Retrieval (IIR) experiments.
The framework consists of a number of high level components, including the simulation, the searcher and the system, all of which must be configured. The SimIIR framework provides a series of interchangeable components. Examples of these components include the querying strategies (how simulated queries are formulated) and stopping strategies (the depth to which a searcher will examine snippets and documents) that a simulated searcher will employ.
We have implemented various existing strategies so that they can be used by other researchers to not only replicate and reproduce past experiments, but also create new experiments. This paper describes the SimIIR framework and the different components that can be configured and extended as required.