The Impact of Domain Knowledge and Search Training on Search Effectiveness

To what extent do MeSH terms improve search effectiveness of different kinds of users? We observed four different kinds of information seekers using an experimental information retrieval system: 1. search novices; 2. domain experts; 3. search experts and 4. medical librarians. The information needs were a subset of the relatively difficult topics originally created for the Text REtrieval Conference (TREC). Effectiveness of retrieval was based on the relevance judgments provided by TREC. Participants searched either using a version of the system in which MeSH terms were displayed or another version in which they had to formulate their own terms. The results suggest that MeSH terms are more helpful in terms of precision for domain experts than for search experts. We speculate that this is because of the highly technical nature of the topics; only the domain experts had the knowledge to understand and therefore make use of the MeSH terms. The results advance our understanding of the usefulness of controlled vocabulary in interactive information retrieval systems.