Developing and testing quality criteria for research in the humanities

The aim of this project is to develop quality criteria for research in selected subjects of the humanities that cannot be effectively represented using conventional bibliometrics. The project will be conducted in two steps. In the initial exploratory phase, quality criteria for literary studies and art history will be compiled using empirical methods, an analysis of specialist literature, and an in-depth study of current practices. The second phase is a testing procedure in which the existing data from the disciplines will be correlated with the quality criteria compiled in the first phase. The goal is to translate quality criteria into a set of indicators that can be used to assess the quality of research output and to enhance its visibility.

 

The quality criteria are developed using various methods (repertory grid technique, the Delphi method, review of specialist literature, structured interviews, good practice reviews) and differing paradigmatic focuses are taken into consideration (e.g. in literary studies: hermeneutics and reception theory). To begin, an initial set of quality criteria for research will be defined using the repertory grid technique to survey selected experts in a given subject; this information is then supplemented with a documentation of the criteria from reviews of specialist literature and from established international practices for evaluating research output. The good practice review will commence parallel to the repertory grid survey and continue while the Delphi method is applied. The review of specialist literature is generated via surveys of selected researchers (perspective of those being evaluated) as well as via interviews with representatives of the institutions (perspective of decision-makers and, in some cases, evaluators). Interim findings are incorporated in the Delphi questionnaires; final results are incorporated in the final report.

 

The results from the survey of experts, the literature review and initial findings from the good practice review provide the basis for an international-European Delphi questionnaire; this will subsequently serve to adapt quality criteria, to complement them and to place them into context. The Delphi questionnaire comprises three rounds of questioning in which widely renowned scholars from the relevant disciplines in Europe are asked the following: 1) to name quality criteria for research in their discipline, 2) to evaluate the quality criteria and 3) to assess the applicability of the criteria. Once generated, the quality criteria will undergo statistical analysis and be assigned dimensions, with sets of indicators being allocated to each dimension. Translating quality criteria into indicators and incorporating available data will occur in the framework of a feasibility study in close collaboration with specialists at the University of Zurich and the University of Basel. The specialist in Zurich will be employed 50% in the university’s controlling office and 50% at the Professorship of Social Psychology and Research on Higher Education at ETH Zurich (Prof. Dr H.-D. Daniel). In Basel, the specialist is employed at the Vice Rectorate for Research.

 

The initiative’s website: www.psh.ethz.ch/crus