Quality-assuring practices
In addition to the observation of quality-reducing practices, the extent to which quality-assuring practices are routinely carried out in the respective research culture was also investigated. A distinction is made here between measures that could in principle be integrated into all research fields and those that are more field-specific. Measures can occur in all research fields if they are independent of the specific objects, methods and questions used in the field. These include the following measures:
- Detailed documentation of research steps,
- Discussion of research results with colleagues,
- Internal quality assurance (4-eyes principle) before submitting third-party funding applications,
- Internal quality assurance (4-eyes principle) before submitting manuscripts for publication, and
- Publication with publishers that use peer-review processes.
On the other hand, there are measures that are currently being pushed in some disciplines, but which also occupy a great deal of space in the overarching discussion about research quality, and it is questionable whether they can be meaningfully transferred to other fields. These include:
- conducting replication studies,
- publishing null results or
- pre-registering research designs.
While replication studies are useful in laboratory experimental research and are typically carried out, the measure cannot be transferred one-to-one to so-called natural experiments. In turn, null results only arise in hypothesis-testing research. The idea of pre-registering research designs is often based on stereotypical notions of how research should be conducted, so it would be necessary to examine carefully whether and, if so, how pre-registration is useful at all within the respective subject.
Figure 59 shows that cross-disciplinary and method-independent quality assurance measures are quite widespread. For example, 89% of respondents state that they discuss research results with colleagues in the field. 87.2% state that they publishin peer-reviewed journals. Internal quality assurance (four-eyes principle) before submitting manuscripts or third-party funding applications is also quite widespread. In the case of manuscripts, 72.3% of researchers practice this, and in the case of third-party funding applications, 55.6% still do so.
A comparison of the various subject groups reveals some differences (see Figure 60). In the natural sciences, internal quality assurance is carried out slightly less often before the submission of third-party funding applications (see Figure 60). The humanities fall slightly behind the other subject groups when it comes to the detailed documentation of research steps. The life sciences are leading the way here. Internal quality assurance (4-eyes principle) before submitting manuscripts is also practiced somewhat less frequently in the humanities than in other subject groups. This is partly understandable, since work is more often done alone and not in a team in the humanities. On the other hand, this circumstance does not per se exclude the possibility of “cross-reading” by colleagues. In addition, the humanities are not quite as far along as the other subject (groups) when it comes to publishing in peer-reviewed journals. The differences in publication cultures are well known. For example, researchers in the humanities and, to some extent, the social sciences, publish more frequently in the form of monographs and edited volumes (Kulczycki et al. 2018). These subject groups seem to be under increasing pressure to align their publication cultures with peer-reviewed articles (Schneijderberg et al 2022).
As expected, field-specific quality control measures, i.e. those dependent on the research method, are much less common (see Figure 61). Only 13.1% of respondents said that replication studies were conducted, while 14.1% said that zero results were published. When comparing the subject areas, it is clear that field-specific quality control measures are most commonly used in the life and social sciences (see Figure 62). There are two reasons for this: on the one hand, the life sciences and parts of the social sciences tend to work experimentally and by testing hypotheses. At the same time, in some of these subjects, such as medicine, psychology and economics, such measures are currently being strongly promoted by reform movements.
Research quality is one of the major topics in research management and in debates on research policy. In the BSS, an attempt was made to approach the topic not via cases of research misconduct, but via the research cultures, the implemented practices and the research orientations of the scientists and scholars. It was found that research quality is of very high importance. The research-immanent goals of “methodological rigour” and “originality of research results” are seen by respondents as significantly more important goals than, for example, “third-party funding” and “publication output,” and are accordingly prioritised most highly in practice. This is the case even though other goals are associated with greater pressure on researchers to meet expectations.
With regard to research practices, it can be seen that quality assurance measures are implemented or routinely carried out in the vast majority of research contexts. However, there is also a minority for whom one or other quality assurance measures could be used even more regularly. Quality risks arise not least from excessive workloads. Of all respondents, 28.5% stated that they “often”, “very often” or even “always” have to compromise on quality at work due to heavy workloads. In 80% of cases, this is at the expense of research.