The issue of preregistration in psychology has divided opinions across the scientific community
The issue of preregistration in psychology has divided opinions across the scientific community. Recent cases of fraud and the use of questionable research practices (QRPs) in research studies has led to the belief that the current method of publication in psychological research is no longer appropriate (Nosek & Bar-Anan, 2012). To increase transparency in research practices, some have called for preregistration to be adopted (van ‘t Veer ; Giner-Sorolla, 2016; Chambers ; Munafo, 2013). In reviewed preregistration (RPR), the requirement to detail underlying theories, hypotheses and all data analyses prior to conducting the research leaves little room for researchers to subjectively manipulate the data once it is analysed (van ‘t Veer & Giner-Sorolla, 2016; Kerr, 1998). Preregistration also aims to reduce the established bias towards only significant results (Gonzales & Cunningham, 2015). There are challenges to implementing this practice across all of psychological research, however by using incentives and encouraging a shift in attitude from the scientific community, preregistration can help pave the way for better research practices as a whole.
There are two kinds of preregistration, namely reviewed preregistration (RPR) and unreviewed preregistration (UPR). UPR employs a “self-registration” model, and a review is only conducted after data has been collected (van ‘t Veer ; Giner-Sorolla, 2016, p. 3). This method is problematic as researchers are still able to engage in QRPs. Alternatively, the process of RPR requires a review of research design, methods and analytic strategy prior to data being collected (van ‘t Veer & Giner-Sorolla, 2016). RPR prevents researchers from abusing the self-registration model in UPR. This essay will focus mainly on RPR as the specificities to this method may be more beneficial to psychological research.
The current bias towards publishing positive significant results in research papers may be improved by the use of RPR (Nosek & Lakens, 2014). The original model of publishing research has led to a bias towards studies with a statistically significant result, rather than evaluating the research based on its implications for the community (Giner-Sorolla, 2012). Replication studies and negative/null results are thus overlooked and rarely published. Rather than emphasising only significant results, with a RPR approach, the goals shift to having stronger research designs and theories behind the research (van ‘t Veer ; Giner-Sorolla, 2016). With the incentive of in principle acceptance (IPA) that RPR grants, researchers are not motivated to generate novel, statistically significant studies to appeal to publishers (Nosek ; Lakens, 2014). Without negative or positive results to base an opinion on, publishers are thus unable to reject strong research on the basis of unfavourable results. Adopting RPR across all areas of psychological research can not only affect bias, but help to reduce QRPs.
The negative stigma attached to undesirable results has created a “hypercompetitive academic climate” whereby significant results are prioritised over sound research and research practices (Pashler ; Wagenmakers, 2012). Aesthetically pleasing data and results are often published ahead of those that do not meet such criteria (Giner-Sorolla, 2016). As such, there have been numerous documented cases of QRPs among psychological research (Stroebe, Postmes ; Spears, 2012). One major motive of RPR is to reduce common QRPs such as p hacking, hypothesising after the results are known (HARKing) and “inappropriate statistical analyses” (Kerr, 1998, p. 209). As previously highlighted, the negative bias towards null results can motivate researchers to perform QRPs in the attempt to have their study published. For the paper to appear more appealing, researchers are able to redirect hypotheses after seeing the data. This practice clearly undermines the credibility of the research (van ‘t Veer & Giner-Sorolla, 2016). Highlighting more transparent research practices may help foster a better environment in the scientific community. Emphasis is placed on sound research practices rather than on the aesthetics of research (Giner-Sorolla, 2012).
Preregistration may help reduce type 1 errors and reporting biases (Gonzales & Cunningham, 2015; van ‘t Veer ; Giner-Sorolla, 2016). Simmons, Nelson, and Simonsohn (2011) argue that flexibility in data analysis procedures leads to an increase in type 1 errors, meaning one falsely discerns that a significant effect has taken place. If data analysis procedures are not outlined a priori, researchers can choose to perform analyses they know will give them ideal results (van ‘t Veer & Giner-Sorolla, 2016). RPR requires the researcher to outline beforehand what analyses will be performed on the data, preventing this reporting bias. Whilst some may view RPR as restrictive, reducing flexibility in data analysis may lift the doubt clouding the psychological community (Pashler & Wagenmakers, 2012). Without the ability to subjectively analyse the data, reporting bias is reduced (van ‘t Veer ; Giner-Sorolla, 2016).
Despite the benefits preregistration provides, there are a number of concerns. One such issue is that exploratory research will be negatively biased. This is due to the fact that RPR is not applicable to exploratory research in the same way as confirmatory research. Exploratory research generally requires a post hoc interpretation based on the results obtained. This contradicts the general idea of RPR. If required to preregister exploratory studies, researchers may feel pressured to identify theories without results, leading to false interpretations (van ‘t Veer & Giner-Sorolla, 2016; LeBel & Peters, 2011). Gonzales and Cunningham (2015) also highlight that this may lead to a reduction in funding for exploratory research. In order to prevent a negative stigma towards these studies, Watt and Kennedy (2015) recommend an alternative approach to reviewing exploratory research that requires researchers to clearly define their research as either confirmatory or exploratory prior to registration. Researchers are thus encouraged to make post hoc interpretations in exploratory settings, provided they label them as such (Watt & Kennedy, 2015). Having a clearly defined study that is either exploratory or confirmatory can also aid in the reliability of the results obtained (Wagenmakers et al., 2011).
Another proposed drawback of preregistration is that it does not prevent fraud (van ‘t Veer ; Giner-Sorolla, 2016). It partly relies on the assumption that researchers will act honestly, which is not always the case. The process of preregistration is vulnerable to the fabrication of dates as well as registering a study under a different hypothesis (van ‘t Veer & Giner-Sorolla, 2016). However fraud is not limited to preregistration, with cases of falsification and scientific misconduct still occurring outside this method (Bem, 2011). Prominent psychologist Diederik Stapel fabricated data across numerous studies throughout his career, many published in prestigious journals. Stapel’s cases of misconduct only came to light following the concerns of a whistle-blower (Stroebe et al., 2012). Had Stapel’s research been subject to RPR, the higher level of scrutiny required may have allowed for earlier detection of his data fabrication. Ultimately, preregistration will not completely prevent fraud. However as previously stated, through the reduction of publication bias, researchers may not feel as strongly motivated to engage in research misconduct.
The transition to adopting preregistration across psychological research will take time and commitment from researchers and publishers. The focus for the future should be on the education of preregistration as well as expanding incentives for engaging in preregistered research. Currently, the incentives for many researchers to participate in RPR are not widely known or universally applicable, thus the extra work required to register research is not seen as worthwhile (Giner-Sorolla, 2012). Open Science Framework (OSF) offered monetary prizes for preregistration research teams as a way to promote the practice, whilst other Journals have created special preregistration issues (van ‘t Veer ; Giner-Sorolla, 2016). Having a greater range of publishers engaging in such incentives may lead to a greater number of researchers choosing to participate in the practice. Especially for younger researchers, providing information and accessible means to preregister research can be a step towards adopting this practice across psychology.
Moving forward, a major shift is needed in the scientific community in order for preregistration to work effectively. Researchers and publishers alike must emphasise sound research methods over significant results. Without this focus on the aesthetics of research, replication studies and those with null results may not have the same negative stigma attached to them (Giner-Sorolla, 2012). Despite the implication that exploratory research will be negatively impacted, having a clear distinction between exploratory and confirmatory research will mean that different methods of preregistration can be applied. This again can ensure the intentions of one’s research are clearly defined (Watt & Kennedy, 2015). Ultimately, by helping reduce reporting and publication biases, preregistration can help in the reduction of QRPs in psychology.
