Topic > Critical evaluation of evidence-based policy theory and practice

It is often noted that there is something intrinsically attractive about the concept of evidence-based policy (EBP) (Shillabeer et al., 2015). Indeed, the antithesis, that is, politics based on public opinion, is often seen as questionable (Davies, 2004). Inherent in these sensitivities, however, is a preconceived definition of “evidence.” For example, prevailing interpretations of “evidence” include positivist and empirical modes of input-output observation ( Green, 2000 ). In contrast, normative and theoretical reasoning is often ignored as an adequate evidence base (Sarat and Silbey, 1988:107), although it remains an essential and pervasive source of information for policy (Zane and Welsh, 2017). It seems that the term "proof" is very ambiguous and broad. It will be suggested that these ambiguities are central to the disparity between EBP theory and its implementation and practice. Furthermore, implicit in this disparity is the question of whether EBP is first possible and second, desirable. To begin, a review of the definitions and interpretations of the evidence bases, and their conflicts, will be provided. Implications relating to current conceptions of the phase model of the political process will follow. Third, the practice of EBP will be discussed in a sectoral comparison of medical and criminal policy. The importance of networks and “guidelines” for appropriate selection of evidence will be highlighted throughout the article. Say no to plagiarism. Get a tailor-made essay on "Why Violent Video Games Shouldn't Be Banned"? Get an original essay Conflicts between theory and practice are not unique to political studies. Indeed, such tension can also be found in the political (Ozcelik, 2006), mathematical (Malara and Zan, 2003) and aesthetic (Borgdorff, 2012) arenas. However, conflict resolution techniques in these areas remain underdeveloped (Ozcelik, 2006; Wells, 2004). For example, in the context of politics, Green (2000) and McLeroy et al. (1993) suggest that multiple theories may be needed in tandem to successfully develop programs and policies. However, the absence of a “guideline” or “rationale” for selection can significantly hinder the success of a policy (Green, 2000:126; Kriesberg, 1995:170; Buchanan, 1994). In this way, the theoretical foundation of EBP can be seen as incomplete: while there may be support for integrating evidence into policy, there is limited guidance on how the most effective evidence should be selected and assimilated. Furthermore, Davies (2004) notes that, firstly, the uncertainty of knowledge and, secondly, the varying status of knowledge fields, represent significant challenges for EBP. In other words, the status conflict between positive knowledge and normative knowledge and the lack of a coherent conflict resolution strategy between these contribute to an unstable theoretical basis for EBP (Zane and Welsh, 2017). That is, the value placed on empirical research often overlooks the essential function of normative research (Zane and Welsh, 2017). One result is a “knowledge vacuum” in which the surplus and preference for positivist evidence fails to advance policy as alternative forms are neglected (Davies, 2004: 4). Indeed, the report by Bullock et al. (2001) concluded that government departments use a limited range of evidence from a restricted set of resources. Furthermore, “uncertainty of knowledge” may arise from the ambiguity surrounding the definition of evidence and, as suggested above, what form it should take (Nutley et al., 2002). For example, the Oxford English Dictionary (OED) defines evidence as a “set of facts or information whichindicate whether a belief or proposition is true or valid" (OED, 2018). However, this definition is limited in two ways. First, there is an ambiguity regarding the existence of an objective "truth" free from ideological bias social and cultural (Nath, 2014). evidence should, therefore, be considered as universal, determinant and invariable” (Buchanan, 1994:274). (Buchanan, 1994; Green, 2000) Parkhurst (2017:7), for example, notes that technical biases can arise from the neglect of rigorous scientific practice in order to provide evidence to support certain political interests normative and positivist can therefore produce equally distorted results, calling into question the ability of evidence to provide a certain "validity". This is closely linked to the second limitation: implicit in the OED definition is the separation of "facts" and "information". . As Zane and Welsh (2017) suggest, positivist interpretations ('facts') remain largely descriptive while normative evidence ('information') tends towards prescriptive statements – although they differ in intention and methodologies, it is imperative that the two are recognized as inextricably linked. Otherwise, the potential of EBP may be limited to “how the world is” rather than “how the world should be” (Monahan and Walker, 1988:467). The use of normative evidence, for example, integrates unobservable facts, such as equality and fairness, into policy (Faigman, 1999). Furthermore, the theory can be used in policy evaluation to address a variety of inappropriate variables (Green, 2000:126). However, the exclusion of positivist evidence from the political process in exchange for regulation can result in the maintenance of power and social structures harmful to social equity (Sarat and Silbey, 1988: 107). Furthermore, as will be discussed later, policy formation in areas such as medicine is essential (Chalmers, 2003). The two, it seems, are mutually dependent, although this remains to be established in official political logic (Nutley et al., 2002:2). Again, the absence of a “guideline” to guide the appropriate selection of evidence may exacerbate this status conflict. Similarly, there is a conflict between propositional (i.e. formal, academic knowledge) and non-propositional (i.e. informal and obtained through experience) evidence ( Malone et al., 2004:83). On the one hand, the latter emphasizes the conditionality of evidence collection: normative evidence based on non-propositional knowledge can be limited by the number of individuals and their combined life experiences (Malone et al., 2004:83). Consequently, networks that combine expertise and experience are integral to expanding the evidence base (Nutley et al., 2002; Bowen and Zwi, 2005). This can help mitigate the gap between theory and practice as researchers and actors can use non-propositional knowledge to create alternative routes (or “guidelines”) to navigate the appropriate use of evidence (Bowen and Zwi, 2005). Furthermore, the possibility of normative, non-propositional “information” being converted into propositional, positivist “fact” (i.e., theory generation) emphasizes both the importance of partnerships and the interdependence of concepts (Titchen and Ersser, 2001) . However, such reasoning is somewhat idealistic as evaluation practices differ in terms of funding, capacity and rigor, thus limiting the potential for broad assimilation of evidence (Head, 2016). A broader interpretation of “evidence” should therefore be developed (Green, 2000); one that incorporates the wholespectrum of evidence and recognizes the unlikelihood of finding an objective “truth” (Higgs & Titchen, 1995). If a broader definition of evidence were to be adopted, it could be argued, the foundations of EBP would be stabilized, thus aiding its translation into practice. This conditionality is summarized in Nutley et al's (2002:2) four requirements for improving EBP: Agreement on what counts as evidence and under what circumstances A strategic approach to creating and accumulating robust evidence Effective dissemination of evidence and a wider access to knowledge Initiatives to ensure integration of evidence into practice Similarly, Weiss (1998) suggests that for EBP to be successfully implemented, the methodology and findings must be undisputed and support existing political ideologies, which policies have strong “champions” and the consequences reversible and robust (Nutley, Davies, & Walter, 2002; Weiss, 1998). If these are the optimal conditions for EBP, it follows that policies limited by a narrow understanding of the evidence do not provide a broad enough theoretical basis for the practice of EBP (Parkhurst, 2017). Indeed, “Modern Policy Making” (NAO, 2001) concludes that the use of a variety of knowledge sources and analytical skills is essential to risk management and EBP success. Furthermore, further theoretical inconsistency may arise from an inadequate conceptualization of the political process. That is, a simplistic conception of the policy process obscures the point at which evidence informs policy, complicating the translation of EBP into practice. For example, proponents of the linear model of the policy process suggest that policy formation begins with problem identification (Araral, 2012). Here, it is assumed that evidence can, and for some should (Andrews, 2017), inform policy formation from its early stages. Furthermore, feedback loops and the “policy evaluation” phase imply that the process is continually informed by evidence (SOAS, 2018; Sutton, 1999), implicitly suggesting that evidence provides the impetus for policy formulation (SOAS , 2018). Indeed, the usefulness and pervasion of the model are often attributed to its heuristic function and its simplistic idealism (Roe, 1991), thus mitigating the need for accurate and in-depth description (de Leon, 1999). However, the evidence is not used as comprehensively ( Richards, 2017 ), and the simplicity of the model has resulted in the development of more complex theories. For example, recognizing the possibility of externalities impacting the implementation or revision of a policy (i.e. a “policy window”), Kingdon (1999) postulates a more complex model. Drawing on Cohen et al's (1972) “rubbish bin”, Kingdon recognizes that organisations, problems, solutions and processes are non-linear and anarchic (Cairney, 2015). As a result, when problems arise, an organization can draw on a mix of solutions from the “garbage”, that is, a selection of resources yet to be assimilated into a solution that may be ineffective or preferable (ibid). The moment at which the solutions contained in the “rubbish bin” can be used, however, remains undetermined and therefore reflects the reality of the evidence used in the political process: information and facts can be introduced at any stage and with an uncertain purpose (ibid* ). Furthermore, Kingdon's (2003) description of a “primordial policy soup” recognizes that policies are continually evolving due to factors other than evidence, for example the strength of networks (Nutley et al., 2002; Bowen and Zwi , 2005). While Kingdon offers significant advances in the phase model, conceptions of the policy process continue to impede the theoretical foundations of EBP in two ways. First,simplistic models of policymaking continue to pervade and therefore contribute to an inaccurate portrayal of the use of evidence, where it is perceived to dominate the policy process (SOAS, 2018; Sutton, 1999). Secondly, the models fail to provide a causal explanation of how evidence enters the policy process (Howlett et al, 2014:10) and therefore hinder the formulation of a 'guideline' for the use of evidence. However, the development of more complex theories undermines the assumption that evidence is an integral part of the political process. This has implications for the use of the term “evidence-based policy”. Indeed, in recognition of the often subordinate use of evidence in policy, the term evidence-informed policy is preferred, despite the changing agenda (Head, 2016; Nutley, Davies, Walter, 2002). The concept of EBP gained significant traction in the late 1980s and 1990s with the rise of belief-based politics (Nutley, Davies, & Walter, 2002). Specifically, Labour's Modernization Agenda (1997) centralized EBP in an attempt to move from “what works” to “why it works” (Wells, 2004:6). However, some equate the rise with a technocratic politics and managerial agenda (Sanderson, 2002). Furthermore, the policies generated by this movement can be seen to overlook additional elements that inform policy formulation (Parsons, 2002). For example, the economic climate and political context can determine access to resources that influence policy implementation (Head, 2016). Such factors influencing policy are conceptualized by Kingdon (1999) as three streams (policy, issue and politics) that interact and, in the correct “window of opportunity”, converge to produce policy. For example, it has been suggested that EBP developed alongside a political climate of accountability in which taxpayers demand the efficient use of fiscal resources (Myers and Spraitz, 2011; Weiss et al, 2005:28). To take the example of the police; between 2010 and 2014, the “problem flow” of taxpayer demand and limited fiscal resources converged with the “political flow” of austerity, resulting in a 25% reduction in funding (NAO, 2015: 4). However, evidence suggests that these cuts could undermine the statutory foundations of networks and increase demands on police forces, thus having negative effects on crime control (Karn, 2013:21-31). The socio-economic political context, therefore, is given equal, if not more, consideration than evidence in policy formulation. As Davies highlights in Labour's agenda; politics is not just a question of “what works” but “at what cost” (2004: 5). However, similar to the normative-positive debate above, this example should not be indicative of the polarization between ideology and evidence. For example, Chang and Wang's (2016) study of the relationship between political discourse and empirical evidence suggests that they are inextricably linked: the use of empirical research in policy is designed to make causal inferences and are therefore essential to the normative reasoning of rhetoric politics. Furthermore, the interaction between the two is believed to be integral to the functioning of open and democratic societies (Davies, 2004:5; Dillow, 2014). This raises the question; to what extent should evidence inform policy if other factors are of equal importance? Certainly, there is no one correct “ratio” for policy formulation and the egalitarian use of resources and the type of evidence is not necessarily preferable in all public sectors (Nutley, Davies and Walter, 2002; etc etc**). Medicine, for example, has established a "hierarchy of evidence" which on the one hand gives priority tosystematic reviews and randomized experiments, and on the other hand devalues ​​political ideology, observational studies and professional consensus (Hadorn et al, 1996; Nutley, Davies, and Walter, 2002:3). Two reasons can be suggested for this. First, there is a consensus regarding the desired outcome of health policy (reduced mortality and greater health), thus facilitating methodological selection (Nutley et al, 2002:3). Secondly, the consequences of a less rigorous methodology are significant: reduced treatment effectiveness and therefore negative health effects (Moher et al, 1998; Nutley et al, 2002). Indeed, a greater status given to positivist evidence in medical policy has led to theoretical coherence across health care and the establishment of beneficial practices (such as integrating patient values ​​into physician behavior) and institutions (such as the Cochrane Collaboration) (Sackett, 1996). As a result, EBP minimized the effects of bias and chance (Chalmers, 2003). Furthermore, although this does not necessarily prevent the dissemination of methodologically weak evidence, the systematic approach to evidence selection and standardized practices for critical appraisal (e.g., by the Cochrane and Campbell Collaboration) offers a clear example of optimal EBP practice (Davies, 2004). ): 'what matters' as the evidence is easily understandable and guidelines for theoretical coherence are offered which allows for effective translation into practice. However, its practice is not easily transferable to other sectors. For example, the introduction of the “What Works Center for Crime Reduction” in 2013 aimed to “improve how government and other organizations create, share and use high-quality evidence for decision-making” (Hunter et al., 2016: iv ). However, seven limitations have been identified, three of which will be expanded. Firstly, it has been noted that there is no “well-developed road map” offering guidance on the evidence used in practice (Hunter et al., 2016: xi). Significantly, this has been attributed to limited development of a theoretical foundation (Hunter et al., 2016: xi). Secondly, the report identified that limited resources impede the optimal use of “evidence samples” and successful networking (Hunter et al., 2016: xiii). Third, they highlighted the need for “consistency in messages across […] various evidence mechanisms” (Hunter et al., 2016: xiii). These three limitations contrast with EBP in medicine and conflict with Nutley, Davies and Walter's (2002) four requirements for EBP, namely: A strategic approach to building and accumulating robust evidence Effective dissemination of evidence and wider access to knowledge Initiatives to ensure EBP Successful integration of evidence into practice Despite centralization on the political agenda, the use of evidence apparently fails to be successfully translated into practice in all sectors. Indeed, the success of EBP initiatives in education and social care has produced similarly ambiguous results (Bellamy, 2013). This calls into question the possibility that EBP can, and even should, be translated into practice across all sectors due to reasons of theoretical inconsistency. Finally, it is worth noting that although the theory and practice of EBP still contains significant limitations, this should not be taken as support for its abandonment. For example, the Police Research Group (an inter-agency research project) tasked with reducing burglary rates illustrates the potential for evidence to inform practice (Laycock, 2000). Analysis of crime rates concluded that most burglaries were repeated in the same location. Consequentially,.