When There Is Not Enough Evidence and When Evidence Is Not Enough: An Australian Indigenous Smoking Policy Study

Background The Indigenous Tobacco Control Initiative and Tackling Indigenous Smoking Measure were both announced by the Australian Government at a time when its rhetoric around the importance of evidence-based policy making was strong. This article will (1) examine how the Rudd Government used evidence in Indigenous tobacco control policy making and (2) explore the facilitators of and barriers to the use of evidence. Methods Data were collected through (1) a review of primary documents largely obtained under the Freedom of Information Act 1982 (Commonwealth of Australia) and (2) interviews with senior politicians, senior bureaucrats, government advisors, Indigenous health advocates, and academics. Through the Freedom of Information Act process, 24 previously undisclosed government documents relevant to the making of Indigenous tobacco control policies were identified. Interviewees (n = 31, response rate 62%) were identified through both purposive and snowball recruitment strategies. The Framework Analysis method was used to analyze documentary and interview data. Results Government policy design was heavily influenced by the recommendations presented in government authored/commissioned literature reviews. Resulting policies were led by equivocal evidence for improved tobacco control outcomes among Indigenous Australians. Many of the cited studies had methodological limitations. In the absence of high-quality evidence, some policy makers supported policy recommendations that were perceived to be popular among the Indigenous community. Other policy makers recognized that there were barriers to accumulating rigorous, generalizable evidence; in the absence of such evidence, the policy makers considered that the “need for action” could be combined with the “need for research” by introducing innovative strategies and evaluating them. Discussion Despite the absence of high-quality evidence, the formulation and adoption of Indigenous tobacco policy was neither irrational nor reckless. The decision to adopt an innovate and evaluate strategy was justifiable given (a) the potential for the gap between Indigenous and non-Indigenous health outcomes to worsen in the absence of an imminent policy response; (b) the existence of circumstances, which made it difficult to obtain high-quality evidence to guide policy; and (c) the need for policy solutions to reflect community preferences, given sociohistorical sensitivities.


Background: The Indigenous Tobacco Control Initiative and Tackling Indigenous
Smoking Measure were both announced by the Australian Government at a time when its rhetoric around the importance of evidence-based policy making was strong. This article will (1) examine how the Rudd Government used evidence in Indigenous tobacco control policy making and (2) explore the facilitators of and barriers to the use of evidence.
Methods: Data were collected through (1) a review of primary documents largely obtained under the Freedom of Information Act 1982 (Commonwealth of Australia) and (2) interviews with senior politicians, senior bureaucrats, government advisors, Indigenous health advocates, and academics. Through the Freedom of Information Act process, 24 previously undisclosed government documents relevant to the making of Indigenous tobacco control policies were identified. Interviewees (n = 31, response rate 62%) were identified through both purposive and snowball recruitment strategies. The Framework Analysis method was used to analyze documentary and interview data.
results: Government policy design was heavily influenced by the recommendations presented in government authored/commissioned literature reviews. Resulting policies were led by equivocal evidence for improved tobacco control outcomes among Indigenous Australians. Many of the cited studies had methodological limitations. In the absence of high-quality evidence, some policy makers supported policy recommendations that were perceived to be popular among the Indigenous community. Other policy makers recognized that there were barriers to accumulating rigorous, generalizable evidence; in the absence of such evidence, the policy makers considered that the "need for action" could be combined with the "need for research" by introducing innovative strategies and evaluating them.
Discussion: Despite the absence of high-quality evidence, the formulation and adoption of Indigenous tobacco policy was neither irrational nor reckless. The decision to When Evidence Is Not Enough Frontiers in Public Health | www.frontiersin.org October 2016 | Volume 4 | Article 228 inTrODUcTiOn "Evidence-based policy" has become a synonym for "good policy," prompting the observation that "[i]t is difficult to imagine anyone arguing that policy should be based on anything but the best available evidence" (1). In a critical analysis of literature relating to the use of evidence in health policy making, Oliver et al. noted that the majority of studies made normative assumptions that policies should be based on evidence, and that any barriers between evidence and policy ought to be overcome (2). However, few of the studies used empirical data to demonstrate why evidence-based policy making is superior to other forms of decision-making. Similarly, only a small number of studies provided in-depth empirical descriptions of the importance of evidence relative to other considerations. Numerous models and theories conceptualize evidence as just one part of a broader scheme of competing influences on policy. For instance, Walt and Gilson considered that the content of health policies is affected by an interplay between context (e.g., culture, economic factors, demography, history, ideology), actors (i.e., individuals, groups, or organizations and their position in the power hierarchy), and the process through which policy issues are communicated, negotiated, and decided (which can include the consideration of evidence) (3,4).
A contemporary adaptation of this model is Lin's description of health policy as the product of three competing rationalities (5). "Cultural rationality" is defined as "values, ethics, what (perceived) societal opinions feel is right in relation to health policy" and thus aligns with what Walt and Gilson term "context" (5). "Political rationality" relates to the process through which power is exercised and decisions are made and includes such factors as "the willingness of policymakers to have transparent processes and be accountable, the ability of interest groups to participate … and the role of commentators (be it media, experts, or lobbyists)" (5). Finally, "technical rationality" describes the knowledge produced by researchers and can include diverse forms of evidence, such as epidemiology and economics (5). Lin argues that these rationalities (and the policies that they create) are shaped by "historical political legacies" and reflect "ongoing processes of social learning" (5).
In view of such rich theoretical explanations, Oliver et al. call for a new research agenda, which focusses on the "influences on and processes of policy" through in-depth, empirical descriptions of how evidence "fits with the other drivers and triggers that affect policy" (2). This article uses the development of tobacco control policies for Indigenous Australians as a case study for examining the real-world tensions inherent in the health policy-making process.
When the Labor Party assumed government in 2007 under the leadership of Prime Minister Kevin Rudd, Indigenous Australians were expected to have lives that were 17 years shorter than their non-Indigenous counterparts (6). The Labor Party argued that such enduring disparities were a product of ineffective, ideologically driven policies and that "from here on our guiding principle will be the evidence of what works and what does not work in reducing disadvantage" (7).
The Indigenous Tobacco Control Initiative and Tackling Indigenous Smoking Measure committed over AUD 120 million in government funds and marked the beginning of a period "that has seen more action on Indigenous smoking than any other time in our history" (8). Yet, perhaps surprisingly, in light of the foregoing context, it was said of some aspects of the policies that "the government hasn't produced evidence to back up its campaign" (9). This article will (1) examine how the Rudd Government used evidence in Indigenous tobacco control policy making and (2) explore the facilitators of and barriers to the use of evidence relative to other factors.
In so doing, this article reveals difficulties in the application of evidence-based policy making. These difficulties include debates as to what constitutes "evidence" and a tension between the need for good evidence and the need for urgent policy action. The article also demonstrates how, in some circumstances, the need for evidence of effectiveness can be exceeded by the need to empower the target population in the decision-making process.

MaTerials anD MeThODs
Data were collected through a combination of document reviews and interviews. The overarching purpose of data collection with respect to each method is set out in Table 1 below. Data collection and subsequent analysis were informed by awareness of the possibility that key factors might turn out to relate to "context/ cultural rationality" and "political rationality/process" as much as "technical rationality. " It is considered prudent to combine these two methods when conducting policy process analyses to mitigate their respective limitations (4,10). These limitations include the fact that (1) some important events in policy processes are often not recorded, or the documents may be withheld from public access; (2) documents can possess official and perhaps incomplete versions of events; (3) respondents may underrepresent or overrepresent their role in a policy process; and (4) the passage of time might impede a adopt an innovate and evaluate strategy was justifiable given (a) the potential for the gap between Indigenous and non-Indigenous health outcomes to worsen in the absence of an imminent policy response; (b) the existence of circumstances, which made it difficult to obtain high-quality evidence to guide policy; and (c) the need for policy solutions to reflect community preferences, given sociohistorical sensitivities. respondent's ability to recall events accurately or with sufficient detail (10).

Documentary Data collection
A two-stage process was adopted to identify relevant documents.
In the first instance, publicly available documents were obtained by searching the following websites/databases: Where search functions were available, the search was limited to items containing the search terms "(Aboriginal OR Indigenous) AND (tobacco OR smok*). " Where search functions were not available, items were individually screened by title for relevance. Relevant documents were limited to those produced between 2007 (election of Rudd Government) and 2008 (announcement of policies).
The second-stage of the document review was focused on obtaining relevant documents produced for the private use of members of the policy-making community. These documents were either provided by the interview respondents or accessed under the federal Freedom of Information Act 1982 (FOI).
The initial request (FOI Request 309-112) for "written reports/ letters of advice and internal reports or memoranda used by the Government to inform the design of the Indigenous Tobacco Control Initiative" resulted in an estimated charge of AUD 10,845 for the time that FOI officers and document custodians would likely spend searching for, retrieving, and assessing the documents. Further particularized requests were then submitted (FOI Requests 006-1314 and 067-1213) seeking access to the following: The revised requests yielded 24 previously undisclosed documents relevant to the making of the policies. One document was not disclosed on the basis that it was produced for Cabinet deliberations and therefore exempt under the legislation.

interview Data collection
Interview participants were recruited through a combined purposive and snowballing sampling strategy, as recommended for studies of this nature (10,11). A preliminary list of individuals involved in the policy-making process was created using the documents described above. These purposefully selected respondents were asked to name other people who influenced the policy process in relation to Indigenous tobacco control.
Fifty potential respondents were invited to participate in interviews. A source population of 50 policy actors was considered reasonable, given that the study was limited to two specific policies. Thirty-one individuals agreed to participate in the study, representing a response rate of 62%. The response rate is consistent with those obtained in other studies of the public services sector (12).
The sample comprised advisors to the Federal Minister for Health (n = 2), senior Federal politicians (n = 2), senior Federal health bureaucrats (n = 4), members of and assistants to the National Preventive Health Taskforce and its Tobacco Working Group (n = 9), members of the National Indigenous Drug and Alcohol Committee (n = 2), researchers/academics (n = 4), and Indigenous health advocates (n = 8). Anonymity was guaranteed to protect reputations and interests. Of those individuals who did not respond to or declined interview invitations, four were politicians, seven were government advisory group members, one was a senior bureaucrat, four were researchers, and three were Indigenous advocates.
Interviews were audio recorded and transcribed. Interviews were semi-structured, using a combination of open-ended and probing questions. Topic guides were prepared before each interview and were tailored to the knowledge and experience of each respondent. Where relevant, respondents were asked to describe the following: • factors that might explain the policy attention given to Indigenous tobacco control under the Rudd Government; • how the policies were developed; • factors that might explain why certain approaches to tobacco control were prioritized over others; • the role of evidence in the policy process; • facilitators of and barriers to the use of evidence.

Data analysis
Data were analyzed using the Framework Analysis method. The process involved (1) becoming familiar with the data by listening to/transcribing the recordings and reading the transcripts and documents; (2) noting down recurring themes and subthemes (see Table 2 below); (3) using QSR International's NVivo 10 software to code the data according to the themes; (4) transferring the coded data into charts (one for each theme), which were organized in rows by source and in columns by subtheme; and (5) looking down the chart columns to observe recurring patterns and tensions.

resUlTs
The role of evidence in getting indigenous Tobacco control on the agenda There had long been evidence to indicate high rates of Indigenous smoking. A 1994 survey of over 15,700 Indigenous Australians found that 49.7% of respondents over the age of 13 years smoked daily (13). Similarly, a survey conducted a decade later  found that 50% of Indigenous Australians aged 18 years and over (n = 10,439) were current daily smokers, compared to 23% of the general population (14).
Despite these data, there had been an absence of significant government investment specifically geared toward reducing Indigenous smoking rates prior to 2008 (15). For instance, an expenditure analysis revealed that smoking was a focus in only 2 and 3% of Indigenous-specific alcohol and other drug intervention projects in 1999-2000 and 2006-2007, respectively (16).
The interview findings suggest that the increased political attention given to Indigenous smoking in 2008 was the product of a confluence of three factors -namely -(1) the publication of evidence showing that tobacco was the single largest contributor to the "gap" between Indigenous and non-Indigenous health outcomes; (2) the success of the Closing the Gap advocacy campaign, which was led by Indigenous and civil society groups and supported by the incoming Labor Government; and (3) the appointment of a Health Minister committed to tobacco control, perhaps best evidenced in the Rudd Government's decision to make Australia the first country in the world to introduce a law to mandate plain cigarette packaging.

Burden of Disease Study
Six respondents cited a burden of disease study by Vos et al. published in 2007 (hereafter, the Vos study) as an explanation for why Indigenous smoking became more politically prominent in 2008 (17). The study found that, of 11 risk factors, tobacco was the largest single contributor to both the overall Indigenous disease burden (12%) and the health "gap" between Indigenous and non-Indigenous Australians (17%) measured in disabilityadjusted life years.
One senior bureaucrat said of the Vos study, "the study was so important. " The significance of the Vos study was that it "show [ed] people what were the contributing factors to excess mortality" (senior bureaucrat). An Indigenous leader and health advocate, recalled that Vos' work prompted an Indigenous research organization to focus on smoking:

[The Cooperative Research Centre for Aboriginal and Torres Strait Islander Health] put briefings together and all that and started to tell their story to the Minister … And then they did this presentation in Parliament House. Within a fortnight [the Government] found $14 million.
The presentation to which the respondent referred formed part of the Parliamentary Showcase of Aboriginal Health Research, which was held in March 2008 and opened by the Health Minister. As part of the program, Viki Briggs of the Centre for Excellence in Indigenous Tobacco Control (CEITC) delivered a presentation entitled "Reduce Smoking, Reduce the Gap" in which the findings of the Vos study were highlighted (18). Another respondent considered that "that was probably an absolutely crucial event" (Indigenous tobacco control academic). A Ministerial advisor considered that the event "helped crystallise the view that tobacco was a critical factor. " The "Closing the Gap" Movement Four respondents were of the view that the impact of the Vos study must be considered in its sociopolitical context. The study was published in 2007 at a time when the Closing the Gap campaign for Indigenous and non-Indigenous equality was gaining momentum (19). In the lead up to the 2007 Federal election, the Labor Party made a commitment to improve Indigenous social and health outcomes in response to the Closing the Gap campaign.
Evidence demonstrating the impact of smoking on the gap between Indigenous and non-Indigenous health outcomes found fertile ground in this sociopolitical landscape. According to a senior politician: Similarly, a senior bureaucrat explained that a "catalyst" for political action "was that we had a figure, a statistic that said smoking causes 20% of all mortality. It came at the same time as Closing the Gap. So, okay, to close the gap … you've got to prevent premature mortality. " A Ministerial advisor agreed that "we were never going to achieve this [aim to close the gap in health outcomes] if we didn't do something about Indigenous smoking. "

Political Leadership
While the confluence of evidence and political agendas created a positive environment for Indigenous tobacco control to become more politically salient, four respondents emphasized the crucial role of the Health Minister in actually securing policy action. It was noted that DoHA's interest in Indigenous tobacco control predated the policy developments in 2008 but that, despite this, the issue never gained political attention. According to a senior bureaucrat, "[w]henever we put a case around a smoking intervention, every minister would say, ' • offer support and training for local health staff to deliver brief tobacco cessation interventions and programs that include nicotine replacement therapy; • provide more intensive interventions in targeted communities/ regions; • develop media campaigns featuring local identities or artwork; • fund the CEITC to audit evaluations of tobacco control initiatives. 1 Table 3 below contains the studies cited in DoHA review to support each policy proposal and provides a summary of their major findings and relevant limitations.   (29).
The interview data suggest that DoHA played a key role in the development of this policy also. According to a Ministerial advisor, "the Department developed the proposals that we kind of further refined in cooperation with them I guess. " Interviews with DoHA staff confirmed the central role of the Department in designing the policy and also highlighted the informal influence of external advisors. A senior bureaucrat recalled: In the early stages of [ DoHA also commissioned CEITC to draft a literature review and a scoping paper to inform its response to Indigenous smoking rates. The resulting paper was provided to DoHA in September 2008.
Additionally, advice on the formulation of the policy was received from the National Preventative Health Taskforce. The National Preventative Health Taskforce was established by the Government to assist in the development of strategies to prevent diseases related to tobacco, alcohol, and obesity. While the Taskforce did not release its final recommendations until 2009, a Government report revealed that interim advice was provided in 2008 (30). According to a member of the Taskforce's Tobacco Working Group, "[t]hey involved us in the development of the Tackling Smoking Initiative … We were doing the same recommendations formally and informally. " The Director of CEITC was a member of the Tobacco Working Group.
The formal recommendations published by the Tobacco Working Group reproduced almost verbatim the recommendations contained in the literature review and scoping paper drafted by CEITC for DoHA (31). The relevant recommendations in the CEITC paper were to • increase the Indigenous tobacco control workforce; • improve health worker access to appropriately designed training; • improve access to pharmacotherapies; • develop a well-researched and focus-tested Indigenousspecific social marketing campaign at national and local levels; • raise the profile of Quitlines to Indigenous communities (32).
These recommendations closely aligned with the Tackling Indigenous Smoking Measure, which committed AUD 100.61 million to reduce Indigenous smoking rates and the burden of tobacco-related disease through a number of initiatives, the most prominent of which were the following: • establish a national network of tobacco action coordinators; • develop a national Indigenous tobacco action training program for health workers and community educators; • strategies to improve delivery of smoking-cessation services, including nicotine replacement therapy; • social marketing campaigns to reduce smoking-related harms among Aboriginal and Torres Strait Islander peoples; • enhance Quitline to provide culturally sensitive services (29,33). Table 4 below summarizes the evidence cited in the CEITC paper to support each policy proposal and notes major findings and limitations of each.

rationale for adopting Policy Proposals in the absence of strong evidence
It is clear from Tables 3 and 4 that not all of the policy initiatives were grounded in strong evidence demonstrating their likely effectiveness. A critical analysis of the studies revealed that many had limitations, such as inadequate reporting of methods, selection bias, response bias, and small sample sizes. Most Indigenous Australian studies measured subjective preferences for interventions, perceptions of likely impact of interventions, or thoughts and attitudes about smoking. In the absence of rigorous evidence from the Australian context, some reliance was placed on studies of indigenous populations in other countries. Interviews were used to understand why the policy makers  Policy proposal supporting evidence cited in ceiTc review

Type of evidence relevant finding(s) limitations
Ivers et al. (48) • 351 (mostly Indigenous) people from a source population of 1,228 residents participated in a community survey at both baseline and 12-month follow-up • Populations were exposed to multicomponent tobacco control interventions, including advertising • 86% of smokers recalled seeing non-Indigenous anti-tobacco advertisements • 10 people who claimed to be smokers at baseline visit had quit at follow-up • Those who recalled seeing the advertisements were not significantly more likely to quit than those who did not • Logistic regression showed that exposure to individual tobacco interventions was not associated with an increased chance of cessation during intervention year

Impractical to Wait for Rigorous Indigenous-Specific Evidence
Faced with both a dearth of evidence and an understanding that Indigenous smoking rates were inexorably high despite the presence of mainstream tobacco control strategies, some policy makers saw the need to take action based on inference. An Indigenous tobacco control academic with close links to key policy makers noted that "the problem is we didn't have any evidence from this [Indigenous] setting so … people had to make guesses. And they probably weren't bad guesses. " A Tobacco Working Group member used similar language, describing the research conducted to develop policy priorities as "predominately guess-based. " An innovate and evaluate approach was accepted as being an appropriate solution in the circumstances. Internal documents obtained under FoI demonstrate the extent to which policy makers considered that it was both necessary and justifiable to act without a strong evidence base, so long as there was a commitment to accumulating knowledge in the process: Currently there is little available evidence for what works in Indigenous tobacco control. In partnership with research bodies … the Government can make an immediate start on building that evidence-base … In tandem with the research program, it is proposed to run five or six pilot projects in selected locations to trial innovative approaches to smoking prevention and cessation … Evaluation of these pilots could inform future phases of the initiative … The approach recommended here combines action (the pilot programs) coupled with a strong research focus to ensure that resources are not wasted on misdirected initiatives. 2 The alternative approach -that is, waiting for evidence to emerge before taking policy action -was not considered practical. Two policy makers noted that the Indigenous context presented distinct challenges to the conventional understanding of evidence-based policy making, arguing that evidence needed to play more of a back-end role. According to a Tobacco Working Group member:

[W]e need to look at ways of finding types of intervention that would resonate better with Aboriginal communities and I think a part of what came with that [was] also an acceptance that certainly when it comes to mainstream
programs, it is driven by research … but … when it comes to Aboriginal smoking there's a need to be willing to take a few more risks, and trial different ways of addressing the problem.
Researchers with links to policy makers expressed the view that the Indigenous research context presented distinct challenges. One respondent offered the following example of the limitations: [T]he logistics of doing high level randomised controlled trials in [remote] populations … it's ridiculous … There were a whole lot of issues with cross-contamination between communities … We did the numbers for a randomised control trial in giving quit advice … the numbers to do a proper randomised control trial in the Northern Territory were going to be 30,000 participants. You couldn't get it … (Indigenous tobacco control academic).
An Indigenous advocate with research experience added: [T]here's always challenges from the remoteness of going out to do the pre-and post-questionnaires … a lot of these people move around in their communities … They will travel around and visit family and the neighbouring community. Could be anything … even the weather … From all those kind of logistic things, to even language being a barrier.

Policy Proposals Were Championed by Indigenous Representatives
Not all members of the policy community subscribed to the view that mainstream tobacco interventions were ineffective for Indigenous people, or that the Indigenous-specific interventions recommended in the CEITC paper were needed. One Tobacco Working Group member warned against "overdoing the differences" and noted that "Aboriginal people are people. They smoke like other people. They quit like other people. And I think we need to be almost wary of assuming that you need to have something incredibly and dramatically different. " Another Working Group member concurred, arguing that "there is absolutely zero evidence that those [mainstream] approaches do not impact Indigenous people in the same sort of was as they affect the rest of us. " Longitudinal studies lend some support for this view showing that, in the absence of major Indigenous-specific policies, daily smoking prevalence for Indigenous males over the age of 18 years decreased from 58.5% in 1994 to 52.6% in 2008 with a statistically significant annual decrease of 0.4% per year (52).
The apparent mismatch between the advice provided to Government and the view of some respondents that some Indigenous-specific interventions were not necessary was explained in the following way:

DiscUssiOn
This case study has demonstrated empirically what the theoretical models of policy making suggest -namely -that evidence does not exist and cannot be used in a vacuum. Rather than viewing evidence as being in competition with other policy factors, this case study presents evidence as being dependent on them.
The intuitive appeal of evidence-based policy making lies in the perception of objectivity -evidence "speaks for itself " (53). Certainly, the data suggested that evidence played an important role in getting Indigenous smoking onto the political agenda. The salience of the Vos burden of disease study was repeatedly described in interview responses and relevant documents (17). However, the findings show that while "[s]cience may be able to deliver empirical knowledge … this knowledge has to be interpreted and warranted in order to be understood" (54).
In this instance, there were two factors that enabled the evidence to "speak" in the policy process. First, the evidence emerged against the backdrop of a popular sociopolitical campaign to reduce health disparities between Indigenous and non-Indigenous Australians; this aligns with both Walt and Gilson's notion of "context" and Lin's definition of "cultural rationality" as "what (perceived) societal opinions feel is right" (3,5). Second, the evidence was understood and valued by key figures in Government (including the Minister for Health) who had the power to take action in response; this reflects the importance of "actors" and "process" in Walt and Gilson's model, and "political rationality" in Lin's schema (3,5).
The symbiotic relationship between evidence and other policy factors was further demonstrated in the policy design stage. The recommendations presented in the government authored/ commissioned literature reviews (obtained under FoI) seem to have heavily influenced the content of the policies. In relation to the Indigenous Tobacco Control Initiative, "political rationality" and "context" seem to be major explanations for the fact that most of the recommendations emerging from the literature review were adopted in the policy. The political pressure to make a major policy announcement at the Indigenous Health Equity Summit meant that policy proposals needed to be developed by DoHA within a short period of time. In this situation, the importance of the evidence seems, at least in part, to have been associated with its accessibility (the literature review was drafted internally by the Department in the previous year). The accessibility of research has been identified as a facilitator to evidence-based policy making in other studies (55)(56)(57)(58).
Similarly, the Tackling Indigenous Smoking Measure closely reflected the recommendations in another literature review commissioned by DoHA and authored by the CEITC. Viewed through the prism of Walt and Gilson's model of health policy making, the value that was placed on the CEITC's research can be partly ascribed to the organization being regarded as a key "actor" in the policy community. If the policy community is conceptualized as a network of interconnected individuals and organizations, CEITC's influence derived from its centrality in that network (59). DoHA provided CEITC with a majority of its funding with a view to building the evidence base in relation to Indigenous tobacco control and, as demonstrated in the interview data, relied on the organization for policy advice. Furthermore, the Director of CEITC was a member of the Tobacco Working Group, which was also consulted in the development of the policy. From the perspective of Lin's model, "political rationality" overlapped with "technical rationality" in the sense that a research organization was invited into the decision-making process.
While it is clear that the tobacco control policies were developed based on evidence in the form of literature reviews, a critical analysis of the studies contained in those literature reviews revealed that many possessed methodological limitations. The evidence relied on was equivocal as to whether the recommendations would lead to positive tobacco control outcomes for Indigenous Australians. The case study therefore highlights the importance of conceptual clarity when using the term "evidence-based policy. " However, it does not follow that policies based on weak or equivocal evidence are "bad policies. " Far from being irrational or reckless, the interview data suggest that the policy makers' acceptance of the policy proposals was based on logical reasoning, notwithstanding the absence of a rigorous evidence base.
Respondents noted the importance of other influences on the policy process, including the need to ensure that the policies had the support of members of the Indigenous population. Historically, policy making for Indigenous Australians has often been a top-down exercise associated with marginalization and disempowerment of the population. For instance, an inquiry into a controversial 2007 Indigenous policy initiative (the Northern Territory National Emergency Response) found that the policy "diminished its own effectiveness through its failure to engage constructively with the Aboriginal people it was intended to help" (60). For some of the policy makers in the present case study, there was a desire to avoid repeating mistakes of the past by ensuring that the needs and preferences of Indigenous people were captured in the design of the Indigenous Tobacco Control Initiative and Tackling Indigenous Smoking Measure. Evidence cannot trump all other factors; rather it must coexist with other inputs in the policy process, including the "thoughtful identification and compassionate use of individual patients' [or populations'] predicaments, rights and preferences" (61). This can be illustrated by another example from the Australian Indigenous health context. While there is a strong evidence base to support the use of dialysis for patients with renal failure, historically, many Indigenous patients living in remote areas refused treatment on the basis that it would involve spending extended periods of time in urban and regional centers, away from their family, culture, and land (62). Put simply, "an efficacious treatment cannot produce public health benefits if it is not adopted widely by its intended recipients" (63).
Ideally, solutions that are "acceptable" to the target population will also be supported by evidence of their likely effectiveness.
However, the Indigenous tobacco control case study demonstrates that there are situations in which it may be justifiable to introduce policies in the absence of high-quality evidence.
It has been noted that barriers to research in Indigenous contexts include • the fact that the Indigenous Australian source population is small (670,000), meaning that sample populations in Indigenous-specific studies are also likely to be small, and only very large program impacts are likely to be statistically significant (64); • Indigenous distrust of research on the basis that it was historically associated with "the politics of colonial control" and, in the present day, can be perceived to be exploitative, stigmatizing, or unresponsive to Indigenous needs and concerns (65)(66)(67); and • the fact that, in order to be culturally sensitive, there is often a call for research in Indigenous communities to be participatory, which can result in a conflict between "the values of the academic setting and those of the community" (65,68).
The consequence is that "'best' evidence is often gathered on simple interventions and from groups that are easy to reach in a population, " and evidence from those that are harder to reach tends to be regarded as inferior (69).
This research bias is not unique to the Indigenous Australian context. It has been noted that low-income countries often have limited resources to invest in research, and cultural minorities within high-income countries are often excluded from research protocols because of the logistical and financial costs, such as those related to translation (70). With respect to tobacco control research, a systematic review of studies examining the effectiveness of NHS stop smoking services in the United Kingdom found there is currently a lack of high-quality research into the potential differential impact of interventions in other subpopulations, such as black and minority ethnic groups, despite qualitative and anecdotal evidence suggesting different beliefs and attitudes to smoking and smoking cessation (71).
Birch presents the potential resulting paradox of a "no action without supporting evidence" mentality thus: If there is no evidence of effectiveness among the most deprived group… [e]fficient use of scarce resources… would imply the intervention not be given to these individuals with the anticipated increase in social inequalities in health. The resources of a health care system would become increasingly concentrated on less deprived populations. This results not necessarily because the problems of the more deprived groups are insoluble, but because the mechanisms we choose to evaluate for dealing with the observed problems are particularly suited to less deprived groups, i.e. it is a feature of the 'service focus' (72).
Applied to the Indigenous tobacco control context, a rigid adherence to the belief that there should be "no policy without evidence" could feasibly have resulted in a widening of the health gap between Indigenous and non-Indigenous Australians simply because there was more research to demonstrate the effectiveness of interventions in the latter population.
When the potential corollaries of the "no policy without evidence" approach are laid bare, the policy makers' decision to adopt the Indigenous Tobacco Control Initiative and Tackling Indigenous Smoking Measure in the absence of strong evidence is justifiable. The policies were not made with reckless disregard for what would work; instead, evidence from other settings and expert opinions were used to generate conceptually plausible responses to the policy problem in a manner that accommodated the apparent preferences of members of the Indigenous community. Moreover, the policy proposals were accompanied by a commitment to evaluate their effectiveness. This approach is consistent with that advocated by Sanderson, who suggests that policies should be viewed as hypotheses to be tested through pilots or trials and/or rigorous monitoring and evaluation (73).
In May 2015, the Government committed a further AUD116.8 million to Indigenous tobacco control initiatives over 3 years (74). Future research needs to examine whether rigorous evaluations of the Indigenous Tobacco Control Initiative and Tackling Indigenous Smoking Measure were conducted as planned and, if so, the extent to which lessons learnt have informed more recent policy developments.
In terms of theories of the policy process, this case study has provided clear evidence of the role of "context/cultural rationality" and "process/political rationality" as having at least as much weight as "technical rationality" in understanding how this particular policy area evolved.
Furthermore, as this case study demonstrates, public health policy making does not always lend itself to the literal translation of rigorous research findings. More empirical case studies are needed to explore the role of other factors -such as the needs and preferences of the target population -in order to provide a clearer understanding of their relative importance in the policy process.
aUThOr cOnTriBUTiOns