Peer Reviewed Papers
Coming of Age and Taking Stock: The State of Academic Health Policy Research Centres in Canada
AbstractThis descriptive study takes stock of the nation's health services and health policy research capacity by profiling the organizational models, operational challenges and success strategies utilized by Canadian academic health policy research centres. While each such centre is unique, the results point to some common themes, including symbiotic relationships between centres and their ministries of health, pervasive infrastructure funding challenges and the importance of having a supportive academic home. [To view the French abstract, please scroll down.]
Certain developments attest to the coming of age of a discipline. Among these are the emergence of stable funding sources, the formation of professional organizations and the establishment of peer-reviewed journals. The field of Canadian health policy and health services research has recently undergone such rites of passage. In 1997, the Canadian Health Services Research Foundation was established to fund such research and was joined in 2000 by the Institute of Health Services and Policy Research at the Canadian Institutes of Health Research (Canadian Health Services Research Foundation 2004; Canadian Institutes of Health Research 2005). Professional organizations have begun to emerge as well, starting with the Canadian Health Economics Association's evolution into the broader Canadian Association of Health Services and Policy Research and the formation of the nascent Network of Applied Health Services Research Centre Directors. Finally, in 2004, the journal Healthcare Policy was launched. The inaugural issue of Healthcare Policy provides an appropriate venue for taking stock of the nation's academic health policy research centres.
The purpose of this study was to collect descriptive data on Canada's academic health policy research centres, from which to identify the challenges they face and the strategies they deploy in achieving success.
MethodsThis descriptive study was conducted using semi-structured telephone interviews with the directors of selected Canadian academic health policy research centres. The interview tool, which contained approximately 50 questions, covered five broad areas: (1) general information (i.e., history, target audiences, etc.); (2) staffing and collaboration; (3) structure; (4) funding; and (5) external resources and performance measures. These themes were chosen a priori by the investigators, based on personal experience in directing a centre (SS) and on a previous study of American health policy centres (MM) that confirmed the relevance of these domains to identifying organizational challenges and coping strategies.
Because detail and description were deemed critical to garnering a complete understanding of the structure and operations of participating centres, the study was designed to be primarily qualitative rather than quantitative (Creswell 1998). Moreover, as not all questions were applicable to every centre, the semi-structured nature of the interviews allowed the interviewer (MM) to tailor questions based on responses and elicit further information where warranted. Interviews were conducted between October and December 2004, and generally lasted one hour.
SampleFor inclusion, centres had to meet the following selection criteria: (1) having a primary focus on health services, health policy research, or both, and being formally established for and devoted to such research generally; (2) designation in name as a "centre," "unit," "institute" or an equivalent; (3) being located in Canada; and either (4a) having a university affiliation or (4b) being included in the Network of Applied Health Services Research Centre Directors. University departments or schools were deemed inappropriate to include, given that their funding and organizational structures differ significantly from centres and that they have a more prominent pedagogical orientation.
Analysis and Results
ParticipationThirteen entities were identified that met the inclusion criteria, and all participated (Table 1).
|Table 1. Participating Canadian health policy centres|
Centre Name: Centre for Health Economics and
Centre Name: Centre for Health and Policy
Centre Name: Centre for Health Services and
Centre Name: Centre for Health Services and
Centre Name: Centre for Rural and Northern
Centre Name: Groupe de recherche
interdisciplinaire en santé
Centre Name: Institute for Clinical Evaluative
Centre Name: Institute of Health
Centre Name: Institute of Population
Centre Name: Manitoba Centre for Health
Centre Name: Newfoundland and Labrador Centre
for Applied Health Research
Centre Name: Nursing Health Services Research
Centre Name: Population Health Research
*These centres have multiple sites and
university affiliations; only the site marked with an asterisk (*)
**The Institute for Clinical Evaluative Sciences has no formal university affiliation, but it is located on the Sunnybrook and Women's College campus and draws its affiliated investigators from university faculty.
Centre audiencesCentres reported focusing on themes of healthcare quality, efficiency, effectiveness, equity or access, and many emphasized the policy relevance and interdisciplinary nature of their research. Nearly all centres identified their primary audiences as including healthcare policy makers, especially provincial ministries of health, which were listed by 12 of the 13 centres as one of their target audiences. This focus on provincial healthcare policy makers is an example of the close ties between most centres and their respective ministry. In fact, centre-ministry linkage is a central theme arising from the study. This phenomenon is attributable to provincial ministries' role as core funders of the majority of centres, as well as provinces' primary responsibility for health services provision under the Canada Health Act. The next most commonly identified centre audience, mentioned by nine centres, was federal healthcare policy makers, such as Health Canada. Whether referring to federal or provincial policy makers, however, centres generally eschew legislative policy makers and target those in the executive branch instead; this choice appears to stem from centres' concern over tarnishing their reputation for objectivity and non-partisanship. Other frequently mentioned audiences were researchers and other research organizations; healthcare entities, including provider organizations and professional associations; clinicians; regional health authorities; and the public.
Communications strategiesWhile the needs of these varied audiences differ, centres cited relationship-based activities involving face-to-face interaction as a universally effective outreach strategy. Examples of these activities include regular meetings with key funders, such as ministries of health; collaborative research projects that engage the target audience from design through dissemination; informal, individual centre investigator-audience member linkages; and audience member appointments to centre work groups and advisory panels. A few centres designate an audience liaison charged with conducting and coordinating such relationship-building efforts.
Other reportedly effective communication tools employed by centres include educational events, ranging from large annual symposia to tailored workshops geared towards the interests of a particular audience; publications, especially one- or two-page project briefs summarizing key findings; and electronic media, such as websites and newsletters distributed via email. Centres typically utilize multiple communication vehicles, and those centres most attuned to audience outreach, relationship building and knowledge transfer stressed the need for centre leadership to formulate a communications strategy and designate an individual to oversee its day-to-day implementation. In addition, a pithy observation was made: no matter how good a centre's research may be, if the topic is not on policy makers' radar at the time, the results will garner little interest or uptake. Yet, centres acknowledged their ability to temper this phenomenon by jointly selecting research projects with policy maker partners and involving these partners in all project phases. This approach, however, comes with a caveat for maintaining centre autonomy: academic-affiliated centres must balance this approach with supporting purely investigator-driven research and declining policy maker-requested projects with little academic relevance or unrealistic timeframes.
Tracking contactRegardless of the mechanisms used, tracking these centre-audience communications is increasingly important because, with growing frequency, funders are utilizing such interactions as a proxy measure for centre effectiveness. Thus, while centres may dispute the accuracy of this proxy, the vast majority monitor their interactions either through informal or formal means, or both. Among the informal mechanisms in use are direct contacts, inquiries and unsolicited feedback from audience members, as well as invitations to provide presentations and consultations. Formal tracking mechanisms include website "hits," peer-reviewed article placements, centre-related media contacts and coverage, and project evaluations focused on uptake. Nevertheless, tracking centre-audience interactions is fraught with difficulty for some centres because they either lack the resources to institute or adequately maintain such efforts, or the number and geographical distribution of their affiliated investigators make monitoring virtually impossible.
University affiliationThe overwhelming majority of participating centres is affiliated with a university, either as a stand-alone entity within the institutional rubric or as a faculty-based unit, typically within medicine, health sciences or nursing. A small minority of centres, however, has looser institutional ties - whether through renewable membership agreements with affiliated universities or through location on university property and affiliations with university-based investigators. Regardless of the nature of the affiliation, however, centres universally prize this institutional association because of the heightened perception of integrity and objectivity that accrues to the centre and its products as a result.
In addition to claiming an academic "home," centres engage in varied efforts to maintain the external perception that they and their research are objective. Some employ legal mechanisms, such as contract language addressing academic freedom, publication rights and conflicts of interest. Centres also often decline industry support, instead seeking funding through grant competitions. In terms of approaches to research methods, centres avoid proprietary projects and projects with little relevance, apply academic protocols in their traditional and applied research, utilize external project reviewers prior to dissemination, publish results in peer-reviewed journals and adopt only evidence-based positions.
Another university-oriented key to health policy centre success, given the complexities of the area of study, is developing and maintaining a multidisciplinary team of core and affiliated investigators. University linkage is a boon in this endeavour, as well, as access is provided to researchers in the full range of disciplines. To capitalize, centres offer incentives such as support services, funding opportunities, collaborative projects and co-location to entice faculty to become affiliated investigators. Four participating centres offer two additional, unique and highly prized incentives owing to their role as delegated repositories for and custodians of provincial health data (data centres): access to health data and data analysis services. The benefit of serving as a data centre, as related to attracting investigators, is constrained, however, by privacy and confidentiality concerns, which generally mandate more formal affiliation agreements and geographical proximity between the centre and the investigator.
Adopting a multidisciplinary approach means that centres must work across numerous university faculties, creating a matrix structure. While this structure enables the necessary affiliations, it also creates unique challenges for centre management. Because investigators report to their respective departments, rather than to the centre, centres tend to have little formal control over affiliated investigators and may find themselves in competition with home departments for researcher-generated overheads. Additionally, centres hold little sway over departmental reviews of investigators. This situation was of particular concern to a number of participating centres because their associated universities fail to reward applied research and knowledge transfer activities on par with traditional research, peer-reviewed publication and teaching.
Centre fundingFunding is, by and large, the predominant challenge that centres face. Of particular concern is stagnant and, often, shrinking infrastructure funding. This trend can constrain centres because they tend to rely on single sources for the majority of their infrastructure support - typically ministries of health, which provide core funding to nearly all participating centres and serve as the primary funding source for just under half of the participating centres. As a result, centres are forced to do more with less - constricting growth and curtailing new and existing services. Data centres, in particular, are especially vulnerable because they tend to be almost exclusively dependent on ministry funding for infrastructure support, and their core operations require that they maintain a cadre of highly skilled technical staff. The other logical providers of infrastructure support - affiliated universities - are increasingly short of resources, given the state of higher education funding in Canada (Rae 2005). Nevertheless, universities provide limited infrastructure support to the majority of centres. Due to the continuing financial stress they are under, however, the sustainability of these contributions is questionable. Moreover, universities' reticence to fill open tenure-track positions and to create new ones, under the present financial picture, diminishes centres' ability to assemble and maintain a core of multidisciplinary investigators, especially as most grants make no provision for faculty salary support.
Another major anxiety surrounds stability of funding. This issue stems from the cyclical nature of grants and contracts, which are key funding sources for the vast majority of centres. Such term-limited funding requires that intensive effort be focused on applying for grants and on ensuring contract renewals - ultimately reducing the resources available to centres' core research and knowledge transfer functions. In addition, grant and contract funding generally comes with restrictions on how the funds can be spent. Thus, unfunded activities - often knowledge transfer and performance measurement - may fall by the wayside.
A few centres have sought out non-traditional funding sources, such as industry - the pharmaceutical industry, in particular. In these instances, however, private-sector support has not constituted the primary source of centre revenues. Nevertheless, other centres flatly refuse such industry support because of objectivity-related concerns.
To cope with funding woes, centres typically tend to engage in one of two strategies: (1) being guided by centre-defined research themes and areas of expertise in the pursuit of funding or (2) being opportunistic. Yet, regardless of the strategy undertaken, centre success in the funding arena seemingly comes down to a handful of fundamental factors. The most commonly cited of these is developing and maintaining a critical mass of well-respected, high-calibre, committed investigators. The next is the exogenous factor of working in a booming research domain, where project funding is increasingly available. The remaining three factors are building relationships with and getting buy-in from key funders; producing quality, relevant work; and retaining a well-respected, connected director. Data centres noted an additional factor - their role as data custodians.
Performance measurementTied directly to funding issues is the need for and utilization of performance measures. Funders, especially ministries of health, increasingly emphasize accountability. As a result, both funders and centres alike have begun to look to metrics and benchmarks as a means of quantifying performance.
In the absence of any consensus in practice or in the literature on appropriate metrics by which to gauge research centre performance, most centres select their own indicators and compare their performance internally over time. Typically, indicators include such standard academic metrics as the number of peer-reviewed publications, the ratio of core funding to other research dollars generated, research dollars per researcher, overall annual funding and the number of graduate students supervised.
A second, though largely informal, method of performance measurement engaged in is "best-in-class" benchmarking, whereby centres compare themselves to others that they view as leaders in the field. The centres most commonly perceived in this fashion are the Centre for Health Services and Policy Research at the University of British Columbia, the Institute for Clinical Evaluative Sciences, Groupe de recherche interdisciplinaire en santé at the University of Montreal and the Manitoba Centre for Health Policy at the University of Manitoba. Interestingly, two of these three entities, are longstanding data centres. While such comparisons may be relevant for other data centres, it is not clear that such centres are appropriate benchmarks for centres that do not play a data-repository role.
A third mechanism employed by centres to monitor their performance is external reviews. These audits, conducted every few years, assess all areas of centre performance and operations. They are generally an internal requirement of the home university, but in some cases are mandated by core funding contracts with ministries of health.
Discussion and ConclusionsThis descriptive study has found that Canadian university-based health policy research centres are notable for their diversity of size, funding and areas of research strength. Despite such heterogeneity, however, they are strikingly similar in the challenges they identify and the coping strategies they devise. Key challenges identified in our study include communicating effectively with target audiences, developing strong university support, ensuring stable funding and demonstrating appropriate performance by objective criteria. Among the strategies for success reported by respondents were nurturing ongoing relationships with decision-makers; recruiting affiliated faculty from across disciplines and lobbying university officials for a better understanding of applied health research; actively seeking stable funding from both government and universities, as well as private endowments; and finally, developing the capacity to demonstrate high-calibre academic research of relevance to policy makers.
Among the various challenges faced by centres, one stands out as dominant for most: the struggle to maintain operational continuity in the face of absent or scant infrastructural funding. There are clearly several factors contributing to this instability. Most centres depend heavily on core funding from ministries of health, a contribution that must be periodically renegotiated and is unpredictable in size or longevity. While additional support may be received from home institutions, for more than a decade universities across Canada have operated under severe resource constraints. Moreover, as extra-departmental structures, centres tend to be excluded from the normal departmentally based flow of internal university funds. Indeed, organizational change within institutions does not appear to have kept pace with enthusiasm for spawning interdisciplinary research groups. Finally, the availability of funding-agency program or team grants may supplement, but is not an adequate substitute for, stable infrastructural funding. Such grants are term-limited and restricted in how they may be spent, and focus on a designated series of research projects.
Health policy research centres in Canada face another, more conceptual issue: the challenge of serving two quite different masters. Provincial ministries provide support for centres and, in return, generally have some claim on research time. Questions of interest to decision-makers may have little academic interest. The rapidity of response time is often at odds with both a researcher's view of academic thoroughness and prior commitments. Frequently, decision-makers attempt to apply conditions of confidentiality to projects that compromise university views of intellectual property rights. For their part, universities tend to discount the worth of providing advice to government or doing applied research that does not translate into academic output. Indeed, merit in the university is generally gauged by receipt of peer-reviewed funding and peer-reviewed publication, neither activity being of primary interest to the other master. Striking a balance in allocating time and resources to serve the divergent interests of these two masters, and educating each to respect the perspectives of the other, represents a defining task for the health policy centres.
Despite such divergent world views, however, there may be a critical area of accord between ministries of health and academic institutions upon which to build a broader understanding. There is a shared recognition in government and academe of the important capacity-building role that health policy centres can play. For example, recent reviews of the Canadian health system directed by both Roy Romanow and Senator Michael Kirby (Romanow 2002; Kirby 2002) drew heavily on commissioned academic research, the existing peer-reviewed literature and expert testimony in formulating their conclusions. This academic resource, in contrast to the advocacy role associated with many "think tanks," provided what is generally seen as objective opinion on key issues. Ensuring a capacity for academically informed decision-making across all levels of the healthcare system will demand the ongoing production of post-graduate trainees in health policy and health services research. This need is no different from other human resources requirements in the health system and, arguably, would justify ministries' creating permanent funding solutions. Universities, for their part, would need to respond by recognizing the academic role of applied health systems research.
Moving forward with conjoint support from government and universities, however, would confer a critical responsibility upon health policy research centres: they must be able to demonstrate their applied and academic value. This will require the development of performance indicators that are as compelling to a provincial auditor as they are to a faculty promotion committee. Achieving absolute consensus on measures from 13 centres, funded by various provincial ministries of health and located within different universities, is unlikely; however, the development of a generic template readily modifiable to suit local circumstances is an achievable goal.
It seems reasonable to conclude that Canadian academic health policy research centres are, paradoxically, both thriving and yet precarious. The field of health services and policy research shows signs of vibrant maturation, a process to which the research output of the centres has significantly contributed. At the same time, however, centres lack the stability of funding and academic recognition that will ensure future research productivity and capacity development. Whatever other interventions may be suggested by this study, it is clear in aggregate that the centres deserve periodic scrutiny of their challenges and successes.
Prise de conscience et bilan: État des centres de recherche universitaires sur les politiques de santé au Canada
RésuméCette étude descriptive fait le point sur les capacités de recherche en matière de services et de politiques de santé du pays en décrivant les modèles organisationnels, les problèmes opérationnels, ainsi que les stratégies de réussite qu'utilisent certains centres de recherche universitaires sur les politiques de santé au Canada. Bien que chaque centre soit unique, les résultats semblent indiquer quelques thèmes communs, notamment les relations symbiotiques entre les centres et leur ministère de la Santé respectif, les problèmes répandus de financement des infrastructures et l'importance d'être rattaché à un établissement d'enseignement propice à l'épanouissement.
About the Author
Michele Mekel, J.D., MHA, MBA
Executive Director/Legal Fellow
Institute on Biotechnology and the Human Future and
Center on Nanotechnology and Society
Illinois Institute of Technology/Chicago-Kent College of Law
Institute on Biotechnology and the Human Future and
Center on Nanotechnology and Society
Illinois Institute of Technology/Chicago-Kent College of Law
Samuel E.D. Shortt, MPA, MD, PHD
Director, Centre for Health Services and Policy Research
Directeur, Centre de recherche sur les services et les politiques de santé
Contact Information: Michele Mekel, JD, MHA, MBA, Executive Director/Legal Fellow, Institute on Biotechnology and the Human Future Center on Nanotechnology/Chicago-Kent College of Law, 565 W. Adam St., Chicago, IL 60661, USA, firstname.lastname@example.org.
Canadian Health Services Research Foundation. 2005. History. Retrieved July 31, 2005. < http://www.chsrf.ca/about/history_e.php >.
Canadian Institutes of Health Research. 2004. Who We Are. Retrieved July 31, 2005. < http://www.cihr-irsc.gc.ca/e/7263.html >.
Creswell, J.W. 1998. Qualitative Inquiry and Research Design. Thousand Oaks, CA: Sage Publications.
Kirby, M. 2002 (October). The Health of Canadians - The Federal Role. The Standing Senate Committee on Social Affairs, Science and Technology (Chair: The Honourable Michael J.L. Kirby, Deputy Chair: The Honourable Marjory LeBreton). Retrieved July 31, 2005. < http://www.parl.gc.ca/37/2/parlbus/commbus/
Rae, B. 2005. Ontario: A Leader in Learning: Report and Recommendations. Retrieved July 31, 2005. < http://www.raereview.on.ca/en/pdf/finalReport.pdf >.
Romanow, R. 2002 (November). Commission on the Future of Healthcare in Canada: Building on Values - The Future of Healthcare in Canada. Final Report. Retrieved July 31, 2005. < http://www.hc-sc.gc.ca/english/care/
Be the first to comment on this!You must sign in to comment. Sign In / Create an Account