Abstract

This paper reports on a research collective in primary healthcare (PHC) conducted in Quebec in 2004. A lead team of investigators synthesized 30 ongoing or recently completed studies from project description forms filled out by the participating researchers.

The process of the collective is examined by addressing the three main challenges met in the course of its completion, namely, (a) the need to derive an analytical framework to regroup variables in a meaningful way, (b) the assessment of strength of evidence and (c) coping with a mix of quantitative and qualitative studies. Advantages of the collective over other forms of research synthesis   include timeliness, low cost relative to the total cost of the studies it comprises and the information it generates and, finally, context linkage, which enhances relevance but which could limit transferability of the findings. Overall, the research collective appears to be a promising tool for research synthesis.

[To view the French abstract, please scroll down.]

Financial resources devoted to research, and more particularly to health services and policy research, have increased considerably in recent years (CIHR 2004a). Funding agencies expect that such research will help policy makers and managers solve the numerous problems faced by our healthcare system (CIHR 2004b; Lomas 2000). Yet, most of the time, research findings are produced and disseminated one project at a time, and links with other ongoing projects dealing with similar subjects are not generally made. Since research findings take many years to appear in the published literature, decision-makers often disregard or distrust them because they are seen as outdated (Innvaer et al. 2002; Lavis et al. 2005). To be useful to decision-making, research findings need to be produced, synthesized and disseminated more rapidly (Sheldon 2005; Lavis et al. 2005).

The need for timely synthesis of research findings seems difficult to satisfy using traditional methods (Innvaer et al. 2002; Lavis et al. 2005). The research collective offers an alternative approach that is designed to meet this need (Pineault and Tousignant 2001).  

Although both a research collective and a systematic review of the literature can produce a synthesis, a research collective differs from a systematic review of the literature in several respects:

  • it involves active participation of the researchers whose studies are selected;
  • it also involves the participation of decision-makers, as their viewpoints are considered in the analysis;
  • its findings are timely, based on ongoing or recently completed research, and are rapidly released;
  • it is context linked, that is, the studies upon which it rests come from well-identified contexts that increase the collective's usefulness and pertinence for decision-makers operating under comparable conditions.

The research collective can thus be defined as a process of dynamic exchange between a group of researchers and a lead team of investigators, resulting in the research synthesis of a limited number of projects on a given subject. In a general sense, the term research synthesis refers to the overall process of gathering and synthesizing data from different sources; in this sense, a research collective, like a systematic review of the literature, can be viewed as a particular form of research synthesis. Alternatively, in a stricter sense, research synthesis represents one stage in the overall process of a systematic review or of a research collective (Mays et al. 2005).

The objective of this paper is to present the approach, process and methods followed in a research collective on primary healthcare (PHC) conducted in Quebec in 2004 and to discuss the main advantages and challenges arising from this experience (Pineault et al. 2005).

The choice of PHC as a focus for the collective was based on several considerations. First, major problems in healthcare systems have been identified in relation to continuity, accessibility and comprehensiveness, and these phenomena are closely related to PHC (Romanow 2002; Wilson et al. 2004). Second, the central role played by PHC in healthcare systems has been underscored by several authors, based on empirical evidence (Starfield 1998; Macinko et al. 2003). Finally, reforms have been initiated to address these issues (Showstack et al. 2004). It is thus important to inform decision-makers about the elements of PHC that seem to be most promising for achieving the expected results.

Overview of Methods

The research collective was led by a group of investigators who worked on the synthesis and prepared the report. Initially, a list of Quebec researchers whose work focused on the organization of healthcare services was drawn from different sources (research agencies, universities, research centres and groups). These researchers were asked to submit research projects that would meet the terms of reference of a research collective on the organization of PHC. More than 90 submissions were received.

To be selected, a research project had to:

  1. focus on PHC services in the broadest sense (projects dealing only with hospitals or long-term care facilities were excluded);
  2. examine healthcare services (projects restricted to social or community services were excluded) and, more specifically, the organization of PHC services;
  3. be recent, that is, still underway or completed within the last three years; and
  4. have results to present.

Thirty projects, listed in Appendix 1, satisfied these criteria. Researchers from the 30 projects participated in different steps of the research collective. The high participation rate throughout the collective can presumably be explained by two factors: (a) among the investigators were respected senior researchers and academics who knew and shared the values inherent to the research culture and (b) the researchers were informed from the start that their papers would be published in the collective as "chapters in a book," and that their contributions would be recognized and valued as concrete accomplishments in terms of knowledge exchange activities.

For each project, the researchers were required to fill out a project description form that investigators and researchers agreed upon at a first meeting held in April 2004. The form specified the information to transmit regarding research questions, study objectives, methodology and findings (Pineault et al. 2005).

The researchers briefly presented their projects during a seminar held in June 2004. A group of decision-makers representing different constituencies (ministry of health, regional agencies, public health, PHC organizations) also attended this meeting and were asked to react to and comment on the projects. Visual aids used by researchers in their presentations, as well as notes taken during discussions with decision-makers, were gathered and retained as material to be analyzed and incorporated into the final report. After the meeting, researchers were asked to revise the project description forms they initially completed in the light of other researchers' presentations and the discussions. They were asked to pay particular attention to links among the projects.

Investigators' initial examination of the 30 project description forms identified several recurring themes dealing with concepts and phenomena that are especially relevant to ongoing reflections on PHC services, specifically: integration of services; interprofessional collaboration; new information and communication technologies; and dimensions of service utilization (continuity, accessibility and comprehensiveness) and their effects on recourse to emergency room (ER) services, responsiveness, health and costs. Investigators cross-referenced the projects to produce a first draft of the synthesis, which was sent to researchers for validation.

Researchers' suggestions and corrections were integrated into the final version of the report, which was released at a knowledge exchange meeting on February 3, 2005. Half the 100 participants at the meeting were decision-makers.

Figure 1. Analytical framework: contextual blocks

Challenges and Issues

The synthesis findings are available elsewhere (Pineault et al. 2005). This paper focuses on the main challenges faced and on the issues raised in the process of conducting the collective.

Deriving an analytical framework

The first challenge was the necessity to derive an analytical framework in order to regroup the variables in a meaningful way. This analytical framework emerged from the investigators' thematic reading of the project description forms (Figure 1).

Following a configurational approach, which sees organizations in terms of constellations of attributes rather than individual characteristics, the investigators examined the variables, by block rather than individually, because variables were found to be closely interrelated within each block (Meyer et al. 1993). For instance, integration of services and interprofessional collaboration were found to be core elements of integrated healthcare networks, and new communication and information technologies, when implemented, supported integration. Similarly, continuity, accessibility and comprehensiveness were three interdependent dimensions of service provision/utilization that very often overlapped in the studies. Consequently, they were deemed more meaningful when examined together. Finally, the block of consequences, such as responsiveness and use of ER, clearly constituted a conceptually distinct category. These three blocks of variables are shaped and influenced by such contextual elements as geographic characteristics (rural/urban), level and type of resources, availability of technology and so on. Theories and concepts referred to in individual studies were implicitly put together to elaborate a sort of meta-theory underlying the analytical framework (Pawson et al. 2005; Dixon-Woods et al. 2005). Examining the 30 studies together within this analytical framework made it possible to clarify the links among integrated services, interprofessional collaboration, new communication and information technologies and indicators of utilization and outcome.

Assessing strength of evidence

The second challenge was assessing the strength of evidence of the findings. The investigators did not formally assess the scientific quality of the studies for two reasons. First, the projects had been funded by research granting agencies and, hence, had successfully passed the test of scientific merit. Second, the researchers might have been reluctant to participate had they known that investigators would evaluate the strength of their study methodologies.

In the synthesis, however, the investigators considered certain factors in weighing their conclusions, namely, the number of studies associated with specific results and their convergence or divergence. Furthermore, the investigators' judgments and conclusions always referred to the supporting studies, and thus validity could be checked in all cases. It must be added that in filling out project description forms, the researchers were asked to assess the validity of their own research. Internal validity/credibility and external validity/transferability were assessed by filling out the grids shown in Appendix 2. Note that the term internal validity (used in quantitative studies) is coupled with the term credibility (used in qualitative studies), while the term external validity (used in quantitative studies) is paired with transferability (its counterpart in qualitative studies). Both quantitative and qualitative criteria were considered. For example, research design, sample size, size of reference population and statistical inference all refer to quantitative criteria, whereas intervention and implementation analyses refer more specifically to qualitative criteria (Devers 1999). It is interesting to note that the researchers' assessment of their own studies showed high variability and revealed an acute sense of critical judgment. For example, concerning internal validity/credibility, 20% of the researchers rated their research "low" on a three-point ordinal scale, whereas 37% rated it "high." Likewise, 20% of the researchers rated their research "low" on the scale of external validity/transferability and 30% rated it high.

The investigators did not take researchers' self-assessment of the methodological value of their research into formal account. They applied implicit rather than explicit criteria, in the same manner as expert judgment on quality of care most often rests on implicit criteria of what constitutes good medical care (Donabedian 1980). The investigators went through implicit yet reproducible steps to produce the research synthesis. This approach is more in line with Patton's (1999) view that the confidence placed by decision-makers on research findings is mainly based on the credibility of researchers/investigators, which, in turn, depends on their perceived competence, integrity and judgment.

Figure 2. Complementary use of integrative and interpretive approaches in the synthesis

Mixing qualitative and quantitative studies

As is the case for most syntheses in healthcare, a third challenge was mixing qualitative and quantitative studies in the same synthesis (Dixon-Woods et al. 2005; Mays et al. 2005). Of the 30 studies, 18 were qualitative and 12 quantitative. The quantitative studies focused more on measures of utilization and outcomes, whereas the qualitative ones focused more on organizational modes (e.g., integration of health services and interprofessional collaboration). The quantitative studies addressed the question of what works, whereas the qualitative ones addressed, to a greater extent, the questions of how and why it works (Dixon-Woods et al. 2005; Mays et al. 2005).

Both types of studies refer to different theoretical and methodological perspectives: quantitative studies rest upon indicator and statistical analyses, whereas qualitative studies rest upon intervention and implementation analyses (Figure 2). The quantitative approach addresses the "what" question and is more amenable to integrative synthesis that combines, aggregates and summarizes data. The qualitative approach addresses the "how" and "why" questions and is better suited for interpretive synthesis in which interpretation of data and inductive judgments are formulated on the basis of concepts and theory emerging in the course of the synthetic analysis (Dixon-Woods et al. 2005).

Mixing integrative and interpretive approaches in the synthesis clearly yielded added value, as illustrated in Figure 3, which shows four hypothetical studies, two quantitative (A   and C) and two qualitative (B and D). Through a re-analysis of the indicator studies, it was possible to seek qualitative elements related to intervention or implementation analysis but not initially identified by the researchers, and thus to extend them into the domain of qualitative studies. Similarly, it was possible to move qualitative implementation/intervention studies into the domain of indicator studies by paying more attention to the analysis of indicators, which was not the focus of the original studies. The net gain, as shown in Figure 3, is represented by the shaded portions of the bars. In this sense, our approach draws upon both thematic analysis and narrative synthesis (Mays et al. 2005).

Figure 3. Value added in synthesizing integrative (quantitative) and interpretative (qualitative) approaches

Discussion

Compared to more traditional syntheses, the collective appears to yield significant benefits. At the same time, it raises some problems that need to be addressed.

Benefits

The first advantage of the collective is its timeliness, since it reports on studies that are either underway or were recently concluded - a clear advantage over systematic reviews based on published material. Adding to the timely character of the collective is its rapid completion. Our synthesis took eight months to complete.

The collective thus appears to be an efficient way of synthesizing research findings. Not counting researchers' and investigators' time, the cost of this collective was about $125,000. Of this amount, $44,000 came from cash contributions from research funding agencies and $61,000 represented contributions in kind from the host institutions. We estimated $150,000 to be the average cost of each of the 30 research projects, for a total of $4.5 million.Thus, the cost of the collective represents a small proportion of the total cost of the projects. This small incremental cost should encourage research funding agencies to support such syntheses on a larger scale.  

An assessment of the return on investment from research collectives could be refined by coupling cost figures to utilization of the information generated by the synthesis. For example, a survey done among users of this information would enable us to determine the type of use decision-makers have made of the synthesized findings: symbolic, conceptual or instrumental (Denis et al. 2004; Huberman 1994). Such a survey has already been commissioned by the Canadian Health Services Research Foundation (CHSRF) in the case of one policy synthesis (Lamarche et al. 2003). We intend to follow the same approach and evaluate the extent and type of use by decision-makers of the information generated by the collective one year after its release.

Among the main benefits of the collective are the opportunities created for exchanges among researchers, investigators and decision-makers. Direct participation of decision-makers in the early stages of the synthesis was an effective way to formalize personal contact between researchers and decision-makers and ensure that the latter applied the results (Innvaer et al. 2002).

Direct involvement of researchers also distinguishes the collective from traditional systematic reviews. Through their active involvement, researchers were able to provide specific data well adapted to the terms of reference specified by investigators, and to tailor these to the project descriptions. Flexibility and adaptation, as experienced in the collective, are not generally possible in a systematic review.  

Exchange among researchers working within a shared domain is worthwhile in itself. It enables researchers to broaden their perspective through becoming informed about their colleagues' projects and exchanging theoretical, methodological and applied aspects of their work. Such access is rare: owing to the mechanisms of project funding in research, researchers within a same group or centre are often unaware of their colleagues' research, even on subjects of common interest.

Although participating in the collective demanded a considerable investment of time from researchers (who were required to attend three meetings, complete a first and then a revised draft of the project description form and then review the final draft of the synthesis), nevertheless, their evaluation feedback indicates that they appreciated the exercise, particularly their discussions with researcher colleagues and their interactions with decision-makers. These researchers considered the publication of their work in the detailed report of the collective and their inclusion in a knowledge exchange activity with decision-makers to be valuable steps in advancing their academic and research careers.

Finally, the fact that the collective is linked and limited to a given context yields greater in-depth analysis of the context and its interacting components, consequently leading to a better understanding of the contextual environment. In that sense, the collective produces context-sensitive evidence (Lomas et al. 2005). From the standpoint of strict statistical inference, limitation to a single context could reduce external validity/transferability of the results. However, through implementation analysis, transferability of the findings can be ensured by identifying the conditions under which these results are produced. Organizational studies have shown that case studies can be generalized well beyond the case being studied (Yin 1999).  

Limitations

The process of the collective had limitations that need to be considered. In the context of evidence-based decision-making, the greatest and most obvious problem with a collective is assessment of the strength of evidence of the findings. As mentioned earlier, the researchers rated their own projects for both internal validity/credibility and external validity/transferability using both quantitative and qualitative criteria. Although investigators did not use this assessment in their synthesis, it seems an interesting avenue to pursue. Indeed, the researchers were critical of their research, as reflected in their self-assessment scores.  

Another potential problem is the proximity of researchers and investigators, since all come from a small community. Moreover, the investigators participated as researchers in 11 of the 30 studies. In synthesizing, there is always the risk of investigator bias and lack of objectivity regarding participants' own projects. Measures were taken to prevent this, namely, by having investigators check one another's projects and critically revise successive versions of the synthesis. On the other hand, the proximity of researchers and investigators could also be seen as an advantage, since investigators have a better knowledge of the literature dealing with the study topics and of the context in which studies take place. This better understanding of the healthcare system facilitates in-depth and critical analysis of the studies under review.  

By definition, the collective was limited to existing, ongoing or recently completed projects. While this limitation ensures the collective's timeliness, it can also be viewed as a drawback. The synthesis report does not refer to the literature dealing with the organization of PHC services. Ideally, it would be advantageous to interpret and discuss the findings of collectives in light of systematic literature reviews of similar topics. However, adding a systematic review to the process would lengthen the duration of the collective's work and the project's cost. Alternatively, the role played by the content experts among the investigators could obviate the need for complementary systematic reviews while preserving timeliness.

Because of its timely character, the collective runs the risk of disseminating preliminary data that further analysis will reveal to be invalid or, at least, slightly different. The synthesis report included cautionary remarks addressing this point. This risk has to be weighed against the increased degree of certainty that the findings, though preliminary, can bring to decision- and policy making. Ultimately, decision-makers themselves should determine whether the usefulness of the findings outweighs the risk inherent in preliminary data. In the case of this collective, decision-makers participating in the meetings clearly indicated their acceptance of the risk and their high regard for the collective's usefulness.

Conclusion

The collective appears to be a promising tool for research synthesis. The main advantages are its timeliness, its context linkage and the participatory exchange it generates among researchers, investigators and decision-makers. It thus meets many conditions that have been associated with the application of research findings by decision-makers (Innvaer et al. 2002; Lavis et al. 2005). Limitations can also be identified from this experience and must be addressed to improve the tool. Among them is the need to better assess strength of evidence, especially when based on self-assessment by researchers, in order to produce evidence synthesis rather than research synthesis only. Likewise, to strengthen evidence, the collective's findings could be triangulated with those stemming from systematic reviews of the literature. Finally, special attention should be devoted to enhancing participation of both researchers and decision-makers in the process. Overall, researchers' experience of participation was positive but could still be improved by establishing a more open and continuous forum of exchange with investigators. As for decision-makers, their participation is also important, and methods must be developed to consider, analyze and integrate their viewpoints into the broader context of a deliberative process leading to more enlightened decision- and policy making.


Le collectif de recherche : un outil pour produire des synthèses de recherche en temps opportun et liées au contexte

Résumé  

Cet article rapporte l'expérience d'un collectif de recherche sur les services de santé de première ligne menée au Québec en 2004. Trente études en cours, ou récemment terminées, ont fait l'objet d'une synthèse conduite par une équipe d'investigateurs à partir d'une fiche de description des projets complétée par les chercheurs participant au collectif.

Le processus du collectif est analysé en examinant les trois principaux défis rencontrés dans sa réalisation, soit (a) la nécessité d'élaborer un cadre d'analyses, afin de regrouper les variables d'une manière logique, (b) l'appréciation de la force de la preuve scientifique et (c) la façon de procéder lorsqu'on a des données quantitatives et qualitatives. Les avantages découlant du collectif, comparativement à d'autres formes de synthèses de recherche sont discutés, notamment son caractère contemporain, son coût marginal au regard du coût total des études retenues et de l'information produite et, enfin, le contexte, qui rehausse la pertinence mais pourrait limiter la transférabilité des résultats. Dans l'ensemble, le collectif de recherche semble être un outil prometteur pour produire des synthèses de recherche.


Appendix 1: List of the 30 Projects

1. J. Afilalo, A. Marinovich, M. Afilalo, A. Colacone, R. Léger, B. Unger. Characteristics of non-urgent patients who go to emergency services and obstacles to primary care services

2. A. Battaglini, P. Tousignant, L.-R. Poirier, M. Désy, H. Camirand. Matching primary care social and health services to the needs of immigrant populations: Impact of multiethnicity on the organization and delivery of services

3. M. Demers. Primary care services in Québec in 2000: Organization, utilization and changes in general practice

4. N. Leduc, J. Ricard, L. Farand, D. Roberge, A.A. Gbaya. The outpatient services centre as an alternative to hospital emergency services

5. N. Leduc, F. Champagne, S. Bergeron, M. Lafrance, G. Ste-Marie. Study of recourse to emergency paediatric services as a substitute for primary care services

6. L.-R. Poirier, R. Pineault, D. Ouellet, J. Gratton. The reorganization of Montreal's health services network: Long-term impact

7. C. Wolfson, L. Lévesque, H. Bergman, F. Béland, L. Trahan, A. Perrault. Longitudinal study of the unfulfilled needs for assistance and community services among individuals 75 years of age or over

8. M. Clément, D. Aubé, C. Beaucage, M. Tremblay. The continuity of care among individuals affected by the dual disorders of mental illness and drug addiction: Users' responsibility and organizational perspective

9. R. Geneau, R. Pineault, P. Lamarche, P. Lehoux. The structuration process of general practitioners' primary care practice: A qualitative study on the constraining and enabling properties of organizational modes

10. J. Haggerty, R. Pineault, M.-D. Beaulieu, Y. Brunelle, F. Goulet, J. Rodrigue, J. Gauthier (main project); J. Haggerty, D. Roberge, R. Pineault, D. Larouche, N. Touati (hospital emergency services project). The impact of the organization of primary care medical services on accessibility and continuity: Factors associated with differences in the use of hospital emergency services in rural and urban areas

11. R. Ionescu-Ittu, J. McCusker, N. Dendukuri. Continuity of primary care and return visits to emergency departments (ED) based on administrative data

12. J. McCusker, J. Verdon, P. Tousignant, L. Poulin de Courval, N. Dendukuri, P. Jacobs, E. Latimer. Rapid two-stage emergency department (ED) intervention for seniors: Impact on continuity of care

13. L.R. Poirier, M. Caulet, L. Fournier, C. Mercier, A. Lesage, D. White. The impact of service integration on maintenance in the community of individuals affected by major psychiatric disorders who have just experienced a crisis situation  

14. A.-P. Contandriopoulos, M.-A. Fournier, C. Dassa, R. Latour, F. Champagne, H. Bilodeau, N. Leduc. Practice profiles of Québec general practitioners  

15. M.-D. Beaulieu, J.-L. Denis, D. D'Amour, J. Goudreault, L. Lamothe, G. Jobin, J. Haggerty, É. Hudon, R. Geneau, R. Pineault, R. Lebeau. The implementation of family medicine groups: The challenge posed by the reorganization of practice and interprofessional collaboration

16. D. D'Amour, L. Goulet, R. Pineault, J.-F. Labadie. The effect of interorganizational collaboration on perinatal services, health and responsiveness

17. J.-P. Fortin, L. Lamothe. Quebec networked computerized file in oncology (DRIOQ)

18. A. Guttman, M. Afilalo, R. Guttman, A. Colacone, C. Robitaille, E. Lang, S. Rosenthal. An emergency department-based nurse discharge coordinator for elderly patients: Does it make a difference?

19. E. Lang, M. Afilalo, J.F. Boivin, R. Léger, A. Colacone, X. Xue, A. Vandal, S. Rosenthal, B. Unger. An Internet-based standardized communication system (SCS) linking the emergency department with primary care physicians (PCPs): A randomized clinical trial measuring continuity of care

20. A. Tourigny, L. Bonin, D. Morin, M. Buteau, L. Mathieu, L. Robichaud, A. Vézina, P.-J. Durand, R. Hébert. Interdisciplinary, interestablishment geronto-geriatric information system: Perceived usefulness and use in real time

21. L. Trahan, M. Demers, R. Geneau, H. Guay, M. Bowen, J. Tremblay. Evaluation of family medicine groups

22. F. Béland, H. Bergman, P. Lebel, A.-P. Contandriopoulos, J.-L. Denis, P. Tousignant, J. Monette, F. Ducharme. System of integrated care for the frail elderly (SIPA): Evaluation of phase I, June 1999 to May 2000

23. J.-L. Denis, A.-P. Contandriopoulos, C. Sicotte, N. Touati, C. Rodriguez, H. Nguyen. Evaluation of the Capitation Haut-Saint-Laurent project: An integrated care and services network  

24. M.-J. Fleury, A. Lesage, C. Mercier, M. Perreault, D. Aubé and L.-R. Poirier. Integrated service networks and response to the needs of individuals affected by acute mental health disorders

25. M.-J. Fleury, C. Mercier, J. Caron. The integration of mental health services: Comparison of the propensity of different strategies to integrate services in mental health networks

26. M.-J. Fleury, L. Cazale, M. Perreault. Evaluation protocol respecting the mental health service network demonstration project in the territory of the Centre local de services communautaires (CLSC) Longueuil-Ouest

27. J.-P. Fortin, L. Lamothe. The CLSC of the future: Home support and tele-care

28. P.-A. Lamarche, L. Lamothe, C. Bégin. Effects of emerging modes of integration of services in the Laurentian region, Quebec

29. P.-A. Lamarche, L. Lamothe, M. St-Pierre. Concerted action in a territory project (PACTE): Local management of services in the health and social services network in response to the needs of the elderly  

30. A. Tourigny, P.-J. Durand, A. Tourigny, L. Bonin, R. Hébert, M. Paradis. Evaluation of the efficacy of an integrated service network for vulnerable elderly people in a semi-urban territory


[Appendix 2]

About the Author

Raynald Pineault, MD, PhD
Direction de santé publique de Montréal
Institut national de santé publique du Québec
Université de Montréal, QC

Pierre Tousignant, MD, MSc
Direction de santé publique de Montréal
Institut national de santé publique du Québec
McGill University, QC

Danièle Roberge, PhD
Centre de recherche HĂ´pital Charles-LeMoyne
Université de Sherbrooke, QC

Paul Lamarche, PhD
Groupe de recherche interdisciplinaire en santé
Université de Montréal, QC

Daniel Reinharz, MD, PhD
Département de médecine sociale et préventive
Université Laval, Québec, QC

Danielle Larouche, MSc
Centre de recherche HĂ´pital Charles-LeMoyne
Université de Montréal, QC

Ginette Beaulne, BSc, PNP
Direction de santé publique de Montréal
Institut national de santé publique du Québec
Montreal, QC  

Dominique Lesage, RN, MSc
Institut national de santé publique du Québec
Montreal, QC

Acknowledgment

This project received financial support from the Canadian Health Services Research Foundation and the Réseau de recherche en santé des populations du Québec. Some of the institutions to which investigators are affiliated also provided substantial support.

The authors wish to thank the researchers responsible for the 30 studies synthesized in the collective (Appendix 1). We would also like to acknowledge the members of the decision-makers' panel: LĂ©onard Aucoin, Mike BĂ©nigeri, Pierre Bergeron, Denis Roy and Lysette Trahan, as well as Jean-Louis Denis and Lise Lamothe, who acted as moderators at the knowledge exchange meeting in February 2005.

We wish to express our gratitude to Louise Lapierre for her unfailing support and encouragement throughout the project, and to Sylvie Gauthier and Isabelle Rioux, for editorial assistance.

Correspondence may be directed to: Raynald Pineault, MD, PhD, Direction de santé publique de Montréal, 1301 Sherbrooke E., Montreal, QC H2L 1M3; tel.: 514-528-2400, ext. 3480; fax: 514-528-2512; email: rpineaul@santepub-mtl.qc.ca.

References

Canadian Institutes of Health Research (CIHR). 2004a. Finding a Balance in Federal Health Research Funding. Ottawa: Author.

Canadian Institutes of Health Research (CIHR). 2004b. Investing in Canada's Future: CIHR's Blueprint for Health Research and Innovation. Ottawa: Author.

Denis, J.L., P. Lehoux and F. Champagne. 2004. "A Knowledge Utilization Perspective on Fine-tuning Dissemination and Contextualizing Knowledge." In L. Lemieux-Charles and F. Champagne, eds., Using Knowledge and Evidence in Healthcare. Toronto: University of Toronto Press.

Devers, K.J. 1999. "How Will We Know 'Good' Qualitative Research When We See It? Beginning the Dialogue in Health Services Research." Health Services Research 34(5, Part II): 1153-88.

Dixon-Woods, M., S. Angwal, D. Jones, B. Young and A. Sutton. 2005. "Synthesizing Qualitative and Quantitative Evidence: A Review of Possible Methods." Journal of Health Services Research and Policy 10(1): 45-53.

Donabedian, A. 1980. The Criteria and Standards of Quality (Vol. 2). Ann Arbor, MI: Health Administration Press.

Huberman, M. 1994. "Research Utilization - The State of the Art." International Journal of Knowledge Transfer and Utilization 7: 13-33.

Innvaer, S., G. Vist, M. Tremmaid and A. Oxman. 2002. "Health Policy Makers' Perceptions of Their Use of Evidence: A Systematic Review." Journal of Health Services Research and Policy 7(4): 239-44.

Lamarche, P., M.D. Beaulieu, R. Pineault, A.P. Contandriopoulos, J.L. Denis and J. Haggerty. 2003. Choices for Change: The Path for Restructuring Primary Healthcare Services in Canada. Ottawa: Canadian Health Services Research Foundation.

Lavis, J. 2004. "A Political Science Perspective on Evidence-Based Decision Making." In L. Lemieux-Charles and F. Champagne, eds., Using Knowledge and Evidence in Healthcare. Toronto: University of Toronto Press.

Lavis, J., H. Davies, A. Oxman, J.L. Denis, K. Golden-Biddle and E. Furlie. 2005. "Towards Systematic Reviews That Inform Healthcare Management and Policy Making." Journal of Health Services Research and Policy 10 (Suppl. 1): 35-48.

Lomas, J. 2000. "Using 'Linkage and Exchange' to Move Research into Policy at a Canadian Foundation." Health Affairs 19(3): 236-40.

Lomas, J., T. Culyer, C. McCutcheon, L. McAulay and S. Law. 2005. Conceptualizing and Combining Evidence for Health System Guidance. Ottawa: Canadian Health Services Research Foundation.  

Macinko, J., B. Starfield and L. Shi. 2003. "The Contribution of Primary Care Systems to Health Outcomes within Organization for Economic Cooperation and Development (OECD) Countries, 1970-1998." Health Services Research 38(3): 831-61.

Mays, N., C. Pope and J. Popay. 2005. "Systematically Reviewing Qualitative and Quantitative Evidence to Inform Management and Policy Making in the Health Field." Journal of Health Services Research and Policy 10 (Suppl. 1): 7-20.

Meyer, A., S. Tsui and C.R. Hinnings. 1993. "Configurational Approaches to Organizational Analyses." Academy of Management Journal 36(6): 1175-95.

Patton, M.Q. 1999. "Enforcing the Quality and Credibility of Qualitative Analysis." Health Services Research 34(5, Part II): 1189-208.

Pawson, R.T., G. Greenhalgh and K. Walshe. 2005. "Realist Review - A New Method of Systematic Review Designed for Complex Policy Interventions." Journal of Health Services Research and Policy 10 (Suppl. 1): 21-34.

Pineault, R. and P. Tousignant. 2001. Transformations of the Montréal Network: Impact on Health Research Collective. Montreal: Direction de santé publique de Montréal.

Pineault, R., P. Tousignant, D. Roberge, P. Lamarche, D. Reinharz, D. Larouche, G. Beaulne and D. Lesage. 2005. Research Collective in the Organization of Primary Care Services in Québec. Summary Report. Retrieved March 3, 2006. http://www.chsrf.ca/research_themes/pdf/
summary_report_e.pdf and
http://www.greas.ca/publication/
pdfrapport/rapportsynthese.html
.

Romanow, R. 2002. Building on Values. The Future of Heathcare in Canada. Report of the Commission. Ottawa: Commission on the Future of Healthcare in Canada.

Sheldon, T.A. 2005. "Making Evidence Synthesis More Useful for Management and Policy Making." Journal of Health Services Research and Policy 10 (Suppl. 1): 1-5.

Showstack, J., N. Lurie, E.B. Larson, A.A. Rothman and S. Hassmiller. 2004. "The Future of Primary Care." America's Health Insurance Plans (AHIP) Coverage 45(3): 55-56, 58, 60.  

Starfield, B. 1998. Primary Care: Balancing Health Needs, Services, and Technology. New York: Oxford University Press.

Wilson, R., S.E.D. Shortt and J. Dorland, eds. 2004. Implementing Reform: Barriers and Facilitators. Montreal and Kingston: McGill-Queen's University Press.

Yin, R.K. 1999. "Enhancing the Quality of Case Studies in Health Services Research." Health Services Research 34(5, Part II): 1209-24.