Healthcare Policy
Organization and Dissemination of Working Knowledge
Abstract
This commentary discusses the approach taken by the Manitoba Centre for Health Policy to systematize its working knowledge. Web-based knowledge tools facilitate maintenance of the information-rich environments present in almost every province. The malleability of administrative data means that variables can be defined in many different ways. Keeping track of what has been done becomes critical to facilitate reuse. New social data sets pose particular challenges and opportunities. Some more general issues of design are highlighted by a perusal of websites associated with other research efforts (the Panel Study of Income Dynamics, the National Bureau of Economic Research and the Institute for Clinical and Evaluative Sciences).
[To view the French abstract, please scroll down.]
Websites are constantly incorporating more and more information; decisions related to organization are increasingly necessary. Here we describe the approach taken by the Manitoba Centre for Health Policy (MCHP) in systematizing its working knowledge, and also look at some general issues of website design. Sites are now being used for multiple purposes - by teachers and students, by researchers and consumers of research. What tips have emerged from our experience?
Web-based knowledge tools at MCHP have facilitated the development and ongoing maintenance of an environment rich in information (Figure 1). These tools have been used to document file structures and field names for the data sets that comprise the Manitoba Population Health Research Data Repository, to record conceptual and operational definitions and SAS codes for relevant research constructs and to capture critical steps in the implementation or plans for projects using repository data. Organizing new knowledge so that others can use it, and then passing such knowledge on to succeeding generations of researchers, is important for ensuring research productivity.
The malleability of administrative data represents both problem and opportunity. Since variables can be coded in many different ways, keeping track of what has been done and imposing order on the activities become critical for cost-efficient research. The MCHP's Documentation Management Group seeks to plan the Centre's approach to maintaining and updating the Web-based tools for knowledge documentation and management. With some additional effort, the material used for internal purposes can be organized for a wider audience.
The Manitoba Centre for Health Policy (2006) has structured its research resources in terms of:
- Research Protocol - guide to conducting a new research study using administrative data at MCHP
- Data Dictionaries - generally restricted to staff
- Glossary and Related Terms - brief definitions of terms used and links to more detailed definitions
- Concept Dictionary - extensive definitions of research concepts used by MCHP studies.
Documentation has also been developed on how to maintain the Glossary and Concept Dictionary on an ongoing basis. Such documentation for maintaining the Research Protocol and Data Dictionaries is planned.
Need for Details
Most of the provincial databases that are used for research across Canada do not contain as many files as does the Manitoba database. Nonetheless, several sites with which the authors are familiar would seem likely to benefit from the Research Protocol/Glossary/Concept Dictionary approach outlined above. For example, to the best of our knowledge, British Columbia handles length of hospital stay differently than Manitoba does. After a stay of 30 days, an individual in British Columbia is classified as a long-stay patient and a new abstract for the remainder of the stay is produced. In Manitoba, just one hospital separation abstract is generated no matter how long the length of stay.
The need for details is at least as great for those wishing access to Statistics Canada material. The Research Data Centres (jointly sponsored by Statistics Canada, the Social Sciences and Humanities Research Council and university consortia) can be expected to generate high levels of demand for "nitty-gritty" information about files. Statistics Canada's own staff would want such information well documented and organized for their own purposes.
Concepts
Documenting complicated material that might be reused (by the author or other investigators) is a priority addressed in the Manitoba Concept Dictionary. Key working knowledge is often too extensive to be published. Even if published in an academic paper (as text, table or appendix), such material might be difficult to find again. Relatively small amounts of text (100 or 200 words) seem to be reproducible from journals with acknowledgments on websites. At any rate, we have never had problems with journals or authors requesting changes or deletions to entries in the Concept Dictionary.
For example, the Concept Dictionary lists 12 "ambulatory care sensitive conditions" which, although traditionally defined by four- or five-digit (ICD-9-CM) codes on hospital discharge abstracts, can be closely approximated by the three-digit codes on Manitoba physician claims. Our research appears to have been the first population-based study using both physician and hospital data; the results suggested that avoiding hospitalization through "good" ambulatory care for these conditions is much more difficult than previously thought (Roos et al. 2005).
More generally, not only have diagnostic codes for certain conditions not been standardized, but journals have not insisted on structured presentation of certain facts about research (ICD codes, loss to follow-up, etc.). A review by Guyatt et al. (2006) comparing American and Canadian surgical outcomes highlighted this for the senior author. Although referenced in published papers, information that was important for assessment of the quality of the author's own work was not easily available to the abstractors performing their literature review. Since different journals have markedly different audiences and styles, a central, Web-located listing that addresses issues of quality, such as follow-up, is useful for any research group.
Who Does What
Who is to develop and maintain the content of such "working knowledge"? Because brief Glossary entries have been made part of the Manitoba Centre's deliverables, their write-up is part of the lead investigator's responsibilities. The Concept Dictionary and Research Protocol need staff assigned with specific responsibilities for such work. Even the Glossary needs frequent attention to coordinate entries across deliverables.
How should decisions be made about what is to be included and what not? One driver has been requests ("frequently asked questions") for more or clearer information by staff, by collaborators outside Manitoba and by external reviewers. As some of our efforts have expanded from health research to social research, new complicated variables (such as indices of educational achievement) and more work with the research registry (specification of family structure) have had to be explained in considerable detail.
New Health and Social Data Sets
Large data sets specifically collected for research purposes - particularly long-term efforts like the University of Michigan's (2006) Panel Study in Income Dynamics (PSID) - will have a structure imposed upon them. In contrast, administrative data will usually need to be carefully organized after the fact. Each new healthcare data set has its own idiosyncrasies and problems. Data from provincial ministries dealing with social issues seem even more complicated and more likely to vary over time than the healthcare data sets. Provincial health ministries have been relatively constrained by efforts (such as ICD coding) to establish national and international credibility.
Researchers using social data sets will often have questions about the populations served that are largely not asked for health data sets (given Canada's national health insurance). The possibilities for following children throughout their educational careers are intriguing, but test and grade data may be uneven or unavailable for other than recent years. In the most practical terms, descriptions of data sets, including their strengths and weaknesses, are generally helpful for both actual research and proposal writing. Warburton and Warburton (2004) have highlighted the desirability of making provincial administrative data sets on welfare dependency and education available for research. If progress were to be made along these lines, reference to a publicly accessible website would help provide information to interested academics, administrators and reviewers.
Organizing Websites
Work on our Manitoba website and perusal of other sites has generated suggestions for improvement.
- Structure output (papers and reports) by topic. Although PubMed, ISI (Institute of Scientific Information) and Google Scholar searches will find material on a variety of topics, additional organization remains useful for a number of purposes. The Panel Study of Income Dynamics has categorized papers relying on this complex data set simply but effectively. The National Bureau of Economic Research (NBER Working Papers) and the Institute for Clinical and Evaluative Sciences (ICES) have catalogued their output by subject. Data being used across disciplines may particularly benefit from such attention; PubMed is likely to miss material not published in health-related journals.
- Locate a navigation bar at the top of every page. Some of this has been done for the MCHP website, but the navigator bar is not as ubiquitous as that at the PSID site.
- Develop a site guide that displays all pages that can be accessed at a given site.
Organisation et dissémination des connaissances pratiques
Résumé
Ce commentaire traite de l'approche adoptée par le Manitoba Centre for Health Policy pour systématiser ses connaissances pratiques. Les outils de connaissance axés sur Internet facilitent le maintien des environnements riches en renseignements qu'on retrouve dans presque chaque province. La malléabilité des données administratives permet de définir les variables de nombreuses façons différentes. Il devient donc essentiel de suivre ce qui a été déjà fait afin d'en faciliter la réutilisation. Les nouveaux ensembles de données sociales posent des défis particuliers et offrent des occasions uniques. Certaines des questions plus générales liées à la conception deviennent évidentes lorsqu'on visite les sites Web consacrés à d'autres initiatives de recherche (le Panel Study of Income Dynamics, le National Bureau of Economic Research et l'Institut de recherche en services de santé).
About the Author(s)
Leslie L. Roos, PhD
Manitoba Centre for Health Policy
Faculty of Medicine, University of Manitoba, Winnipeg, MB
Lisa Lix, PhD
Manitoba Centre for Health Policy
Faculty of Medicine, University of Manitoba, Winnipeg, MB
Ruth Bond, MA
Manitoba Centre for Health Policy
Faculty of Medicine, University of Manitoba, Winnipeg, MB
Stephanie Smith
Manitoba Centre for Health Policy
Faculty of Medicine, University of Manitoba, Winnipeg, MB
Correspondence may be directed to: Leslie L. Roos, Manitoba Centre for Health Policy, Department of Community Health Sciences, Faculty of Medicine, University of Manitoba, Rm. 408 - 727 McDermot Avenue, Winnipeg, MB R3E 3P5; tel.: 204-789-3773; fax: 204-789-3910; email: Leslie_Roos@cpe.umanitoba.ca.
Acknowledgment
We would like to thank the Lupina Foundation for financial support. As always, Jo-Anne Baribeau provided excellent assistance in preparing the manuscript.References
Guyatt, G.H., P.J. Devereux and J. Lexchin. 2006. "A Systematic Review of Studies Comparing Health Outcomes in Canada and the United States." Unpublished.
Manitoba Centre for Health Policy. 2006 (February 13). Research Resources Index. Retrieved April 18, 2006. http://www.umanitoba.ca/centres/mchp/concept.
Roos, L.L., V. Menec and R.J. Currie. 2004. "Policy Analysis in an Information-Rich Environment." Social Science and Medicine 58(11): 2231-41.
Roos, L.L., R. Walld, J. Uhanova and R. Bond. 2005. "Physician Visits, Hospitalizations, and Socioeconomic Status: Ambulatory Care Sensitive Conditions in a Canadian Setting." Health Services Research 40(4): 1167-85.
University of Michigan Institute for Social Research. 2006. Panel Study of Income Dynamics. Retrieved April 18, 2006. http://psidonline.isr.umich.edu.
Warburton, R.N. and W.P. Warburton. 2004. "Canada Needs Better Data for Evidence-Based Policy: Inconsistencies between Administrative and Survey Data on Welfare Dependence and Education." Canadian Public Policy 30(3): 241-55.
Comments
Be the first to comment on this!
Personal Subscriber? Sign In
Note: Please enter a display name. Your email address will not be publically displayed