Purpose: To develop and evaluate a toolkit for Registered Nurse/Registered Practical Nurse (RN/RPN) staff mix decision-making based on the College of Nurses of Ontario's practice standard for utilization of RNs and RPNs. Methods: Descriptive exploratory. The toolkit was tested in a sample of 2,069 inpatients on 36 medical/surgical units in five academic and two community acute care hospitals in southern Ontario. Survey and focus group data were used to evaluate the toolkit's psychometric properties, feasibility of use and utility. Results: Results support the validity and reliability of the Patient Care Needs Assessment (PCNA) tool and the consensus-based process for conducting patient care reviews. Review participants valued the consensus approach. There was limited evidence for the validity and utility of the Unit Environmental Profile (UEP) tool. Nursing unit leaders reported confidence in planning unit staff mix ratios based on information generated through application of the toolkit, specifically the PCNA, although they were less clear about how to incorporate environmental data into staff mix decisions. Conclusions: Results confirm that the toolkit consistently measured the constructs that it was intended to measure and was useful in informing RN/RPN staff mix decision-making. Further refinement and testing of the UEP is required. Future research is needed to evaluate the quality of decisions resulting from the application of the toolkit, illuminate processes for integrating data into decisions and adapt the toolkit for application in other sectors.

Introduction

There is a growing body of research linking nurse staffing with nurse and patient outcomes (Canadian Health Services Research Foundation 2006; Canadian Nurses Association [CNA] 2004). In particular, having the appropriate mix of nursing staff is seen as essential to providing safe and effective nursing care. Studies conducted over more than a decade provide support for the value of a skill mix that is rich in regulated professional nurses (Aiken et al. 2002; Blegan et al. 1998; Lichtig et al. 1999; McGillis Hall 2003; McGillis Hall et al. 2003, 2004; Needleman et al. 2002; Sochalski et al. 1997; Tourangeau et al. 2006).

Research evidence from the United States (US) and Canada has demonstrated superior patient outcomes associated with higher levels of RN care compared with models with higher proportions of care provided by Registered/Licensed Practical Nurses (R/LPNs¹) and unregulated care providers (Needleman et al. 2002; Tourangeau et al. 2006). In addition, registered nurses are more frequently involved in most elements of the decision-making process, suggesting that they may be better equipped to make decisions where client complexity is increased (Boblin et al. 2008).

Because differences exist between Canada and the US and across provinces in Canada in the educational preparation, entry-to-practice requirements and regulatory structures for RNs and R/LPNs, it is important to interpret current literature with an understanding of these variations. In Canada, baccalaureate education is (or is becoming) the minimum entry to practice for RNs in all provinces and territories except Quebec and the Yukon (CNA 2009). Educational preparation of R/LPNs varies across the country, with some jurisdictions accepting a certificate as the minimum qualification, while others require a diploma. Beginning in 2005 in Ontario, where the current project was undertaken, a two-year (four-semester) diploma program was established as the requirement for entry to practice for registered practical nurses (Baumann et al. 2009; College of Nurses of Ontario [CNO] 2004). The practice of both RNs and RPNs in Ontario is regulated by the CNO.

The CNO (2005)² has developed a practice standard to guide the utilization of RNs and RPNs. This standard proposes that the more complex the client situation, and the more dynamic the clinical environment, the greater the need for an RN to provide the full range of care, assess changes, re-establish priorities and determine the need for additional resources. For this reason, decision-making related to nurse staffing and staff mix should be done at the unit level to ensure appropriate consideration and appreciation of patient, nurse and environmental characteristics (Registered Nurses' Association of Ontario [RNAO] 2007).

Evidence suggests that application of frameworks for nurse staffing remains a challenge for decision-makers (McGillis Hall et al. 2006). Despite the increasing body of knowledge and evidence linking staffing with patient outcomes and safety, there remains a gap in available evidence-based tools and literature to guide decision-making processes that can assist managers in making staffing decisions that best meet patient care needs.

This article will describe the results of a project that addressed this gap. The project involved the development and testing of a "toolkit" of instruments and processes designed to assist nurse leaders to make informed, evidence-based decisions for unit staffing plans that include the appropriate mix of RNs and R/LPNs in the staffing complement. The toolkit has been designed based on a model of total patient care on adult medical/surgical units. The intent is to support managers with an evidence-based approach to determining their overall nursing staff mix plan, with consideration of the trending patterns of their patient population and the characteristics of the nursing unit environment.

Background to the Project

The study was undertaken as a demonstration project in nursing health human resource planning funded through the Nursing Secretariat of Ontario's Ministry of Health and Long-Term Care. The purpose of the project was to develop and evaluate a toolkit of instruments and processes to inform decision-making regarding nursing staff mix in adult acute care medical and surgical hospital units. Seven hospital organizations in southern Ontario participated in the project. A previous version of the tools and processes had been developed at one of the participating hospitals to support nursing staff mix decision-making. Three of the other hospitals had subsequently used the tools and processes with some modifications.

The toolkit was based on the CNO's (2005) standard for the utilization of RNs and RPNs. The standard asserted that decisions regarding the appropriate category of nurse to care for a client's needs must account for client (complexity of care needs, predictability of outcomes, risk of negative outcomes), nurse (knowledge, skill, judgment) and environmental (support tools, consultation, stability of the environment) factors. The goal of the toolkit was to operationalize client, environmental and nurse factors and provide a process for integrating this information to support decisions regarding nursing staff mix. It consisted of four components, designed to work together to ensure an integrated, complementary approach:

• The Patient Care Needs Assessment (PCNA – Appendix A see https://www.longwoods.com/content/21733)
• A consensus-based unit review procedure
• The Unit Environment Profile (UEP – Appendix B see https://www.longwoods.com/content/21733)
• A decision-making process for determining nursing staff mix

The specific aims of the project were to:

• Assess the reliability, validity and feasibility of the PCNA,
• Assess the validity and feasibility of the UEP,
• Describe the process through which nurses use these tools in formulating nursing staff mix decisions, and
• Identify critical success factors and lessons learned regarding application of the tools and review process

Study Design and Methods

A descriptive, exploratory design using a convenience sample from seven hospital organizations was used. Both quantitative and qualitative analyses were employed. Ethics approval was obtained from the Research Ethics Boards of each participating hospital.

Setting and Sample

The study was conducted across 36 medical/surgical adult inpatient units from five academic teaching hospitals and two community hospitals in southern Ontario.

Development of the Toolkit

Patient Care Needs Assessment (PCNA) Tool

The blueprint for the PCNA consisted of four key patient elements from the CNO's (2005) standard: stability, complexity, predictability and risk for negative outcomes. Items were derived from the original tool developed at Sunnybrook Health Sciences Centre and were revised based on a review of the relevant literature. The questionnaire was subjected to review and critique by a panel of 10 nursing practice experts from the partner hospitals to establish content validity. The resulting version was pilot tested on two surgical units and revisions were made. The final version of the PCNA consisted of one open-ended question designed to elicit information about the patient's priority needs for nursing care; 11 yes/no items related to specific aspects of the patient's and family's situation and needs; and four Likert-type scaled questions that addressed the patient's stability, complexity, predictability and risk for negative outcomes. Definitions were developed for each of these four dimensions, based on the CNO standard and experience during the pilot. It was intended that the yes/no items would illuminate factors that affected each of the dimensions of stability, complexity, predictability and risk for negative outcomes, such that the scoring on the four dimensions would be informed by the yes/no items.

Unit Environment Profile (UEP)

The UEP tool was based on an instrument previously developed at the University Health Network in Toronto. Revision of the tool for the current study was guided by relevant literature, particularly the work of Dr. Linda O'Brien-Pallas and colleagues (1997) on nursing unit environmental complexity, and the CNO (2005) standard. The tool was reviewed for content validity by the panel of 10 nursing experts from the participating hospitals. The UEP was pilot tested with three nurse managers, and modifications were made to enhance clarity and consistency of interpretation. The final UEP contained 41 items addressing characteristics of the patient population, nursing staff and supports available to nurses.

Consensus-Based Unit Reviews

Reviews were conducted of every patient's care needs on each participating unit on two separate occasions approximately one week apart. Using a consensus-based approach, the review process involved application of the PCNA to each patient on the unit. The review team consisted of the patient's nurse (RN or RPN) and unit nursing leaders. The type of nursing leader varied depending on the nursing roles within the organization, but generally included the manager, advanced practice nurse, nurse educator and charge nurse or team leader. All reviews were facilitated by a nursing leader who was external to the unit team; a second external nurse leader documented the results for each review. The facilitator posed each PCNA question in turn to the entire unit nursing team, who collaborated to determine the appropriate response. To achieve respectful engagement of all participants during the review process, guidelines were created for facilitators based on feedback from the pilot (Appendix C see https://www.longwoods.com/content/21733).

Decision-Making Process

Following the unit patient reviews and completion of the UEP by the managers, debriefing meetings were held with nursing leaders from each unit to review their summarized PCNA and UEP data and explore ways in which the data could be used to support nursing staff mix decision-making.

Evaluation of the Toolkit

Central to the development of the toolkit was the belief that appropriate skill mix decisions require the participation of the nurses who will be involved in their implementation. Therefore it was critical to the evaluation of the toolkit that the perspectives of the nurses who participated in the reviews – the clinical nurses, managers, advanced practice nurses, charge nurses and educators – were solicited. A combination of surveys and focus groups was employed to elicit data from these groups.

Instruments

Three surveys were developed for this study to gather information about the nurses' experiences with the toolkit. The Patient Care Needs Assessment Opinion Questionnaire (PCNAOQ) was a self-report survey consisting of five questions that captured the nurses' perceptions of the comprehensiveness and accuracy of the items with respect to patient care needs, and two items related to the clarity and ease of use of the instrument. This provided information regarding the face validity and feasibility of the patient assessment tool. Responses were documented on a three-point Likert-type scale with anchors 1 = "Not at all" and 3 = "Very much so" and one open-ended question for any additional comments.

The Unit Environment Profile Opinion Questionnaire (UEPOQ) was designed to understand the opinions of managers regarding the applicability and use of the UEP tool. It was similar in form to the PCNAOQ and included four questions that addressed face validity, two that addressed feasibility and one open-ended question for comments. Response options for this instrument were the same as those for the PCNAOQ.

The Questionnaire for Unit Nursing Leaders (QUNL) was a self-report survey consisting of five open-ended questions to elicit nursing leaders' perceptions and opinions about the review process. Questions addressed participant thoughts regarding the impact of the review process on their nursing staff mix decision-making, the content of decisions they would make resulting from the review and the strengths and limitations of the review/decision process.

The fourth tool employed in the toolkit evaluation was the Patient Requirements for Nursing Care (PRNC) instrument (Fulton and Wilden 1998). This 15-item tool was designed to quantify medical/surgical patients' needs for nursing care, based on the patient's complexity of care requirements, acuity of illness and level of dependency. The PRNC was used for convergent validation of the PCNA.

Data Collection Procedures

Responses to the PCNA questions were documented during the unit reviews. On Day 1 of the reviews, three randomly selected patients per unit across all hospitals (N = 106) were also assessed with the additional tool (PRNC), testing for convergent validity. Using the consensus-based review process, two assessments were conducted on these patients, one using the PCNA and one using the PRNC. In half of the patient reviews, the PCNA assessment was conducted first, followed by the PRNC assessment. In the other half, the order was reversed.

Following each review, all participants were given a copy of the questionnaire to assess the PCNA tool and were asked to voluntarily complete it anonymously and deposit it in the designated collection box on each unit. Additionally, on the first review day every manager was given the UEP, with direction that it be completed within two weeks after the patient reviews were conducted. Completed UEPs were returned to each site's project coordinator. Unit managers were then given a copy of the questionnaire (UEPOQ) to provide feedback on the UEP tool itself and were asked to voluntarily complete it anonymously and return it to the study coordinator. Further, following the debriefing meetings, the unit nursing leaders were given a copy of the questionnaire focused on understanding their thoughts related to the review process and its impact (QUNL), which they also returned (anonymously) to the study coordinator.

Finally, focus groups were conducted with a volunteer sample of members from all clinical review teams to seek their opinions regarding the comprehensiveness and accuracy of the unit reviews, as well as their views on the review process itself. Focus groups were facilitated by an external nurse consultant.

Statistical analysis of the PCNA data was conducted through the Institute for Clinical Evaluative Sciences using SAS Version 9.1.3. Responses to the yes/no items were dummy coded for this analysis, with "yes" responses coded as 1 and "no" coded as 0. Questions where a "yes" response reflected a favourable patient condition were reverse coded so that all scores operated in the same direction.

Descriptive statistics were calculated for variables on the patient assessment tool (PCNA) and the two opinion questionnaires regarding the patient assessment tool and the environmental tool (PCNAOQ and UEPOQ). Inter-item and internal consistency reliability coefficients were calculated for PCNA data; correlation analyses were conducted to compare results between the two patient assessment tools (PCNA and PRNC). Focus group discussions were audiotaped, transcribed and analyzed for thematic content.

Results³

PCNA Reviews

A total of 2069 patient reviews were conducted. Distribution of responses to the yes/no questions is summarized in Table 1. Questions that addressed anticipation of complications, need for adjustment in the plan of care, presence of high-risk interventions, unexpected events and complex patient or family needs showed the most balanced distribution between "yes" and "no" responses. Questions related to vital signs within predetermined limits and level of consciousness within expected range elicited positive responses in over 90% of the patient reviews. Similarly, almost all patients were assessed as having no fluctuations in level of consciousness at the time of the review.

Frequency of vital signs measurement ranged from continuous monitoring of vital signs to once every 24 hours (M = 6.76 hours; SD = 3.13). More than half of the patients reviewed required monitoring of vital signs more frequently than every eight hours.

Items 11 through 14 on the PCNA questionnaire assessed the patient on the dimensions of stability, complexity, predictability and risk for negative outcomes. As demonstrated in Table 2, the most frequent response for each dimension was in the middle range (2.1–4.0), with fewer patients assessed at the higher and lower extremes of the scales.

Reliability and Validity of the PCNA

Cronbach's alpha coefficient for the yes/no component of the PCNA was 0.70. Polychoric correlation coefficients for the relationship between individual yes/no questions and the dimensions of stability, complexity, predictability and risk for negative outcomes (Table 3) were all significant (p <.001). Analysis of the magnitude of the correlations revealed that the items most strongly associated with both patient stability and predictability were those that assessed the need for more frequent monitoring for complications, and reassessment and revision to the patient's plan of care. Complexity was most closely aligned with questions related to revision of the plan of care, patient and family support needs, and supporting patients and/or families with decision-making. Correlations for all of these relationships were 0.60 or higher. Inter-item correlations for the dimension of risk for negative outcomes were generally lower than for the other dimensions; however, items related to increased monitoring, increased adjustment in the plan of care, high-risk interventions, support needs and complex patient or family decisions were all correlated with risk for negative outcomes at r values of 0.50 or higher.

The consistency of the yes/no questions with the scores on the individual dimensions of stability, complexity, predictability and risk for negative outcomes was also analyzed by comparing the total score for the yes/no items to the scores on each of the dimension scales (Table 4). While the PCNA tool is not intended to yield a total score for the purpose of identifying patients' needs for nursing care, comparison of the sum of the yes/no items to each of the dimension scales provided insight into the extent to which the process of answering these questions supported the determination of the ratings on the four dimensions. Spearman correlation coefficients demonstrated a strong positive relationship between the yes/no questions and each of the four dimensions. In each case the coefficients were significant and greater than 0.62. There was also a strong positive relationship among the four dimensions of stability, complexity, predictability and risk for negative outcomes, with correlation coefficients ranging from 0.67 to 0.79.

Face validity of the PCNA tool was assessed through responses to the PCNAOQ and focus group comments. Responses to the PCNAOQ were positive, with mean scores of 2.3 or higher for each question, with the exception of the item related to information missing from the PCNA, which yielded a mean score of 1.9. Similar positive responses were elicited during the focus group discussions, as illustrated by the following sample comments:

We went through the patients that I had that day, and a lot of those criteria met my patients' needs, and I felt it gave a reflection of my patients that particular day.

It's a pretty broad scope. It encompasses everything.

I think it is comprehensive in terms that you're getting input from a variety of sources versus just it being like on a management level or a senior team level, it's actually asking the nurses.

Convergent validation of the PCNA was performed by comparing PCNA results with those generated with the comparison instrument, the PRNC. The Spearman correlation coefficient for the summary scores from the two instruments was 0.63 (p < .0001), indicating considerable congruence between the results generated by the two instruments in the 106 patients reviewed. Significant correlations were also noted between individual items on the PCNA and PRNC that addressed level of consciousness, confusion or disorientation, need for increased monitoring and adjustments to the plan of care, and support for patients and families. In addition, the stability dimension score on the PCNA was associated with the need for analysis of condition and problem-solving items on the PRNC. The PCNA measurement of complexity was also associated with problem solving on the PRNC, while the PCNA dimension of predictability was associated with need for analysis of condition on the PRNC. The PCNA dimension "risk for negative outcomes" did not exhibit significant correlations with any of the PRNC items. Conversely, PRNC items that were more task focused – such as number of systems requiring assessment; number of medications, treatments and dressings; provision of education and counselling; and physical care needs – were not correlated strongly with any of the PCNA items.

Feasibility and Validity of the UEP

The number of responses obtained for the UEPOQ and QUNL was quite low (7/36 and 16/36 respectively), limiting conclusions regarding the UEP. However these responses together with comments during the focus group discussions suggested that the managers had a moderate level of satisfaction with the tool but felt additional items were needed to provide a complete picture of the unit environment. Managers also noted that some data were not readily available to them. Comments from the nursing leaders during the debriefing meetings also suggested that they were less clear about how to incorporate environmental information into staff mix decisions.

Evaluation of the Review Process

One hundred and fifteen nurses participated in the focus groups, including staff nurses, unit managers, team leaders, advanced practice nurses and educators. Five themes related to the process were identified. The collaborative approach was seen as supporting front-line staff, and the discussions leading to consensus provided additional clinical knowledge and assisted in answering questions and determining the scores on the rating scales. Participants also valued the support of an extra nurse on the day of the review, and having an opportunity to review the PCNA instrument prior to the unit review.

Both staff and leaders noted that the reviews required considerable time. Some nurses commented that they felt anxious going into the review and were uncomfortable answering questions in front of their unit leaders.

Responses to the QUNL indicated that participants found the review process beneficial, both in terms of informing nursing staff mix decisions and as an opportunity to witness nursing staff discussing patient care. They noted that some questions were subjective in nature but acknowledged that the consensus approach balanced this subjectivity.

Utility of the Toolkit in Supporting Nursing Staff Mix Decision-Making

Data from the focus groups and unit leader questionnaires indicated that overall nursing leaders felt the toolkit would be helpful in supporting nursing staff mix decision-making and that they would make changes based on the information generated by the tools and review process.

Discussion

Results from this evaluation project suggest that the RN/RPN Utilization Toolkit can support nursing staff mix decision-making. The PCNA appears to capture information about patient stability, complexity, predictability and risk for negative outcomes. By considering each patient's constellation of scores on these four dimensions, a coherent and comprehensive picture of the patient's needs emerges. In particular, the dimension of patient risk for negative outcomes, a recommended consideration in staff mix decisions (CNO 2005), was addressed in the PCNA but not in the comparison tool. Conversely, the lack of correlation between PCNA data and task-focused items on the PRNC suggests that the PCNA captures information that is distinct from workload, as intended.

The results also suggest some opportunities to enhance the utility of the PCNA tool. Vital signs and level of consciousness showed little variability in this sample, suggesting that these items may not contribute substantially to assessment of patient care needs in this population. The lower correlation of the risk dimension with the yes/no items may have been due to inconsistency in the interpretation of this question. In light of this, it is recommended that the question be revised to include a timeframe for the assessment of risk (e.g., within the next 48 hours).

Engagement of clinical nurses and unit leaders in generating the data on which decisions are made, as recommended in best practice guidelines (RNAO 2007), provides an opportunity for nurses to contribute to key decisions regarding their work and may enhance confidence in and credibility of decisions. In addition, staff involvement in decision-making may have a positive impact on recruitment and retention.

Study results suggest that incorporation of environmental data into staff mix decisions remains a challenge for nurses. Refinement of the UEP is required to ensure that it captures those elements of the practice environment that are relevant to staff mix decisions. For example, provisions for continuity of caregiver (RNAO 2007), although potentially addressed though UEP items related to percentage of full-time staff, may need to be articulated further. An organization's vision of its model of care must also be considered. In addition, ensuring access to required information, and supporting nurse leaders to incorporate this information into staff mix decisions, is necessary.

One aspect of nursing work that is not captured through the toolkit is the amount of time and physical demands associated with meeting the needs of patients. Nursing workload measurement processes are designed to assess the volume of work that patients' needs generate and to estimate the time required to meet those needs. In applying the results of the toolkit to staff mix decision-making, it is important that this aspect of nursing work be integrated into the decision.

Implications for Future Research

Results of the current study provide support for the reliability and validity of the inferences drawn from the application of the PCNA to identify patients' needs for nursing care. However, additional research is required to understand the processes through which clinical nurses and nurse leaders can use this information to support effective decisions regarding nursing staff mix. For example, prior application of the unit review process included an additional step in which each patient was identified as appropriately assigned to either an RN or an RPN. Studies exploring this and other procedures for moving from data collection to staff mix decisions are required.

Further work is also urgently needed to evaluate the quality of staff mix decisions as determined by patient, nurse and system outcomes. In particular, the use of nurse sensitive outcomes data to measure the impact on patient outcomes is key. The Canadian Nurses Association Evaluation Framework to Determine the Impact of Nursing Staff Mix Decisions (2005) can provide direction regarding evaluation metrics and processes. Studies that track these outcomes prior to and following staff mix changes, and comparisons of units with similar patient needs and unit environments but different staff mixes, need to be undertaken.

Conclusions

In today's economic environment, provincial governments are facing record deficits and healthcare funding reductions. Nursing resources have historically been viewed as a source of potential savings by healthcare organizations in times of budget constraints. Nursing resources have historically been viewed as a source of potential savings in times of budget constraints. As a result, pressure is placed on nursing administration and managers to reduce nursing care hours per patient day and to consider various staff mix changes, including different RN/RPN ratios as well as the addition of unregulated workers. Although there is a difference in compensation levels between RNs and RPNs, staff mix should be driven by patient care requirements rather than the potential gain in financial savings.

Staff mix decisions must be made with empirical evidence that aligns the knowledge and skills of different workers with patient stability, complexity, predictability and risk for negative outcomes, as well as characteristics of the work environment. Time-based parameters such as hours per patient day and workload data are inadequate for measuring these factors. The PCNA and UEP tools and the consensus-based review process, when used together, can help to improve decision-making in matching patient needs to nursing human resources, focusing on the patient's needs, the nurse caring for the patient and the unit environment. In addition, utilization of a consistent framework for assessing patient needs and nursing environments will enable meaningful comparisons across organizations and over time.

Another positive outcome of the work to develop this toolkit is that it provides nurse leaders with a mechanism previously not available in the literature to apply the CNO (2005) standard for utilization of RNs and RPNs. The basis of the framework requires consideration of the patient, nurse and environmental characteristics. The four components of the toolkit in this study provide the mechanisms to comprehensively assess the patient (PCNA), nurse and environment characteristics (UEP). Although other factors, such as knowledge of the scope of practice of RNs and RPNs, dynamics of the healthcare environment, labour relations, workforce demographics, attrition and regulatory changes also represent components to consider in decision-making, the toolkit provides a thorough and user-friendly mechanism to assess the fundamental considerations of patient, environment and nurse factors.

Overall, the results of this study support the use of these tools and processes to guide managers in deciding the appropriate nursing staff mix on adult medical/surgical units. Further understanding of the application of the toolkit itself, such as frequency of application and review of nursing staff mix, will be important. Additional research is also now required to evaluate the quality of decisions resulting from application of the toolkit, and to illuminate the processes through which nursing leaders can best translate the information generated through the tools into staff mix decisions. Modification and further testing of these tools and process for application in other sectors, such as long-term care, mental health and community settings, is also being explored. The RN/RPN Utilization Toolkit is an important advancement toward facilitating the determination of the right staff mix for each clinical context and maximizing the use of nursing health human resources.

Appendix A – Patient Care Needs Assessment

Appendix B – Unit Environment Profile

Appendix C – Guidelines for Review Facilitator

Comments

Be the first to comment on this!