Challenges Inherent in Outcomes Measurement

Collection of outcomes data requires time. With the present workload of nursing staff, it seems obvious that collecting data does not rank highly on the priority list of nursing activities. Yet, without evidence that nursing is indeed impacting the quality of care, decisions will be made that may be detrimental to quality patient care. In addition, linking outcomes data to nursing structural variables such as staffing levels could have an immediate impact on health and institutional policy. Huston (1) advocates that outcomes measurement begin at the unit level and actively involve all nurses. This may require adjustment of budgets to include both direct and indirect costs associated with outcomes tracking (10). The challenge for both nursing administrators and staff nurses is to change their focus from tasks to evidence-based practice.

Data Collection.

The specific procedures for collecting data must be planned in advance, written, and understood by all involved to ensure consistency during the process. Copies of a written data collection plan should be available for reference in all areas where data collection is occurring. To complete a written plan, decisions that address the who, what, when, where, and how of data collection must be resolved prior to beginning the process.

Who refers to the data collectors. Will they be staff nurses, quality improvement specialists, clinical specialists, other clinicians, or non clinicians? Who will they report to for questions and who will make decisions about the process? Data collectors need to have a knowledge base about the topic, process, and how the varied instruments are used. They also need to be flexible yet precise. In clinical settings, last minute changes frequently occur. Consistency and integrity in the process of data collection, however, is critical. Training will have to be available for all data collectors so that questions can be answered and interrater reliability can be assured.

What refers to the specific research question/s, unit of analysis, data definitions, the forms and protocols required, and the sample. The specific question/s to be answered provides the foundation for the data collection effort and guides the entire process. The unit of analysis is the basic unit of an investigation (7). In healthcare, it often is the patient, although sometimes it can be the department or the provider. Operational definitions that are readily available will assist the data collectors when questions occur about specific variables. One way to provide consistency is to include a data dictionary for all staff involved in the data collection process. A data dictionary provides definitions for the study variables and examples for the data collectors to review. All required forms and specific procedures pertinent to the data collection process should be included with the data dictionary. This assures a complete set of reference materials that will guide the collectors through the process.

The sample from which data are collected must be considered prior to the implementation of data collection. When investigating specific questions, researchers consider the type of sample, sample criteria, and the desired sample size. In clinical situations, we rarely have the opportunity to prospectively consider these factors. With a little preplanning, however, clinicians can use the sampling principles from research to derive a powerful sample from an established clinical service (11). The goal is to determine a subset of patients that will allow us to learn about the care they received without needless data collection on all patients. Consultation with a statistician can be helpful in determining sample size and methods.

When is the frequency of data collection. Prior to implementation, the responsible individual must decide how frequently the data will be collected. There is no right way or magic associated with this decision. It depends on the measure being collected and its importance to the overall performance improvement plan. Many process indicators are collected concurrently, but, are analyzed quarterly. An example of this includes a "skin integrity" risk assessment. Some indicators, such as patient falls, are dependent on additional data collection forms (the incident report). Others still are only collected at discharge or even post discharge (patient satisfaction). Nonetheless, decisions about frequency of data collection will need to be made for each indicator.

Where refers to the actual place or setting of data collection. Will it be on the patient unit, in the patient records department, or in the patient's home? If patients are required to directly answer questionnaires, those who are acutely ill will require more time compared to those who are already discharged. Abstracting data from patient records, while not involving interaction with patients, may present a variety of problems. Often, the data may be missing from the patient record or not legible. Data may be recorded in the chart, but, not be consistent with the operational definition. This requires the data collector to clinically interpret and make decisions about indicators.

How refers to the actual administration of an instrument. Several methods are used to collect data such as questionnaires, interviews, or observation. Knowledge of the instrument and its intended administration process is essential since the data must be accurately recorded. Participant problems such as patient refusal or patient absence and external influences such as family members can influence the administration of an instrument (12). The data collectors themselves may find difficulties when patient needs appear more crucial than the data collection process or they are asked to stop data collecting and provide patient care. Finally, data entry, lost forms, or not enough forms can complicate the implementation of data collection.

Using a step by step thoughtful approach prior to implementing data collection procedures will prevent errors, minimize burden, and lead to successful program implementation. As answers to these five questions become evident, a plan for data collection should be written and serve as the basis for training.