HomePilots & Projects National Demonstration Project NDP Background NDP Data Collection

Data Collection

The instruments used in the evaluation are based on validated measures from the published literature and build on the Center for Research in Family Practice and Primary Care's 14 years of experience conducting similar research and evaluation projects.

Key informant and informal interviews of practice members

A set of 3 interview guides are used to direct interviews with NDP practices. Some of these assessment tools will be used with facilitated practices only and others will be used with both facilitated and self-directed practices. Notes are made by the facilitator conducting the interview. The key stakeholder interview and practice information interview typically are scheduled during the facilitator visit to the practice. The information interviews often occur as conversations as the facilitator observes practice operations.

Key Stakeholder Interview Guide (Guide for formal interviews) – Assesses factors found in the Center's prior research to be important for facilitating practice change.

Practice Interview Guide (Guide for more informal conversations) – Assesses stories important in understanding the practice and the process of change.

Practice Information Form (Structured interview for practice manager or other knowledgeable staff member or clinician) – Assesses key practice features and components of the new model.

Practice Financial Self-Assessment Form (A form to be completed by the office manager and/or lead physician) – Assesses financial status, which is an important outcome and therefore is important for the practice to monitor during any change process.

Practice Genograms

Practice genograms are a visual depiction of the key personnel within and affiliated with a practice and the relationships among these agents. The genogram is very helpful in understanding a practice and is useful to facilitators in guiding the change process. Practices will be asked to supply a staff list and an organizational chart if they have these. The facilitator will use these, the interviews, and practice observations to construct the genograms.

Contact Logs

• Notes that show the date and time of telephone, email or in-person contact with the practice by TransforMED staff.
• Direct observation of practice operations (ethnographic fieldnotes &structured checklists)
• Observation Template (A guide for the facilitator or research nurse, to be used to structure observations and to guide ethnographic fieldnotes.) – Assesses key practice features found to be associated with practice change.
• Practice Environment Checklist (Observational rating scales to be completed by the facilitators and the research nurse.)– Assesses key practice features found to be associated with practice change.

Clinician/Staff Survey

• Clinician Staff Questionnaire – assesses factors found to be important in understanding practices and the process of change.

Medical Record Review

This is currently under development, but will include measures of evidence-based preventive services, disease management and contextual data for interpreting the quality measures.

• AHRQ Quality Measures
• ULTRA Medical Record Review Form

Patient Surveys

Patient Outcomes Surveys will be mailed to a newly selected consecutive sample of 60 patients at each practice on an index day(s) at the beginning, middle and end of the project. Patient-report measures include the following:

• Primary Care Assessment Survey (PCAS) - Assesses seven domains of care through 11 summary scales that have shown strong measurement characteristics and have been associated with patient outcomes and health systems change.
• Components of Primary Care Index (CPCI) - Assesses 10 domains of patient-centered primary care, shows excellent psychometric properties, and has been associated with patient satisfaction, preventive service delivery, chronic illness care, and health care system features and change.
• Patient Enablement Index - Measures the degree to which healthcare helps patients to conduct valued life activities and patient satisfaction, as assessed by the reliable and valid MOS 9-item Visit Rating Scale.

The TransforMED facilitators, qualitative analyst, and research nurse will record their observations and data in secure, password-protected computers and will load these data daily onto a secure FTP server for analysis by the qualitative analyst and the Center evaluation team

Sampling and/or Recruitment Information

The 36 practices already have been selected and randomized. The sample of practice clinicians and staff members completing the anonymous surveys will consist of those who choose to complete and mail back the survey after reading the cover letter and survey. The surveys will be provided to the practice at the beginning, middle and end of the project.

Patient samples at each practice will be selected by the practice manager under guidance from the research nurse. At the beginning, middle and end of the project, (beginning in June, 2006 and approximately 9 and 18 months later) the research nurse, functioning under a business use agreement, will contact the practice manager, asking him/her to provide a list of the names and addresses of 120 consecutive patients seen beginning on a recent index date. Practices also will provide practice stationery. The research nurse or project staff will mail the patient surveys, accompanied by a cover letter and pre-addressed, stamped envelope. The patient names and addresses will be shredded immediately after the mailing is completed. When (s)he visits each practice, the TransforMED research nurse will gather anonymous medical record review data from the charts of the first 60 patients on the list.

Assuming a 50% response rate to the survey, this procedure will result in 3 cross-sectional samples of 60 patients for each practice, to be used to generate practice-level summary data on the care provided at the beginning, middle, and end of the National Demonstration Project