HomePartners & Projects National Demonstration ProjectFAQs About the NDP

FAQs About the NDP

1. What was the goal of the NDP?

The goal of the National Demonstration Project (NDP) was to assess the usefulness and impact on quality of care and business performance of the Future of Family Medicine Report's new model. The new model unites the enduring relationship-centered values of Family Medicine with new technologies and approaches to enable practices to better serve the needs of both patients and practices.

The hypothesis was that the new model's best practices will improve patient care, patient satisfaction, physician satisfaction and business performance. Additionally, it was hypothesized that facilitated practice redesign, in which practices receive tailored consultation and support from experts in the area of redesign and change management may result in faster improved outcomes than self-directed transformation.

2. How many practices were studied?

The 36 selected practices were randomly assigned to one of two groups:

• 18 facilitated practices engaged in a transformative process to fully implement a new model of care
• 18 self directed practices were provided the information to implement the model of care in a self directed manner.
  

3. How were practices selected?

Applications for the NDP were available December 14, 2005 – January 31, 2006. AAFP members were notified about the application by email, via online AAFP publications, and by a mailing to active members and third year residents.  A TransforMED Technical Advisory Committee (TAC) comprised of family physicians and healthcare leaders evaluated the practice applications and recommended practices. A total of 538 applications were received, of which 337 were complete. From this the final 36 were selected for participation in the project.   The practices selected for participation in the National Demonstration Project were publicly announced in April 2006.

4. What types of practices were selected?

The 36 pilot projects included a wide variety of practice types and sizes, reflective of the diversity in family medicine practices across the country. Practices were selected to maximize diversity in a number of areas:

• practice size
• age of the practice
• location
• patient population
• ownership
• arrangement
• revenue
• degree to which the practice may have already implemented components of the TransforMED Model of Care

The selection process also attempted to identify geographic clusters of practices that met the aforementioned criteria to facilitate group learning. All practices chosen to participate demonstrated a commitment to transformative practice change and collaborative learning.

5. How will information learned from the NDP be communicated?

Knowledge gained from this project informs the development of practice enhancement tools and processes to benefit fellow family physicians and their patients across the country. This information also contributes to the literature, and policy and practice work on health care improvement.

Key learnings are synthesized and disseminated on an on-going basis among the demonstration sites and within the larger practice community.

Following the conclusion of the pilot project, the evaluation team will conduct an intensive analysis of the results, summarizing and synthesizing both the data and the insights gleaned from the implementation.

6. How are the NDP practices evaluated?

The Center for Research in Family Medicine and Primary Care, (CRFMPC) has been engaged by TransforMED to oversee the evaluation of the NDP.  The Center for Research in Family Medicine and Primary Care is a consortium of 5 institutions including Case Western Reserve University, Robert Wood Johnson Medical School, Lehigh Valley Hospital, University of Texas Health Science Center and the University of Colorado. 

The Center has been funded by a grant from the American Academy of Family Physicians for more than 10 years. In addition to the evaluation team, the NDP was supported and guided by professional staff from TransforMED that includes facilitators, a research nurse and a qualitative data analyst. The Center evaluators and TransforMED staff are using a variety of validated measures, methods, and data collection instruments in the evaluation process. Following the conclusion of the NDP, the evaluation team will conduct an intensive analysis of the results, summarizing and synthesizing the insights gleaned from the implementation. Key findings on the process and outcomes will be submitted for publication in the peer-reviewed literature.  The Center for Research in Family Medicine and Primary Care has the authority to independently publish both positive and negative findings from this project. A final report is expected in early 2009.

7. What quality measures are included in the study?

Outcomes are evaluated based on nationally recognized metrics, including measures developed by the following initiatives and institutions:

• Ambulatory Quality Alliance
• Nursing Care Quality Initiative
• Institute for Healthcare Improvement
• Medical Group Management Association
• American Medical Association
  

Analyses will use a case study approach to elucidate the process of practice change and to identify key factors that affect the transformation process.

Patient outcomes to be assessed include the quality of chronic disease management, preventive service delivery, acute illness care and mental health care, patient satisfaction and patient-centeredness of care.

Practice outcomes include clinician and staff quality of life, practice processes, and financial viability. Key learnings will be synthesized and disseminated on an on-going basis among the demonstration sites and within the larger practice community.

8. Will the quality measures include the ACQA Starter Set?

Yes, the quality assessment will include the list of measures identified as the recommended starter set.  This starter set is a proposed set of measures for ambulatory care that align with agreed-upon parameters and address agreed-upon specific conditions/areas.  These identified measures are based on their ability to meet the following criteria:

• Clinical importance and scientific validity
• Feasibility
• Relevance to physician performance
• Consumer relevance
• Purchaser relevance

While these are a sound set of measures that meet primary goals such as addressing the IOM's priority areas, they are continually recognized as an initial step in a multi-year process.  The group recommending these measures recognizes that additional work needs to be done to build a more complete set of measures which includes additional efficiency measures, sub-specialty measures, cross-cutting measures, patient experience measures and others.

9. Does the evaluation include other quality measures besides the starter set?

The NDP Quality Assessment  includes 5 new depression measures developed specifically for this study:

• Depression diagnosis criteria met – acute phase
• Recommended follow-up – acute phase
• Regular follow-up documented – continuation phase
• Clinical course and/or medication adjustment documented – continuation phase
• Addressing the diagnosis in previous year – chronic care phase

10. What is the goal of the evaluation?

Although TransforMED has used the NDP to study practice metrics issues related to components of the model such as open access scheduling, the main purpose of the overall Evaluation of the NDP is to study:

• The process of practice change and to identify key factors that affect the transformation process
• Patient outcomes to be assessed include the quality of chronic disease management, preventive service delivery, acute illness care and mental health care, patient satisfaction and patient-centeredness of care
• Practice outcomes include clinician and staff quality of life, practice processes, and financial viability.

11. Will I be able to use the study results to determine if my practice should take on open access scheduling?

In order to conduct scientifically valid research, it is necessary to first identify what you are studying.  As the illustration below shows, if you are studying a patient that has coronary artery disease, you would isolate interventions and only study one intervention at time, such as exercise, in order to determine its effectiveness on the condition.  If we look at the medical practices in the NDP with the same lens, they have implemented several interventions all at the same time – from chronic disease management, e-visits, group visits, improved practice management, to open access scheduling.  The evaluation is studying how implementing the entire model of care affects the practice and its patients.  It would be similar to taking the same patient illustrated below and studying how a change in medicine to lower cholesterol, a change in medicine to lower blood pressure, a change in diet, a change in exercise, and the addition of a daily aspirin tablet affect the patient's condition.  If the patient's condition improves, it is impossible after the study to determine if it was the medicine or the change in diet that caused the improvement.  Likewise, because the NDP practices are implementing all aspects of the new model during the 24-month study, we will be able to speculate how total organizational transformation, such as incurred by these practices, may affect practices outside of the NDP.

So, the NDP studies the effect the entire model has on a practice, not just one component.  TransforMED is using this data to begin a broad-range transformation effort with practices outside the NDP. Through continued work with additional practices, TransforMED will better be able to identify how certain components of the model effects various types of practices.  In the meantime, the NDP is the first study of its kind in the US healthcare system that is studying such broad practice redesign efforts.

12. How are the Evaluators collecting the data?

The instruments used in the evaluation were based on validated measures from the published literature and build on the Center's 14 years of experience conducting similar research and evaluation projects.

A set of 3 interview guides were used to direct interviews with NDP practices. Some of these assessment tools were used with facilitated practices only and others were used with both facilitated and self-directed practices. Notes were made by the facilitator conducting the interview. The key stakeholder interview and practice information interview typically were scheduled during the facilitator visit to the practice. The information interviews often occurred as conversations when the facilitator observed practice operations.

Practice genograms, a visual depiction of the key personnel within and affiliated with a practice and the relationships among these agents, was also developed. The genogram is very helpful in understanding a practice and is useful to facilitators in guiding the change process. Practices were asked to supply a staff list and an organizational chart. The facilitator used these, the interviews, and practice observations to construct the genograms.

Medical Record Review: The TransforMED Research Nurse visited each NDP practice 3 times throughout the study, pulling charts to assess measures of evidence-based preventive services, disease management and contextual data for interpreting the quality measures.  These included:

• AHRQ Quality Measures
• ULTRA Medical Record Review Form

Patient Outcomes Surveys were mailed to a newly selected consecutive sample of 60 patients at each practice on an index day(s) at the beginning, middle and end of the project. Patient-report measures included the following:

• Primary Care Assessment Survey (PCAS) - Assesses seven domains of care through 11 summary scales that have shown strong measurement characteristics and have been associated with patient outcomes and health systems change.
• Components of Primary Care Index (CPCI) - Assesses 10 domains of patient-centered primary care, shows excellent psychometric properties, and has been associated with patient satisfaction, preventive service delivery, chronic illness care, and health care system features and change.
• Patient Enablement Index - Measures the degree to which healthcare helps patients to conduct valued life activities and patient satisfaction, as assessed by the reliable and valid MOS 9-item Visit Rating Scale.

The TransforMED facilitators, qualitative analyst, and research nurse recorded their observations and data in secure, password-protected computers and loaded these data daily onto a secure FTP server for analysis by the qualitative analyst and the Center evaluation team.