Introduction and Context
Wilson Workforce and Rehabilitation Center (WWRC) is a comprehensive rehabilitation center within a federal Vocational Rehabilitation (VR) system. As such, WWRC shares complex systemic processes in Program Evaluation (PE) and Quality Assurance (QA) common to all VR organizations. WWRC's renewed commitment to continuous improvement in the delivery of VR services, performance, and accountability form the basis of its comprehensive Performance Management System. All components of this system support WWRC's vision, mission, and values. The variety of activities that comprise the Performance Management System complement each other, are inter-related and ultimately are used to develop strategic and tactical plans and to document the organization's performance metric.
The evolution to a formalized organizational culture and infrastructure supportive of a purposeful, holistic, and integrated approach to the measurement and use of data for strategic planning, program decision-making, and process improvements began in 2007 with the creation of WWRC's Organizational Development & Quality Assurance (OD&QA) Division. Prior to that date, WWRC had historically garnered a variety of strategies and tools to measure performance against defined program goals, objectives, and indicators across administrations; however theses strategies and tools were not documented as a consistent written strategy with a clear vision and scope. For the first time beginning in 2007, WWRC gave intentional and simultaneous attention to both Program Evaluation and Quality Assurance, recognizing them as distinct, yet critically interrelated functions necessary to growth and development of the Center.
In 2012, the OD&QA Team published a baseline report to the WWRC Executive Team to document the Center's then-current level of performance vis-à-vis its revised vision, with further opportunities for consideration. One of the recommended opportunities was to adopt a formal, systematic Performance Management System for WWRC, based on the 36th Institute on Rehabilitation Issues (IRI) study titled "Performance Management: Program Evaluation & Quality Assurance in Vocational Rehabilitation". This recommendation was accepted and fully integrated as a major initiative launched under WWRC's Blueprint for Direction.
What is a Performance Management System?
Performance Management is defined in the IRI monograph as a "system that a VR Agency uses to establish its mission, set goals, implement service programs and develop measures to ensure continuous quality improvement" (page 9). Program Evaluation (PE) and Quality Assurance (QA) are operationally defined as distinct, yet critically interrelated components. While PE "focuses specifically on acquiring the data associated with measures, analyzing the data, and reporting results so that appropriate decisions can be made about policy and practice directions", QA "occurs when an agency systematically takes corrective action based on the ongoing flow of information that provides signals as to how well it is performing" (pages 9-10). The two elements represent a balance between an examination of the effectiveness of a rehabilitation program and intentional actions taken to improve and strengthen it.
An effective Performance Management System answers questions such as:
A summary of key system assumptions, components, and anticipated outcomes/impact is contained in a Logic Model for WWRC's Performance Management System. Details are provided in the rest of this Reference Document.
What are the Guiding Principles and Components of WWRC's Performance Management System?
WWRC is committed to being a high performance, data-driven organization reflective of its mission. WWRC's Performance Management System is the organized, purposeful structure and corresponding set of interacting and interdependent PE/QA components through which this vision is realized. WWRC's Performance Management System is based on the following guiding principles:
WWRC's OD&QA Division houses two staff dedicated to WWRC's Performance Management System, one as a shared resource through the Virginia Department for Aging and Rehabilitative Services (DARS) Policy and Planning Unit. The Lead Analyst for Program Evaluation and the Lead Analyst for Quality Assurance work as a team to deliver staff training, provide consultation and technical support, and facilitate an organizational culture that supports an effective Performance Management System. DARS's Policy and Planning Unit and DARS/WWRC Information Technology Services provide consultative and collaborative staff expertise and resource support, as indicated. Related role clarifications and business functions are contained in Section 2 of this Administrative Governance Manual.
Operating together, PE and QA sub-systems provide a consistent framework for monitoring WWRC's performance across Center operations relative to Agency (DARS/WWRC) policies, the federal VR State Plan, WWRC Strategic Plans, and the annual Blueprint for Direction. Sanctioned teams and committees, to include the WWRC Executive Team, WWRC Manager/Supervisor Team, Pegboard Team, and Performance Management Steering Committee, provide routine leadership forums for review of PE/QA findings and recommendations, management level decision-making, and organizational accountability (Appendix 5.2.2). The WWRC Administrative Governance Manual provides detailed protocols, standards, and expectations for effective implementation and maintenance of WWRC's overall Performance Management System. Employee Work Profiles specify standards and expectations specific to individual position responsibilities. A thorough Staff Education, Training, and Development Plan ensures that staff at all levels of the organization understand what a Performance Management System is, what it means in performing daily job responsibilities, and how it benefits our clients and VR sponsors. In addition to these essential components of WWRC's Performance Management System, the OD&QA Team meets monthly with the IS Director to review joint operational issues related to organizational PE and QA functions and to mutually address any identified issues. Similarly, the OD&QA Team meets routinely with the WWRC Director for organizational strategy sessions related to performance accountability and data integrity. Program Evaluation and Quality Assurance, as the core elements of WWRC's overall PM System, are described in more detail below. Program Evaluation
Program Evaluation is a systemic method for collecting, analyzing, and using information for purposes of program planning, service delivery, staff training, and public accountability. Program Evaluation activities allow managers and supervisors within an organization to make reasonable judgments about the effectiveness, adequacy, efficiency, and comparative value of program options. Program Evaluation presents an opportunity to review organizational operations to ensure they are appropriately focused. Program Evaluation also ensures compliance with federal and state requirements governing WWRC operations. Timing is a critical factor when using information derived from a system of PE for management activities. Data must retain enough currency to be useful in deliberations.
Accountable for these functions and outcomes, macro level position responsibilities for WWRC's Lead Analyst for PE include:
The Lead Analyst for PE follows standardized Agency (DARS/WWRC) business processes to extract data from AWARE (the official recognized source for VR client information), import and validate data from offline systems, and create routine and customized reports/dashboards. These standardized Agency business processes, along with integrated satisfaction data, form what is known as the Agency PE Data System. This PE Data System ensures that analyses are based on reliable and valid data, with active management of data anomalies.
A major strategy within WWRC's Program Evaluation sub-system is the formally adopted Business Intelligence (BI) Model. The BI model transforms raw data from AWARE and linked offline databases into meaningful information that is easily accessible and can be used in organization-wide and program level decision-making. BI technologies provide historical, current, and predictive views of business operations and can help determine when current business practices do and do not readily support the reporting of information/data.Quality Assurance
Quality Assurance refers to the systematic monitoring, evaluation, and correction of various aspects of a project or service to ensure that standards of quality are being met. Standards are critical indicators for the achievement of excellence in VR practice and can be used as a benchmark against which to evaluate their practice and performance. Quality Assurance activities present opportunities to drive changes in business practices and policies that result in strengthened program operations and improved VR client outcomes.
Accountable for these functions and outcomes, primary position responsibilities for WWRC's Lead Analyst for QA include:
A major strategy within WWRC's Quality Assurance sub-system is the formally adopted industry-recognized Rummler-Brache Performance Improvement Model. This model is used in combination with Business Intelligence to review critical operational processes that drive WWRC's performance metric. The Lead Analyst for QA also assists with accreditation reviews and preparations, internal case audits, and AWARE support, education, and training.
What Are the Anticipated Benefits of an Effective Performance Management System?
Excerpted from the previously referenced 2011 IRI Study: "Managing the overall performance of your agency is the mainstay of survival in this environment of government accountability, outcomes, and evaluation of cost benefit ratios. Developing sound PE processes gives timely information about your program which can be used over and above established standards and indicators. It can inform you as to whether your vision of what your agency should be and do in terms of outcomes for consumers is being translated into action…… Well implemented QA processes assure the effectiveness of your service delivery system and provide information that can be incorporated into strategic planning… to drive decisions about priorities and the allocation of resources. Information from PE efforts can also be used to advertise success, prove the economic value of the program, and justify requests for increased funding…." (pages 10-11).
If all components of WWRC's Performance Management System are functioning properly the following benefits will be realized:Employee Level