|
|
POLICY AND PROGRAM EVALUATION
Rigorous evaluation of policies and programs can have a powerful influence on program management and improvement. In an environment of tight budgets, evaluation can be a critical component of resource allocation decisions. Nexus Associates specializes in the design and implementation of studies relating to the performance of economic and industrial development programs.
 Program evaluation of Pennsylvania's Industrial Resource Centers.
The state of Pennsylvania commissioned Nexus Associates, Inc. to conduct an evaluation of the economic impact of the program, determine the likely future of manufacturing particularly with respect to small and medium-sized manufacturers (SMMs), and assess the implications for the IRC program in the decade ahead. The analysis is based on panel data for individual manufacturing plants - the Longitudinal Research Dataset (LRD) - maintained by the Center for Economic Studies at the U.S. Bureau of the Census. A technique known as a quasi-experimental design was employed to isolate program impacts from other factors affecting company performance including firm size, geographical location, industry trends and general economic conditions. This comparative analysis provided a solid foundation for estimating the statewide impact of the program in terms of gross state product (GSP), employment and tax revenues between 1988 and 1997 through the use of REMITM, a regional economic model. The study also included an in-depth analysis of the manufacturing sector in Pennsylvania and a Delphi exercise on future trends. The Delphi panel consisted of 23 people, including representatives from manufacturing companies, management consulting firms, industry associations, labor unions, and academia.
 Impact assessment of the Ben Franklin Partnership.
Created to promote technological innovation in order to spur economic growth in Pennsylvania, the Ben Franklin Partnership retained Nexus Associates, Inc. to conduct an independent evaluation of the economic impact of the program. Direct program impacts were estimated using a quasi-experimental design that compared 842 Ben Franklin clients to a statistically similar, individually matched, control group of 2819 companies. Differences between the two groups (client impacts) were then used to model the program's overall economic impact on the state's employment, gross state product (GSP) and tax revenues. Data for the comparison group were drawn from the ES202 database, a collection of employment and wage data from employers participating in government unemployment insurance (UI) programs. The assessment also included eight in-depth case studies, describing the specific role of the BFP program in the success of clients and regional initiatives.
- Comprehensive evaluation of the New York Manufacturing Extension Partnership (MEP).
In response to the needs of the New York Legislature and a new executive administration, Nexus Associates conducted a comprehensive evaluation of the impact of the New York MEP on the state economy. The core analysis was based on a quasi-experimental research design involving MEP clients and a similar group of manufacturers that had not received services. As part of the study, Nexus Associates developed and administered a detailed survey regarding plant-level manufacturing and business performance to a stratified random sample of 275 New York MEP clients. Data for the comparison group was drawn from the performance benchmarking dataset maintained by the Industrial Technology Institute (ITI). Net impacts on gross state product, employment and tax revenues were compared to state investment in the program. In addition, five detailed case studies demonstrated specific program outcomes and how they were achieved.
- Evaluation of the Robert C. Byrd Institute manufacturing programs.
The Robert C. Byrd Institute asked Nexus Associates to evaluate their programs to promote advanced flexible manufacturing in West Virginia. The study involved a comprehensive survey of all manufacturers served by the program to determine its impact on firm performance and employment. A series of in-depth case studies provided additional insights into program services and their impacts.
- Evaluation of flexible manufacturing networks for USNet.
In order to assess the impacts of organized business networking activities and interfirm collaboration, USNet -- a federal-state initiative to foster interfirm collaboration -- asked Nexus Associates to conduct a quantitative evaluation through a survey of network-involved businesses. Together, ninety-nine survey respondents reported $5.5 million in benefits to date from their work with thirteen business networks.
- Assessment of current and prospective tax policies for the City of Providence, Rhode Island.
The Providence Revenue Committee and the Rhode Island Public Expenditure Council contracted Nexus Associates to assess the Providence economy and tax base. Analysis included examination of economic activity in Rhode Island, the property tax base in Providence and adjacent communities, city tax policies, and property tax revenues. The report demonstrated the relationship between economic activity, the tax base, and tax revenues.
- Assessment of the economic impacts of a school of veterinary medicine.
Nexus Associates completed a study entitled "The Impact of Tufts University School of Veterinary Medicine on the Massachusetts Economy" evaluating the regional economic impacts of Tufts Veterinary School. The study assessed both the immediate impacts of Tufts Veterinary School's budgetary expenditures and the spillover effects of its involvement with local biotechnology companies. In each case, the impact of Tufts Veterinary School on employment, sales, personal income, taxes and several additional measures of economic activity in the state was evaluated. The report included the findings of both detailed case studies of local biotechnology firms conducting collaborative research with the school as well as the results of a comprehensive regional input-output econometric model analysis.
- Performance evaluation training workshop for Enterprise Florida.
Enterprise Florida retained Nexus Associates to conduct a training workshop on performance monitoring and evaluation. The training addressed the evaluation needs of four strategic business units -- capital development, technology development, workforce development, and international trade -- to provide participants with an analytical framework and techniques for developing and implementing evaluation plans.
- Preparation of a program evaluation guide for manufacturing extension centers (MECs).
Nexus Associates was hired by NIST MEP to develop a comprehensive guidebook to planning and implementing manufacturing extension program evaluations. The guidebook instructs MEC directors how to formulate key evaluation questions most appropriate to their organizational needs. From there, it defines a series of steps and tools to assist managers in developing measures to address these questions, and in designing and instituting a thorough process to monitor program activity and assess impacts. The guidebook will be distributed to all manufacturing extension centers funded by NIST MEP.
- Evaluation of a Flexible Manufacturing Network.
Under subcontract to the Georgia Institute of Technology School of Public Policy, Nexus Associates developed a survey and managed the collection and analysis of survey data from 204 small businesses and micro-enterprises involved in business networking projects in Ohio.
- Evaluation of the SBDC Technology Assistance Pilot Program.
Nexus Associates was retained by the University of Houston and the National Institute of Standards and Technology (NIST) to conduct an evaluation of a pilot project involving five small business development centers (SBDCs). The evaluation focuses on an assessment of the degree to which clients are satisfied with technical information provided by the centers.
- Evaluation of the Oklahoma Alliance for Manufacturing Excellence.
Nexus Associates worked with the Alliance to formulate a comprehensive evaluation plan. The evaluation was designed to address a variety of organizational issues, particularly the nature of collaboration among various manufacturing extension service providers in the state. It will also determine the extent to which program objectives have been achieved.
- Analysis of NIST MEP cost-benefit survey data.
Nexus Associates is participating in two concurrent projects to assist NIST MEP in the analysis of data gathered from NIST MEP client companies through the U.S. Bureau of the Census. First, Nexus Associates is developing a guide to cost-benefit analysis using data collected through the NIST MEP company follow-up survey. Second, Nexus Associates is conducting preliminary analysis of all data collected by the Census to measure various aspects of service delivery and its financial impacts.
- Development of a bibliography on performance measurement and program evaluation for economic development programs.
Nexus Associates is preparing a performance measurement and program evaluation bibliography for the Economic Development Administration (EDA) of the U.S. Department of Commerce. The document will provide a comprehensive resource for U.S. economic development programs and initiatives that have come under pressure to demonstrate their impact. Nexus Associates staff reviewed literature and evaluation reports from North America and oversaw a literature search in Europe by partner firm Segal Quince Wicksteed. Nexus Associates developed an annotated bibliography of over 200 entries and an introduction summarizing the current state of program evaluation of economic development programs.
- Evaluation of the New York Industrial Technology Extension Service (ITES).
Nexus Associates was hired by the New York Science and Technology Foundation to evaluate the ITES program. The evaluation addressed the ability of the organization to deliver services at an appropriate scale and cost, the extent to which companies implemented changes in their operations due to their involvement in the program, and the extent to which clients outperformed similar companies which had not participated in the program. The evaluation was based on detailed statistical analysis of monthly reports, case studies of selected firms, mail survey of program participants, and an econometric analysis of participants and a constructed control group. The study also included an examination of current monitoring and evaluation practices of other industrial extension service programs in the United States. The results of this evaluation were reported in the journal Research Policy.
- Evaluation of the NIST manufacturing extension centers (MECs).
Nexus Associates was retained by NIST to determine the extent to which MECs have actually improved the performance of small manufacturers. Based on a conceptual model developed by the firm, the evaluation examines the impact of the program on the rate of technology adoption, internal manufacturing performance, overall business performance, labor productivity and employment.
- Evaluation workshops.
Nexus Associates has conducted a series of workshops on evaluation sponsored by the NIST Manufacturing Extension Partnership (MEP). The workshops have been directed toward MEP program staff as well as representatives of organizations funded under the Technology Reinvestment Project (TRP).
- Report on evaluating manufacturing extension programs.
Nexus Associates prepared a report on program evaluation under the auspices of the Modernization Forum, an association for the U.S. manufacturing extension community. The report, "Evaluating Manufacturing Extension Programs: A Tool for Effective Management," provides an overview of evaluation design, reviews basic principles in program monitoring and impact assessment, and discusses the roles of internal and external evaluators in program evaluation.
|
|
