THE GALLUP ORGANIZATION

for
U.S. Department of Transportation
Federal Highway Administration (FHWA)

FHWA Study — Results
Abridged Final Report

Implementing Performance Measurement in Environmental Streamlining

Submitted to:

Federal Highway Administration
Planning and Environment, 3222 HEPE
400 7th Street, SW
Washington, D.C. 20590

January 2004

Prepared by:

Alison Simon, Ph.D., AICP
Sameer Abraham, Ph.D.
Calvin Jones, Ph.D.
The Gallup Organization
Government Division
901 F Street, NW
Washington, D.C. 20004

FOREWORD

This report is the culmination of a two year effort to establish a baseline assessment of the Federal Highway Administration environmental streamlining program. FHWA is pleased to sponsor this work as an outcome of the streamlining efforts mandated by the Transportation Equity Act for the 21st Century (TEA-21) of 1998.

FHWA intends to publish excerpts and reports from this document over the next few months on their Internet site. We encourage individuals to keep track of these publications on our website at http://www.fhwa.dot.gov.

ACKNOWLEDGEMENTS

The authors would like to thank those who contributed to the production of this report: Kreig Larson who served as the government project officer, and Fred Skaer, Director of Project Development and Environmental Review Office, who provided guidance and support throughout the process.

The National Technical Review Board also provided feedback and encouragement throughout the process. This was crucial to ensuring the long-term feasibility of the project. The board members:

Ms. Janine Bauer, Coalition to Defend NEPA
Mr. John Devierno, Montana Department of Transportation
Ms. Robin Elliott, U.S. Fish & Wildlife Service
Mr. Marlin Gottschalk, Senior Policy Advisor, Georgia Department of Natural Resources
Ms. Dorothy Guzzo, Deputy Historic Preservation Officer, New Jersey State Historic Preservation Office
Ms. Julie Hunkins, North Carolina Department of Transportation
Mr. Hal Kassof, Vice President, Highway Program Area Manager, Parsons Brinckerhoff
Ms. Susan McDonald, Pennsylvania Department of Transportation
Mr. Gary McVoy, Director of Transportation Maintenance, New York State Department of Transportation
Ms. Anne Miller, Director, Office of Federal Activities, Environmental Protection Agency
Mr. Sam Morigeau, U.S. Department of Agriculture
Mr. Joe Ossi, Federal Transit Administration
Ms. Nancy Schamu, National Conference of State Historic Preservation Officers
Mr. Vaughn Stokes, Director of Engineering, U.S. Department of Agriculture
Ms. Gina Whitehill-Baziuk, Metro
Ms. Sandy Straehl, Chief, Programs of Policy & Analysis, Montana Department of Transportation
Mr. Benjamin Tuggle, Chief, Division of Habitat Conservation, U.S. Fish & Wildlife Service
Ms. Janet D'Ignazio, Chief Planning & Environmental Officer, North Carolina Department of Transportation
Mr. Michael Ryan, Deputy Secretary for Highway Administration, Pennsylvania Department of Transportation
Ms. Susan Borinsky, Director, Office of Human & Environment, Federal Transit Administration
Mr. Richard Brandman, Metro
Mr. Brent Felker, Chief Engineer, California Department of Transportation
Mr. Gary Gallegos, Executive Director, San Diego Association of Governments
Mr. Doug Allen, Dallas Area Rapid Transit
Ms. Dinah Bear, General Counsel, Council on Environmental Quality
Mr. John Studt, Chief, Regulatory Branch, Corps of Engineers
Ms. Bambi Kraus, President, National Association of Tribal HPO Officers
Mr. Nicholas Mpras, Chief, Office of Bridge Administration, U.S. Coast Guard
Mr. Jonathan Deason, American Road and Transportation Builders Association
Ms. Sara Campbell, Surface Transportation Policy Project
Mr. Dave Burwell, Surface Transportation Policy Project
Ms. Nancy Jakowitsch, Director, Smart Growth & International Programs, Surface Transportation Policy Project
Ms. Patricia White, Transportation Associate, Defenders of Wildlife

Contents

• Executive Summary
• Overview
• Research Objectives and Study Design
• Specifying the Population of Interest
• Data Collection
• Follow-Up Procedures
• The 2003 FHWA Survey of Transportation Agencies and Resource-Permitting Organizations
• Report of Major Findings
• Recommendations for Management

Exhibits & Charts

Exhibit 1. Distribution of Reviewers and Managers by Type of Organization

Exhibit 2. Recency of Experience of Reviewers with NEPA Projects by Type of Organization

Exhibit 3. Types of Projects Evaluated by Reviewers by Type of Organization

Exhibit 4. Comparison of Mean (Average) Satisfaction Scale Scores Given by Transportation and Resource Reviewers to Their Named Counterpart Organizations

Exhibit 5. Stage of NEPA Project at which Problems Reported by Reviewers Occurred

Exhibit 6. Problems Experienced by Reviewers on NEPA Projects by Type of Organization

Exhibit 7. Reviewers' Perceptions of the Relationships, Communications, and Timeliness of Interactions Between Transportation and Resource-Permitting Organizations

Exhibit 8. Ratings by Reviewers of the Performance of Named Counterpart Organizations on Cited NEPA Projects

Exhibit 9. Perceptions of Managers and Reviewers of the General Performance of Named Counterpart Organizations on all NEPA Projects on Which They Have Interacted

Exhibit 10. Comparison of Mean (Average) Agree/Disagree Scale Scores for Counterpart Organization Ratings

Exhibit 11. Overall Evaluations by Transportation and Resource Officials of the Current Relationship with Their Named Counterpart Organizations

Exhibit 12. Change in Relationships with Their Named Counterpart Organizations Perceived by Transportation and Resource Officials during the Last Three Years

Chart 1. Overall Satisfaction Expressed by Transportation Reviewers with the Performance of Named Counterpart Organizations on the Cited NEPA Project

Chart 2. Overall Satisfaction Expressed by Resource Reviewers with the Performance of Named Counterpart Organizations on the Cited NEPA Project

Chart 3. Transportation Reviewers: Problem Occurrence and Corresponding Levels of Satisfaction with the Performance of Named Counterpart Organizations on the Cited NEPA Project

Chart 4. Resource Reviewers: Problem Occurrence and Corresponding Levels of Satisfaction with the Performance of Named Counterpart Organizations on the Cited NEPA Project

Executive Summary

During 2003, the Federal Highway Administration (FHWA) contracted with The Gallup Organization to conduct a nationwide survey of the perceptions of stakeholders and participants in the environmental streamlining process. Telephone interviews were conducted with over 700 senior officials in Federal, State, and local government transportation agencies and with more than 600 senior officials in both government and private resource-permitting organizations. All survey participants were either agency managers or senior technical staff directly involved in the completion of environmental review of large scale transportation projects under the requirements of the National Environmental Policy Act of 1969, as amended (NEPA).

Survey questions focused on the views of managers and technical staff in both types of organizations concerning their overall relationships with counterpart organizations. Technical staff members in both types of agencies were also asked about their assessments of the performance of counterpart organizations on recently completed transportation projects.

The primary intent of the survey was to differentiate between the aspects of the relationships and interactions between transportation agencies and resource organizations that are working smoothly and productively and those that are viewed as needing improvement to enhance the efficiency of the NEPA process. A second key goal was to establish a baseline measure of stakeholder perceptions against which data from future surveys might be compared to assess progress in inter-agency relationships and performance.

The Technical Reviewers' Perspective

Technical staff members who served as Reviewers on recently completed NEPA projects were asked a series of 40 questions about specific dimensions of their experiences on a recently completed NEPA project on which they had the most interaction with counterpart organizations. (Transportation agency Reviewers were asked for their perceptions about resource-permitting organizations and vice versa.) Technical Reviewers were also asked for an overall assessment of their satisfaction with the performance of their designated counterpart organization on all aspects of that project.

Reviewers in both transportation and resources organizations answered most of the survey questions about project interactions in very similar ways. The pattern of responses to the question about overall satisfaction (shown below) typifies the results for most of the questions about project interactions.

Approximately two-thirds of the Reviewers in both transportation agencies (67 percent) and resource organizations (64 percent) reported that they were either satisfied or very satisfied with the performance of their counterparts on the NEPA project. One fifth (20 percent) of the Reviewers in both types of organizations gave a neutral response (neither satisfied nor dissatisfied). Only one eighth (12 percent) of Reviewers in transportation agencies and one-sixth (16 percent) of those in resource organization evinced dissatisfaction or strong dissatisfaction with their counterparts' performance.

The responses to the overall satisfaction question correlated strongly with answers to another question asking Reviewers to report on any problems experienced on the recently completed NEPA project they cited — just over one-third of both types of Reviewers reported having problems with their named counterpart organization. Reviewers who reported problems were far more likely than those who did not report problems to indicate that they were dissatisfied with their counterparts' performance or to give a neutral response to the overall satisfaction question.

Scope of Problems Raised

Of the transportation Reviewers who reported problems with their resource organization counterparts, nearly two-thirds indicated that the problems occurred during the analysis of impacts and during the finalizing documents or responses to comments on their NEPA projects. About half indicated that problems arose during the process of obtaining commitment to mitigation measures.

Of the resource Reviewers who reported problems with their transportation counterparts, about two-thirds said that the problems occurred during the analysis of impacts (mirroring the views of transportation Reviewers), and also during the collection of data and information. Over half of resource Reviewers also reported problems during the finalizing of documents or responses to comments (also matching reports by transportation Reviewers) and the development and analysis of alternatives.

The most common type of problem cited by transportation Reviewers was "lack of timely response" by the counterpart resource agency, mentioned by 21 percent. Resource Reviewers gave even more varied descriptions of project problems, with 11 to 15 percent citing "poor communication," "poor coordination," "receiving wrong or incomplete information," as well as "lack of timely response."

Response Patterns of Project Reviewers to Survey Questions About Agency Interactions and Relationships

Response patterns expressed by both types of Reviewers on several dimensions of agency interactions on the recently completed NEPA project closely resembled the distribution of answers to the "overall satisfaction" question, including:

• 5 of 8 questions about the tenor of the relationship between the counterpart organizations on the NEPA project, specifically:
  • whether the counterpart organization provided materials, information, or documentation that the respondent's agency needed;
  • whether the contribution of the respondent's organization was appreciated by the counterpart agency;
  • whether the counterpart organization wanted to play an active role in the NEPA project;
  • whether the respondent's agency's opinions seem to count in the process; and
  • whether the counterpart organization felt the mission of the project was important;

• 6 of 11 survey items on the quality and character of communications with counterpart organizations on the NEPA project, namely:
  • whether the counterpart organization involved the respondent's agency early in the project;
  • whether the counterpart organization responded in a timely way to the respondent agency's requests;
  • whether the counterpart organization was open and honest with the respondent's agency;
  • whether the counterpart organization was open to suggestions and alternatives;
  • whether the counterpart organization gave clear explanations if they did not agree with the respondent agency's proposals; and
  • whether there was a sufficient level of communication between the two agencies on the project.

• 3 of 4 questions about the timeliness of performance by the counterpart organization, including:
  • whether the counterpart organization adhered to schedules throughout the project;
  • whether the counterpart organization gave the respondent's agency enough time to accomplish tasks; and
  • whether the entire process took a reasonable amount of time.

Differences in Opinions Between Reviewers

A number of noteworthy exceptions to the response pattern described above were found among the evaluations given by technical review staff in each of the three aspects of project interactions. These exceptions include some noteworthy differences in opinions on key issues:

• On inter-agency relationships:
  • Nearly all Reviewers of both groups indicated that their own agencies understood what was expected of them on the NEPA project;
  • Resource organization Reviewers were substantially more likely than transportation agency Reviewers to agree that the counterpart organization helped to move the project forward (75 percent for resource Reviewers vs. only 56 percent for transportation Reviewers);
  • Transportation Reviewers were much less likely than resource Reviewers to agree that their counterpart organization made efforts to improve the process during the NEPA project (36 percent for transportation Reviewers vs. 57 percent for resource Reviewers).

• With respect to inter-agency communications:
  • resource Reviewers were considerably more likely than transportation Reviewers to agree that the counterpart organization invited their participation in key meetings (71 percent for resource Reviewers vs. 54 percent for transportation Reviewers);
  • transportation Reviewers were far less likely than resource Reviewers to agree that the counterpart organization kept them informed about their progress (41 percent of transportation Reviewers vs. 58 percent of resource Reviewers);
  • both groups of Reviewers were less likely to agree that their counterpart organizations
    • gave reasonable suggestions or alternatives on the NEPA project; or
    • were willing to compromise on the project;
  • resource Reviewers were substantially more likely than transportation Reviewers to agree that their counterpart organization had adequate participation in key meetings (84 percent for resource Reviewers vs. 65 percent for transportation Reviewers);

• For the questions about timeliness:
  • resource Reviewers were far less likely than transportation Reviewers to agree that the project could have been shortened without compromising the intent of NEPA (35 percent for resource Reviewers vs. 55 percent for transportation Reviewers).

Responses to Survey Questions about Agency Performance

Transportation and resource Reviewers were asked 8 additional questions about the characteristics and performance of their counterpart agencies on the recently completed NEPA project. On 5 of the 8 questions, the response patterns from both groups of reviewers remained similar to and highly correlated with the answers they gave to the overall satisfaction question (noted in Table I), including satisfaction with:

• the quality of information provided by the counterpart organization;
• the completeness of information provided by the counterpart organization;
• the counterpart organization's ability to stay organized throughout the project;
• the counterpart organization's understanding of the respondent agency's mission; and
• the counterpart organization's willingness to consider a range of mitigation measures.

Significantly different response patterns were found on 3 of the survey questions:

• Reviewers from both groups of organizations were much more likely than average to indicate satisfaction with the competence of the counterpart agency staff they interacted with;
• Reviewers from both groups of organizations were much less likely than average to indicate satisfaction with the range of reasonable alternatives their counterparts suggested for the project;
• Resource Reviewers were much more likely than transportation Reviewers to express satisfaction with the level of resources the counterpart organization devoted to the project (70 percent of resource Reviewers vs. only 48 percent of transportation Reviewers).

When asked to report their satisfaction with the job done by their transportation counterparts in protecting the environment, the response pattern was nearly identical to that given to the overall satisfaction question described above (see Table I).

The Perspective of Agency Managers and Reviewers on General Agency Relationships

Senior Managers and technical Reviewers in both transportation agencies and resource organizations were asked to make an additional series of 8 general evaluations of their named counterpart organizations — these evaluations were to be broader than those provided to the preceding questions about the recently completed NEPA project. These high-level evaluation questions asked whether:

1. the counterpart organization understands your agency's mission;
2. the counterpart organization cares about your agency's mission;
3. the counterpart organization is committed to doing quality work;
4. the counterpart organization has competent staff;
5. there is a sufficient level of trust between your two agencies;
6. the counterpart organization is committed to making the environmental review process run efficiently;
7. the counterpart organization is willing to compromise; and
8. there is a sufficient level of communication between your two agencies.

In addition, transportation Managers and Reviewers were asked whether their resource counterpart organization is committed to transportation improvements, while resource Managers and Reviewers were asked whether their transportation counterpart organization is committed to protecting the environment.

For items numbered 1, 3, 4, and 8 in the preceding list, most groups of respondents exhibited answer patterns that closely matched the distributions observed for the "overall satisfaction" question (shown in Table I). However, on the first of these questions (whether the counterpart understands the respondent's agency's mission), transportation Managers were much less likely than the other three types to express agreement (only 48 percent agreed or strongly agreed).

Response patterns to the remaining four items (2, 5, 6, and 7 in the list above) were considerably different from the overall satisfaction pattern (see Table I). In general, all four groups of respondents were less likely to agree with these statements about inter-agency relationships. Transportation Managers were more likely to Disagree (38 percent) than to agree (33 percent) with the statement that their counterpart resource organizations cared about their own agency's mission (item 2 above). Well under half of the Reviewers (39 percent of transportation, 44 percent of resource) agreed with this statement.

Fewer than half of the transportation officials agreed that there was a sufficient level of trust between their agencies and their resource counterparts (item 5 above). Just over half of the resource officials agreed with this statement. Nearly identical response patterns were observed for item 6, focusing on the commitment of the counterpart organization to making the environmental review process run efficiently, and item 7, about whether the counterpart organizations is willing to compromise. On the latter item, more transportation Managers expressed Disagreement (35 percent) than agreement (28 percent), similar to the pattern observed on item 2 above.

Transportation officials, especially Managers, gave significantly more negative responses than did resource officials to the two parallel questions about commitment to their own agency's priorities. Only 26 percent of transportation Managers agreed that their counterpart resource organizations were committed to transportation improvements, while 44 percent Disagreed. Transportation Reviewers were equally divided between agreement, Disagreement, and the neutral (neither agree nor Disagree) response.

In contrast, nearly half of resource Managers (49 percent) agreed that their transportation counterparts were committed to protecting the environment and far fewer (19 percent) Disagreed. The corresponding figures for resource Reviewers were 44 percent and 24 percent.

Characterization of the Quality of Interagency Relationships

Finally, all four groups of officials were asked to characterize the quality of the overall relationship with their named counterpart organizations on a 5-point scale (where 1 = "poor" and 5 = "excellent"). With the exception of transportation Managers, the officials all gave similar responses: 61 to 63 percent giving scores of "very good" or "excellent," 30 to 32 percent giving a neutral score of "3," and only 1 to 3 percent giving scores of "poor." However, only 47 percent of transportation Managers indicated scores of "4" or "5" on this item, repeating a pattern found on the more specific evaluation items.

Most officials involved in the environmental streamlining process expressed belief that their agency's relationship with their counterpart organizations have improved over the last three years (ranging from 49 percent for transportation Reviewers to 59 percent for resource Reviewers. However, just over one-third of the officials (33 to 38 percent) indicated that the relationship had not changed over the past three years. A small but significant minority (14 to 16 percent of transportation officials, and only 7 percent of resource officials) reported believing that the relationships had gotten worse over the past three years.

Overview

Environmentally responsible transportation improvements, delivered on time and within budget, is a simple vision that all too often evades the U.S. Department of Transportation (DOT) and its partner agencies. Public expectations and demands for effective transportation solutions on a reasonable time frame are understandable, given the magnitude and pervasiveness of America's highway transportation problems. Equally understandable is the public's desire for environmentally sound ways of providing for sustainable transport solutions. Inevitably, these two societal goals occasionally come into conflict.

The term "Environmental Streamlining" describes the effort to combine the timely delivery of transportation projects with the protection and enhancement of the environment. First enacted into legislation for highway and transit projects with the Transportation Equity Act for the 21st Century (TEA-21), Environmental Streamlining requires transportation and environmental agencies to cooperate in developing realistic project time frames, and then work collaboratively to adhere to that schedule. Because major transportation projects are affected by dozens of Federal, State, and local environmental requirements administered by a multitude of agencies, improved inter-agency cooperation is critical to the success of environmental streamlining.

Efforts currently underway within the Federal Highway Administration (FHWA) of the U.S. DOT focus on solidifying the inter-agency partnership through a series of actions that include pilot efforts, process reinvention, alternative dispute resolution, and a focus on process and outcome evaluation. Experience in implementing environmental streamlining will lead to critical policy choices that may point to the need for revisions to transportation or environmental laws or regulations.

The objectives of the environmental streamlining provisions, as described in the explanatory materials on TEA-21, prepared by the Committee on Transportation and Infrastructure, are as follows:

• to establish an integrated review and permit process that identifies key decision points and potential conflicts as early as possible,
• to integrate the National Environmental Protection Act (NEPA) process into each project as early as possible,
• to encourage full and early participation by all relevant agencies that must review a highway construction project or issue a permit, license, approval, or opinion relating to the project, and
• to establish coordinated time schedules for agencies to act on a project.

As part of its response to the charge of Section 1309 for streamlining environmental reviews of proposed transportation projects, FHWA is investigating both qualitative and quantitative data sources aimed at creating sound, actionable performance measures for the project development process. These include:

1. A study focusing on establishing an historic baseline for assessing time frames for project development is currently underway. The goal of this research is to establish a baseline against which to assess project delays over time; to attempt to identify causes for such delays (reviews, applications for permits, etc.); and to assess through quantitative statistical analyses the trends and determinants of delays. The baseline study is examining those transportation projects for which Environmental Impact Statements (EIS) were prepared.
2. FHWA, in conjunction with the Center for Transportation and the Environment (CTE), has proposed a series of informal interviews with Federal agencies as well as a limited number of non-Federal agencies/entities for the purpose of assessing the perspectives of and attitudes towards environmental streamlining that are held by those involved in the delivery and environmental review of transportation projects. The interviews will be designed to gather background information on the target stakeholders' perceived roles, expectations, problems, and obstacles (e.g., differing interpretation of rules) with regard to streamlining the project development process. This background information is expected to contribute to the development of performance measures and indicators to assess the effectiveness of environmental streamlining.
3. The Transportation Committee of the American Consulting Engineers Council (ACEC) proposed conducting a survey to measure the performance of stakeholders in the transportation project development process (i.e., resource and transportation agencies). This study would measure perceptions of key participants in the project development process, and, by means of applying scientifically reliable and valid survey methods, explore how stakeholders in the process view the quality of the environmental work and services performed by their counterparts. The survey will include questions referring to:
   • Whether staff members of resource agencies believe that State Transportation agencies are submitting thorough documentation in fulfillment of permit application requirements
   • Whether staff members working in State Departments of Transportation (DOT) perceive that resource agencies are providing timely and constructive responses to the permit process.
   Findings from this survey will enable both parties identify potential problems and to improve inter-agency communication throughout the process.

Research Objectives and Study Design

The Gallup Organization was commissioned to design and conduct the third study listed above with the objectives of:

1. Measuring the performance of agencies involved in environmental streamlining in order to provide a benchmark for agencies to gauge their own performance and that of the process itself.
2. Identifying aspects of the process that represent opportunities for improvement.

Questionnaire Development

Gallup's research team developed initial drafts of both the resource and transportation agency questionnaires. The project began in the second half of 2001, with Gallup conducting qualitative focus group sessions in Chicago, Denver, Atlanta, Boston, Dallas, and San Francisco¹ . Using the results received from the qualitative findings, a draft survey questionnaire was developed and tested in cognitive process interviews conducted through a second series of focus groups. This approach permitted evaluation of multiple versions of the survey questions before finalizing the instrument.

¹ FHWA: Environmental Streamlining and the Development of Performance Measures: A Detailed Analysis of Focus Group Findings. The Gallup Organization, January 11, 2002.

Responses from focus group participants revealed common themes concerning relationships among the counterpart transportation and resource-permitting agencies. For that reason, two sets of questions were developed:

• items focusing on broad aspects of the relationship between the agencies that develop and extend over considerable time, and
• items focusing on specific interactions on a particular project, emphasizing activities over which the agencies are believed to have considerable management discretion and control.

Resource Agency and Transportation Agency Questionnaires

Two questionnaires were designed for use in this study, one intended for transportation officials and the other for resource agency officials. The questionnaires were almost identical in their layout.

Both questionnaires contained four sections organized around the following topics:

1. 16 screener questions to identify eligible respondents, where eligibility is defined as having recent experience reviewing and acting on NEPA documents or managing the review process for NEPA projects,
2. 36 project-specific ratings on several dimensions of the quality of the process and the performance of counterpart agencies (reviewers in resource-permitting agencies rated the performance of transportation agencies and vice versa). These included a rating of the respondent's overall satisfaction with the project, 4 questions on problem incidence, 7 general relationship evaluation questions, 10 communication questions, 5 timeliness questions, and 9 general project performance questions.
3. 11 overall agency ratings not focusing on any specific NEPA project, including 9 questions about the interagency relationship and two questions about the agency's general performance.
4. 4 demographic questions about the respondent.

Using questionnaires developed from focus group findings, a pilot study was conducted in DOT Region 5 to assess both the instruments and procedures for identifying qualified respondents.² Based on findings from the pilot test, the questionnaires were modified slightly prior to conducting the full-scale survey in the remaining nine DOT regions using Computer Assisted Telephone Interviewing (CATI) technology.

To accommodate respondents who wished to respond to the survey but were not able to schedule a telephone interview during the survey period, the CATI questionnaires were adapted for self-administration using an electronic survey form on the World Wide Web (the CATI and Web questionnaires were identical in content). Please contact Kreig (Chip) Larson at 202-494-2056 or kreig.larson@fhwa.dot.gov to receive a copy of these questionnaires.

² FHWA: Environmental Streamlining and the Development of Performance Measures: FHWA Study - Pilot Study Results. The Gallup Organization, October, 2002.

Specifying the Population of Interest

Identifying Eligible Respondents in Transportation and Resource Agencies

Because no list of environmental streamlining participants existed, the list of potential survey respondents was constructed using several methods. First, FHWA and Gallup assembled a comprehensive inventory of agencies and organizations that are routinely involved in environmental streamlining activities on transportation projects. Gallup staff then located telephone numbers from a variety of information resources, and called the organizations to obtain the names of managers and reviewers involved in transportation projects. Members from the National Technical Review Board also supplied names and telephone numbers for likely sample members. Following quality control checks for accuracy, the initial list of names and telephone numbers was entered into the CATI scheduling and management system and the survey was initiated.

The initial segment of each survey call consisted of a series of screening questions designed:

• to confirm that staff of the contacted organization had been directly involved in the environmental streamlining process during the past six months, and especially within the last 90 days,
• to determine whether individuals being called were eligible for the survey by virtue of their specific roles in the environmental streamlining process, and
• to use the initial contacts at the organization as informants to identify all other persons in the same organization who were also eligible to be interviewed as a result of their roles and duties on transportation projects that included environmental streamlining efforts.

The list of agencies contacted for the transportation survey included:

• Federal Highway Administration
• Federal Transit Authority
• State Departments of Transportation
• Federal Aviation Administration
• Mass Transit Agencies
• City and/or County Transportation Authorities

The list of agencies contacted for the resource survey included:

• National Park Service
• National Forest Service
• Fish and Wildlife Service
• Bureau of Land Management
• National Marine Fisheries
• Environmental Protection Agency
• U.S. Army Corps of Engineers
• Federal Lands Program
• U.S. Coast Guard
• State Department of Natural Resources/Department of Environment
• State Historic Preservation Organizations

Respondents who completed the interview were asked if they could identify others in their agency performing similar work who would thus qualify to be interviewed. If so, additional names and telephone numbers were recorded and these potential respondents were called to confirm eligibility and conduct the interview. Using this "networking" approach, we were able to identify the vast majority of managers, program reviewers, and other technical staff who had direct experience with transportation projects during the six-month period preceding the interview date.

Data Collection

The CATI survey was conducted from Gallup's Executive Telephone Interviewing Center in Lincoln, Nebraska. The self-administered electronic questionnaire was programmed and operated on Gallup's secure Web servers.

Selection and Training of Telephone Interviewers

Five Executive Interviewers were selected based upon their prior experience interviewing similar types of professionals using similar screening methods and survey questions. Gallup survey managers produced a training manual to provide project-specific guidelines for the screening, interviewing, and "networking" procedures to identify additional eligible respondents.

Training sessions were conducted in Lincoln, Nebraska, with a videoconference from Washington, D.C. attended by Gallup and FHWA project staff for the pilot study. Interviewers were trained for three hours on the use of the FHWA questionnaires, techniques for understanding screening procedures, and a review of standard responses to commonly asked questions associated with the questionnaires. The training materials included a glossary of acronyms for agencies that interviewers would contact, and detailed specifications for each of the survey questions.

Survey Notification Letters

Prior to contact by telephone, all prospective survey respondents were sent notification letters signed by the FHWA Administrator explaining the purpose of the survey and asking for their cooperation. A copy of the Administrator's letter is attached to this report.

CATI Interviews

Telephone interviewing began on October 28, 2002, and continued through April 18, 2003 for officials in transportation agencies and through May 16, 2003, for those in resource organizations. Interviewing was temporarily suspended from November 1, 2002, through February 1, 2003, in order to obtain an extension of the expiration date for the forms clearance issued by the Office of Management and Budget.

Web-based Self-Administered Interviews

To maximize response rates, a Web-based, self-administered electronic form was developed to provide respondents with an alternative mode to telephone interviewing. The Web-based form was made available to respondents from November 10, 2002, through the conclusion of the CATI survey period. In total, 92 individuals completed the self-administered questionnaire rather than the CATI interview. The availability of the Web-based survey form increased the overall response rate by four percentage points (from 58 percent to 63 percent).

Web responses by region were examined, and in only one region, Region 10, did the number of Web responses go above 15% of total responses (Web responses in Region 10 were 20% of the total transportation sample). However, in the data, Region 10 does not show up as a region with extensive difficulties, and for that reason the modal effects are not considered a hindrance to the results. It is something that needs to be carefully considered in the future and among future survey work done for this group of respondents.

Follow-Up Procedures

As many as seven calls were made on different days of the week and during different hours of the day in order to contact each potential respondent. Once contact was made, as many as seven follow-up calls were made in an attempt to complete an interview with that person. When an answering machine or voice mail was encountered, interviewers left a message that included a toll-free telephone number encouraging individuals to call Gallup at any time they found convenient. Numbers found to be non-working or otherwise incorrect were checked for accuracy and updated by contacting the resource organization directly.

Interviewers attempted to contact a total of 1,596 officials in resource organizations and 1,171 officials in transportation agencies. The table below shows the final disposition of all cases included in the final list of potential respondents. A total of 649 of the officials contacted were ineligible to be surveyed (80 in transportation agencies and 583 in resource organizations). Survey data were obtained from a total of 709 transportation officials and 608 resource officials for an overall response rate of 63 percent (65 percent among transportation officials and 60 percent among resource officials).

These completion rates are based on the most conservative assumptions about the eligibility — namely that all the individuals who were not contacted would have been eligible to be interviewed. However, among the cases that were contacted, we determined that 80 of 1,171 transportation officials we contacted and screened were ineligible (7.0 percent), and 583 of the 1,596 resource officials we screened were also ineligible (37.0 percent). Applying these ineligibility rates to the non-contacted cases, the adjusted response rates would be 66 percent for transportation officials and 70 percent for resource officials, and an overall adjusted rate of 68 percent.

Only 47 of the nearly 2,800 individuals we attempted to contact refused to participate in this voluntary survey.

Final Completion Rates

	Transportation	Resource	Total	Combined % of Total
Total Contact/Screening Attempts	1,171	1,596	2,767
Screen Failure/Ineligible	62	73	125
Screen Failure/Don't Know-Refuse	18	29	43
Ineligible (due to responsibility level)	0	481
Total Potentially Eligible	1,091	1,013	2,104	100%
Complete CATI Interview	630	595	1,225	58%
Completed Web Questionnaire	79	13	92	4%
Total Completed Cases	709	608	1,317	63%
Completion Rate	65%	60%	63%
Non respondents
Call Back/Answering Machine/No Answer	180	329	509	24%
Other	124	20	144	7%
Refusal	15	32	47	2%
Disconnected Numbers	10	11	21	1%
Total Nonrespondents	329	392	721	34%

The findings presented in this report are based on the following numbers of completed interviews in each DOT Region:

The remainder of this report presents the findings of the full-scale survey in nine regions, integrating data from the pilot study when wording and context of the survey items were identical in the pilot and full-scale surveys. These findings represent baseline data that we believe may be used to develop an ongoing performance measurement system on the environmental streamlining process. Subsequent data collection efforts will be able to compare changes in performance measures resulting from a variety of process improvement efforts.

The 2003 FHWA Survey of Transportation Agencies and Resource-Permitting Organizations

The following section provides an overview of the survey respondents who participated in the 2003 survey.

Whom did we interview?

As shown in Exhibit 1, interviews were conducted with a total of 709 individuals working in transportation agencies and 608 persons working in resource-permitting organizations. Within each type of agency or organization, roughly half of the interviews were conducted with people who conduct reviews or work on obtaining NEPA document approvals on a day-to-day basis (referred to as "Reviewers" in this report). The remaining interviews were conducted with individuals who have upper-level management responsibility for NEPA projects and are not typically in projects on a daily or frequent basis, yet have considerable knowledge about the relationship between their agency or organization and their counterparts (referred to as "Managers"). As shown in Exhibit 1, by a small margin, the majority of interviews in transportation agencies were conducted with Reviewers, whereas the slight majority of interviews in resource organizations were conducted with Managers. The numbers of interviews completed in each type of organization are more than adequate to carry out the planned analyses of the survey data on a national level. When the same questions were asked of both Reviewers and Managers, the numbers of observations are also sufficient to examine results within the 10 DOT Regions.

Back to Table of Contents

Respondents from transportation agencies had considerable seniority in their agencies and experience in NEPA-related responsibilities. Those identified as day-to-day Reviewers reported working an average of 13.1 years in their agencies and 12.0 years on NEPA-related projects. Managers in transportation agencies had an average of nearly 17.7 years tenure in their agencies and 14.6 years with NEPA projects.

Respondents from resource organizations reported similar levels of seniority. Day-to-day Reviewers reported an average of 13.3 years working in their organizations and an average of 12.5 years working on NEPA-related projects. Managers in resource organizations reported working in their agencies for 16.5 years, with an average of 15.2 years experience working on NEPA projects.

Male/female ratios were generally consistent in the two types of organizations. Among respondents from transportation agencies, 68 percent of the Reviewers interviewed were male and 32 percent were female, while 81 percent of the Managers were male and 19 percent were female. In resource or permitting organizations, 67 percent of the Reviewers interviewed were male and 33 percent were female, while 70 percent of the Managers were male and 30 percent were female.

Respondents served in a wide variety of professional positions. For the resource agencies, these included:

• Archaeologist
• Biologist
• Chief Environmental Officer
• Design Engineer
• Director of Planning
• District Engineer
• Division Chief
• Engineering Manager/Supervisor
• Environmental Assessment Manager
• Environmental Coordinator/Program Manager/Specialist/Analyst
• Environmental Manager/Supervisor
• Environmental Policy
• Environmental/Civil Engineer
• Fish and Wildlife Biologist
• Landscape Architect
• Preservation Specialist
• Transportation Planner/Specialist

For transportation, the professions of respondents included:

• Chief Environmental Officer
• Design Engineer
• Director of Planning
• District Engineer
• Division Chief
• Engineer/Senior Engineer
• Environmental Policy/Planner/Coordinator/Specialist/Analyst/Program Manager
• Environmental Engineer/Manager/Supervisor
• Transportation Specialist
• Archaeologist
• Environmental Planner
• Biologist
• Civil Engineer
• Operations Engineer
• Transportation Planner/Engineer

Exhibit 2 shows that the great majority of Reviewers in both types of organization had recent experience working on a NEPA project: 97 percent of respondents in transportation agencies and 87 percent of respondents in resource organizations had completed work on a NEPA project within the last 90 days. Moreover, all remaining Reviewers had direct involvement in a NEPA project within six months prior to the date of interview (but had not completed a project in the last 90 days). Thus, all Reviewers were able to provide judgments about relevant and recent performance by both their own organization and their counterparts.

Back to Table of Contents

Reviewers in both types of organizations had roughly similar levels of experience with NEPA projects in the 90 days prior to the interview. Reviewers in transportation agencies reported working on an average of 11.2 Federally funded permits or approvals. Reviewers in resource organizations reported working on an average of 14.5 reviews, approvals, or permits in the 90 days prior to the interview. (Reviewers in resource organizations were not restricted to only Federally-funded projects, consequently they reported more projects.) Reviewers in both types of organization were asked to count only those projects that were completed and to ignore projects that were still in process. Managers were not asked about the number of projects they oversaw in the 90 days prior to their interview.

What Types of NEPA-Related Projects Were Evaluated by Reviewers?

Reviewers in transportation agencies were asked to name the most recent Federally-funded NEPA-related project they had worked that required the most interaction on their part with staffs of resource or permitting agencies. By identifying projects with high levels of interaction with counterpart agencies, we sought to ensure that the performance ratings given by each reviewer would focus on a recently completed project with a sufficient frequency of interaction with resource organizations to provide meaningful data on project processes, problems, and organizational interactions.

Similarly, reviewers in resource-permitting organizations were asked to name the recent NEPA-related project they had worked that required the most interaction on their part with staffs of transportation agencies. As noted above, Reviewers in resource organizations were not asked to restrict their attention to only Federally funded projects in identifying the project to be evaluated. Our purpose was primarily to identify the project that involved the greatest quantity of interaction with their transportation agency counterparts at the Federal or State levels.³

³ If respondents named more than one NEPA project as having similarly high levels of interaction, a random selection of one project was made using a function of the interviewing software.

Once a single high-interaction project was identified, Reviewers in both types of organization were then asked to classify the project using the categories shown in Exhibit 3:

• EIS (Environmental Impact Statement)
• EA FONSI (Environmental Assessment — Finding of No Significant Impact)
• FHWA CE (Federal Highway Administration — Categorical Exclusion)
  or
• Programmatic Agreements

Over 99 percent of the transportation Reviewers and over 93 percent of the resource organization Reviewers were able to classify their selected projects using one of the categories provided.

The distributions of project types differed somewhat between transportation and resource Reviewers. For transportation Reviewers, there was a roughly even distribution across EIS (27 percent), EA FONSI (35 percent), and CE projects (32 percent), with only 5 percent focusing on Programmatic Agreements as the projects with the most interaction with resource or permitting organizations. Reviewers in resource organizations were about evenly divided in specifying EIS projects (41 percent) and EA FONSI projects (39 percent), with only 8 percent naming CE projects and 12 percent identifying Programmatic Agreements as the projects with the greatest interaction with transportation agencies. In our view, the differences in distributions across project types are not large enough to affect the distribution of attitudes and ratings by Reviewers from either type of organization.

Back to Table of Contents

Which Transportation Agencies and Resource-Permitting Organizations Did Reviewers Evaluate?

Transportation and resource Reviewers were also asked which counterpart agency or organization they dealt with most on the project they named. Reviewers in transportation agencies were most likely to report interacting with the Army Corps of Engineers or ACOE (27 percent), the US Fish and Wildlife Service or US FWS (18 percent), and State Historic Preservation Offices or SHPO (14 percent). These three agencies were identified as the primary counterpart organizations by about 60 percent of the transportation Reviewers. In response to a parallel question, over 85 percent of resource Reviewers named a State (81 percent) or local (4 percent) Department of Transportation or DOT. Most of the remainder named a Federal agency such as FHWA, ACOE, FAA, or Coast Guard, with FHWA cited by 11 percent.

Managers in transportation agencies were asked to name the resource organizations (not a specific project) that their staffs interacted with most on NEPA-related projects. Altogether, about 60 different resource organizations were named by the 318 transportation Managers. Of these, the most commonly cited were the ACOE by 24 percent, the US FWS by 20 percent, SHPO by 9 percent, and the US Environmental Protection Agency (EPA) by 9 percent. Together, these four agencies account for over 60 percent of the citations by transportation Managers.

Similarly, Managers in resource associations were asked to name the transportation agencies their staffs interacted with most on NEPA projects. The 326 resource Managers listed about 40 different transportation organizations. The most commonly mentioned agencies were State or county DOTs (by 59 percent, in 35 States), and FHWA by 21 percent. Together, the State and local DOTs and FHWA account for over 80 percent of the agencies cited by resource Managers.

Report of Major Findings

A Note on Interpreting the 2003 NEPA Process Survey Results

In the following sections, we analyze the survey data on perceptions and experiences of Reviewers and Managers working in transportation agencies and resource or permitting organizations. Reviewers in both types of organization were asked for their evaluations of their experiences on the NEPA project they cited in the screening section of the questionnaire (the project that required the most interaction with their counterpart organizations on the other side), and with the organization or agency they identified as the one they interacted with most. Because both types of Reviewers named a large number of different projects and worked with several different counterpart agencies or organizations, we will use the phrases "cited project" and "named organization" to refer to the diverse list of projects and organizations identified in the screening section of the interview. These two phrases are intended to remind the reader that all of the perceptions and beliefs by transportation and resource officials about the NEPA process and performance attributes of involved organizations must be understood as referring to a self-selected sample of projects and interaction patterns.

Readers should bear in mind that participating Reviewers were permitted and encouraged to identify the single project in recent experience that had the highest level of interactions between them and their counterpart agencies on the other side of the process. No constraints requiring maintained focus on the same project were placed on transportation and resource Reviewers — even those located in the same DOT regions. Not surprisingly, a large number of different projects were cited by the 391 transportation agency Reviewers and the 282-resource organization Reviewers.

The data suggest that in a few instances, some of the Reviewers from both transportation agencies and resource organizations in the same DOT region may have provided ratings on their counterparts with whom they interacted on the same project.⁴ In those very few cases, the survey data from transportation and resource Reviewers contain responses in which the reviewers are actually rating organizations and possibly individuals who are also respondents to this survey. However, the frequency that transportation and resource Reviewers focused on a common project and on each others' organizations was far too low to allow this survey to be used to evaluate either the process or agency performance on specific NEPA projects. This survey was not designed for that purpose and cannot be validly used for that purpose.

⁴ Because the survey did not ask respondents to provide the formal name and identifying number of their selected project, the survey data cannot be used to identify specific NEPA projects with certainty. The absence of formal project designations is a design feature of the survey that was intended to enhance respondents' privacy and confidentiality, and hence the validity and accuracy of the performance evaluations they provided.

The analyses and descriptions in the following sections represent characterizations of general project and agency performance attributes by the Reviewers and Managers who were interviewed. They cannot and should not be interpreted as evaluations of particular projects in any DOT region. For this reason, our emphasis in what follows is on the evaluation of the process, problems, perceived agency attributes, and relationships between the two types of agencies all across the country and, in some cases, within the ten DOT regions. With project-level ratings from almost 700 Reviewers and almost 650 managers of NEPA-related activities across the country, these data provide a unique opportunity to understand what the participants in the environmental streamlining process believe to be both the major strengths and the most important opportunities for improvement for both the NEPA process and the relationships among the organizations that must interact to complete the projects. By understanding these general characteristics of the process and the interactions and relationships among participating organizations, senior managers representing both sides of the NEPA process can identify or develop adjustments that have the greatest potential for improving the effectiveness and efficiency of agency collaboration and for removing perceived barriers to better, more timely results in NEPA actions.

How Did Transportation and Resource Reviewers Rate Their Experiences on Recent NEPA Projects?

Following the screening questions, the first substantive section of the survey instrument asked Reviewers in transportation agencies and resource organizations (those with day-to-day responsibility for working directly on NEPA projects), a set of questions about their experiences on the cited project and with the named organization recorded in the screener. Managers in transportation agencies and resource organizations were not asked project-level questions but were skipped to a set of questions about their perceptions of organizational relationships (that were also asked of reviewers). Organizational relationship data collected from Reviewers and Managers are analyzed in a later section of this report.

Reviewers Rate Cited Projects and Named Organizations

Reviewers in both types of organization were told:

"Now I'm going to ask you a series of questions about the [cited project] with regard to [named organization]. Overall, how satisfied were you with [named organization]'s performance on this project? Please use a scale from one to five, where 1 is 'very satisfied,' and 5 is 'very unsatisfied.'"

Charts 1 and 2 show the response distributions obtained from transportation and resource Reviewers.

Chart 1. Overall Satisfaction Expressed by Transportation Reviewers with the Performance of Named Counterpart Organizations on the Cited NEPA Project

Back to Table of Contents

Chart 2. Overall Satisfaction Expressed by Resource Reviewers with the Performance of Named Counterpart Organizations on the Cited NEPA Project

Back to Table of Contents

On balance, Reviewers from both types of organizations report moderately high satisfaction in their interactions with counterpart agencies on recent projects. However, clearly there is room for process improvements that could further increase the satisfaction of these key officials.

About one-quarter (26 percent) of the 390 transportation agency Reviewers indicated that they were "very satisfied" with the named resource organization's performance on the cited project, while only one in twenty (5 percent) said that they were "very unsatisfied" — over a 5 to 1 ratio of high satisfaction to deep dissatisfaction. One in five (20 percent) of transportation Reviewers selected the neutral response (a value of 3 on the five-point scale) indicating neither satisfaction nor dissatisfaction. However, the remaining 50 percent of the transportation Reviewers were far more likely to indicate they were satisfied with the process (41 percent) than dissatisfied (7 percent) — nearly a 6 to 1 ratio in positive to negative ratings.

In total, two-thirds (67 percent) of the transportation Reviewers reported being either satisfied or very satisfied with their named resource organization's performance on their cited projects, while less than one-eighth (12 percent) reported being either dissatisfied or very dissatisfied.

A strikingly similar pattern is evident in the responses from resource Reviewers, with over one-fourth (28 percent) indicating they were "very satisfied" with the named transportation agency's performance on their cited projects, and fewer than one in thirty (3 percent) reporting that they were "very dissatisfied." Also similarly, 20 percent of the resource Reviewers chose the middle response category. Among the remaining 49 percent, nearly three times as many resource Reviewers (36 percent) reported they were "satisfied" with the performance of their counterparts, in contrast to only 13 percent who indicated they were "unsatisfied."

In summary, nearly two-thirds of resource Reviewers (64 percent) said they were either satisfied or very satisfied with the named transportation agency's performance, while less than one-sixth (16 percent) responded that they were dissatisfied or very dissatisfied.

The relative symmetry in the distributions of overall satisfaction scores from Reviewers in both types of organization supports the conclusion that in the majority of recent projects, interactions with counterpart agencies are seen as satisfactory. The sizeable percentages of both types of Reviewers reporting satisfaction also support the conclusion that positive ratings by one type do not tend to be balanced by negative ratings by counterparts. We cannot examine this possibility directly because Reviewers in transportation and resource organizations cannot be linked to a specific common project. However, by examining these results at the regional level, we may obtain a somewhat closer

Regional Results in Reviewers' Overall Satisfaction Ratings of Counterpart Organizations

The percentage distributions across the five categories of the overall satisfaction measure varied somewhat by region, but not substantially. Analysis of regional distributions of reviewers is hampered by the total sample size of 390 transportation Reviewers, leaving an average of only 39 observations per region (ranging between 23 Reviewers in Region 1 and 60 Reviewers in Region 9). The regional numbers are even smaller for the resource Reviewers, ranging from only 11 observations in Region 2 to 37 cases in Region 9. Given these small numbers, we examined the percentages in the two "satisfaction" categories combined, and the two "dissatisfaction" categories combined. At this level of aggregation, regional differences were relatively minor.

Across regions, 12 percent of transportation Reviewers reported they were "unsatisfied" or "very unsatisfied" with the performance of the named resource organization. This percentage varied by a maximum of only +/- 6 percentage points across all 10 regions, from 6 percent in Region 6 to 18 percent in Region 9. The combined dissatisfaction levels for most of the DOT Regions were within 4 percentage points of the overall percentage (12 percent), supporting the conclusion that dissatisfaction with NEPA project interactions is at a relatively low level in all regions.

Slightly greater variation was found on the satisfaction side of the scale, with Region 3 (at 54 percent) being 13 percentage points below the overall average of 67 percent, and Region 5 (at 81 percent) being 14 percentage points above the overall average. However, most of the DOT regions were within the range of 4 percentage points below and 7 percentage points above the overall average.

For the smaller number (282) of resource Reviewers, regional variation in satisfaction distributions appeared slightly larger than for transportation Reviewers, but this is primarily a consequence of the smaller numbers of resource Reviewers in all regions. Distributions that deviated most from the overall results arose because of very small cell sizes (zero to 3 cases) occurring in Regions with very few respondents. Despite the smaller cell sizes, the regional distributions of satisfaction for resource Reviewers were sometimes similar and sometimes different from those of transportation Reviewers. Regions 2, 9, and 10 showed somewhat higher levels of dissatisfaction, while Regions 3, 4, and 7 reported lower levels of dissatisfaction compared to the overall average for resource Reviewers. Conversely, resource Reviewers in Regions 1, 3, and 7 reported higher levels of satisfaction, and those in Regions 2, 8, and 10 indicated lower percentages of satisfaction than the overall average for resource Reviewers.

A comparison of mean (average) satisfaction scores calculated for the two types of Reviewers in the 10 DOT regions is displayed in Exhibit 4 as follows.

Back to Table of Contents

Across regions, the difference in mean satisfaction score between transportation and resource Reviewers is extremely small (+0.03), and is of no relevance or significance for policy or management purposes. Within regions, the differences are somewhat larger. In half of the Regions, the transportation Reviewers report slightly higher satisfaction scores, and the opposite is true in the other five. All ten of the Regional differences fall within the range of -0.3 to +0.4 points on the five-point satisfaction scale.

An intuitive understanding of the magnitude of these differences may best be gained with an example: To raise the mean score of all Reviewers from the current level of about 3.74 to a 4.00 would require just over one-fourth of the Reviewers (175 individuals) to increase their satisfaction rating by 1 point on the five-point scale over their 2003 rating (e.g., from "dissatisfied" to "neutral," or from "neutral" to "satisfied"). In the 2003 survey, a total of 230 Reviewers (34 percent) reported being less than "satisfied" (values of 1, 2, or 3 on the scale); from that number, 138 reported "neutral" scores (3).

Because of the inherent unreliability and wide swings of percentage distributions based on small numbers of observations, we will restrict most of our analyses and interpretation of Reviewer responses to project-specific questions to the overall sample.

Implications for Management

Overall satisfaction with the performance of counterpart agencies is an important indicator of the efficiency and effectiveness of NEPA projects. Under ideal circumstances, all Reviewers from both types of organizations would report they were "very satisfied" with their counterparts' performance on projects that required the highest levels of agency interaction. However, our survey data indicate that about three-fourths are less than "very satisfied" with their counterpart agency's performance, leaving a great deal of room for "improvement" in the relationships and interactions.

That said, given the partly adversarial nature of the NEPA process, universal reports of the highest level of satisfaction among reviewers is an unrealistic expectation. Moreover, a survey measurement approach that asks Reviewers about the project that required the most interaction with a counterpart organization unavoidably results in selection of projects likely to have more contentious elements than the average NEPA project. An alternative goal of all reviewers reporting that they are either satisfied or very satisfied would be a strongly positive outcome for the environmental streamlining effort. At this stage, for both transportation and resource Reviewers, one-third or slightly more did not report that level of satisfaction on their cited projects. The goal of having no Reviewers report overall satisfaction scores of 3 or lower may also be practically unachievable, given the nature of NEPA projects.

The mean (average) satisfaction scale scores given to their counterparts by both transportation and resource Reviewers were nearly identical at values of 3.76 and 3.73, respectively, on the five-point scale. By definition, a mean value of 5 on the five-point scale would indicate that all reviewers reported that they were "very satisfied" with their counterpart's performance on the cited project — again, an unrealistic standard. However, a mean value of, 4.5 on the five-point scale would be a meaningful target, as it would represent a high level of average overall satisfaction by Reviewers with their counterpart organizations, but would not require the complete elimination of scores of 3 or lower. The current mean satisfaction level for Reviewers in both types of organizations is at about 83 percent of a hypothetical target mean value of 4.5. Interventions that would raise the performance evaluations to a mean of 4.5 within a one- to two-year period may be developed using information from the 2003 survey on those areas of project performance that are most frequently cited as problem areas by both transportation and resource Reviewers.

Raising overall average satisfaction scores to 4.5 is a realistic goal for the next few years. A useful management strategy will be to determine the types of problems and organizational relationship and performance attributes most associated with reviewers assigning scores of 3 or lower on the overall satisfaction measure, so that interventions can be designed to reduce the occurrence of those problems, organizational characteristics or behaviors, and other aspects of NEPA project interactions. Even more importantly, senior Managers should identify areas of excellent performance as identified by Reviewers and Managers in both types of organization across the ten DOT regions, and conduct further research to determine whether the responsible staff and organizations in those Regions have developed or identified solutions to problems that may be observed in other Regions. All organizations gain in effectiveness and efficiency when the methods and procedures found in the best performing segments of the organization are identified and diffused throughout the rest of the organization.

Reviewers' Experience of Problems with Their Counterpart Organizations on NEPA Projects

Following the overall satisfaction question, Reviewers for both types of organizations were asked:

"Did you experience any problems with [named organization] during the [cited project]?"

Just over one-third (36 percent) of transportation agency Reviewers reported that they experienced problems with the named organizations on their cited projects, while nearly two-thirds (64 percent) indicated that they had not encountered problems.

Similarly, 37 percent of resource Reviewers indicated that they had experienced problems dealing with their counterpart agencies on their cited projects, and 63 percent reported no problems.

Because these distributions are very similar to those for the overall satisfaction question reported just above, we examined the extent of the relationship between the satisfaction score given and the reports of project problems. As expected, the correlations between the responses were very high, but they were not perfect, so the two questions are not measuring precisely the same reactions. Not surprisingly, the transportation and resource Reviewers who reported experiencing project problems were much more likely to also report being "dissatisfied" or "very dissatisfied" with the performance of the named organization on the cited project. Fully 34 percent of transportation Reviewers who reported problems with their resource counterparts also reported being dissatisfied with their performance, and another 36 percent gave a neutral response to the satisfaction question. Among resource Reviewers, 40 percent of those who reported problems with their transportation counterparts also reported being dissatisfied or very dissatisfied. An additional 31 percent of resource Reviewers who reported problems gave a neutral response to the satisfaction question.

We note that about 30 percent of both types of Reviewers who reported having project problems also indicated they were "satisfied" or "very satisfied" with their counterpart organizations.

For both transportation and resource Reviewers who did not report problems, less than 1 percent indicated that they were dissatisfied, and only 12 to 14 percent gave neutral responses (lower than the 20 percent giving neutral scores overall).

We believe at least two types of Reviewer reactions lead to these results. First, some Reviewers (of both types) may have experienced problems with the named organizations on their projects that were so minor that they did not preclude a satisfactory process and/or outcome. Second, based on results of focus group sessions held prior to the survey, we understand that many Reviewers involved in NEPA projects expect to experience problems with their counterpart organizations. Because these problems are anticipated, and may be viewed by some Reviewers as unavoidable and inherent in the process, they are not sufficient to cause expressions of dissatisfaction as measured in the 2003 survey.

Yet, it is clear that the occurrence of project problems with the named organization is strongly associated with overall satisfaction levels reported by Reviewers. Among transportation Reviewers, 87 percent of those who did not experience problems indicated that they were satisfied or very satisfied with their named counterpart organizations. Similarly, among resource Reviewers who did not experience problems, 85 percent indicated they were satisfied or very satisfied with the performance of their named counterpart organizations. It is reasonable to expect that reducing or eliminating problems by examining root causes and assisting transportation and resource organizations to achieve better, faster responses will be the primary key to improving overall performance and the satisfaction ratings given in response to evaluation survey questions.

Chart 3. Transportation Reviewers: Problem Occurrence and Corresponding Levels of Satisfaction with the Performance of Named Counterpart Organizations on the Cited NEPA Project

Back to Table of Contents

Chart 4. Resource Reviewers: Problem Occurrence and Corresponding Levels of Satisfaction with the Performance of Named Counterpart Organizations on the Cited NEPA Project

Back to Table of Contents

Timing and Nature of Project Problems Reported by Reviewers

To probe the associations between problem occurrence and satisfaction with the named organizations, the 140 transportation Reviewers and the 105 resource Reviewers who reported project problems were asked a series of questions about the nature of the problems they experienced.

First, they were asked, "At what stage of the process did the problem occur?" They then were read the following series of eight stages of project development when problems might be expected to arise:

1. Early project planning or scoping
2. Defining purpose and need
3. Information or data collection
4. Development and analysis of alternatives
5. Analysis of impacts
6. Selection of preferred alternatives
7. Commitment to mitigation measures
8. Finalizing documents or responses to comments

Reviewers were allowed to report problem occurrence at any or all of the eight listed project stages or activities. The following distributions of problem occurrence were reported by transportation and resource Reviewers.

Back to Table of Contents

Transportation and resource Reviewers thus showed some similarities, but also some important differences in their reports on the stages of NEPA projects when problems arose. For both groups of Reviewers, the stage of "defining purpose and need" of the project was the least likely point for perceived problems to occur and the stage of "analysis of impacts" was rated as one of the most frequent problem stages. Nearly two-thirds of the transportation Reviewers who reported project problems indicated that they occurred at both the "analysis of impacts" and the "finalizing documents or responses to comments" stages.

On the resource side, nearly two-thirds of Reviewers cited problems at the "analysis of impacts" stage and the "information or data collection" stage.

Transportation Reviewers were more likely than resource Reviewers to report that problems occurred at the "commitment to mitigation measures" stage.

Resource Reviewers were more likely than transportation Reviewers to report that problems arose at one of the remaining three stages listed, including "early project planning or scoping," "development and analysis of alternatives," and "selection of preferred alternatives."

At least one-third of the Reviewers of one type or the other cited problems in all 8 of the project stages listed in the survey question, indicating that problems occur with significant frequency at all stages of NEPA projects. At least 40 percent of the Reviewers of one type or the other cited problems at 7 of the 8 stages listed. Over half of the Reviewers of one type or the other cited problems at 5 of the 8 listed stages.

More than half of the Reviewers from both types of organizations reported problems with "analysis of impacts" and "finalizing documents or responses to comments," a fact that we believe deserves attention. Beyond these general observations, there are significant differences in the views of the two sides concerning the most problematic stages in NEPA projects.

It is also important to keep in mind, however, that the percentages reported in the prior paragraphs were calculated based on only the 1/3 of transportation and resource Reviewers who reported experiencing any problems in a recent project that required the most interaction with counterpart agencies. In other words, even the most common problems were cited by just over 20 percent of all the Reviewers who were interviewed. Put differently, about 80 percent of the Reviewers on both sides of the NEPA projects did not experience problems with "analysis of impacts." This fact may increase the difficulty of developing effective, targeted interventions that provide universal improvements at this stage of future NEPA projects. That said, data and information collection, impact analysis, and finalizing documents are clearly three areas that may benefit most from management attention on process improvement.

Reviewers were then asked to "please summarize what the (problem was / problems were) in just a few words." Based on earlier focus group sessions, our interviewers were prepared to field-code several "expected" types of responses shown in Exhibit 6 (along with the percentage of transportation and resource Reviewers who named each type):

Back to Table of Contents

For transportation Reviewers, "lack of timely response" was cited more than twice as often as any other issue, and three to five times as often as most of the other types of problems that were mentioned. "Staff turnover or availability" was the second most common type of issue reported by transportation Reviewers, a symptom that could be closely related to the timeliness complaint. Issues such as "problems with mitigation," "poor communications," "wrong or incomplete information," and "Disagreements or differences of opinion" were mentioned by 6 to 9 percent of the transportation Reviewers who reported project problems. In fact, however, these matters were only mentioned by 9 to 12 Reviewers out of the 391 who were interviewed, so they do not appear to us to be very frequent causes of problems.

Among resource Reviewers, there was a more even distribution mentioning each of the listed problems, with "wrong or incomplete information" topping the list, followed by "lack of timely response," "poor communications," "poor coordination," and "problems with alternative interpretation" (cited about equally by 11-12 percent).

In summary, from the transportation side, the single dominant problem associated with dissatisfaction with the performance of counterpart organizations seems to be timeliness of response, which may be related (in the view of transportation Reviewers) to staffing problems at the resource-permitting agencies and may in turn lead to problems with mitigation efforts. From the resource side, project problems that are associated with dissatisfaction with the performance of transportation agencies appear to be a broader mix of process issues led by quality of information provided and timeliness of response, but also including communications, coordination, and other process matters. This understanding provides a framework for the analysis of other ratings collected from transportation agency and resource organization Reviewers, as detailed further below.

Factors Affecting Organizational Interactions on NEPA Projects: The Reviewers' Perspective

Focus groups conducted during the design phase of the evaluation made clear that the nature of organizational interactions on individual NEPA projects are determined by complex combinations of the specific details of each project together with longstanding patterns of agency perceptions and experiences across multiple projects. Depending on their nature, these combinations can produce either of two results: highly efficient processes and successful outcomes, or conflictual interactions that delay project completion and co