OP27 Before-and-After Study Review - Federal Transit Administration

U.S. DOT Federal Transit Administration 

TPM-20 Office of Engineering 

Project Management Oversight 

Oversight Procedure 27 – Before-and-After Study Reviews 

__________________________________________________________________________________ 

1.0 PURPOSE 

The purpose of this Oversight Procedure (OP) is to describe the review, analysis, and reporting 

requirements expected by the Federal Transit Administration (FTA) from the Project Management 

Oversight Contractor (PMOC) with regard to the adequacy of the Grantee’s Before and After Study. 

Specifically, the PMOC will evaluate the Grantee’s: 

• Preservation, analysis, and documentation of forecasts at different project development 

phases; 

• Development of the Before-and-After Study work plan; and 

• Preparation of the Before-and-After Study report. 

This OP covers the project scope and capital cost elements of the Grantee’s before-and-after 

submissions. The other elements are addressed by FTA’s Office of Planning and Environment. 

2.0 BACKGROUND 

In the December 2000 Final Rule on Major Capital Investment Projects, FTA required that Grantees 

seeking Full Funding Grant Agreements (FFGA) submit a plan for the collection and analysis of 

project information to identify the accuracy of the forecasts prepared during project planning and 

development. In 2005, SAFETEA-LU amended Section 5309(g)(2)(c) to codify this regulatory 

requirement and to require that Grantees, as a condition of receiving an FFGA, assemble information 

on five project characteristics: (1) project scope; (2) transit service levels; (3) capital costs; (4) 

operating and maintenance costs; and (5) ridership patterns and revenues. SAFETEA-LU also requires 

FTA to use this information in preparing an annual report to Congress on Before-and-After Studies 

completed during that year. 

For the two characteristics covered in this OP (project scope and capital cost), information from the 

Grantees is required at the following points: (1) entry into preliminary engineering (PE); (2) entry into 

final design (FD); (3) immediately prior to the FFGA award; (4) prior to start of revenue services; and 

(5) two years after opening of revenue service. 

To ensure sufficient information is available for preparation of the forthcoming Before-and-After 

Study, FTA policy guidance issued May 2006 requires preservation and analysis of forecasts prepared 

during project planning and development. The document, “Preparing for Before-and-After Studies,” 

explains why both preservation and analysis is required. A project will not be approved to the next 

stage of project development until FTA determines the Grantee has preserved the required information 

and conducted an analysis that adequately explains the reason(s) for changes in forecasts since the 

previous phase of project development. 

FOR FTA INTERNAL USE ONLY OP 27 Before-and-After Study Reviews 

Rev. 2, May 2010 

Page 1 of 6

The SAFETEA-LU amendment to Section 5309(1)(2) also requires that FTA publish an annual report 

that analyzes the consistency and accuracy of cost and ridership forecasts prepared by contractors. To 

help fulfill this responsibility, FTA requires, as part of the before-and-after submissions, Grantees to 

identify the contractor(s) responsible for the forecasts and to include a description of the contractor’s 

role and responsibilities. 

3.0 OBJECTIVES 

The objectives of this OP are to ensure that the Grantee performing the following and submitting it to 

FTA in a timely manner: 

• Preserving the project scope and capital cost forecasts (predictions) at each stage of project 

development to a level sufficient for use in the predicted-versus-actual analysis for the 

Before-and-After Study; 

• Documenting and analyzing changes in the project scope and capital cost forecasts since the 

previous phase of project development to a level sufficient for use in the predicted-versusactual 

analysis for the Before-and-After Study; and 

• Developing the Before-and-After Study Plan for project scope and capital costs that identifies 

the appropriate work to be performed for their Before-and-After Study, including a sufficient 

allocation of cost to perform this work; 

• Developing interim reports as appropriate; 

• Developing the Before-and-After Study Report that adequately achieves the project scope and 

capital cost analysis objectives of FTA’s Before-and-After Study requirements. 

4.0 REFERENCES 

The following are the principal, but by no means the only, references to Federal legislation, 

codification, regulation, and guidance with which the PMOC should have a good understanding as 

related to the Grantee’s work being reviewed under this OP: 

4.1 Legislative 

• The Safe, Accountable, Flexible, Efficient Transportation Equity Act: A Legacy for Users 

(SAFETEA-LU), Pub.L. 109-59 

4.2 United States Code 

• FTA Statues, 49 U.S.C. Chapter 53 

4.3 Regulations 

• Final Rule on Major Capital Investment Projects, 49 CFR Part 611 

4.4 FTA Circulars 

• C5200.1A, Full Funding Grant Agreements Guidance 


Rev. 2, May 2010 

Page 2 of 6

4.5 Guidance 

• FTA Guidance on New Starts Policies and Procedures, issued May 16, 2006 

• Guidance on Before-and-After Studies, Revised October 2006 (under modification) 

• Preparing for Before-and-After Studies, Revised 2009 

5.0 GRANTEE SUBMITTALS 

The PMOC shall obtain and study the Grantee’s documents below and as listed in Appendix B. 

5.1 Preserved and analyzed project information 

The PMOC shall ensure the Grantee: 

• Complies with the requirements of Appendix B of this OP with regard to the preservation and 

submission to FTA of project information and forecasts at the noted project milestones; 

• Organizes the project information in a clear and consistent manner on CD-ROMs, DVDs, or an 

external hard drive, to ensure that readers can easily follow and understand the information; 

• Provides a list of all the documents contained on each disc or file, at each project milestone. 

The information provided shall be cumulative in nature, including project information from all 

phases up to the current milestone; 

• Includes with its submissions, a copy of any non-commercial off-the-shelf software (if 

possible) that was used to perform analyses; 

• Provides an Analysis of Changes Since Last Milestone. 

5.2 Before-and-After Study Plan 

As shown in Appendix B, it is recommended that the Grantee submit its Before-and-After Study Plan 

at entry to PE or FD, however it is required to be submitted not later than with the application for 

FFGA (or PCGA for Small Starts). 

5.3 Before-and-After Study Report 

As shown in Appendix B, the Grantee is encouraged to submit interim reports, however only the Final 

Before-and-After Study Report is required. 

6.0 SCOPE OF WORK 

6.1 Review of Project Documentation (Preservation and Analysis) 

As required in Appendix B, for each of the project development phases, the PMOC shall assess if the 

Grantee has preserved the project scope and capital cost information needed for use in the predictedversus-actual 

analysis for the Before-and-After Study. 


Rev. 2, May 2010 

Page 3 of 6

The PMOC shall assess if the Grantee has adequately analyzed and explained the causes of changes in 

project scope and capital cost. Explanations are required if project scope or cost has changed from the 

last milestone. The Grantee should provide a detailed explanation of the reasons for changes in the 

project’s capital cost (constant year dollars (constant$) and actual cost (YOE$)), broken down into the 

following categories, including but not limited to: 

• Scope 

• Schedule (i.e., variances in cost due to change in the revenue operations date) 

• Unit pricing/inflation 

• Project reviews (constructability, coordination, risk assessment, etc.) 

• Institutional factors (FTA process, local politics or funding, Agency process, third party 

agreements, selection of project delivery method, etc.) 

• Other changes 

During construction of the project, the PMOC shall ensure that the Grantee preserves the actual (asbuilt) 

project scope and capital cost data as it becomes available in order to sufficiently document the 

changes between the as-built scope/costs versus the projected scope/costs. In addition, the PMOC 

shall ensure that the Grantee’s Analysis of Changes Since Last Milestone outlines why a difference 

exists, if any, between the cost at FFGA execution and the actual (as-built) cost. To more clearly 

document the impacts of inflation, the Grantee should convert their estimated/predicted capital costs 

(in YOE$) and actual costs (also in YOE$) to constant year dollars in which both are referenced to the 

same base year. When analyzing both the predicted and actual capital costs, the Grantee should 

typically use the year of the entry into PE as its base year reference, with all other base year costs 

converted to this same year, and the actual costs (in YOE$) deflated down to this year, in order to 

calculate the actual cost in base year dollars (use ENR or other approved indices to calculate constant 

year costs). 

6.2 Review of Before-and-After Study Plan 

As shown in Appendix B, it is recommended that the Grantee develop and submit its Before-and-After 

Study Plan at entry to PE or FD, since project information is required to be preserved beginning at the 

conclusion of the AA phase. The purpose of the Before-and-After Study Plan is to make sure that data 

collection and analysis are done in ways that ensure that all comparisons made are complete and 

accurate. There are many factors (both internal and external) throughout a project’s life that will result 

in changes; therefore, the Before-and-After Study Plan must include the Grantee’s procedures to 

capture and document changes, preferably as they occur. 

When reviewing a Grantee’s Before-and-After Study Plan, the PMOC should determine the following: 

• Did the Grantee properly identify the work to be done for its Before-and-After Study; 

• Does the Study Plan provide for capturing what has happened to date; 

• Does the Study Plan reflect the project characteristics; 

• Does the Study Plan capture all known issues which have influenced the project scope and 

costs. 


Rev. 2, May 2010 

Page 4 of 6

The PMOC shall confirm that the Grantee’s Before-and-After Study Plan includes but is not limited to 

the following: 

• Introduction (includes purpose and responsibilities of various organizations involved in 

project) 

• Scope of work 

o Organization (identifies procedures that outline how the Grantee will assemble, organize, 

maintain and preserve all applicable data for the Before-and-After Study, as well as which 

parties have responsibility for these activities); 

o Documentation of forecasts, conditions before project implementation, conditions after 

project opening, and completion and documentation of analyses. 

• Work products and their schedule for completion 

• Budget and staffing plan for the Grantee’s efforts to perform activities associated with the 

Before-and-After Study. 

6.3 Review of Interim Reports 

Although the FTA requires only a Final Before-and-After Study Report, the Grantee is strongly 

encouraged to prepare and submit interim reports as noted in Appendix B below. Examples of such 

interim reports include: 

• Project Planning Forecasts Interim Report (to be submitted at PE, FD and prior to award of 

FFGA.) The report is required to include service levels, operating costs and ridership 

forecasts, project scope and capital cost, however the PMOC is required to review only scope 

and cost. 

• Interim Report documenting and describing “before” (pre-construction) conditions (PMOC 

review not required at this milestone unless specifically directed by FTA to do so) 

• Interim Report documenting and describing “after” (post-construction) conditions (PMOC 

review not required at this milestone unless specifically directed by FTA to do so) 

The PMOC shall refer to Appendix C below for the required report outline. The PMOC should review 

the report to verify completeness and the reasonableness of the analysis contained in this report. 

6.4 Review of Final Before-and-After Study Report 

The main purpose of the Before-and-After Study Report is to identify and explain the impacts of the 

project in terms of project scope, capital costs, transit service levels, O&M costs and ridership that 

have occurred between the AA phase and two years after revenue service. 

The Final Before-and-After Study Report will be submitted by the Grantee approximately two years 

after the start of revenue operations. At this time, the PMOC may be tasked with reviewing this report 

to verify completeness and the reasonableness of the analysis contained in this report. 


Rev. 2, May 2010 

Page 5 of 6

7.0 REPORT, PRESENTATION, RECONCILIATION 

The PMOC shall provide FTA with a written report of its findings, analysis, recommendations, 

professional opinions, and a description of the review activities undertaken. After FTA approval, the 

PMOC should share the report with the Grantee. In the event that differences of opinion exist between 

the PMOC and the Grantee regarding the PMOC’s findings, the FTA may direct the PMOC to 

reconcile with the Grantee and provide FTA with a report addendum covering the agreed modifications 

by the Grantee and PMOC. 

The report formatting requirements of OP 01 apply. When necessary, PMOC shall perform data 

analysis and develop data models that meet FTA requirements using Microsoft Office products such as 

Excel and Word and use FTA-templates when provided. The PMOC may add other software as 

required but documentation and report data shall be made available to FTA. 


Rev. 2, May 2010 

Page 6 of 6

APPENDIX A 

Acceptable Quality Level 

________________________________________________________________________________________________________________ 

1 

2 

DESIRED 

OUTCOME 

The PMOC shall 

assess the 

reliability of the 

elements of the 

Grantee's Beforeand-After 

Study 

program. 


provide continuous 

review and 

evaluation of 



program to ensure 

project scope and 

capital cost 

forecasts and data 

are being 

preserved such 

that complete and 

accurate Beforeand-After 


comparisons and 

analyses can be 

performed. 

PERFORMANCE REQUIREMENT 

R1a. The PMOC shall develop and 

document a process for review and analysis 

of the Grantee's preservation of project 

scope and capital cost forecasts and its 

Before-and-After Study Plans and Reports. 

R1b. The PMOC shall use its process and 

project management judgment to validate the 

thoroughness of the Grantee's Before-and- 

After Study Plan and the Grantee's 

preservation of project scope and capital 

cost data at all phases of the project. 

R2a. The PMOC shall assess if the Grantee 

has preserved the necessary project scope 

and capital cost information and the reliability 

of such information for use in the predicted 

versus actual analysis for the Before and 

After Study. 

R2b. The PMOC shall assess if the Grantee 

has adequately analyzed and explained the 

causes of changes in project scope and 

capital cost forecasts from last project 

milestone. 

R2c. During Construction, the PMOC shall 

ensure the Grantee preserves the actual (asbuilt) 

project scope and capital cost data as it 

becomes available in order to sufficiently 

document changes between projected 

scope/costs and as-built scope/costs. 

R2d. The PMOC shall review and assess 

the Grantee's Before-and-After Study Plan 

and provide FTA with its determinations on 

whether (1) the Grantee properly identified 

work to be performed in its Before-and-After 

Study, (2) the Plan provides for capturing 

what has occurred to date, (3) the Plan 

reflects project characteristics and (4) the 

Plan captures all known issues which have 

influenced project scope and costs 

CHECK 

LIST 

ACCEPTABLE QUALITY LEVEL 

Q1a. PMOC provides documentation of 

the process. 

Q1b. Review must be made and PMOC 

provides internal verification that the 

process as documented has been 

followed. 

Q2a. Professional findings and opinions 

of the reliability of the Grantee's 

preservation of project scope and capital 

cost data. 

Q2b. Professional opinion of the 

adequacy of the Grantee's explanations 

of deviations in project scope and capital 

cost forecasts from last project 

milestone. 

Q2c. Professional findings and opinions 

of the reliability of the Grantee's 

preservation of actual (as-built) project 

scope and capital cost data during 

construction of the project. 

PERFORMANCE MEASURE 

M1a. Review of the process 

documentation. 

M1b. Documented review and analysis 

of Grantee's Before-and-After Study 

Plan, preservation of project data and 

and capacity to develop a Federallyassisted 

Project. 

M2a. Documented evidence of a review 

of the Grantee's preserved project 

scope and capital cost information 

demonstrates sound management and 

engineering practices and professional 

experience. 

M2b. PMOC's review and opinion as to 

the Grantee’s explanations for 

deviations in project scope and capital 

cost forecasts demonstrates sound 

management and engineering practices 

and professional experience. 

M2c. Documented evidence of a review 

of the Grantee's preservation of actual 

project scope and capital cost 

information demonstrates sound 

management and engineering practices 

and professional experience. 

M2d. Documented evidence of a 

thorough review of Grantee's Beforeand-After 

Study Plan, supported by 

professional opinion 

MONITORING 

METHOD 

MM1a. Periodic 

review by FTA or 

its agent. 

MM1b. Periodic 


its agent and the 

PMOC's internal 

verification. 

MM2a. Periodic 


its agent. 

MM2b. Periodic 


its agent. 

MM2c. Periodic 


its agent. 

Q2d. Professional findings and opinions 

of the adequacy and completeness of 

the Grantee's Before-and-After Study 

Plan. 

MM2d. Periodic 


its agent. 

OP 27 Before-and-After Study Reviews 

Rev. 2, May 2010 

Page A-1

APPENDIX A 

Acceptable Quality Level 

________________________________________________________________________________________________________________ 

DESIRED 

OUTCOME 


provide continuous 

review and 

evaluation of 



program to ensure 

project scope and 

capital cost 

forecasts and data 

are being 

preserved such 

that complete and 

accurate Beforeand-After 


comparisons and 

analyses can be 

performed. 


provide FTA with a 

written report of its 

findings, analysis, 

recommendations 

and professional 

opinions. 

PERFORMANCE REQUIREMENT 

CHECK 

LIST 

ACCEPTABLE QUALITY LEVEL 

PERFORMANCE MEASURE 

MONITORING 

METHOD 

MM2e. Periodic 


its agent. 

3 

R2e. The PMOC shall review and assess the 

completeness and reasonableness of the 

analyses contained in the Grantee's Interim 

Before-and-After Study Reports. 

R2f. The PMOC shall review and assess the 

completeness and reasonableness of the 

analyses contained in the Grantee's Final 

Before-and-After Study Report. 

R3. The PMOC shall present its findings, 

analysis, recommendations and professional 

opinions to FTA in a written report and, when 

so directed by FTA, seek to reconcile its 

findings with Grantee to the extent possible. 

A supplemental report shall be filed 

describing the results of reconciliation 

attempts. 

Q2e. Professional findings and opinions 

of the completeness and 

reasonableness of the Grantee's Interim 

Before-and-After Study Reports. 

Q2f. Professional findings and opinions 

of the completeness and 

reasonableness of the analyses 

contained in the Grantee's Final Beforeand-After 

Study Report. 

Q3. Reports and presentations are 

professional, clear, concise, and well 

written. The findings and conclusions 

have been reconciled with other PMOC 

reports and have been reconciled with 

the Grantee to the extent possible. 

M2e. Documented evidence of a 

thorough review of the Grantee's Interim 

Before-and-After Study Report, 

supported by professional opinion 

M2f. Documented evidence of a 

thorough review of the Grantee's Final 

Before-and-After Study Report, 

supported by professional opinion 

M3. Review of the PMOC's 

presentation of findings, analysis, 

recommendations and professional 

opinions by the FTA. 

MM2f. Periodic 


its agent. 

MM3. Periodic 


its agent. 


Rev. 2, May 2010 

Page A-2

APPENDIX B 

List of Typical Documentation for Capital Costs and Project Scope Provided by Grantee 

__________________________________________________________________________________ 

Stage 

Entry into PE (or PD 

for Small Starts) 

Entry into Final 

Design 

Application for 

FFGA (or PCGA 

for Small Starts) 

Prior to Start of 

Revenue Service 

Document Name 

• Alternative Analysis Report 

• Draft NEPA Documents (Environmental Impact Statement (EIS), Environmental 

Assessment (EA), or Categorical Exclusion (CE) 

• Technical Memoranda, Engineering Design Reports , Facilities Design Reports 

• Concept Design Drawings 

• Cost estimate in original and SCC format; Capital Cost Methodology Report 

• Project Schedule 

• PE Grant application 

• Draft Before-and-After Study Plan (Recommended) 

• Final NEPA Documents (Environmental Impact Statement (EIS), Environmental 

Assessment (EA), or Categorical Exclusion (Catex) 

• Record of Decision (ROD), Finding of No Significant Impact (FONSI) 

• Annual New Starts Update – Volume 1: New Starts Data 

• PE Drawings, Specifications 

• Basis of Design Report, Utilities Report, Permitting Report 

• Real Estate Acquisition Plan 

• Cost estimate in original and SCC format; Capital Cost Methodology Report 


• FD Grant Application 

• Analysis of Changes Since Last Milestone (Required) 

• Draft Before-and-After Study Plan (Recommended) 

• Interim Before-and-After Study Report (Recommended) 

• FD Drawings, Specifications 

• Basis of Design Report, Utilities Report, Permitting Report 

• Real Estate Acquisition Plan 

• Request for Bids or proposals; construction contracts and addenda 

• Letters of No Prejudice 

• Cost estimate in original and SCC format 


• FFGA or PCGA application 


• Final Before-and-After Study Plan (Required) 


• FFGA or PCGA Amendments (if any) 

• Construction documents; as-built drawings to the extent they are available 

• Cost breakdown in original and SCC format 

• Executed change orders; construction contract amendments; claims 

• Project Schedule and Construction Schedule Adjustment(s) 




Rev. 2, May 2010 

Page B-1

Stage 

2 Years After 

Opening 

Document Name 

• As-built drawings 

• Cost breakdown in original and SCC format 

• Executed change orders; construction contract amendments; claims; settlements 

• Project Schedule and Construction Schedule Adjustment(s) 

• Final Before-and-After Study Report (Required) 


Rev. 2, May 2010 

Page B-2

APPENDIX C 

Report Outline 

______________________________________________________________________________ 

I. Introduction 

a. Project Description 

b. Project Chronology and Milestones 

i. History 

ii. Entry into Preliminary Engineering 

1. List of associated documents 

iii. Entry into Final Design 


iv. Full Funding Grant Agreement 


v. Next Steps 

1. Schedule for Before & After Data Collection 

a. Before 

b. After 

c. Procedures for Archiving Forecasting Procedures and Datasets 

Volume 1: Service Levels, Operating Costs and Ridership Forecasts (PMOC is not required to 

Review Volume 1) 

Volume II: Project Scope and Capital Costs 

I. Project Scope and Capital Costs 

a. Entry into Preliminary Engineering 

i. Physical description of system 

1. Length 

2. Stations 

3. Parking facilities 

4. Other elements 

ii. Capital Cost estimation methodology 

iii. Project Capital Costs by element 

1. FTA SCC Worksheets 

2. Total estimated costs 

b. Entry into Final Design 

i. Physical scope 

1. Changes by element since entry into PE 

2. Overall description of the current system 

ii. Capital Cost estimation methodology 

1. Highlight any methodology changes from entry into PE 

iii. Project Capital Costs by Element 


2. Changes from entry into PE 


Rev. 2, May 2010 

Page C-1

a. Change amount, reason for change (inflation, materials, change 

in scope, etc.) 

3. Total project cost estimate at Entry into Final Design 

c. Full Funding Grant Agreement 


1. Changes by element since entry into FD 

2. Overall description of current system 

ii. Describe capital cost estimation methodology 

1. Highlight any changes in methodology since entry into FD 

iii. Project capital costs by element 


2. Changes from entry into FD 

a. Change amount, reasons for changes 

3. Total project capital cost estimate at time of FFGA 

d. Prior to Start of Revenue Service 


1. Changes by element since award of FFGA 



1. Highlight any changes in methodology since award of FFGA 



2. Changes from award of FFGA 


3. Total project capital cost estimate prior to Start of Revenue Service 

e. Two Years After Opening 


1. Changes by element since prior to start of Revenue Service 



1. Highlight any changes in methodology since prior to start of Revenue 

Service 



2. Changes from prior to Start of Revenue Service 


3. Total project capital cost estimate prior to Start of Revenue Service 

II. 

III. 

Summary 

a. Trend in Capital Costs and Scope 

b. Conclusion 

Appendices 

a. List of referenced publications 

b. List of project information 

c. CD/DVD or hard drive containing project information 


Rev. 2, May 2010 

Page C-2

OP27 Before-and-After Study Review - Federal Transit Administration

Create successful ePaper yourself

Delete template?

Save as template?