(RISK-2394) An Introduction to Analysis for Small Data Populations
Authors: James D. Whiteside, II PE FAACE; Gregory J. Whiteside, PE CCP
Abstract: Standard practices for statistical and probability analysis are not good applications for small data populations. Two better tools are introduced for small data population analysis: the discrete Fourier transform and phase space to perform statistical and probability analysis. Rather than rely on estimating practices (statistical and probability analysis), phase space and Fourier transform are objective data-driven empirical analyses to directly compute probability distributions, model data, forecast and evaluate for cyclical responses.
Phase space analysis is an empirical solution for data modeling, risk assessment, statistical analysis and an alternative tool to multivariate regression. People are pre-disposed (biased) to look for coincidences. The basis of estimating practices (statistics and probability) assumes data is stochastic (random) when, in fact, data is engineered, collected and analyzed by people who are behavioral. If a cyclic condition is determined to exist, then the root cause or trend may be behavioral.
(RISK-2404) Holistic Project Assessment (HPA)
Authors: Ken Cressman, CCP EVP
Abstract: Assessment of the risks, rewards and projected success of a program or project will normally fall within established organizational practices, such as “Risk Assessment” or “Financial Assessment”. This paper will refer to a “project” in the industry established definition. The Holistic Project Assessment (HPA) process described can be applied to both projects and programs. HPA can provide an additional assessment tool for both organizations and individuals to use when assessing the viability of a project, as well as their projected role within that project.
(RISK-2406) Practical Risk Management for Airport Projects Case Study
Authors: Hossam A. Eid Mohammed
Abstract: Risk management including identification, assessment, and prioritization of Risks can be one of the most effective tools in controlling Project execution.
In Strategic projects such as airports, applying risk management tools can secure the reliability of the project schedule, budget and minimize operation problems by implementing a realistic project management plan.
With a case study for a real life project, the paper will introduce how the project management team implemented risk management procedures and used risks management tool to enhance the project level of confidence, achieve targeted level of confidence and indicate the correlation between the project major risks and the impact of these risks on the project time and cost estimate to implement the proper risk responses.
(RISK-2441) Integrated Cost-Schedule Risk Analysis Using Monte Carlo Simulation
Authors: Dr. David T. Hulett, FAACE
Abstract: Originally cost contingency calculations were based on historical ratios and did not refer to schedule risk.
Early cost risk analyses were based on Monte Carlo simulation (MCS) of cost models created in spreadsheets. Generally schedule risk was not seen as a needed input.
Gradually schedule variability was introduced to cost risk models. Sometimes the schedule contingency in days beyond the scheduled finish date could be applied to the indirect costs or the average daily cost of the entire project.
Soon, simulating a schedule could produce a probability distribution of possible schedule dates. These schedule-risk iteration data could be represented as a distribution within the spreadsheet MCS software and applied in the cost risk simulation.
Today integrated cost-schedule risk analysis is practiced on schedule platforms with specialized software that simulates schedules and the corresponding cost for each iteration. Implementing the Joint Confidence Level (JCL) approach to risk allows estimating the targets needed for achieving both objectives.
(RISK-2442) Numerical Bias of Risk Teams vs. Project Actual Bid Results
Authors: David A. Hamilton, PE CCP
Abstract: Are risk analysts and constructability review teams perpetually pessimistic or does something happen between the time of the risk study and bid day? True, those of us involved with quantitative risk analysis have a critical perspective and survey our surroundings looking for the “what ifs” of this world to complicate our projects. Sometimes it’s a fact of remembering a project that went awry or a combination of facts that may lead to either cost or schedule complications. A subtle tension sometimes arises among the risk team, engineer and owner since risk projections can severely impact budgets, project goals and stakeholder expectations.
The scope of this paper explores the dynamics of risk identification, risk distribution types and the resulting impact upon design teams, project budgets and owners. The design industry has been supporting the use of risk analysis and more project risk events are being identified and mitigation measures are being incorporated prior to construction bidding. A generic case study of a water treatment plant project will highlight the risk bias and its intricate effects upon the design team and the owner’s project management (PM) staff.
(RISK-2445) Risk and Uncertainty Best Practices in GAO’s Best Practice Guides
Authors: Karen Richey; Jason T. Lee
Abstract: The U.S. Government Accountability Office (GAO) is responsible for assisting the Congress in its oversight of the federal government, including agencies’ stewardship of public funds. With the help of industry and government experts, GAO has published a series of best practices guides related to program management. The GAO Cost Estimating and Assessment Guide established a consistent methodology based on best practices that analysts can use to develop and manage program cost estimates. The GAO Schedule Assessment Guide is a companion to the cost guide and presents ten best practices associated with developing and maintaining a reliable schedule. In this paper, we describe GAO’s development of its best practices guides, with a focus on cost and schedule risk and uncertainty analysis leading practices.
(RISK-2459) Closing Panel Discussion of DRM Special Track on Quantification Methods
Authors: James Arrow; Colin Cropely; David Hulett
Abstract: The DRM committee is putting together a special 2-day track of presentations and panel discussions on the subject of Project Cost and Schedule Risk Quantification Methods: Alternative Methods. A closing panel discussion will use a compiled table as a focal point of discussion to summarize the papers presented over the two days.
The purpose of the track is to present in a single venue, representations of the leading project cost and schedule risk quantification methods in use by various parties and industries in different settings. Presenters are to summarize their approach as they see fit showing how their method addresses the RP 40R-08 principle attributes in a simple table template. Each paper is to include an appraisal of the strengths and weaknesses of the subject method for the presenters situation while avoiding criticizing other methods. In this way, attendees will be introduced to the methods while also being given a consistent way to compare how each may best fit their needs and requirements. One intended outcome of these discussions is to revise 40R-08.
(RISK-2470) From Three-point Ranging to Risk Driver Approach
Authors: Craig Veteto
Abstract: This paper focuses on the ranging techniques applied to cost and schedule risk analysis. More specifically, it will explore the risk driver approach and three-point ranging when conducting cost and schedule risk assessments. There will be a brief overview of the two approaches with a comparison of the key aspects of each. There will be an in-depth discussion of the reasons for a shift in approach from three-point ranging to risk driver approach. The challenges faced from a practitioner perspective as well as those experienced while persuading other stakeholders within the organization will be presented. There will be a discussion of the real-world results of the newly adopted approach. The risk driver approach is not a wholesale revamp of the cost and schedule risk analysis method, but a change in the ranging technique that produces an improved end product. The risk driver approach is not new, and papers published previously have described the technique. This paper will reinforce the benefits and provide insight to the challenges of converting from a legacy approach.
(RISK-2473) Lessons Learned in Completing a Tailored Project Risk Management Assessment Tool
Authors: Bryan A. Skokan, PE CCP; Connie (Wei) Liu; Rodney Lehman; Jake Lefman; Dan Melamed, CCP EVP
Abstract: The Office of Environmental Management (EM) of the U.S. Department of Energy developed a tool to help risk/project managers estimate the reasonableness of their individual risk probability and risk impact estimates for common environmental cleanup project risks. A database was designed, developed, and populated with risks extracted from the risk registers of a variety of both ongoing and completed EM projects for all phases of project cycles. The function of the tool is to help determine the adequacy of project risk for new projects and in addition, to serve as a means to incorporate lessons learned from past risk approaches and mitigation. It also provides a means to evaluate the reasonableness of the entirety of risk registers with respect to the common risks. This tool allows the user to evaluate and compare their common project risks to the probability and impact estimates of others for similar risks. The tool makes automatic adjustments to the data to compensate for differences in project sizes and increase comparability accuracy to the target project. Lessons learned in developing this tool and analyzing historical project risks will also be discussed.
(RISK-2488) A Rationale for Lean Models for Quantitative Schedule Risk Analyses
Authors: Luis O. Figueroa
Abstract: The reliability of quantitative schedule risk analyses (QSRA) outcomes is dependent upon using robust risk models. These models must fairly represent the project plan, be simple to follow, contain a level of detail where duration uncertainties and risks can be meaningfully assigned, and be feasible to infer from historical data. This paper expands arguments in support of fit for purpose summary schedules as an appropriate type of QSRA risk model for large EPC projects. To that end, it discusses issues associated with the stability of Critical Path Method (CPM) networks. Using examples from actual projects, it explores how those issues curtail the usability of detailed schedules CPM networks as risk models, and precludes the use of historical data as a source of risk estimates evaluation. Then it explains how lean, fit for purpose summary risk models, can be designed for improving CPM network stability and can make historical data inferences more feasible to use, thus increasing the predictability of QSRA outcomes.
(RISK-2498) LA Metro Implements ProjectStatus for Collaborative Risk Tracking
Authors: Julie K. Owen, CCP PSP; David Davies; Brian Criss, PSP
Abstract: Los Angeles Metro is implementing the largest transportation infrastructure program in the United States. Measure M is a sales tax initiative that commits $120 billion into transportation upgrades over the next fifty years and is the largest investment in Los Angeles County infrastructure since the 1980’s.
As part of the overall program strategy, Metro desired to provide for a centralized solution for tracking project risks related to the Measure M program. A collaborative team was developed consisting of the transit client, risk management consultant and technology partner in the development of the overall solution.
The implemented solution consisted of ProjectStatus a proprietary web-based, centralized solution whereby all risks could be tracked and stored in a centralized location, namely Oracle Primavera P6.
This paper discusses risk management requirements as mandated by the Federal Transit Administration (FTA) and the rationale behind the implemented solution and the benefits realized for Los Angeles Metro.
(RISK-2505) Fully Integrated Cost Schedule Model – A Stochastic Alternative
Authors: Liwen Ren; Gustavo Vinueza C.
Abstract: The proposed fully integrated Cost and Schedule Contingency Model incorporates several major inputs in a multibillion project: Cost Budget, Tasks Duration in a Gantt chart, Risk Registers, and Contingency Calculation. On top of this blend, a probabilistic layer was included, which internally integrates all these elements.
The model permits configuration of the different modules and by using Monte Carlo Simulation, makes the stochastic calculation possible. Important to consider is that simultaneously generates samples for Cost and Schedule Uncertainties, Risk Register Occurrences and Impacts, and generates potential Tasks occurring in the future, related to specific risks. The result is a probabilistic forecast of project contingency, with strong capability to drill down into each individual item and analyze its significance level to the contingency.
This work describes the design and development process with collaboration between a modeling specialist (consultant) and a project risk management specialist (owner), and proposes a methodology to replicate into any infrastructure project. This methodology will help Risk Analysts to keep focus on risks, and support Senior Management decision making and project approval.
(RISK-2510) Modelling Realistic Outcomes using Integrated Cost and Schedule Risk Analysis
Authors: Colin H. Cropley
Abstract: After several decades of development of project quantitative risk analysis for time and cost forecasting of project outcomes, achievement of consistently accurate results has remained elusive. Realistic time forecasts of projects have been found to be forecast well by sound Monte Carlo Method based SRA processes integrating schedule impact risk events. But realistic cost forecasting has proven more problematic. This paper demonstrates the capacity of Integrated Cost & Schedule Risk Analysis (IRA) to utilise parametric methods to represent and model systemic risks and to include decision-making during iterations to represent realistic treatment of project risks. But IRA can be extended to incorporate modelling of operation of the assets created by the project to produce probabilistic cashflows based on Capex and Opex uncertainties and risk events as well as revenue uncertainties and risk events, escalation and exchange rate uncertainties and risk events. In this way integrated modelling and scenario analyses can embrace the Total Cost Management framework envisaged by AACE to support project FID.
(RISK-2511) Role of Project Novelty, Execution and Bias in Risk Quantification
Authors: Dr. Yuri Raydugin, P.Eng.
Abstract: Selection of most accurate and efficient cost and schedule risk quantification and contingency estimating methodology is undertaken. It is based on review of existing methodologies against their capability to adequately handle fundamental project uncertainties, project novelty, features of execution strategy, and various realizations of psychological and organizations bias. A notion of ‘methodology’s distance to project reality’ is introduced to facilitate a selection process.
It is accentuated that risk quantification is one of several important project risk management process steps that provide required inputs to quantification.
Fundamental mission of contingency estimating is revised and redefined from ‘prognostication’ to development of credible and achievable stretched targets.
Primary accuracy ranges (PAR) of base estimates are introduced; their relations with AACE® International classes are established. PAR’s introduction indicates a shift from parametric and other deterministic methodologies to probabilistic ones that are coming of age.
Specification of a most accurate and efficient hybrid risk quantification and contingency estimating methodology is proposed as a result of this paper. A practical example of methodology’s application is demonstrated.
(RISK-2515) Realistic and Practical Project Risk Quantification (without CPM)
Authors: John K. Hollmann, PE CCP CEP DRMP FAACE Hon. Life
Abstract: From 2007 to 2013, the AACE® International Decision and Risk Management (DRM) committee produced several Recommended Practices (RPs) for risk analysis and contingency determination methods that aligned with principles documented in RP 40R-08. These included RP 42R-08 for the Parametric Estimating method for systemic risks and RP 65R-11 for the Expected Value method for project-specific risks. This paper describes how these two methods can be applied together in a way that is both realistic (i.e., yields empirically valid outcomes) and practical (i.e., can be applied on any project at any phase regardless of the project size or quality of base estimates and schedules.) The methods integrate cost and schedule risk analysis (i.e., outcomes can be presented with a Joint Confidence Level chart) without reliance on critical path method (CPM) schedule models. Finally, the methods can be implemented using spreadsheets with a Monte Carlo Simulation add-in (in fact, RP 43R-08 provides working parametric model spreadsheets.) The paper centers on a table summarizing how the methods best address the risk quantification principles in RP 40R-08.
(RISK-2516) Variability in Accuracy Ranges: A Case Study in the Canadian Power Transmission Industry
Authors: John K. Hollmann, PE CCP CEP DRMP; Andrea Almeida; Raminder S. Bali, P.Eng.; John M. Boots, P.Eng.; Kevin Burnham; Victor Hsieh; Qaiser Iqbal, P.Eng.; Donald Konan, P.Eng.; Guillaume Lafortune, P.Eng.; Joe Ly; Clement Wu, P.Eng.
Abstract: This paper presents a case study of the variability in accuracy ranges for cost estimates in the Canadian overhead power transmission industry. The study sought to improve the participant’s understanding of risks and estimate accuracy for their major overhead power transmission projects. The study team also sought to verify the theoretical accuracy curves identified in the AACE® International Recommended Practice (RP) 18R-97: “Cost Estimate Classification System – As Applied in Engineering, Procurement, and Construction for The Process Industries”. The study team collected and analyzed actual and phased estimate cost data from 39 projects with actual costs from 2 million to 655 million (2016$CAN) completed from 2007 to 2016. Greenfield and brownfield overhead transmission projects from across Canada were included. This study compares the range bandwidth (uncertainty) as stipulated in AACE® RP 18R-97 with the actuals from this study. The accuracy ranges and the project’s under or over estimation of contingency is compared with published data from other industry studies.
(RISK-2523) The Application of Chaos Model in Mega Project Risk Analysis
Authors: Jin Feng; Xiang Wenwu; Zhang Bochen
Abstract: Mega projects gradually show the phenomenon of huge cost overrun and schedule delay with the characteristic of chaos. In order to analyze the tipping point of changing from the disorder state to the order state as could be controlled by the project manager, the risk drivers of mega project are classed into three categories including complexity, stressor and mis. risks. Based on this classification, two new models of project risk analysis were respectively proposed to calculate contingency for only considering complexity and stressor and only mis. risks. Meanwhile, the tipping point was brought forth to make effective of the risk control. Eventually, one case was studied to illustrate the effectiveness and validity of the proposed models.
(RISK-2555) Contingency Cage Match: Simultaneous Contingency Assessment Methods, A Case Study
Authors: Matthew Schoenhardt, P.Eng.; Vachel Pardais
Abstract: Of the four main methods of contingency assessments support by AACE, two offer reasonable sophistication and detail: recommended practice 41R-08 Range estimating (Monte Carlo simulation); and, 42R-08 Parametric estimating (systemic analysis). Both offer probabilistic results, risk register integration and leverage project team’s knowledge and expertise. There are proponents within AACE of both methods who expound their favored approach (this Author being no exception). The ideal rapprochement is to use both methods simultaneously to offer two sets of “data points” on possible project cost outcomes. In the real world, budgets and project team availability, preclude this contingency Utopia.
In 2012, the Author had the opportunity to simultaneously complete both methods on a large, now substantially complete, project. This paper will review the project’s post mortem implementation of the two contingency approaches; their predictive results against actual results; and, consultant and project team hours expended for each method.
Who will win this Contingency Cage Match? Will there be a clear winner? This paper will review the effort – accuracy relationship between the two approaches along with other time saving methods.
(RISK-2570) An Engineering Approach to Schedule Risk Assessment
Authors: Angela Tuffley; Adrian Pitman; Dr. Elizabeth Clark
Abstract: The Schedule Compliance Risk Assessment Methodology (SCRAM™) is an engineering-focused framework for reviewing complex programs to schedule drivers and identify root causes of schedule slippage, recommend remedial actions and forecast future milestone dates.
SCRAM provides executive decision makers with an understanding of why programs are slipping, the probability of achieving major schedule milestones schedule and what practical actions can to taken to mitigate risk and remediate the impact of issues to minimize further slippage.
SCRAM has been successfully applied to over 30 different major acquisition programs encompassing a variety of domains such as aerospace, maritime, communications, aircrew training, satellite ground control, and command and control. SCRAM has been applied at various points in the program lifecycle including pre-contract award, prior to Earned Value Management (EVM) Integrated Baseline Review (IBR), System Integration and Test as well as production and sustainment.
This paper provides an overview of the SCRAM method, the Root Causes Analysis of Schedule Slippage (RCASS) model and SCRAM’s use of Schedule Risk Assessment (SRA) then summarizing key issues seen on programs reviewed to date.
(RISK-2592) Integrated Cost/Schedule Risk Analysis for Pre-Concept Alternatives Analysis/Technology Selection
Authors: Samuel Steiman, PE; Molly Donovan
Abstract: Quantitative risk analysis is commonly performed for mature projects with a well-defined baseline to estimate appropriate contingency reserves. This paper demonstrates how the integrated cost/schedule risk analysis process described in AACE® International Recommended Practice 57R-09 can be applied to projects on the opposite end of the maturity spectrum, in the pre-concept phase, where alternatives analysis and technology selection are driving considerations.
The application is a company evaluating advanced nuclear reactor technologies in different sizes/configurations to commercialize a full-sized multi-module power plant. The integrated cost/schedule risk results provide insights not available from the best-guess/deterministic cost/schedule estimates, and are key to identifying the preferred alternative from a cost/schedule perspective.
This paper describes the process for building a functional cost/schedule model, demonstrating a practical application of the 57R-09 guidelines for mapping a billion dollar cost estimate into summary-level resources, building a schedule at the appropriate level of detail, adding risks/uncertainties, and interpreting the results unique to integrated cost/schedule analysis. Finally, this paper presents a unique approach to applying probabilistic cashflow results to develop a recommended project funding plan.
(RISK-2596) Backward-Looking Schedule Risk Analysis - Determining Acceptable Variability in Project Start
Authors: Molly Donovan; Samuel Steiman, PE
Abstract: Typical schedule risk analyses are “forward-looking” to provide confidence levels in finish dates when the project start date is defined. But what if the finish date is mandated and there is uncertainty in the project start? This paper documents a unique “backward-looking” application of schedule risk analysis for a combined-cycle natural gas power plant where the owner performs commissioning following the hand-off of system turnover packages (TOPs) from the EPC contractor. The owner had concerns that the TOP delivery dates were uncertain and the liquidated damage dates for the TOPs were insufficient to protect the schedule contingency required for commissioning and start-up. This probabilistic model of the integrated schedule considered uncertainty in the EPC TOP dates, as well as the owner’s commissioning activities. Through various sensitivity runs, the analysis identified the most critical TOPs driving the commercial operation date to focus negotiation efforts with the EPC, and identified areas within the commissioning schedule where mitigation efforts would be most impactful. The owner’s initiative to explore creative “backward-looking” analyses proved valuable in protecting contingency.
(RISK-2694) An Innovative Application of an Escalation Risk Mitigation Approach
Authors: Warren Fletcher
Abstract: The potential for project proponents to use inappropriate escalation rates represents a significant risk to any organisation assessing projects for funding from a limited capital pool. This is the case for the Australian Department of Infrastructure and Regional Development, which assesses project proposals, many with long delivery timeframes, from each state and territory, and subsequently manages, on behalf of the Australian Government, funding for approved major infrastructure projects. This paper describes the innovative application of an escalation risk mitigation approach, part of a suite of cost estimation guidance that the Department is developing, which utilises specialist economic forecasters, informed by experienced cost estimators, to develop a range of road infrastructure project composite quarterly index series from which annual historical and forecast escalation rates specific to each state and territory can be derived. The paper will explain how these composite index series draw on forecasts and actuals for accessible subordinate index series relating to key project cost drivers and how consistency of application is achieved via a robust escalation calculation tool that proponents populate.
(RISK-2700) Quantitative Cost Risk Analysis at Bid Stage (for selecting the most suitable offer)
Authors: Didier Lagrange
Abstract: Bids submitted for main EPC contracts in the Oil & Gas Industry generally contain a number of qualifications or exclusions associated with the proposed price. The project execution strategy foreseen by each bidder is generally not clearly defined and may induce additional cost during the project execution. In order to better assess what is associated to the contractors’ offers, a detailed quantitative cost risk analysis at bid stage will help. It will also support the level of contingency required for each bidder considering the risk the Contractor and the Owner respectively will have to bear. The risk manager will be heavily involved in the bid evaluation process in order to register and assess all risks associated with each of the bids including the options not necessarily validated at this stage.
(RISK-2701) Risk Dependency Analysis in Complex Projects
Authors: Paolo Cavanna; Franco Caron; Filippo Fratoni
Abstract: The rising complexity in projects, the dependencies and the interdependencies between major risks are critical issues for large engineering projects. Current Project Risk Analysis and Management (PRAM) does not clearly manage relationships between risks, though threats are not independent in complex projects. For that reason, a method was developed in order to support PRAM, introducing the phases of Dependency Identification, Dependency Assessment, Dependency Quantification. The aim of Dependency Identification is to identify risk dependencies and interdependencies with a Design Structure Matrix (DSM) and gather them in clusters. In Dependency Assessment, an unconventional approach considers risk dependencies as events with their own probabilities. A procedure for evaluating probability of occurrence of dependencies has been developed and the effects of risk dependencies are quantified during Dependency Quantification. The output of this method is a tool estimating with Monte Carlo delays and cost overruns caused by dependencies between major risks. An analysis of six Oil & Gas projects tests this method out.