Parametric Analysis Overview (PT 01)
Jason Dechoretz – Chief Strategy Officer, MCR, LLC
Rick Garcia – Technical Director, MCR Federal
This introductory session provides an overview of parametrics. It covers the process of building parametric models and describes how those models are used. The training session discusses cost estimating relationships and introduces the topics of:
• Data Characteristics and Sources
• Data Normalization
• CER Development
• CER Validation
• Complex Models
• Hardware
• Software
• Complex Tools in the Estimating Process
• Development of Complex Tools
And finally, the Parametric Analysis Overview asks the question:
• How good is the estimate?
CEU Credit: 0.15
Keywords: Data Characteristics and Sources, Data Normalization, CER Development, CER Validation, Complex Models, Hardware, Software, Complex Tools in the Estimating Process, Development of Complex Tools
Cost Estimating Relationships (CERs) (PT 02)
Cassandra Capots – Cost Analyst, Technomics, Inc.
Kyle Thomas – Operations Research Analyst, TASC, Inc.
Cost Estimating Relationships (CERs) are critical to cost analysis, as they are one of the key tools available to analysts. At the most basic level, CERs are mathematical expressions developed using historical cost and technical data to relate cost with other independent variables. The Cost Estimating Relationships training module (PT02) will provide key definitions, describe the CER development process (i.e., regression analysis), and how to test for significance. A walkthrough of several notional CERs will be provided, including an example analysis highlighting the basics of creating a CER. The session will close with a discussion of CER validation and some advanced topics, including CER calibration.
Linear Regression (PT 03)
Peter Braxton – Senior Cost Analyst and Technical Officer, Technomics, Inc.
Stacy Dean – Senior Procurement Finance Analyst, Boeing
Upon completion of this session, the student will be able to:
• Determine the linear regression equation for a bivariate data set using ordinary least squares (OLS);
• Calculate appropriate statistics related to the linear regression and use them to determine goodness of fit and statistical significance of the model
• Use these mathematical modeling techniques to create Cost Estimating Relationships (CERs) from provided normalized data.
This session covers the linear regression subset of Module 08 Regression Analysis of CEBoK.
Multivariate Regression (PT 04)
Kevin Cincotta – Technical Director, ICF International
David Harris – Booz Allen Hamilton
Upon completion of this session, the student will be able to:
• Apply logarithmic transformations to enable the determination of best fit using OLS for data exhibiting power, exponential, or logarithmic functional forms;
• Determine the linear regression equation for a multivariate data set using ordinary least squares (OLS);
• Calculate appropriate statistics related to the linear regression and use them to determine goodness of fit and statistical significance of the model; and
• Use these mathematical modeling techniques to create Cost Estimating Relationships (CERs) from provided normalized data.
This session covers the non-linear and multivariate regression subsets of Module 08 Regression Analysis of CEBoK.
Multiplicative-Error Regression (PT 05)
Herve Joumier – Head of ESA Cost Engineering, European Space Agency
Development of cost-estimating relationships (CERs) from historical data has, in the past, been based on explicit solutions of the classical least-squares linear regression equation Y = a+bX+E, where Y is cost, X is the numerical value of a cost driver, E is a Gaussian error term whose variance does not depend on the numerical value of X, and a and b are numerical coefficients derived from the historical data.
The coefficients of nonlinear forms such as Y = aXbE are derived by taking logarithms of both sides and reducing the formulation to log(Y) = log(a)+ b*log(x)+log(E). This approach has a number of well-documented weaknesses, one of which is that it a priori excludes from consideration certain potentially attractive nonlinear forms, such as Y = a+bXc, because a logarithmic (or any other reasonable) transformation fails to reduce the problem to the classical linear-regression format.
All known weaknesses can be circumvented by applying “general-error” regression, which allows the analyst to determine the optimal coefficients for any curve shape and to choose the error model independently of the CER’s shape. The optimal (error-minimizing) solution is found by sequential computer search rather than by explicit solution of simultaneous equations, as in the classical regression methods. This tutorial focuses on development of CERs having multiplicative (i.e., percentage) errors, rather than additive (i.e., dollar-valued) errors.
CER Risk and S-Curves (PT 06)
Christian Smart – Director of Cost Estimating and Analysis, Missile Defense Agency
Marc Greenberg
This training session compares S-Curves generated using standard errors of the estimate (SEE’s), confidence intervals (CI’s), and prediction invervals (PI’s), arguing for PI’s as the appropriate technique. It is shown how and why SEE-based techniques, while commonly used, almost always underestimate both risk and uncertainty. Finally, we provide a step-by-step guide on how to generate PI-based S-curves, with the mathematical intuition behind each step.
Complex Hardware Models (PT 08)
Greg Kiviat – Sikorsky Aircraft
David Bloom – Senior Engineering Manager, Raytheon Space and Airborne Systems
The Complex Hardware Models session (PT08) of the ICEAA Parametric training track provides a short history of complex model parametric estimating and continues with an overview, examples and recommendations to develop Basis of Estimates (BOE for Rough Order Magnitude (ROMs), Business Case Analysis, and Proposals. Best practices and lessons learned from the Parametric Estimating Reinvention Laboratory and personal experience provide insight into practical applications and limitations of the toolsets.
Government Compliance (PT 10)
Joyce Friedland – Program Manager, Defense Contract Audit Agency
The use of parametric estimating techniques has been accepted for many years. Industry and government drafted the first Parametric Estimating Handbook. Yet, many companies and government organizations are reluctant to use the parametric tools and techniques available to them. This presentation provides a brief overview of Chapter 7, Government Compliance, of the Parametric Estimating Handbook. Here we will discuss the requirements of the Truth in Negotiations Act (TINA), Federal Acquisition Regulations (FAR), Defense Federal Acquisition Regulations Supplement (DFARS), and the Cost Accounting Standards (CAS) as they pertain to parametric estimating practices and techniques when cost or pricing data are required to establish fair and reasonable prices. Additionally, we will discuss the Government’s perspective when auditing and otherwise reviewing parametric proposals, pricing proposals in general and estimating systems. Attendees should leave the presentation with the knowledge that properly calibrated and validated parametric models are acceptable for use in preparing proposals for government contracts and represent a significant opportunity for industry and government to move away from proposals based on significant “judgmental estimates” or “expert opinion” based estimating techniques.
Other Uses of Parametrics (PT 11)
David Eck – Director, Dixon Hughes Goodman
This training will address what to consider when developing and implementing parametric estimating tools. We will also give descriptions of general and specialized applications where parametric tools can be used. We will show examples of where and how these techniques have been implemented.