Below are copies of all of the presentations and handouts from this year’s Workshop. We encourage you to share them with colleagues or others who may find them of use, but ask that the files not be distributed to large groups or used for demonstration without the author’s permission. If you are unsure whether or not an intended use of these files is appropriate, please email us at iceaa@iceaaonline.org
Authors and presenters: many presentations were changed just before the Workshop or other versions of the presentations were used. If the file here is not the latest version, please email the new version to us and we will post the new file here.
Use the links in the table below to easily navigate to your desired track or presentation.
Professional Papers & Presentations
Government Processes Track
Health CERs – Using an interdisciplinary approach to estimating the cost of caring for Canada’s veterans
Erin Barkel
Carleigh Malanik
Government Processes Track
GP01 Presentation Barkel Health CERs
Reports from the Veterans and the National Defence and Canadian Forces Ombudsmen have brought attention to the prevalence of mental illness among veterans. However, there are no published estimates for the cost of treating this growing population. A key obstacle to this research is that the relevant data is dispersed across several government organizations. We fill this research gap by leveraging actuarial and epidemiological studies of the veteran population to create new cost estimating relationships.
NASA Commercial Crew Cost Estimating – A Look at Estimating Processes, Challenges and Lessons Learned
Lance Cole
Rick Battle
Government Processes Track
GP03 Presentation Cole NASA Commercial Crew
To support annual PPBE budgets and NASA HQ requests for cost information for commercial crew transportation to the International Space Station (ISS), the NASA ISS ACES team developed system development and per flight cost estimates for the potential providers for each annual PPBE submit from 2009-2014. This paper describes the cost estimating processes used, challenges and lessons learned to develop estimates for this key NASA project that diverted from the traditional procurement approach and used a new way of doing business.
Federal Shared Services: The Future for Administrative Shared Services and Challenges Facing Shared Service Providers (SSPs) and Their Customers
Ruth Dorr
Marilyn Fleming
Sarah Pope
Government Processes Track
GP04 Presentation Dorr Federal Share Services
This paper will discuss the progress being made in the “Shared-First” approach for sharing of administrative services. Federal Agencies are being asked to execute their missions to ever increasing standards in a resource constrained environment. Administrative shared services have been identified as a way to reduce redundancy and improve efficiency in Federal business solutions. The identification of shared services as a Presidential cross-agency priority along with the designation of several providers for administrative shared services are recent developments in the movement toward shared services within the Federal government.
Lessons learned from applying cost capability curves on an Air Force AoA
Robert Georgi
Government Processes Track
GP05 Presentation Georgi Lessons Learned AoA
New direction from Air Force leadership established a requirement for the presentation of life cycle cost versus capability tradeoff analysis for all AoA final reports. This presentation examines the approach that was used for a recent AoA that was a pilot program for the Cost Capability Curve process. When appropriately implemented cost capability analysis can provide significant advantages. There are lessons learned from attempting to implement this approach. We will also discuss examples where this approach does not apply.
2015 Best Paper: Government Processes Track
The Navy Modernization Program: Estimating the Cost of Upgrading AEGIS Guided Missile Cruisers
Jeremy Goucher
Krystian Jenkins
Government Processes Track
GP06 Presentation Goucher Navy Modernization
GP06 Paper Goucher Navy Modernization
The Navy is well into its 20 year, $16B (CY10$) plan to modernize 84 AEGIS warships. This cost estimate covers eleven of the planned ships and accounts for maintenance and upgrades to the ship and the combat systems. The estimate leverages recent contract and shipyard performance data and interviews with engineers, resulting in a detailed study and recommended cost savings initiatives. The methods and data in this estimate will assist ship modernization cost efforts across the fleet for the foreseeable future.
Analysis of Alternatives from a Cost Estimating Perspective
Kirby Hom
Kenny Ragland
Government Processes Track
GP07 Presentation Hom Analysis Alternatives
GP07 Paper Hom Analysis Alternatives
Subsequent to the Weapon Systems Acquisition Reform Act of 2009, there have been other significant DoD policy changes that require Service sponsors to conduct an Analysis of Alternatives (AoA) prior to Milestone A. “About three-quarters of a program’s total life cycle cost is influenced by decisions made before it is approved to start development“ [GAO-09-665]. From a cost estimating perspective, a successful AoA rests with the study approach, overall costing methodologies, key insights from trades analyses, and lessons learned.
Mining for Cost Estimating Relations from Limited Complex Data
Mark Jacobs
Shawn Hayes
Government Processes Track
GP08 Presentation Jacobs Mining For Relations
NASA’s robotic Earth and space science missions cover a diverse group of projects addressing complex science objectives that include challenging implementation approaches. Progress applying Principle Component Analysis techniques covering project management, systems engineering, mission assurance, integration & test, and spacecraft subsystems is described. Supporting data analysis efforts include a large detailed set of technical and programmatic input candidates that are analyzed to identify the primary spacecraft cost drivers.
Manufacturing Assembly Plan (MAP) Tool: Bridging the Gap between Performance and the Construction Processes
Richard Lee
Haitham Ghannam
Edward Walsh
Government Processes Track
GP09 Presentation Manufacturing Assembly Plan
This presentation explains the four module build plan for the VIRGINIA Class Submarine (VCS) program and provides an in-depth look into labor hours by Section 1/2A, Section 2B/5, Section 6/7, Section 8/9, Final Assembly and Test (FAT), and Post Shakedown Availability (PSA). Further understanding these concepts will not only allow cost estimators and program managers to better understand the current construction build strategy, but it will also aid in better aligning cost estimates with reality at the deck plate.
It Ain’t Easy Being Green… Sustainable Manufacturing with an eye on Cost Avoidance and Stewardship
Kammy Mann
Jessica Boatwright
Government Processes Track
GP10 Presentation Mann Easy Being Green
GP10 Paper Mann Easy Being Green
The DoD, one of the largest energy consumers in the world, has committed to sustainability. An important element of sustainable weapon systems acquisition currently not under the purview of the DoD’s strategic plan is sustainable manufacturing. This paper will leverage current research and industry best practices to identify the impact of sustainable manufacturing on cost avoidances relating to human health and environmental benefits. Recommendations will also be provided for complying with the FAR sustainability requirements.
Business Case Analysis Wizard
Lauren Nolte
Cassie Capots
Marcie Pfeuffer
Government Processes Track
GP11 Presentation Nolte Business Case Analysis Wizard
This paper describes the purpose and design of a “Business Case Analysis (BCA) Wizard” that was created for the Air Mobility Command (AMC) Enterprise Learning Office (ELO). As the value of BCAs within the learning transformation at AMC has grown, ELO identified the need to develop a tool that allows “non-cost analysts” to develop high-quality BCAs. This “BCA Wizard” guides users through a standard process of conducting a BCA via a user-friendly interface.
Masters in Cost Estimating and Analysis
Dan Nussbaum
Greg Mislick
Government Processes Track
GP12 Presentation Nussbaum Masters Cost Estimating
This presentation provides an update on the all-Distance Learning Masters Degree and/or Certificate Program in Cost Estimating and Analysis (MCEA / CCEA) offered at the Naval Postgraduate School (NPS).
Methods Used in Pricing and Conveying NASA Assets for Use by Commercial Programs
James Roberts
Torrance Lambing
Government Processes Track
GP13 Presentation Roberts Methods Used NASA Assets
Different agreement types including EUL’s, Use Permits, Land Leases and others are used to convey NASA assets no longer needed by programs such as Space Shuttle and construction of the Space Station. By law, some situations prohibit NASA from pricing facilities in competition with the local market. Costs are estimated either via market-based or actual cost methods, according to circumstances. This paper will explain and clarify how these costs are estimated and pricing is established.
ATC Zero: Estimating the True Impact of the Chicago Control Center Fire
Kellie Scarbrough
Government Processes Track
GP14 Presentation Scarbrough ATC Zero
GP14 Paper Scarbrough ATC Zero
For seventeen days, the Chicago Air Route Traffic Control Center went offline after a contractor set fire to the intricate communications network that controls some of the busiest airspace in the country. Workers scrambled around the clock for seventeen days to restore functionality to the center. This paper seeks to determine the true economic impact of the event, including the costs of time, operations, and environmental impacts.
Where Does Your Cost Estimate Go?
Emily Stafford
Christopher Metz
Government Processes Track
GP15 Presentation Stafford Where Cost Estimate Go
You have spent several months developing your cost estimate and presenting it to the customer. Now what? This paper will track what happens to your cost estimate after it has been delivered to the customer. It will follow the estimate through the budget formulation process, ultimately becoming a part of the President’s Budget. It will also discuss the cost estimator’s role during the budget process, and the influence of the budget process on future estimates.
Facilitating Predictive Cost Analytics via Modelling V&V
John Swaren
Government Processes Track
GP17 Presentation Swaren Predictive Modeling VV
We are often asked to “fine tune” estimating software to unique situations. That “calibration” effort is key to Verification & Validation of parametric models. In this presentation, we show an example of using Predictive Analytics to develop a method to predict a Software-estimating model’s inputs as a function of Key Performance Parameters (KPPs). We demonstrate how a calibrated knowledgebase can best be used to V&V an estimating approach reflective of past and future requirement metrics.
Hyperbolic Discounting and Defense Acquisitions
Eric Lofgren
Government Processes Track
GP18 Presentation Lofgren Hyperbolic Discounting
GP18 Paper Lofgren Hyperbolic Discounting
Cost estimation has largely been approached as an engineering problem which can be solved using data and techniques. While these are undoubtedly invaluable considerations, the broader production process is an inherently open-ended system subject to human interactions which are ill-defined and non-deterministic. This paper explores how behavior and institutions affect cost outcomes through the avenue of hyperbolic discounting, a time-inconsistent model of personal valuations. It will be argued that evidence for hyperbolic discounting in both government agents and contractor management have negative effects on cost estimates and will proffer some solutions based on commitment schemes.
Methods & Models Track
Cost Estimating Challenges in Additive Manufacturing
Joe Bauer
Patrick Malone
Methods & Models Track
MM01 Paper Bauer Additive Manufacturing
MM01 Presentation Bauer Additive Manufacturing
Additive manufacturing possibilities are virtually endless: from rapid prototyping, to printing food, to recreating human tissue. This presentation highlights the challenges of estimating 3D printing processes and offers some solutions to the estimator. Challenges include: amortizing non-recurring tooling across multiple programs, capturing software-centric nature of 3D printing, printing speed, and printing media limitations. This presentation highlights the use of existing cost estimating tools and areas for future development.
Long Term Impact of Ship Concepts on Operating and Support Cost Affordability
Eric Buller
Stuart Merrill
Bryan Miller
Methods & Models Track
MM03 Presentation Buller Long Term Impact Ship Concepts
Achieving affordability can be challenging especially once a program enters full rate production. Developing accurate Operating and Support (O&S) cost estimates that reveal tradeoff impacts and sustainment risks provides decision makers with affordability analysis during key acquisition events. We intend to demonstrate through analysis the impact of legacy and new ship concepts on O&S costs and program affordability. We will examine the cost tradeoffs for each ship concept in each O&S cost element.
The Peculiar World of Sole Source Contracting; Maybe There is a Better Way
Sandy Burney
Shawn Larson
Methods & Models Track
MM04 Presentation Burney Peculiar World of Sole Source
In 2014, 21% of U.S. Government contracts were sole source, accounting for 31% of total contracted value. They typically require negotiation and certification of all cost estimates, which can take up to 18 months. This light-hearted presentation will explore the peculiar world of developing and negotiating sole source estimates, including differences from budgetary and competitive estimating, as well as offer suggestions for achieving fair and reasonable sole source prices for less time and money.
Migration of Microsoft Excel Tools to Next Generation Platforms: “Can You Hear the Footsteps?”
Jeremy Eden
Methods & Models Track
MM05 Presentation Eden Footsteps
Analysis tools are popular solutions for extended deployments and at the same time an acceleration of platform migrations is occurring to keep up with rapidly changing technology and end user demands. This paper provides specific examples of platform migration challenges, resulting failures, and how they can be best addressed. It will also provide specific best practices for ICEAA member take-away that can be followed to help ensure smoother transitions across platforms.
2015 Best Paper: Methods & Models Track
Generalized Degrees of Freedom (GDF)
Shu-Ping Hu
Methods & Models Track
MM06 Paper Hu Generalized Degrees
MM06 Presentation Hu Generalized Degrees
Minimum-Percentage Error/Zero-Percentage Bias (ZMPE) method is commonly used for multiplicative-error models. But ZMPE users do not adjust degrees of freedom (DF) for constraints included in the regression process. This generates misleading ZMPE CER fit statistics and underestimates the CER error distribution variance. Hence, ZMPE results are incomparable with those derived without constraints. This paper details why DF should be adjusted and proposes a Generalized Degrees of Freedom measure to compute fit statistics for constraint-driven CERs.
Bottom Up Methods of Estimating Depot Level Repairables
Tim Lawless
James Black
Dave Goldberg
Qing Qing Wu
Methods & Models Track
MM07 Presentation Lawless Bottom Up
Depot Level Repairables (DLR) costs comprise a significant share of total Operations and Support costs as they capture the scope of repairing and/or replacing failed hardware items. This paper will provide detailed descriptions of bottom up methods to estimate DLR costs using programmatic technical baseline documentation and acquisition costs as well as supply system requisitions and repair price data sources. Emphasis will be placed on producing data-driven analysis to inform cost estimates and estimate customers.
Seven Degrees of Separation: The Importance of High-Quality Contractor Data in Cost Estimating
Crickett Petty
Methods & Models Track
MM08 Presentation Petty Seven Degrees
MM08 Paper Petty Seven Degrees.pdf
The popular notion that any two people are linked to one another on average via a chain with “six degrees of separation” is based on a relatively small sample. Using massive data sets, researchers have since discovered that the average degrees of separation is closer to seven. This highlights the need for complete and thorough data in any analysis, and cost estimating in particular. Despite its importance the collection of high-quality contractor cost data is challenging. Processes for meeting these challenges are presented.
Biometric Analytics Cost Estimating
Joseph Sarage
Methods & Models Track
MM09 Paper Sarage Biometric Analytics.pptx
MM09 Presentation Sarage Biometric Analytics
The purpose of this paper is to present cost estimating methods as it pertains to the Biometric Field by which statistical analysis techniques can be applied to better predict the costs and relationships with Biometric data. The paper will explore analytical methods in which Biometric data can be used for trend analysis, forecasting, and strategic decision making within various organizations and applications.
Interviewing Subject Matter Experts
Melissa Teicher
Methods & Models Track
MM10 Presentation Teicher Interviewing SME
MM10 Paper Teicher Interviewing SME.pptx
As estimators, we often have to obtain technical information from Subject Matter Experts (SMEs) in order to provide thorough and accurate analysis. But sometimes, getting the information you need is easier said than done. This paper discusses the steps for interviewing SMEs in order to make the most of their time and your time. It explores the benefits of building a cooperative relationship of mutual respect and the “dos” and “don’ts” of SME consultations.
The Impacts of Design Change on Reliability, Maintainability, and Life Cycle Cost
Andreas Viberg
Oskar Tengo
Methods & Models Track
MM11 Tengo Presentation Impacts Design Change
When making decisions about a design change to a technical system, it is essential to understand how it affects reliability and maintainability, as that directly impacts both overall performance and cost of operations and maintenance throughout the life cycle. LCC analysis based on modeling and simulation of relevant scenarios is an effective way to gain that understanding. This paper presents an approach for such analysis and illustrates its successful application in a recent project.
A Prime Contractor’s View Using JCL Tools and Techniques
Chip Woods
Methods & Models Track
MM12 Presentation Woods Prime Contractor View
For several years now, Lockheed Martin has been integrating cost and schedule data into a single model to perform integrated analysis. This paper will discuss Lockheed Martin’s JCL process, the tools used to develop JCL curves, and how this analysis can be used to shape and steer the program – including modeling funding constraints.
Class List of Characteristics
Paul Breon
Methods & Models Track
MM13 Presentation Breon Class Characteristics
MM13 Paper Breon Class List Characteristics
CLOC (Class List of Characteristics) is a coding system wherein each digit of the code indicates one of the cost drivers for the product/activity. CLOC gives the ability to “Call a thing by what it really is” and creating a universal language that can be utilized for any purpose. CLOC does NOT include Bias like quantity, cost, etc. These things are used in conjunction with the code to do any kind of analysis needed.
Achieving Affordability through high fidelity ROM cost estimates
Eric Buller
Methods & Models Track
MM14 Presentation Buller Achieving Affordability ROM
Achieving affordability can be challenging especially if early decisions are made without proper cost estimates. Developing high fidelity ROM cost estimates and identifying capability tradeoffs during the requirements generation phase can be critical to the success of achieving long term affordability. We intend to demonstrate through analysis that preforming high fidelity ROM cost estimates with capability tradeoffs allow decisions makers to informed choices on mission requirements that achieve affordability for future programs.
Parametrics Track
A Regression Method for Sparsely-Populated Datasets
Daniel Barkmeyer
Parametrics Track
PA01 Presentation Barkmeyer Regression Method
A method is presented for including sparsely-populated independent variables in a regression analysis without introducing subjectivity or statistical bias. The method is shown to reproduce the result of a traditional regression method for a fully populated dataset, and to converges to the same result on a sparsely-populated dataset as that dataset decreases in sparseness. Application of the method to develop a new, empirically-derived model for spacecraft testing schedule duration will also be presented.
2015 Best Paper: Parametrics Track
Improvement Curves: An Early Production Methodology
Brent Johnstone
Parametrics Track
PA03 Paper Johnstone Improvment Curves
PA03 Presentation Johnstone Improvment Curves
Learning slope selection is a critical parameter in manufacturing labor estimates. Incorrect ex ante predictions lead to over- or understatements of projected hours. Existing literature provides little guidance on ex ante selection, particularly when some actual cost data exists but the program is well short of maturity. A methodology is offered using engineered labor standards and legacy performance to establish a basic learning curve slope to which early performance asymptotically recovers over time.
The NRO CER Analysis Tool
Donald MacKenzie
Parametrics Track
PA05 Presentation MacKenzie NRO CER Analysis
PA05 Paper MacKenzie NRO CER Analysis.pdf
The NRO Cost and Acquisition Assessment Group (CAAG) uses its CER Analysis Tool, CERAT, to inform CER development. CERAT analyzes CERs for sensitivity to influential data points (IDPs). The impacts on estimated costs for a target data point when data points are removed are the primary outputs. The changes in recorded CER constants when data points are removed also help in assessing overall CER “stability.” CERAT also provides several other aids to CER developers.
A Parameter Driven World
Jeffrey Platten
Parametrics Track
PA06 Presentation Platten Parameter Driven World
PA06 Paper Platten Parameter Driven World
Parametric estimating is gaining acceptance. DFARS requires DOD contractors to maintain an estimating system. Parameters are the characteristics, features, or measurable factors that define a product or program. What makes a good parameter? What are the pros and cons of parametric cost estimating? How do you develop and maintain a parametric estimating database/model? What does the future hold, beyond just estimating? How do you gain parametric control over touch labor, support labor, and parts costs to improve affordability?
Generating a Semi-Automated “Similar To” Basis Of Estimate from a Complex Parametric Hardware Cost Model for Antennas
Danny Polidi
David Bloom
Parametrics Track
PA07 Presentation Polidi Similar To Antennas
This paper discusses the development of a “Similar-To” Basis Of Estimate (BOE) generation tool used in conjunction with a Complex Parametric Antenna Cost Model. Starting with the parametric model based on quantifiable sizing mechanisms which are designed to quickly and accurately calculate the “top-down” cost for all engineering and operations disciplines and functions required for antenna development and test, cost is generated, “similar to” programs are identified, and a BOE can be auto-populated.
Macro-parametrics and the applications of multi-colinearity and Bayesian to enhance early cost modelling
Dale Shermon
Parametrics Track
PA08 Presentation Shermon Macro Parametrics
PA08 Paper Shermon Macro Parametrics
This paper will consider the spectrum of parametric cost models from cost estimating relationships (CER) to micro and finally macro-parametric models. This will lead to a description of the Family of Advanced Cost Estimating Tools (FACET) parametric suite of models and their top-down capability to estimate costs incorporating the application of multi-colinearity and bayesian techniques.
Using Historical Cost Reports to Derive System Specific Factors
Jocelyn Tague
William Banks
Parametrics Track
PA09 Paper Tague Using Historical Reports
PA09 Presentation Tague Using Historical Reports
Non‐Recurring Engineering (NRE) is often a cost driver for new, complex systems and must be given high attention during Milestone B cost estimating. Our team developed an approach for developing system specific Cost Estimating Relationship (CER) using actual costs for from combat weapon systems with shared capabilities, resulting in a credible defensible, yet flexible approach that can be replicated for a wide range of estimates.
A CER Development Process for Spares Estimating
Cheryl Wilson
Parametrics Track
PA10 Presentation Wilson CER Development
This presentation shows the direction of a group of Spare Parts Estimators and how we are advancing our statistical skills and CER development processes. It shares the progress in our CER development journey attained through education, research, analysis and teamwork.
Improving Parametric Software Cost Estimation Through Better Data Collection
Bob Koury
Parametrics Track
PA11 Presentation Koury Improving Parametric Better Data
In 2010 PRICE Systems LLC in conjunction with PEO STRI agreed to work together to improve the data being collected to support software development cost estimation in the command. The paper will compare and contrast the experiences of two other collection efforts with those of original effort. The paper will identify core findings which may be used universally to improve the quality, quantity, and applicability of data being collected.
Building an Integrated Aerospace Electronics Development Parametric Model from the Ground Up
David Bloom
Parametrics Track
PA12 Presentation Bloom Integrated Aerospace Model
This paper presents the results of a multiple year effort to create a business specific set of integrated parametric cost models for complex electronics hardware development in such a way that all disciplines and activities that participate in that development process have a piece of the model and a stake in the data collection that drives the model. This “grounds up” approach provides that added benefit of cost allocation guidance once the development is approved.
Program Management Track
Essential Views of the Integrated Program Management Reports (IPMRs)
Thomas Coonce
Glen Alleman
Program Management Track
PM01 Presentation Coonce Essential Views IPM
During this session, the authors will describe key views of the IPMRs and other data that will allow a government stakeholder to a) understand the cost, schedule and technical status on applicable contracts, b) investigate the nexus of performance problems, and c) observe likely future problems. The presentation represents a “synthesis” of existing traditional earned value management metrics and other metrics designed to provide an integrative picture to assist in keeping the program “green”.
Meeting Fiscal Constraints: the Evolving Role of Performance Measurement
Steve Green
Kevin Davis
Kurt Heppard
Program Management Track
PM02 Presentation Meeting Fiscal Constraints
Performance measurement tools can reflect the strategic intent of management and help them prioritize initiatives to meet strategic objectives. In fiscally austere times, the Balanced Scorecard can help organizations address these tough decisions. Applying concepts from the Balanced Scorecard to cost analysis and budgeting, this paper develops an integrated aligned system of strategic goals, performance metrics, and cost parameters at a university facing pressure to manage costs. There may be lessons learned for the DoD.
The graph is always greener on the other side: examples of how to interpret and fix bad visual representations of data
Stephen Ketcham
Program Management Track
PM03 Presentation Ketcham Graph Greener Handout
In my role performing independent reviews of estimates, I’ve seen a quality spectrum of different types of graphs, charts, and forecasts. These visuals summarize key points and remain in decision-makers minds, often providing the convincing point to win funding approval for the project. Unfortunately they can also be poorly crafted, employ scare tactics or even frame the data in a misleading way. This presentation provides bad examples of visuals, and discusses possible improvements.
Seeing What We Need to See vs. What We Hope to See: Visualizing Integrated Cost and Schedule Data for Earned Value Analysis
Brian Leach
Program Management Track
PM04 Presentation Leach Seeing What We Need
When applied properly, data visualization can serve analysis by showing us what is otherwise difficult to see (and not always what we hope to see). With the emergence of the UN/CEFACT XML standard, integrated cost and schedule data can be shown in exciting new ways. This paper will examine and explore innovative new ways to show integrated cost and schedule data which have practical applications in earned value management system analysis.
Different Incentives in Government Contracting
Jennifer Leotta
Program Management Track
PM05 Presentation Leotta Different Incentives Contracting
PM05 Paper Leotta Different Incentives Contracting
This paper shows stock prices are used as the primary incentive for executives of publicly traded government contractors rather than profit sharing contracts. Looking at the reaction of companies’ stock prices to public and private sector news, this paper shows stock prices do not capture “bad news” for both the public and private sector customers compared to overall market indicators; and would make poor contract incentives. Finally, alternate methods of contract incentives will be explored.
Agile Project Management Controls: Issues with EVM and a Viable Alternative
Omar Mahmoud
Blaze Smallwood
Program Management Track
PM06 Presentation Mahmoud Aglie Project Management
As more government software programs adopt agile practices, program managers are trying to find effective ways to monitor and control cost and schedule. The challenge is that traditional project control techniques, like EVM, are not well suited to deal with the requirements fluidity and new terms/metrics that come with agile projects. This paper will describe the inherent issues with using EVM on agile projects and offer a recommendation for a more effective solution.
2015 Best Paper: Program Management Track
2015 Best Paper Overall
The Psychology of Cost Estimating
Andy Prince
Program Management Track
PM07 Presentation Prince Psychology Cost Estimating
PM07 Paper Prince Psychology Cost Estimating
Research into human psychology has yielded amazing findings into how we process information and how we use information to make decisions. Cost estimators can use psychology and behavioral economics to improve not only our cost estimates but how we communicate and work with our customers. By understanding how our customer’s think, and more importantly, why they think the way they do, we can have more productive relationships and greater influence. The end result will be better decisions by the decision makers.
Cost Management
Anthony DeMarco
Program Management Track
PM08 Presentation DeMarco Cost Management
Cost Management has a key role in project management and is the management trend due to the actual financial and economic worldwide constraints. Cost Management solution was built to be in line with the recommendations of the GAO (US Government Accountability Office) that stated that a large number of Defense projects are at risk of experiencing cost overruns and schedule delays. GAO found that the two major causes of project failure were first an inaccurate initial estimate and secondly poor subsequent cost control.
Using Total Lifecycle Cost as the Basis for Proposal Cost Evaluation
Ryan Timm
Program Management Track
PM09 Presentation Timm Total Lifecycle Cost
This paper discusses benefits and obstacles of using the TLCC of an entire system instead of the bid price for the purpose of proposal cost evaluation. An example acquisition of a space vehicle (SV) ground system explains the impact to TLCC of portions of the ground system developed by the SV manufacturer, SV operations, external software interfaces, government furnished IT infrastructure, and legacy sustainment until transition. Using TLCC allows higher-level requirements and enables innovative architectures.
A Maturity Analysis of the American and European Cost Community
Mark Gilmour
Program Management Track
PM10 Presentation Gilmour Maturity Analysis
PM10 Paper Gilmour Maturity Analysis
The Cost Engineering Health Check (CEHC) is a cost capability benchmarking tool used to identify strengths and weaknesses within organizations costing functions. In 2014 the authors of this paper conducted a CEHC at the ICEAA conference in Denver. That workshop used e-voting to anonymously gather the views of the audience in attendance. This paper presents a summary of these results and contrasts them against data gathered from European workshops, providing a snapshot of the international costing capability.
Presenting Estimates
Melissa Teicher
Program Management Track
PM11 Tiecher Presenting Estimates
Decision‐makers rely on cost estimators to guide them in making the most educated judgments on what is best for the program or organization. A thorough and well‐organized presentation can make the difference between an easy decision and a difficult one. This paper discusses how public speaking skills contribute to an effective estimate presentation. It also delivers tips and tricks on how to make an estimate presentation more visually pleasing and easier to understand.
Risk Track
Using Stochastic Optimization to Improve Risk Prioritization
Eric Druker
Risk Track
RI02 Presentation Druker Stochastic Optimization
RI02 Paper Druker Stochastic Optimization
Minimizing the cost of complex programs is critical for government agencies trying to meet their missions in today’s fiscal. Today, identifying cost-savings measures is a manual procedure where an analyst must make an educated guess as to what risks to mitigate. This process is manual and slow. This paper will outline a methodology for using the emerging field of stochastic optimization to automate the identification of cost-savings measures on complex programs.
Why Are Estimates Always Wrong: Estimation Bias and Strategic Misestimation
Dan Galorath
Risk Track
RI03 Presentation Galorath Estimates Always Wrong
RI03 Paper Galorath Estimates Always Wrong
The biggest source of estimation error usually comes from people, either by accident or strategically. This is a disaster because viable estimates are core to project success as well as ROI determination and other decision making. Most people don’t know how to estimate. Those who estimate are often optimistic, full of unintentional bias and sometimes strategically mis-estimating.
This paper discusses the issues of estimation bias and strategic mis-estimation as well as how to mitigate these issues.
A “Common Risk Factor” Method to Estimate Correlations between Distributions
Marc Greenberg
Risk Track
RI04 Presentation Greenberg Common Risk Factor
A “common risk factor” method uses expert-derived risk factors to estimate correlation between two distributions. The premise is to estimate mutual information among risk factors that underlie each distribution. Approximation of such mutual information leverages properties of the joint probability distribution of a unit square. Geometric outputs of each pair of common random variables are compared leading to an estimated ‘intersection’ of pair of common risk factors. This intersection is then assumed to be a proxy of correlation.
Data-Driven Guidelines for Correlation of Cost and Schedule Growth
Sidi Huang
Mark Pedigo, PhD
Chris Shaw
Risk Track
RI05 Paper Huang Data Driven Guidelines
RI05 Presentation Huang Data Driven Guidelines
When running Monte Carlo simulations, one of the most difficult aspects to account for is correlation between the independent variables. Ignoring correlation generally reports overly optimistic analysis results; industry standard suggests that when no additional insight is available, a default value of 0.2 can be used. This study analyzes historical NASA program data to develop data-driven guidelines for correlation. Moving forward, this study can be replicated to other industries and organizations with critical schedules.
Risk Mitigation: Some Good News after the Cost / Schedule Risk Analysis Results
David Hulett
Risk Track
RI06 Presentation Hulett Risk Mitigation News
The results of your cost‐schedule risk analysis say you are likely to be overrun on both schedule and budget. Are you are out‐of‐luck? No. Do not forget the risk mitigation phase. Risk mitigation typically proceeds from a prioritized risk list developed from analysis of the risk and uncertainty‐loaded schedule. Complete mitigation of risks is not usually feasible, and uncertainty is usually deemed to be irreducible. Some good news comes from the risk mitigation actions.
A Mathematical Approach for Cost and Schedule Risk Attribution
Fred Kuo
Risk Track
RI07 Presentation Kuo Mathematical Approach
RI07 Paper Kuo Mathematical Approach
The level of sophistication in cost estimate has progressed substantially in the last decade. This is especially true at NASA where, spearheaded by the Cost Analysis Division (CAD) of NASA Headquarters, cost and schedule risks in terms of confidence level have been codified. In this paper, the author attempts to employ the portfolio approach in calculating risk attribution; first applying to a portfolio of cost risks and extend the same concept to schedule risks analysis.
Triangular Distributions and Correlations
Jennifer Lampe
Risk Track
RI08 Presentation Lampe Triangular Distributions
Triangular distributions are often used in estimating cost risk because the math is relatively simple. The concept of an estimate having a lower limit, an upper limit, and a mode is easy to understand.
We show the math behind triangular distributions and correlations.
What does it mean to say that two risks are 30% correlated?
How do risk analysis programs handle triangular distributions and correlations?
How can you use random numbers in Excel to generate a Monte Carlo simulation for any defined triangular distribution?
The Impact of Selected Assumptions and Core Tenets on Schedule Risk Assessment Results (A Progressive Model Comparison)
James Quilliam
Risk Track
RI09 Paper Quilliam Impact Selected Assumptions
RI09 Presentation Quilliam Impact Selected Assumptions
In the quest to ensure the sound representation of Schedule risk assessment (SRA) simulations this case study will provide a progressive model comparison of schedule risk assessment assumptions and core tenets. The elements of this approach will focus on the: methodology and tools; the progressive assumptions and core tenets applied; conclusions and lessons learned for practitioners. This will greatly enhance the understanding and confidence that leadership and project teams have in the schedule risk assessment results.
Wine as a Financial Investement
Michael Shortell
Adiam Woldetensae
Risk Track
RI10 Presentation Shortell Wine as Financial Investment
The purpose of this presentation is to look into wine as an investment and analyze its characteristics as a financial asset. The goal here is to look into vintage wines through time, analyze its profitability and predict how investing in wine could be a good source of diversification. Our objective is to detect the types of wines that are a good source of investment and their associated risk. We aim to identify a good investment portfolio for a wine collector in order to maximize returns and minimize the associated risk.
2015 Best Paper: Risk Track
Beyond Correlation: Don’t Use the Formula That Killed Wall Street
Christian Smart
Risk Track
RI11 Presentation Smart Beyond Correlation
RI11 Paper Smart Beyond Correlation
Risk models in cost estimating almost exclusively rely on correlation as a measure of dependency. Correlation is only one measure (among many) of stochastic dependency. It ignores the effect of tail dependency, when one extreme event affects others. This leads to potential outcomes that do not make sense such as a program with a large schedule overrun but no cost overrun. This method has been widely blamed for the financial crisis a few years ago. The copula method is introduced as a way to overcome these deficiencies.
Connecting the Dots: Integrating the risk cube with the POE
Christina Snyder
Risk Track
RI12 Paper Snyder Connecting the Dots
RI12 Presentation Snyder Connecting the Dots
DoD guidance instructs program managers to include risk and uncertainty analysis in their cost estimates. Program managers also maintain a risk cube that tracks known program risks and assigns a likelihood and consequence. Risks, regardless of if they are programmatic, technical, or schedule defined, will have an effect on the overall program cost. This paper investigates the linkage between identified risks and POE cost and shows how to translate the risk cube into dollars.
Methodology for Integrating Risk mitigation Activities into JCL analysis
James Taylor
Justin Hornback
Risk Track
RI13 Presentation Taylor Methodology Risk Mitigation
RI13 Paper Taylor Methodology Risk Mitigation
The intent of this whitepaper is to outline a an improved methodology for capturing risk mitigation activities for both cost and schedule into a Joint Confidence Level (JCL) analysis and assessing mitigation effectiveness.
Software Estimating Track
The Continual Pursuit of the One True Software Resource Data Reporting (SRDR) Database
Remmie Arnold
Peter Braxton
Software Estimating Track
SW01 Presentation Arnold Continual Pursuit
This paper will demonstrate the ability for one database to store Software Resource Data Report (SRDR) source data in its original form, as submitted by WBS element and reporting event. This database allows evaluations, interpretations, and annotations of the data, including appropriate pairing of Initial and Final reports; mapping of SLOC to standard categories for the purposes of determining ESLOC; normalization of software activities to a standard set of activities; and storage of previous assessments.
Agile Estimation Using Functional Metrics
Thomas Cagley
Software Estimating Track
SW02 Presentation Cagley Agile Estimation
Agile methods have matured and are now being integrated into many different approaches to the development of software. Estimation has been problematic for all methods, agile to plan based and therefore it tends to be a lightning rod for experimentation and synthesis such as is being described in this paper. Agile Estimation Using Functional Metrics has presented a path for integrating the discipline found in functional metrics with the collaborative approaches found in agile parametric estimation.
Put some SNAP in your estimating model
David Lambert
Software Estimating Track
An effective estimating model is dependent upon having the capacity to accurately size the problem domain for the solution being developed. Function Point Analysis is a widely accepted industry and ISO standard for sizing functional requirements; however, it does not take into account the sizing of non-functional requirements. Learn how the newly released Software Non-Functional Assessment Process (SNAP) can provide a more complete and accurate assessment of project size.
Forecasting Software Maintenance
Alea DeSantis
Software Estimating Track
SW04 Paper DeSantis Forecasting Software Maintenance
SW04 Presentation DeSantis Forecasting Software Maintenance
Providing a defensible estimate for software maintenance can be complicated. By overlaying historical data with lifecycle events one can derive event driven factors and use trend analysis to forecast future Software Problem Reports. Topics addressed are: common-cause variation, special-cause variation, severity level, complexity level, and decay rate. This method moves away from using a general defect rate calculation to a more complete product-specific calculation.
Effective use of Function Points for Analogous Software Estimation
Daniel French
Software Estimating Track
SW05 Presentation French Effective Function Points
Analogous estimation is often used to estimate software projects, especially in the early phases, because it is believed to be quicker and easier than other techniques. However, many organizations do not employ this technique effectively and correctly. One pitfall is improperly applying the technique & basing these estimates on Software Lines of Code (SLOC). This presentation will discuss common mistakes made when using analogous estimates as well as how the use of IFPUG function points can greatly improve this estimation technique
NASA Software Cost Estimation Model: An Analogy Based Estimation Method
Jairus Hihn
Software Estimating Track
SW06 Presentation Hihn NASA Software Model
Software development activities are notorious for their cost growth. While there has been extensive work on improving parametric methods there is very little focus on the use of models based on analogy and clustering algorithms. In this paper we will describe the methodology being used in the development of a NASA Software Cost Model using data mining clustering algorithms and evaluate its performance by comparing it to estimates from COCOMO II, a calibrated COCOMO II, liner regression, and K-nearest neighbor models.
Estimating Agile Software Development
Bob Hunt
Software Estimating Track
SW07 Presentation Hunt Estimating Agile
The use of Agile Software Development continues to grow in large and small Federal programs. These programs often utilize Story Points as a key size metric. This presentation will roovideman update on Agile Cost Estimating techniques and emphasize how to,utilize Story Points as a key size metric.
How Much Does Software Maintenance Cost?
Cheryl Jones
James Judy
John McGarry
Software Estimating Track
SW08 Presentation Jones Software Maintenance
To accurately estimate, budget, allocate, and justify the resources required to meet evolving mission and service affordability requirements across the system life-cycle we must be able to answer the fundamental question: How much does software maintenance cost? This presentation provides a summary of the findings from an ongoing U.S. Army study that is focused on developing improved software maintenance estimation processes, cost relationships, and associated models.
Exploring DoD Software Growth: A Better Way to Model Future Software Uncertainty
Nicholas Lanham
Software Estimating Track
SW09 Paper Lanham DoD Software Growth
SW09 Presentation Lanham DoD Software Growth
This presentation highlights trends and estimating relationships derived from the April 2014 SRDR data. This analysis provides the estimating community with several new variables to better model future software growth uncertainty. In addition, the analysis includes growth by contract type, domain, and program type as well as introduces a multivariate model to predict final hours. All of the relationships included within this slide set have been derived from the best 400+ data points, were approved by DCARC, and are DoD-specific.
Considerations in Data Center Migration Life Cycle Cost Estimating: A Joint Information Environment (JIE) Perspective
Dennis McKeon
Software Estimating Track
SW10 Presentation McKeon JIE Migration
The potential benefits of consolidating data centers has led to the creation of initiatives such as the DoD Joint Information Environment (JIE). The goal is to reduce IT infrastructure costs by consolidating system support functions, improve hardware utilization, and lower energy and facility costs. This paper is from the perspective of an organization migrating to a consolidated data center. It will describe the migration steps, cost estimating approach, benefits, risks, and organizational challenges from their perspective.
At the Intersection of Technical Debt and Software Maintenance Costs
Arlene Minkiewicz
Software Estimating Track
SW11 Paper Minkiewicz Technical Debt
SW11 Presentation Minkiewicz Technical Debt
Ward Cunningham introduced the technical debt metaphor in 1992 to ease business leaders understanding of the cost of making poor decisions during software development. While this author has seen many different definitions of technical debt, it is basically an indication of the amount of ‘should fix’ issues that remain in production code. This paper reports on an on-going research effort to understand technical debt and its implications on software maintenance cost and effort estimation.
2015 Best Paper: Software Estimating Track
Early Phase Software Cost and Schedule Estimation Models
Wilson Rosa
Software Estimating Track
SW12 Paper Rosa Early Phase Software
SW12 Presentation Rosa Early Phase Software
Software cost estimates are more useful at early elaboration phase, when source lines of code and Function Points are not yet available. This study introduces effort estimation models using functional requirements as a size predictor and evaluates development schedule using estimated effort based on data from 40 military and IT programs delivered from 2006 to 2014. Statistical results show that estimated functional requirements and peak staff are significant contributors and that estimated or actual effort is a valid predictor of development duration.
Software size measures and their use in software project estimation
Harold van Heeringen
Software Estimating Track
SW14 Paper VanHeeringen Software Size Measures
SW14 Presentation VanHeeringen Software Size Measures
Using the right size measure is extremely important for software cost engineering purposes. However, the number of software size measures is growing rapidly and there is a lot of confusion in the industry with regard to the methods that should be used. The advantages and disadvantages of the major software size measures are discussed and recommendations are given on what size measures should be used to estimate the different types of software systems.
Software Size Growth
Corinne Wallshein
Software Estimating Track
SW15 Presentation Wallshein Software Size Growth
NCCA will present their findings including cost estimating relationships (CERs) of software size growth using selected subsets. CERs will be compared using initially estimated parameter distributions against CERs using software size. Other parameters include requirements, staff size, hours, and duration. As the 2014 update of the Joint Agency Cost and Schedule Risk and Uncertainty Handbook advises use of standard error for uncertainty bounds, NCCA will compare bounds for recommended CERs, by selected data subsets.
Training Tracks & Sessions
Cost Estimating Basic Training Track
Cost Estimating Basics
Chris Dalton
Ted Mills
CEB 01 – Cost Estimating Basics
The Cost Estimating Overview session introduces the basics of cost estimating and analysis and the reasons for doing cost estimates. Additionally, the motivation for cost estimating, why and how it is used, and an overview of the challenges and issues will be presented. Cost estimating products will be reviewed and a general overview of the process and methods used to develop cost estimates will be discussed, providing the background information and fundamental knowledge needed for the other CEB sessions. This session will provide an overview of the ICEAA certification process and give trainees an understanding of what to expect in the subsequent CEB sessions.
Cost Estimating Techniques
Patrick Malone
Matthew McGovern
CEB 02 – Cost Estimating Techniques
The Cost Estimating Overview session introduces the basics of cost estimating and analysis and the reasons for doing cost estimates. Additionally, the motivation for cost estimating, why and how it is used, and an overview of the challenges and issues will be presented. Cost estimating products will be reviewed and a general overview of the process and methods used to develop cost estimates will be discussed, providing the background information and fundamental knowledge needed for the other CEB sessions. This session will provide an overview of the ICEAA certification process and give trainees an understanding of what to expect in the subsequent CEB sessions.
Data Collection
Eric Plumer
Christina Snyder
This session covers the Core Knowledge section of Module 04 Data Collection of CEBoK. All parametric estimating techniques and cost estimating models require credible data before they can be used effectively. Training in Data Collection and Analysis discusses the processes needed to collect and analyze the data used in parametric applications, as well as data types, sources, and adjustment techniques.
Inflation
Sam Toas
Bill Seeman
This session covers the Core Knowledge section of Module 05 Inflation and Index Numbers of CEBoK. Proper inflation analysis is essential to the success of any cost estimate or economic analysis. Calculating inflation correctly and understanding the fundamental concepts will enable you to produce cost estimates that are timely, accurate, and credible to support your program’s lifecycle needs. It will also empower you to communicate with key stakeholders on the need to adjust your financial estimates based on changes in the economy.
Data Analysis
Walter Bednarski
Alex Wekluk
This session discusses the first analytical steps to take after obtaining a set of cost data, with a particular emphasis on techniques for displaying and analyzing data graphically. It addresses univariate, bivariate, and multivariate data sets, with a focus on statistical analysis of univariate data sets and graphical analysis of bivariate data sets via scatter plots. It introduces measures of central tendency, such as mean, median, and mode; measures of variability, such as standard deviation, variance, and coefficient of variation (CV); and measures of uncertainty, such as confidence and prediction intervals. A couple of vivid examples illustrate the perils of drawing conclusions from your numerical data without looking at a graph.
Learning Curve Analysis
Michael Mahoney
Gurney Thompson
CEB 06 – Learning Curve Analysis
This is a training track presentation of the CEBoK Module 7 (Learning Curves). The presentation will cover the key ideas, analytical constructs and applications of the learning curve module. The target audience are those preparing to take the ICEAA certification exam. Beyond the theoretical information, we will present the study questions for Module 7 with steps required to solve the problems using only a calculator as is required on the certification exam.
Risk Analysis Basic
Eric Druker
Sarah Green
This session will provide motivation for the need for risk analysis and introduce the basic types and uses of risk. It will focus on the practical execution of the general risk analysis process: (1) Develop a point estimate; (2) Identify the risk areas in the point estimate; (3) Determine uncertainty around the point estimate; (4) Apply correlation between uncertainty distributions; (5) Run the Monte Carlo simulation; (6) Assess the reasonableness of results; (7) Calculate, allocate, and phase risk dollars; and (8) Show the results.
Probability and Statistics Basic
Tim Anderson
Stacy Dean
This will be a review of statistical and probability concepts relevant to cost estimation and uncertainty. Topics will include statistics; measures of central tendency; measures of dispersion; and important probability distributions and their linkage to cost estimates. We also introduce the concept of a random variable; Monte Carlo simulation; the Inverse CDF method; and thoughts on the differences between the normal and lognormal distributions. Finally, we discuss hypothesis testing and its relationship to cost estimating.
Software Cost Estimating
Arlene Minkiewicz
Blaze Smallwood
CEB 09 – Software Cost Estimating
This session will convey the principles of this critical area of specialized cost estimating. It addresses the basics of software development and some of the things to look out for in software estimating, and includes specific examples of applying software estimating techniques. The presentation will walk through examples of how to approach a software cost estimate using a number of estimating relationships and ideas, commercial models that can be used to construct estimates, and various methods of software development.
Economic Analysis Basic
Kammy Mann
Joseph Sarage
CEB 10 – Economic Analysis Basic
This session covers the Core Knowledge section of Module 13 Economic Analysis of CEBoK. It will be of particular interest to anyone studying for the ICEAA certification exam. The session provides a practitioner’s perspective for conducting an economic analysis (EA) by reviewing EA concepts, terminology, variables and measures-of-merit. By accounting for monetized costs, monetized benefits, opportunity costs and time-value-of-money (“discounting”), an EA enables one to calculate economic measures-of-merit.
Contract Pricing
Peter Braxton
Rick Collins
This session takes a holistic view of the contracting and contract pricing from a cost estimator’s viewpoint, with emphasis on acquisition planning, contract pricing and cost/price analysis. The discussion of acquisition planning will focus on contract types, specifically how cost risk is shared between parties and the profit/fee potentials commensurate with that risk. The use of cost estimating techniques to establish a basis of estimate (BOE) as the foundation of the cost/price proposal (i.e. cost pricing) will be explained in detail.
EVM Basic
David Bach
Thomas Harless
This session will provide an introduction to the basic concepts of earned value management (EVM), with a focus on practical application in support of a project or program. Specific topics will include basic EVM components and data elements, as well as standard earned value analysis techniques. A sample case study will be introduced and built upon throughout the session to demonstrate and reinforce these basic principles of EVM.
Cost Estimating Advanced Training Track
GAO Cost Assessment Guide
Neil Chakrabarti
Bruce Fad
CEA 01 – GAO Cost Assessment Guide
In March 2009, the United States Government Accountability Office (GAO) published, GAO Cost Estimating and Assessment Guide, Best Practices for Developing and Managing Capital Program Costs. The guide was developed to establish a consistent methodology that is based on best practices and that can be used across the federal government for developing, managing, and evaluating capital program cost estimates. This training will introduce the twelve-step process, describe the characteristics and contents of a high quality estimate, and illustrate the application of the guide to GAO assessments.
Technical Baselines
Jason Dechoretz
Paul Tetrault
Sam Toas
Understanding the technical and programmatic issues of the program/product you are estimating is key to developing a good cost estimate. This session discusses what information you need to baseline the technical and programmatic aspects of the program, the questions that need to be asked to ensure a clear understanding and how these issues translate into a cost perspective.
Work Breakdown Structure (WBS)
Chris Dalton
CEA 03 – Work Breakdown Structures
The Work Breakdown Structure (WBS) is a critical part of managing and planning any project. The WBS is the best tool for ensuring consistency and focus on the project. This presentation will discuss the key attributes of a good WBS in scoping and planning of projects. The presentation will provide the generally accepted concepts of the use and application of the WBS as well as why it is often misunderstood and not used as it should for maximum effectiveness.
Cost and Software Data Reports
Emily Beltramo
Brandon Bryant
CEA 04 – Cost and Software Data Reports
This training session will give an overview of Cost and Software Data Reports (CSDRs) and how they can be employed to facilitate the development of better and more credible cost estimates. This training session will also demo and discuss the Cost Assessment Data Enterprise (CADE) tool. CADE collects, organizes, and displays data, including EVM, CSDRs, Selected Acquisition Reports (SARs), budget data, and technical data all in one centralized location.
Risk Analysis Advanced
Christian Smart
Amanda Wilson
CEA 05 – Risk Analysis Advanced
This session will delve into advanced topics in risk, including a short and entertaining refresher on basic cost risk analysis. We will perform an analysis of historical risk data by studying the effect of program size on cost / schedule growth and how we can use this information in our risk analysis. This session covers the Related and Advanced Topics section of Module 09 Cost and Schedule Risk Analysis of CEBoK.
Monte Carlo Simulations
Blake Boswell
John Swaren
CEA 06 – Monte Carlo Simulations
Monte Carlo simulations are the most commonly used tool for performing risk analysis. This training will walk through a brief history of Monte Carlo simulations, from their use in evaluating Pi, through their application on the Manhattan project. The bulk of the training will be an interactive demonstration where the attendees create their own Monte Carlo simulation, alongside the instructor, in Excel (using a few lines of Visual Basic coding). The simulation will be run and the results analyzed in real-time to provide attendees a deeper understanding of where S-Curves come from and how to interpret them.
Probability Distributions for Risk
Peter Braxton
CEA 07 – Probability Distributions for Risk
Cost Estimating Advanced builds on the topics covered in Cost Estimating Basic. Topics addressed include: understanding the shape of a distribution and how it is related to that distribution’s parameters; alternative specification of distributions, such as by any two of a central point (mean, median, and/or mode), a coefficient of variation (CV), and a percentile; correction of distributions for understatement of both mean and CV; and useful rules of thumb for characterizing distributions.
Probability and Statistics Advanced
Marc Greenberg
Christian Smart
This training session builds upon the Probability / Statistics Basic concepts relevant to cost estimation and uncertainty analysis. Topics to be presented include: ordinary least squares, outlier analysis, maximum likelihood estimation, and method of moments. Instructors will provide you with useful approaches to apply advanced statistical concepts to data sets used in your cost estimate generation efforts and approaches for the techniques to ensure you can apply the techniques appropriately in their real-world estimating scenarios.
Manufacturing Cost Estimating
John Deem
Charles Kurtz
Kammy Mann
CEA 09 – Manufacturing Cost Estimating
The goal of the Manufacturing Cost Estimating module is to arm the student with a set of techniques used to address issues unique to estimating in the manufacturing environment. It will be our objective in this module to raise a few of the most common general issues, considerations and concerns the estimator must be aware of in a typical major manufacturing environment and to provide techniques for addressing them.
Software Cost Estimating Advanced
Bob Hunt
Harold van Heeringen
CEA 10 – Software Cost Estimating Advanced
This session: 1) explores functional size measurement techniques with parametric models and historical data; 2) explains functional size and the estimation laws that should be understood before estimating a software project; 3) presents a case is given where a reality check based on functional size and historical data prevented a project disaster; 4) explores current practices in agile software development and present alternative approaches to estimating the cost and schedule for an agile program.
Integration Training Track
Basic Schedule Analysis
Justin Hornback
INT 01 – Basic Schedule Analysis
Examination of Program performance continues to identify the development and monitoring of program schedules as a weakness in programs that do not meet technical, cost, and schedule goals. This workshop presentation will cover scheduling basics, desired attributes of a valid project schedules, and basic schedule analysis techniques important to programmatic and cost estimator analysts. Schedule analysis tools and techniques will include, but are not limited to, three point estimates and schedule risk analysis.
Advanced Schedule Analysis
Justin Hornback
INT 02 – Advanced Schedule Analysis
This workshop will build on topics covered by Basic Schedule Analysis, expanding into the detailed process to conduct schedule analysis. Topics include common schedule analysis metrics and how to use them to analyze a schedule. Using an Integrated Master Schedule (IMS) to create an analysis schedule and employing schedule risk analysis with resource/cost loading to build the framework for Joint (Cost and Schedule) Confidence Level (JCL) creation.
Schedule Risk Analysis
David Hulett
INT 03 – Schedule Risk Analysis
Project scheduling is only the start of understanding when your project will finish. This workshop shows how simulating uncertain durations allow the user to determine how much overrun there might be at different target levels of certainty. We introduce the Risk Driver method of driving the Monte Carlo simulations using the risks usually found starting from the Risk Register. We then prioritize the risks for effective risk mitigation.
Joint Cost and Schedule Risk
Eric Druker
David Hulett
INT 04 – Integrated Cost Schedule Risk Analysis
Recognizing the need to ensure results from cost and schedule risk analyses are compatible, agencies, such as NASA, have recently implemented policy requiring the integration of these two, formerly separate, practices. This training course will provide analysts an overview on how they can integrate their cost estimates, schedules, and risk register into a single cohesive model that can then be analyzed to predict future cost and schedule growth and identify their sources so that mitigative actions can be taken. A live demonstration of an integrated cost/schedule risk analysis (ICSRA) tool, Polaris, will be conducted in class so that analysts can familiarize themselves with how an ICSRA model works.
Contracts Risk
Peter Braxton
This session will present cost estimating and risk analysis as tools to aid in facing contract management challenges. This session is adapted from “Cost Estimating and Risk Analysis as Tools for Contract Management,” twice presented at the Government Contract Management Conference (GCMC), sponsored by sister society National Contract Management Association (NCMA). It also draws from “Analysis of Large O&S Proposals: Lessons Learned,” presented at ICEAA 2013.
Cost Management
Tom Dauber
John Deem
Zachary Jasnoff
This session provides an introduction to cost management, discussing tools and techniques currently in use by leading-edge organizations, the role of cost estimating/analysis, and the related discipline of Cost As an Independent Variable (CAIV). Cost management takes a holistic approach, incorporating cost considerations into the overall management approach, and together with other measures of performance, help lead to decisions that provide optimal value to organizations and customers, be they shareholders, consumers, or warfighters.
Cost Implications of WSARA
Bob Hunt
INT 07 – Cost Implications of WSARA
The Cost Implications of WSARA presentation places the legislation in the context of an ever-evolving acquisition framework. Further, it covers the role played by acquisition in management of the defense enterprise, especially its connection with the two other major decision-support systems employed in the Department: the Joint Capabilities Integration and Development System, and the Planning, Programming, Budgeting, and Execution System. These other two systems, requirements and resourcing, can and do influence acquisition outcomes, sometimes significantly.
Cost Estimating in PMBoK vs. CEBoK
Cassie Capots
Lauren Nolte
INT 08 – Cost Estimating in PMBoK vs CEBoK
The Project Management Body of Knowledge and Cost Estimating Body of Knowledge each give guidance on effective cost estimating for successful project execution. There is a good deal of overlap between the two curricula, but also significant differences. More importantly, in many instances, the same terms are used differently. Mastering cost estimating as it exists in both requires you to remember everything you learn as you study these bodies of knowledge. And then forget it.
Target Costing and Trades
John Deem
Marc Greenberg
INT 09 – Target Cost Management Implementation
This session provides a practical introduction to tools and techniques used by government and industry to incorporate cost considerations into enterprise cost management efforts by supporting decisions that provide optimal value to the enterprise. The presentation is based on Module 16 Cost Management of CEBoK© still covering the essence initiatives to reduce total ownership costs, CAIV, Target Costing and Activity-Based Costing while incorporating additional references useful to estimators and cost analysts preparing for assignments to support the affordability mission.
Data Science and Cost Estimating
Eric Druker
Slides available upon request: iceaa@iceaaonline.org
For decades, cost analysts have relied on the same statistical approaches to building estimates. These approaches face numerous limitations stressing their utility for estimating cost. Data Science is the extraction of knowledge from data. With the recent rise in computing power and democratization of complex methodologies we now have access to an array of approaches previously unthinkable when it came to cost estimating. This exploration will introduce data science to the cost estimating community.
Parametrics Training Track
Parametric Analysis Overview
Anthony Demarco
Steve Glogoza
PAR 01 – Parametric Analysis Overview
This introductory session provides an overview of the parametric estimating technique. Parametric estimating develops cost estimates based upon the examination and validation of the relationships which exist between a project’s technical, programmatic, and cost characteristics as well as the resources consumed during its development, manufacture, maintenance, and/or modification. Attendees will learn the key ideas, primary analytical constructs, practical applications, and applicable commercial-off-the-shelf tools.
Cost Estimating Relationships (CERs)
Anthony DeMarco
PAR 02 – Cost Estimating Relationships
Cost Estimating Relationships (CERs) are critical to cost analysis. At the most basic level, CERs are mathematical expressions developed using historical cost and technical data to relate cost with other independent variables. This module provides key definitions, describes the CER development process and significance tests. A walkthrough of notional CERs will be provided, including an example of basic CER creation. The module ends with a discussion of validation and advanced topics, including calibration.
Regression Linear
Remmie Arnold
Peter Braxton
This session covers the linear regression subset of Module 08 Regression Analysis of CEBoK. Upon completion of this session, the student will be able to: 1) Determine the linear regression equation for a bivariate data set using ordinary least squares (OLS); 2) Calculate appropriate statistics related to the linear regression and use them to determine goodness of fit and statistical significance of the model: and 3) Use these mathematical modeling techniques to create Cost Estimating Relationships (CERs) from provided normalized data.
Regression Nonlinear and Multivariate
David Harris
PAR 04 – Regression Nonlinear and Multivariate
Upon completion of this session, the student will be able to: 1)Apply logarithmic transformations to enable the determination of best fit using OLS for data exhibiting power, exponential, or logarithmic functional forms; 2) Determine the linear regression equation for a multivariate data set using ordinary least squares (OLS); 3) Calculate appropriate statistics use them to determine goodness of fit and statistical significance of the model; and 4) Use these mathematical modeling techniques to create Cost Estimating Relationships (CERs) from provided normalized data.
Regression Multiplicative
Herve Joumier
PAR 05 – Regression Multiplicative
Development of CERs has, in the past, been based on explicit solutions of the OLS Y = a+bX+E, where E is a Gaussian additive error term or the logarithmic transform of Y=aX^b*E. Applying “general-error” regression allows to determine the optimal coefficients for any curve shape and to choose the error model. This tutorial focuses on development of CERs having multiplicative (i.e., percentage) errors.
CER Risk and S-Curves
Chris Price
Christian Smart
PAR 06 – CER Risk and S-Curves
This training session compares S-Curves generated using standard errors of the estimate (SEE’s), confidence intervals (CI’s), and prediction invervals (PI’s), arguing for PI’s as the appropriate technique. It is shown how and why SEE-based techniques, while commonly used, almost always underestimate both risk and uncertainty. Finally, we provide a step-by-step guide on how to generate PI-based S-curves, with the mathematical intuition behind each step.
Complex Models Company-Developed
Chris Price
PAR 07 – Complex Models Company-Developed
“Company Developed Complex Models” covers the material in Chapter 4 of the Parametric Estimating Handbook (PEH), which describes a proven ten-step approach for developing a multi-CER cost prediction model from historic data. The handbook defines a “company-developed” model as a special-purpose model, typically developed by an organization for a specific platform or product. A brief comparison of the GAO Cost Estimating Process and the PEH approach is also provided.
Complex Models Hardware
David Bloom
The Complex Hardware Models session (PT08) of the ICEAA Parametric training track provides a short history of complex model parametric estimating and continues with an overview, examples and recommendations to develop Basis of Estimates (BOE for Rough Order Magnitude (ROMs), Business Case Analysis, and Proposals. Best practices and lessons learned from the Parametric Estimating Reinvention Laboratory and personal experience provide insight into practical applications and limitations of the toolsets.
Other Parametric Applications
John Swaren
PAR 09 – Other Parametric Applications
Other Parametric Applications covers the material in Chapter 8 of the Parametric Estimating Handbook (PEH) This training will address what to consider when developing and implementing parametric estimating tools. We will also give descriptions of general and specialized applications where parametric tools can be used. We will show examples of where and how these techniques have been implemented.