2014 Best Papers

Congratulations to the winners of ICEAA’s 2014 Best Paper Awards!

2014 Best Paper: Overall

Bayesian Parametrics: Developing a CER with Limited Data and Even Without DataICEAA-4

Author:
Dr. Christian Smart
Director of Cost Estimating and Analysis
Missile Defense Agency

Summary: Cost estimators are often confronted with the need to provide cost estimates when there is limited applicable historical data. Bayesian methods can provide a sound and time-proven way to develop estimates in these situations. This paper introduces Bayesian techniques to parametric cost estimates. It provides a theoretical framework, two different ways to apply Bayesian methods to cost estimating, and detailed examples.

PA-3 – Handout – Bayesian Parametrics Developing a CER with Limited Data and Even Without Data

PA-3 – Paper – Bayesian Parametrics Developing a CER with Limited Data and Even Without Data


Best Paper: The Business and Art of Cost Estimating Track

A Comprehensive CES and BCA Approach for Lifelong Learning

Authors:

Kevin Cincotta
Technical Director
ICF International

Dr. Darcy Lilley
Chief Learning Officer
Air Force Air Mobility Command

Lauren Nolte
Senior Cost Analyst
ICF International

Summary:  The Air Force Air Mobility Command (AMC) Enterprise Learning Office (ELO) mission is to transform AMC into a premier Air Force learning organization. AMC ELO is initially focusing on training. Training is generally represented as only one line within a cost estimate. We developed a first-of-its-kind comprehensive Cost Element Structure (CES) for training. The CES decomposes training into its core elements. This paper presents the training CES, and preliminary results of the BCA.

BA-3 – Handout – A Comprehensive CES and BCA Approach for Lifelong Learning


 Best Paper: Earned Value Management Track

Trust but Verify – An Improved Estimating Technique Using the Integrated Master Schedule (IMS)

Author:
Eric Lofgren
Cost Analyst
Technomics, Inc.

Summary: What’s the state of the Integrated Master Schedule (IMS) today? Analysis of actual contract data suggests declining task performance, numerous binding constraints, unraveling logic, and optimistic forecasts. These factors in part lead to a failure of IMS predictive power. By analyzing IMS development across submissions, this paper will show that early performance to baseline is a good measure of realized slip and provides the framework for a new estimating technique.

EV-3 – Handout – Trust but Verify – An Improved Estimating Technique Using the Integrated Master Schedule (IMS)

EV-3 – Paper – Trust but Verify – An Improved Estimating Technique Using the Integrated Master Schedule (IMS)


Best Paper: Information Technology Track

Improved Method for Predicting Software Effort and Schedule

Authors:
Dr. Wilson Rosa
AIS/C4ISR Branch Head
Naval Center for Cost Analysis

Dr. Barry Boehm
Professor Emeritus
University of Southern California

Dr. Bradford Clark
Vice-President
Software Metrics Incorporated

Joseph P. Dean
Operating Location Chief, Hanscom AFB
Air Force Cost Analysis Agency

Dr. Raymond Madachy
Associate Professor
Naval Postgraduate School

Summary: This paper presents a set of effort and schedule estimating relationships for predicting software development using empirical data from 317 very recent US DoD programs. The first set predicts effort as a function of size and application type. The second predicts duration using size and staff level. The models are simpler and more viable to use for early estimates than traditional parametric cost models. Practical benchmarks are also provided to guide analysts in normalizing data.

IT-3 – Handout – Improved Method for Predicting Software Effort and Schedule

IT-3 – Paper – Improved Method for Predicting Software Effort and Schedule


 Best Paper: Life Cycle Costing Track

Cost Overruns and Their Precursors: An Empirical Examination of Major Department of Defense Acquisition Programs

Authors:
Alan Gideon
Senior Systems Engineer
Booz Allen Hamilton

Enrique Campos-Nanez
Senior Software Engineer
Epsilon Group

Pavel Fomin
Aerospace Engineer
United States Air Force

Summary: This paper modifies the 30 year old Asher-Maggelet model of future program acquisition cost to account for trends in Department of Defense cost outcomes by using recent platform-specific median cost growth outcomes. An exploratory analysis of cost trends from the 1960s to the present shows that the original model significantly over-estimates cost, making it unusable. The relationships between accurate individual program cost estimates, contract negotiations, and overall portfolio/department risk are discussed.

LC-3 – Handouts – Cost Overruns and Their Precursors An Empirical Examination of Major Department of Defense Acquisition Programs


Best Paper: Cost Management Track

Innovative Business Agreements and Related Cost & Pricing Methods at NASA in Support of New Commercial Programs

Authors:
James Roberts
Senior Cost Analyst
NASA Kennedy Space Center

Torrance Lambing
Lead in the Business and Cost Analysis Office
John F. Kennedy Space Center, NASA

Summary: Following completion of NASA’s Space Shuttle program, emphasis has been placed on enabling Commercial use of Kennedy Space Center’s infrastructure. This presentation addresses changes and new methods of pricing and estimating costs of NASA facilities and services being provided to commercial entities and other non-NASA users. It also provides an overview of legislation and NASA regulations that establish policy and guidance for entering into Agreements – and how pricing may vary under different Agreement types.

MG-4 – Handout – Innovative Business Agreements and Related Cost & Pricing Methods at NASA in Support of New Commercial Programs


Best Paper: Methods & Models Track

ICEAA-2001001001001001001001001

Relating Cost to Performance: The Performance-Based Cost Model

Authors:
Michael Jeffers
Senior Cost Analyst
Technomics, Inc.

Anna Irvine
Technomics, Inc.

Robert Jones
Senior Cost Analyst
Technomics, Inc.

Kelly Meyers
Surface Combatant Team Lead
Naval Surface Warfare Center Carderock Division

Robert Nehring
Cost Analyst
Technomics, Inc.

Jean-Ali Tavassoli
Cost Analyst
Naval Surface Warfare Center Carderock Division

Summary: A Performance Based Cost Model (PBCM) is an early-stage rough-order-of-magnitude cost-estimating tool that is focused on relating cost to performance factors. PBCMs are parametric cost models that are integrated with a parametric engineering model so that they estimate cost as a function of performance by simultaneously estimating major physical characteristics. This paper will discuss how to build a PBCM, the mathematics behind a PBCM, and the graphical outputs and insights that a PBCM can produce.

MM-24 – Handout – Relating Cost to Performance The Performance-Based Cost Model

MM-24 – Paper – Relating Cost to Performance The Performance-Based Cost Model


Best Paper: Parametrics Track

Bayesian Parametrics: Developing a CER with Limited Data and Even Without DataICEAA-4

Author:
Dr. Christian Smart
Director of Cost Estimating and Analysis
Missile Defense Agency

Summary: Cost estimators are often confronted with the need to provide cost estimates when there is limited applicable historical data. Bayesian methods can provide a sound and time-proven way to develop estimates in these situations. This paper introduces Bayesian techniques to parametric cost estimates. It provides a theoretical framework, two different ways to apply Bayesian methods to cost estimating, and detailed examples.

PA-3 – Handout – Bayesian Parametrics Developing a CER with Limited Data and Even Without Data


Best Paper: Risk Track

DSC_0288 A Critique of Cost-Risk Analysis and Frankenstein Spacecraft Designs: A Proposed Solution

Authors:
Mohamed Elghefari
Eric Plumer

Summary: We present a historical, data-driven probabilistic cost growth model for adjusting spacecraft cost Current Best Estimate. The model is sensitive to when, in the mission development life cycle, the spacecraft cost CBE is generated. The ICEAA-6model is based on historical spacecraft data obtained from the NASA CADRe database. This alternative cost-risk modeling approach encompasses the uncertainties of underlying design parameters of the spacecraft without violating laws of physics or axioms of the theory of probability.

RI-8 – Handout – Critique of Cost-Risk Analysis and Frankenstein Spacecraft Designs A Proposed Solution

 


Best Paper: Space Track ICEAA-15

The NASA Project Cost Estimating Capability

Authors:
Andy Prince
Manager, Cost Engineering Office
NASA/Marshall Space Flight Center

Brian Alford
Operations Research Analyst
Booz Allen Hamilton

Blake Boswell
Analytic Tool Developer
Booz Allen Hamilton

Matt Pitlyk
Operations Research Analyst
Booz Allen Hamilton

Summary: The paper provides a detailed description of the capabilities and shortcomings of the NAFCOM architecture and how what we learned from NAFCOM, plus new estimating requirements, is leading NASA to develop the Project Cost Estimating Capability (PCEC). The requirements for the PCEC are discussed, followed by a description of the PCEC architecture. Finally, the paper provides a vision for the future of NASA cost estimating capabilities.

SP-4 – Handout – The NASA Project Cost Estimating Capability