ICEAA Archives

Search past ICEAA Workshop Proceedings in the table below and click the title to access the downloadable files.

2007-2017 Workshop Proceedings are available online. For 2006 and earlier, please email us.

How Should We Estimate Agile Software Development Projects and What Data Do We Need?Tom Coonce, Glen AllemanThis paper proposes and demonstrates how to estimate agile software projects using collected data from prior similar projects. The authors discuss traditional ways of estimating software projects and show why these approaches are not feasible in estimating agile projects. They demonstrate estimating agile projects using actuals from historical features. They recommend a few additions to DoD's Software Resource Data Report (SRDR) as well as a Standard Feature Breakdown Structure (FBS) to permit using this approach.2017Agile
Agile 'Mumbo Jumbo'Jeremy EdenYou don't put any stock in this Agile "mumbo jumbo" do you? Well, Agile is here to stay and is gaining popularity.This paper will discuss current issues and some solutions for cost estimating Agile projects, clear up common misconceptions about Agile, and Introduce an Agile approach to cost estimating, including a draft Agile Cost Estimating Manifesto with suggestions on how to incorporate it with current cost estimating training, so ICEAA and its members can BE Agile.2017Agile
Agile: You're Doing it Wrong (or how to know you're doing it right)Daniel B. FrenchAgile is a hot topic in the IT world today. If your organization is using Agile, or considering it, there are a number of things you need to know. This presentation will discuss the pros and cons of Agile, how to determine if your organization is ready for it, proper Agile implementation, hybrid methodologies, and the organizational challenges when trying to make an organization Agile.2017Agile
Beyond the Manifesto: Tracking Agile Performance in a DoD EnvironmentGordon M. Kranz, Michael ThompsonThe use of Agile methods are becoming prolific within the DoD Enterprise and Weapons systems development environment. Agile supports the incremental discovery of end system capability within the constraints of the DoD parameters. This paper investigates, through real world experience, the use of earned value techniques to strategically manage the Agile development process. One of the essential elements key to success is traceability of user requirements to Agile features.2017Agile
Agile and GAO Cost Estimating Best PracticesJennifer Leotta, Karen RicheyThis paper will examine how GAO's cost estimating process can be applied to programs that are using an Agile framework. First, it will provide a brief overview of Agile processes and methods. Second, it will examine each of the 12 steps in the GAO cost estimating process and how those steps relate to an Agile framework. Finally, it will discuss how Agile artifacts can be leveraged to fulfill cost estimating documentation needs.2017Agile
CMMI and Agile Development: A Binary Choice?Arlene F MinkiewiczThe Capability Maturity Model Integration for Development has a long and impressive history for progressing the cause of process improvement. Agile development is a paradigm for software projects that are characterized by collaborative, cross functional teams working closely with customers to deliver functionality regularly. Many say these two approaches can't work in tandem. This paper explores this question with examples of both successes and failures.2017Agile
Agile Delphi Estimating and Analyzing Schedule/Resource Realism for Software ProjectsBlaze SmallwoodOne of the most important tasks for a software project manager is to accurately forecast the schedule and resources needed to complete the project. This paper will describe a novel approach to this problem that combines Monte Carlo-based uncertainty analysis with an innovative new technique for estimating software requirements, which utilizes an agile-style Delphi estimating methodology. This method has been successfully implemented on several government projects and is a viable option for any software project.2017Agile
A Cost Model for Early Cost Calculation of Agile DeliveriesEric van der VlietAgile software development methodologies provide the IT industry with the flexibility they need to keep up with the faster change of business requirements. In agile software delivery, upfront detailed specifications are absent, yet investment decisions need budget input. The challenge is to build a cost model that takes essential (size) and additional cost drivers into account. This presentation explains a cost model that supports the cost calculation for agile delivery early in the process.2017Agile
Assessing ERP Cost, Schedule and Size GrowthHasetetsion Gebre-Mariam, Abishek KrupanandThis study will examine percentage changes in cost, schedule, and size across Milestones A, B, C, and full deployment for DoD Enterprise Resource Planning (ERP) programs. The analysis is based on 9 fielded systems collected from DoD authoritative data sources. Cost contributors, drivers, and factors by major cost elements will also be examined. Results may be used for crosschecking cost estimates or business case analyses at an early phase to inform funding decisions.2017Data Analysis
Production Rates: Do They Really Matter?Brent JohnstoneProduction rate is widely assumed to be an important contributor to unit cost -- higher production rates lead to lower unit costs, and vice versa. Examination of published data, however, leads to a more ambiguous picture. This paper examines the impact of rate by functional cost element, including the impact on learning curves. It concludes production rate impacts are real, but the impacts are uneven and sometimes reveal themselves in surprising ways.2017Data Analysis
Lessons Learned in Leveraging Historical Cost, Schedule and Technical DataGreg Kiviat, John SwarenThe process of applying historical cost, schedule and technical data to develop new program estimates is often more difficult than textbooks suggest. The paper provides real-world insight to the issues that arise when capturing, analyzing and applying data for the next estimate. The goal is to integrate information from multiple products, systems and subsystems. Lessons learned include experience and best practices when working with financial, engineering, manufacturing and program management.2017Data Analysis
Technology Readiness Level (TRL vs. Percent Development Cost)James LinickAdvanced technology acquisition often proceeds in steps characterized by TRL of systems or components. The relationship between TRL ordinal number and percent of total acquisition cost could be used as a factor in estimating the cost of new acquisitions at each phase from concept definition through production decision. A study of historical data will be performed to establish such a relationship based on a specific definition of TRLs, which vary across government agencies.2017Data Analysis
Machine Learning Approach to Cost AnalysisKaren Mourikas, Joseph A. King, Denise J. NelsonAt ICEAA 2016, the keynote speaker presented a challenge to the cost estimating & analysis community: to develop cost estimating methodologies based on machine learning. Machine learning provides an alternate to parametric modeling. At Boeing, we are experimenting with several machine learning techniques for cost estimation and analysis; Random Forest prediction methodology, in particular, has shown encouraging results. This paper describes our cost applications of Random Forest.2017Data Analysis
Technology Development Cost and Schedule ModelingChuck AlexanderA tangible need exists in the scientific, technology, and financial communities for economic forecast models that improve new or early life-cycle technology development estimating. Industry models, research, technology datasets, modeling approaches, and key predictor variables are first examined. Analysis is then presented, leveraging a robust industry project dataset, applying technology and system-related parameters to deliver higher performing parametric cost and schedule models.2017Data Analysis
DHS Cost Analysis OverviewDarrin DeReusCost Estimating and Analysis is a vital resource for program and project managers, regardless of agency or activity. July 2014 the cost analysis responsibilities and personnel were transferred from the DHS's Office of Program Accountability and Risk Management to the Office of the Chief Financial Officer. This presentation will provide a history of the creation of a cost analysis capability in a new agency, changes over time, lessons learned and current successes of the Cost Analysis Division.2017Data Analysis
Using Real Options to Quantify Portfolio Value in Business CasesGeorge Bayer, With the challenges in quantifying program risks, interdependencies, and the business case impact on portfolios, cost estimators can use real options to estimate probabilistic value in business cases and assess portfolio value. Many business cases not only add value as stand-alone investments, but they provide the opportunity for subsequent investments. Cost estimators can measure this incremental program value and the impact of a specific investment on a larger portfolio by using real options.2017Economic Analysis
Discount Rate for Government InvestmentRon Beheler, Laura BarkerWe often use a discounted-cash-flow construct to evaluate the Economic Viability of an investment. OMB-94 provides discount-rate guidance for internal investment (treasury rates) and public investment/regulatory programs (7% - rate of return on an average private sector investment). Since any resources consumed by the government must be foregone by the private sector, we will argue that the discount rate should be based on a rate that at least proxies the value of private investment.2017Economic Analysis
Are the Rates Right? Benchmark Protection Against Escalation SWAGBrian Flynn, Brian Torgersen, Greg Mihalek, Adam JamesMedical escalation rates continue to trend higher than general inflation - with significant impact on defense firms and acquisition affordability. Modest deltas in rates translate into billions of additional costs. This research presents benchmark escalation values for employer costs of employee medical compensation. The benchmarks, in turn, support company planning & collective bargaining and liberate government cost estimates from their traditional reliance on solely FPRPs and FPRAs.2017Economic Analysis
The Rate You Want – Rethinking Cost Escalation in FAA DevelopmentSriram Krishnan, Andrew DrennonLike most civilian agencies, the FAA has used 'raw' OMB escalation indices to produce then-year costs for its life cycle cost estimates. However, as an agency which incrementally funds many development programs, could the FAA benefit from recognizing time value of money principles at the estimation stage? We look at outlay data from across the agency to determine whether defense appropriations styled weighted indexes would significantly impact then-year FAA solution development costs.2017Economic Analysis
Terminal Facility Realignment: A Business Case ApproachDeji Oladipupo, Ayn Smith, Melvin Etheridge, Richard SheproThe Federal Aviation Administration applies a business case analysis approach to the decision process for realigning and consolidating Terminal facilities. A Microsoft Excel model has been developed to produce these analyses. It uniquely displays both costs and benefits in one place and includes graphical outputs of NPV over time. A key output of the analysis shows interesting cyclical trends of NPV rising, falling, and rising again over time in almost all scenarios.2017Economic Analysis
Kennedy Space Center's Transformation to a Multi-User SpaceportTerry LambingIn the past, launch pads were used almost exclusively for government missions. To support a growing private sector space economy, NASA’s Kennedy Space Center has transformed into a multi-user spaceport capable of handling the needs of a variety of companies from launch processing and operations through recovery. This presentation will provide an overview of the transformation and the NASA Business Case requirements needed in order to gain approval, and successfully establish these unique partnerships.2017Economic Analysis
NRO CAAG Spacecraft Test Schedule Estimating RelationshipDaniel BarkmeyerThe NRO CAAG's parametric model for estimating spacecraft testing schedule duration has been updated. The updated model incorporates data from government and commercial spacecraft contracts. The intent of this presentation is to share the model with the ICEAA community. The latest model will be presented, along with a case study of a satellite program that illustrates the importance of the chosen schedule drivers to system test schedule.2017EVM & Scheduling
Fully Integrated Cost & Schedule Method (FICSM) Analysis Schedule ImplementationJonathan Brown, Benjamin T. UnruhThe Joint Agency Cost Schedule Risk and Uncertainty Handbook released in March 2014 introduced the FICSM which integrates cost, schedule and risk estimating. Key to the FICSM process is creation of an analysis schedule including duration uncertainty. This study leveraged data from an active program to develop an analysis schedule as part of building a FICSM model. This paper describes the methods used to develop the analysis schedule and duration uncertainty used for this program.2017EVM & Scheduling
Beyond RIFT: Improved Metrics to Manage Cost and ScheduleNicholas DeToreRisk-Informed Finish Threshold (RIFT) presented an innovative solution to the problem inherent in schedules that risk analysis results (time) cannot be allocated the same way as in cost models (dollars). Developing RIFT validation methods inspired an exploration into analyzing simulation data more meticulously. Methods described here provide unique insight into cost and schedule uncertainty results while introducing powerful new techniques to improve a project's potential to complete on time, on budget.2017EVM & Scheduling
Adopting DOD Best Practices in IPM to non-DOD NeedsDavid L. Wang, Daniel Schwartz, David T. ChiangNon-DoD federal acquisition agencies have their own unique acquisition requirements and acquisition culture. However, many non-DoD acquisition agencies face similar challenges, specifically in the areas of acquisition program execution and program affordability. To address these challenges, there has been increased interest to consider tailoring DoD best practices in Program Management, Earned Value Management and acquisition lifecycle schedule management. This presentation discusses a systematic approach to tailoring DoD best practices for non-DoD acquisition needs.2017EVM & Scheduling
Cost Associated with Acquisition Complexity and Differing Levels of Mission AssuranceErik BurgessIn Government acquisition programs, the cost of what we buy is largely driven by how we buy. For satellites, this includes contracting, funding, oversight, reporting, testing, parts assurance requirements, and other factors. All of these are related, and no single attribute defines the cost curve. The NRO has developed and implemented a method that quantifies the cost of these factors and circumvents the usual debate about whether or not an acquisition is streamlined or commercial-like.2017Methods
Don't Dis LOE: Modeling Production Sustaining Labor Across Multiple LotsSandy BurneyThe estimation of sustaining labor is often given little analysis; just assuming some constant level. However, in estimating production hours on large Programs, spanning multiple lots, where each lot is negotiated separately, conflicts often occur over the cost of sustainment labor. This briefing will look at different approaches to estimating and modeling sustaining labor, based upon varying cycle times for completing a single unit, and the number of units being procured in each lot.2017Methods
Automated Data Collection Using Open Source Web Crawling TechnologyAnna FooteFor many reasons, data gathering is often one of a cost estimator's biggest challenges. Even in cases where data is available, the logistics of data collection can be daunting. This paper explores a methodology developed by PRICE utilizing free, open source web crawling technology to automatically seek and find commodity pricing for information technology hardware and software items. This presentation shares the nuts and bolts for everyone's benefit.2017Methods
A Framework to Price and Cost IT Network ServicesJohn LeahyThe correct pricing of an IT network offering aimed toward internal users can be an involved process. Different services targeted to some, but not all customers makes proper cost allocation challenging. Considerations such as mirroring the structure, if not the rates, of commercial offerings; reconciliation of differences between cost allocations and rational pricing; and influencing customer purchasing behavior adds to the complexity. An approach for costing and pricing IT services that address these requirements will be presented.2017Methods
Don't Get Caught in the Learning Curve VacuumPatrick McCarthyProduction environments featuring multiple end items benefit from a manufacturer designing products and processes with a high degree of commonality and standardization. When estimating labor hours using learning curves, the estimator often overlooks commonality when considering quantities, slope, production rate and lost learning. This paper will analyze alternative approaches to defining commonality and how to account for commonality using learning curves in integrated production environments.2017Methods
Modeling Hardware Development Cost in a Low TRL / Pre-Acquisition EnvironmentJack SnyderEstimating the cost of pre-concept programs, often with very immature TRL's, is always challenging. This case study highlights the use of analogous programs and predictive modeling to overcome the challenge. We will explore the strategy of finding analogous components of a program, even when the overall program is unrelated. Predictive models can easily be tailored for the inherent uncertainty of these early programs, when we are unable to find relevant data.2017Methods
Exploring the of Results of two Methodologies for Unmanned Space EstimationJohn SwarenIndependent validations of two separate predictive analytics methodologies were performed over the last two years. In both cases, common detailed assessments of past missions integrated technical and programmatic requirements to mimic a grass-roots bottom-up methodology. This presentation compares the process and top-level results from both approaches as well as explores several lessons learned.2017Methods
Deployment Cost Estimation for Electronic/IT SystemsF. Gurney Thompson IIIThis paper will discuss our research into cost estimation for the deployment of Electronic and IT systems, also known as Operational/Site Activation. The deployments can span multiple sites/locations, and often require technical studies, end user working groups, and site survey visits in addition to the actual installation activities. This paper will discuss our research approach, sizing metrics, cost drivers, model structure, and future work.2017Methods
Budgeting for Canadian Shipbuilding– Examining Predictive Cost Analytic MethodsPeter Weltmann Zachary JasnoffPredictive Cost Analytics plays a major role Canada's Shipbuilding Strategy in developing cost and budget estimates. However, these estimates must be based on sound historical cost data. This paper discusses challenges faced by the Canadian PBO in developing reasonable estimates for the JSS, AOPS and CSC shipbuilding programs based on the quality of historical data available, the methodologies employed to address these challenges, and the influence of these estimates on decision-makers2017Methods
Decision Trees and Cost EstimatingJosh WilsonDecision trees are predictive modeling approaches that recursively partition data, and apply simple prediction models to the resulting subgroups. Decision tree based prediction approaches have the benefits of being easy to interpret and explain, requiring little data preparation or cleaning, and the ability to model complex nonlinear relationships and variable interactions. This presentation explores the applicability of decision tree based prediction methods to cost estimating.2017Methods
Cost Estimating Canada's Future Surface CombatantsRod StoryCanada has not built a new surface combatant since it completed the last of its 12 Halifax class frigates in 1996. Currently Canada is in the middle of a request for proposal to build, using a common hull, 12 new frigates and 3 new destroyers. This paper presents the first public cost estimate of these ships highlighting both the challenges that were encountered and the solutions that were used in determining a reliable cost estimate.2017Methods
Modeling the Influence of System and Application Complexity on the Cost of Cloud HostingDaniel J. Harper, Kevin BuckTo refine cost estimates produced within a Cloud Total Ownership Cost model, MITRE is now prototyping an additional modeling feature that estimates the cost impact of migration and hosting complexity. While many cloud cost models rely on a limited set of cost drivers, other factors associated with general complexity significantly influence costs and are often ignored. Within the prototype, complexity assessments are plotted using intuitive spider diagrams for ready comparisons of candidate systems and applications for hosting in various cloud solutions.2017Methods
Marco-Parametrics; Its Unique Capability and ApplicationDale Shermon, Arlene F MinkiewiczThe Family of Advanced Cost Estimating Tools (FACET) has been used around the world as a macro parametric cost model at the early stages of a project life cycle. It has a unique capability to seamlessly transition from performance to design based estimating. FACET will also combine uncertainty in the inputs with those of the algorithm to produce a true uncertainty range. Now implemented in TruePlanning the FACET model is easy to access and apply.2017Methods
Projecting Program Spare Parts Sustainment with Incomplete DataBryan Anderson, George BayerAnalyzing a government acquisition, the team discovered that the existing agency supply chain system lacked transparency, and historical failure data was unavailable due to implementation of a new supply chain system. The team examined how to accurately project parts demand using incomplete data by conducting a comparative analysis between data sources, using subject matter experts to estimate upper and lower bounds, and applying statistical analysis fit curves to project spare parts needs.2017Operating & Support
Sustainment Cost Data CollectionSandi EnserOperations and support costs can exceed 60% of the total Life Cycle Cost of DoD systems. Currently, the individual services have their Visibility & Management of Operating and Support Costs (VAMOSC) systems. However, with many sustainment functions performed by contractors, improved reporting on sustainment contracts is essential. This paper presents several data collection initiatives for sustainment efforts and their impacts on estimating O&S costs.2017Operating & Support
Army Software Maintenance Cost Estimating RelationshipsCheryl Jones, James Doswell, John McGarry, Jenna MeyersFor the past four years, the Army, under the leadership of DASA-CE, has been collecting and analyzing Army system software maintenance cost and technical execution data to support the development of more accurate cost estimation methods. The presentation will present the cost methods and cost estimation relationships developed from the analysis of the execution data sets. The results of the Army's analysis efforts, including the detailed statistical analysis will be made available.2017Operating & Support
Implementing Additive Manufacturing Technology into the Logistics Supply ChainPatrick K. Malone, Bruce FadRecent explosive growth in Additive Manufacturing (AM) or 3D printing is providing logistics supply chain economic opportunities. We investigate self-sufficient repair and maintenance capabilities for isolated environments, impacts on strategic readiness, increasing responsiveness and cost efficiencies not available in traditional supply chains. Our research will evaluate and contrast legacy logistics architectures against AM elements that will drive cost downward over lifecycles to meet current and future affordability goals within the Government and commercial organizations.2017Operating & Support
Expanding the Range of Your Data: A Small Ships Case StudyKathleen Hudgins, Robert Nehring, Elizabeth Koza, Anna IrvineWhile the Navy has comprehensive Operating and Support (O&S) data for current Navy ships, there are a limited number of smaller boats available for inclusion. With increasing technological advances and a renewed effort to reduce personnel, smaller boats are receiving increased interest. This paper explores using Coast Guard O&S data to supplement Navy data for use in estimating O&S costs for smaller boats. Topics include data sources, normalization, and comparisons of data between the services.2017Operating & Support
Assessing Confidence Levels in Funding and Budgeting NASA Science MissionsRobert E Bitten, Charles D. HuntNPR7120.5E requires that NASA utilize joint confidence level (JCL) analysis to set budget and funding guidelines for projects within its portfolio of missions. This paper addresses analysis that was conducted to determine the effect of different confidence levels on the performance of different cases of mission portfolios. The results show that the most effective confidence level varies depending on the case. An overview of the policy, methodology, cases and results are discussed.2017Parametrics
Data Driven Confidence Regions for Cost Estimating RelationshipsChristopher JarvisCost estimating models typically contain many uncertain parameters. The uncertainty of the parameter values drives the uncertainty of the model outputs. In this paper we review and compare the techniques to compute the confidence regions for linear and nonlinear regression methods. Application to two common CERs is presented with comments regarding the practical implementations and limitations within the current cost estimating environment.2017Parametrics
Moment One, PleaseChad Krause, Erik BurgessCost models constructed without statistical correlation are designed to underestimate the mean, unless they are simple sums. Any model that uses a factor or other instance so functional correlation will miss the first moment, not to mention others, unless statistical correlation among the CERs is also applied. An internal audit of 13 recent satellite cost estimates comprising of 52 CERs shows the mean cost changes by as much as 9% when this correlation is applied.2017Parametrics
Modeling with Gumby: Pros and Cons of the Weibull CurveMichael Mender, Ann HawpeThe Weibull function is used to model various cost phenomena. This popularity is driven by its significant flexibility, driven by its nature as a two parameter function. Additional parameters result in flexibility, which while advantageous, also places incorporating estimates at risk of over fitting (i.e., a model that fits the observed data well, but has little predictive power). Presentation relays theory and exhaustively reviews pros and cons for cost estimating.2017Parametrics
Maximum Likelihood Estimation for Regression of Log Normal ErrorChristian SmartThe use of Log-transformed Ordinary Least Squares (LOLS) has been criticized for the use of transformation, which results in biased estimates. LOLS is a maximum likelihood estimate (MLE) of the median when the residuals are lognormally distributed, for which we provide evidence. We discuss MLE and show how to use this technique to directly model untransformed lognormal error. We also discuss two other popular methods, ZMPE and MUPE, in the context of MLEs.2017Parametrics
Novel Manufacturing Methods ~ Characterizing the Impacts to CostZachariah Sayre, Frank CampanileAverage unit procurement costs for military aircraft are growing at a rate that far exceeds inflation. Efforts such as the Air Force Research Lab's Composite Affordability Initiative and the Advanced Composite Cargo Aircraft have demonstrated alternative manufacturing methods utilizing composites to achieve significant part and fastener count reductions. This analysis highlights the concepts of large composite unitization in fixed wing aircraft manufacturing and its resulting effects to cost.2017Parametrics
Assuring Credibility in the Cost EstimateHenry ApgarCredibility can be the most important attribute of a cost estimate. This paper traces the evolution of quality metrics that assess cost credibility in the words of senior government executives, industry leaders, estimating and engineering handbooks, professional journals, and government auditing manuals. The presentation concludes with recommendations for the estimating professional.2017Policy & Standards
In Theory Or In Practice? The Optimistic World of Pessimistic Cost EstimatorsErin K. BarkelIn 2015, Andy Prince won the Best Paper award for The Psychology of Cost Estimating, a piece which highlighted a number of cognitive biases which impact the work of cost estimators. Of these cognitive biases, the one that seemed to resonate most strongly with the community was optimism bias. However, not everyone is worried that the glass is half-full. Where do negativity and pessimism bias fit? How does our understanding of these biases change how we communicate with stakeholders?2017Policy & Standards
How to use Predictive Analytics for a DCAA-Compliant Estimating SystemAnthony A DeMarcoThe DCAA ensures that Defense contractors use an acceptable estimating. They establish acceptable estimating system compliance criteria and audit's contractors to determine compliance. Non-compliance can be devastating. This presentation demonstrates a data-driven estimating methodology that complies and reduces the time and labor necessary to produce estimates. It illustrates the roles and procedures necessary for success. Six key compliance tenants are identified and key benefits quantified.2017Policy & Standards
Developing a Cost Capability RoadmapJohn FitchDeveloping cost estimating tools and databases is critical to analyst productivity and the quality of analyses. Historically, organizations struggled to systematically improve capabilities due to a lack of a long-term plan, unstable budgets, and priorities that shift with changes in leaders. This presentation walks through how one organization confronted this challenge by developing a roadmap that links the mission to requisite tools, data, and training; and a plan to achieve the roadmap.2017Policy & Standards
The Shortcomings of Cost Estimating TemplatesMeagan Gadreault, Faye KimTemplates are seen as a best practice in many industries to standardize workflow and reduce redundancy. However, templates may not be the ideal solution for cost estimating as they can suppress innovation and constrain analysts. Instead, best practices should be determined using process improvement methods and implemented to create customized estimates. This paper explores leveraging Lean Six Sigma techniques to determine alternatives to templates in order to put forth the best estimates.2017Policy & Standards
The Art of Cost: Sun-Tzu's Strategic Insight in Cost EstimationBrian A. GillespieStrategy is the process employed to determine how political purpose is translated into action. Clausewitz in his grand work "On War" argued that war serves a political purpose. So too does cost estimation, for at its most fundamental level, cost estimation attempts to determine the level of resources needed to achieve a political outcome. As one of the greatest enduring works on military strategy, what does The Art of War teach us about cost estimation?2017Policy & Standards
Coherence & Oddities: A Retrospective of Cost Estimating Publications 1978-2016Ross A. Jackson, Bradley C. BoehmkeIntellectual sedimentation can operate unobserved within professions. Assessing publication trends serves to make such conventional wisdom explicit. Our analysis of articles from the Journal of Cost Analysis and Parametrics (and its predecessors) provides insight regarding research coherence and oddities within the cost estimating community. This knowledge is essential for envisioning alternative futures and could be of benefit to those engaged in the praxis of cost estimating or its research.2017Policy & Standards
A History of Thought in Defense AcquisitionsEric M. LofgrenAs Congress debates another round of defense acquisition reform, the necessary role for the cost estimator is affirmed. But how did this role come about and what are future implications? From the famed RAND systems analyses of the 1950s to the introduction of data reporting systems still in use today, this paper will explore the rich history of thought in defense acquisition, giving a special eye to controversies and continuing challenges that affect cost estimators.2017Policy & Standards
Establishing and Implementing Cost Estimating StandardsHetal Patel, Denise Dulee, Danielle SpencerAs part of its efforts to create a world-class cost estimating capability, the Missile Defense Agency (MDA) wrote a cost estimating handbook and established an independent assessment team to ensure adherence to the handbook. As a result of this, MDA has received positive feedback from GAO in its progress towards this goal. This briefing describes in detail the processes and procedures that MDA has developed.2017Policy & Standards
Estimating Challenges and Solutions at NASA: Past, Present, & FutureCabin Samuels, Jeff BrownNASA's cost estimating landscape is fraught with unique challenges. Missions often incorporate cutting-edge technology and operate in harsh environments with no margin for error. Goddard's Cost Estimating, Modeling, & Analysis (CEMA) Office has developed robust methodologies to create realistic, transparent cost estimates for missions with little or no cost heritage. This paper discusses past and current states of cost estimation at Goddard and provides insights to shape future improvement.2017Policy & Standards
Anatomy of the Future DoD Cost EstimatorTamiko L. Ritschel, Jonathan D. RitschelThe need for a specialized, properly qualified DoD cost estimating workforce has resulted in a migration of civilian billets from a financially focused job series to the 1515 Operations Research series. We analyze the origins of this change, implications for the current workforce, and then discuss the anatomy of the future DoD cost analyst. While we focus on recent changes occurring within the U.S. Air Force workforce, the future implications are relevant to all Services.2017Policy & Standards
Masters in Cost Estimating and AnalysisDan Nussbaum, Greg MislickThis presentation provides an update on the all-distance Learning Master's Degree and/or Certificate Program in Cost Estimating and Analysis (MCEA / CCEA) offered at the Naval Postgraduate School (NPS).2017Policy & Standards
Contract Geometry Best Practices for Incentive ContractingPeter J. BraxtonWith the current emphasis on Incentive Contracting, the cost analyst plays a vital role in establishing target cost, fee, and shareline. Drawing from the Joint Contract Price Database, this paper examines actual contract geometries across a wide range of programs and their effectiveness in encouraging cost management. Using the published Risk-Based ROS methodology, it provides a framework for implementing incentive arrangements for both competitive and negotiated procurements.2017Risk
A 'Common Risk Factor' Method to Estimate Correlations Between DistributionsMarc GreenbergA 'common risk factor' method uses expert-derived risk factors to estimate correlation between two distributions. The premise is to estimate mutual information among risk factors that underlie each distribution. Approximation of such mutual information leverages properties of the joint probability distribution of a unit square. Geometric outputs of each pair of common random variables are compared to estimate common risk factor 'intersections' that are, in turn, proxies of correlation.2017Risk
The End of S-Curve Alchemy: Gold from a New SAR DatabaseTodd Andrews, Jeffrey Pincus, Brian FlynnS-curves typically address only "known unknowns." They're built upon an analysis of cost risk and uncertainty – element by element. But, they fail to address the ambiguity of the curve itself, or the degree of confidence that its proffered probabilities are actually correct. This confidence factor, of course, is hard to measure. The "unknown unknowns," in other words, are not explicitly captured since their probabilities are a mystery. This research fills the gap in knowledge.2017Risk
Being Certain About Uncertainty, Part 1Andy PrinceDoing cost risk analysis is hard because we don't really know what a good cost risk analysis looks like. In this paper we will explore the challenges to doing good cost risk analysis and discuss ways to know if your cost risk analysis is any good. We will also examine the phenomena of extreme cost growth and lay the groundwork for future work.2017Risk
Integrated Cost-Schedule Risk Analysis Improves Cost Contingency CalculationDavid T. HulettThe main benefits of integrated cost-schedule risk analysis are improvement of the estimates of cost contingency and identification of the main risks to cost for mitigation purposes. The main focus will be on estimating the cost contingency needed and identifying risks to cost, which may be independent of schedule or indirectly due to schedule risk. New simulation software including iterative risk prioritization will be used to illustrate these points.2017Risk
Using Quantum Theory and Monte Carlo Multiverse to Manage RiskNathan EskueNeed to better predict your EAC? Tired of being surprised? Great! Not an expert on quantum theory or the multiverse? No problem! This hands-on session will give a high level view of how quantum theory can help us understand our EAC by viewing it as a multiverse. We'll then explore an Excel model that takes your EAC and uses Monte Carlo analysis to reveal all possible futures. Attendees get a copy of the model!2017Risk
Software Projects Estimation & ControlAlain AbranThis talk presents 1) an outline of the design of the '2nd generation' COSMIC Function Point method for measuring a functional size of software; 2) Industry evidence of its versatility & value in software project benchmarking & estimating in business, real-time, infrastructure, component, mobile, IoT, cloud software, via traditional & agile methods; 3) how it can be used for early and rapid sizing at estimation time; 4) how size measurement can be automated with very high accuracy.2017Software
Bottom Up Methods of Estimating Software SE/PM and Non-DCTI CostsJames BlackSystems Engineering/Program Management (SE/PM) and additional non-Design, Code, Test, and Integration (Non-DCTI) activities performed during software development efforts are often significant and drive estimates of total project costs. Yet, cost estimates often omit the detailed research and analysis needed to adequately model SE/PM & Non-DCTI costs. This brief will present bottom up methods useful for understanding and estimating these costs and share analysis of recent SE/PM & Non-DCTI data.2017Software
Reliable Non-Design, Code, Test, and Integration Cost RelationshipsJeremy Goucher, Brittany StaleySoftware cost estimates require ratios derived from historic cost reports for non-design, code, test, and integration (NDCTI) cost elements. Since NDCTI accounts for as much as 50% of the estimate, a comprehensive historical data set is critical to ensuring an accurate estimate. The authors have recently analyzed over ten years of actual cost data from DoD command and control systems to develop a new set of NDCTI ratios. The results also bring new insight into "fixed" versus "variable" cost.2017Software
Objective SLOC: An Alternative Method to Sizing Software Development EffortsAndrew KicinskiEquivalent Source Lines of Code (ESLOC) is the basis of methodology used by many organizations for collecting and estimating software development costs. Selecting ESLOC parameters requires insight into the software reuse. Too often data collectors are unable to verify the appropriateness of the assigned ESLOC parameters and validate their implementation. This paper examines the drawbacks of ESLOC, and presents an alternative and more objective method to estimating software development effort.2017Software
Software Effort Estimation Models for Contract Cost Proposal EvaluationWilson Rosa, Corinne WallsheinThis study will introduce regression models and benchmarks for predicting software development effort using input variables typically available at contract bidding phase. The analysis is based on 200 DoD projects delivered from 2004 to 2016. The first set predicts effort using initial software requirements along with peak staff, application domain, and other categorical variables. The second set predicts effort specifically for agile software development using data from 16 agile projects.2017Software
Software Size GrowthMarc Russo, Wilson RosaSoftware cost estimating relationships often rely on software size growth percentages. Actual delivered source lines of code (SLOC) may be predicted with categories of early code estimates such as new, modified, reuse, and auto-generated SLOC. Uncertainty distributions will be presented to represent growth by code category for use in cost modeling.2017Software
Analysis of Software Cost Estimation Models Using Normalized, Stratified DataDan StricklandSoftware Resources Data Reports (SRDR) are the DoD's effort to improve cost estimation efforts by collecting metrics on software programs. Existing SRDRs were normalized in a manner consistent with previous efforts and productivities were analyzed using three popular software cost estimation models (SLIM, SEER, and COCOMO). Calculated values were tested against model thresholds and the results were tested for accuracy. Findings indicate some models outperform others and schedule impacts results.2017Software
The Journey from "Bottom-up" to Predictive Modelling BOELori SaleskiPredictive Cost Modeling has the potential to save significant money over conventional bottom-up bidding with better accuracy. This paper outlines a process to implement predictive cost analytics in an organization that been dependent on bottom-up estimating for decades. Walk with us down the path discussing implementation, change management, adoption and stakeholder challenges. Explore architecting the solution for maximum leverage at ROI.2017Software
Observation of Production Rate Effect observed in Historical Weapon Systems CostWilliam Banks, Timothy LawlessProduction Rate Effect has been observed in several weapon systems program’s historical cost data during the procurement phase of the acquisition life-cycle. The main contribution of this paper is to illustrate real life observations of production rate effect and the economies of scale shown when purchasing end items from major defense contractors.2016Acquisition
Federal Agency Independent Government Cost EstimateCassandra M. Capots, Lauren Nolte, Lavanya YeleswarapuTo achieve mission success in today’s constrained fiscal environment, federal agencies must establish a fair and reasonable cost baseline to make informed decisions. This baseline is commonly called an Independent Government Cost Estimate (IGCE). We have developed a customizable process that forms a starter kit for any agency that wishes to quickly and effectively perform IGCEs. We discuss best practices, lessons learned, and implementation tips for both DoD and civilian agencies.2016Acquisition
Early Stage Cost Estimating for Radars and SensorsJeremy GoucherThe majority of costs for programs are locked in even before a program enters production, which makes accurate early stage cost estimates vital for effective resource management and program success. This study proposes a method for analyzing combat system cost prior to a complete requirements description. The data set includes old, new, big, small, ground, sea, air, domestic, and foreign systems. The result is a model that requires limited data and is widely applicable.2016Acquisition
The ABCs of the UCF: A look at Section L and Section MJennifer LeottaThis paper will examine the government cost analyst’s role in the acquisition/contracting process. Specifically, it will look at the Uniform Contract Format (UCF) and how to create effective language for Section L that provides clear directions for offerors and includes all information necessary to complete a thorough proposal evaluation. It will also look at developing Section M language for varying contract types (cost plus, fixed price, etc.) that is consistent with the Federal Acquisition2016Acquisition
GAO-COFC Concurrent Bid Protest Jurisdiction: Are Two Fora Too Many?James LinickThe Government Accountability Office (GAO) and the Court of Federal Claims (COFC) are the bid protest fora where a disappointed offeror may challenge an agency’s procurement decision. Critics claim that concurrent bid protest fora, which have differences in jurisdiction, standards of review and remedies, creates inefficiency in the procurement system, e.g. serial protests. This presentation examines whether concurrent bid protest fora are healthy for the US federal procurement system.2016Acquisition
A Weapons Acquisition Case Study: Cost Overruns and Schedule SlipsRandy Bowen, David L. Peeler, Jr.This story is true. Certain details have been omitted to “stump the audience” and protect the guilty. The trials and tribulations of an acquisition program cost overrun are explored for persistent lessons and future applications. This saga includes materiel difficulties, workforce issues, congressional interest, funding and schedule perturbations. The aim is to educate, as the persistence of fundamental acquisition problems and the insignificance of acquisition reforms are framed.2016Acquisition
Bridging the Gap between Capital Investment Valuation in Private & Public SectorGeorge Bayer, Jr.The public and private sectors have different valuation techniques and considerations when evaluating capital budgeting decisions. Understanding the difference between private and public sector capital investment analysis (discounted cash flow, cost/benefit, tax impacts, stakeholders) helps decision-makers make better informed decisions. In the cost estimating community, understanding investment distinctions makes us better stewards of information and more effective Finance professionals.2016Business Case Analysis
ROS vs. ROC: Government Can Lower Price by Raising FeeSandy BurneyMany Government Contractors measure profitability using Return on Sales (ROS), which includes Fee in the denominator. The FAR regulates fee percentages or Return on Cost (ROC). In negotiating sole-source contracts, this metric difference can result in the Contractor negotiating a higher risk adjusted cost and price to offset lower fees offered by the Government. The briefing provides a mathematical example of the tradeoff between Cost and Fee for a CPFF contract.2016Business Case Analysis
Developing a Dynamic Expense-Volume-Profit Model to Determine Break-Even pointWilliam KentAnalysts must be able to determine when a program will be profitable. This insight is used to drive contract negotiation, pricing, and strategy. Many factors need to be evaluated: expenses, revenue, and volume of sales. These factors must be determined based on limited historical data and extrapolated over a time horizon. This paper introduces a framework for analyzing these factors using learning curves, regression, and financial statement analysis to determine break-even.2016Business Case Analysis
Fixing the Flawed Nature of DoD BCA'sWilliam G. WilliamsonThe government is not a business that is profit-oriented. Monetary business metrics as a means of determining what items to consume are inherently flawed for the government since metrics such as Return on Investment’s assume the organization is a provider of the goods or services, not a consumer. A better approach for the government would be to measure the areas that the government most values, i.e. schedule and requirements. Using those areas as anchors, a best value metric can be developed, pointing the government to smarter acquisition decisions.2016Business Case Analysis
3D Printing in the DoD; Employing a Business Case AnalysisNicole Santos, Richard Shea, Robert Appleton3D Printing is the family of technologies that enables users to produce items on demand from CAD files or 3D scanning. The potential benefits to military logistics include cost savings, weight reduction, and responsiveness to the warfighters’ needs. To demonstrate and measure the benefits in the Department of Defense (DoD), a rigorous Business Case Analysis (BCA) will identify benefits and challenges to implementation including evaluating its costs, risks, and benefits.2016Business Case Analysis
Training Cost Analysts, a Cohesive Pedagogical Framework for SuccessKammy Mann, Danielle LainezHow do we train cost analysts? This paper will take a critical look at peer-reviewed academic research regarding pedagogy, or the method of teaching. Leveraging existing knowledge, current best practices and resources available, a framework for teaching and training will be proposed to the cost community. By establishing a consistent methodology for educating our professional workforce we can ensure that new members of the cost analysis field have the tools and skills to succeed.2016Business Case Analysis
Analyses of Alternatives for Space SystemsR. Alex WeklukThis presentation provides an overview of recent experiences performing satellite analyses of alternatives, including technical and cost trades. Our team developed a scalable mechanism to perform satellite sizing using physics and stochastic modeling. Once modeled, an array of costing methods is applied to develop repeatable and defensible estimates. The team made great strides in presenting the large number of nuances to decision makers.2016Business Case Analysis
Begin With a Joke? What John Oliver Can Teach Us About Communicating Cost EstimatesErin K. BarkelAre you having a hard time holding management’s attention when you explain the results of your estimates? It’s time to change your approach. Consider this: millions of people tune-in and click to watch John Oliver talk about cost estimates gone wild, and they stay tuned-in for 20 whole minutes. What can you learn from a comedian about explaining cost estimates? Maybe you should start with a joke.2016Business Case Analysis
Department of Defense Commercial-Off-The-ShelfHeather L. BrownNCCA partnered with DoD Enterprise Software Initiative (ESI) to periodically receive commercial-off-the-shelf (COTS) software orders with the intent of improving software cost estimating. As of today, more than 250,000 orders from the Army, Navy, Marine Corps, and other federal entities have been recorded. Together, these represent over six billion dollars in sales. NCCA examines this data and categorizes various product descriptions as licensed products, maintenance and support, or services in order to present statistical trends.2016Methods & Models
Naval Sea Systems Ship Command Logistics Requirements Funding Summary ToolJeremy EdenThe United States Navy has focused on planning and estimation of operations and support costs for programs. A task force created by Secretary Stackley (Assistant Secretary Navy Research, Development & Acquisition) to identify actions for achieving this goal recommended creating a logistics tool similar to that developed by the United States Marine Corps (USMC). This paper demonstrates the tool, development approach, cost estimating methodologies, interface, and deployment to the next generation2016Methods & Models
Footprints in the Sand: A Conversational Approach to Basis of EstimateFrank R. FlettThe Basis of Estimate (BOE) is the mechanism by which the cost estimator or cost proposal preparer hopes to convince the evaluator or reviewer that the method he/she took in preparing the estimate was “reasonable” (not the same as “correct”). This paper will present a technique of writing BOEs as if they were a part of a conversation with the evaluator/reviewer, who is, after all, a human being who will early on develop a distinct impression of the “goodness” of the end result.2016Methods & Models
Estimating the Cost of the Aegis Flight III Test PlatformJeremy GoucherThe Flight III upgrade to the Aegis Destroyer program brings unparalleled new capabilities to the surface Navy fleet which will require in depth testing prior to deployment. For live fire testing, the Navy is considering four unique options to procure, install, and test the required systems on an unmanned vessel. This paper discusses the available data, methodology, and results of the cost estimate. The paper will also discuss some of the technical challenges associated with each option.2016Methods & Models
Beyond Anderlohr: An Alternate Approach To Estimating Production BreaksBrent JohnstoneEstimating the cost impacts of production breaks has long been problematic. Use of the Anderlohr method is widely accepted, but requires a significant degree of estimating judgment and can produce substantially different answers based on the individual user’s assumptions. This paper suggests an alternate empirical methodology based on recent research on organizational learning and forgetting.2016Methods & Models
Delphi Methodology for Cost EstimatingCole J. Kupec, IIThe Delphi methodology can be a practical and valuable tool for cost estimators. This paper explores its proper implementation based on current research to include group selection, question formulation, statistical analysis, and the survey process. Special attention is paid to tailoring the Delphi method to cost estimating applications. A case study is presented for example purposes. The history of the Delphi method as well as modifications to the traditional Delphi method are also presented.2016Methods & Models
Beyond CSDRs: Collecting Contractor Cost Data for Detailed Cost EstimatesTim LawlessQuality cost estimates are based on actual costs, and for the Department of Defense, usually are acquired through Cost and Software Data Reports (CSDRs). However, CSDRs may not be available, complete or consistent for quality analytical purposes. This paper examines the collection and use of detailed contractor actual costs for estimates, such as with Priced Bills of Material and invoices, and their analytical benefits, perhaps supplanting CSDRs as the preferred data source for cost estimators.2016Methods & Models
Using Physics Based Reliability Methods for Should Cost EstimatesPatrick K. MaloneValue based should cost methodologies supporting Better Buying Power tenants can help realize significant MDAP cost savings over the life cycle. The use of Physics Based Reliability techniques in program development is described to meet affordability goals, identify cost drivers affecting reliability, availability and maintainability of weapon systems, and demonstrate PBR advantages over the classical reliability growth model with actionable plans to maintain technological superiority.2016Methods & Models
Workload-Adjusted Labor Cost ModelMichael MenderGovernment estimators typically use Forward Pricing Rate Agreements (FPRAs) to estimate future labor costs. FPRAs make assumptions about future workload, and can be inaccurate as realized workload often differs from projections. To enable understanding of this uncertainty, the Naval Center for Cost Analysis (NCCA) developed a lagged regression model that predicts future overhead costs as a function of workload, allowing NCCA to quickly and accurately assess the impact of Navy procurement decisions on acquisition programs.2016Methods & Models
Challenges in Estimating the Development Cost of Microcircuit TechnologyChristopher Price, Gurney ThompsonElectronic Technology continues to change at a rapid pace. Significant changes in the level of integration of electronic components dictate that circuit designs using these parts be redesigned every three to five years. Estimating the development cost of these new designs can be challenging. This paper will discuss these challenges in more detail, and describe some methods and tools available to rapidly and accurately estimate the development, production and O&S cost of these new technologies.2016Methods & Models
Estimating the Costs of Future Air-to-Ground WeaponsTom SandersThe weapons acquisition community has been acquiring air-launched weapons in a slowly-evolving manner for years. While technologies experience great leaps forward, acquisition strategies evolve at a slower pace. The AF Research Lab is evaluating revolutionary weapons designs that facilitate newly-emerging acquisition strategies…thereby presenting real challenges to cost estimators. This paper documents techniques for estimating costs of those future weapons.2016Methods & Models
The Space Situational AwarenessJames Smirnoff, Brennen WoodruffToday there are over a thousand operating satellites orbiting the earth providing a variety of critical functions such as communications, navigation, weather sensing, reconnaissance, and astronomy. As the space environment becomes more crowded, Space Situational Awareness (SSA) is increasingly important. This paper describes a new tool called the Space Situational Awareness (SSA) Cost Model which provides an objective and data-driven framework to estimate the cost of ground-based electro-optical components of future SSA architectures.2016Methods & Models
Crew and Space Transportation Systems Cost ModelRichard WebbAs part of the MSFC Engineering Cost Office’s new Program Cost Estimating Capability (PCEC) suite of cost estimating tools and capabilities, we are developing the Crew and Space Transportation Systems Cost Model (CASTS), a new, unique cost model for use in estimating crew and space transportation systems. This paper will provide an overview of the capabilities, estimating approach, historical database, and key features of CASTS as well as plans for future improvements.2016Methods & Models
Organization Cost Estimation as a Human Intensive Systems Engineering ProblemDavid Bloom, Robert WrightThis paper will compare and contrast the difference between Intensive and Non-Intensive Human Interactive Systems. Further, this paper will investigate the role of Governance, Culture, Process and Tools (GCPT) in each of the 2 systems, Intensive and Non-intensive. Finally, this paper will apply the GCPT Framework to the system of Organizational Cost Estimation and provide examples of results that Raytheon has seen in its effort to improve the accuracy, affordability and accountability of the current bidding/estimation process2016Management, EVM & Scheduling
Conducting Root Cause Analysis for Should Cost ManagementNancy A. R. Droz, Mr. Robert L. HernandezRoot-cause analysis (RCA) is typically performed after a technical failure or after a program anomaly is found. However, the technique is key to the development and selection of effective initiatives to reduce program cost. RCA is a critical aspect of successful Should Cost Management execution. The presentation will provide a how to approach of applying root cause analysis during the evaluation of program data to develop more effective costs reductions. Lessons learned will be presented.2016Management, EVM & Scheduling
EVM’s Potential for Enabling Integrated Cost-Risk ManagementDavid R. Graham, Bryn TurnerIntegrating the cost estimating, EVM analysis and technical risk management disciplines to realize comprehensive cost-risk management has proven elusive. Recently developed EVM cost-risk tools offer the potential to successfully enable integrated cost-risk management. The paper will illuminate the process of how integrated cost-risk management can be realized through cooperation of cost estimating, EVM analysis and technical risk management disciplines and briefly describe these new EVM cost-risk tools.2016Management, EVM & Scheduling
Applying Earned Value to Agile Development ProgramsBob Hunt, Michael ThompsonAgile Software and Agile Development continue to dominate the Federal acquisition arena. This presentation will address current trends in estimating and applying Earned value to Agile programs.2016Management, EVM & Scheduling
Provide Agile Analysis to Stakeholders Using Category Tagging ApproachEric HongIt is becoming more and more important for organizations to use data analytics for improved decision-making to create a competitive advantage and/or operate efficiently. This paper will describe the challenges in creating and implementing a strategy. It will identify potential reporting requirements and how to leverage existing work products as input into a standard structure feasible to perform analysis for different stakeholders. The proposed solution will increase the accuracy and agility of an organizations’ analysis.2016Management, EVM & Scheduling
Putting Schedule Quality Checks to the TestEric M. LofgrenAnalysts often use the 14-Point Assessment for Schedule Quality as a benchmark for determining the overall reliability of a schedule. But how much of the variation in schedule error can be explained by the results of the 14-Point check? This paper will use actual defense contract data to find the correlates of schedule reliability, measured using both the accuracy and the timeliness with which the schedule slip is predicted.2016Management, EVM & Scheduling
Preventing Program Management Pitfall Using Portfolio EstimatingNick Morales, Christopher DewberryEstimating the effort, time, and resources needed to complete project activities is one of the most challenging tasks that project managers must face. Along with the inherent uncertainty associated with managing these activities, additional management uncertainty occurs when there are multiple incremental efforts associated with that project. In order to handle the complication of managing several program increments, each with their own budget lines, portfolio estimating is a method that helps to provide program managers with a fully comprehensive overview of the resource requirements across multiple projects. Program managers can use the portfolio estimate as a tool to help make informed management decisions at the appropriate level and identify how those decisions impact the entire program.2016Management, EVM & Scheduling
Cost Consistency and Completeness as an Impossible ExerciseDavid L. Peeler, Jr.In previous papers, Hilbert’s Problems were used to propose and revisit Hilbert’s Problems for Cost Estimating. This paper employs Gödel’s theorem w/respect to Hilbert’s application onto cost. What can we learn about ourselves as estimators and where can we exert greatest impact with our estimates? Using Godel’s two theorems of undecidability as catalyst, we explore the effect and utility of exacting math and other notions on cost estimates specifically and programmaitics generally.2016Management, EVM & Scheduling
The Performance Metrics Model and StudyDerreck Ross, Haitham Ghannam, Richard C. LeeThe Performance Metrics Model and Study (PMMS) is a comprehensive technique for selecting a performance metric TCPI. The PMMS uses historical data of completed products to determine the best EVM estimation technique for similar products that are in progress. The result is an auditable and repeatable approach that increases the objectivity of an Estimate at Completion. This approach can be applied across any program that has historical data.2016Management, EVM & Scheduling
Fiscal Sustainability of Canada's National Defence ProgramPeter WeltmanThis report provides PBO’s estimate of the fiscal gap between the status quo budget allocations and the cost of sustaining Canada’s status quo national defence forces. Parliamentarians may wish to examine scenarios that will reduce or eliminate the gap between the cost of maintaining the current force structure and the amount of funding being allocated to paying for it.2016Management, EVM & Scheduling
Liars! Why Cost Estimators and Budgeters Inflate and Underestimate Costs!Travis Winstead, Ann HawpeProjects and programs regularly suffer from poor cost estimating techniques that result in insufficient budgets and unanticipated cost overruns. To issue a contract or set a realistic budget, the cost analyst must verify and validate the mountain of cost information and determine a fair and reasonable price for a contractor’s goods and services. This paper will discuss strategies to assist program analysts in reviewing cost estimates and strategies used to support Government negotiations.2016Management, EVM & Scheduling
Multiple Regression: Prediction & Confidence Intervals DemystifiedStacy M. DeanWe all know the value of prediction/confidence intervals as a measure of the certainty in the best fit line for a regression equation; however, once we get past bivariate regressions how to actually calculate these statistics gets a little murky. This paper attempts to de-mystify multivariate confidence/prediction interval calculation as well as provide some fun facts on the origin and history of these infamous intervals.2016Parametrics
How Predictive Analytics is Improving Parametric Cost EstimationAnthony A. DeMarcoPredictive analytics is defined as the use of data, statistical algorithms and machine-learning techniques to identify the likelihood of future outcomes based on historical data. This paper will explore the history and parallels of parametric estimating and predictive analytics. It will highlight how cost management professionals can use current predictive analytics tools, methods and talent to improve their estimates and their success rates.2016Parametrics
The Collinearity Kill Chain: Detect, Classify, Localize, NeutralizeBrian Flynn, Adam JamesThis paper examines the issue of multicollinearity from the fresh perspective of a statistical warfighter. A kill-chain of steps is offered to defeat this illusive foe. Heuristics, chi-squared tests, t-tests, and eigenvalues are the Aegis, Predator, Bradley, and Ma Deuce 50-caliber statistical equivalents used to detect, classify, localize, and neutralize the problem.2016Parametrics
Dialing for Dollars: Improving Cost Estimating Accuracy Through Data MiningBrittany Holmes, James Glenn, Christian SmartThis paper shows the benefits of leveraging obligation and expenditure data from DoD Enterprise Resource Planning (ERP) Systems to supplement cost data from CCDRs or IPMRs. One benefit is that cost trends and funding patterns can be analyzed with more precision because the ERP data is updated real-time vice monthly or annually. Another benefit is that more comprehensive CERs can be created because ERP systems capture all cost vice only contracts with cost reporting CDRLs.2016Parametrics
Develop PRESS for Nonlinear EquationsShu-Ping HuPredicted residual sum of squares (PRESS) and Predicted R2 statistics are commonly used in regression analysis outside the cost analysis industry to determine (1) how well the model predicts new observations and (2) if the model includes more independent variables than necessary. Currently, these two leave-one-out statistics are only available for ordinary least squares. This paper develops these two statistics for nonlinear equations and demonstrates their value in practical applications.2016Parametrics
Including Escalation in Cost Estimating RelationshipsChristopher JarvisPrior to developing a Cost Estimating Relationship (CER) the data should be normalized for effects not explained by the assumed CER model. This typically includes the effects of inflation and/or escalation which has recently generated considerable discussion within the DoD. In this paper we demonstrate two common CERs models and augment them to include an escalation term. Discussion will be made regarding the reliability of the solutions of the augmented models.2016Parametrics
Dangers of ParametricsAndy PrinceWhat if our models are not solving our estimating problems, but instead are the source of our problems? The purpose of this paper is to address this question. We will look at what a cost model is, and what it isn’t. We will examine how cost models appeal to our need for certainty and helps us create a good story for our cost estimate. We will take a look at a simple cost model to see what can go wrong when we trust the model over trusting the data. Finally, we will identify specific actions2016Parametrics
Development of an Additive Manufacturing Cost ModelF. Gurney Thompson, III, Grady NollThis paper discusses our research and development of a cost model for additive manufacturing (AM), also known as 3D printing. Working with academia and multiple AM companies, we investigated cost drivers, collected cost and technical data, derived AM CERs, and developed an approach for including AM components within a larger parametric cost estimate. This paper discusses the research approach, data collection, results, cost model implementation, and future work.2016Parametrics
X-Planes to X-Wings ~ Developing a Parametric Cost ModelSteve SterkIn today’s cost-constrained environment, NASA needs an X-Plane database and parametric cost and schedule model that can quickly provide rough order of magnitude predictions of cost and schedule from initial concept to first flight of potential X-Plane aircraft. This paper takes a look at the steps taken in developing such a model and reports the results. The challenges encountered in the collection of historical data and recommendations for future database management are discussed.2016Parametrics
Integrating Cost Estimating and Data Science Methods in RJosh Wilson, Laura BarkerData science is a growing, interdisciplinary field about the extraction of knowledge from data. It emphasizes the utilization of statistics and advanced analysis techniques to answer questions in a well-documented and reproducible manner. R is a free, open source programming language that is popular among data scientists. This presentation discusses the benefits of using R to supplement traditional cost estimating and data analysis practices.2016Parametrics
AoAs the Right WayAmanda Wilson, David Macdonald, Mariam UzunyanThe follow-ons to two large space acquisition programs from the Space and Missile Systems Center completed the AOA processes last year. They were set apart from past AoAs and AoA-like studies with the leadership and involvement of OSD CAPE and the AFCAA teams, with broad community involvement strengthening the teams. This change resulted in benefits, challenges and lessons learned; various visual tools were created to highlight risks, opportunities, and investment strategies to senior leaders.2016Parametrics
Cost Estimating Relationship Development HandbookJohn Fitch, Adam James, Alfred SmithDeveloping appropriate and well-defined parametric relationships is a foundational requirement for Cost Estimators. Current training for analysts focuses on various ordinary least squares regression (OLS), techniques, but ignores logical decision sequences employed by good analysts applying OLS and omits methods beyond OLS. The CER Handbook includes these techniques in a flow-chart format and guides analysts along each step in developing a parametric relationship.2016Parametrics
Cost Risk for Firm Fixed-Price ContractsDavid Biron, Christian SmartThe terms “cost risk” and “firm fixed-price contracts” seem contradictory. By design the contractor bears all risk under a firm fixed-price (FFP) contract. In spite of this, overruns often occur, and contractors have recourse to a Request for Equitable Adjustment (REA) when cost grows beyond the contract value. We present statistics on cost overruns for FFP contracts, some of which are significant, and show how to model risk for such contracts.2016Risk & Uncertainty
Modeling Prediction Intervals Using Monte Carlo Simulation SoftwareJames Black, Qing Qing "Q" Wu, The use of a prediction interval (PI) is a simple method of quantifying risk and uncertainty for a Cost Estimating Relationship (CER) derived from an Ordinary Least Squares (OLS) regression. Yet, few cost estimators implement PIs in their estimates despite their frequent use of CERs. This presentation will provide a step-by-step tutorial for modeling a PI for an example CER using Monte Carlo Simulation software and will identify the beneficial impact on the coefficient of variation (CV).2016Risk & Uncertainty
Introducing RIFT to Protect Your Uncertain ScheduleNicholas DeTore, Peter FredericThere are industry-accepted methods for allocating cost risk and uncertainty analysis results to detailed WBS elements; schedule results cannot be allocated the same way since duration behaves differently than cost. We present an innovative solution to this issue. The RIFT algorithm calculates a threshold date, for any task or milestone, that if exceeded puts the probabilistic project finish date in jeopardy. RIFT provides a new tangible metric to guide decision makers.2016Risk & Uncertainty
Risk vs. Uncertainty – What’s the difference?Melvin R. Etheridge, Jr.After years of applying “risk”, many in the profession still do not understand the difference between program risk and cost uncertainty in cost estimates. Determining confidence levels goes beyond simply applying the effects of programmatic risks. Simply put, risks have a probability of occurrence, while we know that quantitative inputs have some degree of uncertainty and must be treated as random variables. This paper explores analytical methods in Excel-based cost models using Monte Carlo s2016Risk & Uncertainty
A 'Common Risk Factor' Method to Estimate Correlations between DistributionsMarc GreenbergA ‘common risk factor’ method uses expert-derived risk factors to estimate correlation between two distributions. The premise is to estimate mutual information among risk factors that underlie each distribution. Approximation of such mutual information leverages properties of the joint probability distribution of a unit square. Geometric outputs of each pair of common random variables are compared to estimate common risk factor “intersections” that are, in turn, proxies of correlation.2016Risk & Uncertainty
How Regression Methods Impact Uncertainty ResultsBoyan Jonov, Shu-Ping Hu, Log transformation and weighted least squares are commonly used to develop multiplicative error cost estimating relationships. We objectively compare the two regression techniques and provide a sound defense of log-linear ordinary least squares (LOLS) to counter arguments against its use. We then demonstrate how uncertainty modeling varies substantially based on regression method. Lastly, we establish criteria to use so that LOLS leads to a justifiable uncertainty assignment.2016Risk & Uncertainty
The Zone System of Uncertainty AnalysisJeff McDowellThis paper presents a methodology for selecting cost uncertainty distributions and their dispersion inspired by the photography Zone System. The method seamlessly addresses five groupings of uncertainty across a sliding complexity scale. The distribution shapes and their parameters are drawn from patterns developed from the AFCAA Cost Risk and Uncertainly Metrics Manual (CRUAMM) body of work.2016Risk & Uncertainty
Cloud Total Ownership Costing: Considering the Technologies, Costs and BenefitsDan Galorath, Steven WoodwardCloud abounds with promises of drastically reduced costs, which are all attractive. However, Cloud computing can be ineffective in terms of technology, affordability and total cost of ownership in many situations. The assumption that cloud technology reduces government costs is often flawed. The benefits from cloud computing relating to time-to-market, savings, innovation and agility should be analyzed, providing balanced perspectives for the cloud opportunities presented.2016Software & IT
Data Collection for Agile ProjectsBlaze SmallwoodAgile software development projects produce a unique set of metrics, such as points and velocity, that can provide interesting insight into project progress. However, collecting this data requires specialized mechanisms, since no established standards exist, like SRDRs. This paper will discuss the types of data that are useful to collect for agile software development projects and mechanisms that have been used to collect them for several government projects.2016Software & IT
Apples and Oranges: a Presentation and Analysis of Results of Cloud Cost Calculators and Rate CardsDaniel J. HarperA recent effort for an Army customer examined over a dozen calculators and rate cards for estimating storage and hosting costs for cloud applications. This presentation will provide an overview of several calculators and tools, guidance for cost estimators on interpreting IT-centric inputs, and a discussion of similarities and variation in results. We will also present a cloud complexity plotter which provides a visual tool for explaining cloud cost and complexity drivers.2016Software & IT
A Primer on Agile Software Development for Cost AnalystsEric Hawkes, Cole J. Kupec IIAs Agile software development continues to become more common, cost estimators are compelled to adapt their methods to the Agile approach. This paper begins by explaining what Agile really is and how it differs from the common waterfall methodology. It also presents two case studies that detail how Agile was successfully implemented in a DoD acquisition program. Lastly, the paper proposes four innovative cost estimating methods for an Agile environment.2016Software & IT
The Cost of CyberAnn Hawpe, effrey VothA digital revolution is changing the nature of warfare and the cyber domain is now a top priority for all MDAP and MAIS programs. As the continued advancement of cyber warfare changes the future of land, air, sea, and space over the next decade, the cost community will need to consider the challenges facing defense programs and how to estimate the cyber security requirements within the construct of a reformed acquisition process.2016Software & IT
Maturing the Economic Aspects of Agile Development in the Federal GovernmentJennifer ManringAgile development has emerged in the commercial domain as a leading software development methodology, with growing adoption across federal agencies. This new approach for software development challenges traditional acquisition areas including cost estimation, economic analysis, and system engineering. This research provides a deeper understanding of the economic implications of agile development for the Federal Government and provides insights needed to understand the value proposition of an agile development environment compared with a traditional waterfall environment.2016Software & IT
Using Predictive Analytics for Cost Optimization Across Cloud WorkloadsZachary Jasnoff, David A. CassAs organizations move to the cloud, estimating costs proves difficult, presenting many challenges. Once estimated, cloud costs must be optimized taking into account workloads. Organizations failing to estimate cost based on workloads will either underestimate or over spend on cloud services, often with disastrous results. A framework for optimizing costs (spanning IaaS, PaaS, and SaaS) and determining the cost/benefit between public clouds, private clouds and hybrid clouds will be presented.2016Software & IT
SURF Process Summary & Initial Findings: A Deeper Focus on Software Data QualityNicholas Lanham, Corinne Wallshein, Ranae P. WoodsAs a result of the joint Software Resource Data Report Working Group (SRDRWG), several recommendations were generated by the supporting, multi-agency SRDRWG members to the Cost Leadership Forum. One of which led to the development of the first joint-agency SRDR Validation and Verification (V&V) guide as well as the SRDR User Review Function (SURF) subgroup. This study summarizes the results and data-quality improvement areas identified from 319 V&V comments generated by the SURF team.2016Software & IT
Cloud Solutions - Infrastructure, Platform or Software - Where should you go?Arlene F. Minkiewicz, Ashley HoenigkeCloud computing technologies promise to create some new and interesting challenges for the cost estimating community. How does one determine the effort and costs associated with migration of existing applications to the cloud? What challenges do the various cloud solutions (IaaS vs. PaaS,vs. SaaS) present? This paper presents a case study outlining the migration activities and cost implications for migrating the same application on each of the three cloud solution models.2016Software & IT
Estimating Small Scale Software Integration Efforts with a Large Scale Code BaseJoe BauerEstimating the cost and effort for testing and integration is often one of the most challenging aspects of software estimation. Integration effort depends greatly on the components under evaluation. When relatively small capabilities are introduced into large existing code bases, the integration impact due to the existing code may surprise stakeholders even when requirements are reduced. This presentation highlights software integration modeling for a major defense acquisition program.2016Software Sizing
Alternatives to SLOC in the Modern EnvironmentJeremy GoucherSource Lines of Code (SLOC) based estimating relies heavily on subject matter expert input. However, viable alternatives to SLOC are few. By mapping deliverables to functionality and functionality to effort, the authors propose an alternate technique with reduced reliance on SLOC. Agile development and some recent innovations represent alternatives as well. The authors will present an overview of the topic with examples, required data inputs, and estimating methodology.2016Software Sizing
Agile and Function Points: A Winning CombinationDaniel B. FrenchMany users of Agile have an incorrect belief that projects do not need to adhere to proper project management practices. However, it doesn’t advocate getting rid of metrics or PM processes. The most critical aspect of software development is software size. Function Points can be effectively used with Agile development and this presentation will demonstrate the benefits of using FP in many aspects of project management including estimation, schedule and product backlog.2016Software Sizing
Cost Element Structure for Cloud Migration Business Case AnalysisLorena AguilarIn 2010, the White House released its initial “cloud first” policy. Agencies have since struggled to identify all of the costs associated with cloud migration, and recent audits have criticized agencies for not properly assessing cloud-related costs and benefits. We have developed a cloud migration cost element structure (CES), which simplifies and standardizes cost data collection and estimating. We will present the structure and its applicability to business case and cost benefit analyses.2016Software Sizing
Collaborative Scoring: An Innovative Approach for Sizing Software ProjectsBlaze SmallwoodA major challenge facing government organizations is developing defendable cost and schedule estimates for software development projects, driven by adequately estimating software size. This paper will describe a viable new method for sizing software requirements that utilizes a collaborative scoring methodology similar to those used during agile sprint planning. It has been successfully implemented on several government projects and is a viable option on all software projects.2016Software Sizing
Process-Related Effort and Schedule Estimating Relationships for Software Cost EstimatingNichlas Lanham, Corinne Wallshein, Wilson RosaThe Naval Center for Cost Analysis will present comprehensive, updated findings of software size growth factors, effort estimating relationships (EER), and schedule estimating relationships (SER) with subsets of Department of Defense Computer Software Configuration Item records. This presentation focuses on software size (new, modified, reused, auto-generated, and total code). Subsets include maturity, application and super domain, language, contract type, and operating environment.2016Software Sizing
Health CERs - Using an interdisciplinary approach to estimating the cost of caring for Canada's veteransErin Barkel, Carleigh MalanikReports from the Veterans and the National Defence and Canadian Forces Ombudsmen have brought attention to the prevalence of mental illness among veterans. However, there are no published estimates for the cost of treating this growing population. A key obstacle to this research is that the relevant data is dispersed across several government organizations. We fill this research gap by leveraging actuarial and epidemiological studies of the veteran population to create new cost estimating relationships.2016Government Processes
NASA Commercial Crew Cost Estimating - A Look at Estimating Processes, Challenges and Lessons LearnedLance Cole, Rick BattleTo support annual PPBE budgets and NASA HQ requests for cost information for commercial crew transportation to the International Space Station (ISS), the NASA ISS ACES team developed system development and per flight cost estimates for the potential providers for each annual PPBE submit from 2009-2014. This paper describes the cost estimating processes used, challenges and lessons learned to develop estimates for this key NASA project that diverted from the traditional procurement approach and used a new way of doing business.2016Government Processes
Federal Shared Services: The future Shared-First and challenges facing the Federal Shared Services Providers (FSSPs) and their customersRuth Dorr, Marilyn Fleming, Sarah PopeThis paper will discuss the progress being made in the "Shared-First" approach for sharing of administrative services. Federal Agencies are being asked to execute their missions to ever increasing standards in a resource constrained environment. Administrative shared services have been identified as a way to reduce redundancy and improve efficiency in Federal business solutions. The identification of shared services as a Presidential cross-agency priority along with the designation of several providers for administrative shared services are recent developments in the movement toward shared services within the Federal government.2016Government Processes
Lessons learned from applying cost capability curves on an Air Force AoARobert GeorgiNew direction from Air Force leadership established a requirement for the presentation of life cycle cost versus capability tradeoff analysis for all AoA final reports. This presentation examines the approach that was used for a recent AoA that was a pilot program for the Cost Capability Curve process. When appropriately implemented cost capability analysis can provide significant advantages. There are lessons learned from attempting to implement this approach. We will also discuss examples where this approach does not apply.2016Government Processes
The Navy Modernization Program: Estimating the Cost of Upgrading AEGIS Guided Missile CruisersJeremy GoucherThe Navy is well into its 20 year, $16B (CY10$) plan to modernize 84 AEGIS warships. This cost estimate covers eleven of the planned ships and accounts for maintenance and upgrades to the ship and the combat systems. The estimate leverages recent contract and shipyard performance data and interviews with engineers, resulting in a detailed study and recommended cost savings initiatives. The methods and data in this estimate will assist ship modernization cost efforts across the fleet for the foreseeable future.2016Government Processes
Analysis of Alternatives from a Cost Estimating PerspectiveKirby Hom, Kenny RaglandSubsequent to the Weapon Systems Acquisition Reform Act of 2009, there have been other significant DoD policy changes that require Service sponsors to conduct an Analysis of Alternatives (AoA) prior to Milestone A. "About three-quarters of a program’s total life cycle cost is influenced by decisions made before it is approved to start development" [GAO-09-665]. From a cost estimating perspective, a successful AoA rests with the study approach, overall costing methodologies, key insights from trades analyses, and lessons learned.2016Government Processes
Mining for Cost Estimating Relations from Limited Complex DataMark Jacobs, Shawn HayesNASA’s robotic Earth and space science missions cover a diverse group of projects addressing complex science objectives that include challenging implementation approaches. Progress applying Principle Component Analysis techniques covering project management, systems engineering, mission assurance, integration & test, and spacecraft subsystems is described. Supporting data analysis efforts include a large detailed set of technical and programmatic input candidates that are analyzed to identify the primary spacecraft cost drivers.2016Government Processes
Manufacturing Assembly Plan (MAP) Tool: Bridging the Gap between Performance and the Construction ProcessRichard Lee, Haitham Ghannam, Edward WalshThis presentation explains the four module build plan for the VIRGINIA Class Submarine (VCS) program and provides an in-depth look into labor hours by Section 1/2A, Section 2B/5, Section 6/7, Section 8/9, Final Assembly and Test (FAT), and Post Shakedown Availability (PSA). Further understanding these concepts will not only allow cost estimators and program managers to better understand the current construction build strategy, but it will also aid in better aligning cost estimates with reality at the deck plate.2016Government Processes
It Ain't Easy Being Green... Sustainable Manufacturing with an eye on Cost Avoidance and StewardshipKammy Mann, Jessica BoatwrightThe DoD, one of the largest energy consumers in the world, has committed to sustainability. An important element of sustainable weapon systems acquisition currently not under the purview of the DoD’s strategic plan is sustainable manufacturing. This paper will leverage current research and industry best practices to identify the impact of sustainable manufacturing on cost avoidances relating to human health and environmental benefits. Recommendations will also be provided for complying with the FAR sustainability requirements.2016Government Processes
Business Case Analysis WizardLauren Nolte, Cassie Capots, Marcie Pfeuffer This paper describes the purpose and design of a "Business Case Analysis (BCA) Wizard" that was created for the Air Mobility Command (AMC) Enterprise Learning Office (ELO). As the value of BCAs within the learning transformation at AMC has grown, ELO identified the need to develop a tool that allows "non-cost analysts" to develop high-quality BCAs. This "BCA Wizard" guides users through a standard process of conducting a BCA via a user-friendly interface.2016Government Processes
Masters in Cost Estimating and AnalysisDan Nussbaum, Greg MislickThis presentation provides an update on the all-Distance Learning Masters Degree and/or Certificate Program in Cost Estimating and Analysis (MCEA / CCEA) offered at the Naval Postgraduate School (NPS).2016Government Processes
Methods Used in Pricing and Conveying NASA Assets for Use by Commercial ProgramsJames Roberts, Torrance LambingDifferent agreement types including EUL’s, Use Permits, Land Leases and others are used to convey NASA assets no longer needed by programs such as Space Shuttle and construction of the Space Station. By law, some situations prohibit NASA from pricing facilities in competition with the local market. Costs are estimated either via market-based or actual cost methods, according to circumstances. This paper will explain and clarify how these costs are estimated and pricing is established.2016Government Processes
ATC Zero: Estimating the True Impact of the Chicago Control Center FireKellie ScarbroughFor seventeen days, the Chicago Air Route Traffic Control Center went offline after a contractor set fire to the intricate communications network that controls some of the busiest airspace in the country. Workers scrambled around the clock for seventeen days to restore functionality to the center. This paper seeks to determine the true economic impact of the event, including the costs of time, operations, and environmental impacts.2016Government Processes
Where Does Your Cost Estimate Go?Emily Stafford, Christopher MetzYou have spent several months developing your cost estimate and presenting it to the customer. Now what? This paper will track what happens to your cost estimate after it has been delivered to the customer. It will follow the estimate through the budget formulation process, ultimately becoming a part of the President’s Budget. It will also discuss the cost estimator’s role during the budget process, and the influence of the budget process on future estimates.2016Government Processes
Facilitating Predictive Cost Analytics via Modelling V&VJohn SwarenWe are often asked to “fine tune” estimating software to unique situations. That “calibration” effort is key to Verification & Validation of parametric models. In this presentation, we show an example of using Predictive Analytics to develop a method to predict a Software-estimating model’s inputs as a function of Key Performance Parameters (KPPs). We demonstrate how a calibrated knowledgebase can best be used to V&V an estimating approach reflective of past and future requirement metrics.2016Government Processes
Hyperbolic Discounting and Defense AcquisitionsEric LofgrenCost estimation has largely been approached as an engineering problem which can be solved using data and techniques. While these are undoubtedly invaluable considerations, the broader production process is an inherently open-ended system subject to human interactions which are ill-defined and non-deterministic. This paper explores how behavior and institutions affect cost outcomes through the avenue of hyperbolic discounting, a time-inconsistent model of personal valuations. It will be argued that evidence for hyperbolic discounting in both government agents and contractor management have negative effects on cost estimates and will proffer some solutions based on commitment schemes.2016Government Processes
Cost Estimating Challenges in Additive ManufacturingJoe Bauer, Patrick MaloneAdditive manufacturing possibilities are virtually endless: from rapid prototyping, to printing food, to recreating human tissue. This presentation highlights the challenges of estimating 3D printing processes and offers some solutions to the estimator. Challenges include: amortizing non-recurring tooling across multiple programs, capturing software-centric nature of 3D printing, printing speed, and printing media limitations. This presentation highlights the use of existing cost estimating tools and areas for future development.2016Methods & Models
Long Term Impact of Ship Concepts on Operating and Support Cost AffordabilityEric Buller, Stuart Merrill, Bryan MillerAchieving affordability can be challenging especially once a program enters full rate production. Developing accurate Operating and Support (O&S) cost estimates that reveal tradeoff impacts and sustainment risks provides decision makers with affordability analysis during key acquisition events. We intend to demonstrate through analysis the impact of legacy and new ship concepts on O&S costs and program affordability. We will examine the cost tradeoffs for each ship concept in each O&S cost element.2016Methods & Models
The Peculiar World of Sole Source Contracting; Maybe There is a Better WaySandy Burney, Shawn LarsonIn 2014, 21% of U.S. Government contracts were sole source, accounting for 31% of total contracted value. They typically require negotiation and certification of all cost estimates, which can take up to 18 months. This light-hearted presentation will explore the peculiar world of developing and negotiating sole source estimates, including differences from budgetary and competitive estimating, as well as offer suggestions for achieving fair and reasonable sole source prices for less time and money.2016Methods & Models
Migration of Microsoft Excel Tools to Next Generation Platforms: "Can You Hear the Footsteps"Jeremy EdenAnalysis tools are popular solutions for extended deployments and at the same time an acceleration of platform migrations is occurring to keep up with rapidly changing technology and end user demands. This paper provides specific examples of platform migration challenges, resulting failures, and how they can be best addressed. It will also provide specific best practices for ICEAA member take-away that can be followed to help ensure smoother transitions across platforms.2016Methods & Models
Generalized Degrees of Freedom (GDF)Shu-Ping HuMinimum-Percentage Error/Zero-Percentage Bias (ZMPE) method is commonly used for multiplicative-error models. But ZMPE users do not adjust degrees of freedom (DF) for constraints included in the regression process. This generates misleading ZMPE CER fit statistics and underestimates the CER error distribution variance. Hence, ZMPE results are incomparable with those derived without constraints. This paper details why DF should be adjusted and proposes a Generalized Degrees of Freedom measure to compute fit statistics for constraint-driven CERs.2016Methods & Models
Bottom Up Methods of Estimating Depot Level RepairablesTim Lawless, James Black, Dave GoldbergDepot Level Repairables (DLR) costs comprise a significant share of total Operations and Support costs as they capture the scope of repairing and/or replacing failed hardware items. This paper will provide detailed descriptions of bottom up methods to estimate DLR costs using programmatic technical baseline documentation and acquisition costs as well as supply system requisitions and repair price data sources. Emphasis will be placed on producing data-driven analysis to inform cost estimates and estimate customers.2016Methods & Models
Seven Degrees of Separation: The Importance of High-Quality Contractor Data in Cost EstimatingCrickett PettyThe popular notion that any two people are linked to one another on average via a chain with “six degrees of separation” is based on a relatively small sample. Using massive data sets, researchers have since discovered that the average degrees of separation is closer to seven. This highlights the need for complete and thorough data in any analysis, and cost estimating in particular. Despite its importance the collection of high-quality contractor cost data is challenging. Processes for meeting these challenges are presented.2016Methods & Models
Biometric Analytics Cost EstimatingJoseph SarageThe purpose of this paper is to present cost estimating methods as it pertains to the Biometric Field by which statistical analysis techniques can be applied to better predict the costs and relationships with Biometric data. The paper will explore analytical methods in which Biometric data can be used for trend analysis, forecasting, and strategic decision making within various organizations and applications.2016Methods & Models
Interviewing Subject Matter ExpertsMelissa TeicherAs estimators, we often have to obtain technical information from Subject Matter Experts (SMEs) in order to provide thorough and accurate analysis. But sometimes, getting the information you need is easier said than done. This paper discusses the steps for interviewing SMEs in order to make the most of their time and your time. It explores the benefits of building a cooperative relationship of mutual respect and the “dos” and “don’ts” of SME consultations.2016Methods & Models
The Impacts of Design Change on Reliability, Maintainability, and Life Cycle CostAndreas Viberg, Oskar Tengo When making decisions about a design change to a technical system, it is essential to understand how it affects reliability and maintainability, as that directly impacts both overall performance and cost of operations and maintenance throughout the life cycle. LCC analysis based on modeling and simulation of relevant scenarios is an effective way to gain that understanding. This paper presents an approach for such analysis and illustrates its successful application in a recent project.2016Methods & Models
A Prime Contractor's View Using JCL Tools and TechniquesChip WoodsFor several years now, Lockheed Martin has been integrating cost and schedule data into a single model to perform integrated analysis. This paper will discuss Lockheed Martin’s JCL process, the tools used to develop JCL curves, and how this analysis can be used to shape and steer the program – including modeling funding constraints.2016Methods & Models
Class List of Characteristics: Changing the Landscape of Estimating by Removing BiasPaul BreonCLOC (Class List of Characteristics) is a coding system wherein each digit of the code indicates one of the cost drivers for the product/activity. CLOC gives the ability to “Call a thing by what it really is” and creating a universal language that can be utilized for any purpose. CLOC does NOT include Bias like quantity, cost, etc. These things are used in conjunction with the code to do any kind of analysis needed.2016Methods & Models
Achieving Affordability through high fidelity ROM cost estimatesEric BullerAchieving affordability can be challenging especially if early decisions are made without proper cost estimates. Developing high fidelity ROM cost estimates and identifying capability tradeoffs during the requirements generation phase can be critical to the success of achieving long term affordability. We intend to demonstrate through analysis that preforming high fidelity ROM cost estimates with capability tradeoffs allow decisions makers to informed choices on mission requirements that achieve affordability for future programs.2016Methods & Models
A Regression Method for Sparsely-Populated DatasetsDaniel BarkmeyerA method is presented for including sparsely-populated independent variables in a regression analysis without introducing subjectivity or statistical bias. The method is shown to reproduce the result of a traditional regression method for a fully populated dataset, and to converges to the same result on a sparsely-populated dataset as that dataset decreases in sparseness. Application of the method to develop a new, empirically-derived model for spacecraft testing schedule duration will also be presented.2016Parametrics
Improvement Curves: An Early Production MethodologyBrent JohnstoneLearning slope selection is a critical parameter in manufacturing labor estimates. Incorrect ex ante predictions lead to over- or understatements of projected hours. Existing literature provides little guidance on ex ante selection, particularly when some actual cost data exists but the program is well short of maturity. A methodology is offered using engineered labor standards and legacy performance to establish a basic learning curve slope to which early performance asymptotically recovers over time.2016Parametrics
The NRO CER Analysis ToolDonald MacKenzieThe NRO Cost and Acquisition Assessment Group (CAAG) uses its CER Analysis Tool, CERAT, to inform CER development. CERAT analyzes CERs for sensitivity to influential data points (IDPs). The impacts on estimated costs for a target data point when data points are removed are the primary outputs. The changes in recorded CER constants when data points are removed also help in assessing overall CER "stability." CERAT also provides several other aids to CER developers.2016Parametrics
A Parameter Driven WorldJeffrey PlattenParametric estimating is gaining acceptance. DFARS requires DOD contractors to maintain an estimating system. Parameters are the characteristics, features, or measurable factors that define a product or program. What makes a good parameter? What are the pros and cons of parametric cost estimating? How do you develop and maintain a parametric estimating database/model? What does the future hold, beyond just estimating? How do you gain parametric control over touch labor, support labor, and parts costs to improve affordability?2016Parametrics
Generating a Semi-Automated "Similar To" Basis Of Estimate from a Complex Parametric Hardware Cost Model for AntennasDanny Polidi, David BloomThis paper discusses the development of a "Similar-To" Basis Of Estimate (BOE) generation tool used in conjunction with a Complex Parametric Antenna Cost Model. Starting with the parametric model based on quantifiable sizing mechanisms which are designed to quickly and accurately calculate the "top-down" cost for all engineering and operations disciplines and functions required for antenna development and test, cost is generated, "similar to" programs are identified, and a BOE can be auto-populated.2016Parametrics
Macro-parametrics and the applications of multi-colinearity and Bayesian to enhance early cost modellingDale ShermonThis paper will consider the spectrum of parametric cost models from cost estimating relationships (CER) to micro and finally macro-parametric models. This will lead to a description of the Family of Advanced Cost Estimating Tools (FACET) parametric suite of models and their top-down capability to estimate costs incorporating the application of multi-colinearity and bayesian techniques.2016Parametrics
Using Historical Cost Reports to Derive System Specific FactorsJocelyn Tague, William BanksNon-Recurring Engineering (NRE) is often a cost driver for new, complex systems and must be given high attention during Milestone B cost estimating. Our team developed an approach for developing system specific Cost Estimating Relationship (CER) using actual costs for from combat weapon systems with shared capabilities, resulting in a credible defensible, yet flexible approach that can be replicated for a wide range of estimates.2016Parametrics
A CER Development Process for Spares EstimatingCheryl WilsonThis presentation shows the direction of a group of Spare Parts Estimators and how we are advancing our statistical skills and CER development processes. It shares the progress in our CER development journey attained through education, research, analysis and teamwork.2016Parametrics
Improving Parametric Software Cost Estimation Through Better Data CollectionBob KouryIn 2010 PRICE Systems LLC in conjunction with PEO STRI agreed to work together to improve the data being collected to support software development cost estimation in the command. The paper will compare and contrast the experiences of two other collection efforts with those of original effort. The paper will identify core findings which may be used universally to improve the quality, quantity, and applicability of data being collected.2016Parametrics
Building an Integrated Aerospace Electronics Development Parametric Model from the Ground UpDavid BloomThis paper presents the results of a multiple year effort to create a business specific set of integrated parametric cost models for complex electronics hardware development in such a way that all disciplines and activities that participate in that development process have a piece of the model and a stake in the data collection that drives the model. This "grounds up" approach provides that added benefit of cost allocation guidance once the development is approved.2016Parametrics
Essential Views of the Integrated Program Management Reports (IPMRs)Thomas Coonce, Glen AllemanDuring this session, the authors will describe key views of the IPMRs and other data that will allow a government stakeholder to a) understand the cost, schedule and technical status on applicable contracts, b) investigate the nexus of performance problems, and c) observe likely future problems. The presentation represents a “synthesis” of existing traditional earned value management metrics and other metrics designed to provide an integrative picture to assist in keeping the program “green”.2016Program Management
Meeting Fiscal Constraints: the Evolving Role of Performance MeasurementSteve Green, Kevin Davis, Kurt Heppard Performance measurement tools can reflect the strategic intent of management and help them prioritize initiatives to meet strategic objectives. In fiscally austere times, the Balanced Scorecard can help organizations address these tough decisions. Applying concepts from the Balanced Scorecard to cost analysis and budgeting, this paper develops an integrated aligned system of strategic goals, performance metrics, and cost parameters at a university facing pressure to manage costs. There may be lessons learned for the DoD.2016Program Management
The graph is always greener on the other side: examples of how to interpret and fix bad visual representations of dataStephen KetchamIn my role performing independent reviews of estimates, I’ve seen a quality spectrum of different types of graphs, charts, and forecasts. These visuals summarize key points and remain in decision-makers minds, often providing the convincing point to win funding approval for the project. Unfortunately they can also be poorly crafted, employ scare tactics or even frame the data in a misleading way. This presentation provides bad examples of visuals, and discusses possible improvements.2016Program Management
Seeing What We Need to See vs. What We Hope to See: Visualizing Integrated Cost and Schedule Data for Earned Value AnalysisBrian LeachWhen applied properly, data visualization can serve analysis by showing us what is otherwise difficult to see (and not always what we hope to see). With the emergence of the UN/CEFACT XML standard, integrated cost and schedule data can be shown in exciting new ways. This paper will examine and explore innovative new ways to show integrated cost and schedule data which have practical applications in earned value management system analysis.2016Program Management
Different Incentives in Government ContractingJennifer LeottaThis paper shows stock prices are used as the primary incentive for executives of publicly traded government contractors rather than profit sharing contracts. Looking at the reaction of companies’ stock prices to public and private sector news, this paper shows stock prices do not capture “bad news” for both the public and private sector customers compared to overall market indicators; and would make poor contract incentives. Finally, alternate methods of contract incentives will be explored.2016Program Management
Agile Project Management Controls: Issues with EVM and a Viable AlternativeOmar Mahmoud, Blaze SmallwoodAs more government software programs adopt agile practices, program managers are trying to find effective ways to monitor and control cost and schedule. The challenge is that traditional project control techniques, like EVM, are not well suited to deal with the requirements fluidity and new terms/metrics that come with agile projects. This paper will describe the inherent issues with using EVM on agile projects and offer a recommendation for a more effective solution.2016Program Management
The Psychology of Cost EstimatingAndy PrinceResearch into human psychology has yielded amazing findings into how we process information and how we use information to make decisions. Cost estimators can use psychology and behavioral economics to improve not only our cost estimates but how we communicate and work with our customers. By understanding how our customer’s think, and more importantly, why they think the way they do, we can have more productive relationships and greater influence. The end result will be better decisions by the decision makers.2016Program Management
Cost ManagementGeorges TeologlouCost Management has a key role in project management and is the management trend due to the actual financial and economic worldwide constraints. Cost Management solution was built to be in line with the recommendations of the GAO (US Government Accountability Office) that stated that a large number of Defense projects are at risk of experiencing cost overruns and schedule delays. GAO found that the two major causes of project failure were first an inaccurate initial estimate and secondly poor subsequent cost control.2016Program Management
Using Total Lifecycle Cost as the Basis for Proposal Cost EvaluationRyan TimmThis paper discusses benefits and obstacles of using the TLCC of an entire system instead of the bid price for the purpose of proposal cost evaluation. An example acquisition of a space vehicle (SV) ground system explains the impact to TLCC of portions of the ground system developed by the SV manufacturer, SV operations, external software interfaces, government furnished IT infrastructure, and legacy sustainment until transition. Using TLCC allows higher-level requirements and enables innovative architectures.2016Program Management
A Maturity Analysis of the American and European Cost Community Mark GilmourThe Cost Engineering Health Check (CEHC) is a cost capability benchmarking tool used to identify strengths and weaknesses within organizations costing functions. In 2014 the authors of this paper conducted a CEHC at the ICEAA conference in Denver. That workshop used e-voting to anonymously gather the views of the audience in attendance. This paper presents a summary of these results and contrasts them against data gathered from European workshops, providing a snapshot of the international costing capability.2016Program Management
Presenting EstimatesMelissa TeicherDecision-makers rely on cost estimators to guide them in making the most educated judgments on what is best for the program or organization. A thorough and well-organized presentation can make the difference between an easy decision and a difficult one. This paper discusses how public speaking skills contribute to an effective estimate presentation. It also delivers tips and tricks on how to make an estimate presentation more visually pleasing and easier to understand.2016Program Management
Using Stochastic Optimization to Improve Risk PrioritizationEric DrukerMinimizing the cost of complex programs is critical for government agencies trying to meet their missions in today’s fiscal. Today, identifying cost-savings measures is a manual procedure where an analyst must make an educated guess as to what risks to mitigate. This process is manual and slow. This paper will outline a methodology for using the emerging field of stochastic optimization to automate the identification of cost-savings measures on complex programs.2016Risk
Why Are Estimates Always Wrong: Estimation Bias and Strategic MisestimationDan GalorathThe biggest source of estimation error usually comes from people, either by accident or strategically. This is a disaster because viable estimates are core to project success as well as ROI determination and other decision making. Most people don’t know how to estimate. Those who estimate are often optimistic, full of unintentional bias and sometimes strategically mis-estimating. This paper discusses the issues of estimation bias and strategic mis-estimation as well as how to mitigate these issues.2016Risk
A "Common Risk Factor" Method to Estimate Correlations between DistributionsMarc GreenbergA “common risk factor” method uses expert-derived risk factors to estimate correlation between two distributions. The premise is to estimate mutual information among risk factors that underlie each distribution. Approximation of such mutual information leverages properties of the joint probability distribution of a unit square. Geometric outputs of each pair of common random variables are compared leading to an estimated ‘intersection’ of pair of common risk factors. This intersection is then assumed to be a proxy of correlation.2016Risk
Data-Driven Guidelines for Correlation of Cost and Schedule GrowthSidi Huang, Mark Pedigo, Chris Shaw When running Monte Carlo simulations, one of the most difficult aspects to account for is correlation between the independent variables. Ignoring correlation generally reports overly optimistic analysis results; industry standard suggests that when no additional insight is available, a default value of 0.2 can be used. This study analyzes historical NASA program data to develop data-driven guidelines for correlation. Moving forward, this study can be replicated to other industries and organizations with critical schedules.2016Risk
Risk Mitigation: Some Good News after the Cost / Schedule Risk Analysis ResultsDavid HulettThe results of your cost-schedule risk analysis say you are likely to be overrun on both schedule and budget. Are you are out-of-luck? No. Do not forget the risk mitigation phase. Risk mitigation typically proceeds from a prioritized risk list developed from analysis of the risk and uncertainty-loaded schedule. Complete mitigation of risks is not usually feasible, and uncertainty is usually deemed to be irreducible. Some good news comes from the risk mitigation actions.2016Risk
A Mathematical Approach for Cost and Schedule Risk AttributionFred KuoThe level of sophistication in cost estimate has progressed substantially in the last decade. This is especially true at NASA where, spearheaded by the Cost Analysis Division (CAD) of NASA Headquarters, cost and schedule risks in terms of confidence level have been codified. In this paper, the author attempts to employ the portfolio approach in calculating risk attribution; first applying to a portfolio of cost risks and extend the same concept to schedule risks analysis.2016Risk
Triangular Distributions and CorrelationsJennifer LampeTriangular distributions are often used in estimating cost risk because the math is relatively simple. The concept of an estimate having a lower limit, an upper limit, and a mode is easy to understand. We show the math behind triangular distributions and correlations. What does it mean to say that two risks are 30% correlated? How do risk analysis programs handle triangular distributions and correlations? How can you use random numbers in Excel to generate a Monte Carlo simulation for any defined triangular distribution?2016Risk
The Impact of Selected Assumptions and Core Tenets on Schedule Risk Assessment Results (A Progressive Model Comparison)James QuilliamIn the quest to ensure the sound representation of Schedule risk assessment (SRA) simulations this case study will provide a progressive model comparison of schedule risk assessment assumptions and core tenets. The elements of this approach will focus on the: methodology and tools; the progressive assumptions and core tenets applied; conclusions and lessons learned for practitioners. This will greatly enhance the understanding and confidence that leadership and project teams have in the schedule risk assessment results.2016Risk
Wine as a Financial InvestementMichael Shortell, Adiam WoldetensaeThe purpose of this presentation is to look into wine as an investment and analyze its characteristics as a financial asset. The goal here is to look into vintage wines through time, analyze its profitability and predict how investing in wine could be a good source of diversification. Our objective is to detect the types of wines that are a good source of investment and their associated risk. We aim to identify a good investment portfolio for a wine collector in order to maximize returns and minimize the associated risk.2016Risk
Beyond Correlation: Don't Use the Formula That Killed Wall StreetChristian SmartRisk models in cost estimating almost exclusively rely on correlation as a measure of dependency. Correlation is only one measure (among many) of stochastic dependency. It ignores the effect of tail dependency, when one extreme event affects others. This leads to potential outcomes that do not make sense such as a program with a large schedule overrun but no cost overrun. This method has been widely blamed for the financial crisis a few years ago. The copula method is introduced as a way to overcome these deficiencies.2016Risk
Connecting the Dots: Integrating the risk cube with the POEChristina SnyderDoD guidance instructs program managers to include risk and uncertainty analysis in their cost estimates. Program managers also maintain a risk cube that tracks known program risks and assigns a likelihood and consequence. Risks, regardless of if they are programmatic, technical, or schedule defined, will have an effect on the overall program cost. This paper investigates the linkage between identified risks and POE cost and shows how to translate the risk cube into dollars.2016Risk
Methodology for Integrating Risk mitigation Activities into JCL analysisJames Taylor, Justin HornbackThe intent of this whitepaper is to outline a an improved methodology for capturing risk mitigation activities for both cost and schedule into a Joint Confidence Level (JCL) analysis and assessing mitigation effectiveness.2016Risk
The Continual Pursuit of the One True Software Resource Data Reporting (SRDR) DatabaseRemmie Arnold, Peter BraxtonThis paper will demonstrate the ability for one database to store Software Resource Data Report (SRDR) source data in its original form, as submitted by WBS element and reporting event. This database allows evaluations, interpretations, and annotations of the data, including appropriate pairing of Initial and Final reports; mapping of SLOC to standard categories for the purposes of determining ESLOC; normalization of software activities to a standard set of activities; and storage of previous assessments.2016Software Estimating
Agile Estimation Using Functional MetricsThomas CagleyAgile methods have matured and are now being integrated into many different approaches to the development of software. Estimation has been problematic for all methods, agile to plan based and therefore it tends to be a lightning rod for experimentation and synthesis such as is being described in this paper. Agile Estimation Using Functional Metrics has presented a path for integrating the discipline found in functional metrics with the collaborative approaches found in agile parametric estimation.2016Software Estimating
Put some SNAP in your estimating modelDavid LambertAn effective estimating model is dependent upon having the capacity to accurately size the problem domain for the solution being developed. Function Point Analysis is a widely accepted industry and ISO standard for sizing functional requirements; however, it does not take into account the sizing of non-functional requirements. Learn how the newly released Software Non-Functional Assessment Process (SNAP) can provide a more complete and accurate assessment of project size.2016Software Estimating
Forecasting Software MaintenanceAlea DeSantisProviding a defensible estimate for software maintenance can be complicated. By overlaying historical data with lifecycle events one can derive event driven factors and use trend analysis to forecast future Software Problem Reports. Topics addressed are: common-cause variation, special-cause variation, severity level, complexity level, and decay rate. This method moves away from using a general defect rate calculation to a more complete product-specific calculation.2016Software Estimating
Effective use of Function Points for Analogous Software EstimationDaniel FrenchAnalogous estimation is often used to estimate software projects, especially in the early phases, because it is believed to be quicker and easier than other techniques. However, many organizations do not employ this technique effectively and correctly. One pitfall is improperly applying the technique & basing these estimates on Software Lines of Code (SLOC). This presentation will discuss common mistakes made when using analogous estimates as well as how the use of IFPUG function points can greatly improve this estimation technique2016Software Estimating
NASA Software Cost Estimation Model: An Analogy Based Estimation MethodJairus HihnSoftware development activities are notorious for their cost growth. While there has been extensive work on improving parametric methods there is very little focus on the use of models based on analogy and clustering algorithms. In this paper we will describe the methodology being used in the development of a NASA Software Cost Model using data mining clustering algorithms and evaluate its performance by comparing it to estimates from COCOMO II, a calibrated COCOMO II, liner regression, and K-nearest neighbor models.2016Software Estimating
Estimating Agile Software DevelopmentBob HuntThe use of Agile Software Development continues to grow in large and small Federal programs. These programs often utilize Story Points as a key size metric. This presentation will roovideman update on Agile Cost Estimating techniques and emphasize how to,utilize Story Points as a key size metric.2016Software Estimating
How Much Does Software Maintenance Cost?Cheryl Jones, James Judy, John McGarryTo accurately estimate, budget, allocate, and justify the resources required to meet evolving mission and service affordability requirements across the system life-cycle we must be able to answer the fundamental question: How much does software maintenance cost? This presentation provides a summary of the findings from an ongoing U.S. Army study that is focused on developing improved software maintenance estimation processes, cost relationships, and associated models.2016Software Estimating
Exploring DoD Software Growth: A Better Way to Model Future Software UncertaintyNicholas LanhamThis presentation highlights trends and estimating relationships derived from the April 2014 SRDR data. This analysis provides the estimating community with several new variables to better model future software growth uncertainty. In addition, the analysis includes growth by contract type, domain, and program type as well as introduces a multivariate model to predict final hours. All of the relationships included within this slide set have been derived from the best 400+ data points, were approved by DCARC, and are DoD-specific.2016Software Estimating
Considerations in Data Center Migration Life Cycle Cost Estimating: A Joint Information Environment (JIE) PerspectiveDennis McKeonThe potential benefits of consolidating data centers has led to the creation of initiatives such as the DoD Joint Information Environment (JIE). The goal is to reduce IT infrastructure costs by consolidating system support functions, improve hardware utilization, and lower energy and facility costs. This paper is from the perspective of an organization migrating to a consolidated data center. It will describe the migration steps, cost estimating approach, benefits, risks, and organizational challenges from their perspective.2016Software Estimating
At the Intersection of Technical Debt and Software Maintenance CostsArlene MinkiewiczWard Cunningham introduced the technical debt metaphor in 1992 to ease business leaders understanding of the cost of making poor decisions during software development. While this author has seen many different definitions of technical debt, it is basically an indication of the amount of ‘should fix’ issues that remain in production code. This paper reports on an on-going research effort to understand technical debt and its implications on software maintenance cost and effort estimation.2016Software Estimating
Early Phase Software Cost and Schedule Estimation ModelsWilson RosaSoftware cost estimates are more useful at early elaboration phase, when source lines of code and Function Points are not yet available. This study introduces effort estimation models using functional requirements as a size predictor and evaluates development schedule using estimated effort based on data from 40 military and IT programs delivered from 2006 to 2014. Statistical results show that estimated functional requirements and peak staff are significant contributors and that estimated or actual effort is a valid predictor of development duration.2016Software Estimating
Software size measures and their use in software project estimationHarold van HeeringenUsing the right size measure is extremely important for software cost engineering purposes. However, the number of software size measures is growing rapidly and there is a lot of confusion in the industry with regard to the methods that should be used. The advantages and disadvantages of the major software size measures are discussed and recommendations are given on what size measures should be used to estimate the different types of software systems.2016Software Estimating
Software Size GrowthCorinne WallsheinNCCA will present their findings including cost estimating relationships (CERs) of software size growth using selected subsets. CERs will be compared using initially estimated parameter distributions against CERs using software size. Other parameters include requirements, staff size, hours, and duration. As the 2014 update of the Joint Agency Cost and Schedule Risk and Uncertainty Handbook advises use of standard error for uncertainty bounds, NCCA will compare bounds for recommended CERs, by selected data subsets.2016Software Estimating
NPS and AFIT Masters Degree in Cost Estimating and AnalysisDaniel Nussbaum, Greg MislickThis presentation provides an update on the Joint, all-Distance Learning Masters Degree in Cost Estimating and Analysis offered at the Naval Postgraduate School (NPS) and Air Force Institute of Technology (AFIT).2014Business and Art of Cost Estimating
A Comprehensive CES and BCA Approach for Lifelong LearningKevin Cincotta, Darcy LilleyThe Air Force Air Mobility Command (AMC) Enterprise Learning Office (ELO) mission is to transform AMC into a premier Air Force learning organization, achieve learning through optimum approaches and develop Mobility Airmen into life-long learners who demonstrate institutional Air Force competencies with a positive approach to managing their own learning. In this context, learning has three main components: training, education, and experience. ...2014Business and Art of Cost Estimating
BOE Development: Scope Evaluation and CriteriaMichael Butterworth, Demetrius PradoCurrent basis-of-estimates (BOEs) do not identify succinctly, how the scope of the estimate was derived and how actuals used for build-up, analogous and parametric methodologies fit the technology and product under consideration. We believe that one of the many problems leading to poor BOE development revolves around the lack of identifying the scope of work and understanding the effort needed in particular skill mix to perform the task. ...2014Business and Art of Cost Estimating
Long Term Affordability through Knowledge Based Bid & ProposalZachary JasnoffAccording to a 2013 GAO report, "positive acquisition outcomes require the use of a knowledge-based approach to product development that demonstrates high levels of knowledge before significant commitments are made. In essence, knowledge supplants risk over time." Often times, acquisition proposals that are not knowledge-based introduce significant risk and while seeming reasonable in the short term, cannot sustain long term affordability. ...2014Business and Art of Cost Estimating
What Happens to a Cost Estimate When It's Done?William Barfield, David BachWhat happens to a cost estimate when it is done, or "finished?" Cost estimates are used to support a wide variety of financial decisions. When estimates are done and the decision has been made, is the estimate still useful after the decision, or does it become "shelf-ware?" We surveyed the international cost community to determine how we develop, document, use, and archive our various kinds of cost estimates. ...2014Business and Art of Cost Estimating
Update On The Cost FACTS (Factors, Analogies, CER’s & Tools/Studies) Group – Enhancing KM By Leveraging Enterprise Social NetworkingDaniel Harper, Ruth DorrCost FACTS is a community-driven initiative to bring together the cost estimating community across MITRE, Government agencies, the contractor community, and academia to share reusable, non-restricted cost information and to encourage dialogue throughout the worldwide cost community. Cost FACTS provides an easily accessible repository of reusable cost estimating FACTS, i.e., Factors, Analogies, Cost Estimating Relationships (CERs), and Tools/Studies. ...2014Business and Art of Cost Estimating
Space Shuttle Cost Analysis: A Success Story?Humboldt MandellIn the aftermath of the highly successful Apollo lunar program, NASA struggled for a few years to find a meaningful program which would satisfy long range national space strategies, as well as reflecting the realities of the rapidly changing political environment in the nation. The Space Shuttle emerged from the need to lower the costs of orbital cargo delivery for construction of a space station and enabling Mars exploration, but also was highly constrained by DOD requirements ...2014Business and Art of Cost Estimating
Big Data Meets Earned Value ManagementGlen Alleman, Thomas CoonceWhen the result of an action is of consequence, but cannot be known in advance with precision, forecasting may reduce decision risk by supplying additional information about the possible outcomes. Data obtained from observations collected sequentially over time are common. Earned Value Management is an example where project performance data (BCWP) is collected from the status reports of planned work (BCWS) and a forecast of future performance is needed to manage the program. ...2014Earned Value Management
Don't Let EVM Data Mislead YouSteve SheamerEVM data is a popular data source for cost estimators and for good reason; in theory, it should provide most of the data needed to develop an estimate for a program. For completed programs, it provides historical costs by WBS and for programs that are in work it provides a measure of the work completed, work remaining, and forecast of the work remaining. But during a period of frequent cost overruns, estimates built using EVM data often fail to forecast the extent of program overruns...2014Earned Value Management
Trust but Verify - An Improved Estimating Technique Using the Integrated Master Schedule (IMS)Eric LofgrenIt has long been the wonder of management why the Integrated Master Schedule (IMS) fails to give advanced warning of impending schedule delays. The oft-touted Government Accountability Office (GAO) 14-Point Check for Schedule Quality analyzes schedule health using key metrics, leading one to assume that such a test authenticates schedule realism. Why, then, do practitioners find themselves caught off guard to slips when their IMS appears in good health? ...2014Earned Value Management
A Cure For Unanticipated Cost and Schedule GrowthGlen Alleman, Thomas CoonceFederal programs (DoD and civilian) often fail to deliver all that was promised and many times cost more than estimated and are often late. Delivering programs with less capability than promised while exceeding the cost and planned undermines the Federal government’s credibility with taxpayers and contributes to the public’s negative support for these programs. ...2014Earned Value Management
Unleashing the Power of MS Excel as an EVM Analysis ToolAllen Gaudelli, Steve SheamerWhat do you do if you need to analyze or report on EVM data and you don't have access to (or can't afford) the latest industry software? Nearly everyone has a very powerful analysis and reporting tool on their desktop with the flexibility and capability to consolidate cost, schedule, and risk drivers into a single view. In this presentation, we will show you how to leverage and manipulate the inherent capabilities of Microsoft Excel to build interactive EVM dashboards ...2014Earned Value Management
Design to Cost: Misunderstood and misappliedErin Barkel, Tolga YalkinThe Canadian Department of Defence maintains that concerns over cost overruns are overstated because it adopts a design to cost approach. According to the US Government, design to cost "embodies the early establishment of realistic but rigorous cost targets and a determined effort to achieve them." From the beginning of a project to its completion, "[c]ost is addressed on a continuing basis as part of a system's development and production process." ...2014Earned Value Management
Testing Benford's Law with Software Code CountsChuck Knight, Chris KaldesWhen analyzing a data set, common thinking may lead one to suspect that the leading digit of each data point would follow a uniform distribution, where each digit (1 through 9) has an equal probability of occurrence. Benford's law, to the contrary, states that the first digit of each data point will conform to a nonuniform distribution. More specifically, it states that a data point is more likely to begin with a one than a two, a two more likely than a three, a three more likely than a four, and so on. ...2014Information Technology
Improved Method for Predicting Software Effort and ScheduleWilson Rosa, Barry Boehm, Ray Madachy, Brad Clark, Joseph P. DeanThis paper presents a set of effort and schedule estimating relationships for predicting software development using empirical data from 317 very recent US DoD programs. The first set predicts effort as a function of size and application type. The second predicts duration using size and staff level. The models are simpler and more viable to use for early estimates than traditional parametric cost models. Practical benchmarks are also provided to guide analysts in normalizing data.2014Information Technology
Costs of Migration and Operation in the CloudArlene MinkiewiczAt one level cloud computing is just Internet enabled time sharing. Instead of organizations investing in all the Information Technology (IT) assets such as hardware, software and infrastructure they need to meet business needs; cloud computing technology makes these resources available through the Internet. Cloud computing allows an organization to adopt a different economic model for meeting IT needs by reducing capital investments and increasing operational investments. ...2014Information Technology
How I Continued to Stop Worrying and Love Software Resource Data ReportsNicholas LanhamThis presentation highlights the trends and cost estimating relationships derived from detailed analysis of the August 2013 Office of the Secretary of Defense (OSD) Software Resource Data Report (SRDR) data. This analysis was conducted by Nicholas Lanham and Mike Popp and provides as follow-on analysis to the August 2012 SRDR brief developed and previously presented by Mike Popp, AIR 4.2. ... 2014Information Technology
Mobile Applications, Functional Analysis and Cost EstimationTammy PreussThis presentation will demonstrate how to derive cost estimates at different stages in a project's lifecycle by using function points and the advantages of using an FP based size estimate over a SLOC based estimate. The intended audience is software cost estimators, project managers, and anyone who is interested in software measurement.2014Information Technology
In Pursuit of the One True Software Resource Data Reporting (SRDR) DatabaseZachary McGregor-DorseyFor many years, Software Resource Data Reports, collected by the Defense Cost and Resource Center (DCARC) on Major Defense Acquisition Programs (MDAPs), have been widely acknowledged as an important source of software sizing, effort, cost, and schedule data to support estimating. However, using SRDRs presents a number of data collection, normalization, and analysis challenges, which would in large part be obviated by a single robust relational database. ... 2014Information Technology
Optimizing Total Cost of Ownership for Best Value IT Solutions: A Case Study using Parametric Models for Estimates of Alternative IT Architectures and Operational ApproachesDenton Tarbet, Kevin Woodward, Reggie ColeBecause of a variety of architectures and deployment models, Information Technology (IT) has become more and more complex for organizations to manage and support. Current technology IT system architectures range from server based local systems to implementations of a Private Cloud to utilization of the Public Cloud. Determining a "best value architecture" for IT systems requires the ability to effectively understand not only the cost, but the relative performance, schedule and risk associated with alternative solutions. ...2014Information Technology
Estimating Hardware Storage CostsJenny Woolley, William BlackEstimating Commercial-off-the-Shelf (COTS) hardware storage volume and cost requirements can be challenging. Factors such as storage type, speed, configuration, and changing costs can potentially lead to estimating difficulties. This is especially true when a Redundant Array of Independent Disks (RAID) configuration is implemented. Due to the multiple attributes that can vary within each RAID level, as well as other factors that may influence the total storage volume needed, developing relationships for estimating long-term storage costs can become complicated. ...2014Information Technology
Relating Cost to Performance: The Performance-Based Cost ModelMichael Jeffers, Robert Nehring, Jean-Ali Tavassoli, Kelly Meyers, Robert JonesFor decades, in order to produce a cost estimate, estimators have been heavily reliant on the technical characteristics of a system, such as weight for hardware elements or source lines of code (SLOC) for software elements, as specified by designers and engineers. Quite often, a question will arise about the cost of adding additional performance requirements to a system design (or in a design-to-cost scenario, the savings to be achieved by removing requirements). Traditionally, the engineers will then have to undertake a design cycle to determine how the shift in requirements will change the system. ...2014Information Technology
Lessons Learned from the International Software Benchmark Standards Group (ISBSG)Arlene MinkiewiczThis paper will introduce the ISBSG and the database that are available from the ISBSG. It then provides details of the data driven approach applied to develop these templates - discussing research approach, methodology, tools used, findings and outcomes. This is followed by a discussion of lessons learned including the strengths and weaknesses of the database and the strength and weaknesses of the solutions derived from it. While particularly relevant to software estimators, this paper should be valuable to any estimator who lacks data or has data they are not quite sure what they might do with it.2014Information Technology
Software Maintenance: Recommendations for Estimating and Data CollectionShelley Dickson, Bruce Parker, Alex Thiel, Corinne WallsheinThe software maintenance study reported at ICEAA in 2012 and 2013 continued to progress in 2013 in spite of the high data variability. This presentation summarizes the past years' software maintenance data collection structure, categorizations, normalizations, and analyses. Software maintenance size, defect, cost, and effort data were collected from Fiscal Years (FY) 1992 - 2012. Parametric analyses were performed in depth on available variables included in or derived from this U.S. Department of Defense software maintenance data set. ...2014Information Technology
An Update to the Use of Function Points in Earned Value Management for Software DevelopmentMichael Thompson, Daniel FrenchThis presentation describes the opportunity that was presented to the team and how the recently completed pilot program was developed and implemented to address it. The authors will address how effective the pilot program was as far as identifying and resolving issues, measuring earned value, as well as the challenges and lessons learned with the development, implementation, and sustainment of the FP based EVM process.2014Information Technology
The Federal IT Dashboard: Potential Application for IT Cost & Schedule AnalysisDaniel HarperFederal agencies have experienced a growing demand for rapid turnaround cost and schedule estimates. This need is increasing as the pressure to deploy systems rapidly mounts. The push for Agile SW development compounds this problem. A critical component in cost estimating is the data collection of costs for the various elements within the estimate. Analogous programs constitute a robust source for credible estimates. The problem is how to find analogous programs and how to capture the cost of elements within those programs at a sufficiently detailed level to use in a cost estimate and in a timely manner so that the cost data is still relevant. ...2014Information Technology
Trends in Enterprise Software Pricing from 2002 to 2011Ian Anderson, Dara LoganOne of the biggest challenges in the cost estimating community is data collection. In the Information Technology (IT) cost community, technology is always evolving, while the data capturing it tend to be scarce and more difficult to use in building solid cost models. Fortunately, NCCA learned the Department of the Navy (DON) Chief Information Officer (CIO) has been collecting benchmarking measures, including pricing, since 2002 under the Enterprise Software Initiative (ESI) Blanket Purchasing Agreements (BPAs). ... 2014Information Technology
Estimating Cloud Computing Costs: Practical Questions for Programs Kathryn ConnorCloud computing has garnered the attention of the Department of Defense (DoD) as data and computer processing needs grow and budgets shrink. In the meantime, reliable literature on the costs of cloud computing in the government is still limited, but programs are interested in any solution that has potential to control growing data management costs. ...2014Information Technology
The Agile PM Tool: The Trifecta for Managing Cost, Schedule, and Scope Blaze Smallwood, Omar MahmoudThe growing number of DoD software projects that are adopting an “Agile” development philosophy requires cost estimators to not only adapt the methodologies and metrics they use to estimate software development costs, but also re-think their models to give PMs the information they need to effectively manage these programs. The Agile PM Tool is one manifestation of this trend as it provides a logical, dynamic approach for helping the government effectively manage the cost, schedule, and scope of their “Agile” projects.2014Information Technology
Which Escalation Rate Should I Use?Nathan HonsowetzConducting life cycle cost estimates requires time frames of 10, 20, even 30 years, and with such long time frames it's important to use appropriate escalation indices. Escalation can have a significant impact on cost estimates, especially estimates with longer time frames. However, often cost estimators insert a "standard" escalation index into their models without considering whether that index is appropriate for their estimate. ...2014Life Cycle Costing
Ground Vehicle Reliability Analysis Using the Mean Cumulative FunctionCaleb FlemingThis paper surveys and outlines the fundamental MCF methodologies and explanations detailed in-depth in Wayne Nelson's Recurrent Events Data Analysis for Product Repairs, Disease Recurrences, and Other Applications. Applying the MCF to vehicle maintenance data reveals recurring component failure behaviors, develops new guidelines for interpretation, and assists in data normalization and validation.2014Life Cycle Costing
Cost Overruns and Their Precursors: An Empirical Examination of Major Department of Defense Acquisition ProgramsAlan Gideon, Enrique Campos-Nanez, Pavel Fomin, James WasekThis paper proposes a model of acquisition program future cost for two specific acquisition domains - aircraft and ships - that takes into account the non-recurring developmental costs defined at program approval and each domain's historic tendencies to exceed planned program cost. Technical and non-technical reasons for these observations are discussed. ...2014Life Cycle Costing
System Utilization: An In-depth Method of Modeling and Measuring Military Manpower CostsOmar MahmoudEstablishing defendable cost estimating methodologies for capturing military manpower costs is a key component in any Program Life Cycle Cost Estimate. With a proven and systematic approach to estimating military manpower costs, a program can be confident in selecting a proper course of action among competing alternatives when conducting an EA, appropriately designate their ACAT level, avoid the pitfalls that lead to over/under estimating or double counting costs, and above all obtain a high level of confidence from their resource sponsor and Milestone Decision Authority.2014Life Cycle Costing
Integrating Sustainability into Weapon System Acquisition within the Department of DefenseWalt Cooper, Remmie ArnoldDoD acquisition and logistics professionals use the term sustainment to describe the support needed to operate and maintain a system over its lifetime. In the context of the DoD acquisition process, sustainability involves using resources to minimize mission, human health, and environmental impacts and associated costs during the life cycle. This paper will present a draft version of "DoD Guidance - Integrating Sustainability into DoD Acquisitions," initial findings from pilot studies, and the challenges and road ahead. ...2014Life Cycle Costing
Cost Analysis & Optimization of Repair Concepts Using Marginal AnalysisJustin WoulfeOPRAL is an analytical model for determining the optimal repair locations and spares allocations in a multi-level hierarchical support organization to optimize Life Cycle Cost. With this model, the analyst can either treat repair decisions as fixed and given as input parameters, or using the OPRAL algorithm, evaluate several different repair strategies in order to find the optimal one, considering all aspects of life cycle cost. ...2014Life Cycle Costing
A Balanced Approach to Meeting Fiscal ConstraintsSteve Green, Kevin Davis, Kurt HeppardThe effective and systematic use of cost and budgeting information is a critical component of strategic planning and decision making in most organizations. The Department of Defense's (DoD) current operational environment, scarce resources, and conflicting stakeholder expectations are resulting in extreme fiscal constraints. The result is the need to reconsider missions and goals, reassess priorities, entertain force structure alternatives, and ultimately reduce budgets. ...2014Cost Management
Cost Overruns and Cost Growth: A Three Decades Old Cost Performance Issue within DoD's Acquisition EnvironmentLeone YoungFor the last three decades, the US Department of Defense (DoD) has been encountering program performance issues such as inaccurate and unrealistic estimations for its acquisition programs, and its effort of eliminating cost overruns and cost growth phenomenon has been deemed ineffective as well. This paper examines and consolidates multiple government reports and studies, which pertain to the cost performance of DoD that has been viewed as unsatisfactory and problematic. ...2014Cost Management
Supplier Cost/Price Analyses - Best Practices for evaluating supplier proposals and quotesMike MardesichDuring proposal planning, preparation, and review, an important but often over looked aspect is the evaluation of a supplier's proposal or quote. Requirements in the Federal Acquisition Regulations (FAR) provide for certain visibility into supplier's proposals and quotes, depending on the value of the proposal/quote. Additionally it is critical to note the FAR puts the onus of supplier cost (or price) analysis on the prime contractor. ...2014Cost Management
Innovative Business Agreements and Related Cost & Pricing Methods at NASA in Support of New Commercial ProgramsTorrance Lambing, James RobertsThis paper and presentation, focusing on Kennedy Space Center, will discuss changes and new methods of pricing and estimating the costs of NASA facilities and services to be provided to outside entities for use in new Commercial Space endeavors. It will also give an overview of new NASA regulations and documents that establish policy and guidance for entering into Agreements and how they are priced under the various types of Agreement currently being used at NASA. ...2014Cost Management
The Other RCA: Restaurant Cost AnalysisPeter BraxtonThe Weapon Systems Acquisition Reform Act (WSARA) of 2009 highlighted the importance of Root Cause Analysis, or RCA, but its conduct remains shrouded in mystery. In illustrating the central role of risk and uncertainty analysis in cost, Dick Coleman often made the provocative pronouncement to the effect of "You can't stand outside a restaurant with a menu, and the people you'll be dining with, and a calculator and get within 10% of the final bill, so what makes you think you can estimate a complex multi-billion-dollar acquisition program with that precision?!" ...2014Cost Management
Intelligence Mission Data Cost Methodology GuidebookEugene Cullen, III, Matthew SchumacherIn 2013, Booz Allen Hamilton developed and authored the Intelligence Mission Data Cost Methodology Guidebook (IMD CMGB) for the Defense Intelligence Agency's (DIA) Intelligence Mission Data Center (IMDC). This guidebook is the official costing manual for life-cycle mission data planning, required by Department of Defense Directive (DoDD) 5250.01 "Management of IMD within DoD Acquisitions," and defines costing methodologies, procedures, and processes that result in OSD Cost Assessment & Program Evaluation (CAPE) and Government Accountability Office (GAO) compliant cost estimates that are required by DoD Acquisition Systems. ...2014Cost Management
Achieving Affordable Programs NRO Cost Analysis Improvement Group (CAIG) Support of Cost Driver IdentificationLinda Williams, Pat Kielb, Eileen DeVillier, Jay MillerThis paper will describe the approach taken over the last several months to identify cost drivers and summarize findings that senior leaders found helpful to the decision making process. In summary, as budgets reductions trigger asset affordability reviews, the NRO CAIG has been able to assist in identifying total organizational cost drivers and provide a framework for future cost reduction opportunities.2014Cost Management
Gödel's Impact on Hilbert's Problems Or Cost Consistency and Completeness as an Impossible ExerciseDavid PeelerIn a previous set of papers, the idea of using Hilbert's Problems as a construct to propose and recently revisit the status of a list of Hilbert's Problems for Cost Estimating. This paper similarly employs Gödel's enlightenment with respect to Hilbert's attempts onto the cost estimating community. What can we learn about ourselves as estimators and where can we exert the greatest impact with respect to the use of our estimates? Using Godel's two theorems of undecidability as catalyst, we will explore the effect and utility of exacting math and other motions on cost estimates specifically and programmaitics generally.2014Cost Management
A New Cost Management Policy for Your Organization: An Integrated Approach?Tom Dauber, Woomi Chase, Ken OdomDeveloping a robust Cost Management Policy is a key driver to the success of any organization, regardless of size or industry. The policy should ensure cost control measures that are valid and effective, risks are mitigated, solutions are delivered on time, and profits/ROIs are maximized. The Cost Management Policy should be a systematic approach to managing cost through-out the life cycle of a program through the application of cost engineering and cost management principles. ... 2014Cost Management
Right Sizing Earned Value Management for Your ProjectGordon KranzEarned Value Management (EVM) is a program management tool that provides data indicators that can be used on all programs to enable proactive decision making throughout the program lifecycle and facilitate communication across the program team. Each program has unique attributes that should be considered when determining program management and reporting requirements, including, but not limited to, contract size and type, scope of work, complexity, risk, technology maturity, and resource requirements. ...2014Cost Management

Military Construction Cost Estimating

Nicole Barmettler (Sullivan)An informative presentation on construction cost estimating specifically dealing with military facilities. Within this topic, the author will identify, define, and explain the cost methodologies and cost adjustment factors considered when developing construction cost estimates for general military facilities. Project costs will be exemplified by illustrating a breakdown and walkthrough of the process. The author will specifically discuss the process involved in a five year facility acquisition timeline that is usually required for a typical major military construction effort, which is defined by a project cost exceeding $750,000.2014Methods and Models
Cost and Performance Trades and Cost-Benefits AnalysisSteven IkelerThis paper will discuss the basics of Trades Analysis and the C-BA. It will include what the cost analyst should do to prepare and what information to collect during the Trades Analysis. One observation is that non-traditional Work-Breakdown Structures need to be considered and the analyst should model potential second and third order effects beforehand. ...2014Methods and Models
Lessons Learned from Estimating Non-Standard Equipment Returning from Overseas OperationsMichael MetcalfThis paper explores challenges in estimating the cost of retaining this equipment. Items that present unique challenges include: the type classification and full materiel release processes; repair, reset, and upgrade; short- and long-term storage; and knowledge retention and loss of wartime experience. We will also explore funding challenges; moving targets in number and configuration of retained systems; the transition of O&S from wartime contractor-based to peacetime organic; training; and system disposal and divestiture.2014Methods and Models
Weibull Analysis MethodErik Burgess, James Smirnoff, Brianne WongThe NRO Cost and Acquisition Assessment Group (CAAG) develops time-phased estimates for space systems in support of milestone decisions, budget formulation, and other program assessment tasks. CAAG relies on parametric budget-spreading models based on historical data to provide objective analysis and credible information before contract award or early in a program when there is little or no execution history available. ...2014Methods and Models
Study of Cost Estimating Methodology of Engineering DevelopmentMyung-Yul LeeThe purpose of this paper is to study the actual engineering labor hours for FSED Program and PE/PI project and to create a parametric estimating model in a weapon system, especially cargo aircraft. The C-17 final assembly facility in Long Beach, California closes in 2015. It is worthwhile to review historical data for the C-17 FSED program and PE/PI project, both programs have engineering development program in view of estimating methodology research.2014Methods and Models
Validation and Improvement of the Rayleigh Curve MethodMichael MenderThe presentation begins with a brief overview of the Rayleigh method, followed by an explanation of R-squared and the issues with non-linear functions. We then discuss the method for linearizing the Rayleigh function and then using it for linear regression. We then provide a brief demo of the tool we developed and then move on to a discussion of our results pertaining to the performance of the Rayleigh method. ...2014Methods and Models
Rotorcraft Cost Model Enhancements for Future ConceptsF. Gurney Thompson IIIThis paper will discuss the ongoing research efforts to improve upon existing rotorcraft cost estimation capabilities, from both a cost estimating relationship (CER) update and a model development perspective. We will share our approach, many of our findings, and any lessons learned. Efforts currently underway include data collection, updates to existing cost models and their CERs, adding support for new aircraft types and technologies, and the addition of new analysis capabilities to better understand total ownership cost.2014Methods and Models
Kill Vehicle Work Breakdown Structurennifer Tarin, Christian Smart, Paul TetraultThis paper provides an alternative to Appendix C: Missile Systems for inclusion in MIL-STD-881C, the Department of Defense standard for Work Breakdown Structures (WBSs). The Missile Defense Agency (MDA) produces interceptors that are similar to missiles with the exception of the payload element. Where Appendix C defines the payload element with a limited set of WBS elements, the MDA interceptor payload, referred to as a kill vehicle, includes a large collection of significant WBS elements. ...2014Methods and Models
Meet the Overlapping Coefficient: A Measure for Elevator SpeechesBrent LarsonYou've seen this picture before. . . a plot of two overlapping probability distributions. You may have created one with an overlay chart. Typically this graphic contrasts two cost distributions so as to illustrate similarity, difference or change. However, seldom seen is a number that quantifies the overlap or area shared by both distributions. ... 2014Methods and Models
Excel Based Cost Roll Up MethodMatthew LeezerThis paper will show the user how to create a custom function (sumlowerlevel) using Visual Basic and apply this function in Excel to generate a report that will save time and increase the accuracy of estimates. The method uses the lookup function to capture the cost of purchased components and subsystems and uses the custom function for all make assemblies to roll up the cost of the purchased parts. ...2014Methods and Models
The Role of Cost Estimating in Source SelectionAnnette BarliaThe analysis will focus on the process of developing an IGCE and then utilizing it to evaluate vendor proposals for the acquisition of new technology. It will demonstrate that a strong IGCE facilitates source selection. If the cost estimate is developed the right way, organizations will have more leverage during contract negotiations with vendors, and the acquisition will run smoothly and meet program goals. ...2014Methods and Models
Automated Costing to Identify Opportunities to Reduce Direct Material SpendJulie Driscoll, Dale MartinThis session will cover how technology is automating the costing process through integrating costing solutions with CAD, PLM and ERP; pulling information about components directly from CAD files; and using an intelligent cost engine that evaluates manufacturing routings for feasibility and cost effectiveness. We will look how these solutions enable automated batch costing of components and are used by manufacturers to support cost reduction projects. ...2014Methods and Models
Moving Beyond Technical Parameters in our CERsEric Druker, Charles HuntOne of the frequent criticisms of JCL analysis (integrated cost and schedule risk analysis) has been that the results typically exhibit coefficients of variation (CV) that are orders of magnitude less than those seen in parametric estimates of similar scope. During a recent NASA research task examining how parametrics estimates can be linked to program management artifacts, the research team stumbled upon a characteristic of our Cost Estimating Relationships (CERs) that almost certainly leads our parametric estimates to have higher than necessary CVs. ...2014Parametrics
Using Dummy Variables in CER DevelopmentShu-Ping Hu, Alfred SmithThis paper explains the reasons for using dummy variables in regression analysis and how to use them effectively when deriving CERs. Specific guidelines are proposed to help analysts determine if the application of dummy variables is appropriate for their data set. This paper also demonstrates some common errors in applying dummy variables to real examples. An application using dummy variables in splines (to derive the fitted equation as well as the intersection) is also discussed.2014Parametrics
Bayesian Parametrics: Developing a CER with Limited Data and Even Without DataChristian SmartThis paper discusses Bayes' Theorem, and applies it to linear and nonlinear CERs, including ordinary least squares and log-transformed ordinary least squares.2014Parametrics
Tactical Vehicle Cons & Reps Cost Estimating Relationship ToolCassandra Capots, Jeffery Cherwonik, Adam James, Leonard Ogborn When estimating Operating and Support (O&S), it is reasonable to assume that as reliability increases, consumable and reparable parts ("cons and reps") cost should decrease (less frequent repairs), while as vehicle price increases, parts cost should increase (more expensive parts). Developing a dataset to support cost estimating relationships (CERs) for the Army's Tactical Vehicle fleet is a significant challenge. ...2014Parametrics
Unmanned Aerial Vehicle Systems Database and Parametric Model ResearchBruce Parker, Rachel Cosgray, Anna Irvine, Brian Welsh, Patrick Staley, Praful PatelThis handbook documents the first two years of research sponsored by NCCA and ODASA-CE. With the inclusion of UAS in the United States' (U.S.) military arsenal, the government has a desire to understand the components of a UAS including the air vehicle, GCS and payloads, the development and production process, and the O&S implications of these systems. ...2014Parametrics
Building a Complex Hardware Cost Model for AntennasDavid Bloom, Danny PolidiThis paper discusses the development of a Complex Antenna Cost Model based on quantifiable sizing mechanisms which are designed to quickly and accurately calculate the "top-down" cost for all engineering and operations disciplines and functions required for antenna development and test. ...2014Parametrics
ESA Project Office Cost ModelHerve JoumierThis paper describes the definition and the implementation of a Project Office parametric cost model aimed at defining a reference manpower allocation based on fair judgement and rational fair modelling. It has been developed to improve the cost estimations capability of ESA providing outputs that are used by agencies for comparison with contractors' proposals. ...2014Parametrics
Improving the Accuracy of Cost Estimating Relationship for Software SystemsDavid WangIn this paper, we leverage recently published results on the statistical characterization of schedule and cost risks to analyze the prediction accuracy of CER for software systems. Our analytical analysis and empirical statistical analysis of actual code size growth data suggest that the statistics of code size estimate can also be characterized by fat-tail distributions....2014Parametrics
Hybrid Parametric Estimation for Greater AccuracyWilliam RoetzheimThis talk will discuss hybrid parametric estimation based on HLO catalogs, and give examples of the application and accuracy of this technique within organizations including the State of California, Halliburton, IBM, Procter and Gamble, and multiple top 25 financial institutions.2014Parametrics
Linking Parametric Estimates to Program Management Artifacts (LPEPM)Mike Smith, Ted Mills, John SwarenA common fate of parametric cost and schedule estimates is that they fall into disuse as a Project's own artifacts (e.g. Work Breakdown Structure (WBS), budget, schedule, risk lists, etc.) are created and mature. Parametric estimates typically do not map cleanly to WBS or schedule-derived artifacts, allowing a sense among Project Managers (PMs) ? rightly or wrongly ? that "parametric estimates are fine, but they don't reflect my project." ...2014Parametrics
Impact of Full Funding on Cost Improvement Rate: A Parametric AssessmentBrianne Wong, Erik BurgessThe NRO Cost and Acquisition Assessment Group (CAAG) currently houses data collected from various U.S. Government organizations, including the Department of Defense and NASA. These data points are pooled with NRO data and used in Cost Estimating Relationships for space hardware, which underpin CAAG estimates for major system acquisition programs, aiding in the development of programs and budgets. ...2014Parametrics
Developing R&D and Mass Production Cost Estimating Methodologies for Korean Maneuver Weapon SystemDoo Hyun Lee, Sung-Jin Kang, Suhwan KimIn this research we have attempted to establish a CER development process to meet the current need, and found certain cost drivers for the Korean historical maneuver weapons system data, using Forward selection, Stepwise Regression and R square selection. We have also developed a CER model for production labor costs, using Learning rate which has been generally applied to estimate valid production labor costs. ...2014Parametrics
Excel Based Schedule Risk and Cost EstimatesWilliam EvansThe emphasis on robust cost and schedule estimating solutions has resulted in the creation of multiple solutions for analysts and clients. Excel based integrated cost and schedule risk is only one methodology for solving client problems. Incorporating cost and schedule risk in Excel leads to an increased ability to audit and trace the schedule and cost risk methodology throughout an Excel based PLCCE, improving the confidence and robustness of the estimate. While there are hurdles to the implementation of an Excel based schedule risk solution, when combined with form controls, the benefits to PLCCE auditability and usability are immense.2014Risk
Using Bayesian Belief Networks with Monte Carlo Simulation ModelingMarina DombrovskayaOne of the main aspects of creating a Monte Carlo simulation cost estimate is the accuracy in defining uncertainty and risk parameters associated with the cost components of the model. It is equally important to assess and accurately represent inter-dependencies between uncertain variables and risks, which are measured via correlation. Since oftentimes historical data is insufficient for a rigorous statistical analysis, both probability distribution and correlation are commonly estimated via a subject matter opinion. ...2014Risk
Expert Elicitation of a Maximum Duration Using Risk ScenariosMarc GreenbergThis paper provides with a brief review on a current method to elicit a most-likely commute time, a "practical maximum" commute time and risk factors that contribute to commute delays. This paper continues by showing how these risk factors can be organized into an objective hierarchy of risk factors, leading to the creation of a customized risk work breakdown structure (WBS). ...2014Risk
Quantifying the Necessity of Risk Mitigation StrategiesJames Northington, Christopher Schmidt, Chuck KnightThis paper will begin by highlighting flaws with the current risk management process, walk through the new proposed methodology for risk mitigation, and provide a quantitative example of the process in action using raw data. In the end, the proposed methodology will provide a greater understanding of program risks, a measurement of importance of implementing a risk mitigation strategy, a measurement of the mitigation strategy's subsequent impact, and a quantitative measurement of benefit for Program Mangers to defend their risk mitigation strategies.2014Risk
Improved Decision Making with Sensitivity AnalysisBlake BoswellIn this study, we review common applications of SA methods to project estimation including a description of each method as well as its advantages and disadvantages. Additionally, we explore the topic of Global Sensitivity Analysis (GSA), which is a process for measuring the overall contribution of uncertain model inputs to variation in model outputs and is a popular technique for model validation in engineering and life sciences. ...2014Risk
Affordability Engineering for Better Alternative Selection and Risk ReductionMarlena McWilliams, Bob KouryThis paper will outline the process and steps to how to implement affordability into your estimating environment to understand system requirements vs. system costs and affordability; and provide best value identifying and accepting the most affordable, feasible, and effective system or alternative. The need to evaluate and assign a best value is essential to both the government (DoD) and the contractors supplying systems / alternatives to the government.2014Risk
Risk Adjusted Inflation IndicesJames BlackIt is often observed that Office of the Secretary of Defense (OSD) inflation rates are different than prime contractor specific inflation rates seen in Forward Pricing Rate Agreements/Proposals (FPRAs/FPRPs) and in commodity group composite rates (e.g. Global Insight indices). Yet, it is a standard practice in many cost estimating organizations to use OSD inflation rates for escalating costs in estimates without giving consideration to a range of different possible inflation rates. ...2014Risk
Critique of Cost-Risk Analysis and Frankenstein Spacecraft Designs: A Proposed SolutionMohamed Elghefari, Eric PlumerIn this paper, we present a historical data driven probabilistic cost growth model for adjusting spacecraft cost Current Best Estimate (CBE), for both earth orbiting and deep space missions. The model is sensitive to when, in the mission development life cycle, the spacecraft cost CBE is generated. The model is based on historical spacecraft data obtained from the NASA Cost Analysis Data Requirements (CADRe) database. ...2014Risk
A Next Generation Software Cost ModelJairus Hihn, Tim Menzies, James JohnsonIn this paper we will summarize our findings on effort/cost model estimation and model development based on ten years of software effort estimation research using data mining and machine learning methods. We will then describe the methodology being used in the development of a NASA Software Cost Model that provides an integrated effort, schedule, risk estimate, as well as identifying the changes in the project characteristics that are most likely to improve a given projects cost-schedule performance and risk exposure.2014Space
NASA's Phasing Estimating RelationshipsChad Krause, Erik Burgess, Darren ElliottCost and schedule estimating in support of budget formulation is limited when cost phasing is not considered. As a result, NASA'a Office of Evaluation (OE) Cost Analysis Division (CAD) initiated a review of historic mission funding profiles for the purpose of corroborating current phasing profiles and optimizing future budgeting performance. Actual expenditures by year, technical parameters, and programmatic information were compiled and normalized from NASA's extensive library of CADRe (Cost Analysis Data Requirment) documents for programs since 1990. ...2014Space
NASA Instrument Cost Model (NICM)Hamid Habib-Agahi, Joseph Mrozinski, George FoxThe NASA Instrument Cost Model (NICM) includes several parametric cost estimating relationships (CERs) used to estimate NASA's future spacecraft's instrument development cost. This presentation will cover the challenges associated with creating cost models in an environment where data on previously built instruments is 1) sparse, 2) heterogeneous and 3) book-kept differently by the various NASA centers and support institutions. ...2014Space
The NASA Project Cost Estimating CapabilityAndy Prince, Brian Alford, Blake Boswell, Matt PitlykThe paper begins with a detailed description of the capabilities and shortcomings of the NAFCOM architecture. The criteria behind the decision to develop the PCEC are outlined. Then the requirements for the PCEC are discussed, followed by a description of the PCEC architecture. Finally, the paper provides a vision for the future of NASA cost estimating capabilities.2014Space
Developing Space Vehicle Hardware Nonrecurring Cost Estimating Relationships at the NRO CAAGRyan Timm, Jan SterbutzelThis paper builds on our 2012 SCEA conference briefing that described the NRO CAAG approach to developing Space Vehicle (SV) hardware Cost Estimating Relationships (CERs) for Nonrecurring (NR) engineering. These CERs are developed from the NRO CAAG's cost database of more than 2300 space hardware boxes, and can stand as alternatives to other popular parametric tools, like the nonrecurring CERs in USCM or NAFCOM. ...2014Space
NASA JCL: Process and LessonsSteve Wilson, Mike StellyOur paper will describe JCL implementation and address the creation, implementation, evolution, inherent benefits, inherent issues, its ultimate place among program management's decision-making toolset, and hard recommendations for organizations hoping to wage successful JCL campaigns. Real-world examples will be referenced, including those from the Constellation, Commercial Crew, and Orion spacecraft development programs. ...2014Space
NASA Project Cost Estimating Capability: New Analyses for Spacecraft EstimatingBrian Alford, Frank A. (Andy) PrinceThe NASA Project Cost Estimating Capability (PCEC) is a free, publicly-available parametric tool for estimating the cost of spacecraft and transportation systems for scientific and human exploration of our solar system and beyond. In development since late 2013, PCEC replaces the NASA / Air Force Cost Model (NAFCOM). This presentation will discuss the new overall philosophy and data analysis underlying PCEC, the challenges encountered during development, and provide a high-level overview of the2016 Int'lGovernment Perspectives
OSCAM - US Navy's Operating and Support Cost Estimating ToolStephen CurramOSCAM is a simulation based tool for operating and support cost estimating that has been used by the US Navy for 20 years. It plays an important role in submissions for US DoD procurement milestones. An overview and brief history of OSCAM will be given, with examples of equipment programmes that have used OSCAM. It will conclude with lessons learned, including insights on the type of data required for operating and support cost estimating.2016 Int'lGovernment Perspectives
Master of Cost Estimating & Analysis BriefGreg MislickGreg Mislick will provide an update on the all-distance learning Master’s Degree and/or Certificate Program in Cost Estimating and Analysis (MCEA / CCEA) offered at the Naval Postgraduate School (NPS). Entering its sixth year, the MCEA program has proven to be a success for both the students who take the program and the DoD Services and other US Government Executive Agencies in which the students work. Expert cost estimates are key underpinnings for critical government processes dependent on credible and reasonable cost estimates, from budgeting to cost-benefit and financial analyses.2016 Int'lGovernment Perspectives
NATO Agency Transformation: Creating a Centralized Cost Estimating CapabilityCandace MahalaA new NATO Agency is born, transforming multiple agencies into one, changing culture, and implementing a centralized robust cost estimating capability. This is the journey of how to grow and cultivate cost estimating from inception, integrating into projects as a value added asset, maturing over time and eventually becoming a fundamental part of the culture in the acquisition and execution processes.2016 Int'lGovernment Perspectives
NASA's X-Plane Database and Parametric Cost Model V2.0Steve SterkIn 2015 the idea of creating an Armstrong X-Plane Cost Model was conceived. Initially the data being used was leveraged from Dr. Joe Hamaker’s Quick Cost v5.0 and verified and supplemented by Armstrong’s summer intern Aaron McAtee (PhD). After several peer reviews, it was suggested to go back and take a hard look at the data collected. Our objective is to produce an X-plane database that ensures that all the data is as accurate, traceable and far-reaching as possible.2016 Int'lGovernment Perspectives
Cost Estimate Credibility from a Government PerspectiveHenry ApgarThis original paper describes a search for the definition of "Cost Estimate Credibility." The author documents his review of government-agency estimating handbooks, contemporary textbooks, previous ICEAA papers and journal articles, and the opinions of contemporary experts. The paper concludes with a concise definition which can be used by government and contractor estimators for their cost models and cost estimates.2016 Int'lGovernment Perspectives
The Value of R&D - a Real Options Analysis ApproachJohn Shimell, Neil DaviesThis study sought to apply Real Option Analysis (ROA) to answer the question “what is the value achieved from the MODs Chief Scientific Advisor’s (CSA) £400m annual budget”. Typically this has been reviewed retrospectively via case studies and surveys. This study developed a tool based on an amended Black & Scholes Formula, combined with a Monte Carlo analysis of the “tree” of R&D to determine a predictive value for the R&D spend of the CSA.2016 Int'lManagement
What Does A Mature Cost Engineering Organisation Look Like?Dale ShermonCost Engineering Health Check; How good are those numbers? is Dale Shermon's second book and co-authored with Dr Mark Gilmour. The book is the paper and the presentation will consider what a mature cost engineering organisation looks like? It will cover the essential requirements for a credible and justified cost forecast.2016 Int'lManagement
Using Public Data for Validation and Winning BidsAlexander KingTo win bids, cost forecasting involves more than just creating a reasonable baseline. This presentation will show how public data can be combined and triangulated to ensure our predictions are competitive as well as reasonable. The presentation will show how parametric estimates, past contract data, financial statement data and statistical data can combine to make powerful metrics for productivity and competitiveness.2016 Int'lManagement
OSA: Cost and Schedule Saver or Driver?Victoria Cuff, Brian D. FerschOpen Systems Architecture employs a modular design and utilized consensus based standards for its key interfaces with full design disclosures [Office of the Deputy Secretary of Defense, Systems Engineering, Initiatives]. A challenge with OSA is the ability to continuously design modular systems with obsolete technology and pieces, evolving technological environment and emerging threats. Examining past successes and failures, this study will analyze means to reduce costs and increase success.2016 Int'lManagement
Is it Worth It? The Economics of Reusable Space TransportationRichard L. WebbMuch has been invested in reusable space transportation systems in the belief that reusing system elements is cheaper than expending costly hardware each flight; a view not held by all industry stakeholders. This paper will define the metrics by which stakeholders measure financial "goodness," provide examples of application of the metrics to investment decisions, and offer observations on their potential influence on current and future decisions regarding space transportation system investment.2016 Int'lManagement
The 10+/-2 Factors for Estimate SuccessAndy NolanA study of completed projects across Rolls-Royce revealed the factors that lead to accurate estimates. This has enabled us to develop an Estimate Readiness Level (ERL) assessment method, to determine the likely accuracy of an estimate. Success is no accident and estimates accuracy can be predicted from the outset of a project. This paper will cover the factors for estimate success and the development of the ERL assessment.2016 Int'lManagement
The Signal and the Noise in Cost EstimatingChristian B. SmartWe seek to extract signal and eliminate noise when building models with historical data to predict future costs. One common problem is normalization, which is necessary when making comparisons but can inject noise when used in modeling. We discuss kernel smoothing and distribution fitting as ways to avoid overfitting peculiarities in historical data and the important issues of cross-validation and parsimony as ways to validate models and avoid the lure of overfitting.2016 Int'lModels & Methods
Facilities Cost Estimates Drivers in the Oil and Gas Field DevelopmentLinda Newnes, David Peacock, Ettore Settani, Jon WrightIn aerospace and defence research has focused on estimating the cost of delivering an outcome, e.g. available aircraft. Our research shows that focusing on the product alone, using product reliability/failures does not reflect the cost of delivering an outcome. In this paper we show why this is the case and how the use of input-output modelling and systems engineering can be used. We then demonstrate how the approach can be used in estimating the cost of waste water treatment.2016 Int'lModels & Methods
Successful Cost Estimating with T1-EquivalentsGrady Noll, F. Gurney Thompson IIIThis paper discusses our research and development of a cost model for additive manufacturing (AM), also known as 3D printing. Working with academia and multiple AM companies, we investigated cost drivers, collected cost and technical data, derived AM CERs, and developed an approach for including AM components within a larger parametric cost estimate. This paper discusses the research approach, data collection, results, cost model implementation, and future work.2016 Int'lModels & Methods
Costing Aircraft Availability and 50-Shades of Grey WaterAndrew Langridge, Anna FooteData gathering is often one of a cost estimator’s biggest nightmares for many reasons. Even in cases where data is available, the logistics of data collection can be daunting. This paper discusses a methodology that PRICE has developed to use open source web crawling software to regularly and repeatedly update guidance on commodity prices for Information Technology items with the push of a button. Uses of this methodology and further directions will be presented.2016 Int'lModels & Methods
Outing the OutliersAlan R. JonesInappropriate inclusion or exclusion of data that may or may not be regarded as outliers may make a project unachievable or uncompetitive. This paper looks at some of the issues involved and what tests are available to support an informed decision process rather than relying on a random guess on what is, or is not, an outlier. The paper will compare the performance of these tests against a number of sample data sets.2016 Int'lRisk Analysis
The Way From a Parametric Estimate to a CAD-Driven CalcualtionJoachim Schoeffer, Herbert SpixThe industrial requirements for quick, rough and exact costings (e.g. for budgeting) and the detailed cost break down to the last screw or processor is as old as both methodologies exist. To satisfy both fractions we need different solutions that address both needs. Ideally both scenarios are integrated and interface between each other. In this presentation the author shows a way of integrate parametric and CAD-driven bottom-up.2016 Int'lRisk Analysis
A Parametric Model for the Cost Per Flight HourMichail BozoudisThe parametric estimating technique provides timely cost estimates for "unknown" systems, utilizing cost estimating relationships deriving from historical datasets. This case study describes the development of a parametric model that estimates the cost per flight hour (CPFH). The cost derives as a function of the aircraft empty weight and the engine’s specific fuel consumption. As an example, the F-35A CPFH is estimated under the hypothetical scenario that it is operated by the Hellenic Air Force.2016 Int'lRisk Analysis
Should You Care About How Good An Estimation Process?Alain AbranThis presentation is based on the author’s 2015 book on Software Project Estimation (Wiley & IEEE Press) and illustrates how organizations (including in the car industry) having collected their own data using international standards have built their own estimation models and developed a key competitive advantage through improved software estimation capabilities. Examples will include data from a large European car manufacturer and a small software organization using Agile.2016 Int'lRisk Analysis
CER Issues And SolutionsKurt BrunnerCost estimating Relationships (CERs) are widely accepted as an effective cost estimating tool. There are some issues with CER development and application that should be addressed. This briefing will explore those probable shortcomings and how they may be alleviated with the goal of refining our cost estimating capabilities.2016 Int'lRisk Analysis
Inherent Risk in Spreadsheets (IRiS)Alan R. JonesMany spreadsheets may be certified as being error-free on the day they are tested, but some are more prone to future errors due to the manner in which they have been designed or compiled.2016 Int'lRisk Analysis
Satellite Mass GrowthDonald MacKenzie, Erik BurgessA new study of satellite mass growth on 21 programs shows that equipment type and design maturity are drivers. Unlike current AIAA guidelines, acquisition strategy is shown to be a significant driver, with competitively awarded programs experiencing much higher growth than sole-source awards, even for units with the same design maturity. Overall, mass growth allowance recommended by AIAA for every equipment type is much lower than the actual growth experienced in this dataset.2016 Int'lSpace Systems
QuickCost 6.0Bob HuntQuickCost 6.0 has just been released by Galorath Federal and NASA. QuickCost 6.0 is an all new implementation QuickCost (QuickCost 1.0 was released in 2004). The philosophy behind the model is that accurate cost and schedule estimates of automated spacecraft missions can be obtained parametrically in a quick and easy fashion estimating at the top 11 NASA WBS elements and using only a few input variables. QuickCost 6.0 is based on data from the NASA ONCE CADRe database.2016 Int'lSpace Systems
Hubble Space Telescope and Its Color PhotographsDavid PineThe Hubble Space Telescope has taken thousands of "color" photographs that have excited people the world over. This presentation examines the "Mirror Problem" and then how scientists change the photons being collected by the telescope into the phenomenal photographs that have thrilled the public. Understanding the color photographs begins with understanding what color is, and how Hubble's cameras operate to eventually produce pictures.2016 Int'lSpace Systems
Halfway to AnywhereFabian EilingsfeldA 'tariff zone map' is proposed for space transportation. Using parametric analysis, applying the paradigm of terrestrial public transport, it displays specific energy cost depending on total energy delivered to payloads. Every space transportation system leaves its own distinctive footprint when its performance data is plotted on the proposed map. It visualizes performance and cost trends with an emphasis on affordability, which helps rational discussion of project options and roadmaps.2016 Int'lSpace Systems
The SSCAG LegacyHenry ApgarActive partnership in this ICEAA Symposium marks the final milestone of a spectacular forty-year journey of the Space Systems Cost Analysis Group. SSCAG provided a unique international forum for space-systems cost analysts. Over those years, more than 300 skilled cost estimators, business analysts, program managers, space engineers, and logisticians filled the international venues, at more than 100 scheduled meetings, to deliberate major issues associated with government and commercial development.2016 Int'lSpace Systems
Cloud Solutions - Infrastructure, Platform or SoftwareArlene F. Minkiewicz, Ashley HoenigkeMigrating capability to the cloud comes with several planning and management challenges. How does an organization determine the right capability to migrate and the right platform for the migration? What challenges do the various cloud solutions (Infrastructure vs. Platform vs. Software as a Service) present? This paper presents a case study outlining the migration activities and other implications for migrating the same application on each of the three cloud solution models.2016 Int'lSoftware
Measurement of Software Size: Contributions of COSMIC to Estimation ImprovementsAlain Abran, Charles Symons, Frank VogelezangThis talk presents 1) an outline of the design of the 2nd generation COSMIC method for measuring a functional size of software; 2) industry evidence of its practical value in software project performance measurement and its accuracy in estimating; 3) an account of the method’s full automation, with very high accuracy, for real-time embedded software specified in Matlab-Simulink and developed through a world network of software contractors.2016 Int'lSoftware
The Monitoring and Early Warning Indicators for a Software ProjectChristine GreenThis presentation will focus on controlling by monitoring and early warning indicators on a software project’s scope, estimate and performance - all factors that highly influence the cost and budget.2016 Int'lSoftware
Software Estimating Model Using IFPUG Standard Sizing MethodsChristine GreenA major input for any cost model for software projects are the scope and effort for a software projects regardless of technology or type of project process. IFPUG Sizing Standards methods can be utilized for software estimating model and thereby establish a valuable input for the cost models produced that will include both scope control and effort estimates.2016 Int'lSoftware
Introduction to Software Obsolescence Cost Analysis FrameworkSanathanan RajagopalAlmost every project in Defence sectors has got software with various degrees of complexity and dependencies. Whilst various research and studies have been conducted on system obsolescence and tools developed to cost systems and components obsolescence, no major research has been undertaken to develop a framework to estimate cost of software obsolescence. This paper will discuss the Software Obsolescence Cost Analysis Framework developed as part this research.2016 Int'lSoftware
Applying Earned Value to Agile Development ProgramsBob Hunt, Michael ThompsonA general movement to Agile software development process is one method used to address and potentially resolve these problems. The movement to Agile Software Development introduces several unique problems for EVM assessments. This paper will discuss some key element associated with: Agile/Hybrid Agile software development; Establishing a process for developing the technical, cost and schedule baseline; Applying Earned Value Management to agile programs.2016 Int'lSoftware
The [Whole] Truth about ANSI-compliant EVMSMichael NosbischOver the past several years, there have been multiple presentations by government agency representatives and consultants alike that have characterized an EVMS that complies with the ANSI/EIA 748(B) Standard as simply "good project management," and as such should not be that difficult for most successful government contractors to implement. It is true that the basic concepts and principles of EVM can absolutely be likened to sound project management. However, the costly, time-intensive process of implementing, validating, and then maintaining an "ANSI-compliant" EVM system is far and above what any contractor would do if it were not a contractual requirement that is also fully reimbursable. ...2013Earned Value Management
Estimating Cost To-Go Without Stable EVM DataPeter Frederic, Ronald K. LarsonInitial independent estimates for development programs are typically created using parametric methods. As a program progresses, it becomes important to separate sunk cost from cost to-go. In the very early stages of a program, it is acceptable to estimate cost to-go by simply subtracting the reported sunk cost from the parametrically-estimated total cost. However, as the program continues to progress and more significant progress accumulates, we can no longer assume that dollars spent equate directly to technical progress. It is then necessary to understand progress to-date in order to estimate cost to-go...2013Earned Value Management
Time is Money: The Importance and Desired Attributes of Schedule Basis of EstimatesJustin Hornback"Time is Money" is a maxim made popular by Benjamin Franklin. It reflects the long understood importance of schedule requirements and their impact on cost. While the importance of cost basis of estimates (BoE) have gained in popularity among industry communities, schedule BoEs are, at least equally, if not more important as cost, but have yet to reach the same level of understanding of importance within the industry...2013Earned Value Management
Cost Estimation and Earned Value IntegrationSissy Gregg, Michelle EhlingerTraditionally, cost estimators and earned value analysts work separately from one another, focusing on their own individual discipline. We have found synergies by merging these disciplines together resulting in stronger analysis through improved methods, more robust data sets, and new tools. The benefits of cost estimating (CE) and earned value (EV) management integration are realized throughout the acquisition lifecycle and enhance senior leadership decision making and improve acquisition outcomes. This paper presents the proven benefits of CE/EV integration, including more accurate cost estimates, cost estimating relationships, improved program management support, and cross program assessments...2013Earned Value Management
Understanding Requirements for Subcontract EV Flow Down and ManagementMark InfantiDo you use Earned Value Management (EVM)? Do you have a government customer that is requiring you to use EVM and report the data monthly? In either of these situations, if you have a subcontractor (SC), you need to know how they will report to you. You need to have a plan on how you will incorporate their data into your data for a complete program understanding. If your SC reports to you at cost level how do you add the fee to their report? How do you add your companies general and administrative (G&A) overhead to their costs? Do you share your Integrated Master Schedule (IMS) with them or just the key milestones?...2013Earned Value Management
Measuring and Managing Organizational Performance in the GovernmentKellie ScarbroughIn this presentation, the author details the challenges that government agencies face when attempting to measure organizational performance and presents a general methodology for successfully implementing a robust performance measurement program. The majority of government agencies do not operate like private industry and face unique challenges when defining meaningful metrics and collecting performance data...2013Earned Value Management
The Use of Function Points in Earned Value Management for Software Development
Ben Netherland, Mike Thompson, Dan FrenchIn this presentation, the authors detail their efforts in the development of an EVM methodology for a government software development project utilizing the International Function Point User Group (IPFUG) function point software sizing metric. Traditionally it has been difficult to apply Earned Value Management (EVM) criteria to software development projects, as no tangible value is earned until the software is delivered to production...2013Earned Value Management
New Air Force Integrated Baseline Review (IBR) Process - A Quick Reaction Capability (QRC) PerspectiveJames Ross, Martin Levitan, Christine Bolton, Barbara MeyersSince the early 1990s, the U.S. Air Force has been using Integrated Baseline Reviews (IBR) as a key technique for early assessment of the efficiency and effectiveness of baseline plans, resource allocations, scheduling and costs. Much of the earlier IBR doctrine does an excellent job describing what is required to complete these assessments, but little detail had been written about the specific processes, procedures, steps and techniques required to undertake these assessments. The January 2012 "Air Force Integrated Baseline Review (IBR) Process Guide", developed by SAF/AQXC personnel, is one of the best documents to date to address this concern. A drawback of this document is that the described IBR Process is based primarily on experiences with, and lessons learned from, the Acquisition Category (ACAT) I KC-46 Tanker Program...2013Earned Value Management
Capacity Cost Model: Balancing the Demand for Software Changes Against the Supply of ResourcesChris KaldesA Program Executive Office (PEO) within the Department of Defense (DoD) is responsible for multiple information systems that support the Services (Army, Navy, and Air Force). This PEO is not only responsible for Operations and Maintenance (O&M) of the information systems, but also for implementing system change requests that are made by the Services' Senior Service Representatives (SSRs). The PEO was relatively satisfied with the status quo, but the executives within the PEO recognized that there were three problems with doing business today. (1) Currently, there was no way to scientifically indicate/prove to the SSRs what software changes they could promise...2013Information Technology
Software Maintenance: Recommendations for Estimating and Data CollectionCorinne Wallshein, Bruce Parker, Vanessa Welker, Thomas Harless, Peter BraxtonEntering a period of fiscal austerity, it becomes more important than ever to estimate and consider operating and support (O&S) costs, which represent the lion's share of life cycle cost (LCC) for most platforms, during acquisition. Given the ubiquity of software in today's complex programs, a key component of O&S is software maintenance. This paper will discuss our work to understand the process of software maintenance, data collected and normalized to date, and resultant benchmarks for use in developing and cross-checking software maintenance estimates. It represents an update to "Software Maintenance Data Collection and Estimating Challenges" (Welker, et al., SCEA/ISPA, 2012)...2013Information Technology
Domain-Driven Software Cost, Schedule, and Phase Distribution Models: Using Software Resource Data ReportsWilson Rosa, Barry Boehm, Brad Clark, Joe Dean, Ray MadachyInstead of developing cost and schedule estimation models with many parameters, this paper describes an analysis approach based on grouping similar software applications together called Productivity Types. Productivity types are groups of application domains that are environment independent, technology driven, and are characterized by 13 product attributes. Also consideration is given to the operating environment that the software operates within. Over 196 actual software projects from DoD's Software Resource Data Reports (SRDRs) were fully inspected and analyzed to produce a comprehensive set of Cost Estimation Relationships, Schedule Estimation Relationships, Software Productivity Benchmarks, and Best Practice Data Normalization Guide. Analysis results will be discussed in this presentation...2013Information Technology
Estimating Software Development Costs for Agile ProjectsBlaze Smallwood, Omar MahmoudAddressing the need to more rapidly develop and field capabilities for the warfighter, more and more software-centric DoD programs are transitioning towards an industry trend called "Agile" software development. While "Agile" software development is geared towards producing usable software products more rapidly than traditional waterfall or incremental methods, it also requires more flexibility with managing requirements. The main challenge this has created for program managers is figuring out how to effectively manage scope, cost, schedule, and performance in this flexible, fast-paced development environment in which requirements are more fluid. In turn, cost estimators have been challenged to develop new data collection approaches and estimating methodologies to more effectively estimate software costs for these "Agile" programs...2013Information Technology
Software Maintenance Cost Estimating Relationship Development for Space SystemsRyan TimmMethods development for Space Vehicle (SV) cost estimating has mainly focused on acquisition cost, as acquisition makes up the majority of a SV's lifecycle costs and has high interest from oversight agencies and congress. Tightening budgets and increased space vehicle life expectancy has bolstered the need to further refine methods for estimating SV maintenance costs...2013Information Technology
The COSMIC Functional Size Measurement Method: An IntroductionH.S. van HeeringenThe COSMIC functional size measurement method is a relatively new and not very well known method to measure the functional size of pieces of software. The main advantages of this method over the existing methods (like IFPUG and NESMA function point analysis) is that COSMIC is designed to also measure other types of software than the traditional administrative software. COSMIC can therefore be used to measure real-time, embedded, infrastructure software, as well as administrative software. Another advantage is the continuous scale that is used in the COSMIC method, which makes it possible to accurately measure the difference in size between different functional processes...2013Information Technology
Estimating Real-Time Software Projects with the COSMIC Functional Size Measurement Method and the ISBSG RepositoryH.S. van HeeringenNowadays, software project estimation of administrative software systems has evolved into quite a mature stage. Standardized functional size measurement methods, like IFPUG and NESMA, are used to measure the functionality that the software application is going to offer to the user and this functional size is considered to be the main cost driver for administrative software projects. These methods are suitable for project estimation, because they measure the functional user requirements (which should be described before the project starts), instead of the actual delivered product (for instance source lines of code, which can only be measured after the project finished)...2013Information Technology
Developing A Business Case for CloudCynthia O'Brien, John BellCommercial organizations and their customers have embraced cloud-based data management solutions with astonishing speed. Government agencies are not far behind, spurred on by the Office of Management and Budget’s “Cloud First” Federal IT policy urging agencies to look to cloud-based solutions whenever possible. Government decision makers are moving to cloud solutions not only because of pressure from OMB and end users; they are also intrigued by the promise of dramatic cost savings. But can a decision to pursue a cloud transition be based on the promise of cost savings alone? There have been numerous examples where transitions to cloud have not yielded the 40% or more expected cost savings...2013Information Technology
ODASA-CE Software Growth ResearchKevin Cincotta, Lauren Nolte, Eric Lofgren, Remmie ArnoldFor several years, the Office of the Deputy Assistant Secretary of the Army for Cost and Economics (ODASA-CE) has used a single growth factor to account for size growth in weapon system software development estimates. This factor is invariant to program characteristics and may, for example, lead to excessive growth estimates for large programs, whereas experience suggests that these grow less in percentage terms than their smaller counterparts. Over the past year, ODASA-CE worked with Technomics, Inc. to research improved methodologies for incorporating growth in software estimates...2013Information Technology
Cloud Computing and Big Data - What's the Big Deal?Arlene MinkiewiczAt one level, cloud computing is just Internet enabled time sharing. Instead of Information Technology (IT) organizations investing in all of the hardware, software and infrastructure necessary to meet their business needs, cloud computing makes access to them available through the Internet. On the one hand, cloud computing allows an organization to adopt a different economic model for meeting IT needs by reducing capital investments and increasing operational investments. On the other hand, cloud computing enables the capture, storage, sharing, analysis and visualization of huge amounts of data in multiple formats from multiple media type: Big Data analysis...2013Information Technology
Can DoD Inflation Indices and Discounting Processes Be Improved?Kathryn Connor, James DrydenCurrently the DoD is facing an uncertain budget environment. This will have an impact on what the DoD can spend for acquisition programs and sustainment of major weapons systems. Current practices for inflation and discounting skew program affordability, especially during operations and sustainment. In this presentation, we look at how well current inflation indices and discount rates serve programs today and whether there are strategies to improve the accuracy of these estimates. After examining the experience of several major weapons systems we have identified potential policy changes and strategies for cost estimators to employ on inflation and discounting. We believe that these can improve a program's understanding of long run affordability and potential risks associated with inflation and discounting.
2013Life Cycle Cost
Lessons Learned from the Joint STARS Analysis of Alternatives for Cost and Risk AnalysesDaniel Mask, David StemThe Joint Capabilities Integration and Development System (JCIDS) plays a key role in identifying the capabilities required by the warfighters to support the National Security, Defense, and Military Strategies. The Analysis of Alternatives (AoA) is a documented evaluation of the performance, operational effectiveness, operational suitability, and estimated costs of alternative systems to meet a capability need that has been identified through the JCIDS process. As weapon system complexities and their interdependencies between other systems increase in the collaborative warfighter space, this evaluation of mission systems and technologies has become more and more challenging...2013Life Cycle Cost
Analysis of Large O&S Proposal: Lessons Learned!James LinickRecently, a $20M contract was awarded for weapon system O&S. The contractor's proposal was subjected to technical evaluation and extensive negotiations ensued. Over 100 BOEs were evaluated for multiple CLINs. Proposal cost components were labor, materials and ODCs. This presentation will focus on how a final agreement was reached, focusing on: 1) Proposal construction, what are CLINs, BOE's and how sub-contractor work was credited, 2)
BOE technical evaluations, the delineations and weighting of on-site and offsite work, 3) Forward price rate agreements, what they are, and the difference between them and final negotiated labor rates, 4) Cost to price calculations, how they are performed and what are Fringe, G&A, Overhead, Fee and Cost of Money and 5) How final contract price and weapon system CONOPS cost estimate compared...
2013Life Cycle Cost
The Dynamic Economic Model: A Flexible Approach to Investment Analysis with AnalyticaKevin King, Jason MehrtensGiven the current economic crises facing the United States, government spending is being placed under more intense scrutiny. As a result, major investments are being more closely analyzed and pressure to be cost effective is of the utmost importance. In the world of cost estimation, the scrutiny is translating to increased demand for cost metrics prior to the establishment of clear scope requirements. For MCR, uncovering new tools and strategies to adapt to this type of environment has become a top priority. One way MCR improved versatility while maintaining our valued reliability has been to create economic cost models using Analytica®, a modeling software designed to enhance the capabilities of the modern spreadsheet tool...2013Life Cycle Cost
The Forgotten Costs and Uncertainty Analysis within the DoD Acquisition EnvironmentRoberto Perez, Elizabeth EvansDespite the best efforts of the Department of Defense (DoD) to capture all costs associated with the life cycle of acquisition programs, it continues to be an elusive. A primary reason for this shortcoming in cost estimating is the lack of time spent on estimating operations & sustainment (O&S) costs compared to the time and effort estimating research, development, and procurement costs. The implications for this imbalance are significant given the fact that 60 - 70 percent of an acquisition program's life cycle cost is comprised of operations and support...2013Life Cycle Cost
Life Cycle Cost Estimate (LCCE) AssessmentsCasey Trail, David Brown, Colleen CraigThis presentation summarizes a scorecard tool and criteria that are used to conduct assessments. The criteria, which are based on the GAO Cost Estimating and Assessment Guide, include four characteristics, which are further broken in to 13 criteria and 75 sub-criteria. Lessons learned, and key findings are presented as examples of LCCE characteristics associated with both positive and negative evaluation outcomes. The presentation is of interest to government and contractor cost estimators, as well as anyone from the cost community wishing to leverage best practices and tools related to life cycle cost assessment. Participants with cost assessment experience are encouraged to share their experiences and lessons learned...
2013Life Cycle Cost
Construction vs. Systems Acquisition Cost Estimating:  A Comparative AnalysisTom Sanders, Steve Essig, Tim AndersonThe continued push to improve DOD's acquisition processes includes several thrusts — regulatory streamlining, improving use of incentives, and workforce optimization. These are not new. To deal with these concerns, the acquisition community has a unique opportunity. DOD Cost estimators have recently received a strong "vote of confidence" in the acquisition community; new positions have been created and existing positions have been protected. That is not necessarily so in the construction industry, another large employer of cost estimators. The construction industry has undergone a transformation over the past six years, beginning with the housing collapse in 2006 and enduring through the years of the Great Recession...2013Management
The Joint Integrated Analysis Tool (JIAT)-  Making Data Sharing EasierNiatika Griffin, Melissa Cyrulik, John McGahanThe silver lining in the current DoD budget turmoil is that it provides us with an opportunity to reexamine how we operate and look for ways to be more efficient. In times of belt tightening, improving access and distribution of established resources is imperative. We are challenged to ultimately improve how we share cost resources. For cost professionals, these resources include databases of cost and technical information, libraries of resources, and approved models that can be used rather than re-invented. The Office of the Deputy Assistant Secretary of the Army - Cost & Economics (ODASA-CE) has invested in developing a tool that facilitates secure access to data needed by the cost analyst...2013Management
Strategic Value of the Business Case Analysis (BCA)Eric BullerThe impact of sequestration has become the main topic of discussion in almost all circles in the Department of Defense since the Budget Control Act passed into law in 2011. In response to reduced budget sequestration, the DoD has been interested in exploring potential cost reductions in weapon system operations. In nearly all lifecycle costs, sustainment generally makes up 60% to 80% of the total program costs. In a defense world seeking increased efficiencies and budget savings it is imperative to examine effective sustainment strategies...2013Management
Back to the Big Easy: Revisiting Hilbert's Problems for Cost EstimatingPeter Braxton, Richard ColemanAt the International Congress of Mathematicians at the Sorbonne in Paris in 1900, German mathematician David Hilbert boldly put forth a list of 23 theretofore unsolved problems in mathematics, which subsequently became quite influential in 20th-century research efforts. At the Joint SCEA/ISPA Conference in New Orleans in June, 2007, the authors audaciously emulated Hilbert with a list of 21 problems for cost estimating and risk analysis...2013Management
Net Present Value (NPV): The Basics and the PitfallsKevin Schutt, Nathan Honsowetz One of the essential metrics of an economic analysis is the Net Present Value (NPV) measure. While it is a fairly simple calculation, made even more routine and efficient by modern technology, be wary of pitfalls in its application. This presentation will cover the basic components and methodology of NPV and discuss the pitfalls and easily-made mistakes in its application. These include incorrect discount rate formulas, inconsistent treatment of inflation, incorrect understanding of the discount rate, incorrect application of Excel's "=NPV()" formula (hint: it's not really "Net"!), and including irrelevant cash flows. Come learn about Net Present Value and avoid these potentially career-limiting mistakes!2013Management
Fifteen Undeniable Truths About Project Cost Estimates, or Why You Need an Independent Cost EstimateTim AndersonAll project cost estimates are wrong, some more so than others. It is exceedingly difficult to accurately estimate the cost of a large project (e.g., major defense acquisition project). Reasons for this are manifold, but the primary causes seem to be general uncertainty, insufficient technical detail, unanticipated design/technical changes, funding and schedule perturbations over the acquisition life cycle, other programmatic "difficulties," and otherwise attempting to predict the future with imperfect information. The end result is that project cost estimates tend to be too low, with early estimates being the most likely to be understated...2013Management
Cost Engineering Heath Check — How Good are your Numbers?Dale Shermon, Mark GilmourHigh quality cost estimating gives a business leader confidence to make rational financial decisions. Whether you are a business leader or a cost estimating manager, you have a vested interest in understanding whether you can depend on your organisations ability to generate accurate cost forecasts and estimates. But how can business leaders be confident that the cost information that they are being provided is of high quality? How can a cost estimating manager be sure that their team is providing high quality cost information?...2013Management
Sailing Blind: Data-Driven Estimating at the Concept Phase, a Case Study of Canada’s Joint Support Ship ProjectTolga Yalkin, Erin BarkelCanada's lack of recent experience in naval shipbuilding and the unique nature of the National Shipbuilding Procurement Strategy (NSPS) present unique estimating challenges. A methodology was needed that was at once data-driven and defendable while at the same time able to address shipbuilding realities at a very high level conceptual stage. This paper outlines a case study where PRICE Systems and the Canadian Parliamentary Budget Officer (CPBO) developed a methodology and model to address these challenges for the Joint Support Ship (JSS) project...2013Management
Supplier Cost/Price AnalysesDavid Eck, Todd BishopUnder United States Federal Government contracts, an important but overlooked aspect is the audit and evaluation of a subcontractor's cost proposal. Requirements in the Federal Acquisition Regulations (FAR) provide for certain visibility into subcontractor's cost proposal depending on the value of the proposal. Additionally it is critical to note the FAR puts the onus of subcontractor cost (or price) analysis on the prime contractor. The expectation is that the same level of detail that the prime includes in its proposal should be included as support to a subcontractor's cost proposal. Effectively the prime must perform subcontractor cost analysis in accordance with the FAR...2013Management
National Reconnaissance Office (NRO) Experiences with the Implementation of Will Cost/Should Cost ManagementSissy Gregg, Greg Lochbaum, Erik BurgessThe objective of this presentation is to address how to assess and incorporate contract performance in an Estimate to Complete, for the purpose of determining a reasonable amount of margin and how that margin should be managed and controlled. The discussion will go beyond cost and schedule variance to forecast total program cost including future requirements, other government costs, etc...2013Management
Avoiding Pitfalls When Applying Learning to your EstimateAndrew BusickMost cost estimators understand the basics of learning curves, but we sometimes fail to recognize the implications of the way that we apply learning.
This presentation points out some major pitfalls and mistakes that can be made when dealing with learning. We address both deriving learning curves from historical data and applying learning to a cost estimate. Additionally, we recommend best practices to use when dealing with these issues...
2013Methods and Models I
Cost Effective Analysis: The Role of Discounting in Government InvestingBrandon ShepelakIn order for the government to best fulfill its obligation to efficiently spend tax dollars, it uses various analysis methods to determine which investment to choose. Cost effectiveness analysis is often the preferred analysis method when an investment is not being judged as to the value added but to determine the least expensive spending alternative. According to the Office of Management and Budget (OMB) in circular A-94, an investment is cost effective when, comparing competing alternatives, its cost are the lowest as expressed in present value terms for a given amount of benefits (OMB, 2011)...2013Methods and Models I
Learning Curve Analysis of Small Data Sets - Spacecraft Bus Cost Improvement AnalysisBrian Welsh, James YorkRecent initiatives to implement cost savings across DoD have produced a number of potential opportunities. Within the Space Portfolio one such opportunity is to acquire satellites more efficiently through block buys and limiting production breaks. To quantify these opportunities, we can analyze historical cost experience. This paper describes three approaches used to pool cost improvement data from multiple programs, allowing for regression of all data simultaneously. The approaches are (1) scaling based on weight, (2) normalizing to a common theoretical first unit, and (3) the use of program binary ("dummy") variables...2013Methods and Models I
Building an Analysis Schedule – Lessons Learned from the SGSS ProgramMatt Blocker, David JacinthoConstructing an integrated cost and schedule uncertainty model involves bringing together important program elements: cost, risk, and schedule. The model's core is the schedule; the linking of costs and risks to the schedule is what allows the implementation of time-dependent costs and, ultimately, an analysis that jointly considers cost and schedule. It follows, then, that one of the first, and often largest, difficulties in constructing an integrated cost and schedule model is the development of the schedule backbone...2013Methods and Models I
A Novel Non-Recurring Production CER MethodologyLisa Hackbath, Raymond CovertProduction costs are generally categorized as either non-recurring or recurring. Typically non-recurring costs include tooling and pre-production activities, among others. Cost estimating relationships (CERs) are generally developed first for recurring hardware costs, and then non-recurring CERs are developed as a function of recurring hardware costs...2013Methods and Models I
Data Collection and Analysis Supporting Defendable Cost EstimatesArlene MinkiewiczCost modeling and estimation has a long and interesting history in the Aerospace and Defense industry starting around the time of World War II. All sorts of mathematical and experiential models have been proposed and used over the years to help with bidding, planning, proposing and executing contracts. While general purpose models are useful, more and more industry and government cost professionals are asking for models built with data very specific to their industry and their organization. Unfortunately many organizations do not have the infrastructure, processes or tools for collecting project data efficiently...2013Methods and Models I
Getting (and sharing!) the FACTS: Factors, Analogies, CERs and Tools/StudiesDaniel Harper, Ruth DorrOne of MITRE's corporate values is "People in partnership." MITRE values […] partnership with the government, collaboration within and without..."
We at MITRE have been charged via our corporate goals to "Apply technical and engineering excellence," by bringing to the customer the best thinking possible by "[…] tapping into a deep technical base, both within MITRE and globally, across the breadth of industry and academia." ...
2013Methods and Models I
Aircraft Operation - Cost Models; Lessons Learned!James LinickIn FY10 the Air Force Cost Analysis Agency (AFCAA) asked one of its cost support contractors to study the processes in place for producing specific aircraft cost products and implement improvements, specifically concentrating on format consistency, maintainability and automation. The periodically produced products to be studied were: 1) Aircraft Operation and Sustainment Cost Estimates 2) Cost per Flying Hour Risk Models 3) Jet Engine Maintenance Procedures versus Time on the Wing...
2013Methods and Models I
Implementing a Data Collection Tool to Estimate and Analyze SpendingThomas Brooks, Daniel MaskAs budgets and resources shrink, the effort to better understand spending has become a priority to many decision makers within the Marine Corps. Since Operation Iraqi Freedom (OIF) began in 2003, military organizations have been able to request and receive immediate Overseas Contingency Operations (OCO) funding without following traditional procurement processes. This readily available funding has led to the emphasis of developing and maintaining operational capability in lieu of program estimates and resource management. The lack of quality budget estimates and use of financial tracking mechanisms has made it difficult for Marines to strategically allocate budget cuts based on efficiencies or priorities; more often than not, budget reductions are distributed uniformly across the organization...2013Methods and Models I
Cost Estimating Tips: Learn Tricks in Excel and Best PracticesEric HongThis presentation for the 2013 International Cost Estimating and Analysis Association (ICEAA) Conference is to present Microsoft Excel tricks and best practices for cost estimating. Many of the presentations at the conference are insightful and show the amazing cost estimating products that are being developed, but the knowledge gained from those presentations usually cannot be applied on a daily basis. The goal for this presentation is to provide the attendees efficient and more effective ways of modeling...2013Methods and Models I
The Dashboard Tool: Taking Cost Analysis to the Next Step By Combining Costs with Capabilities to Evaluate COAsJohn KoAlthough life cycle cost estimates can provide a comprehensive view of the costs for a program, cost estimates do not provide the entire picture needed to make sound business decisions because they only focus on the costs for a program. To determine how programs are actually performing, the system's capabilities must be linked with the program's costs to properly quantify how programs meet their requirements with respect to cost. Booz Allen has developed an Excel Based tool called Dashboard that estimates both the program's costs and the capabilities achieved for a program...2013Methods and Models I
Realizing the True Cost of Energy - Keeping the DoD GreenJohn KoIn recent years, the demand for energy as well as volatility in costs of energy for the Department of Defense has prompted individual programs to monitor these costs closely in order to identify new ways to control these costs. In response to spikes in energy costs in recent years, major programs are now evaluating different alternatives to improve fuel efficiency and ultimately reduce the military's dependence on fuel...2013Methods and Models I
Galaxy Charts: Depict and Color Your WBS in a Meaningful WayRobert Nehring, Katharine Mann, Robert JonesFor centuries, we have searched for new ways to display our thoughts and ideas in ways that will allow the viewer to easily digest and understand our point of view. Displaying quantitative data in interesting yet meaningful ways is no different. In fact, discerning how to display your entire Work Breakdown Structure (WBS) succinctly and clearly has proven to be very difficult. One answer to this challenge is called the Galaxy Chart, which shows both relationships and magnitudes on a single chart...2013Methods and Models II
Computing Fully Burdened Costs of Energy - Fuel (FBCE-F)Walt Cooper, Richard LeeThe Defense Science Board, in studies conducted in the early years of the past decade, concluded that the DoD was not according sufficient emphasis to the consumption of energy in the battle space. Based on those studies, DoD leadership has elevated the importance of seeking opportunities to deliver more capable forces that consume less energy. The Joint Capabilities Integration and Development System now requires an energy Key Performance Parameter. Pilot studies have been conducted to establish a framework for the computation of the fully burdened costs of energy (FBCE) that, by statute, must now be used to inform cost, schedule and performance trade decisions in Analyses of Alternatives (AoAs). Based on those pilot efforts, the Assistant Secretary of Defense (Operational Energy Plans and Programs) has recently published a framework for computing FBCE. A detailed and thorough understanding of this framework is essential for cost analysts supporting AoAs...2013Methods and Models II
Estimating Alternatives for Joint Future Theater Lift (JFTL)Robert GeorgiUS military operations in Afghanistan and Iraq are making military transport aircraft work at rates not foreseen just a decade ago. At the same time the existing inventory of transport aircraft are limited in their ability to transport heavy cargo to austere unimproved landing zones where military forces routinely operate. The US Air Force (USAF) with considerable support from the US Army conducted the Joint Future Theater Lift (JFTL) Technology Study (JTS) to consider how best to equip its theater lift fleets beyond 2020 to address this need. The JTS was established by merging the US Army's Joint Heavy Lift (JHL) and the USAF's Advanced Joint Air Combat Systems (AJACS). The JTS analyzes the cost, risks, and operational effectiveness of theater lift technology alternatives that address capability shortfalls identified in the JFTL Initial Capabilities Document (ICD); an essential capability shortfall being the delivery of a combat-configured medium weight armored vehicle (up to 36 tons) into austere, short, unimproved landing areas without ground handling equipment. One of the alternatives included a hybrid airship design...2013Methods and Models II
Ground Radar Expenditure Phasing AnalysisRick GarciaRealistic budget submissions are critical to a program's success. Budget realism relies on a full understanding of the system's requirements so that a comprehensive cost and schedule estimate can be developed that can inform a realistic budget position. In addition to developing a realistic estimate, the budget position must also reflect a realistic phasing of budget dollars across an appropriate schedule; so that sufficient budget dollars are available at the correct time in a program's acquisition lifecycle to assist the program achieve success. Most existing expenditure phasing models are Rayleigh or Weibull-based distributions and were developed on a variety of DoD space and ground programs and are not commodity specific to terrestrial radar systems...2013Methods and Models II
Improving Cost Estimates in a Medical Acquisition EnvironmentMark Russo, Kimberly WallaceEstimating costs for drug development has proven difficult over the years. Difficulties stem from the overlapping of the Department of Defense (DoD) acquisition process and the Food and Drug Administration's (FDA's) review process, small data pool, industry close hold of cost data, the inherent difficulties in the drug development process, and the lack of using best practices in cost estimating. Over the past couple of years the medical cost community has been working towards improving their cost estimates. These efforts included the establishment of a standard drug development work breakdown structure to ensure that data could be collected and compared in a standard way. A medical cost model was then created in order to use the available data to develop program estimates...2013Methods and Models II
Incorporation of Program Priorities in Funding-Constrained EnvironmentsLeon Halstead, Michael Land, Derrick Kuhn, Bryan ShraderThe recent era of declining Federal (and organizational) budgets has given new impetus to the ability to prioritize funding in an efficient and effective manner. While there are multiple methodologies to pursue these budget reductions, there is a tendency for both allocations and reductions of funding to be top-down, i.e. Organization-driven with a less defensible framework to support Program priorities. This approach heightens the risk that Program goals and objectives will be negatively impacted due to underfunding in this manner due to lessened consideration of Program-specific environmental factors...2013Methods and Models II
Quantifying Uncertainty in Early Lifecycle Cost Estimation (QUELCE)Robert Stoddard, Robert Ferguson, Dennis Goldenson, Jim McCurley, Dave ZubrowEarly lifecycle cost estimation continues to grow in importance within both industry and the U.S. Government. In the commercial arena, shrinking business and product development cycles demand more rapid and early cost estimates. Within the U.S. Defense Department, cost estimates are now mandated prior to Milestone A approval within the DoD Acquisition Lifecycle. One of the primary challenges with early lifecycle cost estimation remains the incongruence of input data required by existing cost estimation models and the data available early in the lifecycle. Recent research into capability-based cost estimation offers improvements but ignores much of what may be viewed as program execution change drivers. These program execution change drivers embody the root causes of changes in program performance leading to changes in the cost of the program. As a result, our team developed the QUELCE method as an innovative approach to elicit domain expert knowledge about the possible future scenarios of change driver behavior for a given program...2013Methods and Models II
Sensitivity Analysis in Cost-Benefits AnalysisSteven IkelerThe Army has been using Cost-Benefits Analysis (C-BA) for all new requirements with cost impact for two years. As with the Analysis of Alternatives (AoA), a sensitivity analysis of the C-BA results is required. The C-BAs frequently use weighted decision matrices to combine the benefits into a single Cost-Benefit Index. This greatly simplifies the sensitivity analysis and facilitates allocation of cost to the individual benefits. The author reviewed the sensitivity analysis in 80 C-BAs. This session will address some of the techniques for presenting sensitivity analysis, the basic math behind the sensitivity analysis and allocating cost to benefits and generalizing the results to more complicated problems.2013Methods and Models II
Probabilistic Technology Investment Ranking SystemPeter FredericThis paper describes the Probabilistic Technology Investment Ranking System (PTIRS), a multi-tool cost estimating system. PTIRS is a new cost estimating tool that has been developed for the NASA Environmentally Responsible Aviation (ERA) Project at Langley Research Center. The ERA Project is charged with developing new technologies for commercial transport aircraft that will reduce fuel consumption, emissions, and noise. The PTIRS tool assesses the integrated, end-to-end benefits of a new technology and allows the benefits to be weighed against the cost of maturing, certifying, and ultimately implementing the technology in a generation-after-next (N+2) subsonic transport system. The tool is both deterministic and probabilistic: it allows inputs with ranges of uncertainty described as statistical distributions and it uses Monte Carlo simulation to produce results with statistically described ranges of uncertainty. The tool produces a two part answer: 1) the life cycle cost impact and noise, emission, and performance benefit of implementing and operating the new technology, and 2) the cost and time required to mature and certify the technology to a readiness level appropriate for full-scale development...2013Methods and Models II
Rapid Generation and Optimisation of Ship Compartment Configuration based on Life Cycle Cost and Operational EffectivenessAidan Depetro, Rhyan HoeyIt has been well established that the majority of Life Cycle Cost (LCC) is incurred during the in-service period. Among other factors, this is strongly linked to the design of the ship and the decisions made during the early design phase. In particular, compartment configuration can have a significant effect on LCC. Poorly considered compartment configuration and hull selection can result in hydrodynamic efficiencies which significantly increase energy consumption and hence fuel costs. Associated space limitations, inadequate or non-existent removal routes and other accessibility problems may result in expensive equipment overhaul and replacement procedures, invasive removal methods, longer maintenance availabilities and increased maintenance costs. Current design methods and decision analysis techniques focus mainly on the trade-off between operational effectiveness and acquisition cost rather than LCC...2013Methods and Models II
Data-Driven Estimating - Quantifying Electronics Complexity Using Public DataF. Gurney ThompsonThe parametric cost estimation industry is being challenged to increase the defendability of their estimates, mainly through comparison to analogous data. However, companies are more protective than ever of their high quality cost data. While this data is often obtainable, it usually comes with strings attached (non-disclosure agreements, etc.) that preclude its usage in defending your estimate. At the same time, government and military databases with cost and technical data are being exposed to the public, and while the noise introduced by granularity issues and the mapping process of this data presents its own set of challenges, this trend also contains the key to a solution...2013Methods and Models II
Affordability Analysis: The Role of Process, Cost and ROI Modeling In Improved Program PerformanceDan GalorathAffordability analysis as part of decision making may be the biggest edge of the decade for both commercial organizations and DoD / government organizations. Affordability has been addressed in the past but never to the level it is today. In an IT context companies struggle to increase profits and often view IT as a necessary evil: one that consumes resources rather than contributes to the bottom line. However, IT can be a significant contributor when IT decisions are made after modeling affordability in multiple dimensions.2013Methods and Models II
Applying Cost Analysis to the DoDAFMichael ButterworthCost analysts in the aerospace community today are being challenged to accurately estimate new and on-going program efforts that will be integrated into the Department of Defense Architecture Framework (DoDAF). The DoDAF provides a foundational framework for developing and representing architecture descriptions that ensure a common denominator for understanding, comparing, and integrating architectures across organizational, Joint, and multinational boundaries. It establishes data element definitions, rules, and relationships and a baseline set of products for consistent development of systems, integrated, or federated architectures...2013Methods and Models II
The Role of Value Engineering In Affordability AnalysisQuentin Redman, Robert Koury, Joseph Bobinis, Paul Tuttle, Kevin Woodward, Hein B.A. de JongThe purpose of this white paper is to describe the role Value Engineering plays within the affordability process. The paper is not a step by step "How To Conduct or Execute" Value Engineering (VE) but is a discussion of the context, input, setup, execution hints, and output of Value Engineering in support of conducting affordability analysis and management. As such it is important to understand the concept of affordability within the Systems Engineering paradigm. This paper is designed to provide insights, lessons learned, and suggestions for using Value Engineering in the affordability process...2013Methods and Models II
Adding Process Tailoring to Product Size for Better Cost EstimationDavid Bloom, Chris FudgeGenerally, when one is looking to create a parametric model to predict costs of an activity, then one looks to size relationships associated with that activity. For example, if one were to develop a parametric model for estimating the development of a Field Programmable Gate Array (FPGA), then, because most of the work to complete an FPGA revolves around software code development, building a parametric model around new code, modified code and reused code (with other size relationships) would probably be a good place to start...2013Parametrics
Evaluating Cost Relationships with Nonparametric StatisticsCaleb FlemingParametric statistics are often used to analyze data, evaluate hypotheses, and determine the significance of a given set of inputs. Most commonly, cost estimators use the following parametric testing measures for significance: Pearson's product-moment correlation, z-tests, t-tests, and Analysis of Variance (ANOVA) tests. While each of these tests is subjectively appropriate to evaluate varying scenarios, each test also objectively depends on the validity of a fixed set of assumptions. Though the assumptions differ by test, all root themselves in the general idea that the data is contained within an adequately large sample and follows a particular probability distribution with estimable parameters...2013Parametrics
Fit, Rather Than Assume, a CER Error DistributionShu-Ping HuAnalysts usually assume a distribution (e.g., normal, log-normal, or triangular) to model the errors of a cost estimating relationship (CER) for cost uncertainly analysis. However, this hypothetical assumption may not be suitable to model the underlying distribution of CER errors. A distribution fitting tool is often used to hypothesize an appropriate distribution for a given set of data. It can also be applied to fit a distribution to...2013Parametrics
Developing a Military Aircraft Cost Estimating Model in KoreaSung Jin Kang, Dong Kyu KimA parametric cost estimating model is very useful for the weapon acquisition process. But it requires validated and normalized historic cost data to develop the Cost Estimating Relationships (CERs). Statisticians seek hundreds of data points to assure a good statistical fit between CERs and the supporting data points comprised of cost, technical, and program information...2013Parametrics
Improving Program Affordability Through The Application of Data AnalyticsDavid Wang, Austin LeeIn the current budgetary environment, program affordability is a key concern. Improving program execution and minimizing schedule delays and cost growth are keys to improving program affordability. However, there is a lack of quantitative analysis of schedule risks, and a lack of understanding of the root causes for schedule delays and cost growth. Consequently, most of the common affordability improvement suggestions centered on relaxation of test and verification requirements, and streamlining of engineering processes may not address the true root causes. The effectiveness of these affordability improvement suggestions is uncertain...2013Parametrics
Portfolio Management: A Parameter ApproachAndy NichollsMy paper outlines one high level method of managing a portfolio of future projects from pre-concept into acquisition phases and examines current practices and tool set requirements at the same time; examines the limitations of the current toolset and the possibilities of inclusion of probabilistic risk calculations via use of individual program risk registers...2013Parametrics
On General Purpose Model CredibilityWendy Lee, Evin StumpIn the aero/defense community and elsewhere, cost estimation of proposed large scale hardware projects by prospective bidders or sponsors is often done using general purpose parametric models, i.e., models based on statistical analysis of a collection of historical data. Of interest in this paper are the parametric models that provide estimates of costs of hardware development and production for a wide variety of hardware, especially hardware that may be used in many situations and environments...2013Parametrics
Benefits of Integrating Schedule and Cost Risk AnalysisRafael HartkeProject managers have the arduous task of running their projects within budget and deadlines, all while facing a wide range of risks like prices and labor costs uncertainty, suppliers performance and reliability, technological challenges, weather, currency exchange rates, etc. Thus, assessing the likelihood and impact of risks is of utmost importance to manage not only the project itself, but also the expectations of the various stakeholders...2013Risk I
Modeling the Risk and Uncertainty of Inflation Rate ProjectionsBrian Flynn, Peter Braxton"Experience in controversies such as these brings out the impossibility of learning anything from facts till they are examined and interpreted by reason; and teaches that the most reckless and treacherous of all theorists is he who professes to let facts and figures speak for themselves, who keeps in the background the part he has played, perhaps unconsciously, in selecting and grouping them, and in suggesting the argument post hoc ergo propter hoc." Alfred Marshall...2013Risk I
Probabilistic Mass Growth UncertaintiesDarren Elliott, Eric PlumerMass has been widely used as a variable input parameter for Cost Estimating Relationships (CER) for space systems. As these space systems progress from early concept studies and drawing boards to the launch pad, their masses tend to grow substantially hence adversely affecting a primary input to most modeling CERs. Modeling and predicting mass uncertainty, based on historical and analogous data, is therefore critical and is an integral part of modeling cost risk. This paper presents the results of an ongoing NASA effort to publish mass growth datasheet for adjusting single-point Technical Baseline Estimates (TBE) of masses of space instruments as well as spacecraft, for both earth orbiting and deep space missions at various stages of a project's lifecycle...2013Risk I
Deciphering JCL: How to use the JCL Scatterplot and IsocurvesEric DrukerAs the use of integrated cost and schedule risk analysis (ICSRA) methodologies rapidly expands across U.S. government programs there has been increasing confusion on how to interpret its seminal result: the Joint Confidence Level scatterplot and its associated isocurve. Over the past few years, the following statements have been made about JCL:...2013Risk I
Understanding the Results of an Integrated Cost/Schedule Risk AnalysisJames Johnson, Darren ElliottThe recent rise of integrated risk analyses methods has created a new opportunity for complex projects to understand the dynamic inter-relationship of cost, schedule, and risk. NASA has been implementing Joint Confidence Level (JCL) analysis of cost, schedule, and risk for all major projects since 2009 in an attempt to proceduralize and codify the requirements for an integrated risk analysis product...2013Risk I
Top-level Schedule Distribution ModelsPeter FredericIn the initial stages of a development project, it is sometimes necessary to build a summary-level schedule for planning and budgeting purposes before the day-by-day details of the project are fully defined or understood. However, when uncertainty assessments are performed on schedule networks containing few activities, the distribution forms chosen for individual activity durations can have a significant impact on the overall results. It is therefore important to choose uncertainty distribution forms that accurately represent the behavior of the sub-network of activities represented by each summary activity...2013Risk I
Analysis Schedules: Danger at a Higher LevelJustin HornbackA government organization charged with evaluating the adequacy of schedule and funding of large projects using project provided schedules and cost estimates receives a detailed (~2000 line) schedule capable of analysis with limited modification or summation. Analysis conducted using detailed schedule revealed interesting results when compared with high level schedule used in previous evaluations of the same project when a detailed analysis was not available...2013Risk I
Cost of Mission Assurance for Space ProgramsErik Burgess, Joe Frisbie, Michelle Jones, Chad KrauseStandards and strategies for assuring successful space missions have evolved in recent years, with some programs moving toward traditional government standards and others moving to industry's commercial standards. This has prompted questions from several government leaders about the cost of mission assurance. Although there are decades of engineering studies, test data, and on-orbit failure data to support detailed evaluation of the technical merits of various approaches, there is very little on the cost side of the equation. The National Reconnaissance Office Cost Analysis Improvement Group (NRO CAIG) has begun to address this by focusing on two of the more costly areas of mission assurance—environmental testing and high-reliability electronic parts...2013Risk I
Base Realignment and Closure (BRAC) Savings and Acquisition RiskPeter Braxton, Kevin Cincotta, Richard LeeThe Government Accountability Office (GAO) recently released the study Military Base Realignments and Closures: Updated Costs and Savings Estimates from BRAC 2005 (GAO-12-709R, June 29, 2012). Its appendices contain a wealth of risk data, with initial estimates (2005 BRAC commission) and final costs (Fiscal year 2011 DOD budget) for 175 distinct BRAC initiatives. Applying an innovative method for modeling within-program risk and uncertainty using cross-program data, this paper derives cost growth factors (CGFs) and coefficients of variation (CVs) for BRAC initiatives. Furthermore, the pattern in these data is astoundingly similar to that found in major defense acquisition program (MDAP) data, a strong confirmation of this modeling approach (meta analysis)...2013Risk II
Correlation Matrices RevisitedSteven IkelerIn a cost risk analysis, risk distributions are sometimes assigned at a low level and then combined, most commonly using Monte Carlo simulation. When using this technique, it is important to use a realistic correlation matrix. It is usually impractical to know all the pairwise correlations. This session will address some of the techniques for completing a credible correlation matrix at differing levels of input information and certainty. Additionally, the session will show examples and review the mathematics behind legitimate correlation matrices. Past sessions have discussed these topics and typically deal with correction techniques for expert-based correlation matrices. This session will be different since it deals less with correcting expert based correlation matrices and more with dealing with uncertainty in the correlations or incomplete correlation matrices. It will also address an extension of the problem to a parametric analysis problem.2013Risk II
Joint Cost Schedule Risk and Uncertainty HandbookDuncan Thomas, John Fitch, Alf Smith, Jeff McDowellThe Naval Center for Cost Analysis (NCCA) has created a Cost Schedule Risk and Uncertainty Handbook (CSRUH) that clearly presents simple, well-defined cost risk and uncertainty analysis processes that are repeatable, defendable and easily understood. The CSRUH is based upon the Air Force Cost Analysis Agency (AFCAA) Cost Risk Uncertainty Handbook published in 2007. It updates key processes to address cost risk and uncertainty methods with an emphasis on providing guidance to capture the impact of schedule uncertainty and the risk register while avoiding "double counting"...2013Risk II
Cost Contingency Analysis using PolytopesBohdan KaluznyContingency is defined as a possibility that must be prepared for—an event that may occur but is not likely or intended. In the realm of cost estimation, contingency refers to a reserve above and beyond a baseline estimate that would be tapped if one or more unexpected events led to higher program costs. Ideally, the selection of a contingency amount should be based on achieving a particular level of confidence derived from cost uncertainty and risk analyses, but often a mere rule-of-thumb percentage (e.g. 15% of the baseline estimate) is applied...2013Risk II
Use of the Risk Driver Method in Monte Carlo Simulation of a Project ScheduleDavid HulettIdentifying the root causes of project schedule and cost risk requires that the risk to the project schedule is clearly and directly driven by identified and quantified risks. In the Risk Driver Method the risks from the Risk Register drive the simulation. (As a side note, we find that the Risk Registers are not complete—during the interviews to collect risk data the interviewees introduce important risks that are, surprisingly, missing from the Risk Register.) The Risk Driver Method differs from older, more traditional approaches in which 3-point (low, most likely and high) estimates of the activity durations are applied directly to activity durations...2013Risk II
Robust Default Correlation for Cost Risk AnalysisChristian SmartCorrelation is an important consideration in cost risk analysis. Exclusion of correlation from cost risk analysis results in the de facto assumption that all risks are independent. The assumption of independence leads to significant underestimation of total risk. However, figuring out the correct correlation values between work breakdown structures elements can be challenging. For instance, it is difficult to estimate the exact correlation value between the structures and thermal protection subsystems in a cost risk estimate...2013Risk II
A Step-Wise Approach to Elicit Triangular DistributionsMarc GreenbergAs the federal government acquires less mature, more advanced and more complex systems, there is an ever-increasing burden on the cost analyst to employ methods of eliciting requirements, schedule and cost uncertainties from one of more subject matter experts (SMEs). Arguably, the most common technique a cost analyst uses today to elicit such data is to ask each SME for the lowest, most likely and highest value which, consequently, produces a triangular distribution...2013Risk II
Innovative Business Agreements and Resulting Cost & Pricing Methods at NASA in Support of New Commercial ProgramsJames Roberts, Torrance LambingOn April 15, 2010 President Obama delivered a speech at Kennedy Space Center in which he outlined his new vision for the U.S. space program. Emphasis was placed on enabling the exploration of Space by Commercial entities instead of by the Government. Since that time, NASA’s role has changed in many instances from being a program manager – overseeing development of space launch hardware and conducting space exploration missions – to one of support and a provider of space-related facilities and infrastructure...2013eTrack
Touch Labor Estimate ModelingMichael Yeager, Lyle DavisTo support its mission for the F-35, the production cost estimating team developed touch labor estimate models which include the flexibility to run production rate effects, loss of learning, commonality adjustments, affordability initiative and outsourcing impacts, multiple learning curve break point analyzes, and estimates for touch labor in hours or by realization. Given the scrutiny to the F-35 Lightning II program by the Department of Defense and Congress detailed and accurate cost modeling allows for better budgeting and more credibility within the services themselves and with the American public...2013eTrack
Methods to Analyze Services Portfolio Cost Drivers and EfficienciesValerie Reinert, Virginia WydlerThis presentation provides a statistical methodology used to evaluate the "health" of a services portfolio through performance data analysis and cost analysis, with the goal to identify cost drivers and efficiencies for performance improvement. Managing a Federal government portfolio is a complex endeavor. Portfolio Managers have a major responsibility to evaluate whether their portfolio is operating efficiently and effectively. Thus, they need to use analytical procedures that, when executed, will evaluate the "health" of the portfolio and identify areas for increased efficiency and effectiveness. If the analysis is not data-driven, this leads to a lack of proper business case analysis and cost benefit analysis to weed out bad actors in the portfolio...2013eTrack
Utilization of Visual Basic in Cost Estimating and Analysis Tools - Anyone Can CookJeremy EdenAs collaborative environments become more prevalent in all industries, the cost estimating and analysis industry is no exception to this movement. Increasing amounts of pressure are put upon cost estimators and analysts to develop tools that are low cost, fast, robust in design, have long standing methodologies, do not require proprietary software or licenses, and are and easy to use by both the advanced cost estimator looking for maximum control and the novice simply trying to diligently support the early stages of a development program...2013eTrack
How to Estimate and Measure "the Cloud" and Make COCOMO Cloud EnabledDavid Seaver, John RosbrughCloud and Big Data have become the new buzzwords in the commercial, federal and DoD technology arenas. This presentation will: 1. Define Cloud & Big Data 2. Identify what's new and different with the Cloud and Big data. 3. Identify the cost drivers for cost estimates...2013eTrack
Got a Defense Business System? The effect of the new defense business system policy on cost estimating, tools, and fundingCharles GuThe Department of Defense (DoD) budgets over $7 billion a year for business system investments and has mandated that all information technology systems must be assessed and certified through a formal process before funding can be obligated in FY 2013. This new law, Section 901 of the Fiscal Year 2012 National Defense Authorization Act (FY2012 NDAA), significantly expands the scope of systems requiring certification to include any business system with a total cost in excess of $1M over the period of the current future-years defense program, regardless of type of funding or whether any development or modernization is planned...2013eTrack
Cloud Computing Starter Kit: Cost and Business Case ConsiderationsJennifer Manring, Raj AgrawalCloud computing is a major trend in the information technology (IT) industry that is affecting both commercial companies and Federal Government agencies due to its promise of agility, scalability, and cost savings. Prior to investing in cloud computing, decision makers are asking "what is my return on investment (ROI)?' However, there is no one size fits all answer to this question and caution should be exercised when applying generic rules of thumb and quick assessment cost and economic models. Costs, benefits, and risks must be carefully assessed for each situation...2013eTrack
From Bid Package to Detailed EVM Baseline in One Easy StepKaren StiffYour Company has put together a cost estimate and basic schedule along with the technical elements as a bid package for a Request for Proposal for a Government contract requiring Earned Value Management (EVM). The proposals have been evaluated, discussions have been held and now it's time for the award. Congratulations your company's won the contract! Now the manager assigned to the program has the task of setting up an EVM baseline for the program. Why is it so hard to do a detailed schedule and EVM baseline, when you've got the schedule and cost that the company bid? Why doesn't this baseline look in any way similar to the bid package cost and schedule? Should it look similar?...2012Earned Value Management Track
A New EVM Performance Index: The MRPIMichael NosbischIt has long been understood that the accuracy of the cost performance index (CPI) is closely tied to how management reserve (MR) has been historically utilized on a specific project/program. However, a formal assessment of MR usage is still not consistently performed in conjunction with CPI calculation and subsequent analysis. This presentation will offer a potential solution to this issue, by proposing a "Management Reserve Performance Index," or "MRPI." Once calculated, the author will demonstrate how the MRPI can be then used to develop a "risk-adjusted IEAC" that provides a more accurate assessment of a project's or program's overall cost performance.2012Earned Value Management Track
EVM for the Rest of UsJavier SloninskyEarned Value Management (EVM) is the practice of evaluating cost and schedule data to measure productivity and progress throughout the lifecyle of a project. EVM, however, often has the stigma of being overly complex and burdensome to implement in a useful way. This session will discuss best practices in implementing a simple yet robust earned value management solution for organizations that benefit from progress measurement and the ability to forecast cost performance, but don't need to comply with ANSI 748, the EVM standard often used by government agencies and contractors. We'll discuss topics including the integration of required data, developing key performance indicators (KPI) and metrics, application of rules of credit, updating of percent complete, and automated EV reporting.2012Earned Value Management Track
Accepted Standards and Emerging Trends in Over Target Baseline (OTB) ContractsSimon DekkerOver Target Baseline (OTB) projects or programs are those that have run significantly over cost and require formal reprogramming - essentially a complete replanning of the project - in order to help the contractor regain management control over the effort. The OTB process has been well documented and become an established part of Earned Value Management practice. Much of the literature-to-date has focused on OTBs from the contractor perspective, including the steps to take in order to propose and implement an OTB, and the proper channels and occasions for engaging the customer in the process...2012Earned Value Management Track
Successful Implementation of an Over Target Baseline/Schedule from a Government PerspectiveJoseph C. Annunziato, Daniel W. DonaldsonIn recent years, Over Target Baseline/Schedule implementations have been accomplished with varying degrees of success. Using the Over Target Baseline (OTB) and Over Target Schedule (OTS) Handbook as a guide, this presentation will address the proper way to implement an Over Target Baseline/Schedule from a Government perspective. The purpose is to provide both government and industry professionals a general understanding of the processes and decisions that must be considered when implementing an OTB and to assist the earned value management (EVM) community in the understanding and implementation of this important management tool. It will also incorporate examples and lessons learned from recent OTB/OTS implementations...2012Earned Value Management Track
Critical Chain Applied to an MRO - Surface Repair FacilityHoward RainerThe project management methodology of Theory of Constraints, called Critical Chain, is examined for application to Scheduling, Planning and Control (SP&C) for a defense-related Maintenance Repair Operation (MRO). The opportunities for Critical Chain, as an extension of Theory of Constraints (TOC), have been reported for well over a decade. With this proven record comes a proposal for similar application in the SP&C at the Surface Repair Facility (SRF) at Cecil Field, Florida. The SRF has established reporting systems such as Earned Value Methodology (EVM) - mentioned here to distinguish the differences between execution and reporting in SP&C.2012Earned Value Management Track
Integrated Project Management (IPM) - Transforming Data into InformationAndrea Mozzo, Bruce KoontzEarned Value Management (EVM) integrates the cost, technical and schedule performance. EVM is not just tracking Schedule Performance Index (SPI) and Cost Performance Index (CPI), but has been transformed into Integrated Project Management (IPM). IPM involves three key areas: Managing cost, schedule, and technical performance within constraints; initiating effective cost, schedule, and performance tradeoffs when constraints are not achievable; and continually evaluating progress to predict and mitigate problems...2012Earned Value Management Track
WBS Development: Rules, Aberrations and MIL-STD-881C Implementation ChallengesMichael MetcalfIssued on October 3, 2011, Military Standard 881C (MIL-STD-881C) provides the framework for developing Work Breakdown Structures (WBSs) for use throughout the acquisition process, as well as a standard set of structures that fit eleven commodity types. These WBSs are used for management, system engineering, and cost estimation by a variety of stakeholders. Though comprehensive, the MIL-STD does not address some of the complex details encountered when developing a weapon system WBS, and there are many special cases and exceptions that could not be captured in the document...2012Earned Value Management Track
"The Answer is 5": Observations on Cost & Schedule in Small Defense ProgramsBrian Fersch, Wesley Tate, Colleen LeonardCost & schedule overruns have been an enduring issue in government acquisitions. Our largest programs are typically dissected and their flaws illuminated with 20/20 hindsight. DoDs smaller programs, though comprising the majority of the budget and quantity of programs, do not get this same attention. Over the course of the past year this study team has collected historical & current cost and schedule data on numerous programs that has been collected at similar points in each programs acquisition lifecycle. The end result is a database that allowed the team to examine how cost estimates and schedule expectations changed from year to year for the same events...2012Earned Value Management Track
Joint Analysis of Cost and Schedule (JACS) - A New Tool for JCL AnalysisAntonio Rippe, Rey CarpioNASA has recently implemented a policy that requires a program/project to be approved at a 70% Joint Budget and Schedule Confidence Level (JCL) which are to be generated from models that integrate cost and schedule. To comply with the policy, a tool framework is needed that can conduct a risk and uncertainty analysis on schedule logic as well as integrate cost uncertainty analysis to account for phasing and costs that are dependent on overall duration. NASA is currently using several platforms, but has seen some issues in extensibility, cost integration, and overall performance...2012Earned Value Management Track
Optimization of Cost and Performance in Complex Systems Using System Dynamics ModelingMichael Polly, Benjamin HarrisAnalysis of trade spaces between cost and performance is critical to win new contract pursuits in today's highly competitive environment. The overall goal in a program is to design and produce a system that maximizes both performance and affordability. In attempting to balance cost and performance, traditional cost models only allow for experimentation with one variable at a time. This can be very cumbersome on large systems, and inhibits how much of the trade space can be explored under time constraints. Altering one variable at a time also limits the number of cost estimating relationships that can be incorporated into the analysis...2012Earned Value Management Track
Software Maintenance Data Collection and Estimating ChallengesVanessa Welker, Peter Braxton, Wilson Rosa, Corinne Wallshein, Joe DeanEntering a period of fiscal austerity, it becomes more important than ever to estimate and consider operating and support (O&S) costs, which represent the lion's share of life cycle cost (LCC) for most platforms, during acquisition. Given the ubiquity of software in today's complex programs, a key component of O&S is software maintenance. This paper presents the results of a research study co-sponsored by the Air Force Cost Analysis Agency (AFCAA) and the Naval Center for Cost Analysis (NCCA) to collect software maintenance data from government support activities and development contractors to enable high-fidelity cost estimates for software maintenance...2012Life Cycle Cost
Estimating Software Maintenance Costs for U.S. Army Weapons SystemsCheryl Jones, James Judy, Brad ClarkThe emerging defense economic environment is characterized by scarce resources, extended system life cycles, and evolving mission capability requirements. As a result, the Army's investment in software maintenance, sustaining engineering, and operational support efforts is coming under increased scrutiny by decision makers at all levels. Of particular interest is the validity and defensibility of the estimates used to determine, allocate, and evaluate the value of software maintenance funds...2012Life Cycle Cost
How VAMOSC VIEWS Can Help You!Brian Welsh, Walt Cooper, Paul Hardin, Elizabeth Koza, Robert NehringSeveral recent developments underscore the importance of identifying and analyzing operating and support (O&S) costs for weapon systems. This brief will present the VAMOSC VIEWS, innovative graphical displays of O&S cost data that serve as an invaluable tool for the cost analyst. In response to current economic conditions, we are entering an environment of reduced budgets within the Department of Defense (DoD). Legislative initiatives, starting with the Weapon Systems Acquisition Reform Act (WSARA) of 2009, are aimed squarely at increasing the visibility and use of O&S costs in weapon systems decision-making...2012Life Cycle Cost
Software Sustainment: Pay Now or Pay LaterArlene MinkiewiczWith today's budget constraints the Department of Defense (DoD) is likely to build less new systems. In light of this, the systems that we have today will need to be stretched further. This means more time and effort will be going into maintenance. And since many systems fielded today rely on software to deliver much of their mission critical capability; this means more time and money will be devoted to software sustainment...2012Life Cycle Cost
Discrepancy Report Prioritization and Software Maintenance ImpactsJennifer Woolley, Kyle ThomasIn the current budget-constrained environment, programs and organizations throughout the Intelligence Community (IC) are searching for new strategies to reduce costs. Rather than sacrificing new capabilities, maintenance is often one of the first areas of reduction. Many organizations perform developed software maintenance based on the submission of Discrepancy Reports (DRs), which are ranked according to their level of severity/importance. One approach to cutting maintenance costs involves only focusing on higher levels of DRs while leaving lower level DRs postponed or unaddressed. This strategy allows for a reduction in personnel because fewer DRs must be completed...2012Life Cycle Cost
Tactical Wheeled Vehicle (TWV) Fuel Economy Improvement Breakeven AnalysisRaymond KleinbergWith the rising costs of and attention towards fuel usage in the Department of Defense (DoD), efforts are being made to procure more fuel efficient vehicles and modernize the Army's existing Tactical Wheeled Vehicle (TWV) fleet. This research is part of an ongoing effort to help program managers make decisions regarding how much investment can be made into new technologies in order to see benefits within the estimated useful life of individual systems and the fleet as a whole.2012Life Cycle Cost
DCARC Contractor Sustainment Cost Data CollectionSandra B. EnserIn the past, Contractor Sustainment (including Contractor Logistics Support (CLS), Contractor Performance Based Logistics (PBL), and other contractor sustainment efforts) were omitted from DoD centralized O&S databases, or were captured as an annual total with little or no supporting detail. For many programs, Contractor Sustainment costs are a major cost driver. They represent an increasing percentage of operating costs. As DoD faces major budget reductions, valid and timely Contractor Sustainment cost data supports analysis which enables Services to use declining operations funds effectively...2012Life Cycle Cost
Designing a Conceptual Framework for Estimation and Analysis of Total Ownership CostF. Gurney Thompson III, Robert KouryIn recent years, the push for greater efficiency and productivity in Defense spending has yielded an increased focus on affordability analysis. Understanding and estimating Total Ownership Costs (TOC) is key in assessing affordability, and the cost community must adapt to support TOC estimation. This paper discusses the development of a conceptual framework for estimating TOC in support of a broader audience, from the acquisition community to program managers and even as a decision support tool for entities such as Congress, DoD Financial / budgetary community, and G-8 Program Analysis & Evaluation...2012Life Cycle Cost
Life Cycle Costs in CBANiatika Griffin, Ronako CarsonCost Benefit Analysis (CBAs) are of growing importance to the acquisition process. The Vice Chief of Staff for the Army signed a memo dated December 20, 2009 stating that, in this era of constrained resources, Army leaders need to be responsible stewards of government funds. Any new requirement that is unfunded and/or newly expanded program proposals will be accompanied by a thorough CBA. The CBA must identify the total cost proposal to the Army in the form of life cycle cost, benefits that will result, and the bill-payers that would be used to pay for the unfunded or new requirement...2012Life Cycle Cost
Preliminary Study of Palm-Oil Biodiesel Life Cycle Cost Analysis in IndonesiaSidhartha Sahirman, Dr. Saparso, Agus Sarjito, Mokhtar Awang, Lr. Shaharin A. SulaimanRecently, Biodiesel have been gaining popularity and getting accepted as an alternative fuel for diesel engines. In the last few years, Indonesian government has a special interest in the development of this renewable energy, the use of which can help reduce oil imports. Biodiesel has a bright future in this country, as Indonesia is the world's largest producer of crude palm oil (CPO) - a desirable feedstock for biodiesel production. Biodiesel has the potential to become a significant industry sector in Indonesia...2012Life Cycle Cost
Evaluating the Life Cycle Cost and Effort of Project Management for Complex Systems Development ProjectsLeone YoungAcross industries, systems development effort has become more costly and complex in nature over the decades, and the costs of project management that are associated with systems development effort have shown an upward growth as well. The estimation of systems development cost and effort has been studied by many scholars and researchers in several different areas such as hardware, software, integration and systems engineering, but the area of project management cost and effort has had very limited exploration, and the literature is void of project management costing methodologies as of today...2012Management
Modeling R&D Budget ProfilesErik BurgessPrior research sponsored by the National Reconnaissance Office (NRO) and published in the Journal of Parametrics (Summer 2006) has become the principal method for developing space and ground segment budget profiles for NRO programs. Since then, the practice of budgeting to independent cost estimates has been signed into law for the intelligence community. The resulting scrutiny and reliance on budget-phasing models has motivated several improvements that should be relevant to the estimating process for other DoD commodities. The presentation first reviews the mathematical formulation of expenditure profiles that are based on historical data, and how some of the resulting accuracy metrics have been useful in defending annual budgets. New models for space and ground systems are presented. Since the annual budget authority required to support an expenditure stream is heavily dependent on a program's execution rates (i.e., outlay rates), we also present a new approach for estimating what those outlay rates will be. This is a new area of responsibility for most cost estimating organizations, yet its impact on annual budget requests can be significant.2012Management
IT Service-Based Costing: Standardizing Provisioning and Servicing IT ResourcesEmily Jessen, Paul BrownIn today's current economic environment, there is a strong focus on constraining spending of appropriated funds to meet the mission particularly within the Intelligence Community and Department of Defense (DoD). Part of the DIA's mission includes the delivery of IT products and services. In the past, IT services have been provided to non-Core customers for free, but in the new fiscal environment, the DIA does not have sufficient resources to continue to operate in this manner. In compliance with the Economy Act of 1932 and DoD Financial Management Regulations, the DIA has been taking steps to quantify the IT services it deploys in order to recover costs of those services...2012Management
Trying To Do Too Much with Too Little: How Poor Portfolio Management Can Lead to Schedule Delays and Cost OverrunsChristian SmartGovernment organizations often try to accomplish too much with the resources with which they are provided. The genesis of new projects in government organizations often begins with the search to spend idle funds. An organization's leadership may discover that it has a relatively small amount of money that can be used to research a new technology or to begin initial development of a new project. New ideas to improve systems, develop better, newer ones, or to advance scientific understanding abound. In order to sell leadership, project management provides leadership with success-oriented, optimistic projections...2012Management
Case Study of Army Cost Management: Managing Variability in Analytical Cost Products for an ACAT ID ProgramTomeka WilliamsArmy Cost Management requires an understanding of the level of commitment the Department of the Army has made to cost management that goes beyond the Program Management Office (PMO) through, TRADOC, and HQDA levels. By requiring any program that is a new start, or major upgrade of an existing program to submit a Cost-Benefit Analysis based on guidance from the VCSA, is one of the unique cost management products the Army has instituted. This is now the first step to securing a viable and executable acquisition program within the Department of the Army...2012Management
How does a $20B Program Become a $30B Program?  Lessons Learned in Production Cost ManagementSteve Sheamer, Allen GaudelliHow does a $100M ship become a $200M ship or a $75M fighter jet turn into a $125M fighter jet? As we have seen recently with several high profile acquisition programs, systems integrators and cost estimators continue to struggle when estimating the production costs of development systems. This presentation focuses on production-related cost drivers and why contract incentive-fee structures are often part of the problem, then details steps for better management of these production costs...2012Management
NRO Program Assessments - Best Practices and Lessons LearnedGreg Lochbaum, Linda Williams, Lisa Keller, Ken OdomThe National Reconnaissance Office (NRO) performs program assessments for a variety of purposes: the management of acquisition programs; portfolio analysis for resource allocation; and accountability to NRO's oversight organizations - Execution to Oversight. The NRO Cost Analysis Improvement Group (CAIG) recently stood up a Program Assessment group to continuously improve the NRO's program assessment process to become more efficient, consistent and standardized. This presentation provides the NRO's approach to each level of the Execution to Oversight pyramid...2012Management
Capabilities Based Portfolio Analysis (CBPA)Paul Gvoth, Brad EllisTotal cost is no longer the singular driver of defense acquisition program economic decision making. Defense programs and projects must be evaluated according to their contributions to overall monetary and non-monetary Return on Investment (ROI) relative to the capabilities of a Family of Systems (FoS), System of Systems (SoS) or as a part of a portfolio of products that together meet documented operational requirements. This is needed because future defense strategies, including Smart Defence are being driven by budget cuts...2012Management
Using the Tools of Persuasion to "Sell" Your EstimateJennifer KirchhofferWhat is the difference between a good cost estimator and a great cost estimator? Both have robust estimating skills and knowledge, but only great cost estimators are able to garner acceptance and usage of their estimates from even their harshest critics. The ability to "sell" the estimate is key to the utilization of the estimate in the proper context by the decision makers. Our jobs often entail delivering news that our customers and leaders are not anxious to hear. Many factors work against us as we try to garner support and acceptance of the estimates...2012Management
A Case Study in Broad System Analysis: DoD Spectrum Reallocation Feasibility Study, 1755-1850 MHzBrian Wilkerson, Guenever AldrichWith the increasing demand placed on the electromagnetic spectrum by the proliferation of web enabled cell phones, the President has begun an initiative to sell 500 MHz of dedicated bandwidth from the federal government to the wireless industry. The 1755-1850 MHz band was selected as the first band under this initiative for a study of the cost and technical feasibility of relocating DoD assets that use these frequencies. The study was incredibly broad as all services were involved, and the systems affected include Precision Guided Munitions, Point to Point Microwave, High Resolution Video, Multiple Radio Systems, Electronic Warfare, Air Combat Training, Telemetry, Satellite, and Small Unmanned Aerial Systems...2012Management
Cloud Computing: Federal Mandates and the DoDHeather Nayhouse, Eric LumsdenThe Federal CIO has laid out a plan to implement cloud computing in all federal agencies and consolidate 800 data centers by 2015. The Office of Management and Budget has required agencies to default to cloud computing solutions wherever those solutions are secure, reliable, and cost-effective. In response, the Department of Defense has launched a number of pilot programs as well as its own strategic plan to foster their deployment of cloud computing systems. Several case studies of cloud solutions within the Department of Defense are already available. The DoD is analyzing these cases for guidance into a larger push into cloud computing systems...2012Management
Building a DHS Cost Estimating & Analysis Center of ExcellenceKatie Geier, Colleen Craig, David Brown, Kevin CincottaThe Department of Homeland Security (DHS) Cost Estimating & Analysis (CE&A) Center of Excellence (COE) serves as the Department's primary advocate and resource for cost estimating and analysis. Formed in October 2011, the CE&A COE's primary goal is to take a proactive, streamlined approach to improving the Department's resources and skills in cost estimating and analysis. This presentation highlights the roles and responsibilities of CE&A COE and summarizes the work accomplished to date and planned future efforts for accomplishing the following objectives:...2012Management
Social Media ApplicationsMarc Wear, Michelle Ehlinger, David Pearson, Sara WiseThis paper explores the impact of adopting social media tools in the work environment as a means to improve communication, collaboration, and productivity for cost and earned value analysts. Public and private entities have taken steps to adopt social media platforms to benefit their organization; this paper demonstrates how cost analysts can implement these tools to improve methods and processes...2012Management
A Holistic Approach to Multiyear ProcurementsAnn Hawpe, Travis WinsteadMulti-year contracts have the potential to save the Government a substantial amount of money if they are executed on stable programs in an environment with stable requirements and funding. With the current economic pressure for defense programs to do more with less, multi-year contracts are a good option for programs that meet these criteria. The largest amount of savings is the result of procuring a larger quantity of parts than would be procured with a year-over-year contract structure, thus realizing economies of scale. Savings are also captured when the labor hours to develop and submit year-over-year proposals are factored into the calculations...2012Management
Using Treasury Securities to Develop Inflation IndicesC. Tyler Cunningham, Joseph ParisiThis paper examines inflation assumptions in Department of Defense cost estimates. Professional guidance allows and encourages the development of custom inflation indices, but there are few resources detailing the methodologies to develop those indices. We address that need by laying out detailed steps to develop defensible, custom, risk-adjusted inflation indices using techniques that leverage analysis of the spread and volatility between traditional US Treasury Securities and Treasury Inflation-Protected Securities. 2012Management
Job Satisfaction: The Link to Retention and the Correlation to Age, Gender and Organizational PositionDarrin L. DeReusIn an attempt to balance resource availability and workload, leaders in the United States Air Force have attempted to manage the budgets by fluctuating manpower levels. There is minimal research of multiple affiliations (active duty military, government civilians and contractors) and the effects of manpower reductions on the organization. This study collected data on career anchors and satisfaction levels to find the similarities and differences of multiple affiliations in the United States Air Force. The results of this study showed that there were correlations between career anchors and satisfaction scores...2012Management
Building an Agile, Collaborative Environment for Capturing Productivity Based Cost Model DataDavid Bloom, Wanda Grant, Mason WexlerUnderstanding and analyzing productivity within an Aerospace electronics development organization requires continuous oversight and adjustment. For example, in digital electronics, a corollary to Moore's Law says that the capability of digital processor components doubles every 18 months. This would imply that the cost to design a set of digital electronics products to deliver a specific state-of-the-art capability would change (become less) as a function of the ability to incorporate more features on any one component...2012Methods and Models I
Galaxy Charts: The 1,000-Light-Year View of the DataRobert Nehring, Katharine Mann, Robert JonesThe old adage "A picture is worth a thousand words" is one that we have all heard and often try to use to our advantage. This paper applies the principles of visual display of information as advocated by Edward Tufte and others to develop an innovative graphic that will prove invaluable to cost estimators and consumers of their estimates everywhere...2012Methods and Models I
One NASA Cost Engineering (ONCE)Eric Plumer, Mike BlandfordThis presentation is the first roll out of NASA's ONCE (One NASA Cost Engineering) database to the SCEA/ISPA community. Concept and early prototyping of the One NASA Cost Engineering (ONCE) database began several years ago and it has been developed to fully automate the CADRe data in a GUI database allowing for easy search and retrieval of data for the cost community. By automating the retrieval of CADRe data this tool is greatly improving the space cost community's ability quickly acquire the data needed to improve the quality and credibility of these cost estimates...2012Methods and Models I
Defense Cost & Resource Center (DCARC) Executive Overview BriefMike AugustusThe OSD stood up an independent office to manage the CCDR (Contractor Cost Data Report) requirement in the late 1990s. Today, that office is known as the Defense Cost and Resource Center (DCARC), which is part of the Office of the Secretary of Defense (OSD) Cost Assessment and Program Evaluation (CAPE). The original charter has been expanded to accommodate the collection of not only cost data but software metrics data as well, from industry on ACAT 1 Major Defense Acquisition Programs and Major Automated Information Systems (MAIS) programs. Today the Cost and Software Data Report (CSDR) requirement spans over 140 MDAP ACAT 1C/1D programs and a large number of MAIS 1AM and 1AC Programs...2012Methods and Models I
An Approach to Improving Cost Estimating and Budget Integration in Federal ProgramsMichael Noonan, Kristin Jackman, Eric HongOver the past few years, political standoffs in Washington, D.C. have delayed or prevented passage of Federal Appropriations Bills and significantly reduced Federal spending. As Agency budgets continue to shrink, programs must increasingly compete with each other for a slice of dwindling resources. Program and Departmental managers responsible for resource decisions are demanding more detailed and agile resource justifications which can accommodate multiple budget scenarios. In response, program offices require tools that efficiently translate estimated program costs into flexible and defensible budgets...2012Methods and Models I
Enhancing Excel-Based Cost Models with PivotTable ReportingBlaze Smallwood, Omar MahmoudA major challenge facing DoD cost estimators is developing Program Life Cycle Cost Estimates (PLCCEs) that serve as more than just an acquisition reporting "check-in-the-box" document. The ideal is to simultaneously create a dynamic, flexible tool that the program office can use to support their day-to-day costing needs, from answering a wide-range of cost-related data calls to integrating PLCCE outputs into their Program, Planning, Budgeting, and Execution (PPBE) processes. For DoD cost estimators that utilize Microsoft (MS) Excel to create their PLCCE models, a valuable tool that can be leveraged to achieve these goals is Excel's PivotTable reporting capability...2012Methods and Models I
A Canadian F-35A Joint Strike Fighter Cost Estimation ModelBohdan L. KaluznyThe F-35 Joint Strike Fighter (JSF) is a single-engine, stealthy (radar-evading), supersonic multi-role fighter. Canada intends to purchase 65 F-35A-conventional takeoff and landing variant (CTOL) - jets for delivery between 2016 and 2022. In order to secure production, Canada will likely have to commit to procurement as early as 2012. Until recently, Canada relied on the United States (U.S.) JSF Program Office (JPO) for projected costs. In early 2011, the Department of National Defence (DND) F-35A cost estimate came under public scrutiny as a result of a Parliamentary Budget Officer report (Canada's equivalent to the U.S...2012Methods and Models I
Developing Standardized Cost Element Structures for the United States Marine CorpsJeremy EdenThis presentation explains the development of a Cost Element Structure (CES) for use in United States Marine Corps (USMC) Logistics Requirements Funding Summaries (LRFS). Industry and the Assistant Commander for Life Cycle Logistics formed an Integrated Product Team (IPT) to create a new CES. The IPT comprised logisticians and cost estimators who worked together over the course of one year to create the new USMC LRFS CES...2012Methods and Models I
United States Marine Corps Logistics Requirements Funding Summary (LRFS) Cost Estimating Tool (CET) - A Quick Cost Estimator for Logisticians Part IICharles GuSimilar to a Life Cycle Cost Estimate (LCCE), the Logistics Requirements Funding Summary (LRFS) captures Integrated Logistics Support (ILS) related costs for a program throughout its life cycle. The LRFS provides visibility into the logistics requirements of a program which is needed during Program Objectives Memorandum (POM) and budget submissions. Additionally, it helps to plan and quantify requirements, identify and defend funding, and serves as the ILS input to the LCCE...2012Methods and Models I
Modeling Potential Cost Savings by Synchronizing Commercial Derivative Acquisition & Lifecycle ProgramsBrad BoehmkeSynergy is defined as "the combined or cooperative action of two or more stimuli for an enhanced effect" (Razzetti, 2009). As Department of Defense (DoD) system acquisition costs continue to grow and commercial derivative programs becoming more accepted, it is critical that the Air Force look for ways to create cost savings in every possible way. With commercial derivative programs becoming more popular in defense acquisition, there may be acquisition programs running concurrently that have system commonalities and could benefit from synchronizing acquisition efforts. By leveraging commonalities in requirements, the Air Force and the Department of Defense may be able to "obtain cost savings through acquisition and logistics planning" (Poulin, 2010)...2012Methods and Models I
Zero Based Review MethodologyMartha Wells, Peter MeszarosThe days of incrementally funding activities based on past performance are over at DIA. Increased budget pressure throughout the federal government has created an environment for improvements to the way activities are funded. DIA is approaching this increasingly challenging budget environment with a new programming methodology - Zero Based Reviews. The Zero Based Review (ZBR) process incorporates programmatic analysis and activity-based costing to approach budgeting from a right sizing methodology. Rather than reviewing the enterprise based on current structure, the ZBR takes a topical approach to program review to determine if the right resources are being used in the right way for the right mission...2012Methods and Models I
Cost Benefit Analysis OverviewDarrell HamiltonThere are many different requirements to do some type of cost and benefit tradeoff analysis as part of the acquisition process, IT investment decisions, performance based logistics decisions and budget change proposals. Most of them have nuance definition differences and are known by their acronyms as CBA, EA, BCA, AoA, COA and several other terms. Every one of those analysis requirements is aimed at demonstrating or helping others to decide on what is the most reasonable and cost effective decision that needs to be made...2012Methods and Models I
Accuracy Matters: Selecting a Lot-Based Cost Improvement CurveShu-Ping Hu, Alfred SmithThere are two commonly used cost improvement curve (CIC) theories: unit cost (UC) theory and cumulative average cost (CAC) theory. Ideally, analysts develop the CIC by analyzing unit cost data. However, it is common that instead of unit costs, analysts must develop the CIC from lot cost data. An essential step in this process is to estimate the theoretical lot midpoints (LMP) for each lot, to proceed with the curve-fitting process. LMP is generally associated with UC theory, where the midpoint is always within the lot. The more general lot plot point (LPP) term is used in the context of both the UC and CAC theories...2012Methods and Models I
Will-Cost and Should-Cost Management: It's Not Business As UsualZachary JasnoffIn April , 2011 Under Secretary of Defense for Acquisition, Technology & Logistics Ashton Carter issued the Memorandum: "Implementation of Will-Cost and Should-Cost Management". The memo defines implementation of Should-Cost and Will management for all ACAT I, II and III programs and lists "Selected Ingredients of Should Cost Management". Thus, each organization involved with these programs must successfully deal with the challenges or planning, coordinating and managing Should Cost/Will cost programs and have the necessary tools to quantitatively manage them through their life cycle...2012Methods and Models II
DoD Contracts Database and Interactive ToolBrian OcteauProposal Summary Good cost estimates, analyses and assessments should consider all sources of data available. One of those sources - contract data - is often overlooked, and sometimes misused. For many, the collection, compilation, and interpretation of contract data can be an unwieldy and time consuming venture. However, once compiled, the data can be an invaluable resource for cost analysts, providing insight relative to cost, schedule, and technical growth, highlighting data completeness gaps inherent in other data sources, and revealing data trends and/or patterns...2012Methods and Models II
Rapid Business Case Development Using Macro-Based Excel ToolAndrew HutchinsonPurpose: To present a tool that creates business cases based on user-entered customizations such as the start and end year of the investment. With the increasing scrutiny of federal government budgets the development of business cases for future investments will take on added significance. A best practice within industry is the development of business cases to analyze the net present value of potential investment opportunities. The question this analysis answers is how much value will potential initiatives create within your organization and allows for an "apples to apples" comparison of multiple investment opportunities...2012Methods and Models II
An Intuitive Application of Cost Risk Analysis to a LRFSBlake BoswellThis presentation explains the implementation of an Uncertainty Modeling Capability (UMC) into a USMC LRFS. Similar to Life Cycle Cost Estimate (LCCE), the Logistics Requirements Funding Summary (LRFS) captures Integrated Logistics Support (ILS) related costs for a program throughout its life cycle. The LRFS provides visibility of logistics requirements for Program Objectives Memorandum and budget submissions. Additionally, it helps to plan and quantify requirements, identify and defend funding, and serve as the ILS input to the LCCE...2012Methods and Models II
Creating a Framework for Cost Analysis: The Cost Analysis Support Tool (CAST)Eric LumsdenThere are many redundancies encountered when creating new cost models, and many operations that can be automated. CAST is a tool created by TASC that provides a framework to accelerate the process of building a cost model. Estimators are able to execute processes at the click of a button that would otherwise be very time consuming to create from scratch or complete by rote. They are able to transfer over work from other projects to eliminate redundant operations...2012Methods and Models II
Identifying the Cost Capabilities of the DoDAF Architecture FrameworkHolly A. H. Handley, Resit Unal, Andreas TolkThe Department of Defense Architecture Framework (DoDAF) is a key tool used by the United States engineering and acquisition communities to describe the overall structure for designing, developing, and implementing systems. In 2009, version 2.0 of DoDAF was released. This version represented a major shift in focus from output products, or documentation, to the collection of the underlying data that represents the architecture. This version describes a new data model, the DoDAF Meta Model (DM2); the DM2 provides the mechanism needed to collect, organize, and store data. While many pre-defined models reside in the DoDAF, none of them are considered cost models of the system...2012Methods and Models II
EDGARS CB - A New Memory Tool for CostersWilliam BarfieldAre you befuddled by what it takes to do a cost estimate? What kind of information is used and when is it needed so that a defendable cost and benefits estimate can be provided to the investment decision authority? Since beginners and professionals alike rely on tricks-of-the-trade to help get their job done, a new and easy memory aid is presented...2012Methods and Models II
Significant Reasons to Eschew Log-Log OLS Regression when Deriving Estimating RelationshipsStephen A. BookLog-Log Ordinary Least Squares (LLOLS) regression, considered in the 18th and early 19th Centuries as the best (and, in fact, the only) method for fitting nonlinear algebraic relationships of the form y = axb to data sets of (x,y) pairs, has a number of serious defects that make it far from adequate for CER development in the 21st. No other option was available 200 years ago, but the advances in computing power and techniques of statistical optimization available to us today leave no reason to stick with an obsolete method. In the 21st Century, we insist that 21st Century engineering technologies be applied, so why would we continue to develop CERs to estimate them using 18th Century statistical methods?...2012Methods and Models II
Valuation in Cost Estimating: Taking a Page from the Investment Banker's PlaybookKevin SchuttValuation is "the process of estimating what something is worth". It is commonly undertaken in investment banking to support activities such as initial public offerings of company stock and mergers & acquisitions. The need for valuation analysis arose in a cost estimation setting in order to satisfy Office of Management and Budget budgetary treatment criteria, yet valuation techniques in general may be a useful addition to the cost estimator's toolkit, facilitating price analysis and cross-checks of cost estimates. Likewise, cost estimation techniques may inform valuation analysis. While cost estimation generally results in market-independent estimates, valuation provides the viewpoint of what the market will bear. This presentation will provide an overview of four major valuation techniques: Price multiple; Replacement cost; Comparable transaction, and Discounted Cash Flow. A case study in valuation will be presented. 2012Methods and Models II
Cost Estimating Training for Non-Cost EstimatorsJeremy EdenThe effort to improve the accuracy and fidelity of cost estimates is critical given budget restraints. As this effort continues, non-cost estimators (program managers, engineers, logisticians, etc.) are more frequently facing the demand of preliminary cost estimates prior to the development of formal cost estimates. These preliminary estimates are done in order to streamline the cost estimating process and guard against risk for a program in development. While most non-cost estimators have the information and tools available to address early program cost estimating needs, many do not have the skills and experience necessary to make good use of the resources at their disposal...2012Methods and Models II
Just-In-Time Cost AnalysisJohn KoBooz Allen developed and maintains a robust Life Cycle Cost Estimate (LCCE) for the acquisition of the Urgent Universal Need Joint Mine Resistant Ambush Protected (MRAP) vehicle program, the Defense Secretary's top acquisition priority. Building and maintaining the LCCE was challenging because the MRAP Program is unique from other urgent needs Acquisition Category I-D programs in two ways. First, to accommodate the aggressive and accelerated schedule, the MRAP Program conducted an open competition, which required a review of several designs from various contractors in a short period...2012Methods and Models II
Estimating Alternatives for Joint Future Theater Lift (JFTL)Robert Georgi, Bruce PurselUS military operations in Afghanistan and Iraq are making military transport aircraft work at rates not foreseen just a decade ago. At the same time the existing inventory of transport aircraft are limited in their ability to transport heavy cargo to austere unimproved landing zones where military forces routinely operate. The US Air Force (USAF) with considerable support from the US Army conducted the Joint Future Theater Lift (JFTL) Technology Study (JTS) to consider how best to equip its theater lift fleets beyond 2020 to address this need...2012Methods and Models II
Exploring Methods of Conflating Data from Various Data SourcesAshley MosesWhen creating a cost estimate, it is not out of the realm of possibilities that one might be required to combine data from multiple sources to get an accurate estimate. Coleman, Braxton, Druker, et al. have presented research on which methods might be the best to combine opinions given by subject matter experts (SMEs). Additionally, they have also discussed what adjustments are needed to make the estimates provided by SMEs more accurate as studies suggest that SMEs will tend to underestimate the range of possible data values when appropriate feedback is not given to them...2012Methods and Models II
Volatility and Cost EstimatingJennifer LeottaThis paper will examine the properties and uses of implied volatility, stochastic volatility, and historic realized volatility. Further discussion will focus on what applications an assessment of market volatility brings to the field of cost estimating through the application of a volatility range derived for fuel prices at varying intervals over the course of a generic program's life cycle.2012Methods and Models II
Estimating Relationship Development Spreadsheet and Unit-as-an-Independent Variable RegressionsRaymond P. Covert, Noah L. WrightMCR has constructed an estimating relationship (ER) development spreadsheet based on the zero percent bias, minimum percent error (ZMPE) regression technique to help with more credible and efficient development of cost improvement curves and cost estimating relationships (CERs). The CER-development method accommodates linear, power and triad functions with single and multiple technical and dummy independent variables. Furthermore, the ER development spreadsheet may be modified to accommodate other functional forms that may be of value in particular contexts...2012Parametrics
Are Parametric Techniques Relevant for Agile Development Projects?Arlene MinkiewiczAgile software development practices are predicated on the following tenets as introduced in 2001 in the Agile Manifesto [1] - Individuals and interactions over processes and tools - Working software over comprehensive documentation - Customer collaboration over contract negotiation - Responding to change over following a plan2012Parametrics
Comparing Bottom-Up and Top-Down Estimating Approaches in a Custom Cost ModelDonald L. Trapp, Noah L. Wright, Lisa HackbarthDeveloping a cost model involves several steps such as data collection, normalization, cost estimating relationship (CER) development, model building and documentation. Choosing an appropriate level of detail for each CER is an important consideration. This is influenced by several factors such as user needs, data availability as well as the statistical behavior of the data. The cost modeler must sometimes choose to estimate costs for particular work breakdown structure (WBS) elements with either a bottom-up or top-down approach...2012Parametrics
Error Analysis of a Custom Cost ModelRaymond P. Covert, Donald L. Trapp, Noah WrightHow do you know how well a custom cost model performs? If it is based on historical data, then the simplest performance measurement is the model's percent standard error (PSE). The PSE measures a historically-derived model's ability to re-predict its underlying database. In this presentation we will demonstrate an analytic error analysis of a custom cost model using historically-derived cost estimating relationships (CERs) and factors. We will provide a sample model and show how we calculated the PSE of each of the CERs from the data as well as developing the error calculations for each step in the model. We will show how we used the concepts of effective correlation coefficients, propagation of errors, and the calculated variances of the sums and differences of two random variables.2012Parametrics
The Function Point Based Pricing Model: The Price is Right!Daniel B FrenchMany IT organizations face a difficult challenge managing contractual engagements with vendors for application development and maintenance services. Regardless of the pricing model or acquisition strategy, projects come in late and over-budget more often than not. Be it a time and materials, cost plus, fixed price or other contractual arrangement, both the vendor and client face the challenge of providing the functionality desired, minimizing risk, and ensuring that the vendor is able to generate a reasonable profit from the engagement...2012Parametrics
Tactical Missile Bluebook Cost Model DevelopmentDonald L. Trapp, Noah L. Wright, Lisa HackbarthAs part of a quality review process, MCR constructed a cost model based on the cost estimating relationships (CERs) and cost factors contained in the Tactical Missile Bluebook. This effort proved to be valuable in refining the existing CERs and factors and in improving the Bluebook. While building the model, the CER developers found instances where CER input definitions in the documentation needed to be more specific to better serve CER users. When comparing candidate CERs to each other in the regression development process, CER developers relied on error statistics and goodness-of-fit metrics, but these metrics provided an incomplete comparison...2012Parametrics
Tactical Missile Bluebook and Cost Model OverviewRaymond P. Covert, Donald Trapp, Noah Wright, Andrew DrennonMCR recently developed the Tactical Missile Bluebook together with a cost model based on the information contained in it. The Bluebook contains cost estimating relationships (CERs) for tactical missile development and production based on 13 programs and 153 recurring production lots. In this presentation, we show how the Bluebook is organized and how the CERs and factors were developed, including the filtering methods used to determine data points to be included and excluded from CER development, the challenges of building a useful estimating method from this compilation of data, the methods used to develop the estimating relationships, the results obtained, and significant general findings...2012Parametrics
Understanding and Measuring the Impact of Design and Systems Engineering Decisions on AffordabilityZachary JasnoffMethodologies for understanding and measuring the impact of design and systems engineering decisions on affordability In today's data driven cost estimating environment, it is critical to understand the impact of design and systems engineering decisions on cost. It is also important to leverage actual cost data in developing data-driven Cost Estimating Relationships (CERs) that relate engineering design and performance parameters to cost. In addition, the ability to conduct sensitivity analysis on not only the CER, but on the entire system estimated is needed to understand the full impact on Measures of Effectiveness and Measures of Performance...2012Parametrics
Introducing Reliability and Availability Requirements into TOC ModelsEvin J. Stump, Wendy W. LeeTwo important system requirements are reliability and availability. Reliability is the probability of no disabling failures over a certain span of time, while availability is the ratio of the time the system will actually fulfill its operational expectations, to the total time it is expected to fulfill them. Both of these are often included in system specifications...2012Parametrics
Applying Parametric Cost Models as a Predictive Parameter for CMMI ComplianceDavid Bloom, Wanda Grant, Robert Wright, Gary BosworthThe Electronic Engineering Center (EEC) of Raytheon Space and Airborne Systems (SAS) recently breezed through a Capability Maturity Model Integration (CMMI) Level 5 assessment, jumping from an existing CMMI level 3 capability and without needing a separate CMMI Level 4 assessment in the process. Although this success was the result of a lot of hard work by many contributors, the bottom line is that there were very few changes in terms of business processes that were required to obtain a CMMI Level 5 organizational assessment, the hallmark of a learning and optimizing organization...2012Parametrics
Using Method of Moments in Schedule Risk AnalysisRaymond P. CovertA program schedule is a critical tool of program management. Program schedules based on discrete estimates of time lack the necessary information to provide a robust determination of schedule uncertainty and therefore the risk that the proposed schedule will be completed on time. To determine the risk that a proposed, discrete schedule will meet a particular schedule and to find the probable critical paths (i.e., the criticality index), a probabilistic schedule risk analysis (SRA) is performed. SRA is a process by which probability density functions (PDFs), usually triangular, are defined at the task level in an effort to quantify the uncertainty of each task element...2012Risk
Fitting Absolute Distributions to Limited DataBlake BoswellThe choice of probability distributions is a critical component for cost risk and uncertainty modeling. When data is available, distribution fitting techniques, such as Goodness of Fit (GoF) tests and Information Criteria (IC), can be applied to determine distributions that accurately describe potential cost realizations; however, with limited data GoF tests and IC based methods provide little or no insight into the best distribution choice. Therefore, when data is limited it is standard practice in cost risk and uncertainty modeling to solicit expert opinion in the construction of triangular distributions with vertices representing the best case, typical, and worst case scenarios...2012Risk
A Systematic Approach for Empirical Sensitivity Analysis on Monte Carlo ModelsMatt PitlykA key component of risk analysis is the sensitivity analysis performed on the input variables for Monte Carlo models with the goal of determining those variables that cause the most variation in the final distribution and identify the best candidates for risk mitigation plans. While the standard technique of calculating the correlation coefficient between the final distribution and each input distribution is appropriate for linear models, it is not sufficient to accurately identify the largest uncertainty drivers. In the case of the non-linear models it is neither appropriate nor accurate...2012Risk
SAR Data Analysis, CV Benchmarks, and the Updated NCCA S-Curve ToolRichard Lee, Peter Braxton, Kevin Cincotta, Brian Flynn, Ben BreauxTo support the development of better probabilistic cost estimates, the Naval Center for Cost Analysis (NCCA) has championed the development of the S-Curve Tool, which was well received at both the 44th Annual Department of Defense Cost Analysis Symposium (ADoDCAS) in February, 2011 (1), and the joint Society of Cost Estimating and Analysis (SCEA) / International Society of Parametric Analysts (ISPA) conference in June, 2011. This paper presents ongoing research to support both continued improvement of the S-Curve Tool and greater understanding of the nature of cost growth for major acquisition programs; its mean value (risk) and variability (uncertainty); and the components thereof...2012Risk
Enhanced Scenario-Based Method for Cost Risk Analysis: Theory, Application, and ImplementationBryan Flynn, Peter Braxton, Paul Garvey, Richard LeeThere is a growing realization within the cost-analysis community that estimates of cumulative probability distributions of cost, or S-curves, too often understate true, underlying risk and uncertainty. Several organizations cite cases where return program acquisition costs, or actuals, fall at the 99th+ percentile on S-curves estimated years previously. This degree of deviation from the mean is a legitimate possibility for any one acquisition program. After all, there's no such thing as an "average" program. Variation is expected...2012Risk
Real Data, Real UncertaintyAlfred Smith, Jeff McDowell, Lew Fichter, Wilson RosaCentral to any cost risk analysis and model are the uncertainty distributions assigned to point estimates. Ideally, the analyst will have a database of historical cost and technical information that can be used to objectively develop Cost Estimating Relationships (CERs) using approved statistical methods. From that analysis the analyst will be able to objectively define the shape and dispersion of the CER uncertainty distributions. Similarly, with a suitable database or expert opinions to draw upon, the analyst will be able to develop uncertainty distributions for the inputs to the CERs...2012Risk
Diagnosing the Top Level Coefficient of Variation: An Alternative ApproachDaniel J. AndelinThe coefficient of variation (CV), defined as the standard deviation divided by the mean, is a useful metric to quantify the uncertainty inherent in a probability distribution, as it provides a relative (and thereby more intuitive) measure of spread than the variance or standard deviation. In the field of cost analysis, the CV can be used as a diagnostic to determine the level of risk and uncertainty built into an estimate and whether those levels fall within the expected range for a program of a given scope and complexity and at a given stage in its life-cycle...2012Risk
Inflation Cost Risk Analysis to Reduce Risks in BudgetingMichael DeCarlo, Eric DrukerFor any project there is a danger of unanticipated cost growth because inflation rates are extremely difficult to estimate. This presents a significant challenge to estimators. Predicting future inflation rates with some precision is possible, however, when the appropriate analysis is implemented. Even with previous recommendations from a major federal and commercial consulting firm, there is evidence indicating that the government has not been making adequate assessments of inflation rates for future budgeting. Without applying proper attention and techniques to the analysis and prediction of inflation rates, budgets run a higher than necessary risk for increased cost growth due to inflation prediction error...2012Risk
Applying the Pareto Principle to Distribution Assignment in Cost Risk and Uncertainty AnalysisJames R. Glenn, Christian Smart, Hetal Patel, Lawrence JohnsonSignificant effort and statistical knowledge is required in order to complete an accurate uncertainty analysis of a cost estimate. First, a representative sampling of data points must be collected in order to model appropriate distributions for each random variable in the estimate. Additionally, correlation between the random variables in the estimate must be properly assessed and applied. Finally, to generate the cumulative distribution of total costs (i.e., S-Curve) either the joint cumulative distribution function must be computed or an approximation technique such as Monte-Carlo simulation must be used...2012Risk
The Unseen: Statistical Inference with Limited DataTrevor VanAttaObjective measurements of probability are often unavailable, and most significant choices under risk require an intuitive evaluation of probability.' -Nobel Laureates Daniel Kahneman and Amos Tversky. What are the odds of rolling a sum total of seven when tossing two dice? What is the probability of red turning up after a spin of a European roulette wheel? Most analysts, given a little time and a calculator, could answer these two questions with exact precision. For both of these questions, there is only one true correct answer. Such is the nature of probability analysis for questions that are decompositional (all possible outcomes can be determined), frequentistic (the experiment can be repeated an infinite number of times), and algorithmic (the results can be measured with numbers)...2012Risk
Utilizing Optimization Technique to Enhance Cost and Schedule Risk AnalysisColin Smith, Brandon HerzogAdvancements in Monte Carlo simulations have enabled cost and schedule uncertainty analysis to be conducted on large, complex programs which could never be analyzed using conventional methods. Analysts are now able to predict with increasing confidence the budgetary requirements, schedule forecast, and risk mitigation measures on projects at all points during their lifecycle. These advancements in Monte Carlo technologies and methodologies have further lowered the barrier to entry for conducting uncertainty analysis; have reduced simulation times to mere fractions of a second; and have allowed for a comprehensive, integrated cost, schedule and risk analysis...2012Risk
Is My Schedule Ready for Risk Analysis?Mario FountanoThe ultimate goal of schedule risk analysis is the ability to provide project stakeholders with a risk mitigation plan that allows the team to make informed decisions concerning future cost and schedule performance. However, in order to perform schedule risk analysis, a project schedule must first be valid, and provide a clear path to project completion. This study presents the steps our team takes in order to evaluate the readiness of project schedules prior to integrating a risk analysis. Often, project schedules contain flaws in duration and logic, are left un-statused, or were never developed to capture the true nature of project work. The assessment process our team follows helps to build more confidence and transparency into project work. 2012Risk
Joining Effort and Duration in a Probabilistic Method for Predicting Software Cost and ScheduleMichael A. RossThis paper describes a data-driven method for estimating the cost and schedule of software development projects. This method correlates the estimates of cost and schedule such that constraining (perhaps reducing) the cost will impact the estimated schedule and constraining (perhaps compressing) the schedule will impact the estimated cost. This method provides these estimates of cost and schedule that are probabilistic (i.e., provide a range of possible outcomes with associated probabilities of attainment); a capability that is essential to analyzing the impact that affordability and budget constraints have on program cost and schedule and their associated risks...2012Risk
A Closed-Form Solution for the Production-Break Retrograde MethodDarrell Hamilton, Brian GillespieThis article explores and discusses concepts surrounding the multi-step retrograde analysis process for learning curve production breaks that was popularized by George Anderlohr, in his 1969 Industrial Engineering article "What Production Breaks Cost". Mr. Anderlohr based much of his analysis using the cumulative average curve method, but the basic principles have been widely accepted and used to calculate the equivalent calculation using the unit theory learning curves...2012Software/Hardware
Estimation of Expedited Systems Engineering SchedulesBarry Boehm, Dan Ingold, JoAnn LaneA major objective of many organizations is to reduce the calendar time needed for a project to perform its systems engineering (SE) functions, without compromising the resulting product's needed functionality and quality attributes. This paper will present the derivation of a model to estimate the necessary calendar time to perform the project's SE functions as a function of its product drivers (size, domain familiarity), process drivers (maturity, streamlining), project drivers (team cohesion, collaboration technology support), people drivers (knowledge, skills, and agility), and risk drivers (vs. relative needs for speed, quality, and scalability)...2012Software/Hardware
COTS Estimating Metrics for Increased Cost AccuracyJoshua Patapow, Eric TiminskiObserved cost analysis issue was that multiple Commercial Off-The-Shelf (COTS) Hardware/Software cost estimates were significantly higher than recent contract award values. Using data from those contract awards, an analytical approach and metrics were developed to ensure increased accuracy on future estimates. For COTS Hardware (HW) and Software (SW), a metric of the actual contract price as a percentage of the mean online price/list price (Part I) was developed. Additionally, an estimating relationship was developed (Part II) to calculate the cost of annual maintenance support as a percentage of the actual HW/SW initial cost and the annual maintenance support actual contract price as a percentage of a vendor quote...2012Software/Hardware
A Standard Process for Software Code CountingBetsy Legg, Brian EnserSoftware Lines of Code or SLOC has long been one of the critical inputs for measuring the effort and time involved with software development projects. SLOC is by no means the only way to measure the size of a project but it is the most mature measure and is well understood by decision makers. Software cost estimating that relies on SLOC as an input depends on accurate and consistent counts. The problem is that no standard has been universally accepted for defining SLOC. Numerous code counting tools in use throughout the industry rely on interpretations of the definition of a line of code...2012Software/Hardware
Utilization of Visual Basic in Cost Estimating ToolsJeremy EdenAs collaborative computing environments become more prevalent in all industries, the cost estimating industry is no exception to this movement. Increasing amounts of pressure are put upon cost estimators to develop tools that are robust in design, have long standing methodologies, do not require proprietary software or licenses, and are and easy to use by both the advanced cost estimator looking for maximum control and the novice simply trying to diligently support the early stages of a development program...2012Software/Hardware
Cloud Computing - Changing the Way We 'Do' SoftwareArlene MinkiewiczIn 1961 at the MIT Centennial, John McCarthy opined "if computers of the kind I have advocated become the computers of the future, then computing may someday be organized as a public utility just as the telephone system is a public utility&. The computer utility could become the basis of a new and important industry."[1] In 2006, Amazon Web Services was launched providing computing on a utility basis. Since that time the notion of cloud computing has been emerging and evolving...2012Software/Hardware
Software Cost Estimating for Iterative and Incremental Development ProgramsBob Hunt, Jon Kilgore, Jennifer SwartzIterative and Incremental Software Development processes are at the heart of a cyclic software development process. Iterative and Incremental Software Development processes evolved in response to the weaknesses of the traditional waterfall model. Iterative and Incremental Software Development start with an initial planning and ends with deployment with the cyclic interactions in between. The goal is to deliverer what the user needs at the end of the process, not only what was envisioned at the beginning of the process. Iterative and incremental development is an essential part of the Rational Unified Process, Extreme Programming, and generally the various agile software development frameworks...2012Software/Hardware
Estimating for Lifecycle and Product Line AffordabilityJoAnn Lane, Barry Boehm, Ray Madachy, Supannika KoolmanojwongA significant challenge in systems engineering and acquisition is to justify investments in new systems and the evolution of existing systems and product lines. In an era of shrinking budgets, especially in the U.S. Department of Defense, choices are driven by affordability, and not just the affordability of the initial development. Affordability includes development, deployment, usage, maintenance, and retirement/disposal costs. A way to assess or estimate system or product line affordability is through Total Ownership Cost (TOC) analysis...2012Software/Hardware
Learning Curve Analysis with EZQuant - An OverviewMichael MahoneySure you learned how to calculate learning curves by hand while studying for the CCEA exam, but what will you use to calculate them from day to day? This session will provide an overview of the freely available DCAA EZQuant regression analysis shareware. EZQuant takes minutes to learn and is easy to use. Unit and Cumulative Average learning curve calculations and projections are supported. Since DCAA recognizes the software you don't spend time validating the tool. Natural logarithms are fascinating but EZQuant is just fast.2012Software/Hardware
The Importance of Software Cost Estimating Standards among a Diverse CommunityKyle Thomas, Jenny Woolley, William BlackMany cost analysis organizations within elements of the United States Intelligence Community (IC) have developed cost estimation and data collection methods that are based on years of Agency-specific historical programmatic data. These methods have been established over time, effectively creating independent approaches across the Community. While these techniques are valuable, the cost estimation and reconciliation process highlights differences in estimation methods that make analysis across the IC difficult...2012Software/Hardware
The EVP Certification: A Bridge Between SCEA and AACEMichael NosbischIn the 1990s, SCEA and AACE International, formerly known as the Association for the Advancement of Cost Engineering, not only were both members of the now defunct Joint Cost Management Association, but they also maintained a cooperative agreement that intended for members of the two associations to "benefit from increased knowledge of the activities and services available from the other organization." While that document agreement unfortunately did not extend beyond 1995, there has recently been a joint effort to renew a similar type of agreement...2012Informational Sessions
A Structured Approach to CCEA® CertificationMichael Mahoney, Rick Beavers (Presenter)The structured approach to SCEA certification details the methods used by Lockheed Martin Chelmsford to prepare employees for the CCEA examination. In 2010, three of us set out to earn certification at the CCEA level and all three of us passed on the first attempt. The session will introduce a generic schedule for studying the CEBoK modules including the time allotted for each, an approach to taking sample tests, and lessons learned. We'll discuss what worked and what didn't and share tips and tricks.2012Informational Sessions
Touchpoints between the SCEA Cost Estimating Body of Knowledge (CEBoK®) and the Program Management Institute (PMI) Body of Knowledge (PMBoK)Leon HalsteadThe General Accounting Office (GAO), in its publication of the "GAO Cost Estimating and Assessment Guide" in May of 2009, focuses extensively on the importance of Cost Estimation and its contribution to overall project success. While this publication offers specific technical examples of cost estimation and analysis capability, e.g. pre-Program approval cost estimates/baselines, Affordability Analysis, et al, there is less focus on how these technical examples "fit" into the larger Project Management context...2012Informational Sessions
Targeting Affordability and Controlling Cost Growth through Should-Cost AnalysisAnthony DeMarcoOn September 14th, 2010, The Honorable Ashton B. Carter; Under Secretary of Defense for Acquisition, Technology and Logistics, released a memorandum addressed to the acquisition professionals of the Department of Defense. The primary thrust of the memorandum was the current need for greater efficiency and productivity in defense spending. Secretary Carter provided guidance organized into five initiatives...2011Applications
An Application of Data Mining Algorithms for Shipbuilding Cost EstimationBohdan L. KaluznyThe North Atlantic Treaty Organization (NATO) Research and Technology Organization (RTO) Systems Analysis and Studies (SAS) 076 Panel (NATO Independent Cost Estimating and its Role in Capability Portfolio Analysis) is a working panel generating independent cost estimates for NATO systems with the aim of standardizing how NATO countries conduct cost estimation. One of the systems analyzed by the SAS-076 Panel in an ex post exercise was Her Netherlands Majesty's Ship (HNLMS) Rotterdam Landing Platform Dock (LPD), an amphibious transport dock ship that was launched in 1997...2011Applications
Selection of Data Source for Systems Contractor Labor Rates and Overheads and Their ApplicationBrian Wilkerson, Wallace RigginsSelection of the best data source can reduce the uncertainty in system contractor labor rates and overheads, facilitate improved government program budgeting, and produce savings for the government in contract negotiations. This presentation compares and contrasts the use of historical contract data vs. Forward Pricing Rate Recommendations (FPRRs) in determining contractor labor rates and overheads in the preparation of Independent Government Cost Estimates (IGCEs). The advantages and disadvantages of each approach are discussed. The development of forward pricing rates beginning in the proposal stage, through DCMA recommendation and DCAA audit to a bilaterally signed agreement is covered...2011Applications
Rolling On The Affordability River (While Managing The Acquisition Program In The Rapids)Christopher SvehlakFrom inception through execution, every acquisition program in the Department of Defense is now expected, even mandated, to incorporate aspects of affordability. This paper explains the role and importance of affordability; then, taking an empirical approach, it presents scenarios depicting actual events based on the author’s observations and experience within the context of two separate program offices. Affordability, cost estimating, and financial management processes are described and detailed within the scenarios. Problem root causes and areas of strength are briefly discussed, and some topical research is offered as supporting and explanatory material throughout. Finally, affordability guidelines, relevant public law, current Department of Defense Instructions, and senior leader perspectives are presented, reinforcing the importance of affordability in acquisition programs. 2011Applications
Commercialization Activities at NASA and Resulting Cost ImplicationsJames Roberts, Torrance LambingDue to mandates of the current Obama Administration, much of the space launch activities that have traditionally been led and contracted out by NASA are being turned over to the Commercial Sector. NASA’s role is changing in many instances from being a program manager - overseeing development of space launch hardware and conducting space exploration missions - to that of a supplier and manager of facilities and infrastructure to support the space development and launch activities of commercial entities...2011Applications
Reducing Maintenance Costs Using Beyond Economic Repair AnalysisJerry Le MayA Beyond Economic Repair (BER) analysis compares the cost of repairing a product with the cost of replacement giving a company information to help decide if repairing a product is more economical than replacement. Using information from a BER analysis, repair procedures can be written so that once a pre-determined amount of time has been spent on repair without success, a product can then be replaced, spending the additional time on replacement rather than further attempts at repair. A BER analysis starts out as a prediction using anticipated repair costs for a new product to establish the amount of hours spent to attempt repair before stopping repair work and replacing the product...2011Applications
Cost by Capability: Funding the Right Mission Capabilities in a Cost Constrained EnvironmentJohn ScardinoIn an environment of increasing demands on constrained resources, it has become imperative that Department of Defense (DoD) organizations be able to correctly identify the cost and proper allocation of resources across units to provide a sustained level of mission readiness. As operational units train and deploy across the globe, it is important to provide decision makers with the processes that will enable, implement, measure, and manage cost-wise readiness objectively...2011Applications
Earned ReadinessJohn Williams, John ScardinoThe Department of Defense (DoD) faces complex challenges in today’s globally strategic and economic environment. During these times of increasing demands and constrained and limited resources, it is imperative that organizations be able to correctly identify the proper alignment and allocation of resources across operational units. The ability to sustain readiness and performance levels over time at lower cost is critical. At multiple levels across the enterprise, leadership must be able to determine solidified readiness and financial goals, as well as have the ability to quickly analyze the readiness and financial performance achieved in meeting those goals...2011Applications
INCOSE Affordability Work Group – Design for AffordabilityEdwin B. Dean, Joseph BobinisThe Affordability Working Group of The International Council on Systems Engineering (INCOSE) is striving to understand and promulgate affordability within the systems engineering community. The group began at the 2010 INCOSE International Conference last summer, meets monthly by telephone, and will meet for two days at the end of January 2011 at the annual INCOSE Workshop. In the interest of intersociety sharing, this paper will share the perspective and status of the Affordability Working Group meeting at the INCOSE Workshop.2011Applications
Use of JCL Data for Programmatic SuccessRey CarpioWith surging interest in Joint Confidence Level (JCL) and corresponding expectations levied on the program management communities, what is a PM to do with JCL data and information? This presentation puts emphasis on the value and use of data and information that JCL tools and processes bring to the table. Appreciating the pedigree of the inputs used in JCL development, and the rigor of the JCL process, the presentation will cover the multiple JCL products and the interpretation of the data and information. The presentation will also cover the effective way to communicate the JCL results to the program managers and the stakeholders...2011Applications
Budgeting to the MeanRick Garcia, Casey WallaceDecision makers (and policy) often require cost estimators and analysts to move to a higher percentage on the S-Curve to ensure enough budget is requested so that a program does not overrun its budget target. Although the request to budget at a higher confidence level is a pragmatic attempt to avoid overruns (under budgeting), there are other factors to consider besides simply “moving to the right” on an S-Curve; such as ensuring a cost estimate captures all relevant uncertainty and acknowledging acquisition changes that will impact a program after a budget has been set...2011Applications
Evolved Expendable Launch Vehicle (EELV) Discrete Event Simulation: Ensuring the Buck Results in a BangColleen Craig, Scott DeNegreAn accurate cost estimate is an essential prerequisite when considering expansion or modification of a major government system. To make an informed decision, cost must be considered in light of the expected benefit of the expenditure, to ensure the investment is justified. We describe one method of performing this analysis, using the EELV supply chain as a demonstrative example. We demonstrate the methodology of Discrete Event Simulation (DES), and its application in the cost community, as a tool to measure the current performance of and potential improvement in the EELV system...2011Applications
Multiply or Divide? A Best Practice for Factor AnalysisShu-Ping Hu, Alfred SmithFactors are commonly used in engineering build-up equations to derive cost estimates. For example, software development hours are often estimated based on an estimate of source lines of code (SLOC) divided by a productivity rate: SLOC per hour. To develop the productivity rate, the analyst collects software size (SLOC) and development time (hours) metrics from a variety of programs. Ideally, these programs have similar characteristics to the one being estimated, for instance, similar complexity, language, resources (people and tools) and environment...2011Applications
Interconnected Estimating Relationships: Their Derivation and ApplicationStephen Book, Amanda FeatherBy the term “Interconnected Estimating Relationships”, we mean estimating relationships for hardware and software costs, schedules, weights, and below-the-line programmatic costs that are jointly impacted by each other or by the same drivers. There are two common examples of this phenomenon: (1) cost-estimating relationships and schedule-estimating relationships, which are interconnected for many reasons, but primarily because a projects schedule is a significant driver of its cost; and (2) cost-estimating relationships for hardware and software and techniques for estimating below-the-line costs, the latter of which is typically done by applying a factor or percentage to the hardware and/or software cost estimate...2011Applications
Using the New 881 WBS/CES for ERP Acquisition: Lessons LearnedVirginia Stouffer, Gerry BelcherMIL HDBK 881's work breakdown structure/cost estimating structure (WBS/CES) is under revision, and one of the key new WBSs is an automated information systems WBS for enterprise resource planning (ERP). We used the draft 881 to create the program office estimate for the US Coast Guard's CG-LIMS system. Even with help from key committee members in defining the categories, we encountered some difficulties in assigning ERP costs to their proper categories, in accounting for all the ERP costs, and interpreting some of the newer subelements...2011Applications
Determining Cost Estimating Relationships for Nine FAA Solution Development ElementsWilliam BarfieldThe Federal Aviation Administration is responsible for management of our National Airspace System, which requires massive amounts of software development and maintenance. The writing and testing of large-scale software is expensive and involves many substantial costs in addition to the development of the basic software itself. In an effort to improve financial management practices within the FAA, new Cost Estimating Relationships (CERs) were determined for nine FAA Work Breakdown Structure elements pertaining to software development and delivery life cycles activities. CERs are regression equations are typically based on normalized actual costs of prior analogous software development...2011Applications
Standardizing the Cost Technical Baseline - Template and TutorialDiane Butler, Jason NewmanOver the last four years, the Department of Defense (DoD) cost community has experienced changes in cost policies, regulations and implementation procedures. Decision makers want more insight into cost estimate development, improvements in estimate quality, and a better understanding of program changes that affect funding levels. As managers of cost estimators, it is our responsibility to provide guidance on how these changes impact our analysts in the program offices...2011Applications
A Methodology for Multivariate Regression on Large DatasetsMatthew PitlykLarge datasets can present a formidable challenge to the analyst. Proper analysis of a long list of potential cost drivers can require enough regression computations and scatter plots to make discovering good relationships difficult. A methodical approach helps to keep the analysis organized and ensures that nothing is overlooked. This presentation will put forth a standardized way to approach analyzing independent variables in regression and an application of automating this approach in VBA...2011Applications
Life Cycle Cost Growth Study - 20 Science Mission Directorate (SMD) MissionsClaude FreanerPrevious studies have concentrated on examining development cost growth, excluding launch vehicle, mission operations and data analysis. This study looks at cost growth from a Life Cycle Cost perspective, including launch vehicle and mission operations and data analysis. Costs are separated into major WBS elements to see if any elements are more likely to have cost growth than others. The different WBS elements are tracked over time with snapshots of costs at major milestones to see where in the project life cycle the cost growth occurs. 2011Management
How Cost Arises - How We Can Reduce CostEdwin B. DeanThis presentation is based upon nine years of research for NASA on how cost arises and how we can reduce cost. A summary of this research was first published as the 500+ page NASA Design for Competitive Advantage web site (1994-1998).
This paper will illustrate how cost arises, suggest various means of reducing cost, and provide resources for those who desire to further this research.
Constructing a Price-to-WinFrank R. FlettWinning a competitive federal procurement requires more than just writing a good technical proposal. A win strategy that has been formulated without consideration of the competition is bound to fail. A thorough competitive assessment includes the identification of the customer’s Most Important Requirements (MIRs), the assessment of our own and our competitors’ strengths and weaknesses versus those MIRs, the investigation of our competitors’ current market position and behavior on recent competitive procurements, and the postulation of the competitors’ strategy options, probable strategy, and price for the opportunity being bid. Only then can we determine our own win strategy and our Price-to-Win (PTW)...2011Management
EELV Should Cost Review Overview and Lessons LearnedJames Smirnoff, Karen Schaben, Joe KabeisemanIn his September 14, 2010 “Memorandum for Acquisition Professionals”, Ashton Carter, the Undersecretary of Defense for Acquisition Technology and Logistics, directed that “the manager of each major program conduct a Should Cost analysis justifying each element of program cost...” In early March of 2010, as a response to significant pending contract price increases, supplier readiness uncertainty, and unresolved Defense Contract Audit Agency (DCAA) audit issues, Michael Donley, the Department of Defense (DoD) Executive Agent for Space directed a Should Cost Review (SCR) of the Evolved Expendable Launch Vehicle (EELV) program...2011Management
What to Know When Estimating Virtualized Environment CostsJennifer Woolley, Ryan Boulais, Sandra WilliamsVirtualization, which allows a single piece of hardware to function as multiple “virtual” pieces of hardware, is one of the principal trends for current IT systems. As the technology has matured over the last several years, various organizations in both the public and private sectors have made the transition to operating in a virtual environment in an effort to reduce costs and streamline data access. With server, storage, and network virtualization becoming more prevalent, cost analysts are frequently asked to estimate and assess the costs associated with virtualization...2011Management
Building a Cost Analysis Improvement Group - Best Practices and Lessons LearnedKeith Robertson, Linda Williams, Erik Burgess, Jay JordanThe NRO Cost Analysis Improvement Group (CAIG) has been in existence in various forms for over 20 years. It has migrated from an organization that provides independent cost estimates (ICE) for specific product lines to one that provides end to end program acquisition support...2011Management
Lessons Learned: A Case Study in Labor Cost Estimating in a Data-Scarce EnvironmentKevin SchuttObservations and lessons learned from labor rate and utilization development in a data-poor environment are discussed. A case study of an independent cost estimate for a large federal services contract forms the basis of the examples, which include a method for labor rate build-up. The method combines forward pricing rates and GSA schedules, and a mapping of labor categories from multiple disparate sources to those prescribed in the Request for Proposal (RFP)...2011Management
How to Estimate and Use Management Reserve in an EVM SystemMark InfantiWhen using an Earned Value Management System (EVMS), a Management Reserve (MR) is both expected and a ‘best practice’. But the misunderstanding of how to establish it, what it is for and how to use it, extends from government agencies to contractors.
MR is defined as “An amount of the total budget withheld for management control purposes, rather than being designated for the accomplishment of a specific task or set of tasks” (ANSI/EIA 748-B)
Enhanced Cost Analysis in Support of Aerospace Corporation’s Decision Support Framework (DSF)Mel Broder, Lubo Jocic, Inki MinSince its inception fifty years ago, the Aerospace Corporation has been supporting government decision making in many programs and at different levels. An objective technical analysis employing subject matter experts using many tools, models, and methodologies is the backbone of Aerospace’s Decision Support Framework. Strong customer demand for timely analysis and advances in modeling capabilities have enabled introduction of fast, concurrent, engineering processes into decision support. The Concept Design Center (CDC) and Concurrent Program Definition Environment (CPDE) engines were built to enhance timeliness of design and program definition processes...2011Management
You Really Don’t Have to Lose a Million Dollars a Year, a Cost/Price Analytical Journey through the World of WinemakingMichael ThompsonIt’s been said that the best way to lose a million dollars a year is to open a winery, the question that as cost analysts is “why?”
By conducting interviews with several small winery owners I found that their business model is more often than not reliant on the pricing structure of large wineries in their region, rather than determining how much they can afford in direct and indirect costs.
Cost Analysis Process in Manufacturing IndustryMostain Dara BillahIn the common conflict of interest, there exists between daily operations/manufacturing, sales/marketing, and cost and finance. These conflicts can contend to be as healthy; however it cannot be avoided in a manufacturing environment. Even though these facts may be right, but for last 35 years I have experienced a different opinion that I would like to share with you all, today...2011Management
Best Practices in Aerospace Cost Estimation: Observations from US Air Force and NASARobert Georgi, Benjamin WatsonBoth the United States Air Force and the National Aeronautics and Space Administration (NASA) develop aerospace projects with cutting edge technology. This paper will provide an overview of cost estimation for the respective organizations based on the authors experience with NASA and Air Force projects. When developing aerospace projects applying cost estimation in a formal manner in organizational structure and personnel training can improve program estimates and risk management...2011Management
Integrating Earned Value Analysis and Independent Cost Estimating for Large, Multiyear System ProcurementsGreg Lochbaum, Cole Kupec, Eric Mosier, Chris MasseyThe NRO Cost Analysis Improvement Group has integrated the Earned Value Analysis Center of Excellence (ECE) with the Independent Cost Estimating (ICE) group. The intent of this consolidation is to continue to improve the estimating and earned value products that are provided to Agency decision makers. Both the ECE and ICE groups are finding the integration mutually beneficial. The NRO CAIG often uses earned value (EV) information to help develop and validate its high level conclusions reached in independent cost estimates (ICEs). This approach has, at times, led analysts astray...2011Management
Mathematical Lessons Learned from a Year’s Worth of ICEsRyan W. Boulais, Brett DickeyHaving conducted over 20 Independent Cost Estimates (ICEs) across the Intelligence Community (IC) over the past year, there were a few common mathematical concerns that needed addressing in multiple estimates. These concerns appeared time and again from a cost method and application point of view. Three such topics will be discussed in this paper: calculating and applying correlation, calculating a cost factor from an average “rate”, and properly implementing distributions in Monte Carlo simulations such that the “Mean” point estimate and the “Mean” output from the simulation are the same...2011Management
PARS II: Redefining Program Oversight & Assessment at the Department of EnergySimon DekkerRequiring a new method for capturing actionable project performance data from its contractors, the Department of Energy recently implemented its Project Assessment and Reporting System (PARS II), built on Dekker PMIS™ technology, and revamped its project reporting procedures to better align project performance with strategic Departmental goals. PARS II provides a web-based system that captures accurate, timely, complete, and verifiable project performance data from contractor sites around the country. The system fosters greater transparency and provides project oversight staff a single, online, and secure vantage point from which to monitor up-to-date performance data generated directly from the project sites Earned Value Management systems...2011Management
NASA Implementation of JCL PolicyJames Johnson, Charles HuntFor the past two years NASA has been implementing Joint Cost and Schedule Confidence Level (JCL) analysis within its projects. This analysis is sponsored by NASA Agency level policy, performed by individual NASA projects (and is some cases programs), and reviewed by a NASA non-advocate organization. This presentation will cover the Agency’s rationale for implementing JCL, the evolution of the JCL policy, and the lessons learned with regards to project implementation. This presentation will also address the current measures that NASA is sponsoring to make the JCL methodologies, processes, products, tools, and communication better for the JCL community.2011Management
Obtaining Greater Efficiency and Productivity in Defense Spending – The Carter MemoTom DuPre, Kristen SchulteThe Nov 3rd, 2010 Ashton Carter memorandum directing initiatives within the Department of Defense to obtain greater efficiency and productivity in Defense spending presents an ongoing challenge to the services. How do PEOs and PMs reduce the costs of their programs without reducing capabilities? This presentation outlines processes to set affordability goals early in the acquisition life cycle and a strategy to help PMs focus on achieving “should cost” versus “will cost” estimates. There is an essential role for the professional cost estimator to help set affordability targets at Milestone A that will be treated as Key Performance Parameter (KPPs), and to participate in systems engineering tradeoff analysis in support of Milestone B decisions...2011Management
Comprehensive Assessment of Contract Performance using Earned Value Management DataWilliam Laing, Colleen Craig, Richard Lee, Scott DeNegrePurpose: This presentation outlines a set of data views that provide a comprehensive assessment of contract performance using Earned Value Management (EVM) data. The proposed views provide alternatives to traditional EVM metrics and are intended to provide early-warning indicators of contract performance...2011Management
Overcoming Challenges in Estimating Advanced Technology ProgramsZachary Jasnoff, Dan NussbaumDuring the Source Selection process, many different types of cost estimating are required by both the Source Selection Board (Government) and the Contractor (Performer). This includes analysis such as Cost Realism, Analysis of Alternatives, Ghosting the Competition and Growth Estimating. Often times, both Government and Contractors have no consistent, repeatable and traceable way of performing these types of analysis. This leads to poor and/or inconsistent decision making during the Source Selection process. However, using parametric methods, these types of analysis are easily accomplished in a fraction of the time of traditional methods...2011Management
Improving Baseline Execution - A Parametric ApproachSteve SultzerToo often projects are undertaken without good visibility into the executability of the program baseline. Although baseline analysis and review (including IBR’s) may include staffing and Integrated Master Schedule (IMS) review, the baseline is often not compared back to the underlying cost drivers upon which the program was based. This can lead to baselines which appear to be consistent from a staffing/hours/schedule perspective, but which may be unrealistic based on the size, scope, and risk profile of the program...2011Management
The Shortcut to Fully Burdened Cost of Fuel AnalysisKristopher Atkins, Richard ClineFrequent attacks against resupply convoys, along with their growth in both length and size, have caused the Department of Defense to investigate ways of shrinking the military’s logistics tail. A large logistics tail impedes the mobility, flexibility, and concealment of tactical forces and grows with fuel demand. The DoD is aiming to reduce fuel demand by integrating energy-based analysis of alternatives into the early stages of systems acquisition. An analytic capability that measures fuel delivery costs and assesses the value of investments in efficiency is needed to evaluate competing alternatives...2011Management
Instrument Schedule Delays Potential Impact on Mission Development Cost for Recent NASA Projects (Follow-On Study)Kristina Kipp, Stephen Ringler, Erin Chapman, Luke RinardThis study explores instrument schedule delays and their potential impacts on mission development cost for recent NASA projects. Schedule data collected at key milestones for a number of NASA instruments is used to compare planned and actual instrument development times. The study shows average instrument development schedule growth is on the order of 30%. Comparing last instrument delivered development time and mission development cost growth shows a positive correlation, indicating that instrument schedule delays may increase total mission development cost. Instruments are binned by various categories such as instrument type, mass, power, etc. to explore specific trends. The results of this study can be used for planning purposes by project and program managers in charge of future NASA development efforts.2011Management
United States Military RankMichael BrozynaMany new cost analysts have no idea about the U.S. military rank structure. This presentation goes through the enlisted, warrant and officer ranks of the Services. Insignias and salutations will be offered. It also will give a frame of reference of pay. It also shows how the ranks are abbreviated differently across Services. No math involved.2011Management
Extending FEA and DODAF to Support Cost ModelingAndreas Tolk, Resit Unal, Charles Keating, Johnny Garcia, Holly HandleyThe Federal Enterprise Architecture (FEA) and the Department of Defense Architecture Framework (DoDAF) are widely recognized, applied, and often mandated methods to define future systems and their interplay with current solutions to close capability gaps with fit for purpose solutions.
However, the focus is currently on operational capabilities and the cost aspect is easier done separately or as an afterthought once the 'optimal technical solution' is specified. Current research at Old Dominion University in two related domains have the potential to contribute to a paradigm shift in cost modeling based on architectural descriptions.
2011Models and Methods
Operating and Support Cost Estimating Methods: An approach to estimate the US Navy’s future cost of Ballistic Missile DefenseBrian A. Welsh, Elizabeth A. Koza, Paul L. Hardin III, Marc GreenbergThe Missile Defense Agency’s (MDA) Aegis BMD program provides a defensive shield against short and mid-range ballistic missiles. Currently a portion of US Navy ships are dedicating some of their at-sea operations to BMD. Additionally, the Sea Based X-Band (SBX) radar system is supporting BMD. Over the next seven years, MDA will add Aegis BMD capabilities to Romania and Poland. After these land-based sites are constructed, the US Navy will be responsible for operating and maintaining the systems and supporting infrastructure at each site...2011Models and Methods
Utilizing The Capabilities Knowledge Base for Cost Benefit Analysis and Analysis of AlternativesChadd SibertDepartment of Defense (DoD) leadership is rigorously evaluating potential acquisition programs much earlier in the programs lifecycle than they have in the past due to the increased scrutiny of defense funding. Two key items are the December 2008 DoD Instruction 5000.02 addressing Milestone-A Analysis of Alternatives (AoA) and recent Department of the Army guidance requiring Cost Benefit Analysis (CBA) for all decision points. This in turn greatly increases the demand for cost information to support these early programmatic decisions. However, are there any tools available to support this effort? If so, how would you utilize them?... 2011Models and Methods
Assessing Impact of Funding Constraints to Cost and ScheduleDarren Elliott, Antonio RippeCost risk and uncertainty over the past few years have drawn high interest in the cost estimation industry. Significant effort has been invested by the cost community to develop standard practices and refining tools to standardize how analysts to conduct risk and uncertainty analysis and examine the results. The most common output has been the cumulative distribution function (S-Curve) from which a target confidence level can be identified and a resulting cost target can be generated. This result is normally defined as the risk-adjusted estimate and phased over time to generate a desired funding level for the program...2011Models and Methods
Applying Development Cycle Electronics Hardware Sub-Product Cost Models to Project ExecutionDavid BloomOver the last decade, many advances have been accomplished by notable organizations in applying parametric model techniques to establish cost estimates for deliverable electronics systems and electronics products. At the Electronics Center of Raytheon Space and Airborne Systems, we utilize commercial software applications to perform these kinds of tasks...2011Models and Methods
Trade Space, Product Optimization and Parametric AnalysisDoug HowarthThis paper shows how to bound, build and assemble trade spaces for product optimization.
The advent of computerized tools that describe available trade spaces has changed not only the nature of optimized product design, but that of parametric cost studies as well. Because these tools allow broader analysis, engineers produce more potential designs and parametricians must produce many more estimates in support of them.
2011Models and Methods
Objective System Acquisition Decision Making Utilizing the Analytic Hierarchy Process (AHP)Hisham JalilToday’s increasingly complex government acquisition environment engenders increasingly critical decision points for acquisition leaders and decision makers. Perpetual advancements in technology coupled with increasing vendor capability often present one such key decision point for today’s technologically complex, mission critical systems. Government decision makers must ask and answer the following question: Do we choose an organic or service acquisition approach for this system? That is, do we build, implement, operate, and maintain the system?...2011Models and Methods
A Comparison of Military and Commercial Submersible Systems, Cost Environments, and Methods for Estimating Submersible Development and Production CostsJanet Vacca-LeBoeuf, Greg C. Bell, Brian HarrisThe U.S. Navy and the U.S. Special Operations Forces (USSOCOM) have established requirements for wet and dry submersible vehicles capable of transporting combat teams and their equipments to and from the shoreline of enemy territory. The submersibles are carried to the vicinity of the enemy either by surface vessels or mounted upon the decks of fleet attack submarines. A recent high profile program, the Advanced Seal Delivery System (ASDS), was initiated to be the principal vehicle for performing this kind of mission. A fleet of as many as six or eight of the 55 ton units was planned...2011Models and Methods
QuickCost 5.0: A Space Project Cost ModelJoe HamakerThis paper describes the latest release version of the QuickCost suite of cost estimation models for space projects. This version of QuickCost includes databases and models in spreadsheet format for five space project application areas which are 1) automated spacecraft, 2) orbital transfer stages (including a variable for human rated), 3) orbital modules (including a variable for human rated), 4) launch vehicle stages (including a variable for human rated) and 5) liquid rocket engines (including a variable for human rated)...2011Models and Methods
Cost Analysis using Random Forest PredictionKaren Mourikas, Denise Nelson, James SchimertWhat to you do when your cost data is not well suited for linear regression analysis? As an alternative, we experimented with Random Forests, an example of using a computer intensive learning algorithm instead of assuming a parametric model. Random Forests are a collection of Decision Trees. Trees can capture non-linear relationships and interactions among predictors. Individual decision trees are interpretable, however, not necessarily great predictors. By suitably fitting a collection of trees, and averaging the tree predictions, Random Forests results compete with the best machine learning predictors...2011Models and Methods
Multicollinearity in Zero Intercept Regression: They Are Not Who We Thought They WereKevin CincottaAsked about the identity of the members the opposing team after being defeated by them, a famous football coach once said, “They are…who we thought they were.” Unfortunately, this is not the case when it comes to multicollinearity in zero intercept linear regression. It is not what we think it is, and the consequences of ignoring the distinction between it and its nonzero intercept counterpart can be devastating...2011Models and Methods
The Infrastructure Service Provider (ISP) Cost ModelKyle Thomas, Belinda Nethery, Carol WilsonThe current political and economic environment has created a focus on efficiency and productivity, particularly with Intelligence Community and Department of Defense spending. One area for improvement that has been highlighted is that of Information Technology (IT) services. A Government Agency identified IT as an area in which efficiencies could be achieved and has been taking steps to deliver IT as services across the enterprise. Beginning with the FY12-17 Program Build, there has been a desire to project the cost of IT services as an enterprise activity rather than by specific programs or contracts, as they had been in the past...2011Models and Methods
Analysis of Parametric and Database-Driven Cost Estimates in the Transit IndustryL. Brian EhrlerMost major transit projects require some form of grant, either Federal or State, to be financially viable for a city. For those cities considering a transit project there are billions of dollars available each year from the US Department of Transportation (1). However, to qualify for these dollars the transit agency for the city, must demonstrate that the proposed transit project is cost effective...2011Models and Methods
An Elicitation Method to Generate Minimum-Bias Probability DistributionsMarc GreenbergAs the US Department of Defense develops less mature, more advanced and more complex systems, there is an ever-increasing burden on the cost analyst to employ methods of eliciting requirements, schedule and cost uncertainties from one of more subject matter experts (SMEs). Arguably, the most common technique a cost analyst uses today to elicit such data is to ask each SME for the lowest, most likely and highest value which, consequently, produces a triangular distribution...2011Models and Methods
Economic Elasticity of Tactical Missile CostsRaymond P. CovertEconomic elasticity is defined as the ratio of the percent change in one variable to the percent change in another variable. In missile production, it is hypothesized that two forms of elasticity dominate: price elasticity of demand and production cost improvement. The former implies prices and demand are inversely proportional, meaning if like goods (e.g., missiles) are cheaper, we tend to buy more of them. Production cost improvement is defined simply as the reduction of costs of successive units. Cost improvement curve theories, such as unit learning, explain the converse effect: “If we can buy a greater quantity of an item, its relative cost decreases between successive units.”...2011Models and Methods
Modeling to Establish the Affordability KPPSamuel Toas, Greg HoganDr. Ashton Carter’s November 2010 memo provided a number of initiatives aimed at achieving better buying power for Defense spending. The guidance presented several new requirements to be completed during Milestone-A and Milestone-B such as the establishment of Affordability Targets. One or the common threads across both acquisition phases is an analysis of performance vs. cost. Programs are required to generate a target cost KPP and provide the necessary cost tradeoff curves which justify the established target...2011Models and Methods
Dynamic Help Desk Resource ModelingSujoy K. Roy, Nolin Huddleston, Steve SheamerThis paper discusses the concept of Dynamic Helpdesk Resource Modeling for a major Information Technology System. This methodology utilizes regression analysis based on the concept of learning curves to project the resource load for a three tiered help desk system based on a number of factors including system user population and maturity. The paper uses the implementation of an Enterprise Resource Planning System throughout the United States Navy as the user system with a three tiered help desk model as the basis for analysis...2011Models and Methods
A Probabilistic Approach to Determining the Number of Widgets to Build in a Yield-Constrained ProcessTimothy P. AndersonMany cost estimating problems involve determining the number of Widgets to build when it takes, on average, M attempts to produce N successes. Examples include computer chips, focal plane arrays, circuit boards, FPGAs, etc. The simplistic approach to this problem is to multiply the number of Widgets needed, N, by the average number of attempts needed to produce a success, M...2011Models and Methods
A Methodology to Improve the Predictability of the CER with Insufficient Data in Korean Weapon System R&D EnvironmentDong Kyu Kim, Sung Jin Kang, Yong Bok LeeParametric cost estimating models have been used widely to obtain appropriate cost estimates in early weapon system acquisition process. Parametric cost estimating models are composed of some CER(Cost Estimating Relationship)s based on regression analysis with historical data.
However, there are many restrictions in developing Korean version CER because of the insufficient number of projects and the abnormal data characteristic such as multicollinearity, outlier existing, heteroscedasticity, etc. So, the diverse regression methods have been studied in Korea to improve the predictability and stability of the each CER respectively.
2011Models and Methods
RealTime AnalyticsEric Druker, Blake BoswellAs organizations increasingly rely on complex analytics using simulation software in their decision making, an inescapable trade-off has arisen: The more robust the analysis, the longer it takes to use to make a decision. One of the primary reasons for this is the time required to run “excursion scenarios” using new or modified assumptions. Running these excursion scenarios often takes hours, requiring the scheduling of additional meetings before decisions can be made. And, since these scenarios are necessarily run outside of the meeting room, leadership is limited in the number of scenarios they can request...2011Models and Methods
Faster & Better Ways to Determine an Engineering Labor Rate for Pre-Award Development Contract Cost EstimatesMichael Sweeney, Brian FerschThe accuracy of cost estimates is an enduring issue in the DoD acquisition community. DoD programs too often overrun cost and are subsequently cancelled - a reality which puts the American warfighter at risk. One chief aspect of cost estimates with significant risk and uncertainty surrounding it is the labor rate used for contractor engineering in support of the specific project; if the wrong labor rate is used it can very detrimentally impact an entire estimate, thereby putting DoD programs at risk of cancellation. Cost estimators across the Government are continuously looking for ways to improve their methods of labor rate formulation...2011Models and Methods
Building CERs and SERs for Enterprise Resource Planning (ERP)Dr. Wilson Rosa, Travis Packard, James Bilbro, Max Hodal, Brian KolstadAFCAA is pleased to present an abstract for an Enterprise Resource Planning (ERP) paper. We will present the results of several years of data collection and study of ERPs, with the intention of providing and improving ERP cost metrics…2011Models and Methods
Introduction to the Operating and Support Cost Analysis Model (OSCAM)Mary Mertz, Geoffrey PawlowskiThe Operating and Support Cost Analysis Model (OSCAM) is a family of tools developed and sponsored by the Naval Center for Cost Analysis to estimate operating and support (O&S) costs for current and future Naval weapon systems. The US OSCAM family of models includes: OSCAM Ship, OSCAM Shipboard Systems, OSCAM Air, OSCAM EFV, and OSCAM USAF. OSCAM is a system dynamics-based model, which provides a structured methodology for dealing with complex systems having many interacting components. This approach enables the user to capture the dynamic behavior of a system while allowing for a flexible design, which can easily be enhanced and expanded...2011Models and Methods
Joint Cost Schedule Model(JCSM) - Recent AFCAA Efforts to Assess Integrated Cost and Schedule AnalysisAntonio Rippe, Greg Hogan, Darren ElliottThe Space Division of Air Force Cost Analysis Agency (AFCAA) supports Air Force (AF) and Department of Defense (DoD) Major Space Acquisition programs by providing thorough, effective, independent cost estimates (ICEs) and conducting special studies for decision makers. Recently AFCAA has initiated a research task to assess the potential for developing a joint cost-schedule model and the usability of the model...2011Research
Fuel Cells - Turn Up the Heat!Arlene MinkiewiczThe fuel cell concept was first introduced in 1893 by Sir William Grove. Fuel cells remained a conceptual curiosity until the 1960s when the US space program identified a requirement for extended life batteries for which fuel cells offered a promising solution. The current focus on green technologies has caused an increase interest in consumer uses of fuel cells for transportation, residential and commercial power supply, emergency power backup and portable power supply for consumer electronics...2011Research
Using Project Performance Data in Cost and Schedule AnalysisFred Kuo, Mike Stelly, Darren ElliottNumerous projects developed at NASA now use the Joint Confidence Level (JCL) approach to simultaneously estimate cost and schedule uncertainty associated with the project. One crucial step in JCL analysis is the development of proper uncertainty bounds. These bounds are often the opinion of subject matter experts, or are derived from cost and schedule estimating relationships. However, as a project progresses, over time the bounds around certain elements and tasks are likely to change...2011Research
Ideas from Measurement Theory for Quantifying IntangiblesMitch RobinsonMeasurement theory, the mathematical study of measurement scales, helps us quantify attributes consistently with: 1) the attributes' “key properties"; and 2) how we intend to use the attributes.
Psychologists, economists, and mathematicians originally developed measurement theory to provide a rigorous basis for quantifying intangible attributes, such as subjective probability and utility. The theory similarly has promise for quantifying intangibles in which the cost community has novel interests, such as “complexity”, “difficulty”, “technology level” and “design heritage”...
NRO CAIG O&M WBS and Duration GuidanceRyan Timm, Gary Kanady, Jenny Moose, Sara Wise, Lori ZondloThe NRO CAIG has developed new guidance for O&M duration and an O&M WBS for cost estimation purposes. The goal of these efforts is to aid in standardizing the scope of O&M activities to be estimated for NRO program independent cost estimates. The new O&M WBS may be used in conjunction with the Space Acquisition WBS for total life-cycle cost estimation. The O&M WBS can be used for estimation of:
• Site O&M Costs (Mission Ground Station, COOP site, Factory, etc.)
• Space and Ground System O&M Tails
• Individual Projects/Programs O&M Costs
Enhancing Cost Realism Through Risk-Driven Contracting: Designing Incentive Fees Based on Probabilistic Cost EstimatesSean P. Dorey, Dr. Josef Oehmen, Dr. Ricardo ValerdiCost overruns in defense system development efforts do not seem to be caused by a lack of cost estimation tools and best practices. A number of sophisticated cost estimation guides have been published, for example by the Army, Air Force, NASA, GAO, RAND, and SSCAG. There also exist extensive recommendations by professional societies like ISPA, SCEA, SSCAG, SCAF, and EACE. In an unbiased world, applying these best practices would generate reliable and accurate cost estimates. However, the problem is that the cost estimation community tends to focus on the technical variables contributing to cost risk...2011Research
An Approach to Estimate the Life Cycle Cost and Effort of Project Management for Systems Centric ProjectsLeone Z. Young, Dr. Jon Wade, Dr. Ricardo Valerdi, Dr. John V. Farr, Dr. Young Hoon KwakOne of the key challenges in predicting life cycle costs (LCCs) is to develop an accurate top down method that can be used in the early phase of system s life cycle to estimate hardware, software, integration, and management costs. Models have been developed for systems engineering (SE) costing component of management, but the literature is void of project management (PM) costing methodologies. Thus, the research study proposes a framework for evaluating PM costs required for the conceptualization, design, test, and deployment of large-scale systems centric projects...2011Research
Cloud Nine, Are We There Yet?Arlene MinkiewiczIn 1961 at the MIT Centennial, John McCarthy opined “if computers of the kind I have advocated become the computers of the future, then computing may someday be organized as a public utility just as the telephone system is a public utility…. the computer utility could become the basis of a new and important industry” [1]. In 2006, Amazon Web Services was launched providing computing on a utility basis. Since that time the notion of cloud computing has been emerging and evolving...2011Research
Effective Use of Cost Risk ReportsAlfred SmithCost estimates are generated in Constant Year (CY), Then Year (TY) and something called cost risk-allocated TY dollars. Several papers have been published describing various ways to allocate risk dollars from a selected level in the Work Breakdown Structure (WBS) to ensure WBS elements sum at the percentile of interest. Cost risk-allocated results are typically used as the basis for estimating the projects budget, but, rarely used as the basis for the subsequent analysis to identify cost and uncertainty drivers in the model (cost risk reports)...2011Risk
Quasi-Monte Carlo Methods: Combating Complexity in Cost Risk AnalysisBlake BoswellCost analyses rely on probabilistic numerical methods to estimate the impact of risk and uncertainty associated with systems technical definitions and cost estimating methodologies. Such methods involve modeling risk and uncertainty as probabilistic distributions, applying iterative sampling techniques using Monte Carlo (MC) methods, and deriving risk and uncertainty adjusted statistical measures; however, the accuracy of statistical measures resulting from probabilistic analyses is directly dependent on the number of samples considered. Therefore, in probabilistic numerical methods, a trade-off exits between accuracy of results and computational complexity...2011Risk
Covered with Oil: Incorporating Realism in Cost Risk AnalysisChristian SmartWhen Jimmy Buffett sang the words “All of those tourists covered with oil” in his song Margaritaville he probably never imagined that this phrase might apply to crude oil instead of suntan lotion. Both the cost and the environmental impact from the 2010 oil spill in the Gulf of Mexico were much worse than anyone had expected or could have predicted. It was, in the words of financial writer Nassim Taleb, a “black swan” an unexpected event with tremendous consequences. These types of events, like Hurricane Katrina in 2005, the giant tsunami in the Indian Ocean in 2004, and the financial crisis that began in 2007 are all examples of events with huge impacts that are hard to predict...2011Risk
Joint Confidence Level Analysis and the Dynamic Integrated Cost Estimator (DICE) ModelGraham Gilmer, Colin SmithSince 2006, NASA has been leading the way in the establishment of Joint Confidence Level (JCL) guidance and the integration of cost and schedule estimation. Recently, in response to the standardization of NASA JCL policy, Booz Allen Hamilton identified the need for and developed a comprehensive model to further enable this technique. The Dynamic Integrated Cost Estimator (DICE) model prototype is Booz Allen’s initial response to NASA’s emerging area of JCL analysis...2011Risk
Risk Based Estimating and Alternative Selection Using Value AnalysisGregory BrinkThe selection between project alternatives and corresponding plans of action is one that is fraught with many facets of uncertainty and risk. In order to enhance the reliability and quality of the information provided to decision makers, an approach of performing a Risk-Adjusted Value Analysis is suggested for project, process, and product alternative evaluation and selection. The use of this assessment technique based in Value Methodology allows for the multi-dimensional weighing of performance, cost, and duration attributes under the influences of risk and uncertainty...2011Risk
Real-time Risk for the Operations EnvironmentJohn TealTraditional risk management and cost risk analysis is an important and evolving field, but many analysts admit that it is as much of an art as a science. Although quantitative in nature, assessing the likelihood and impact of risks is generally a series of predictive guesses made by subject matter experts. When successful, the recipients of risk reports embrace the process and make use of risk reports; in other programs the process of tracking risks is regarded as an academic exercise, which often relegates analysts to the back-office, out of sight and out of touch with real-world production and operations. Ideally, the risk management process should provide users quantitative, objective, timely and relevant risk information...2011Risk
Joint Confidence Level of a Parametric Software Cost and Schedule EstimateMichael A. RossThis paper summarizes a regression-based approach for developing, describing, and calibrating Cost and Duration Estimating Relationships (CDERs) (Ross, 2008) and then describes a simple method for determining joint and conditional probabilities of estimates based on these CDERs. The paper concludes with a practical example of developing a CDER from Software Resources Data Report (SRDR) data collected by Dr. Wilson Rosa of the U.S. Air Force Cost Analysis Agency (AFCAA) and then implementing the developed CDER with joint and conditional probability calculation in a Tecolote ACE model. 2011Risk
The Perils of Portability: CGFs and CVsPeter Braxton, Richard Coleman, Richard Lee, Brian Flynn, Megan Guild, Jack Smuck, Kevin CincottaWith the provisions of the Weapon Systems Acquisition Reform Act of 2009, or WSARA, raising the importance and visibility of risk analysis, it is an opportune time to revisit how risk benchmarks based on historical data are both derived and applied. This paper will focus on two key summary risk metrics: (1) the cost growth factor, or CGF, which captures historical cost growth for individual programs and is used collectively to capture the pervasive tendency to underestimate cost; and (2) the coefficient of variation, or CV, as a measure of the uncertainty in both historical and new estimates...2011Risk
Testing S-Curves for Reasonableness: The NCCA S-Curve ToolRichard Coleman, Peter Braxton, Richard Lee, Brian FlynnThe requirement of the Weapon Systems Acquisition Reform Act of 2009 to “ … disclose in accordance with paragraph (2) the confidence level used in establishing a cost estimate for a major defense acquisition program or major automated information system program…” and the austerity of current Department of Defense (DoD) budgets has brought about an increased interest in risk analysis and widespread use of the S-curve (the cumulative distribution function) of cost. As this interest intensified, experience in the use of S-curves widened...2011Risk
Use of EVM Trends to Forecast Cost RisksRoy SmokerThis paper develops a risk burn-down metric based on a forecasted variance at complete estimated from trend (time series) data. Using only the first 18 data points of a 43 point series, projections of the budgeted cost of work performed (BCWP) and the budget at complete (BAC) are solved simultaneously to identify the value of BCWP that equals the value of BAC at some future date in time...2011Risk
Understatement of Risk and Uncertainty by Subject Matter ExpertsPeter Braxton, Richard ColemanThe authors’ paper “The Correct Use of Subject Matter Experts in Cost Risk Analysis,” presented at the Naval Postgraduate School (NPS) Acquisition Research Symposium (ARS) and Department of Energy Cost Analysis Symposium (DoECAS), explored alternative methods for conflating the risk distributions of multiple subject matter experts (SMEs). This paper builds on the results therein by establishing a mathematical foundation for the conflation of triangular distributions and by presenting the results of an independent experiment on expert assessment of uncertain quantities...2011Risk
The Implementation of Risk Analysis in Proposal Evaluation and Cost ModelsMarcus Oberholzer, Travis WinsteadHerren Associates has been performing cost analyses and proposal evaluations for many Navy clients over the past 20 years. After the receipt of a sole source production proposal, Herren Associates provides a cost and pricing evaluation to determine if the proposal is fair and reasonable. To enhance this service offering and respond to our clients’ need to incorporate risk and uncertainty into its cost modeling process, we developed a solution using Monte Carlo simulation and innovative statistical methods to provide negotiation cost ranges rather than simply a point cost estimate...2011Risk
Cost Risk Allocation Theory and PracticeChristian SmartRisk allocation is the assignment of risk reserves from a total project or portfolio level to individual constituent elements. For example, cost risk at the total project level is often allocated to individual work breakdown structure (WBS) elements. This is a non-trivial exercise in most instances because of issues related to the aggregation of risks, such as the fact that percentiles do not add. For example if a project is funded at a 70% confidence level then one cannot simply allocate that funding to WBS elements by assigning each its 70% confidence level estimate...2011Risk
SEER-SEM to COCOMO II Factor ConvertorAnthony Peterson, Dr. Derrick Tate, Dr. David A. Wyrick, Dr. Ricardo ValerdiPreviously [Madachy, Boehm, 2008], Rosetta Stones have been developed for converting COCOMO II estimate inputs into corresponding SEER-SEM or True S inputs, or vice-versa. Most of those Rosetta Stone mappings between factors are one-to-one, but some are one-to-many. Often these factors use disproportionate definitions. Additionally, each model contains factors that unique to that model...2011Software
Estimation Challenges for 21st Century Software SystemsBarry Boehm, Bradford Clark, Ray Madachy, Wilson Rosa, Thomas TanSeveral future trends will present significant future challenges for the sizing and cost estimation of 21st century software systems. Prominent among these trends are:

1. Rapid change, emergent requirements, and evolutionary development. These trends will require more attention to cost uncertainty ranges, requirements volatility, and a new factor called the Incremental Development Productivity Decline (IDPD).
Function Point Analysis: One Size Fits AllDan FrenchThere are many challenges when faced with creating estimates for software development projects. However, without a doubt the single largest driver for cost, effort and duration is project size.
The typical approach to sizing a software project, if it is tried at all, is to “estimate” software lines of code (SLOC). While this may be the most expeditious method, history has shown that this approach usually produces highly inaccurate estimates.
IBM developed function points as an alternative method to bring about more consistent and accurate project sizing for use in software project estimation. This methodology, while not a perfect solution, has enjoyed great success in more than 30 years of use world-wide...
Is There Magic Associated with Software Benchmarks?Donald ReiferBenchmarks have proven to be a useful tool for determining whether or not software cost, productivity and quality estimates are realistic and make sense. When used properly, benchmarks can be a very powerful tool during negotiations because they set expectations. While many may question the numbers, benchmarks that are produced with rigor and care can pass as solid bases for comparison. However, when done haphazardly, they can be very dangerous. This talk will discuss how benchmarks are developed...2011Software
The Challenge of Agile EstimatingHeather Nayhouse, Christina DonadiAgile development is one of the buzzwords of 2011. But what exactly is agile development and how is it impacting the software world? Agile development refers to a group of software development methodologies based on iterative development, where requirements and solutions evolve through collaboration between self-organizing cross-functional teams. Agile strives to set goals and plan the path of a project to a release while maintaining ability and openness to change based on feedback and new information. An added challenge to understanding and estimating is that, across the software development community, agile development may have different meanings and levels to which it is incorporated into a project...2011Software
Software Cost Estimation Using a Decision Graph Process: A Knowledge Engineering ApproachSherry Stukes, Dr. John SpagnuoloAt The California Institute of Technology/Jet Propulsion Laboratory, methodologies for Flight Software (FSW) cost estimation and documentation are determined that allow for efficient concurrent and consistent analysis within a tight schedule constraint. This knowledge is structured or "engineered " to facilitate the implementation of FSW cost estimation by others who wish to serve as practitioners in the field...2011Software
Enterprise Resource Planning Systems: Sizing Metrics and CER DevelopmentDavid BrownOver the last 10 years, Enterprise Resource Planning (ERP) systems have seen increasing use within government agencies. One of the key components to any software cost estimate is a sizing metric. Unlike estimates for a traditional AIS, ERP estimates cannot use a traditional metric such as source lines of code (SLOC) because (1) ERPs are almost always based on commercial off the shelf (COTS) software; and (2) ERP implementation requires a significant amount of effort outside the categories of software development and software procurement. Reports, interfaces, conversions, and extensions (RICE) objects are often suggested as the sizing metric of choice for ERP...2011Software
Defense Acquisition University’s (DAU) EVM Checklist WorkshopKim HunterAs the Department of Defense (DoD) increases emphasis on Earned Value Management (EVM) as a project management tool, more and more program offices are finding themselves having to implement and then subsequently execute EVM on their contract. Although EVM is a project management tool, EVM responsibilities are routinely assigned to the business or financial management section which is typically staffed with financial management analysts and cost estimators who may have little or no EVM experience or training. Even if an EVM analyst is on staff, his or her experience and training can vary greatly...2011e-Track
An Improved Method for Predicting Software Code Growth: Tecolote DSLOC Estimate Growth ModelMichael RossThis paper describes the Tecolote DSLOC Estimate Growth Model, which provides probabilistic growth adjustment to Technical Baseline single-point Estimates (TBEs) of Delivered Source Lines of Code (DSLOC) for New software and for Pre-Existing software, these estimates being sensitive to the “maturity” of the estimate; i.e., when, in the Software Development Life Cycle (SDLC), the TBE DSLOC estimate is performed. The model is based on Software Resources Data Report (SRDR) data collected by Dr. Wilson Rosa of the U.S. Air Force Cost Analysis Agency (AFCAA). This model provides an alternative to other software code growth methodologies such as Mr. Barry Holchin’s (2003) code growth matrix.2011e-Track
Parameters in Parametric Cost EstimatingMyung-Yul LeeParametric estimating methods are one of the most desirable and have high creditable due to the fact that this method use actual hours for the weapon system we are estimating. The purpose of this paper is to study how to create a parametric estimating model for weapon system in the aircraft. C-17 Engineering organization will be utilized for generating estimating model...2011e-Track
Integrated Cost-Schedule Risk Analysis using Risk Drivers and Prioritizing RisksDavid HulettMany cost estimates assume that the project schedule is engraved in stone. A risk analysis of the cost estimate is conducted using only cost risk drivers. In this analysis the impact on cost risk of uncertainty in schedule is often ignored or incompletely considered.
This presentation discusses how cost and schedule risk analysis is integrated so that cost risk is affected by both traditional cost risk elements (e.g., uncertain material costs or labor rates) and schedule risk elements (e.g., fabrication or testing that may take longer and thus cost more)...
Review of Three Small-Satellite Cost ModelsMelvin Broder, Eric Mahr, Daniel Barkmeyer, Erik Burgess, Wilmer Alvarado, Samuel Toas, Gregory HoganThe problem of estimating costs for small satellites is more vexing that it would seem. Small satellites, as described here, generally weigh less than 1,000 pounds, and are sometimes much smaller and significantly different from much of what exists now. Costing these kinds of satellites is not simply a matter of scaling down from larger systems. It requires incorporation of new data sets and consideration of new modeling methods. The challenge of addressing small?satellite development is important, considering the emerging priority for developing much smaller, modular or interactive space systems...2010Estimating
Determining the Cost of the Certification and Accreditation Process using Expert Opinion and Monte Carlo SimulationAmanda Flynn, Belinda Nethery, Kyle Thomas, Amanda Gerstner, Brett Dickey, Christina Kanick, Peter BraxtonThe Certification and Accreditation (C&A) paper will address an empirical approach to determining the total cost and schedule of completing the C&A process for a given network or system. The requirement for and policy guidance on how to conduct C&A is covered by Director of Central Intelligence Directive 6/3 (DCID 6/3) along with other DoD and Federal guidance. DCID 6/3 requires agencies to complete a rigorous process of reviews to ensure that the networks and systems are secure. The customer felt that C&A was consuming extraordinary amounts of time and effort but wanted to document and understand the real impact before pursuing changes to the process...2010Estimating
Developing Far-Term Portfolio Cost Estimates for a System of SystemsKevin Foley, Ian CappitelliTraditional cost estimation relies upon data such as requirements, schedules, and relevant historical data provided by a program office to develop an estimate. Significant problems arise for programs with far-term (>10 years) start dates because this data may be unavailable and program offices may not exist. Historically, many commercial and government agencies would either ignore these future costs or develop a simple Rough Order of Magnitude (ROM) estimates. Additional complexities arise when these far-term programs interact with existing near- and mid- term programs...2010Estimating
Lessons Learned from New Subsystem Development in Manned SpaceRobert GeorgiThe effort to develop new technology is complicated. We will examine the effect when a bottom up risk adjusted schedule is not used. Low technical readiness level (TRL) components carry particular risks which can play havoc with budgets and deadlines. Commercial parametric models such as Galorath’s SEER are designed for estimating hardware at a TRL of at least 4, component and/or breadboard validation in laboratory environment. When engineers are challenged with realizing truly advanced technology concepts the cost risk (amount it can exceed the parametric estimate) is 110% to 400% based on limited historical evidence...2010Estimating
Applying the Army Fully Burdened Cost of Fuel Methodology to Analyses of AlternativesDavid HullIn the current conflicts in Iraq and Afghanistan, enemies are attacking United States (U.S.) military convoys that transport necessary supplies such as fuel, ammunition, food, and water in an effort to sever logistical resupply routes. Department of Defense (DoD) leadership is concerned about a having a large logistics footprint that supports
U.S. forces because large convoys are attractive targets. Energy inefficiency is one of the main causes of the large logistical convoys. In an effort to eventually decrease the size of the U.S. logistics footprint, DoD leadership is putting into place procedures to better understand military fuel demands and become more fuel efficient...
Estimating Issues Associated with Agile DevelopmentBob HuntNew software development methodologies with combinations of old and new ideas are getting increasing public attention. These ideas all emphasized close collaboration between the programmer and business experts; face-to-face communication; frequent delivery of software units; and tight, self-organizing teams. Agile is a frequently used developmental process that follows this paradigm. As the use of Agile Development expands, the cost analyst is faced how this process affects the basic estimating fundamentals. This paper will discuss the best estimating practices that should be applied to agile programs and will recommend a set of techniques for agile estimation including...2010Estimating
Cost Benefit Analysis for Incorporation of PrognosticsJerry Le MayPrognostics and Health Monitoring (PHM) technology report information about the operational condition of a system or product. This information can be used to notify the user of necessary service, warn of impending failure, order replacement parts and\or consumables, notify Police and Fire personnel of an emergency and in some cases save maintenance costs by reducing unnecessary maintenance or service. A Cost Benefit analysis provides the basic cost information for a customer to determine if the benefits of incorporating PHM into a system are worth the cost...2010Estimating
Improving ERP Estimating in the DoDWison Rosa, Charles Silvanic, Max Hodal, Brian KolstadAFCAA is pleased to present an abstract for an Enterprise Resource Planning (ERP) paper in two parts. The first part presents the results of several years of data collection and study of ERPs, with the intention of improving ERP estimating. The second part is a case study where AFCAA used this information to estimate a major ERP. AFCAA envisions this paper resulting in two presentations in the same track, one for each part...2010Estimating
Cost Factors Contributing to the Choice of Aerospace Advanced MaterialsPaul ShieldsThe materials selection process for the most recently proposed military and civil aircraft has centred substantially around the use of advanced carbon composites versus conventional high strength aluminium alloys. The former material type has been selected for use in increasing proportion. For example, Airbus’s A400M is 35% advanced composite by weight; the JSF is 40%; the Boeing 787 Dreamliner is circa 50%; the proposed Russian MS21is 45%; the A350 XWB is 52% and the Bombardier C-Series is more than 50%...2010Estimating
A Case Study in EAC GrowthRichard ColemanThe responsibility for the reporting of EACs is well established, and the methods for prediction, as well as the shortcomings in those methods are well known. This should result in good EACs and a good sense of the risk in those EACs. Occasionally, however, growth of apparently totally unexpected scope occurs. Why does this happen, with all of our experience and all of our processes? This paper will describe the growth of several EACs from the original prediction through several snapshots of what was known and what was reported...2010EVM & Scheduling
Programs and the Future of Project ManagementSimon Dekker, Vance KotrlaEvery government agency is concerned about funding. Those able to make the wisest capital investments succeed while others fall behind in cost and schedule overruns. For contractors and suppliers, increasing globalization has resulted in widespread competition in what used to be domestic markets. Vendors and service providers now routinely bid against global companies, even for local business. In order to remain competitive, businesses are re-examining their management processes to ensure all operations demonstrably serve the larger goals of the organization...2010EVM & Scheduling
Integrated Master Planning Formulation-PMAG ApproachNancy Droz, Mun KwonThe Integrated Master Plan (IMP) is an essential program tool that spans from the beginning of the life- cycle acquisition framework through source selection, through program execution and up to program sell- off. Though the IMP has only three or four levels, the determination of program elements, significant accomplishments(key events), accomplishment criteria and possible tasks its development provides crucial insight to successful execution of the remainder of the program’s life-cycle. Furthermore, the conduct of risk assessments to include risk identification and mitigation, as well as accounting for unknowns-unknown is a key step in the development of an executable IMS...2010EVM & Scheduling
SMC/PMAG - Control Account Manager (CAM) Notebook EvaluationEddie Hall, Nhung Tran, Mun KwonThe Control Account Manager (CAM) notebook is the key document for assisting the CAM and government action officer in the integration and management of the control account. The CAM notebook enables “single thread” analysis of the technical scope, integrated master schedule (IMS), resource loading profile, earned value (EV) performance data, and subcontractor costs. Past and present practice tends to focus on the financial and EV performance data, schedule and program plan data. The technical scope of the control account has largely been ignored...2010EVM & Scheduling
Integrated Critical SchedulingDonna Pucciarello, Jared Mathey, Dustin SextonThe Integrated Critical Scheduling (ICS) methodology is an in-depth guide to ensure client and stakeholder overall objectives are exceeded. Integrated scheduling is one of the most critical facets of project management within a developmental government program. An Integrated Master Schedule (IMS) is the foundation upon which a project’s cost, schedule and risks are objectively managed. It also serves as the foundation for time-phasing of the project’s resources. Thorough IMS data analysis yields accurate current and predictive indications of the health of the project as well as increases the value of complimentary products such as the Work Breakdown Structure (WBS), Integrated Master Plan (IMP) and Organizational Breakdown Structure (OBS)...2010EVM & Scheduling
Ship Construction Estimates at Completion: A New Technique Using the Weibull FunctionShawn RudolphThis presentation summarizes recent earned value management (EVM) research at Technomics, Inc. exploring new, improved methods for generating Estimates at Completion (EAC). Our approach is based on a significant amount of “burn rate” data resulting from serial ship procurement. We define burn rate as the incremental change in the cumulative value of Actual Cost of Work Performed (ACWPCum) divided by the time over which the change occurred. Though some research suggests that Rayleigh modeling of the burn rate is preferred, we have studied an alternative regression technique based on the Weibull probability density function, known as the Hybrid Weibull EAC Model...2010EVM & Scheduling
Technical Content and Risk Based Integrated Baseline Review ? PMAG ApproachNhung Tran, Mun KwonIntegrated Baseline Review (IBR) is a technical review process integrated with schedule and cost baselines. Risk identification should be the focus of the IBR process which includes IBR preparation, Performance Measurement Baseline assessment, IBR execution, and management processes. Frequently, many IBRs have evolved into a “box check” process with an emphasis on in-plant CAM interviews and limited preparation by the government program office team. It is crucial to understand that an IBR is a process, not just an event. Properly planned and executed Integrated Baseline Reviews (IBRs) is a key element of a Program Manager’s risk management approach...2010EVM & Scheduling
How to Manage a Program EffectivelyBruce Fad, Dale ShermonThis paper will describe the importance of cost estimating to Program Control. Specifically it will consider the policy for Earned Vale Management (EVM) and the need to consider the deviation from the baseline program in an EVM System (EVMS). Through the integration of a parametric model with an EVMS application, on based upon frequent re- evaluation of the baseline, it is possible to increase the confidence in the EVM metrics...2010EVM & Scheduling
Use of Life Cycle Cost Estimates in OMB-300 Budget ReportingDavid Brown, Kevin CincottaMajor capital asset investments within the federal government are required to submit program budget estimates as part of the OMB-300 Business Case. In cases where the program has already completed an estimate of life cycle cost (LCC) within another document (such as an Analysis of Alternatives or Economic Analysis), it is often desired to use the LCC as the basis for the initial OMB-300 budget submission. However, the process of conversion from an LCC to an OMB-300 budget involves potentially complex adjustments for inflation, scope, and reporting format...2010Life Cycle Cost
Life Cycle Cost Estimation of System of SystemsKishore Gagrani, Manmeet Grover, Yousuf MohammadMark Maier’s System of Systems definition [Maier 1998] dis-integrates systems from SoS. In that it calls each individual system as Constituent System (CS) which is independently developed, and managed. But new CS are often added into SoS in different time frame, with many COTS products and these CS can dynamically come and go...2010Life Cycle Cost
Developing an Efficient and Repeatable Life Cycle Cost Estimation Process within an IPT Framework by Leveraging Parametric Tools and Constant Client CommunicationZachary Jasnoff, Kevin McKeelOne of the major challenges that federal IT program offices encounter is to develop accurate and defendable life cycle cost estimates, given shrinking budgets and limited availability of key resources. Additionally, the program offices frequently have neither “embedded” cost estimation resources nor tools required to develop Life Cycle Cost Estimates (LCCEs). Working in a collaborative environment, PRICE Systems, Alon Inc, and Logapps have developed a repeatable process to deliver high-quality LCCEs that can stand up to GAO and OMB scrutiny...2010Life Cycle Cost
A Cost Estimating Framework for Materials Obsolescence in Product-Service SystemsFrancisco Romero, Rajkumar Roy, Essam Shehab, Kalyan CheruvuObsolescence happens when a component becomes no longer available from stock of own spares or being procurable or produced by its supplier or manufacturer. This is an issue that will definitely affect long-life support systems. The rapid change in technology during the last few decades has exacerbated this problem, especially for EEE (electronics, electromechanical and electrical) components, as their life-cycle is becoming increasingly shorter. This is the reason why most of the research on obsolescence carried out so far has been focused on dealing with EEE components obsolescence...2010Life Cycle Cost
Estimating Life-cycle Cost of West Virginia Fiber Reinforced Polymer (FRP) Bridge DecksSidharta SahirmanThe main objective of the research was to study the economic viability of West Virginia Fiber Reinforced Polymer (FRP) bridge decks. Life?cycle cost of those bridge decks were estimated for conducting such analysis. Three main differences that distinguish the way the life?cycle cost of FRP deck was estimated are: (1) the manufacturing cost of a FRP bridge deck was estimated using learning curve theory; (2) cost savings in support structures when FRP is chosen as opposed to the alternative bridge deck was modeled; and (3) the service life was estimated based on factor method to minimize the subjectivity of the estimates. The three case studies for West Virginia FRP deck projects show that based on the estimated life?cycle cost, FRP decks are financially viable under certain conditions.2010Life Cycle Cost
Developing a Milestone-A Analysis of Alternatives Cost EstimateChadd SibertDepartment of Defense (DoD) leadership is rigorously evaluating acquisition programs much earlier in the system’s lifecycle than they have in the past due to the increased scrutiny of defense funding. This in turn greatly increases the demand for cost information to support these early programmatic decisions...2010Life Cycle Cost
Risk Based BOE Analysis – PMAG ApproachImran Ahmed, Mun Kwon, David WangThe Government Accountability Office (GAO) Cost Guide is a compilation of best practices from across the Federal Government that can serve as a benchmark for cost estimation. The Guide provides useful information to assist the acquisition community in the development of reliable, comprehensive, defensible, and well- documented cost estimates...2010Management
Federal Budget Process: A Quick PrimerMichael BrozynaMost people have little understanding of the Federal appropriation process. Yet, it has an enormous impact on the cost estimating community. This briefing is a light-hearted romp through the process in an attempt to explain some of the relationships and terms used. It may also help explain why your children and grand children owe tens of thousands of dollars each to foreign creditors. It won’t explain why you think you pay too much in taxes. There will be no equations used in this presentation.2010Management
The 1989 NASA Space Exploration Initiative Cost Risk Estimate and Related IssuesEdwin DeanThis paper examines cost related issues that arose during and after performing the 1989 cost risk estimate for returning to the Moon and going on to Mars (The Space Exploration Initiative (SEI)).

It will describe how the cost risk estimate was performed and issues that arose from that process. It will also examine cost related issues that arose from follow on personal research driven by performing that estimate.

The observations and perspective contained in this paper are personal and may differ from the perspective of the SEI project and NASA.
Service Cost Estimation with Uncertainty Using Agent Based SimulationJohn Erkoyuncu, Rajkumar Roy, Essam Shehab, Kalyan CheruvuThis paper aims to contribute to the decision making in integrating uncertainty to service cost estimation by providing a novel approach through agent based modelling (ABM) for the early stage of bidding for Contracting for Availability. The paper presents the ABM architecture describing the agents that represent the customer and industry, which captures responsibilities in areas including failure, repair, supply and availability management. Furthermore, internal and external rules are defined to capture the interaction between the cost drivers. The major outcome relates to the robust and accurate “what-if” scenarios of the dynamic interplay among the many cost uncertain elements...2010Management
"Dan Galorath on Estimating"Dan GalorathDan Galorath for over 25 years has been a leader in the estimating industry, and finally has begun to blog about his experiences and what he's still learning. This incredibly wide?ranging presentation will bring to life some of the high points of his BLOG. Dan will discuss metrics, effectiveness measurement, the real costs of a variety of undertakings, what is ITIL versus ISO 9000, the risk of IT projects, software project costing challenges and solutions, and lots more. He weighs in on a wealth of topics both timeless and contemporary.2010Management
Who Moved My Milestone? Cost and Risk Implications of Selected Changes in the New DOD 5000 Defense Acquisition FrameworkPeter Hantos, Nancy KernAs a significant milestone in the DOD’s continuous self-assessment process, an important document, the Defense Acquisition Performance Assessment (DAPA) report, was released in early 2006. The report – in its sweeping and integrated assessment – attempted to consider all critical aspects of defense acquisition, and made recommendations for each of the major elements of the defense acquisition system. On the basis of the panel’s recommendations, in December 2008 three major changes were instituted in the Technology Development Phase of the DOD 5000.02 Defense Acquisition Framework...2010Management
Innovative Procurement Approach for Satellite Constellations on Institutional MarketHerve JoumierProcurement of satellite constellations on Institutional market has been limited to GPS constellation for a long time. The 1990’s saw the emergence of commercial constellation such as Iridium or Globalstar. Due to the magnitude of the investment, building up such kind of projects has never been easy. All these projects went through bumpy rides with more or less success in meeting their objectives when they actually survived...2010Management
Change Management Data Collection Tool: Findings and Lessons LearnedLaura LucasThis presentation relates the benefits and pitfalls of the online survey that the author designed and implemented. Using example questions and responses from the actual data collection tool, the author relates findings and lessons learned. Topics include: how the data collection tool strengthens future cost estimates, where the data fail to inform estimate inputs, and why the author recommends certain design choices...2010Management
Transforming Estimating and Pricing Organizations to Attract and Leverage the Strengths of the New WorkforceStanton Smith, Ian Callanan, Eva YoungThe incoming workforce to replace the Baby Boom/Reagan Cold War defense buildup professionals is going to require a new organizational model to attract the professional talent to maintain effective Estimating and Pricing organizations. Northrop Grumman Electronic Systems Estimating and Pricing Organization, located near Baltimore Maryland, has piloted and implemented a new organizational and management technique model and will be sharing the concept, implementation, and results in this paper...2010Management
Business Modeling: Cost Analysis In The Operations EnvironmentJohn Teal, Andy BeckTraditionally, cost estimators find themselves estimating elements of major acquisitions, whereas operational environments rarely require the same level of cost support. However, operational programs are under the same pressures to reduce costs and manage limited resources efficiently. This paper explores the application of cost and quantitative analysis in the operations environment, and illustrates practical examples of cost-related analytic toolsets. Specifically, this paper will describe environments apt for business modeling, propose metrics related to cost analysis that usefully measure operational performance, and illustrate tools that allow the cost analyst to perform quantitative analysis and communicate with stakeholders...2010Management
Time Phasing Cost Estimates in MS Excel ModelsWilliam Black, Matt Reiley, Jeffrey JaekleOne of the most critical, and often difficult, elements of developing a cost model is the determination of how to phase costs across the life of an estimate. Agency Cost Positions (ACPs) and Independent Cost Estimates (ICEs) are often conducted early in a program’s lifecycle, before detailed technical specifications, let alone schedules, have been developed. Even in those cases where a decent schedule does exist, issues still arise concerning the best way to translate technical and programmatic milestones into executable cost/budget profiles. This task proves even more daunting as external factors, such as the needs of other programs, are introduced into the estimating and budgeting process...2010Models
GDOT Planning Level Cost Estimation Review StudyJeff Carroll, Dave CoxCost escalation has become a major concern in virtually every field of capital project development. On top of regular inflation, the volatility of the global oil market is tied directly to most construction related items (primarily asphalt), thereby causing shifts in cost by the rising and falling oil prices. Within the transportation sector, cost escalation has attracted attention at the federal, state, regional, and local government levels for highways, transit, and other modes...2010Models
Logistics Requirements Funding Summary (LRFS) Cost Estimating Tool (CET): A Quick Cost Estimator for LogisticiansCharles Gu, John Ko, Jeremy Eden, Zach Pryor, Brandon CainSimilar to Life Cycle Cost Estimate (LCCE), the Logistics Requirements Funding Summary (LRFS) captures Integrated Logistics Support (ILS) related costs for a program throughout its life cycle. The LRFS provides visibility into the logistics requirements of a program which is needed during Program Objectives Memorandum (POM) and budget submissions. Additionally, it helps to plan and quantify requirements, identify and defend funding, and serves as the ILS input to the LCCE...2010Models
Cost Benefit Optimisation to Achieve Affordable Force StructuresZachary Jasnoff, Dale ShermonThis paper will describe the application of parametric cost models to the problem of selecting the optimum Force Structure when limited defence budgets impose considerable restrictions on the shape and magnitude of the armed forces.

The use of Cost Benefit analysis by means of a Combine Operational Effectiveness and Investment Appraisal (COEIA) has been a methodology used to select alternative Force Structures in many procurement agencies. It is now possible to optimise this process through the linkage of parametric models and optimisation tools such as Phoenix Integration’s Model Center.
Standardizing Space Systems O&M Cost EstimatingRobert Koury, John Long, Pete StanleyThis presentation addresses the importance of standardized Operations and Maintenance (O&M) cost estimation within the space cost community. Currently there are at least nine active O&M cost estimating models in use by just one branch of the United States Government. Although there is a relatively large number of models there do not appear to be a comprehensive view of O&M for space systems...2010Models
Adapting Bottoms-up Cost Estimating Relationships to New SystemsDale Masel, Robert Judd, John D. DowlerFor several years, Ohio University has been engaged in research to develop improved cost estimating procedures for jet engine components. The resulting methodology uses bottoms-up cost estimating relationships to estimate the cost for each of the manufacturing processes needed to create the features on the part, but these CERs are implemented in a parametric framework.
This means that most of the user-entered attributes are high-level descriptions of part geometry and these attributes are used to map appropriate values to the attributes in the feature-level CERs...
Simulation-based Experimentation: How to Scope and Estimate Development ProjectsKaren Mourikas, Denise NelsonSimulation-based Experimentation consists of designing, developing, and executing an experiment using computer simulations representing real systems, with or without operators-in- the-loop. Estimating simulation-based experimentation differs from estimating an acquisition program which includes distinct development, production, and operations and support phases. The end result of simulation-based experimentation is an experimentation environment and results of analysis to address both internal and external customer concerns. To improve estimation of the cost of a simulation-based experimentation project, we developed an experimentation cost estimating methodology and toolkit...2010Models
Advancing the Art of Technology Cost Estimating- a Collaboration between NASA and BoeingMark Schankman, John ReynoldsAdvancing the art of estimating the cost of developing technologies is a long term need for both government and industry. This paper describes a collaborative effort by Systems Engineers at Boeing and NASA resulting in cost estimating tools and analysis techniques that assist in evaluating the development cost of aerospace technologies...2010Models
Using the Joint Integrated Analysis Tool (JIAT)Daniel Schwartz, Melissa CyrulikThe Joint Integrated Analysis Tool (JIAT) is administered by the Office of the Deputy Assistant Secretary of the Army – Cost & Economics (ODASA-CE) and Tecolote Research, Inc. JIAT is a web-based application that provides seamless linkages between cost estimating tools, engineering design models, modeling and simulation tools, as well as capability, performance and operations and support databases. JIAT incorporates several cost estimating industry standard tools and databases which allow end-users to run and query these resources from their desktops...2010Models
Return on Investment Metrics: An Army PerspectiveNoel BishopCost analysts have several Return on Investment (ROI) metrics (e.g., Benefit to Investment Ratio (BIR), Net Present Value (NPV, and Break Even) in their tool box when conducting an economic or comparison of benefit analysis. All of these metrics have their utility, however one metric may be more appropriate than another depending on who the decision maker is and the type of project...2010Methods
Detecting Anomalous Cost Data in an Integrated Data WarehouseLawrence BrownData quality is always a key issue when using historical data to predict future costs. Often a small data error in the historical data can lead to much more significant errors in predicted costs. The problem becomes more acute if data is integrated from several sources, the sources systems have evolved, the source systems were not designed to support analysis, or extensive business rules are required to allocate or assign attributes to the data. This paper discusses systematic techniques, used by the Air Force Total Ownership Cost (AFTOC) program, to find anomalous data...2010Methods
New Research in General Error Regression Model (GERM) Significance TestingKevin Cincotta, Andrew BusickA limited but significant literature exists with respect to significance testing in General Error Regression Models (GERM). GERM refers to cost estimating relationship (CER) development in the absence of specific functional form assumptions and distributional assumptions about the error term. It is sometimes called a “distribution-free” method of CER development. Within the cost estimating community, GERM was first popularized by Book and Young (1997). More recently, Anderson (2008) proposed A Distribution-Free Measure of the Significance of CER Regression Fit Parameters Established Using General Error Regression Methods, which won best paper in its track at the 2008 SCEA Conference...2010Methods
Neural Network Cost Estimating RelationshipsEdwin DeanRegression, in various forms, is the tool of choice for parametric cost analysis (PCA). Neural networks are a largely unused analysis tool within the cost community. However, neural networks are an effective alternative to all forms of regression for PCA. This paper will illustrate the use of neural networks for PCA and will highlight similarities and differences with respect to the use of regression for PCA. For example, regression provides a linear or transformed linear approximation of parameters for a given set of data. Neural networks provide a purely nonlinear approximation of the parameters for the data...2010Methods
Testing for the Significance of Cost Drivers Using Bootstrap SamplingDaniel FeldmanMost regression-based statistical modeling aims to derive, on the basis of historical data, an algebraic expression for estimating a quantity of interest, such as cost. Such an algebraic expression is characterized by one or more “fit parameters,” which are typically coefficients or exponents. Once these fit parameters are determined, it is a logical next step to test for their accuracy by a process called “statistical inference.” The fit parameters of a cost-estimating relationship (CER) are each directly linked to a candidate cost driver...2010Methods
Improving CER building: Getting rid of the R2 tyranny - Building a CER with the MedianPierre FoussierThe purpose of the author is always to get the maximum – for prediction purposes – of the data which are available to the estimator (assuming their reliability has been already checked). This paper is a contribution to this effort. We will start by a classification of the cost estimating techniques, showing that parametric – of which a clear definition will be given – is just a technique among others (which should deserve more consideration). Nevertheless this paper deals with parametric studies, because it is still the most used technique...2010Methods
The Business Case for Bootstrapping: When You're Stuck with Incomplete Data, Here's How You Make it Work!Brett Gelso, Glenn Grossman, Eric DrukerThe purpose of inferential statistics is to reliably extend sample characteristics to a population. Mild assumptions such as the Central Limit Theorem and Identically and Independently Drawn (IID) estimators ensure that, based on a given level of precision, samples will be representative of the population from which it was drawn. However, when samples are small, most standard test statistics will be insignificant, because sample characteristics may fail to adequately capture variation in the population. Parametric Bootstrapping is an alternate approach which resamples the underlying distribution of the sample in order to estimate population characteristics. The objective of this presentation is to illustrate how to use Parametric Bootstrapping for Cost Estimating with small samples to enhance defensibility of cost estimates...2010Methods
Simple Mean, Weighted Mean, or Geometric Mean?Shu-Ping HuThere are three commonly used methods to calculate an average (i.e., mean): the simple average, weighted average, and geometric average. Analysts often use averages as estimating guidance when predicting nonrecurring (NR) costs using ratios of NR costs to first unit (T1) production costs. For example, when the “average” of the NR to T1 ratios is determined, the NR cost can be estimated as a “factor” of its T1 cost. Here, the simple average is simply the arithmetic mean of these ratios while the weighted average is derived by weighting each ratio by its T1 cost. Consequently, deciding which average (i.e., factor) is the most appropriate metric for estimating the nonrecurring cost is frequently debated...2010Methods
Study on developing CERs with Insufficient Data in Korean R&D environmentSung Jin Kang, Yong Bok Lee, Dong Kyu KimParametric cost estimating models have been used widely to obtain appropriate cost estimates in the weapon acquisition process. A parametric cost estimating model is composed of some Cost Estimating Relationships(CERs). Generally, CERs are regression models which have been based on cost and technical data. In order to obtain a good cost estimate from CERs, it's important that gathering the sufficient historical data and selecting a proper method for statistical satisfaction under restricted condition is essential...2010Methods
Using Fourier Analysis in Evaluating Program Office EstimatesAngela Lemke, Matthew Latham, Stephen CoxFourier analysis encompasses both the decomposition of a function into simpler pieces, often called “analysis”, and the corresponding operation of rebuilding the function from those simpler pieces, referred to as “synthesis”. While often used as a tool in analyzing harmonic functions, Fourier analysis methods can also be used to investigate and determine contributions of various components to a more complex output function...2010Methods
Contract Negotiation in a Multi?objective Scenario using the Concept of Dynamic Shared Negotiation? Spaces and Soft vs. Hard Affordability LimitsAnil Ray, Jorn Mehnen, Paul ShieldsIn today’s business environment, a shift in the competitiveness paradigm “Business that Manufacture” is becoming essential. It covers the whole bandwidth from designing for performance to designing for affordability and finally delivering a cost effective product with added value that the Buyer is willing to pay for. Making the decision as to whether it is most cost effective to procure or to manufacture for reasons of affordability requires analysis of alternatives to achieve a convincing compromised solution that satisfies Seller and Buyer. Historical data verification or tacit experiences of Seller and Buyer tend to be the norm but often decision makers miss the choice of the best possible solution...2010Methods
Software Cost Estimating RelationshipsJennifer LeottaSoftware cost overruns are a common problem for the majority of software development projects. With the ever increasing amount of software present in current Department of Defense (DOD) programs, it is extremely important to generate an accurate software cost estimates. There are many complex models that estimate software development productivity and costs. This paper builds upon the principles of these models to look for a simple regression model that can be used to generate accurate and defendable cost estimates for software development programs.2010Software & IT
Costs of Achieving Software Technology ReadinessArlene MinkiewiczTechnology is advancing with lightning speed. Many of the complex systems required by the Department of Defense (DoD) and NASA require technological innovations to achieve sophisticated mission and state of the art accomplishments. Technology readiness assessments for complex systems and subsystems are important for both the government and its contractors to ensure proposed technologies meet program requirements. Without good technology assessments, programs may be funded that have little or no chance of success because success assumes use of technologies not yet confirmed to be realistic. The defense acquisition community recognizes this and has developed detailed guidelines for Technology Readiness Assessments (TRAs)...2010Software & IT
Estimating S O A, As Easy as 1 2 3Arlene MinkiewiczMetcalf’s Law tells us that the value of a telecommunications network is proportional to the square of the number of users of the system [1]. Service-Oriented Architectures (SOAs) capitalize on this phenomenon. Through a set of standard interfaces, services (software based capabilities) are made available to any consumer willing to follow the rules for interface and consumption. The loose coupling provided by the standard interfaces enables this plug and play capability. Taking advantage of such a notion promises great gains in efficiency for anyone looking to create interoperable, scalable applications that share information across boundaries...2010Software & IT
Function Point Analysis: Introduction and Basic Overview as an Alternative to SLOC-based EstimationTucker MooreWhen analyzing and estimating the costs for a software application, most cost estimators inherently rely on problematic, “SLOC-centric” estimation techniques. Cost analysts typically strive for concrete, tangible numbers to work with in order to produce relationships and parametric trends in historical and future cost data...2010Software & IT
Improving Software Cost Estimates Using the Univariate ModelBrian OpaskaDespite its drawbacks, the univariate (first-order) model for estimating software remains a widely used technique throughout the government and private sector. This presentation will provide three techniques that can be applied to improve the consistency and accuracy of this method. These techniques involve using a reliable code counting sizing metric, performing thorough analysis of the actual SLOC count, and normalizing the SLOC...2010Software & IT
Comparative Analysis of Software Maintenance Modeling in the COCOMO II, SEER, SLIM and True S Cost ModelsDonald Reifer, Jill Ann Allen, Brian Fersch, Barbara Hitchings, James Judy, Wilson RosaThis paper provides a snapshot of the findings, conclusions and recommendations a year-long Joint U.S. Army and Air Force study on the topic of software life cycle maintenance and cost modeling. The paper starts by defining the work that government life cycle support centers typically perform as part of their normal jobs using a Work Breakdown Structure (WBS). Next, the paper rates how well the popular COCOMO II, SEER, SLIM and True S cost models estimate each of the work tasks in this WBS. Gaps uncovered are then highlighted and rules of thumb for filling them are offered as the paper concludes with recommendations for providing fuller coverage for these important operations, maintenance and support activities.2010Software & IT
Software Maintenance: Debunking the MythsDonald Reifer, Jill Ann Allen, Brian Fersch, Barbara Hitchings, James Judy, Wilson RosaThis paper provides a snapshot of the findings, conclusions and recommendations of a year-long Joint U.S. Army and Air Force study on the topic of software maintenance. Using facts gathered during fact-finding, the paper focuses attention on dispelling myths about maintenance. While focused primarily on weapons systems, the conclusions offered seem generally applicable to any system, either military or commercial. The ultimate goal of our recommendations is to improve practices used for software maintenance in the field.2010Software & IT
The Economics of Cloud Computing: Addressing the Benefits of Infrastructure in the CloudAnn Repczynski, Ted Alford, Gwen MortonCloud computing is a new approach to providing IT services at a greatly reduced cost relative to current distributed IT environments. The potential savings are expected to be significant— so much so that the Office of Management and Budget is embracing cloud computing as the preferred federal IT environment. Why is this necessary? First, the federal IT budget for 2010 has grown to nearly $76 billion, and is expected to reach $88 billion for 2011. The federal government cannot maintain this spending trajectory. Secondly, the proportion of spend required simply to maintain IT infrastructure—at least $20 billion, or more than 25 percent of the 2010 budget—crowds out opportunities to invest in new initiatives and technologies...2010Software & IT
Moving Desktops to the Cloud - The costs and benefits of Virtual Desktop Infrastructure solutionsRyan Timm, Leon Halstead, Michael Richards, Justin SmithCloud computing is revolutionizing IT server operations and organizations are beginning to embrace the cloud computing paradigm for desktop computing. Cloud computing refers to users receiving IT services over the LAN, WAN, or internet from a datacenter with dynamically scalable virtualized resources. Two approaches to serving desktops from the cloud are Virtual Desktop Infrastructure (VDI) and Terminal Services-based remote desktops. These desktop replacement solutions move desktop processing and storage to the datacenter and require a client at the user’s desk which transmits keyboard and mouse inputs to the datacenter and receives display information...2010Software & IT
SOFTWARE SCHEDULE ESTIMATING LINKED TO COST ESTIMATESSarah Springer, Daniel Feldman, Greg HoganRecently there has been growing emphasis within the cost community on schedule estimating, and exploring the link between cost and schedule for Department of Defense (DoD) programs. As software development has become an increasingly important part of acquisition for the DoD, the need for better software cost and schedule models has grown. The intrinsic link between cost and schedule allows the acquisition community to adequately time phase resources to ensure you meet predicted schedules. Although a number of “commercial” models exist that will perform these functions, visibility into the data behind Cost Estimating Relationships (CERs) and Schedule Estimating Relationships (SERs) is preferred in order to build a comprehensive program model...2010Software & IT
Server Virtualization and Government OrganizationsJennifer Woolley, Sandra Williams, Ryan BoulaisServer virtualization, which can allow a single server to replace multiple servers, is rapidly becoming commonplace. As the relatively new technology has matured, various government agencies have begun to make the transition to operating in a virtual environment in an effort to cut costs and streamline data access. With virtualization becoming more prevalent, new data is available to assess the cost impacts associated with this hardware baseline shift...2010Software & IT
Capturing Risk Interdependencies: The CONVOI MethodBlake Boswell, Mike ManchisiTraditionally, risks impacting cost and/or schedules are often modeled as independent. However, it is often the case that the occurrence of one risk influences the probability of another occurring. For this reason, identifying and accounting for relationships between risks is essential to producing accurate cost estimates. Accounting for risk interdependence is often neglected by conventional risk analysis programs because risks are usually generated independently by technical experts focused on a single part of the system making it difficult to formulate and quantify relationships among risks...2010Risk
Understanding Risk in the Budgeting Process from a Portfolio Point of ViewRyan Boulais, Brett Dickey, Matt ReileyThe Office of the Director of National Intelligence (ODNI) Cost Analysis Improvement Group (CAIG) is responsible for conducting independent cost estimates (ICE) for Intelligence Community (IC) acquisitions at or above the major system acquisition (MSA) threshold of $170M. IC agencies are required, by law, to budget to the ODNI CAIG ICE or endorsed Agency Cost Position (ACP) for all acquisitions above a $500M threshold...2010Risk
JCL in a Nutshell: Exploring the Math of Joint Cost & Schedule Risk Analysis Through Illustrative ExamplesEric DrukerWith agencies such as NASA now requiring joint cost and schedule risk analysis, it is important to take time out to understand the mathematics underlying both and how it relates to their integration. Rather than focusing on difficult-to-penetrate equations, this presentation will examine the math behind Joint Cost & Schedule (JCL) risk analysis using illustrative examples. These examples will demonstrate that the combination of cost & schedule risk analysis can lead to counterintuitive results. Following these demonstrations, several recommendations will be made to the community to encourage discussion and feedback regarding this emergent methodology...2010Risk
Budget-Constrained Joint Confidence Level ApproachPeter FredericAs long as we have tried to understand and quantify the uncertainty associated with our cost estimates, we have understood that cost risk and schedule risk are highly interrelated. The current emphasis on Joint Confidence Level (JCL) analysis at NASA has focused a great deal of attention on this relationship...2010Risk
Modeling of Inflation Uncertainty in Cost-Risk AnalysisJames Glenn, Robert Meyer, Paul HardinCompletion of a thorough cost-risk analysis is a vital part of a credible cost estimate. The GAO Cost Estimating and Assessment Guide identifies seven steps which should be completed in order to complete a valid cost-risk analysis. These steps include: placement of probability distributions around WBS elements or inputs to WBS elements to capture various types of uncertainty, accounting for correlation between inputs and interpreting results from a Monte Carlo simulation model...2010Risk
Risk Identification and Visualization in a Concurrent Engineering Team EnvironmentJairus Hihn, Debarati Chattopadhyay, Robert ShishkoJPL is NASA’s prime center for deep space missions. In response to the need to reduce the cost and time to complete early concept studies and proposals JPL created the first concurrent engineering team in the aerospace industry: Team X. Started in 1995, Team X has carried out over 800 studies, dramatically reducing the time and cost involved, and has been the model for other concurrent engineering teams both within NASA and throughout the larger aerospace community. The success of Team X has also spawned new related teams such as Team I for instrument studies and in the past year the Rapid Mission Architecture Team for trade studies...2010Risk
PMAG Risk Based Integrated Program ManagementMun KwonIntroduce the risk based integrated program management framework describing major tenets behind the framework to enhance the space acquisition programs development success. The major tenets of the framework will be discussed are 1) program risk management and the decision based program risk tracking, 2) managing program management assurance 3) life cycle based system integration, 4) application oriented training. This risk based integrated program management framework is designed to primarily focus to coach and mentor the program managers and project managers through the hands on OJT to produce risk based products and services to strengthen the organic integrated program management capabilities. Key findings and lessons learned of space acquisition programs will be addressed during the briefing...2010Risk
Constructing Bounds for S-CurvesChristopher Mehl, Michael PoppAfter years of developing point estimates analysts knew were uncertain, the estimating and acquisition community has embraced the concept of viewing a cost estimate as a potential distribution of cost represented by a Cumulative Distribution Function (CDF), commonly called the S?curve. Unfortunately the very thing the S?curve was intended to counter—the implication of preciseness inherent in a point estimate—has become a preciseness about percentile values and levels of risk. Decision makers, for example, have come to view the 50th percentile as an absolute and wonder why the number budgeted at 50% is shown as 30% when the estimate is updated...2010Risk
Here, There Be Dragons: Considering the Right Tail in Risk ManagementChristian SmartThe portfolio effect is the reduction of risk achieved by funding multiple projects that are not perfectly correlated with one another. It is relied upon in setting confidence level policy for programs that consist of multiple projects. The idea of a portfolio effect has its roots in modern finance as pioneered by Nobel Memorial Prize winner Harry Markowitz. However, in three recent ISPA-SCEA conference presentations, “The Portfolio Reconsidered” in 2007, “The Fractal Geometry of Cost Risk” in 2008, and “The Portfolio Effect and the Free Lunch” in 2009, the author has demonstrated that the portfolio effect is more myth than fact...2010Risk
Build Your Own Distribution FinderAlfred SmithA basic step in a cost uncertainty analysis is to define the distribution of every uncertain element in the cost model. Identifying and then defending these distributions is a fundamental challenge. If data is available, the preference is always to perform a statistical analysis to arrive at an objective assessment. Several commercial tools are available for finding the distribution that best describes the shape and dispersion of the sample data set, but they are not in agreement.. Are they all correct? Have these different methods been subjected to an independent validation and verification (IV&V)? Is there a “best method” to derive the “best fit”?...2010Risk
Risk-Based Return On Sales (ROS) As a Tool For Complex Contract NegotiationsPeter Braxton, Richard Coleman, Michael BurtonWith the requirement in the Weapons System Acquisition Reform Act of 2009 to estimate at the eightieth percentile, both government and contractor organizations need to understand the impact on the acquisition process, in particular on complex contract negotiations. This paper will show how a risk-based approach to negotiations, simultaneously considering the government’s paramount concern of contract price and the contractor’s paramount concern of return on sales (ROS, or margin), can lead to a more mutually beneficial contracting arrangement. This paper builds on the analytical foundations of “Risk-Based Return On Sales (ROS) for Proposals with Mitigating Terms and Conditions” (SCEA, 2009)...2010Risk
AFCAA Cost Risk and Uncertainty Analysis Metrics ManualWilson Rosa, Alfred Smith, Jeff McDowell, Lew FichterThe Cost Risk and Uncertainty Handbook (CRUH), published by the Air Force Cost Analysis Agency (AFCAA) in July 2007, is a reference for approved methods, practices, and reporting requirements needed to produce a realistic, defendable cost uncertainty analysis. CRUH addresses the question “how to do it”. CRUAMM provides guidance on “what to base it on” - that is, it provides data and data-driven uncertainty distributions for potential use in your cost model or for use as cross-checks. This paper will present a status on development of this supplement to the AFCAA CRUH...2010Risk
More Trouble with Estimating at the 80th PercentileTim AndersonThis presentation addresses the recommendation within the Weapon System Acquisition Reform Act (WSARA) of 2009 that defense acquisition program cost estimates be reported at the 80th percentile of their cost probability distributions. The goal of this recommendation is commendable. Defense acquisition program costs have typically been under-estimated. Thus, it is believed that planning for cost at the 80th percentile will reduce the number of defense programs being placed in the unenviable situation of discovering that cost growth has caused the program budget to be insufficient...2010Risk
QUANTIFYING THE IMPACT OF PROPOSED RISK MITIGATION STRATEGIES: DOES THE COST TO MITIGATE EXCEED THE COST OF THE IMPACT?Philip E. BeenhouwerThe risk management process involves a concise, well defined framework for identification, analysis, and mitigation of identified program risks. In order to effectively manage risks, handling options must be established and mitigation actions defined. Typically, mitigation actions are identified and acted upon without consideration of the cost to implement these actions. Decisions of whether to assume, control, transfer, avoid, and watch the risk are likely taken with little regard to their potential effect on the risk. It is not uncommon for the cost to mitigate the risk to exceed the cost impact of the risk itself...2009Risk
Risk-Based Return On Sales (ROS) for Proposals with Mitigating Terms and ConditionsPeter Braxton, Richard Coleman, Eric Druker, Bethia Cullis, Christina Kanick, Ameya Bapat, Jeanne Callahan, Bryan CaccavaleThis paper addresses the unique challenges faced by both cost and risk analysts and decision makers when attempting to compute and convey the percentiles of cost and return on sales (ROS) for proposals with terms and conditions providing for equitable adjustments for certain components of cost growth deemed to be outside the contractor’s control, such as inflation...2009Risk
Compensating for “Lack of Tails” on Triangular DistributionsMelvin BroderThe use of triangular distributions for capturing cost risk has been popular because they have exceptional attributes for the collection of expert information on the uncertainties of cost. However triangles have one major drawback, the absolute limitation of risk at the extremes, i.e. the Low and High values have zero percent probability of exceedence. Conversely the normal or Gaussian distribution has infinite “tails” but information collection, although requiring only two parameters (mean and sigma), is often difficult to elicit from experts...2009Risk
What Percentile Are We At Now (And Where Are We Going?)Richard Coleman, Eric Druker, Peter Braxton, Bethia Cullis, Christina KanickFor decades nearly all cost estimators have held to the premise that their estimates were at the 50th percentile, and most risk analysts have built their work on the presumption that this is so. This paper will show that it is unlikely that this key premise was ever true, and will explore the ramifications of the premise being false. Among other implications, this issue bears very strongly on current initiatives by many contractors (and agencies) to allow for cost growth in their proposals (and budgets.)...2009Risk
MAKING STATISTICAL ANALYSIS ACCESSIBLE: THE RAMS TOOL FOR PERFORMING REGRESSION AND RISK ANALYSISMatt Lytton, Eric Druker, Gregory HoganThe tools for evaluating the uncertainty around cost and schedule estimating relationships (CERs) using statistical techniques remain under utilized by the cost estimating profession today. Three primary estimating techniques, the standard error of the estimate (SEE), SME judgment, and various "industry guidelines" are frequently used to assess uncertainty. These techniques can introduce bias into the estimate and lead to uncertainty being misestimated by, in some cases, orders of magnitude. The false security of high-confidence level budgets developed using these techniques can have a dramatic effect on a program’s chance of success...2009Risk
Why Not Do Project Risk Analysis Parametrically?Daniel Galorath, Evin StumpThe dominant approach to quantitative project risk analysis is troublesome for several reasons:

• Subjective probability distributions assigned to WBS elements
• Subjective correlations assigned between WBS elements
• Limitation of risks to those associated with direct charge work
• Ease of overlooking important risks
• No feedback from project experience or lessons learned...
A Simple, Universal Approach to Cost Risk Analysis: Integrating Simulations and SensitivitiesEric Hawkes, Eric DrukerWithin the last decade, probabilistic cost estimating techniques have increased in popularity as a quantitative method to bound the estimate and as a way to guard against optimism in the planning process. This paper begins with an objective look at the current techniques available to cost analysts. It then presents a new approach titled, Enhanced Sensitivity Analysis (ESA). ESA applies Response Surface Methodology concepts to the cost estimating environment. After walking through a case study using ESA, the paper then compares it with the input-based simulation approach...2009Risk
Techniques for Assessing Cost/Risk in Source SelectionsThomas Sanders, Ken Kennedy, Kristina GoldenArguably the most important decision ever made in an acquisition weapon system program’s life cycle occurs prior to any contract award. That decision recognizes a potential contractor’s understanding of the cost, schedule, and technical risk inherent in his approach to satisfying government requirements as stated in a Request for Proposal (RFP). Although it cannot be validated, many acquisition professionals agree that the selection of the best contractor has a great deal to do with the success of a program –all the money in the world may not necessarily yield a successful program...2009Risk
Risk, Uncertainty, and Trouble: Escaping the RUT of Program InstabilityColonel Brian ShimelWhen we say we are stuck in a rut, we imply our often repeated actions are forced on us by things outside our control. As if to say, we are not really as crazy as we look. But what if we build our own rut and then act as if we have no choice?

Risk and uncertainty are not what get us into trouble. We get into trouble when we ignore, or unwisely discount, risk and uncertainty. I call that Risk, Uncertainty, and Trouble—and it is a RUT of our own making...
The Portfolio Effect And The Free LunchChristian SmartThe portfolio effect is the reduction of risk achieved by funding multiple projects that are not perfectly correlated with one another. It is relied upon in setting confidence level policy for programs that consist of multiple projects. The idea of a portfolio effect has its roots in modern finance as pioneered by Nobel Memorial Prize winner Harry Markowitz. However, in two prior ISPA-SCEA conference presentations, “The Portfolio Reconsidered” in 2007 and “The Fractal Geometry of Cost Risk” in 2008, the author has demonstrated that the portfolio effect is more myth than fact. Additional cost growth data have been collected for an updated study...2009Risk
Predicting ReliabilityVirginia StoufferOne of the biggest causes of operational cost overruns is the failure of delivered systems to meet required reliability targets. Often early fielding to achieve a desired defense function precedes government testing on reliability; in those cases manufacturer estimates of reliability are accepted as truth. This paper describes how manufacturer reliability estimates were evaluated, using early fielding data. The manufacturer used a software failure rate estimating package and adjusted for field derating, thermal conditioning, and operating environments. This paper describes how government evaluators used failure mode analysis and Chi-Square testing on available data to evaluate the contractor methods and in some cases, determine an alternate reliability estimate.2009Risk
Use of Weibull Failure RatesVirginia StoufferOften one of the largest costs of an operational phase, estimating reparable costs remains more an art than
science. A number of different probability distributions are commonly assumed to underlie failure rates for predicting future reparable requirements. In the case where a limited amount of data is available on demonstrated failure rates, the Weibull can be a versatile and useful estimating tool. This paper demonstrates how observed failure rate data from a time-limited test can be adapted to a Weibull distribution for predicting future failure rates. Since the Weibull can be adjusted to represent an exponential, a binomial, or a Gaussian distribution, this paper discusses how to determine which shape is appropriate, and when to assume the shape parameter rather than estimate it. Real program data is used in examples of how the Weibull can be applied.
DON’T LET THE FINANCIAL CRISIS HAPPEN TO YOU: WHY ESTIMATES USING POWER CERS ARE LIKELY TO EXPERIENCE COST GROWTHEric Druker, Richard Coleman, Peter BraxtonKurtosis risk is the risk embodied in wider (fat-tailed) distributions. This is well known in the finance industry since the failure to account for it led to the collapse of the Long-Term Capital Management hedge fund in the late 1990s.
Kurtosis risk occurs when the probability of extreme values is underestimated in a risk model. The cost analysis community has recently begun to acknowledge this risk; fat-tailed distributions for cost risk analysis have been proposed in various papers presented over the last few years...
IMPROVING SOFTWARE COST AND SCHEDULE ESTIMATING WITHIN THE FAAWilliam S. BarfieldThe Federal Aviation Administration (FAA) initiated a data call and analysis to improve the estimation of software cost and schedule life cycles for new programs. The Air Traffic Operations (ATO) Investment Planning and Analysis (IP&A) group has taken the lead in this important effort.

Reverse modeling is a novel approach to calibrate and validate program-specific software cost estimates by using cost estimating best practices and the SEER-SEM software cost estimating tool. This technique produces a data base of sufficient size to support the development of Cost Estimating Relationships (CERs)...
2009Software & IT
Illustrative Example of Flight Software EstimationRobert GeorgiThe effort to effectively model the development of flight software for manned space is complicated because of unique approaches used by NASA. Data from previous efforts are frequently poorly documented. Also the infrequency of flight software development activities for new manned vehicles means that techniques from previous projects are not applicable to the present day due to the maturation of software languages. This presentation will explain the challenges faced in developing a parametric model-based software estimate and how these areas were addressed...2009Software & IT
Use Case Estimating 2009Bob HuntSoftware projects are notorious for being completed behind schedule and over budget. One primary cause is the inability to accurately define project size, which ultimately leads to unreasonable effort and schedule estimates. Many project managers use software modeling tools, such as SEER-SEM™, COCOMO II, and custom in-house tools for developing estimates; however, the accuracy of project effort and schedule estimates, from any modeling tool, is directly proportional to the accuracy of estimated project size. This presentation discusses the advantages and disadvantages of the Use Case concept as a requirements development tool. And, then presents a process for translating Use Case Points into size metrics, effort months, cost and schedule.2009Software & IT
Making the Case for SOAArlene MinkiewiczA Service Oriented Architecture (SOA) is a computing environment in which applications are composed, rather than developed, through a set of standard interfaces. A SOA takes advantage of networking capabilities to compose applications through the coupling of services in a way that is independent of architecture, programming language, development platform and vendor...2009Software & IT
A Holistic Approach to Understanding IT CostsArlene MinkiewiczWhen we talk to Information Technology (IT) leaders about better defining, selecting and managing their IT projects, they are quick to inform us that the projects represent less than a third of their budgets. Most of the IT budget goes toward paying existing people and support existing datacenters. What they really need is the means to perform an activity based analysis of the Total Cost of Ownership (TCO) for their IT operations. With this, organizations will have the right information to determine cost savings and additional costs associated with new projects and emerging technologies...2009Software & IT
Productivity Decline in Directed System of Systems Software DevelopmentRamin Moazeni, A. Winsor Brown, Barry BoehmThe emerging concept of Directed System of Systems (DSOS) describes the large-scale integration of many independent, self-contained systems in order to deliver unique capabilities. From Future Combat Systems to Fractionated Space Systems, these complex multi-systems are very interdependent to achieve DSOS’s objectives. The synthesis of these very large systems often results in different sets of problems than those presented by the design of a single, but complex, system usually addressed by engineers...2009Software & IT
Cost Impacts of Server VirtualizationLeah Roberts, Jennifer Woolley, Alisha Soles, Will BlackIn today’s often budget strained economy, corporations and government agencies are constantly looking for ways to cut costs, streamline data access, and improve security. As a result, newer and more efficient ways of operating, such as the use of virtualized environments, are becoming more prevalent across the IT community. Server virtualization in particular, which can allow a single server to replace multiple servers, is rapidly becoming commonplace...2009Software & IT
Optimized Project Management of Systems and Software ProjectsDenton TarbetSuccessful software projects demonstrate good project management methods incorporating modern processes and effective utilization of metrics. This paper demonstrates the application of control theory methods to project management decisions and incorporates optimization of systems trade-offs to maximize project performance. High fidelity parametric estimation models used as the Basis of Estimate for the project also provide the decision tools for a “Fully Managed Project”...2009Software & IT
Organizing IT Project InformationAlan WilliamsThis paper describes a methodology for estimating IT projects. The span of knowledge required of an IT project estimator is much greater than just being able to characterize the software development. Having an understanding of the elements that comprise the organization infrastructure, deployment concepts, and security issues as well as the complex interactions within the environment are essential to preparing the estimate. The estimation methodology described here is composed of two parts: a formalized estimation structure, and a series of “catalogues” or mini-estimates that provide defaults to use as starting points for all critical estimation elements2009Software & IT
Initial Results Building a Normalized Software Database using SRDRSMichael Gallo, Paul Hardin, Elizabeth Koza, Robert BaileyThe Software Resource Data Report provides DoD cost analysts with a primary source of comprehensive, objective, and model-independent sizing, effort and schedule data on large DoD software programs. The SRDR does not rely on rigid, standardized data fields for collecting software effort and sizing information (a feature advocated by Government, industry, and academia during the design of the SRDR). This flexibility lessens the reporting burden placed on industry and reduces ‘data synthesis’ errors...2009Software & IT
Integrating Systems Engineering with Earned Value ManagementPatrick K. BarkerThis seminar explores the inter-relationship of system engineering (SE) and earned value management (EVM). Often, project control specialists see the planning of the cost and schedule baselines as an isolated activity disassociated from the technical or engineering aspects. This type of thinking can significantly hamstring the development of the earned value baseline and likewise jeopardize the ability to control project scope. This lesson explains basic SE terms and concepts and shows similarities between SE and EVM. Hands-on requirements generation and WBS building exercises in this seminar help students recognize the value of linking the two disciplines together...2009EVM/Schedules
Anchoring EVM Analysis Approaches in the GAO GuidePatrick K. Barker, Mary Kate Lenz, Ammar ChaudryThe growing size and complexity of Defense programs and a higher level of awareness of uncertainty places an ever-increasing emphasis on the integration of risk, earned value, cost, technical performance and schedule in the Acquisition community and creates the need for new communication tools for program managers. The US Department of Defense and related Federal organizations have updated several guidance documents and policies to encourage this integration, the latest evidence being the release of the GAO Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Capital Program Costs This landmark guide places a significant amount of space dedicated to the role played by Earned Value Management (EVM), as well as other key program management disciplines...2009EVM/Schedules
Is Earned Value Management (EVM) in the future of Performance-Based Services Acquisitions (PBSA)?Anthony James ChandlerThree key objectives of the PBSA contracts are to measure, provide incentives and assist in the management of performance provided by service contractors. The dollar value the Federal Government spends on service contracts is estimated over $147B. Given the dollar value, period of performance, criticality and complexity of some service contracts, metrics are being called out to aid in contractor performance visibility...2009EVM/Schedules
PERFORMING STATISTICAL ANALYSIS ON EARNED VALUE DATAEric Druker, Dan Demangos, Richard ColemanSome Earned Value Methods, particularly those described in the equations on the DAU Gold Card, suffer from the shortcomings that they are backwards looking (they do not make a prediction of the final CPI) and do not allow for inferential or descriptive statistics. This leads to the propensity for these estimates to tail-chase as the CPI changes over time, meaning that the EAC for an over running program will systematically lag in predicting the overrun, and the EAC for an under running program will systematically lag in predicting the under run...2009EVM/Schedules
Improving Cost Estimating during Program Execution through Integrated Program ManagementWendy FreemanDuring program execution, Earned Value Management (EVM), the Integrated Master Schedule (IMS) and Technical Performance Measures are integrated with risk management in day-to-day program management. The program’s cost estimate needs to take into account program performance, including contract performance, in order to estimate the cost and schedule resources necessary to complete the program and ensure sufficient budget is available. This presentation will explore an analysis tool which links the program management disciplines together (a best practice as described in the GAO Cost Assessment Guide) in order to produce a probability distribution around the cost estimate while providing actionable information to the Program Manager. The results will be demonstrated on a Major Defense Acquisition Program.2009EVM/Schedules
Comparative Analysis of Spacecraft Schedule by Classical and Quantum Monte Carlo SimulationsDavid R. GrahamA schedule risk management methodology currently accepted at NASA is based on the represen- tation of durations or costs of risky tasks by distribution functions taken from project history or suggested by experts and using Monte Carlo simulation of the project schedule or cost estimate. Though empirical, this “expert opinion” methodology has proven efficient in providing the basic uncertainty of project cost and duration for the current plan without further mitigation. A new method of schedule analysis calculates the correlation function of randomly delayed tasks and exhibits systematic milestone delay with fully symmetric (normal) individual task distributions...2009EVM/Schedules
Cost Estimating—Back to BasicsDr. John C. BredfeldtWe, as cost estimators, are often criticized for inaccurate estimates of new weapon systems, construction projects, or program concepts. We frequently comment that a cost estimate is only as good as the definition of that being estimated and the risks inherent in the effort being estimated. However, how often does the estimator request the program content information in such a manner or in sufficiency to develop the most “credible” estimate possible. I propose to remind estimators of several micro-economic principles involved in estimating, which oftentimes are inviolate due to the nature of their being...2009Methods
Abstract: Cost Estimation as a Linear Programming ProblemKevin Cincotta, Andrew BusickWe tend to think of cost estimating as a non-linear, non-integer problem in which the quantities of items we will buy are known, but the costs of each item are unknown. But what happens when the reverse is true? In other words, what happens when every item procured is commercial off the shelf (COTS), with a known unit price...2009Methods
A Refresher On Engineering EconomicsJoe HamakerAbstract: This PowerPoint Briefing is intended as refresher training or introductory training to the discipline of engineering economics as practiced by cost analysts. Basic concepts such as the purpose and intent of engineering economics, the time value of money, the effects of inflation, the selection of a discount rate, net present value, future value, annual equivalents amounts and internal rate of return will be clearly explained in a jargon free way. The basic set up of engineering economic problems will be covered as well as tricks and tips, common errors and how to avoid them. Realistic examples will be used that cost estimators might normally encounter.2009Methods
Methods and Challenges in Early Cost EstimatingDavid HullDepartment of Defense (DoD) leadership is currently making decisions on acquisition programs much earlier in the system’s lifecycle than they have in the past, and leadership demand for cost information to support these decisions is growing rapidly. Since its inception two years ago, the ODASA-CE Early Cost Team’s primary mission has been to find ways to ensure that the cost information to support these early decisions is available and reliable...2009Methods
The System Capability Architecture: Enabling Capability-Based Cost AnalysisKatherine E. McCormack, Martha A. RoperThe current Department of Defense (DoD) cost environment requires early investment decisions. In order to improve acquisition outcomes, these early decisions require cost analysis at a time when limited program information is available. The limited program information can sometimes be just a list of desired capabilities...2009Methods
ENTERPRISE WIDE COST MODELING: A SYSTEMS ENGINEERING PERSPECTIVEYousuf S. Mohammed, Dr. Han P. BaoIn the present research the problem of Enterprise wide cost modeling is approached from a systems engineering standpoint. What this does is to use each stage of product life cycle to obtain useful information that helps in estimating the cost of the system. Once a generic framework is developed for estimating the core cost, layers of other factors that affect the cost are applied to the core cost such as risk and uncertainty, maintainability, supply-chain and socio-economic conditions...2009Methods
Weapon System Design Trade Offs: Using Life Cycle Cost (LCC)F. Quentin Redman, George L. Stratton, Andrew T. CrepeaCost Performance Trade-Off analysis is fundamental to the Systems Engineering process. A cost performance trade study, is a procedural search for a design solution that balances achieved systems performance, effectiveness and cost against the desired or required values for these features and the system. Life cycle cost (LCC) has become increasingly important as systems are designed with longer useful technological lives. Within this paper we will discuss the methodology and rational behind the methodology for using life cycle cost estimates within these weapon system design trade-offs...2009Methods
Systems Cost Engineering - Program Affordability Management and Cost ControlDale ShermonCost estimating for major engineering hardware, software, service and IT projects is critical for ensuring a realistic evaluation of the project or program, to assess its affordability, and for managing the ongoing costs realistically. Traditional cost engineering methods are time consuming, expensive and occasionally irrational. Organizations are increasingly turning to parametrics as a means of managing cost and program affordability...2009Methods
Capabilities-Based Cost Estimating: Capabilities Knowledge Base (CKB) Analysis Tools and ApplicationsChadd SibertCost analysts in today’s environment often find themselves being asked to analyze solutions identified to fill capability gaps that are poorly defined. Often, only a requirement or desired capability set is known. Is this sufficient information to produce a cost estimate? Are there any tools available to support this effort? If so, how would you utilize them?...2009Methods
A Distribution?Free Measure of the Significance of CER Regression Fit Parameters Established Using General Error Regression MethodsTimothy P. AndersonGeneral Error Regression Methods (GERM) have earned a strong following in the cost estimating community as a means of establishing cost estimating relationships (CERs) using non?linear functional forms. GERM has given rise to a wide variety of functional forms for CERs, but has so far lacked a means for evaluating the “significance ” of the individual regression fit parameters in a way that is analogous to the roles played by the t?statistic in ordinary least squares (OLS) regression...2009Models
Statistical Foundations of Adaptive Cost-Estimating RelationshipsStephen A. Book, Melvin Broder, Daniel FeldmanTraditional development of cost-estimating relationships (CERs) has been based on “full” data sets consisting of all available cost and technical data associated with a particular class of products of interest, e.g., components, subsystems or entire systems of satellites, ground systems, etc. In this paper, we review an extension of the concept of “analogy estimating” to parametric estimating, namely the concept of “adaptive” CERs – CERs that are based on specific knowledge of individual data points that may be more relevant to a particular estimating problem than would the full data set...2009Models
Parametric Cost Analysis Using Neural NetworksEdwin B. DeanRegression, in various forms, is the tool of choice for parametric cost analysis (PCA). Neural networks are a largely unused analysis tool within the cost community. However, neural networks are an effective alternative to all forms of regression for PCA. This paper will illustrate the use of neural networks for PCA and will highlight similarities and differences with respect to the use of regression for PCA. For example, regression provides a linear or transformed linear approximation of parameters for a given set of data...2009Models
Manpower cost Estimating Tool for Enhanced Online Reporting (METEOR)Thomas Demas, Anupam KumarAcquisition of major weapons systems requires an assessment of the program operating and support (O&S) costs. The largest O&S cost contributor is in the area of Manpower, Personnel and Training. To improve the quality of Total Ownership Cost estimates, the Navy recently released a new manpower estimating database/model to help cost analyst capture both the direct and indirect costs associated with Manpower, Personnel and Training...2009Models
Excessive Project Cost – Where It Comes From: A Quantum PerspectiveIlya M. Fishman, David GrahamIt is well known that project costs and durations often exceed their planned values. In the accepted practice, this phenomenon is described by asymmetric distribution functions for project tasks or WBS elements. To make predictions reasonable, Low, Most Likely and High values of these distributions are skewed to higher cost and longer durations, with distribution Means exceeding the Most Likely values. Currently, this empirical “expert opinion” methodology satisfies mounting needs for project cost and duration guidelines...2009Models
Comparing Different Methods for Deriving Cost-Dependent CERsDr. Shu-Ping HuParametric cost estimating relationships (CER) are developed using historical data. Regression analysis is used to determine whether the independent variables can help explain the variation in the dependent variable. There are two different types of independent variables in CERs: technical parameters (e.g., weight, power, etc.) and cost-dependent parameters (e.g., first unit production cost, Prime Mission Product [PMP] cost, etc.). When CERs are generated, they are based upon actual data from completed projects whether or not the independent variables are cost- dependent or hardware design-driven...2009Models
The Lifecycle Integration Framework -- Extending Affordability Simulation through Cost and Engineering Model InteroperabilityZachary JasnoffFor affordability targets to remain effective across all program phases, cost estimating models must be interoperable with engineering tools to avoid a "disconnect" between each phase of the lifecycle and affordability objectives. Initiatives such as Cost as An Independent Variable (CAIV) are enhanced through interoperability with engineering design tools giving engineers, project and business managers insight into achieving affordability goals...2009Models
Contractor Costs for Life Cycle Cost EstimatingLew Messing, Samuel HendrixThe purpose of this study is to analyze contractor costs and develop cost estimating relationships (CERs) in order to increase the fidelity and defendability of costs associated with contractors as part of life-cycle cost and program office cost estimates. At this time we are not aware of any developed methodology to estimate contractor costs for various position types or seniority levels and believe that this type of information could significantly impact the development of cost estimates throughout the DoD and IC...2009Models
Experimentation Estimating ToolkitKaren Mourikas, Denise NelsonExperimentation is a discovery process that evaluates benefits of new technologies or concepts by simulating operations and assessing the outcomes. This process provides a deeper understanding of issues surrounding the problem area, as well as more insight into cause and effect. Experimentation efforts can range from relatively small-scale efforts to highly complex events involving large numbers of participants. In order to estimate the cost of an experimentation effort, the scope of the experiment must be well understood...2009Models
An Analytical Framework for Cost Estimating Relationship DevelopmentChristian Smart, George CulverCost estimating relationship (CER) development, particularly for spacecraft hardware, has traditionally been method-centric. Much attention has been paid to parameter calculation and goodness-of-fit statistics for particular methods. However, parametric modeling involves other critical factors that are often overlooked, including model validation. These factors are discussed, and a complete analytical framework for developing CERs is presented. Three popular methods for spacecraft hardware CER development are compared in the context of this framework, namely Minimum Unbiased Percent Error (MUPE), Minimum Percent Error with Zero Percent Bias Constraint (MPE- ZPB), and log-transformed Ordinary Least Squares (LOLS)...2009Models
A Method for Estimating the Development and Production Cost of Advanced Radar Systems at the Subsystem and Component LevelDavid Stamm, Greg Bell, Paul GrimBudget constraints and high development and production costs have combined to limit the deployment of sensor systems needed for ballistic missile defense and other important military applications. To reduce the cost of new radar systems, designs must be based on reuse of components and subsystems. Similarly, in-service radars are being modified and upgraded to extend system performance and enhance supportability and sustainability. This shift in emphasis has created a need for a procedure to estimate radar modification costs and radar development and production programs where components and subsystems are reused in the early stages of the planning and budgeting cycle...2009Models
Simple Parametric Model for Estimating Development (RDT&E) Cost for Large- Scale SystemsPaul Hardin, Robert Jones, Anna IrvineThis paper summarizes OSD sponsored research regarding the development and implementation of a simple approach for estimating RDT&E cost. The approach is based on the systems engineering analysis data that is resident in system architecture products. The approach uses authoritative data that is normally available for ACAT I, II, and III programs, i.e., Information Support Plans (ISP), Department of Defense Architecture Framework (DoDAF) views, Selected Acquisition Reports (SARs), and Budget Exhibits...2009Models
Cost Analysts/Estimators—the Necessary ProfessionalsDr. John C. BredfeldtMost cost analysts and estimators came to the profession through contact with the Department of Defense either as a contractor, or as a government employee. Yet, cost analysts/estimators have unique skills that could greatly help the decision-makers in a great number of private and public arenas. This briefing will identify a number of skill sets that cost analysts should have and then identify how those skill sets could be applied for decision purposes in the both the public and private worlds. Some micro-economic concepts will be used to show how the cost analyst/estimator should consult in the private world...2009Management
Federal Appropriation Process? A Quick PrimerMichael BrozynaAfter numerous conversations with private industry cost estimators, I realized that most people have no idea how the Federal Govt appropriation process works. You have to understand how your biggest customer works. This briefing explains some of the basic concepts and shows how they relate to our business environment. The budget cycle, appropriation law, important terms and earmarks will all be covered. Go ahead; study the equation adjustments that may change the estimate outcome by a few million dollars at an Earned Value Management or Learning Curve Theory lecture. Then come see how that estimate fits into the big picture. This session will be a light hearted effort to understand the mind? numbing process that now works in the trillions of dollars. Bring your sense of humor.2009Management
An Approach to Corroborating the Impact of Recruit Quality and Recruiting Mission on Resource RequirementsDr. Robert Clemence, Jeremy Heusner, Robert Love, Raissa Nourieva, Meredith SachsThe Army Deputy Chief of Staff for Personnel, G-1 manages people and money to meet the Army’s needs for soldiers and civilian professionals. As the sponsor for all Army manpower programs and appropriations, the G-1 reviews and validates programs and funding requests, distributes people and dollars among competing requirements according to Army policy and priorities, and oversees the execution of approved programs. Manpower costs are the Army’s largest expense. As the Army increases its authorized strength, the rising costs of its recruiting and retention programs are under scrutiny by the Army leadership, the Office of the Secretary of Defense, and by the Congress...2009Management
Dan Galorath on EstimatingDan GalorathDan Galorath for over 25 years has been a leader in the estimating industry, and finally has begun to blog about his experiences and what he's still learning. This incredibly wide?ranging presentation will bring to life some of the high points of his BLOG. Dan will discuss metrics, effectiveness measurement, the real costs of a variety of undertakings, what is ITIL versus ISO 9000, the risk of IT projects, software project costing challenges and solutions, and lots more. He weighs in on a wealth of topics both timeless and contemporary.2009Management
NASA Productivity StudyJoe Hamaker, Tom Coonce, Bob Bitten, Henry HertzfeldThis study was performed at the request of the NASA Administrator and examined NASA productivity in terms of economic efficiency. The analysis examined historical trends of NASA “bang for the buck” metrics over time, and compared NASA productivity to other organizations including the Air Force, the NRO, ESA and commercial space taking into the account the relative complexity of the missions. Past studies with a similar focus were reviewed and their results summarized. Finally, recommendations for improving NASA productivity were solicited from industry, government and academia and are documented in the study. The results of the study were briefed to the Administrator in late December 2008 and a journal quality paper is being prepared for submission to the Journal of Cost Analysis and Parametrics.2009Management
Market Mapping for Production OptimizationDouglas K. HowarthThis paper describes methods for product optimization using market mapping. It is possible to optimize products by examining market conditions as over time and comparing them with evolving cost structures. Market mapping facilitates this analysis.
Products have important attributes associated with them, which includes their demand curves, prices and costs. Market demand maps reveal market boundaries and underserved price and quantity regions that producers can seek in order to improve product viability through reduced competition.

The Challenge of Estimating Space System Costs—Modeling the Relationship with Acquisition StrategyPaul S. KillingsworthIn recent years, a considerable number studies and “blue-ribbon” reports have highlighted the problems associated with the acquisition of military space systems. These problems are manifested by the fact that almost all major space programs under development by both the DoD and Intelligence Community have encountered (or are currently encountering) breaches of their cost, schedule, or technical baselines. The author hypothesizes that understanding the impact of a program cost estimate and its associated risk within the context of the complex program acquisition process may help us better understand the sources of cost growth...2009Management
PMAG’s Program Startup AssistanceColonel Mun KwonThe Program Management Assistance Group (PMAG), Space and Missile Systems Center (SMC), Los Angeles AFB, is enhancing program baseline executability. PMAG is establishing robust integrated baselines for our space acquisition programs by formulating integrated program risk analyses. As a result of PMAG’s accomplishments with the programs it supports, it is the desire of Senior Air Force leadership that the PMAG methodology be spread throughout the USAF Acquisition community...2009Management
The Lockheed Martin Aeronautics Transformed Estimating SystemCleo LilesTaking advantage of company initiatives in the areas of technical planning and earned value management, Lockheed Martin Aeronautics has instituted an initiative to re- engineer the company’s estimating processes and systems. This presentation will describe the needs that drive this action, the plan for this transformation, and the benefits that will result from its implementation...2009Management
Leadership AND Management: Two Necessary TalentsDavid PeelerRecent scholarship and over a decade of management down-sizing have emphasized the value of leadership over management. This prioritization has produced profound detrimental effects on many organizations. Most of the consequences are not fully realized by senior leadership, but are felt by so many “flattened” organizations’ employees as chaos, disorder, confusion and/or turmoil within the workplace...2009Management
Incorporating Financial Sustainability Considerations into the Planning and Design of New Facilities in the National ParksFred Richardson, Matthew TetreaultThe backlog of maintenance in the National Park System is reaching crisis proportions. The total deferred maintenance (DM) backlog was estimated at $8 to $10 billion in 2008. This problem has worsened in recent years despite a concerted effort to combat it. The National Park Service (NPS) cannot hope to limit and reduce deferred maintenance without significant increases to its budgets for the Repair & Rehabilitation and Line Item Construction programs, which combined provide less than $100 million annually to address DM. Faced with this seemingly intractable problem, park planners should heed the maxim, “First, do no harm.” ...2009Management
SOURCING STRATEGY BEST PRACTICES - BRINGING SPEND ANALYSIS TO THE DODChristopher Milo, Patrick W. Baranowsky IIDoD reliance on commercial outsourcing expanded dramatically over the past several years, growing 78 percent in the decade ending FY 2006.1 As outsourcing has grown, so too has appreciation for the potential of cost growth due to lax oversight of acquisition activities. DoD has issued a new mandate through DoD Instruction 5000.2, directing designated managers throughout all components to implement effective strategic sourcing procedures. These procedures explicitly include execution of periodic spend analyses...2009Applications/Tools
Contemporary Alternatives to Classical Risk Analysis ChallengesChris HutchingsRisks intrude on every aspect of our daily routines, whether it be through investing, playing a sport, weather forecasting, or just simple activities such as eating or driving; we are constantly in harms way.

In Project Management there are aspects of risk in several key areas, including the development of requirements, the rapid growth in technology and the instability of the operational environment, all of which ultimately adversely effect the three tangible measurements of effectiveness for any project, Performance, Schedule and Cost.
Estimating the Cost of a Depot Level Mobile Maintenance CapabilityAngela LemkeToday, the US Army is exploring options for an aviation sustainment level maintenance capability that can effectively cover initial drop in readiness. The Mobile Maintenance Capability concept provides key capabilities to sustain expeditionary Army aviation forces during operations in austere or immature theaters.

In order to support early program documentation and budgetary submission requirements, an initial life cycle cost estimate of a mobile maintenance capability was performed. Since, numerous alternatives have been proposed. This paper describes the assumptions, methodology, and results of the life cycle cost estimate for one of those options. The estimate was prepared using current Army guidance for an ACAT III program.
Migrating Calibrated PRICE Hardware Project Files to TruePlanning for HardwareJames E. OtteThis paper is intended to provide enough detail for an experienced PRICE Hardware client to understand the primary features of the True Hardware model. This paper will include a discussion of why PRICE developed a new “architecture” and Hardware estimating model
As well as;
• A brief explanation of enhanced capabilities of TruePlanning for Hardware
• A comparison of PRICE H and TruePlanning for Hardware will be closely examined.
• Explain the underlying modeling methods and assumptions of TruePlanning for Hardware
True Cost Benefit Analysis FrameworkPeter PizzutilloThis paper introduces the innovations implemented in TruePlanning by PRICE Systems. Building on the success of the industry’s first true System of Systems Estimating Framework, PRICE Systems has enhanced this framework to include cost benefits analysis capabilities. Previously, organizations needed a collection of tools to create cost/benefit models; estimate system cost and benefits, and evaluate or rate the assessment of these alternatives. The paper shows how, in a single application, heterogeneous systems can be integrated, costs can be derived parametrically or bottoms up, benefits are estimated and payback and assessment criteria are evaluated...2009Applications/Tools
Integrating Homegrown CERs in a Single Commercial Estimating ToolsetAnthony A. DeMarcoMost organizations use a combination of commercial models, public models and proprietary CERS to develop confidence in their estimates. Homegrown CERs in the form of simple cost factors, equations, curves, and rules of thumb build confidence in estimates if they are derived from known, reliable data. However, most organizations have difficulty hosting, sharing and integrating their CERs with other estimating models. Homegrown CERs are commonly hosted in arcane Excel spreadsheets, and are not easily understood, audited or reused...2009Applications/Tools
Introduction to the Joint Integrated Analysis Tool (JIAT)Daniel L. Schwartz, Melissa CyrulikA major challenge for U.S. Army cost analysts is bring together current engineering design, capability, and performance data in their cost estimating processes and analyses. While individual models and tools address these areas, it is a tedious, manual process to identify and bring together the relevant results. The Joint Integrated Analysis Tool (JIAT) is being developed by the Office of the Deputy Assistant Secretary of the Army – Cost & Economics (ODASA-CE) to address the requirement for a multi-linked system of integrated cost estimating engineering design/capability and performance model tools...2009Applications/Tools
Through-Life Estimating (TLE) – the need to estimate through program lifeArlene Minkiewicz, Dale ShermonThe objective of this paper is to introduce the concept of using one estimating tool to estimate hardware, software, services and IT programs consistently from cradle to grave using appropriate estimating methodologies.
This paper will define the term “Through-Life Estimating” and describe the challenges of estimating an aerospace or defence program over the typically long gestation period of 20 to 30 years – estimating Through-Life.
Parametrics as a Function of Project Management: A Tactical ExampleKrista StrohOver the past few years as budgets have become tighter, government agencies have been trying to do more science and research with less resources. As a result, the National Aeronautics and Space Administration (NASA)’s International Space Station ( ISS) Program employs cost estimation as a function of project management in order to make better decisions. The cost estimators and project controls professionals, through the use of parametric tools, have been able to execute effective project controls for various customers, including the program manager, project engineers, and the procurement office...2009Applications/Tools
EVMS for Program Management: Where Does It Fit?Simon DekkerThis workshop provides insight on Earned Value Management and its function within the broader scope of program management. Specific contributions of EVM to be discussed include the integrated management and analysis of scope, cost, schedule, and resources. Mr. Dekker also demystifies Earned Value Management concepts, and leads students to discover how EVM can be a natural byproduct of sound program management disciplines.2008Earned Value Management/Schedule
Utilization of EVM metrics and data in the development of Life Cycle Cost Estimates and Cost Benefit Analyses.Anthony J. Chandler, Kenneth ThomsonThis paper will help define and examine the challenges and opportunities of using EVM in the development of Life Cycle Cost Estimates (LCCE) and Cost Benefit Analyses (CBA). The purpose of the LCCE and the CBA is to ensure that programs and projects are acquired on the basis of LCC Return on Investment (ROI) or most benefit. The advent of ANSI/EIA-748 and the increased leveling of 0MB 300 requirements on programs have led EVM to the intersection ofLCCE and CBA...2008Earned Value Management/Schedule
Recognizing Earned Value Management (EVM) GamingWalt Majerowicz, Dorothy TiffanyWhy would a contractor or project manager intentionally reduce an Estimate-To-Complete or activity duration? What are the implications of your contractor keeping two sets of Earned Value Management “books,” and how would you even know about it? Under what circumstances would a contractor manipulate schedule logic? What is a “rubber baseline?” These questions and others like it raise serious concerns that all project teams using Earned Value Management should recognize. This presentation examines some common and not-so-common gaming, abuse, and data manipulation techniques that some projects or contracts may employ. At best, these techniques are errors or misunderstandings on the part of the contractor, at worst they are outright fraud and could place the entire project outcome in jeopardy.2008Earned Value Management/Schedule
Early Warning Model for Acquisition Program Cost and Schedule GrowthDr. Dan DavisThis paper uses the non-linear Rayleigh function to model the behavior underlying the problem-solving and engineering activities contained in research and development contracts and reflected in the cumulative cost accrual during the execution of those contracts. This paper examines 115 contract datasets. Using non-linear least squares with monotonically transformed and restricted parameters and the techniques of numerical computation, we examine the ability of the Rayleigh function to explain the variation of the actual cost data over time in each of these contracts...2008Earned Value Management/Schedule
Object-Oriented Estimation TechniquesLeah UpshawMost commonly available software estimation methodologies use either source lines of code (SLOC) or function point (FP) as the basis for the measurement of their size and scope. While these methodologies and tools may suffice for software projects that use structured software development techniques, it is more effective to use a methodology that takes advantage of the information provided by the use cases, scenarios, and more in depth object-oriented (OO) metrics for projects that employ such techniques...2008Software/Hardware/Tools
Improving Development Scheduling and Cost Flow in SEER-HAinsley ChongSEER for Hardware is a leading hardware cost estimating and planning system. Until recently, SEER HW had basic visibility into the scheduling of estimates, limited to annual expenditure. An upcoming version of SEER HW will include a set of new features to help program managers better schedule tasks and manage their expenditures. This effort, sponsored in part by NASA IPAO, will be part of the 7.1 release of the SEER HW product...2008Software/Hardware/Tools
A Basis of Estimate (BOE) Tool for Project EstimatesBob Fairbairn, James Miller, Rosemary BaizeThis Basis of Estimate tool is under development at NASA Langley Research Center, for use in formulating and documenting project build-up estimates, as part of an effort to improve the quality and documentation of proposals…2008Software/Hardware/Tools
Software Total Ownership Cost: Development is Only Part of the EquationDaniel GalorathSoftware development is a costly, often schedule driven activity, prone to compromises to meet schedule. Many of these compromises have far reaching impacts on the cost of software maintenance, total ownership costs and software sustainability…2008Software/Hardware/Tools
Automating SEER for Software: A Quick Cost Estimation ProcessJohn Teal, Gary HellengaSoftware estimates are useful tools for examining project scopes ranging from an entire system- of-systems program, down to a specific component of a small subsystem. In general, however, producing an estimate is neither quick nor easy. Even worse, maintaining the estimates as projects mature and undergo change is laborious – and consequently, tracking the progress of a project’s estimates over the project lifetime is rarely effectively accomplished. What is desired is an estimation process that can respond with agility to changes in project direction and to the assumptions on which the estimate is based, without undue cost and effort in updating the estimates themselves...2008Software/Hardware/Tools
A Structured Framework for Estimating IT Projects and IT SupportKaren McRitchie, Ross HuntIn today's global economy, Information Technology is a critical component of many companies’ core business operations, impacting operational efficiency, customer relationship and the bottom line. These organizations need an agile and responsive IT organization with capability to support dynamic business goals. That means IT teams today must take on a much more diversified class of projects and initiatives and to be part of the business case evaluation team to determine feasibility and advisability of not only IT project development but the companies capability for deployment, implementation and support of these initiatives through the complete project life cycle...2008Software/Hardware/Tools
Hard Disk Storage Deflation: Is There a Floor?Laura M. Friese, Allison Converse, David Wiley, Alisha Ssoles, Christopher ThomasStorage system costs are a major component of many automated information systems (AIS) cost estimates. Vendor quotes or specific storage specifications are sometimes available to estimate the cost of these storage solutions, but often, especially for significantly future storage cost projections, a $/volume of storage is often applied to projected storage volumes. Storage costs historically deflate considerably over time; Cost Deflation vs. Technology Inflation of RAID (Redundant Array of Independent Disks) Storage Systems (Converse, Watkins, SCEA 2006) supports the claim that RAID storage costs deflate 30-40% per year...2008Software/Hardware/Tools
An Approach for Building a Normalized Software Database using SRDRsMichael Gallo, Megan Holzman, Paul Hardin, Elizabeth KozaDuring the last 2 1/2 years Technomics has collected software data for the Army. The Army recognized it will be many years before current ACAT I programs complete their software developments and hence receive the associated SRDR data. Because of this 'software data gap', the Army is funding research to collect software data from Army contractors for recently completed software developments. Technomics' data collection approach is similar to that of the SRDRs: collect data in its raw unadulterated form and collect meaningful definitions of the data to permit correct interpretation of the data.2008Software/Hardware/Tools
Revolutionizing Cost Estimating with TruePlanning 2008Arlene MinkiewiczSuccessful project planning requires good cost estimating practices. Good cost estimating practices require that we think beyond the prediction of the cost, effort and schedule for hardware, software or systems.. The cost estimator must have a toolset and a mindset that facilitates unimpassioned trade-offs between scope, schedule, and budget in order to provide decision makers with all the information necessary to plan and manage the development of complex systems...2008Software/Hardware/Tools
DEVELOPMENT AND PRODUCTION COST EQUATIONS DERIVED FROM PRICE-H – TO ENABLE RAPID AIRCRAFT (MDO) TRADE STUDIESTom HarwickOne of the challenges in performing engineering design trade studies is to meet the required cycle time or trade study tempo and to have sufficient data to model the trade study alternatives for each architecture concept. Reduced-form cost equations, derived from industry cost models, provide a convenient way to allow for rapid single as well as multiple architecture aircraft studies. This capability is particularly important in a Multi-discipline Design Optimization (MDO) environment...
SE/IT/PM Estimating Approaches for Service-Oriented Architecture EnvironmentsHilary Eckberg, Kevin J. SnyderWithin the Intelligence Community, there is a movement towards breaking down stovepiped architectures and implementing service-oriented architectures (SOAs). The idea is that new programs will be integrated into an existing SOA framework, thus allowing data sharing between systems. Estimating costs, particularly Systems Engineering/Integration and Test/Program Management (SE/IT/PM) costs, for these new programs that will be sharing the system poses several problems. The questions that arise from this are as follows:...2008Software/Hardware/Tools
SEER for Hardware's Cost Model for Future Orbital Concepts (Far Out)Lee FischmanThe Advanced Spacecraft Cost Model effort (Far Out) has sought to produce a model that can, with reasonable certainty, estimate orbital concepts lying 15 years into the future. In the process it has assembled a spacecraft database of over 200 missions, 150 of them associated with cost. Far Out has sought to predict the future largely by trending forward estimating methods calibrated in progressive chronologic windows against past missions. A number of high level (from the perspective of information provided) estimating methods have been attempted...2008Software/Hardware/Tools
Managing the People Side of ChangePhil EastmanOver the last decades, many projects met technical requirements but failed to deliver their intended value to the organization. Employee resistance, declining productivity, slower than expected adoption of the solution, miscommunication and poor executive sponsorship presented barriers that were difficult for project managers to overcome.
Change management provides the missing link – it delivers a structured approach to managing the ‘people side’ of change introduced by your projects. Additionally, recent research has shown that the likelihood that a project meets its objectives is directly tied to the change management effectiveness of the project...
2008Lessons Learned
Lessons Learned on Estimating Costs for Unmanned Aircraft Systems (UAS)Charlie Kapaku, Chris TuttleThis paper/presentation is aimed at provided lessons learned on performing cost estimates for the Predator, Reaper, and Global Hawk UAS.

Paper/presentation will compare recent Mil Std 881 WBS guidance with actual estimates for the above programs and discuss unique items such as ground control stations and multiple comm links...
2008Lessons Learned
Multicollinearity – Don’t Throw The Baby (Independent Variable) Out With The Bath WaterJim ByrdOne of the cardinal assumptions with respect to Ordinary Least Squares Regression is the assumption that the Independent Variables are independent of each other. That is to say they are not correlated with each other. Collinear independent variables do not lead to biased estimates, however, they do produce estimates with relatively large variances. This is another way of saying that the estimate may be very far from its true value...2008Lessons Learned
Five Questions a Project Manager Should Ask About Every EstimateAnthony A. DeMarco
Every day, project managers make decisions on estimates. Since each decision can determine whether a project succeeds or fails, credible estimates are critical. Projects launched without a rigorous initial estimate have five times more probability of experiencing delays and cancellations. Even projects with sound initial estimates are doomed to overrun if they are not guided by management rules of thumb and formal estimates-to-complete. Therefore, project managers must be armed with simple metrics along with rigorous estimating models to be successful...
2008Lessons Learned
Cost Overruns and Defense ContractingJennifer LeottaThis paper will show that incentive fee contracts in the Department of Defense (DOD) for the development phase are not effective in eliminating the cost overrun problem faced by the DOD. This will be examined first by analyzing the optimal share ratios with the Weitzman model on two specific DOD development programs, then through a comparison of various programs with extreme cost overruns and varying contract types and fee structures. Finally, reasons explaining the cost overruns will be explored.2008Lessons Learned
Acquisition Intelligence Cost EstimatingTim EdemAir Force weapon systems have become increasingly dependent on integrated intelligence information/data. Without it, some systems cannot execute their mission. Intelligence cost estimating has been a long identified need within Air Force acquisition organizations.

Under the direction of HAF/A2, and HQ AFMC A2/5, the Intelligence Costing Working Group (ICWG) was established to develop an Air Force/Defense framework for identifying and estimating the cost of intelligence requirements. The ICWG is chaired by Air Force Materiel Command’s Intelligence Squadron, Plans & Resources Flight (AFMC IS/A2X).
2008Lessons Learned
Analyzing Cost Growth in Government AcquisitionsRichard G. Cline, Toan Nguyen, David RossThis paper studies a sampling of historical programs that have experienced varying levels of cost growth. The chief purpose is to identify (common) causes; thereby offering an opportunity to make recommendations that can mitigate those causes…2008Lessons Learned
CUSTOMIZATION OF SUCCESS ESTIMATOR TO CREATE FAR-COMPLIANT COST PROPOSALSCharlie GriffinTwo years ago, the federal government-based business unit of Parsons Corporation decided on US COST’s “Success Estimator” as its standard estimating software. Although government agencies have been a cornerstone of the Success user base for many years, the application did not provide “out of the box” cost reimbursable proposals from the contractor’s perspective that were compliant with the Federal Acquisition Regulations (FAR)...2008Lessons Learned
A Process for Translating Project Risks into Cost RiskRobert Kellogg, Robert Bitten, Debra Emmons, Randy PersingerSpace system development projects typically have processes defined to manage technical and programmatic risks. These processes usually include some combination of risk identification, quantification, and mitigation. The standard 5x5 risk matrix can be used to display the current risks and the changes over time. The 5x5 risk matrix requires the quantification of the likelihood of a risk occurring and the consequence if the risk does occur. This information may be incorporated into the project budget as liens or it may be tracked separately from the budget process...2008Risk
Inflation Risk: Assessing Inflation Risk in Multi-Year ProposalsChris Leonetti, Brian Welsh, Bethia Cullis, Richard Coleman, Peter Braxton, Eric DrukerGovernment contracts are often let for long periods of time – up to ten years or more. When contractors compete for these long-term contracts, the inflation built into the bids (or omitted!) introduces a potentially serious risk area. If the inflation numbers included in the bid are off, the contract will perform at a significantly different cost, and thus return on sales than projected, and, given the size of many government contracts, this risk may be very large in dollars as well, adversely affecting either the contractor or the customer, or both, depending upon contractual arrangements...2008Risk
What is Risk?Peter Frederic…Or more importantly, what is Uncertainty? Risk is defined as “the possibility of suffering harm or loss.” Uncertainty is generally defined as “being unsettled or in doubt” but has the more specific statistical definition of “the estimated amount or percentage by which an observed or calculated value may differ from the true value.” Cost estimators tend to use the words “risk” and “uncertainty” interchangeably, which causes confusion, especially when communicating with people outside the cost community. To avoid this confusion, it is the statistical definition of uncertainty that should be the cost estimator’s yardstick when we talk about risk or uncertainty...2008Risk
How Many Iterations Are Enough?Alfred SmithA common question regarding uncertainty analysis is: How many iterations are needed to obtain stable and reliable simulation results? The answer is always: It depends. Many factors influence the stability of the uncertainty simulation: how many uncertain elements are modeled; how many layers deep is uncertainty assigned; to what degree are uncertainty distributions skewed; what proportion of the distributions are left, center or right skewed; what is the proportion of “simple” vs. “complex” distribution shapes; how many cyclic relationships in the WBS; is correlation applied; and to what degree is functional correlation present...2008Risk
Knowledge-based, Risk-based, and Web-based Cost EstimationBrian LewisWe will introduce the Vanguard SystemTM, a new cost estimating software solution that focuses on knowledge-based generative costing models. As technology evolves and diversifies, costing by analysis (generative costing) rather than costing by analogy (parametric costing) is emerging as a critically necessary planning capability. Generative costing methods calculate the aggregate cost of an object by decomposing it into estimates for individual processes, activities, resources, or whatever fundamental units are required...2008Risk
Theory of Project Risk and Schedule Probability (Quantum Modeling)Ilya Fishman, David GrahamReliable calculation of schedule probability as a function of risk remains an extremely important and unresolved problem of project management. It is well known that real project durations might differ from originally planned by a substantial factor of 30%-100%.

Existing methodology addresses the problem by establishing task distribution functions and con- ducting Monte Carlo simulations of milestone probability. These results fully depend on the ac- curacy of task distribution functions...
U.S. Aerospace Industry Cost Risk Analysis SurveyHollis M. Black IIIThe purpose of this research paper is to summarize how the U.S. Aerospace Industry (Government and contractor) develops and applies cost risk analysis to aid business decisions. This paper is based on a survey provided to 2400 SCEA and SSCAG members in early 2008. It summarizes current risk assessment methods and tools, and the benefits to business decisions.
Survey responses (105) were received from 32 parent organizations, many with multiple sites. Organizations included 5 U.S. Government agencies, 12 major corporations, 13 support contractors, and 2 European agencies (Ministry of Defense and European Space Agency)...
Software Risk SimulationJeffrey Jaekle, Eric Druker, Emily Jessen, Jacquelyn McQueston, Brian BrophySoftware cost estimation should always address a certain level of risk within the predicted cost. Major risks within any software development effort include code growth, schedule slips, and productivity rates. Historical data can be gathered for a program that can provide insight into the average factors to use for each one of these risks. By using a Monte Carlo simulation with the statistical distributions around each of these data sets a range of possible cost outcomes can be obtained...2008Risk
Monte Carlo Simulation PracticumDavid AlleyMonte Carlo analysis is commonly used to predict the cost of future activity by acknowledging the uncertainty of the assumptions that contribute to the cost estimate and enabling the expression of those uncertainties in clever ways that bring forth and capture the subtle information that does exist...2008Risk
How to Capture Discrete Cost Risks in Your Project Cost ModelDave Graham, Alfred Smith, Melissa Cyrulik, Robert Kellogg, Robert Bitten, Debra EmmonsDiscrete cost risk analysis is defined in different ways by different cost analysts. It can mean the identification of specific risk under a specific scenario, the estimate of the cost impact due to that specific risk and/or the addition of the cost estimate for the risk into the cost estimate of the element, subsystem or system. Almost always, it can involve more than one risk under a given scenario. However, each scenario requires a risk adjusted cost estimate and there is generally only enough time to develop a few specific scenarios. NASA PA&E/CAD cost analysts are exploring simulation methods to perform discrete cost risk analysis...2008Risk
A Systems Thinking Model for Cost Growth Mitigation: Finding the Leverage PointsMichael Seibel“The incompleteness and inconsistencies of our ideas become clear only during implementation.” This quote by Dorothy Sayers says much as to how cost growth happens. It’s no surprise to the casual observer that our world is getting more and more complex. Unfortunately, we are negotiating the 21st century with a cave man mind that does not deal well with complexity. As a result, project teams continually under scope the content and difficulty of their programs...2008Risk
CUMAV or Unit? Is Cum Average vs. Unit Theory a Fair Fight?Bethia Cullis, Richard Coleman, Peter Braxton, Jacquelyn McQuestonThe statistics of the regressions used to judge significance and “best fit” are inherently biased towards Cum Average because the Cum Average metric is smoothed by the very fact that it is cumulative. This paper will use generated data with and without error terms in an attempt to show the degree of this bias and will seek to generate some thumb rules or some insight to help the analyst decide between the two models...2008Analysis
NASA New Start Inflation IndexBrian Welsh, Kenneth OdomInflation is an important factor for cost estimating and budgeting. The Office of Management and Budget (OMB) publishes annual inflation guidance for all government funded programs. In certain government programs actual inflation is higher than OMB guidance. This is especially true for “New Start” National Aeronautics and Space Administration (NASA) programs. These programs involve complex new design work requiring a specialized work force, limited production quantities (typically one), and limited opportunities for commercial applications...2008Analysis
Parametric Cost-Estimation of an Assembly using Component-Level Cost EstimatesDale Masel, Robert JuddThere are two approaches that can be taken for estimating the manufacturing cost of a product assembly. One approach is to separately estimate the cost of each of the individual components from the work breakdown structure (WBS) and summing these component costs to determine the total cost. Another approach is to derive a parametric model that uses information about the overall assembly to produce a cost estimate...2008Analysis
Taking the Next Step: Turning OLS CER-Based Estimates into Risk DistributionsChristina Kanick, Richard Coleman, Eric Druker, Peter Braxton, Matthew CainThere has been an increased focused on uncertainty and risk in cost estimates as customers now routinely desire a range of potential costs, rather than a point estimate,. Most analysts know that when performing regression analysis the uncertainty of the estimate can be derived from the analysis of variance (ANOVA) statistics; however, only rarely is this uncertainty modeled in the risk analysis. This paper will touch briefly on the concepts behind the prediction interval, and then will delve into a deeper discussion of how prediction intervals can be converted into uncertainty distributions for estimates generated from regression-based cost estimating relationships (CERs)...2008Analysis
Multiplicative-Form CERs: A Progress ReviewMitch Robinson, Eric MosierExponential and power functions, y = a?bc?x and y = a?x , have significant currency as CER
functional forms; and the ready accessibility of the Excel Solver has put a tool for fitting them to data in everyone’s hands. However, compared to their linear cousins, the mathematics of these “multiplicative CERs” is less easy and knowledge of their fundamentals is less widespread. It is thus a good time to improve our understanding of them, to ground future development and application firmly. We speak here to our initial efforts...
New Challenges for Cost Analysts: Creating Technical Baselines for Point EstimatesSamuel Toas, Joseph Frisbie, Duncan ThomasAs cost analysts are integrated into product teams earlier in program lifecycles, we are required to complete estimates and provide budget inputs well before the typical parameters of a program technical baseline are understood. Many times, these conceptual designs (either contractor designs or reference architectures) are based on immature technology rather than historically proven design solutions. This paper details how a satellite working technical baseline can be constructed using a parametric model that utilizes customer requirements, target performance, and cost objectives...2008Analysis
Estimating Support Labor for a Production ProgramJeffrey PlattenCommon methods for estimating support labor are; Percent to Touch Labor, Fixed / Variable, Semi-Variable, and Improvement Curves. The problem with improvement curves is that rate variation may require an adjustment. The problem with the other methods is that they do not significantly address variation due to program maturity. Variation in support labor due to program maturity is usually different than the variation in touch labor due to program maturity. In other words, support labor does not follow the same improvement slope as touch labor...2008Analysis
Systems Engineering Affordability Tracking SystemKaren Mourikas, Denise NelsonIs your program “on target”? How do you define “on target”? How do you measure a program’s progress? Systems Engineering Affordability Tracking (SEAT) can help you answer these questions. The SEAT System is a process and a tool to help produce a more affordable system, project, or program. SEAT goes beyond the current contractual obligations that traditional Earned Value Management (EVM) processes perform, by expanding the scope of the process and the targets tracked. All program phases from Early Development through the end of program are incorporated within the system...2008Analysis
Help or Hype? Estimating the Impact of Reverse Auction on Materials Procurement CostSandra EnserThe first Navy purchase by reverse auction occurred in May, 2000. It was touted as a means of achieving significant cost savings, and initial results looked very promising. Since then, many Government agencies have adopted e-commerce as standard operating procedure, and companies have incorporated reverse auction and web-based procurement methods at varying levels of complexity. The purpose of this paper is to define the e- commerce processes, to identify which materials are most suitable for reverse auction, to assess the “life cycle” of any savings (one time vs. ongoing reductions), and to describe some of the benefits and pitfalls of these techniques...2008Analysis
Missile Defense Agency (MDA) Booster Cost AnalysisDavid Stamm, Greg BellThe purpose of the presentation is to discuss the current fleet of MDA booster systems (employed as both interceptors and as target systems), common booster elements and components, basing, support systems, launch and range operations and cost estimating and analysis methods used to forecast, plan and budget their development, production, and operational costs. In addition to boosters, MDA payloads and launch elements will also be addressed.2008Analysis
R2 vs. r2Shu-Ping HuCost estimating relationships (CER) with multiplicative error assumptions are commonly used in cost analysis. Consequently, we need to apply appropriate statistical measures to evaluate a CER’s quality when developing multiplicative error CERs such as the Minimum-Unbiased-Percentage Error (MUPE) and Minimum-Percentage Error under the Zero-Percentage Bias (ZMPE) CERs...2008Analysis
The Percentile Problem: How Much Is Enough?Kevin CincottaIt is common practice for federal government agencies to require cost estimates at the 80th percentile. For example, the Air Force requires 80th percentile cost estimates as a matter of routine. Recently completed work for the Coast Guard also involved the mandated use of 80th percentile estimates to support budgeting. It is widely thought that this practice reduces risk, by making cost overruns less likely. The difference between the 80th percentile estimate and the point estimate (which may be a very large number, because the point estimate likely resides at a very low percentile of the true cost distribution) is often as defined as “risk dollars.”...2008Analysis
Pre-Milestone-A Cost Analysis: Progress, Challenges, and ChangeMartha RoperPre-Milestone-A Cost Analysis: It’s a relatively new concept in the defense community, but one very familiar to the early cost analysis research team at the Office of the Deputy Assistant Secretary of the Army for Cost and Economics (ODASA-CE). With over eighteen months logged in parallel research and application efforts, the time investment continues to produce dividends of both progress and salient lessons learned. More than ever, it is clear that early acquisition investment decisions must be cost-informed, and the demand for this early cost information is growing...2008Analysis
CAD Model and Parametric Cost Model Integration – A Case StudyChristopher RushThis paper explores the integration of CAD (Computer Aided Design) models and Parametric Cost Models. The author demonstrates, through means of a case study, an approach to CAD and Parametric Cost Model integration. CAD models are widespread in industry; they are an integral element of the design process in most companies. Since as much as 80% of a product cost is committed during the early stages of design, the integration of CAD and Cost models is intriguing. Such integration would seemingly provide enormous benefit to the product development process...2008Methods & Models
Combining Deterministic and Stochastic Processes in FAA Cost-Benefit EstimatingKatrina Hall, Mark KippermanFor many years the FAA has struggled to provide quality products and services to the aviation community while controlling schedule and cost overruns. Agencies such as OMB, GAO, and the DOT OIG have stated that the FAA must become more fiscally responsible in providing these products. The agency has responded by developing in- house processes and procedures that will provide the framework to achieve this directive...2008Methods & Models
The Statistical Approach for USCM9 CER DevelopmentShu-Ping Hu, Michael Pfeifer, Nick LozziThe Minimum-Unbiased-Percentage Error (MUPE) regression technique was used to develop the Unmanned Space Vehicle Cost Model, Seventh Edition (USCM7) and Eighth Edition (USCM8) Cost Estimating Relationships (CERs). This is because MUPE is a well-established technique to model multiplicative error CERs. To prepare for USCM9 CER development, the Air Force Space and Missile Systems Center (SMC) and Tecolote Research have been evaluating various statistical approaches. This effort includes selecting a regression technique, defining CER selection criteria, and evaluating cost improvement modeling alternatives...2008Methods & Models
The Traveling NET Trainer ProblemAngela LemkeAs part of providing a life cycle cost estimate for an Army program, the costs of providing New Equipment Training (NET) to a number of continental U.S. and overseas locations were examined. The proposed solution uses a traveling salesman approach to obtain a close to optimal cost solution for sending a team of NET trainers to a number of CONUS and OCONUS US Army Depot and Intermediate Maintenance locations. Stochastic optimization methods to include more traditional Monte Carlo Simulation and novel approaches such as Ant Colony Optimization can be used to optimize NET costs using common software tools such as Microsoft Excel.2008Methods & Models
Using your Cost Estimate to Answer Budget QuestionsMelissa CyrulikWe can use the budget and estimate together to answer three key questions. First, is the budget sufficient for my program and are there years with budget shortfalls? Second, if the budget changes how will it affect my program? Third, which estimate elements should be watched carefully to remain on budget? The focus of this presentation is to show modeling techniques used to answer these questions and to propose how to display the results to management...2008Methods & Models
Using C-17 Program's Data to Build a Parametric Model for the Engineering OrganizationMyung-Yul LeeThe Boeing Company, C-17 program, located in Long Beach, California has produced more than 171 cargo aircraft to the U.S. Air Force after its first flight on September 15, 1991. The Boeing C-17 Program Office has responsible for managing the Producibility Enhancement/Performance Improvement (PE/PI) contracts. The major efforts under the PE/PI contract are:
Y Perform studies and analyses
Y Design, develop, test and prototype C-17 Weapon System improvements and enhancement
Y Develop change proposal for Production and retrofit incorporation into aircraft
2008Methods & Models
Providing Visibility of the True Cost of Marine Corps MaintenanceTed KuusistoMarine Corps Logistics Command has developed a suite of decision support capabilities to assist in the analysis of materiel life cycle management activities. The Marine Corps Equipment Readiness Information Tool (MERIT) with its Total Support Cost (TSC) module plays a key role in helping the Marine Corps Logistics Command determine affordable readiness. In its graphical and tabular presentation forms, TSC displays cost and cost metrics of Marine Corps weapon and logistics systems to provide reliable investment advice to senior Marine Corps leadership...2008Methods & Models
Adaptive Cost-Estimating RelationshipsStephen Book, Melvin Broder, Daniel FeldmanTraditional development of cost estimating relationships (CERs) has been based on “full” data sets consisting of all available cost and technical data associated with a particular class of Products of Interest (PoIs), e.g., components, subsystems or entire systems of satellites, ground systems, etc. In this paper, we extend the concept of “analogy estimating” to parametric estimating by deriving “adaptive” CERs, namely CERs that are based on specific knowledge of individual data points that may not be reflected in the full data set at the time that the original CER was derived...2008Methods & Models
Translating Cost Models from Excel into ACE: Best Practices and Lessons LearnedSam BresnahanTranslating life cycle cost models from Excel into ACE often proves to be a daunting task, especially for analysts who are novice or intermediate-level ACE users. This session will recommend a general approach to the translation process and identify several ACE best practices: combining Dynamic Equation Columns (DECs) and the SumIf( ) function to manage hardware configurations; accessing lookup tables with relative row addressing; and using the OpFieldedUnits( ) function to phase Operations & Maintenance costs...2008Methods & Models
Identifying Best-Value Technologies Using Analogy-Based Cost Estimating Methods and ToolsMark Schankman, John ReynoldsDefense and Aerospace technology planners and engineers are faced with a challenging problem: how to select from many alternative technology research & development projects to provide the best value (meet needs at lowest cost). The problem is complex because:
• Defense and Aerospace needs depend on many performance requirements where each requirement requires the combination of several system attributes (eg, lethality, range, speed) each with a resulting cost impact.
• New technologies must be identified and developed to provide the required performance.
• The cost to develop these technologies is difficult to estimate due to inadequacies of existing cost estimating tools and methods.
2008Methods & Models
Estimation of Non Recurring Costs in ManufacturingRonal Bradbery, Linda Newnes, Antony MilehamThis research aims to improve estimation of the non recurring costs (NRCs) of manufacture. Investment required prior to the first unit of production varies depending on the industry. However shorter production runs mean that manufacturing investment levels are increasingly relevant. The larger body of cost estimation research is focused on modelling product unit cost. Much cost estimation practice is informed by experience and understanding of costing relationships. The less frequent development of new manufacturing capacity limits options for research and the development of bodies of expert knowledge in the estimation of NRCs...2008Methods & Models
Quantum Modeling of Project Cost and ScheduleIlya Fishman, David GrahamIn conventional cost estimates, project cost is a stochastic sum of individual tasks costs. For the purpose of cost estimate, tasks could be performed in arbitrary sequence thus making the sched- ule links irrelevant. This disregard of task coordination and communication may be responsible for up to 40-50% of the total project cost...2008Methods & Models
Comprehensive Costing Model for WATer and Environmental Research Systems Network (WATERS Network)Akshay Raut, Jaineel Aga, Thariq Kara, Jeffrey GlassThis presentation will review the development of a preliminary costing approach and model for a large scale, multidisciplinary scientific project being developed by numerous principal investigators across the country: the WATERS Network. This project is a joint initiative between the hydrologic science and environmental engineering research communities and funded by the National Science Foundation (NSF) to provide opportunities to further the scientific understanding of water as an integrated resource and the effects of anthropological activities. It is a bold environmental observatory initiative to transform research on the water environment by investing in new infrastructure and facilitate investigations (WATERS Network, 2008)...2008Methods & Models
A Non-Simulation Based Method for Inducing Pearson’s Correlation Between Input Random VariablesEric Druker, Richard Coleman, Peter Braxton, Joel HughesSeveral previously published papers have cited the need to include correlation in risk analysis models. In particular, a landmark paper published by Philip Lurie and Matthew Goldberg presented a methodology for inducing Pearson’s correlation between input/independent random variables. The one subject absent from the paper was a methodology for finding the optimal applied correlation matrix given a desired outcome correlation. Since the publishing of the Lurie-Goldberg paper there has been continuing discussion on its implementation, however there has not been any presentation of an optimization algorithm that does not involve the use of computing-heavy simulations.
This paper reviews the general methodology used by Lurie and Goldberg (along with its predecessor papers) and presents a non-simulation approach to finding the optimal input correlation matrix given a set of marginal distributions and a desired correlation matrix.
2008Methods & Models
Big Iron EVM: Implementation at AEDC – Initial Success Met FailureDave Peelerrnold Engineering Development Center (AEDC) had “experimented” with Earned Value Management (EVM) for several years, under its support contract. With the source selection for FY04 consolidating the test and support contracts, several personnel recognized the opportunity to bring the entire center under the rubric of EVM. Understanding costs and particularly schedules has long been a supreme challenge to managers at AEDC. A focus on EVM would produce greatly enhanced insight into the test and maintenance aspects of Center operation. EVM had certainly helped transform the mission support aspect of things at AEDC...2007Earned Value Management
New Heavy Launchers, a Cost Estimating ChallengeMichael van PeltEstimating development and recurring cost for a new heavy launcher pose some challenges. It has been a long time since Saturn V was last flown, and cost estimate models based its development and launch cost may not accurately predict the effects of the current state of technology or development and production methods...2007Earned Value Management
Ending the EAC Tail Chase - An Unbiased EAC Predictor using Progress MetricsEric Druker, Richard Coleman, Jeff Jaekle, Elisabeth BoyadjisThe world of EVM has traditionally produced results that are unreliable – specifically, traditional EAC predictions are biased low; they are “in a tail chase.” Many papers (by Christensen and others) show that all standard CPI and SPI-based EAC predictions are in essence lower bounds. While producing estimates for a customer whose EACs have traditionally tail-chased from start to finish, the authors stumbled upon a method of that takes into account the performance of completed programs of a similar nature...2007Earned Value Management
A Simplified Earned Value Management ApproachDorothy TiffanyMany organizations, especially agencies and departments within the federal government, are struggling with requirements to implement Earned Value Management Systems quickly. At the same time, issues such as culture, training and system architecture, have stymied those trying to respond to these new requirements in a timely manner. This presentation identifies many of the roadblocks to implementing EVM, and provides some ideas to bridge the gap until fully compliant EVM systems are attainable. Some very basic EVM principles and analysis will be discussed to show that simplified EVM may not be that hard to implement, after all. 2007Earned Value Management
DoD Military Construction Inflation Index vs. Construction Industry Inflation IndicesBrian Welsh. Vince RussoDesigning and constructing a building is a very involved, detailed, and long term project. Buildings are made of many different commodities that fluctuate due to external conditions such as supply and demand forces, political events, and trade agreements. To help plan and estimate the cost and duration of a building, indices are used to determine the future price of construction materials based on previous prices. Department of Defense construction projects are recommended to use the Total Obligation Authority Military Construction Inflation Indices published by the Office of the Undersecretary of Defense...2007Earned Value Management
Accurately Estimating Class C Spacecraft BusDuncan Thomas, Meghan Connelly, Greg HoganThis paper quantifies the cost and explanatory technical differences between Class A and Class C space vehicles, both of which are often included in space cost databases. This effort was completed by the Air Force Cost Analysis Agency (AFCAA)…2007Earned Value Management
The Evolution of Hardware EstimatingArlene MinkiewiczJust as Nintendo evolves gaming and Apple innovates telecommunications, PRICE Systems examines how hardware estimation evolves as technology improves at breakneck speeds. This paper presents the findings of a study on the impacts of improving technology on the development and delivery of hardware and hardware systems...2007Earned Value Management
Automatic Layout of Hardware Effort/Duration Estimates Into Microsoft ProjectLee FischmanWith the introduction of the SEER-H Client for Microsoft Project, Galorath has mated parametric hardware cost estimation with a leading project planning solution. The "H Client" permits SEER-H projects of any size to be exported into Microsoft Project using either default or customized lifecycle templates, permitting estimators to evaluate and modify detailed schedules. This presentation demonstrates this capability and provides insight into how it was achieved.2007Earned Value Management
Simple is Not Necessarily Better:  Why Software Productivity Factors Can Lead to Bad EstimatesMike Gallo, Paul HardinThe use of simple linear productivity factors to estimate software development costs is rampant in DoD. Although these factors are easy to explain and use and are (usually) grounded on historical data, they nevertheless contain several cost estimating risks including:...2007Software
Software Code Growth: A New Approach Based on Analysis of Historical ActualsRobert Jones, Paul Hardin
There are numerous methods available for estimating the growth in software source lines of code (SLOC). Most simply increase New SLOC, Modified SLOC, and Reuse SLOC by growth factors regardless of the characteristics of the SLOC or the distribution of New, Modified, and Reuse...
To support a Make/Buy Business Case to Successfully Respond to an Inspector General (IG) AuditSavita ChoudhryThis document provides an overview of the service PRICE Systems has provided to a Federal Program Agency to support a major IT modernization Project, which helped the Federal Agency pass an Inspector General (IG) audit. The selected case study was a Make/Buy analysis. Because the case study involves an analysis conducted to respond to an IG audit, the paper will review the methodology and steps undertaken by both the government and contractor from completing and delivering the estimate, analyzing the benefits of the COTS implementation scenario, answering specific detailed questions to the audit, to receiving Program Office acknowledgement of the tremendous value of the Cost Estimation (in accordance with GAO guidelines)...2007Software
Understanding the Sensitivity of PRICE and SEER to the Software Development Project CSCI ArchitectureAbbey TurnauSoftware development effort is often estimated using commercial tools such as PRICE and SEER. Obtaining accurate estimates is an important key to program planning and control. Size inputs are organized into Computer Software Configuration Items (CSCIs) and Computer Software Components (CSCs) typically based on the architecture designed by the systems engineering team. Often times when a third party is performing an independent cost estimate, access to information at the CSCI level is not provided.
Without this information, the third party analysts must make assumptions about the architecture...
Applying parametric cost modelling to Technology InsertionArthur Griffiths, Dale ShermonThe paper will describe the UK MOD need to develop a Whole Life Cost modelling tool to enable the exploration and justification of up-front investment in different acquisition scenarios for characteristics that facilitate subsequent Technology Insertion.

This paper will describe a case study regarding how to apply parametric cost modelling to the attributes which are associated with Technology Insertion. The case study applies this modelling to an Unmanned Combat Air Vehicle (UCAV), although it is equally applicable to all generic system types...
Software Total Ownership Costs Development is Only Part of the EquationDaniel GalorathSoftware development is a costly, often schedule driven activity, prone to compromises to meet schedule. Many of these compromises have far reaching impacts on the cost of software maintenance, total ownership costs and software sustainability.

Maintenance itself is often treated as a level of effort activity where insufficient staff are asked to do too much and where a desired level or maintenance rigor is impossible to preserve. Unlike hardware, software doesn’t wear out but with inadequate maintenance it just becomes functionally less useful and less reliable over time...
A Process for Mapping COCOMO Input Parameters to True S Input ParametersJames OtteOversight review panels for major programs require software cost analysts to complete software estimates using several software different cost models. This requires the software cost analysts to collect data for various cost models.

This paper will provide a comparative survey and analysis using NASA flight projects. The paper is an extension of the Ray Madachy, Barry Boehm and Danni Wu paper completed by USC on the same subject. This paper provides a road-map for mapping COCOMO input parameters to True S input parameters. This will enable a software cost analyst to complete two or more software model estimates using one set of input parameters.
Estimates and Measurement for a Software Product LineDavid SeaverIn today’s environment where technology rapidly changes, an increasing number of companies, agencies and organizations are realizing that they can no longer afford to develop multiple software products one product at a time. They are being pressured to deploy new products or add functionality to existing ones at a rapid pace. Simultaneously, companies must develop high quality products that minimize the cost to maintain, make effective use of very limited resources, compensate for an inability to hire, and achieve mass customization...2007Software
Understanding Service Oriented Architectures and Their CostsKevin Snyder, Janelle McDonaldService-Oriented Architecture (SOA) appears to be the next generation of IT architecture. The technology is growing rapidly in both commercial and government markets, with new products being delivered almost monthly. The Director of National Intelligence (DNI) is pushing for SOA implementations of enterprise architectures within the Intelligence Community. Recently, the DNI’s Chief Information Officer released strategy and security documentation, as well as a reference model, for SOAs. Although the community is aware that SOAs are the way of the future, little is known about the costs associated with this implementation and subsequent maintenance...2007Software
Software Estimation Through The Use of EV DataJeff Jaekle, Justin Greene, Eugene Cullen, Eric DrukerService-Oriented Architecture (SOA) appears to be the next generation of IT architecture. The technology is growing rapidly in both commercial and government markets, with new products being delivered almost monthly. The Director of National Intelligence (DNI) is pushing for SOA implementations of enterprise architectures within the Intelligence Community. Recently, the DNI’s Chief Information Officer released strategy and security documentation, as well as a reference model, for SOAs. Although the community is aware that SOAs are the way of the future, little is known about the costs associated with this implementation and subsequent maintenance...2007Software
System of Systems Cost Estimation SolutionsJosh WilsonAn area in which several advocates are attempting to improve industry understanding of system of systems relates to estimating the cost and effort required to deliver capabilities rather than platforms. Currently, the significant offerings in this and closely related realms include COSYSMO, COSOSIMO, SECOST, SEER-H with TSV, PRICE PES,
and PRICE TruePlanning Systems. This paper explores the history of each of these models, the origin of their methodologies and cost estimating relationships, the major size and cost drivers, and the extent to which each solution addresses the need for a system of systems estimating capability...
Software Development Schedule Realism: Prediction Band ToolAlison Converse, Eric Druker, Jeff JaekleKnowing the probability of achieving an originally proposed schedule, as well as updated schedule proposals throughout a contract, provides decision makers insight when considering schedule changes, risk mitigation plans, funding and other decisions. The purpose of a schedule realism tool is to provide this insight by determining the probability that the software development schedule proposed will be met, as well as information about the distribution of the proposed schedule.
The paper will present the research and methodology behind a schedule realism tool which utilizes Prediction Bands. The tool will be based on historical data from the Automated Information Systems (AIS), but the methodology will extend to any program with sufficient historical data.
Pioneers of ParametricsRicardo ValerdiThis paper provides a historical account of the development of the field of Parametrics through information obtained during interviews with twelve pioneers of the field. Cost model developers, users, and practitioners were interviewed with the intent to capture their views on the impact between cost estimation research and practice. The individuals interviewed represent a diverse range of perspectives including academia, government, and industry. Each perspective sheds light on the areas in which the field of Parametrics has had an impact and which synergies have been influential in the development of the field. The implications of the findings are discussed in light of the future challenges for the field of Parametrics.2007Management & Lessons Learned
A Cost Control methodology designed to apply for defense acquisition of South Korea.Kyu Meyung Choi, Jongyou OnDefense system as a long lead program has the high probability of cost growth due to the uncertainty and risk. Therefore, meeting schedule, technical performance and preventing excessive cost overrun is the main concerning point to success defense program. CAIV, Cost as An Independent Variable, is taking shape in Department of Defense Acquisition of USA. The policy to adapt CAIV is to place more emphasis on cost control. The goal of CAIV is to shift the paradigm from “more performance is better” to “an affordable balance between cost and performance is better.”...2007Management & Lessons Learned
Cost Estimating Challenges for the Constellation ProgramChristian Smart
NASA’s Constellation program has introduced numerous cost estimating challenges. This new program is NASA’s first large-scale development program of a new launch vehicle since the Space Shuttle.
The Constellation program is similar to the Apollo program, especially with regard to hardware. However, the Apollo program began development in the 1960s. Thus application of the analogy-based cost methods that estimators often prefer poses significant issues for Constellation. NASA has met this challenge by developing and implementing multivariate cost- estimating relationships (CERs)...
2007Management & Lessons Learned
The Portfolio Effect ReconsideredChristian SmartIt has recently been argued that funding for multiple missions benefits from a portfolio effect – that is, that the various risks of cost overruns for individual missions is less when considered as a group. This is based on modern financial theory that was originally developed by Nobel laureate Harry Markowitz. The result of this diversification, in theory, should allow NASA the ability to fund individual missions at, for example, the 61st percentile, while achieving an overall confidence that is much higher, such as the 80th percentile...2007Management & Lessons Learned
Counterinsurgency Aircraft: An Analysis of Two AlternativesDave PeelerWithin military academic circles and the special operations community, a call is being made for an aircraft dedicated to the Counterinsurgency (COIN) mission. Given the Air Force’s budget constraints brought on buy the dedication to the F-22A, advocacy for a COIN aircraft needs the concreteness of hard numbers. Building on Arthur Davis’ COIN aircraft advocacy paper, this research focuses not on further advocacy, but on a process and method to actually procure a COIN aircraft. The acquisition focus is on US Special Operations Command’s acquisition authority coupled with its GWOT mission responsibility, using commercial-off-the-shelf aircraft procurement to specifically address the need for an airborne COIN capability...2007Management & Lessons Learned
Cost, Technical, and Acquisition Differences Between Military and Commercial Communication SatellitesDuncan Thomas, Meghan Connelly, Linda SnowThe purpose of this briefing is to explain differences between commercial and military communication satellite acquisition and how these differences impact development and production costs.

The goal of Air Force satellite acquisitions is to maximize operational capabilities while minimizing development and procurement costs. While sound in concept, these goals often end up being contradictory. One overarching contribution to this is the common misconception within government satellite acquisition community concerning the applicability of commercial practices to meet operational requirements...
2007Management & Lessons Learned
Realities of Cost As an Independent Variable (CAIV) - Stakeholder PerceptionsGreg Kiviat, Stuart Swalgen, Sebastian BottaCost As an Independent Variable (CAIV) has been shown to be an effective tool to identify and mitigate program risk by balancing effectiveness and affordability. CAIV establishes goals and provides cost feedback to development teams as requirements change and design details emerge. However, depending on the project phase (Concept, Technology Development, SD&D, Production and Operating & Support) and the level of stakeholder support (both customer and contractor), there is great variability in the level and scope of CAIV application and investment in cost analysis capability...2007Management & Lessons Learned
Implementing a robust cost estimating processHerve JoumierHow do cost estimating entities bring value in an organisation and how can survive such groups through time?
These questions may not have easy answers and many counter-examples of short existence are observed both in industrial companies and procurement agencies. They tend to resurface later on in different forms but in most of the cases this reshuffling is accompanied with significant loss of experience, qualified personnel or even loss of reference data and cost models...
2007Management & Lessons Learned
Turning Avoidable Guidelines Into Sensible Requirements– Credible Space Cost Estimating PolicyJill HardashIn the last five years NASA has experienced a renaissance in cost estimating. The small but remarkable crew of remaining estimators at the Agency started an effort to rebuild the cost estimating capability at all of the Centers, including NASA Headquarters. This involved growing the number of trained cost estimators, educating cost estimating customers and providing guidance on the discipline of cost estimating at NASA. Without these three components credible cost estimates within NASA would not be possible. Five years later, with a robust cadre of trained estimators, how has the cost estimating message been delivered to customers?...2007Management & Lessons Learned
GAO’s Cost Assessment GuideKaren Richey, Jennifer Echard, Carol ChaThe U.S. Government Accountability Office (GAO) is responsible for, among other things, assisting Congress in its oversight of the federal government, including agencies’ stewardship of public funds. Legislators, government officials, and the public want to know whether government programs are achieving their goals and what their costs are. The capability to generate reliable program cost estimates is a critical function necessary to effectively use public funds and to support the Office of Management and Budget’s (OMB) capital programming process. Without it, agencies are at risk of experiencing cost overruns, missed deadlines, and performance shortfalls—all of which are recurring problems that our program assessments too often reveal...2007Management & Lessons Learned
Cost Estimating and Management in Virtual Manufacturing NetworksCharlie Stirk, Dennis ThompsonA virtual manufacturing network is a loosely coupled enterprise, which is formed by many partners (whole or parts of real companies) enabling a group of individual real enterprises to operate more efficiently and effectively, as if it is a single global enterprise to fulfill a specific mission. We focus on the costing capabilities needed to enable two particular types of network centric solutions: rapid response to manufacturing legacy parts, and manufacturing readiness and launch of new products/processes. For legacy parts where the original supplier may not be available, a new manufacturing network must be rapidly assembled to reverse engineer, plan the manufacturing processes and supply chain, and estimate lead-time and cost...2007Management & Lessons Learned
Myth Buster: Do Engineers Trust Parametric Models over Their Own Intuition?Ricardo ValerdiThis paper explores the question of whether engineers trust parametric models over their own intuition when performing cost estimates. The approach to answering the question is similar to that of the popular television show MythBusters which aims to separate truth from urban legend using controlled experiments. We performed a controlled experiment with forty systems engineers and discovered that most of them trust their intuition more than a parametric model. This challenges a central assumption in the field of parametrics: that decision makers mainly base their cost estimation decisions on parametric models. We explain how this phenomenon can be scientifically possible based on theories of cognitive decision making.2007Management & Lessons Learned
Improving the Cost Estimating Capability at the FAAKatrina HallFor many years the FAA has struggled to provided quality products and services to the aviation community while controlling schedule and cost overruns. Agencies such as OMB, GAO, and the DOT OIG have stated that the FAA must become more fiscally responsible in providing these products. The agency has responded by developing in- house processes and procedures that will provide the framework to achieve this directive...2007Management & Lessons Learned
The Evolving Launch Vehicle Market Supply and the Effect on Future NASA MissionsRobert Bitten, Debra Emmons, Claude FreanerThe upcoming retirement of the Delta II family of launch vehicles will leave a performance gap between small expendable launch vehicles, such as the Pegasus and Taurus class, and larger vehicles, such as the Delta IV and Atlas V families. This performance gap may lead to a variety of progressions including large satellites that utilize the full capability of the larger launch vehicles, medium size satellites that would require dual manifesting on the larger vehicles or smaller satellites missions that would require a large number of smaller launch vehicles...2007Management & Lessons Learned
COLLABORATIVE ESTIMATING, RESPONSIBLE, ACCOUNTABLE, COMMUNICATED AND INFORMEDShamraz RazzaqFrom their inception, cost estimating models have been operated by individual users interpreting the technical and programmatic information that they have gathered into the cost models for which they have been trained. However, there is now an opportunity for a change in this culture and the way in which Parametricians interact with each other and their customers...2007Management & Lessons Learned
Challenges to Integrated Cost and Performance/Effectiveness AnalysesKurt Willstatter, Andy CampbellThe defense acquisition community has made significant investments into enabling simulation of potential military systems early in the concept development phase. Furthermore, additional challenges in acquisition modeling have been imposed due to military requirements migrating from specifying systems to specifying capabilities. As such, analogous pressures have caused perturbations within the cost analysis community, especially to facilitate decision- making on future investments in a budget-constrained environment. Estimating the cost of concepts and/or capabilities requires new ideas and approaches to address accounting for sunk costs, sharing costs across multi- capability systems and multi-theater systems, and time- phasing of costs across limited numbers of assets. This presentation provides practical examples to address these issue(s).2007Management & Lessons Learned
Life Cycle Cost Normalization for ComparabilityKirk Hoy, Andy CampbellSystem concept contractors are increasingly being tasked to formulate life cycle cost (LCC) estimates on their respective offerings. These estimates are often started during the concept development stage of the system acquisition life cycle. From the US Government standpoint, comparability of LCC data among competing vendors and its own internal LCC modeling is a fundamental goal. One way to accomplish this is through early creation and publication of detailed cost groundrules and assumptions that structure the LCC data/estimates being provided to simplify the comparisons and reduce much of the guess work...2007Models
Update on the Jet Propulsion Laboratory Parametric Mission Cost Model (PMCM)Erik Monson, Leigh RosenbergIn today’s dynamic, fast-paced world of unmanned space exploration, priorities can, and often do, change rather quickly. A single breakthrough discovery can quickly shift community focus to an entirely different direction. Scientists and engineers involved in the early conceptualization of unmanned missions must have the tools and procedures at their disposal to keep up with such changing priorities, while conforming to strict cost and schedule limitations. Parametric cost estimation is an increasingly important tool for use in the fast-turnaround world of mission studies...2007Models
Propulsion System Advanced Cost ModelRoy Smoker, Dan FeldmanMCR has developed an Advanced Cost Model for The Air Force Research Laboratory Space Vehicles Directorate (AFRL/VS) at Kirtland AFB, NM, to operate in concert with the Integrated Propulsion Analysis Tool (IPAT) developed by Advatech Pacific, Inc. IPAT is a conceptual design tool that can provide the physical mass properties of any new launch system that could come off the drawing boards during the next 20 years...2007Models
COINCOMO The COCOTS Integration ChallengesA Winsor Brown, Vu NguyenThe continued vision for the COINCOMO Tool has been to integrate all of the elements of the COCOMO "suite" of software development estimation tools, including the COCOMO II Charts [extension], COPSEMO, COSECMO, CORADMO and/or COAglMO, and COCOTS; and the systems engineering extensions COSYSMO and COSOSIMO. Note: COAglMO is a proposed new extension to COCOMO II similar to CORADMO, but utilizing Agile methods during the construction phase (which assumes that the products overall capabilities and performance requirements have been documented and an adequately documents architecture developed to address those requirements...2007Models
Modeling the Life Cycle Cost of Protecting US Commercial AircraftKirk Hoy, Kurt WillstatterIn support of the Department of Homeland Security (DHS) Counter-MANPADS program, Summit Engineering Group designed and constructed a detailed Life Cycle Cost (LCC) model to address various potential deployment alternatives. Summit Engineering Group leveraged the detailed cost groundrules and assumptions document it created, data from various DHS contractors, and other related data in constructing the comprehensive, releasable LCC model. This presentation provides the structure of the LCC model, rationale for it, and lessons learned in addressing US commercial aircraft costs.2007Models
Integration of Cost History with Project Management Data for Improved Cost EstimatesDavid Graham, Dan WalkovitzNASA is working to improve estimates-to-complete (ETC’s) for projects as they evolve by integrating regularly produced EVMS data from these programs with NASA’s CADRe system which captures historical cost, programmatic and technical data. This integrated body of data will be used during and at each transition between NASA Phases A through E: Concept Development – SDR; Preliminary Design - PDR; Detailed Design– CDR; Fabrication, Assembly, and Test – Launch; Operations and Sustainment - Disposal to generate an enhanced ETC...2007Models
Parametric O&S Cost Estimation Using Markov Chains and the Influence Function MethodEvin Stump, Alexandra MinevichThis paper reports on an effort at Galorath Incorporated to apply parametric methods to the system O&S estimation problem. As a goal, we wanted to reduce the number of user inputs as much as possible in keeping with maintaining decent accuracy.
We also wanted a high degree of model flexibility so that the same general architecture would suffice for most kinds of vehicles and infrastructure. After some preliminary study, we decided that:...
Using Bottoms-Up Cost Estimating Relationships in a Parametric Cost Estimation SystemDale Masel, Robert JuddThe parametric approach to manufacturing cost estimation is preferable because of the limited amount of part data required to generate an estimate. However, the limited data means that the estimate cannot achieve the accuracy of a bottoms-up estimating system. A hybrid cost-estimating system has been developed and implemented to estimate the cost of jet engine components. The system utilizes a bottoms-up estimating approach for the actual cost estimates, but appears as a parametric system to the user. The inputs needed for bottoms-up cost- estimating relationships (CERs) are calculated from user-specified parameters based on relationships derived between part parameters and feature attributes.2007Models
Forecasting Operation & Support Costs For Missiles and MunitionsJerry Le MayOperation and Support (O&S) costs are an important component of Life Cycle Cost for military weapon systems. Accurate forecasting of O&S costs early in the program allows the customer and logistics services provider to construct the most cost effective support program, providing high levels of weapon readiness while saving the customer money. Missiles and munitions have a deployment and operations profile that is different from other weapons systems in that 97% or more of their available service life is spent in protected storage in a dormant state...2007Models
Capabilities-Based Costing: Approaches to Pre-Milestone-A Cost EstimatingMartha RoperThe issue of early, rigorous evaluation of program costs is becoming more important as defense funding come under greater scrutiny. Often at this point in the life cycle, a requirement or desired capability is known, but the manifestation of the solution is unknown or described only at a high level. Can capabilities alone be used to produce a cost estimate? If so, how can we link the proposed solution to existing systems if only a particular solution’s general capability set is known?...2007Models
Cost Modeling Data Standards for Model IntegrationJeff McDowellThe objective of the work described in this paper is to develop data exchange mechanisms necessary to revolutionize the efficiency and accuracy of cost estimating through automated data exchanges with design and performance models and simulations. This development is intended as an early step towards fully integrating cost estimating into the modeling and simulation (M&S) domain, enhancing the prediction of the affordability of future systems, and the understanding of the costs of alternatives courses of action and operational scenarios...2007Models
Defense Acquisition Performance Assessment – The Recommendation for Time Certain Development: Pipe-dream or Reality?Peter HantosAs a significant milestone in the DOD’s continuous self-assessment process, the Defense Acquisition Performance Assessment report was released in early 2006. A far-reaching recommendation of the report is the introduction of Time Certain Development as a preferred acquisition strategy, making schedule a Key Performance Parameter. By mandating high confidence cost and schedule estimates, two arms of the well-known “Iron Triangle” (Cost, Schedule, and Performance) become fixed, leaving only requirements in the project management trade-space. The paper’s objective is to offer engineering perspectives and a discussion of the underlying, parametric estimation issues that impact the successful implementation of the report’s recommendation.2007Models
Why ZMPE When You Can MUPE?Shu-Ping Hu, Alf SmithMultiplicative error terms are commonly used in the cost analysis field because experience tells us that the error of an individual observation (e.g., cost) is generally proportional to the magnitude of the observation rather than some fixed amount. In such cases, it is appropriate to hypothesize a multiplicative error term for a cost estimating relationship (CER). The Unmanned Space Vehicle Cost Model, Eighth Edition (USCM8) CERs were developed using the Minimum-Unbiased-Percentage Error (MUPE) method to model multiplicative errors. The MUPE method is also known as an iterative, weighted least squares regression...2007Methods
Kinetic Energy Interceptor (KEI) Affordability ProcessTim ChristophersonKEI is a development program in the emerging National Missile Defense system. An Affordability process was developed which leverages Design For Six Sigma (DFSS), Design for Manufacture and Assembly (DFMA), and Cost As an Independent Variable (CAIV).
The emerging process was applied to a recent trade study, and a “knee in the curve” chart was developed using Launch Area Denied (LAD) as the measure of performance and Development and Test (D&T) cost as the independent variable. A step by step methodology is presented with unique, real-world results.
An Enterprise Model of Rising Ship Costs: Loss of Learning Due to Time between Ships and Labor Force InstabilityRichard Coleman, Jessica Summerville, Peter Braxton, Eric Druker, Peter BraxtonSince the end of the Cold War, the perceived need for Navy ships has dropped, and so the shipbuilding budget has dropped. Seemingly coincidental with this budgetary pressure, and perversely aggravating the problem, ship costs began to rise steeply. We will set aside that ships have grown in weight by about three percent per year since World War II, and that ever-more weapon systems are being put into them, and confine ourselves to discussions of costs rising for ships beyond the increase in “content.” ...2007Methods
To b or Not to b - The y-intercept in cost estimationRichard Coleman, Jessica Summerville, Peter Braxton, Eric Druker, Peter BraxtonThere is considerable, if low-key controversy over the y-intercept in cost estimation. The controversy, although more common than one would suppose, has received surprisingly little attention in societies and publications, but the authors have found that it is a controversy nonetheless, and one with sometimes significant implications. There are proponents that argue that all or most cost estimating relationships (CERs) should necessarily pass through the origin, and those that argue that they need not do so...2007Methods
Life Cycle Cost Analyses of C4ISR SystemsDavid Stamm, Greg BellCommand, Control, Communications, Computing, Intelligence, Surveillance, and Reconnaissance (C4ISR) systems have emerged as a prominent area of defense investment, modernization, and transformation. In the future, C4ISR will continue to command an important share of defense spending due to the following attributes:...2007Methods
Hilbert’s Problems for Cost EstimatingPeter Braxton, Richard ColemanAt the International Congress of Mathematicians at the Sorbonne in Paris in 1900, German mathematician David Hilbert boldly put forth a list of 23 theretofore unsolved problems in mathematics, which subsequently became quite influential in 20th-century research efforts. Though seven years late, the authors propose a similar list for 21st- century cost estimating and risk analysis. Because cost is an inherently but not purely analytical field, some of the problems take the form of broader issues to be addressed...2007Methods
Desired Characteristics of a Senior Cost EstimatorHollis Black, Harold Bodin, Michael HerringtonIntroduction. This paper provides a comprehensive overview of the skills, strategy, and work style of a Senior Cost Estimator. Our purpose is to encourage younger Estimators to proactively improve their knowledge, methods, and inter-personal skills, thereby increasing their value to the organization. Management relies on the contributions of Estimators in making sound business decisions.2007Methods
Economic Analysis Process Applied To the Worlds’ Largest Wind Tunnel - RC1 CoolerDan Wyman, Robert NelsonEconomic Analysis Process Applied To the Worlds’ Largest Wind Tunnel - RC1 Cooler
• RC-1 Concerns
o A single point of failure
o Contains environmentally hazardous/toxic fluids
o System wear out – aged technology…
Product, Sub-Market and Market DemandDoug HowarthMarkets demonstrate aggregate demand curves in which the number of units purchased varies inversely relative to price. These curves mathematically link to the values that buyers place upon products offered for sale. Over time, multiple sub-markets form within larger marketplaces. Sub-markets have their own demand curves, which sum to the market aggregate demand curve. The demand curve for a hypothetical new product is a measure of buyers’ demonstrated responsiveness to its specific valued attributes and product saturation in general...2007Methods
Work Scope Database for Cost ProposalsRobert WebsterIn most situations the resources necessary to support the creation of proposals to win business opportunities are scarce and sometimes inadequate for the number of opportunities available. Waste for any reason in the proposal process reduces the number of opportunities a business can pursue. On large proposals the customer requires starting from a position of fact whenever possible. That requirement demands a searchable Labor Actuals and Labor Work Scope Database. This discussion focuses on the Labor Work Scope Data of past programs...2007Methods
CER Prediction UncertaintyMatthew Goldberg, Richard SperlingIn this paper we compare the parametric bootstrap to the Delta method as tools for quantifying the uncertainty of predictions from a nonlinear cost-estimating relationship (CER) with a multiplicative error term. The bootstrap is a computationally-intensive resampling technique used to estimate bias and standard errors of statistical estimators. The validity of the bootstrap depends on asymptotic theory and is exact only in very large samples, whereas most CERs are estimated from small samples. The Delta method provides a linear approximation of the mean and variance for a nonlinear estimator...2007Methods
Learning Curves Redux: A New Use for a Familiar ToolEvin Stump, Alexandra MinevichThis paper proposes another use for learning curves, namely the scheduling of production operations. This is not entirely a new idea, but it is usually not formally implemented in the planning of production operations. Learning curves alone can’t do this planning job effectively, but when combined with other appropriate, relatively simple logic, the result can be an automated scheduling process that predicts not only the cost of each produced item, but also the dates when production of each item will start and finish, plus a spread of labor hours and material costs by month, or even by week, if desired...2007Methods
An Analysis of the Economic Impact for Airlines at LGA from a Tower Modernization Alternative that Closes GatesMarc RoseThis paper examines the economic impact on airlines due to delay should the airport lose gates. This paper will describe the techniques and data used to develop the inputs for a queuing model of LaGuardia airport. The results of the analysis allowed us to estimate the economic impacts to the airlines.
This, combined with the estimated costs, helped the FAA in making a decision on a new tower with no loss of gates versus upgrading the existing tower.
SE/IT/PM Trend AnalysisAlex Wekluk, Nathan MentonSystems Engineering (SE), Integration and Test (I&T), and Program Management (PM) costs have traditionally been difficult items to estimate at any given point in a program. Cost Estimating Relationships (CERs) are often used in practice to predict the ultimate SE/IT/PM cost. The utility of a CER naturally lessens as a program matures since CERs do not typically take into account known performance to date or specific details of the program that are not captured in the CER variables...2007Methods
Global Economy – Global RiskBethia Cullis, Christopher Leonetti, Eric Druker, Crystal HauserAs we progress towards an increasingly global economy, proposals are more and more frequently priced in multiple currencies. Exchange rates are not only difficult to predict, the risk associated with these predictions compounds the further into the future the predictions are made. In a recent proposal assessed by the authors, ~1/3 of the total proposed price was initially estimated in pounds (£, GBP.) The proposal team then used projected foreign exchange rates (a single point estimate per year of the period of performance) to determine the price in dollars...2007Risk
Taking a Second Look: The Potential Pitfalls of Popular Risk MethodologiesEric Druker, Richard Coleman, Christopher Leonetti, Peter BraxtonAs risk analysis has come into its own as a specialized field under the larger umbrella of cost estimating, certain practices and methodologies have been developed and generally accepted by the community at large. The creation of more standardized methodologies has opened risk analysis to a much larger population and skill set than previously allowed. Unfortunately, without a certain level of scrutiny, these accepted methods can lead to results that are not in accordance with the spirit of the methodology. In some cases, the combination of methodologies can even lead to results that are mathematically unsound...2007Risk
Examination of Functional Correlation And Its Impact On Risk AnalysisAlfred SmithThe term “functional correlation” is often mentioned in papers and presentations, but the definition of the term remains nebulous enough to confound even senior analysts. Yet, functional correlation often appears in even the simplest parametric cost models. Cost estimate methods that share the same uncertain inputs or are simply a factor of another cost method are examples of “functional” correlation. For example, using the same labor rate (with uncertainty assigned) across many different work breakdown structure (WBS) elements causes these WBS elements to be correlated, intended or not...2007Risk
Cost Risk Allocation Objectives, Tendencies, and LimitationsJohn SandbergAllocating cost risk for building a budget is sweeping through the cost estimating community faster than cost estimators can keep up. Few understand the goals, behavior, implementation, or limitations of results generated by popular risk allocation methods. This paper provides perspective on the issue by going beyond the basic mechanics to examine general tendencies, trade-offs, and objectives of cost risk allocation heuristics with focus on the quantification and verification of the allocated results. Some important limitations of existing heuristics are discussed, including the inability to capture program priorities, schedule impacts, and contract vehicles, followed by recommendations on how to capture these in future allocation methods...2007Risk
A Cost Estimation Risk Assessment Process Implementing Lurie-Goldberg's Algorithm for Generating Correlated Vectors of Random NumbersPaul WetzelA true cost estimate is not a single value. A true cost estimate is a probability distribution of anticipated cost. There are many high-quality cost estimating tools in the marketplace that provide "point solution" estimates, but there are few tools that perform a rigorous risk assessment sufficient to develop an accurate probability distribution of estimated cost...2007Risk
Cost-Risk Analysis of Satellite Bandwidth ServicesSam BresnahanThe purpose of this paper is to demonstrate application of basic risk- analysis techniques to a real-world cost estimating problem. Each year the U.S. Marine Corps must budget millions of dollars for satellite bandwidth services. Communication via satellite is frequently required during operations in theater, disaster relief, and any situation in which a secure communication infrastructure is not already present. The USMC is developing an Expeditionary Command and Control Suite (ECCS) that allows small teams of soldiers to establish secure satellite voice and data links with headquarters...2007Risk
Cost Risk as a Discriminator in Trade StudiesSteve BookPrior to formal program initiation, analysts typically undertake trade studies to investigate which of several candidate architectures or designs can best provide a desired capability at minimum cost. However, the various candidates typically differ significantly in risk as well as in cost, but members of the Government or industry trade- study team do not have the time, and the candidate solutions usually aren't sufficiently detailed at this stage, to conduct a thorough risk analyses. Yet, those differences in risk, as well as in cost, should be taken into account to the extent possible during the trade- study decision process...2007Risk
Developing Risk Adjusted Results with Limited DataGabriela RohlkThis paper expands on work by Dr. Stephen Book, MCR by addressing additional distributions in performing “quick-risk” analyses. We provide an overview of the purpose and methodology and then develop the equations for implementation. Given a simple point estimate, this methodology, when combined with some assumptions, allows the decision maker to estimate the high confidence cost/benefits. The methodology is developed for three different assumed statistical distributions (Triangular, Normal & Log-Normal). Additionally a high-level discussion of cost/benefits risk analysis will be covered.2007Risk
COTS Risk Models: Basic Advice for Avoiding PitfallsJohn TealThis paper will discuss a few of the common pitfalls to cost risk estimating, especially those caused by junior estimators lack of familiarity with popular risk tools. None of these tips break new ground, apply fresh theories, or even offer advice that is outside of the “help” section of each tool’s operating manual. But nonetheless, these tips will help a new estimator achieve a new level of understanding and insight which will ultimately lead to higher quality and more reliable estimates...2007Risk
Do Not Sum Earned-Value-Based WBS-Element Estimates-at-CompletionStephen A. BookEACsare Random Variables, Not Deterministic Numbers, and Must be Handled Statistically –Avoid Meaningless, Contradictory Outcome of Roll-up Procedure –Get Mean, Median, Mode of Total-System Cost at Completion, as well as All Cost-at-Completion Percentiles2000CEBoK Reference 2000
NAVAIR Cost Growth Study: A Cohorted Study of The Effects of Era, Size, Acquisition Phase, Phase Correlation and Cost DriversJessica R. Summerville, Megan E. Dameron, Cari L. Pullen, Richard L. Coleman, Donna M. SneadThe goal of this study was to further analyze Navy programs to determine the cause of cost growth (technical or cost estimating) and to develop a concept for estimating future cost growth. This understanding will allow program offices to plan for cost growth (adjust cost estimates and budget with cost risk dollars) and/or mitigate the high risks in the program to minimize cost growth...2001CEBoK Reference 2001
Relational Correlation: What to do when Functional Correlation is ImpossibleJessica R. Summerville, Megan E. Dameron, Cari L. Pullen, Richard L. Coleman, Donna M. SneadThis paper provides a significant breakthrough in the application of correlation in risk and cost estimation. The paper will supply the missing piece not handled by the procedure called Functional Correlation, put forward by the authors in 1994, and will show the unseen errors generated by other methods of correlation injection. The paper will show how to use the geometry of regression to determine the correct values of correlation and other variables in risk. Powerful and reliable methods of approximating the values of needed parameters will be suggested, and finally, possible applications to other areas of cost estimation will be suggested.2001CEBoK Reference 2001
Predicting Final CPI R.L. Coleman, M.E. Dameron, J.R. Summerville, H. F. Chelson, S. L. Van DrewEstimating the EAC based on current performance has traditionally been a point estimate or, at best, a range based on different EAC calculations (CPI, SPI, CPI*SPI, etc.). NAVAIR is in the midst of revising their EVM Toolkit, which incorporates the formation of an EAC. This paper provides the EVM analyst with a predicted Final CPI (and thus EAC), and also provides a confidence interval around the predicted value. This will enable analysts to determine where the program EAC falls on the cumulative probability distribution and to calculate the likelihood of achieving a favorable Final CPI (e.g., the probability of a Final CPI of 1.0 or better). The rule of thumb that EACs never improve over their values at 20% complete is analyzed and found to be generally true, but with
some exceptions. This paper is based on work by Michael Popp on distributions of Final CPI given Current CPI and % Complete for both development and production programs. This follow-on paper explores the larger patterns at work and discovers overarching trends in CPIs.
2003CEBoK Reference 2003
Analogies: Techniques for Adjusting ThemRichard L. Coleman, Jessica R. Summerville, Shishu S. GuptaThe three basic methods of cost estimation are generally accepted to be Parametric, Analogy and Build-up. Considerable attention is devoted to research in parametric methods, generally in higher-level and oversight organizations, and for early studies. Likewise, great expertise is resident in manufacturing sector in Build-up methods. The technique of analogy receives surprisingly little attention in research or in development of methods. Practitioners are enjoined to “find the best match and adjust it if necessary.” The issue of what exactly constitutes a best match and how to adjust are left to the practitioner. This paper will discuss the usual method, linear adjustment based on a single parameter such as weight or size (often called “J-ing up”), and will recommend two other methods as preferable to straight linear adjustment: one which borrows a slope from a cost estimating relationship, when one exists, and the other that develops a factor from the geometry of correlation, and which arises from a previous paper by the authors. The reasons for these methods being preferable will be presented, and an illustrative case study with actual data will be shown. The problem of finding a best match is not trivial and is not solved in this paper, but it will be discussed briefly for completeness.2004CEBoK Reference 2004
Rayleigh Curves – A TutorialH. Chelson, R. Coleman, J. Summerville, S. Van Drew2004CEBoK Reference 2004
How Age Affects Operations and Support Costs Differently Across Platforms
Sarah Grinnell, Richard L. Coleman, Jessica R. SummervilleThis paper expands on work previously published in Ship Scheduled Overhaul Costs Over Time (J.R. Summerville, R.L. Coleman, M.E. Dameron, S. D. Leach, SCEA 2003) and the more recent installment, Operating & Support Cost Over Time (S.E. Grinnell, J.R. Summerville, M.E. Dameron, R.L. Coleman, SCEA 2004). The first paper was ground-breaking research that showed that age does indeed have an effect on ship Operations and Support (O&S) costs. In the latter paper, the research was further refined to show that the O&S cost for a ship goes up over time and then, because the number of overhauls and repairs decrease at the end of a ship’s life, the cost decreases in the last few years of a ship’s life. The discussion in this paper will focus on more recent research that demonstrates there is also an age effect in aircraft. This discovery is important for a couple of reasons. First of all, it confirms that age is a cost driver for platforms other than ships...2006CEBoK Reference 2006
Making Risk Management Tools More Credible: Calibrating the Risk CubeRichard L. Coleman, Jessica R. SummervilleThe so-called Risk Cube is arguably the most widely used tool in Risk Management (RM) today. The method involves creating a risk register of all foreseeable risks. The attendant register is intended to give risk managers and program offices a tool to identify and manage risks to allow mitigation. The quantitative version of this method assigns a probability of failure (Pf) and a consequence of failure (Cf) to each risk. The qualitative version uses some measure of “likelihood” and consequence which are usually low, and high or ordinal scales. This paper will discuss an improvement to the qualitative method. In reviewing the qualitative method the authors have observed three general shortcomings. First, many programs, despite having the means at their disposal, do not roll up the risk register to determine the expected outcome in total. This is easily remedied and if Monte Carlo methods are used (which will be briefly explained in the paper) a distribution of outcomes can be produced. Secondarily, many programs fail to allow for any “unknown unknowns” which are well established as necessary in risk literature; accordingly, they yield an unreasonably low expected value...2006CEBoK Reference 2006

Back To Top