ICEAA Archives

Search past ICEAA Workshop Proceedings in the table below and click the title to access the downloadable files.


2014-2017 Workshop Proceedings available now and more are being added.

Just uploaded: 2012 and 2013!

TitleAuthor(s)SummaryYearTrack
How Should We Estimate Agile Software Development Projects and What Data Do We Need?Tom Coonce, Glen AllemanThis paper proposes and demonstrates how to estimate agile software projects using collected data from prior similar projects. The authors discuss traditional ways of estimating software projects and show why these approaches are not feasible in estimating agile projects. They demonstrate estimating agile projects using actuals from historical features. They recommend a few additions to DoD's Software Resource Data Report (SRDR) as well as a Standard Feature Breakdown Structure (FBS) to permit using this approach.2017Agile
Agile 'Mumbo Jumbo'Jeremy EdenYou don't put any stock in this Agile "mumbo jumbo" do you? Well, Agile is here to stay and is gaining popularity.This paper will discuss current issues and some solutions for cost estimating Agile projects, clear up common misconceptions about Agile, and Introduce an Agile approach to cost estimating, including a draft Agile Cost Estimating Manifesto with suggestions on how to incorporate it with current cost estimating training, so ICEAA and its members can BE Agile.2017Agile
Agile: You're Doing it Wrong (or how to know you're doing it right)Daniel B. FrenchAgile is a hot topic in the IT world today. If your organization is using Agile, or considering it, there are a number of things you need to know. This presentation will discuss the pros and cons of Agile, how to determine if your organization is ready for it, proper Agile implementation, hybrid methodologies, and the organizational challenges when trying to make an organization Agile.2017Agile
Beyond the Manifesto: Tracking Agile Performance in a DoD EnvironmentGordon M. Kranz, Michael ThompsonThe use of Agile methods are becoming prolific within the DoD Enterprise and Weapons systems development environment. Agile supports the incremental discovery of end system capability within the constraints of the DoD parameters. This paper investigates, through real world experience, the use of earned value techniques to strategically manage the Agile development process. One of the essential elements key to success is traceability of user requirements to Agile features.2017Agile
Agile and GAO Cost Estimating Best PracticesJennifer Leotta, Karen RicheyThis paper will examine how GAO's cost estimating process can be applied to programs that are using an Agile framework. First, it will provide a brief overview of Agile processes and methods. Second, it will examine each of the 12 steps in the GAO cost estimating process and how those steps relate to an Agile framework. Finally, it will discuss how Agile artifacts can be leveraged to fulfill cost estimating documentation needs.2017Agile
CMMI and Agile Development: A Binary Choice?Arlene F MinkiewiczThe Capability Maturity Model Integration for Development has a long and impressive history for progressing the cause of process improvement. Agile development is a paradigm for software projects that are characterized by collaborative, cross functional teams working closely with customers to deliver functionality regularly. Many say these two approaches can't work in tandem. This paper explores this question with examples of both successes and failures.2017Agile
Agile Delphi Estimating and Analyzing Schedule/Resource Realism for Software ProjectsBlaze SmallwoodOne of the most important tasks for a software project manager is to accurately forecast the schedule and resources needed to complete the project. This paper will describe a novel approach to this problem that combines Monte Carlo-based uncertainty analysis with an innovative new technique for estimating software requirements, which utilizes an agile-style Delphi estimating methodology. This method has been successfully implemented on several government projects and is a viable option for any software project.2017Agile
A Cost Model for Early Cost Calculation of Agile DeliveriesEric van der VlietAgile software development methodologies provide the IT industry with the flexibility they need to keep up with the faster change of business requirements. In agile software delivery, upfront detailed specifications are absent, yet investment decisions need budget input. The challenge is to build a cost model that takes essential (size) and additional cost drivers into account. This presentation explains a cost model that supports the cost calculation for agile delivery early in the process.2017Agile
Assessing ERP Cost, Schedule and Size GrowthHasetetsion Gebre-Mariam, Abishek KrupanandThis study will examine percentage changes in cost, schedule, and size across Milestones A, B, C, and full deployment for DoD Enterprise Resource Planning (ERP) programs. The analysis is based on 9 fielded systems collected from DoD authoritative data sources. Cost contributors, drivers, and factors by major cost elements will also be examined. Results may be used for crosschecking cost estimates or business case analyses at an early phase to inform funding decisions.2017Data Analysis
Production Rates: Do They Really Matter?Brent JohnstoneProduction rate is widely assumed to be an important contributor to unit cost -- higher production rates lead to lower unit costs, and vice versa. Examination of published data, however, leads to a more ambiguous picture. This paper examines the impact of rate by functional cost element, including the impact on learning curves. It concludes production rate impacts are real, but the impacts are uneven and sometimes reveal themselves in surprising ways.2017Data Analysis
Lessons Learned in Leveraging Historical Cost, Schedule and Technical DataGreg Kiviat, John SwarenThe process of applying historical cost, schedule and technical data to develop new program estimates is often more difficult than textbooks suggest. The paper provides real-world insight to the issues that arise when capturing, analyzing and applying data for the next estimate. The goal is to integrate information from multiple products, systems and subsystems. Lessons learned include experience and best practices when working with financial, engineering, manufacturing and program management.2017Data Analysis
Technology Readiness Level (TRL vs. Percent Development Cost)James LinickAdvanced technology acquisition often proceeds in steps characterized by TRL of systems or components. The relationship between TRL ordinal number and percent of total acquisition cost could be used as a factor in estimating the cost of new acquisitions at each phase from concept definition through production decision. A study of historical data will be performed to establish such a relationship based on a specific definition of TRLs, which vary across government agencies.2017Data Analysis
Machine Learning Approach to Cost AnalysisKaren Mourikas, Joseph A. King, Denise J. NelsonAt ICEAA 2016, the keynote speaker presented a challenge to the cost estimating & analysis community: to develop cost estimating methodologies based on machine learning. Machine learning provides an alternate to parametric modeling. At Boeing, we are experimenting with several machine learning techniques for cost estimation and analysis; Random Forest prediction methodology, in particular, has shown encouraging results. This paper describes our cost applications of Random Forest.2017Data Analysis
Technology Development Cost and Schedule ModelingChuck AlexanderA tangible need exists in the scientific, technology, and financial communities for economic forecast models that improve new or early life-cycle technology development estimating. Industry models, research, technology datasets, modeling approaches, and key predictor variables are first examined. Analysis is then presented, leveraging a robust industry project dataset, applying technology and system-related parameters to deliver higher performing parametric cost and schedule models.2017Data Analysis
DHS Cost Analysis OverviewDarrin DeReusCost Estimating and Analysis is a vital resource for program and project managers, regardless of agency or activity. July 2014 the cost analysis responsibilities and personnel were transferred from the DHS's Office of Program Accountability and Risk Management to the Office of the Chief Financial Officer. This presentation will provide a history of the creation of a cost analysis capability in a new agency, changes over time, lessons learned and current successes of the Cost Analysis Division.2017Data Analysis
Using Real Options to Quantify Portfolio Value in Business CasesGeorge Bayer, With the challenges in quantifying program risks, interdependencies, and the business case impact on portfolios, cost estimators can use real options to estimate probabilistic value in business cases and assess portfolio value. Many business cases not only add value as stand-alone investments, but they provide the opportunity for subsequent investments. Cost estimators can measure this incremental program value and the impact of a specific investment on a larger portfolio by using real options.2017Economic Analysis
Discount Rate for Government InvestmentRon Beheler, Laura BarkerWe often use a discounted-cash-flow construct to evaluate the Economic Viability of an investment. OMB-94 provides discount-rate guidance for internal investment (treasury rates) and public investment/regulatory programs (7% - rate of return on an average private sector investment). Since any resources consumed by the government must be foregone by the private sector, we will argue that the discount rate should be based on a rate that at least proxies the value of private investment.2017Economic Analysis
Are the Rates Right? Benchmark Protection Against Escalation SWAGBrian Flynn, Brian Torgersen, Greg Mihalek, Adam JamesMedical escalation rates continue to trend higher than general inflation - with significant impact on defense firms and acquisition affordability. Modest deltas in rates translate into billions of additional costs. This research presents benchmark escalation values for employer costs of employee medical compensation. The benchmarks, in turn, support company planning & collective bargaining and liberate government cost estimates from their traditional reliance on solely FPRPs and FPRAs.2017Economic Analysis
The Rate You Want – Rethinking Cost Escalation in FAA DevelopmentSriram Krishnan, Andrew DrennonLike most civilian agencies, the FAA has used 'raw' OMB escalation indices to produce then-year costs for its life cycle cost estimates. However, as an agency which incrementally funds many development programs, could the FAA benefit from recognizing time value of money principles at the estimation stage? We look at outlay data from across the agency to determine whether defense appropriations styled weighted indexes would significantly impact then-year FAA solution development costs.2017Economic Analysis
Terminal Facility Realignment: A Business Case ApproachDeji Oladipupo, Ayn Smith, Melvin Etheridge, Richard SheproThe Federal Aviation Administration applies a business case analysis approach to the decision process for realigning and consolidating Terminal facilities. A Microsoft Excel model has been developed to produce these analyses. It uniquely displays both costs and benefits in one place and includes graphical outputs of NPV over time. A key output of the analysis shows interesting cyclical trends of NPV rising, falling, and rising again over time in almost all scenarios.2017Economic Analysis
Kennedy Space Center's Transformation to a Multi-User SpaceportTerry LambingIn the past, launch pads were used almost exclusively for government missions. To support a growing private sector space economy, NASA’s Kennedy Space Center has transformed into a multi-user spaceport capable of handling the needs of a variety of companies from launch processing and operations through recovery. This presentation will provide an overview of the transformation and the NASA Business Case requirements needed in order to gain approval, and successfully establish these unique partnerships.2017Economic Analysis
NRO CAAG Spacecraft Test Schedule Estimating RelationshipDaniel BarkmeyerThe NRO CAAG's parametric model for estimating spacecraft testing schedule duration has been updated. The updated model incorporates data from government and commercial spacecraft contracts. The intent of this presentation is to share the model with the ICEAA community. The latest model will be presented, along with a case study of a satellite program that illustrates the importance of the chosen schedule drivers to system test schedule.2017EVM & Scheduling
Fully Integrated Cost & Schedule Method (FICSM) Analysis Schedule ImplementationJonathan Brown, Benjamin T. UnruhThe Joint Agency Cost Schedule Risk and Uncertainty Handbook released in March 2014 introduced the FICSM which integrates cost, schedule and risk estimating. Key to the FICSM process is creation of an analysis schedule including duration uncertainty. This study leveraged data from an active program to develop an analysis schedule as part of building a FICSM model. This paper describes the methods used to develop the analysis schedule and duration uncertainty used for this program.2017EVM & Scheduling
Beyond RIFT: Improved Metrics to Manage Cost and ScheduleNicholas DeToreRisk-Informed Finish Threshold (RIFT) presented an innovative solution to the problem inherent in schedules that risk analysis results (time) cannot be allocated the same way as in cost models (dollars). Developing RIFT validation methods inspired an exploration into analyzing simulation data more meticulously. Methods described here provide unique insight into cost and schedule uncertainty results while introducing powerful new techniques to improve a project's potential to complete on time, on budget.2017EVM & Scheduling
Adopting DOD Best Practices in IPM to non-DOD NeedsDavid L. Wang, Daniel Schwartz, David T. ChiangNon-DoD federal acquisition agencies have their own unique acquisition requirements and acquisition culture. However, many non-DoD acquisition agencies face similar challenges, specifically in the areas of acquisition program execution and program affordability. To address these challenges, there has been increased interest to consider tailoring DoD best practices in Program Management, Earned Value Management and acquisition lifecycle schedule management. This presentation discusses a systematic approach to tailoring DoD best practices for non-DoD acquisition needs.2017EVM & Scheduling
Cost Associated with Acquisition Complexity and Differing Levels of Mission AssuranceErik BurgessIn Government acquisition programs, the cost of what we buy is largely driven by how we buy. For satellites, this includes contracting, funding, oversight, reporting, testing, parts assurance requirements, and other factors. All of these are related, and no single attribute defines the cost curve. The NRO has developed and implemented a method that quantifies the cost of these factors and circumvents the usual debate about whether or not an acquisition is streamlined or commercial-like.2017Methods
Don't Dis LOE: Modeling Production Sustaining Labor Across Multiple LotsSandy BurneyThe estimation of sustaining labor is often given little analysis; just assuming some constant level. However, in estimating production hours on large Programs, spanning multiple lots, where each lot is negotiated separately, conflicts often occur over the cost of sustainment labor. This briefing will look at different approaches to estimating and modeling sustaining labor, based upon varying cycle times for completing a single unit, and the number of units being procured in each lot.2017Methods
Automated Data Collection Using Open Source Web Crawling TechnologyAnna FooteFor many reasons, data gathering is often one of a cost estimator's biggest challenges. Even in cases where data is available, the logistics of data collection can be daunting. This paper explores a methodology developed by PRICE utilizing free, open source web crawling technology to automatically seek and find commodity pricing for information technology hardware and software items. This presentation shares the nuts and bolts for everyone's benefit.2017Methods
A Framework to Price and Cost IT Network ServicesJohn LeahyThe correct pricing of an IT network offering aimed toward internal users can be an involved process. Different services targeted to some, but not all customers makes proper cost allocation challenging. Considerations such as mirroring the structure, if not the rates, of commercial offerings; reconciliation of differences between cost allocations and rational pricing; and influencing customer purchasing behavior adds to the complexity. An approach for costing and pricing IT services that address these requirements will be presented.2017Methods
Don't Get Caught in the Learning Curve VacuumPatrick McCarthyProduction environments featuring multiple end items benefit from a manufacturer designing products and processes with a high degree of commonality and standardization. When estimating labor hours using learning curves, the estimator often overlooks commonality when considering quantities, slope, production rate and lost learning. This paper will analyze alternative approaches to defining commonality and how to account for commonality using learning curves in integrated production environments.2017Methods
Modeling Hardware Development Cost in a Low TRL / Pre-Acquisition EnvironmentJack SnyderEstimating the cost of pre-concept programs, often with very immature TRL's, is always challenging. This case study highlights the use of analogous programs and predictive modeling to overcome the challenge. We will explore the strategy of finding analogous components of a program, even when the overall program is unrelated. Predictive models can easily be tailored for the inherent uncertainty of these early programs, when we are unable to find relevant data.2017Methods
Exploring the of Results of two Methodologies for Unmanned Space EstimationJohn SwarenIndependent validations of two separate predictive analytics methodologies were performed over the last two years. In both cases, common detailed assessments of past missions integrated technical and programmatic requirements to mimic a grass-roots bottom-up methodology. This presentation compares the process and top-level results from both approaches as well as explores several lessons learned.2017Methods
Deployment Cost Estimation for Electronic/IT SystemsF. Gurney Thompson IIIThis paper will discuss our research into cost estimation for the deployment of Electronic and IT systems, also known as Operational/Site Activation. The deployments can span multiple sites/locations, and often require technical studies, end user working groups, and site survey visits in addition to the actual installation activities. This paper will discuss our research approach, sizing metrics, cost drivers, model structure, and future work.2017Methods
Budgeting for Canadian Shipbuilding– Examining Predictive Cost Analytic MethodsPeter Weltmann Zachary JasnoffPredictive Cost Analytics plays a major role Canada's Shipbuilding Strategy in developing cost and budget estimates. However, these estimates must be based on sound historical cost data. This paper discusses challenges faced by the Canadian PBO in developing reasonable estimates for the JSS, AOPS and CSC shipbuilding programs based on the quality of historical data available, the methodologies employed to address these challenges, and the influence of these estimates on decision-makers2017Methods
Decision Trees and Cost EstimatingJosh WilsonDecision trees are predictive modeling approaches that recursively partition data, and apply simple prediction models to the resulting subgroups. Decision tree based prediction approaches have the benefits of being easy to interpret and explain, requiring little data preparation or cleaning, and the ability to model complex nonlinear relationships and variable interactions. This presentation explores the applicability of decision tree based prediction methods to cost estimating.2017Methods
Cost Estimating Canada's Future Surface CombatantsRod StoryCanada has not built a new surface combatant since it completed the last of its 12 Halifax class frigates in 1996. Currently Canada is in the middle of a request for proposal to build, using a common hull, 12 new frigates and 3 new destroyers. This paper presents the first public cost estimate of these ships highlighting both the challenges that were encountered and the solutions that were used in determining a reliable cost estimate.2017Methods
Modeling the Influence of System and Application Complexity on the Cost of Cloud HostingDaniel J. Harper, Kevin BuckTo refine cost estimates produced within a Cloud Total Ownership Cost model, MITRE is now prototyping an additional modeling feature that estimates the cost impact of migration and hosting complexity. While many cloud cost models rely on a limited set of cost drivers, other factors associated with general complexity significantly influence costs and are often ignored. Within the prototype, complexity assessments are plotted using intuitive spider diagrams for ready comparisons of candidate systems and applications for hosting in various cloud solutions.2017Methods
Marco-Parametrics; Its Unique Capability and ApplicationDale Shermon, Arlene F MinkiewiczThe Family of Advanced Cost Estimating Tools (FACET) has been used around the world as a macro parametric cost model at the early stages of a project life cycle. It has a unique capability to seamlessly transition from performance to design based estimating. FACET will also combine uncertainty in the inputs with those of the algorithm to produce a true uncertainty range. Now implemented in TruePlanning the FACET model is easy to access and apply.2017Methods
Projecting Program Spare Parts Sustainment with Incomplete DataBryan Anderson, George BayerAnalyzing a government acquisition, the team discovered that the existing agency supply chain system lacked transparency, and historical failure data was unavailable due to implementation of a new supply chain system. The team examined how to accurately project parts demand using incomplete data by conducting a comparative analysis between data sources, using subject matter experts to estimate upper and lower bounds, and applying statistical analysis fit curves to project spare parts needs.2017Operating & Support
Sustainment Cost Data CollectionSandi EnserOperations and support costs can exceed 60% of the total Life Cycle Cost of DoD systems. Currently, the individual services have their Visibility & Management of Operating and Support Costs (VAMOSC) systems. However, with many sustainment functions performed by contractors, improved reporting on sustainment contracts is essential. This paper presents several data collection initiatives for sustainment efforts and their impacts on estimating O&S costs.2017Operating & Support
Army Software Maintenance Cost Estimating RelationshipsCheryl Jones, James Doswell, John McGarry, Jenna MeyersFor the past four years, the Army, under the leadership of DASA-CE, has been collecting and analyzing Army system software maintenance cost and technical execution data to support the development of more accurate cost estimation methods. The presentation will present the cost methods and cost estimation relationships developed from the analysis of the execution data sets. The results of the Army's analysis efforts, including the detailed statistical analysis will be made available.2017Operating & Support
Implementing Additive Manufacturing Technology into the Logistics Supply ChainPatrick K. Malone, Bruce FadRecent explosive growth in Additive Manufacturing (AM) or 3D printing is providing logistics supply chain economic opportunities. We investigate self-sufficient repair and maintenance capabilities for isolated environments, impacts on strategic readiness, increasing responsiveness and cost efficiencies not available in traditional supply chains. Our research will evaluate and contrast legacy logistics architectures against AM elements that will drive cost downward over lifecycles to meet current and future affordability goals within the Government and commercial organizations.2017Operating & Support
Expanding the Range of Your Data: A Small Ships Case StudyKathleen Hudgins, Robert Nehring, Elizabeth Koza, Anna IrvineWhile the Navy has comprehensive Operating and Support (O&S) data for current Navy ships, there are a limited number of smaller boats available for inclusion. With increasing technological advances and a renewed effort to reduce personnel, smaller boats are receiving increased interest. This paper explores using Coast Guard O&S data to supplement Navy data for use in estimating O&S costs for smaller boats. Topics include data sources, normalization, and comparisons of data between the services.2017Operating & Support
Assessing Confidence Levels in Funding and Budgeting NASA Science MissionsRobert E Bitten, Charles D. HuntNPR7120.5E requires that NASA utilize joint confidence level (JCL) analysis to set budget and funding guidelines for projects within its portfolio of missions. This paper addresses analysis that was conducted to determine the effect of different confidence levels on the performance of different cases of mission portfolios. The results show that the most effective confidence level varies depending on the case. An overview of the policy, methodology, cases and results are discussed.2017Parametrics
Data Driven Confidence Regions for Cost Estimating RelationshipsChristopher JarvisCost estimating models typically contain many uncertain parameters. The uncertainty of the parameter values drives the uncertainty of the model outputs. In this paper we review and compare the techniques to compute the confidence regions for linear and nonlinear regression methods. Application to two common CERs is presented with comments regarding the practical implementations and limitations within the current cost estimating environment.2017Parametrics
Moment One, PleaseChad Krause, Erik BurgessCost models constructed without statistical correlation are designed to underestimate the mean, unless they are simple sums. Any model that uses a factor or other instance so functional correlation will miss the first moment, not to mention others, unless statistical correlation among the CERs is also applied. An internal audit of 13 recent satellite cost estimates comprising of 52 CERs shows the mean cost changes by as much as 9% when this correlation is applied.2017Parametrics
Modeling with Gumby: Pros and Cons of the Weibull CurveMichael Mender, Ann HawpeThe Weibull function is used to model various cost phenomena. This popularity is driven by its significant flexibility, driven by its nature as a two parameter function. Additional parameters result in flexibility, which while advantageous, also places incorporating estimates at risk of over fitting (i.e., a model that fits the observed data well, but has little predictive power). Presentation relays theory and exhaustively reviews pros and cons for cost estimating.2017Parametrics
Maximum Likelihood Estimation for Regression of Log Normal ErrorChristian SmartThe use of Log-transformed Ordinary Least Squares (LOLS) has been criticized for the use of transformation, which results in biased estimates. LOLS is a maximum likelihood estimate (MLE) of the median when the residuals are lognormally distributed, for which we provide evidence. We discuss MLE and show how to use this technique to directly model untransformed lognormal error. We also discuss two other popular methods, ZMPE and MUPE, in the context of MLEs.2017Parametrics
Novel Manufacturing Methods ~ Characterizing the Impacts to CostZachariah Sayre, Frank CampanileAverage unit procurement costs for military aircraft are growing at a rate that far exceeds inflation. Efforts such as the Air Force Research Lab's Composite Affordability Initiative and the Advanced Composite Cargo Aircraft have demonstrated alternative manufacturing methods utilizing composites to achieve significant part and fastener count reductions. This analysis highlights the concepts of large composite unitization in fixed wing aircraft manufacturing and its resulting effects to cost.2017Parametrics
Assuring Credibility in the Cost EstimateHenry ApgarCredibility can be the most important attribute of a cost estimate. This paper traces the evolution of quality metrics that assess cost credibility in the words of senior government executives, industry leaders, estimating and engineering handbooks, professional journals, and government auditing manuals. The presentation concludes with recommendations for the estimating professional.2017Policy & Standards
In Theory Or In Practice? The Optimistic World of Pessimistic Cost EstimatorsErin K. BarkelIn 2015, Andy Prince won the Best Paper award for The Psychology of Cost Estimating, a piece which highlighted a number of cognitive biases which impact the work of cost estimators. Of these cognitive biases, the one that seemed to resonate most strongly with the community was optimism bias. However, not everyone is worried that the glass is half-full. Where do negativity and pessimism bias fit? How does our understanding of these biases change how we communicate with stakeholders?2017Policy & Standards
How to use Predictive Analytics for a DCAA-Compliant Estimating SystemAnthony A DeMarcoThe DCAA ensures that Defense contractors use an acceptable estimating. They establish acceptable estimating system compliance criteria and audit's contractors to determine compliance. Non-compliance can be devastating. This presentation demonstrates a data-driven estimating methodology that complies and reduces the time and labor necessary to produce estimates. It illustrates the roles and procedures necessary for success. Six key compliance tenants are identified and key benefits quantified.2017Policy & Standards
Developing a Cost Capability RoadmapJohn FitchDeveloping cost estimating tools and databases is critical to analyst productivity and the quality of analyses. Historically, organizations struggled to systematically improve capabilities due to a lack of a long-term plan, unstable budgets, and priorities that shift with changes in leaders. This presentation walks through how one organization confronted this challenge by developing a roadmap that links the mission to requisite tools, data, and training; and a plan to achieve the roadmap.2017Policy & Standards
The Shortcomings of Cost Estimating TemplatesMeagan Gadreault, Faye KimTemplates are seen as a best practice in many industries to standardize workflow and reduce redundancy. However, templates may not be the ideal solution for cost estimating as they can suppress innovation and constrain analysts. Instead, best practices should be determined using process improvement methods and implemented to create customized estimates. This paper explores leveraging Lean Six Sigma techniques to determine alternatives to templates in order to put forth the best estimates.2017Policy & Standards
The Art of Cost: Sun-Tzu's Strategic Insight in Cost EstimationBrian A. GillespieStrategy is the process employed to determine how political purpose is translated into action. Clausewitz in his grand work "On War" argued that war serves a political purpose. So too does cost estimation, for at its most fundamental level, cost estimation attempts to determine the level of resources needed to achieve a political outcome. As one of the greatest enduring works on military strategy, what does The Art of War teach us about cost estimation?2017Policy & Standards
Coherence & Oddities: A Retrospective of Cost Estimating Publications 1978-2016Ross A. Jackson, Bradley C. BoehmkeIntellectual sedimentation can operate unobserved within professions. Assessing publication trends serves to make such conventional wisdom explicit. Our analysis of articles from the Journal of Cost Analysis and Parametrics (and its predecessors) provides insight regarding research coherence and oddities within the cost estimating community. This knowledge is essential for envisioning alternative futures and could be of benefit to those engaged in the praxis of cost estimating or its research.2017Policy & Standards
A History of Thought in Defense AcquisitionsEric M. LofgrenAs Congress debates another round of defense acquisition reform, the necessary role for the cost estimator is affirmed. But how did this role come about and what are future implications? From the famed RAND systems analyses of the 1950s to the introduction of data reporting systems still in use today, this paper will explore the rich history of thought in defense acquisition, giving a special eye to controversies and continuing challenges that affect cost estimators.2017Policy & Standards
Establishing and Implementing Cost Estimating StandardsHetal Patel, Denise Dulee, Danielle SpencerAs part of its efforts to create a world-class cost estimating capability, the Missile Defense Agency (MDA) wrote a cost estimating handbook and established an independent assessment team to ensure adherence to the handbook. As a result of this, MDA has received positive feedback from GAO in its progress towards this goal. This briefing describes in detail the processes and procedures that MDA has developed.2017Policy & Standards
Estimating Challenges and Solutions at NASA: Past, Present, & FutureCabin Samuels, Jeff BrownNASA's cost estimating landscape is fraught with unique challenges. Missions often incorporate cutting-edge technology and operate in harsh environments with no margin for error. Goddard's Cost Estimating, Modeling, & Analysis (CEMA) Office has developed robust methodologies to create realistic, transparent cost estimates for missions with little or no cost heritage. This paper discusses past and current states of cost estimation at Goddard and provides insights to shape future improvement.2017Policy & Standards
Anatomy of the Future DoD Cost EstimatorTamiko L. Ritschel, Jonathan D. RitschelThe need for a specialized, properly qualified DoD cost estimating workforce has resulted in a migration of civilian billets from a financially focused job series to the 1515 Operations Research series. We analyze the origins of this change, implications for the current workforce, and then discuss the anatomy of the future DoD cost analyst. While we focus on recent changes occurring within the U.S. Air Force workforce, the future implications are relevant to all Services.2017Policy & Standards
Masters in Cost Estimating and AnalysisDan Nussbaum, Greg MislickThis presentation provides an update on the all-distance Learning Master's Degree and/or Certificate Program in Cost Estimating and Analysis (MCEA / CCEA) offered at the Naval Postgraduate School (NPS).2017Policy & Standards
Contract Geometry Best Practices for Incentive ContractingPeter J. BraxtonWith the current emphasis on Incentive Contracting, the cost analyst plays a vital role in establishing target cost, fee, and shareline. Drawing from the Joint Contract Price Database, this paper examines actual contract geometries across a wide range of programs and their effectiveness in encouraging cost management. Using the published Risk-Based ROS methodology, it provides a framework for implementing incentive arrangements for both competitive and negotiated procurements.2017Risk
A 'Common Risk Factor' Method to Estimate Correlations Between DistributionsMarc GreenbergA 'common risk factor' method uses expert-derived risk factors to estimate correlation between two distributions. The premise is to estimate mutual information among risk factors that underlie each distribution. Approximation of such mutual information leverages properties of the joint probability distribution of a unit square. Geometric outputs of each pair of common random variables are compared to estimate common risk factor 'intersections' that are, in turn, proxies of correlation.2017Risk
The End of S-Curve Alchemy: Gold from a New SAR DatabaseTodd Andrews, Jeffrey Pincus, Brian FlynnS-curves typically address only "known unknowns." They're built upon an analysis of cost risk and uncertainty – element by element. But, they fail to address the ambiguity of the curve itself, or the degree of confidence that its proffered probabilities are actually correct. This confidence factor, of course, is hard to measure. The "unknown unknowns," in other words, are not explicitly captured since their probabilities are a mystery. This research fills the gap in knowledge.2017Risk
Being Certain About Uncertainty, Part 1Andy PrinceDoing cost risk analysis is hard because we don't really know what a good cost risk analysis looks like. In this paper we will explore the challenges to doing good cost risk analysis and discuss ways to know if your cost risk analysis is any good. We will also examine the phenomena of extreme cost growth and lay the groundwork for future work.2017Risk
Integrated Cost-Schedule Risk Analysis Improves Cost Contingency CalculationDavid T. HulettThe main benefits of integrated cost-schedule risk analysis are improvement of the estimates of cost contingency and identification of the main risks to cost for mitigation purposes. The main focus will be on estimating the cost contingency needed and identifying risks to cost, which may be independent of schedule or indirectly due to schedule risk. New simulation software including iterative risk prioritization will be used to illustrate these points.2017Risk
Using Quantum Theory and Monte Carlo Multiverse to Manage RiskNathan EskueNeed to better predict your EAC? Tired of being surprised? Great! Not an expert on quantum theory or the multiverse? No problem! This hands-on session will give a high level view of how quantum theory can help us understand our EAC by viewing it as a multiverse. We'll then explore an Excel model that takes your EAC and uses Monte Carlo analysis to reveal all possible futures. Attendees get a copy of the model!2017Risk
Software Projects Estimation & ControlAlain AbranThis talk presents 1) an outline of the design of the '2nd generation' COSMIC Function Point method for measuring a functional size of software; 2) Industry evidence of its versatility & value in software project benchmarking & estimating in business, real-time, infrastructure, component, mobile, IoT, cloud software, via traditional & agile methods; 3) how it can be used for early and rapid sizing at estimation time; 4) how size measurement can be automated with very high accuracy.2017Software
Bottom Up Methods of Estimating Software SE/PM and Non-DCTI CostsJames BlackSystems Engineering/Program Management (SE/PM) and additional non-Design, Code, Test, and Integration (Non-DCTI) activities performed during software development efforts are often significant and drive estimates of total project costs. Yet, cost estimates often omit the detailed research and analysis needed to adequately model SE/PM & Non-DCTI costs. This brief will present bottom up methods useful for understanding and estimating these costs and share analysis of recent SE/PM & Non-DCTI data.2017Software
Reliable Non-Design, Code, Test, and Integration Cost RelationshipsJeremy Goucher, Brittany StaleySoftware cost estimates require ratios derived from historic cost reports for non-design, code, test, and integration (NDCTI) cost elements. Since NDCTI accounts for as much as 50% of the estimate, a comprehensive historical data set is critical to ensuring an accurate estimate. The authors have recently analyzed over ten years of actual cost data from DoD command and control systems to develop a new set of NDCTI ratios. The results also bring new insight into "fixed" versus "variable" cost.2017Software
Objective SLOC: An Alternative Method to Sizing Software Development EffortsAndrew KicinskiEquivalent Source Lines of Code (ESLOC) is the basis of methodology used by many organizations for collecting and estimating software development costs. Selecting ESLOC parameters requires insight into the software reuse. Too often data collectors are unable to verify the appropriateness of the assigned ESLOC parameters and validate their implementation. This paper examines the drawbacks of ESLOC, and presents an alternative and more objective method to estimating software development effort.2017Software
Software Effort Estimation Models for Contract Cost Proposal EvaluationWilson Rosa, Corinne WallsheinThis study will introduce regression models and benchmarks for predicting software development effort using input variables typically available at contract bidding phase. The analysis is based on 200 DoD projects delivered from 2004 to 2016. The first set predicts effort using initial software requirements along with peak staff, application domain, and other categorical variables. The second set predicts effort specifically for agile software development using data from 16 agile projects.2017Software
Software Size GrowthMarc Russo, Wilson RosaSoftware cost estimating relationships often rely on software size growth percentages. Actual delivered source lines of code (SLOC) may be predicted with categories of early code estimates such as new, modified, reuse, and auto-generated SLOC. Uncertainty distributions will be presented to represent growth by code category for use in cost modeling.2017Software
Analysis of Software Cost Estimation Models Using Normalized, Stratified DataDan StricklandSoftware Resources Data Reports (SRDR) are the DoD's effort to improve cost estimation efforts by collecting metrics on software programs. Existing SRDRs were normalized in a manner consistent with previous efforts and productivities were analyzed using three popular software cost estimation models (SLIM, SEER, and COCOMO). Calculated values were tested against model thresholds and the results were tested for accuracy. Findings indicate some models outperform others and schedule impacts results.2017Software
The Journey from "Bottom-up" to Predictive Modelling BOELori SaleskiPredictive Cost Modeling has the potential to save significant money over conventional bottom-up bidding with better accuracy. This paper outlines a process to implement predictive cost analytics in an organization that been dependent on bottom-up estimating for decades. Walk with us down the path discussing implementation, change management, adoption and stakeholder challenges. Explore architecting the solution for maximum leverage at ROI.2017Software
Observation of Production Rate Effect observed in Historical Weapon Systems CostWilliam Banks, Timothy LawlessProduction Rate Effect has been observed in several weapon systems program’s historical cost data during the procurement phase of the acquisition life-cycle. The main contribution of this paper is to illustrate real life observations of production rate effect and the economies of scale shown when purchasing end items from major defense contractors.2016Acquisition
Federal Agency Independent Government Cost EstimateCassandra M. Capots, Lauren Nolte, Lavanya YeleswarapuTo achieve mission success in today’s constrained fiscal environment, federal agencies must establish a fair and reasonable cost baseline to make informed decisions. This baseline is commonly called an Independent Government Cost Estimate (IGCE). We have developed a customizable process that forms a starter kit for any agency that wishes to quickly and effectively perform IGCEs. We discuss best practices, lessons learned, and implementation tips for both DoD and civilian agencies.2016Acquisition
Early Stage Cost Estimating for Radars and SensorsJeremy GoucherThe majority of costs for programs are locked in even before a program enters production, which makes accurate early stage cost estimates vital for effective resource management and program success. This study proposes a method for analyzing combat system cost prior to a complete requirements description. The data set includes old, new, big, small, ground, sea, air, domestic, and foreign systems. The result is a model that requires limited data and is widely applicable.2016Acquisition
The ABCs of the UCF: A look at Section L and Section MJennifer LeottaThis paper will examine the government cost analyst’s role in the acquisition/contracting process. Specifically, it will look at the Uniform Contract Format (UCF) and how to create effective language for Section L that provides clear directions for offerors and includes all information necessary to complete a thorough proposal evaluation. It will also look at developing Section M language for varying contract types (cost plus, fixed price, etc.) that is consistent with the Federal Acquisition2016Acquisition
GAO-COFC Concurrent Bid Protest Jurisdiction: Are Two Fora Too Many?James LinickThe Government Accountability Office (GAO) and the Court of Federal Claims (COFC) are the bid protest fora where a disappointed offeror may challenge an agency’s procurement decision. Critics claim that concurrent bid protest fora, which have differences in jurisdiction, standards of review and remedies, creates inefficiency in the procurement system, e.g. serial protests. This presentation examines whether concurrent bid protest fora are healthy for the US federal procurement system.2016Acquisition
A Weapons Acquisition Case Study: Cost Overruns and Schedule SlipsRandy Bowen, David L. Peeler, Jr.This story is true. Certain details have been omitted to “stump the audience” and protect the guilty. The trials and tribulations of an acquisition program cost overrun are explored for persistent lessons and future applications. This saga includes materiel difficulties, workforce issues, congressional interest, funding and schedule perturbations. The aim is to educate, as the persistence of fundamental acquisition problems and the insignificance of acquisition reforms are framed.2016Acquisition
Bridging the Gap between Capital Investment Valuation in Private & Public SectorGeorge Bayer, Jr.The public and private sectors have different valuation techniques and considerations when evaluating capital budgeting decisions. Understanding the difference between private and public sector capital investment analysis (discounted cash flow, cost/benefit, tax impacts, stakeholders) helps decision-makers make better informed decisions. In the cost estimating community, understanding investment distinctions makes us better stewards of information and more effective Finance professionals.2016Business Case Analysis
ROS vs. ROC: Government Can Lower Price by Raising FeeSandy BurneyMany Government Contractors measure profitability using Return on Sales (ROS), which includes Fee in the denominator. The FAR regulates fee percentages or Return on Cost (ROC). In negotiating sole-source contracts, this metric difference can result in the Contractor negotiating a higher risk adjusted cost and price to offset lower fees offered by the Government. The briefing provides a mathematical example of the tradeoff between Cost and Fee for a CPFF contract.2016Business Case Analysis
Developing a Dynamic Expense-Volume-Profit Model to Determine Break-Even pointWilliam KentAnalysts must be able to determine when a program will be profitable. This insight is used to drive contract negotiation, pricing, and strategy. Many factors need to be evaluated: expenses, revenue, and volume of sales. These factors must be determined based on limited historical data and extrapolated over a time horizon. This paper introduces a framework for analyzing these factors using learning curves, regression, and financial statement analysis to determine break-even.2016Business Case Analysis
Fixing the Flawed Nature of DoD BCA'sWilliam G. WilliamsonThe government is not a business that is profit-oriented. Monetary business metrics as a means of determining what items to consume are inherently flawed for the government since metrics such as Return on Investment’s assume the organization is a provider of the goods or services, not a consumer. A better approach for the government would be to measure the areas that the government most values, i.e. schedule and requirements. Using those areas as anchors, a best value metric can be developed, pointing the government to smarter acquisition decisions.2016Business Case Analysis
3D Printing in the DoD; Employing a Business Case AnalysisNicole Santos, Richard Shea, Robert Appleton3D Printing is the family of technologies that enables users to produce items on demand from CAD files or 3D scanning. The potential benefits to military logistics include cost savings, weight reduction, and responsiveness to the warfighters’ needs. To demonstrate and measure the benefits in the Department of Defense (DoD), a rigorous Business Case Analysis (BCA) will identify benefits and challenges to implementation including evaluating its costs, risks, and benefits.2016Business Case Analysis
Training Cost Analysts, a Cohesive Pedagogical Framework for SuccessKammy Mann, Danielle LainezHow do we train cost analysts? This paper will take a critical look at peer-reviewed academic research regarding pedagogy, or the method of teaching. Leveraging existing knowledge, current best practices and resources available, a framework for teaching and training will be proposed to the cost community. By establishing a consistent methodology for educating our professional workforce we can ensure that new members of the cost analysis field have the tools and skills to succeed.2016Business Case Analysis
Analyses of Alternatives for Space SystemsR. Alex WeklukThis presentation provides an overview of recent experiences performing satellite analyses of alternatives, including technical and cost trades. Our team developed a scalable mechanism to perform satellite sizing using physics and stochastic modeling. Once modeled, an array of costing methods is applied to develop repeatable and defensible estimates. The team made great strides in presenting the large number of nuances to decision makers.2016Business Case Analysis
Begin With a Joke? What John Oliver Can Teach Us About Communicating Cost EstimatesErin K. BarkelAre you having a hard time holding management’s attention when you explain the results of your estimates? It’s time to change your approach. Consider this: millions of people tune-in and click to watch John Oliver talk about cost estimates gone wild, and they stay tuned-in for 20 whole minutes. What can you learn from a comedian about explaining cost estimates? Maybe you should start with a joke.2016Business Case Analysis
Department of Defense Commercial-Off-The-ShelfHeather L. BrownNCCA partnered with DoD Enterprise Software Initiative (ESI) to periodically receive commercial-off-the-shelf (COTS) software orders with the intent of improving software cost estimating. As of today, more than 250,000 orders from the Army, Navy, Marine Corps, and other federal entities have been recorded. Together, these represent over six billion dollars in sales. NCCA examines this data and categorizes various product descriptions as licensed products, maintenance and support, or services in order to present statistical trends.2016Methods & Models
Naval Sea Systems Ship Command Logistics Requirements Funding Summary ToolJeremy EdenThe United States Navy has focused on planning and estimation of operations and support costs for programs. A task force created by Secretary Stackley (Assistant Secretary Navy Research, Development & Acquisition) to identify actions for achieving this goal recommended creating a logistics tool similar to that developed by the United States Marine Corps (USMC). This paper demonstrates the tool, development approach, cost estimating methodologies, interface, and deployment to the next generation2016Methods & Models
Footprints in the Sand: A Conversational Approach to Basis of EstimateFrank R. FlettThe Basis of Estimate (BOE) is the mechanism by which the cost estimator or cost proposal preparer hopes to convince the evaluator or reviewer that the method he/she took in preparing the estimate was “reasonable” (not the same as “correct”). This paper will present a technique of writing BOEs as if they were a part of a conversation with the evaluator/reviewer, who is, after all, a human being who will early on develop a distinct impression of the “goodness” of the end result.2016Methods & Models
Estimating the Cost of the Aegis Flight III Test PlatformJeremy GoucherThe Flight III upgrade to the Aegis Destroyer program brings unparalleled new capabilities to the surface Navy fleet which will require in depth testing prior to deployment. For live fire testing, the Navy is considering four unique options to procure, install, and test the required systems on an unmanned vessel. This paper discusses the available data, methodology, and results of the cost estimate. The paper will also discuss some of the technical challenges associated with each option.2016Methods & Models
Beyond Anderlohr: An Alternate Approach To Estimating Production BreaksBrent JohnstoneEstimating the cost impacts of production breaks has long been problematic. Use of the Anderlohr method is widely accepted, but requires a significant degree of estimating judgment and can produce substantially different answers based on the individual user’s assumptions. This paper suggests an alternate empirical methodology based on recent research on organizational learning and forgetting.2016Methods & Models
Delphi Methodology for Cost EstimatingCole J. Kupec, IIThe Delphi methodology can be a practical and valuable tool for cost estimators. This paper explores its proper implementation based on current research to include group selection, question formulation, statistical analysis, and the survey process. Special attention is paid to tailoring the Delphi method to cost estimating applications. A case study is presented for example purposes. The history of the Delphi method as well as modifications to the traditional Delphi method are also presented.2016Methods & Models
Beyond CSDRs: Collecting Contractor Cost Data for Detailed Cost EstimatesTim LawlessQuality cost estimates are based on actual costs, and for the Department of Defense, usually are acquired through Cost and Software Data Reports (CSDRs). However, CSDRs may not be available, complete or consistent for quality analytical purposes. This paper examines the collection and use of detailed contractor actual costs for estimates, such as with Priced Bills of Material and invoices, and their analytical benefits, perhaps supplanting CSDRs as the preferred data source for cost estimators.2016Methods & Models
Using Physics Based Reliability Methods for Should Cost EstimatesPatrick K. MaloneValue based should cost methodologies supporting Better Buying Power tenants can help realize significant MDAP cost savings over the life cycle. The use of Physics Based Reliability techniques in program development is described to meet affordability goals, identify cost drivers affecting reliability, availability and maintainability of weapon systems, and demonstrate PBR advantages over the classical reliability growth model with actionable plans to maintain technological superiority.2016Methods & Models
Workload-Adjusted Labor Cost ModelMichael MenderGovernment estimators typically use Forward Pricing Rate Agreements (FPRAs) to estimate future labor costs. FPRAs make assumptions about future workload, and can be inaccurate as realized workload often differs from projections. To enable understanding of this uncertainty, the Naval Center for Cost Analysis (NCCA) developed a lagged regression model that predicts future overhead costs as a function of workload, allowing NCCA to quickly and accurately assess the impact of Navy procurement decisions on acquisition programs.2016Methods & Models
Challenges in Estimating the Development Cost of Microcircuit TechnologyChristopher Price, Gurney ThompsonElectronic Technology continues to change at a rapid pace. Significant changes in the level of integration of electronic components dictate that circuit designs using these parts be redesigned every three to five years. Estimating the development cost of these new designs can be challenging. This paper will discuss these challenges in more detail, and describe some methods and tools available to rapidly and accurately estimate the development, production and O&S cost of these new technologies.2016Methods & Models
Estimating the Costs of Future Air-to-Ground WeaponsTom SandersThe weapons acquisition community has been acquiring air-launched weapons in a slowly-evolving manner for years. While technologies experience great leaps forward, acquisition strategies evolve at a slower pace. The AF Research Lab is evaluating revolutionary weapons designs that facilitate newly-emerging acquisition strategies…thereby presenting real challenges to cost estimators. This paper documents techniques for estimating costs of those future weapons.2016Methods & Models
The Space Situational AwarenessJames Smirnoff, Brennen WoodruffToday there are over a thousand operating satellites orbiting the earth providing a variety of critical functions such as communications, navigation, weather sensing, reconnaissance, and astronomy. As the space environment becomes more crowded, Space Situational Awareness (SSA) is increasingly important. This paper describes a new tool called the Space Situational Awareness (SSA) Cost Model which provides an objective and data-driven framework to estimate the cost of ground-based electro-optical components of future SSA architectures.2016Methods & Models
Crew and Space Transportation Systems Cost ModelRichard WebbAs part of the MSFC Engineering Cost Office’s new Program Cost Estimating Capability (PCEC) suite of cost estimating tools and capabilities, we are developing the Crew and Space Transportation Systems Cost Model (CASTS), a new, unique cost model for use in estimating crew and space transportation systems. This paper will provide an overview of the capabilities, estimating approach, historical database, and key features of CASTS as well as plans for future improvements.2016Methods & Models
Organization Cost Estimation as a Human Intensive Systems Engineering ProblemDavid Bloom, Robert WrightThis paper will compare and contrast the difference between Intensive and Non-Intensive Human Interactive Systems. Further, this paper will investigate the role of Governance, Culture, Process and Tools (GCPT) in each of the 2 systems, Intensive and Non-intensive. Finally, this paper will apply the GCPT Framework to the system of Organizational Cost Estimation and provide examples of results that Raytheon has seen in its effort to improve the accuracy, affordability and accountability of the current bidding/estimation process2016Management, EVM & Scheduling
Conducting Root Cause Analysis for Should Cost ManagementNancy A. R. Droz, Mr. Robert L. HernandezRoot-cause analysis (RCA) is typically performed after a technical failure or after a program anomaly is found. However, the technique is key to the development and selection of effective initiatives to reduce program cost. RCA is a critical aspect of successful Should Cost Management execution. The presentation will provide a how to approach of applying root cause analysis during the evaluation of program data to develop more effective costs reductions. Lessons learned will be presented.2016Management, EVM & Scheduling
EVM’s Potential for Enabling Integrated Cost-Risk ManagementDavid R. Graham, Bryn TurnerIntegrating the cost estimating, EVM analysis and technical risk management disciplines to realize comprehensive cost-risk management has proven elusive. Recently developed EVM cost-risk tools offer the potential to successfully enable integrated cost-risk management. The paper will illuminate the process of how integrated cost-risk management can be realized through cooperation of cost estimating, EVM analysis and technical risk management disciplines and briefly describe these new EVM cost-risk tools.2016Management, EVM & Scheduling
Applying Earned Value to Agile Development ProgramsBob Hunt, Michael ThompsonAgile Software and Agile Development continue to dominate the Federal acquisition arena. This presentation will address current trends in estimating and applying Earned value to Agile programs.2016Management, EVM & Scheduling
Provide Agile Analysis to Stakeholders Using Category Tagging ApproachEric HongIt is becoming more and more important for organizations to use data analytics for improved decision-making to create a competitive advantage and/or operate efficiently. This paper will describe the challenges in creating and implementing a strategy. It will identify potential reporting requirements and how to leverage existing work products as input into a standard structure feasible to perform analysis for different stakeholders. The proposed solution will increase the accuracy and agility of an organizations’ analysis.2016Management, EVM & Scheduling
Putting Schedule Quality Checks to the TestEric M. LofgrenAnalysts often use the 14-Point Assessment for Schedule Quality as a benchmark for determining the overall reliability of a schedule. But how much of the variation in schedule error can be explained by the results of the 14-Point check? This paper will use actual defense contract data to find the correlates of schedule reliability, measured using both the accuracy and the timeliness with which the schedule slip is predicted.2016Management, EVM & Scheduling
Preventing Program Management Pitfall Using Portfolio EstimatingNick Morales, Christopher DewberryEstimating the effort, time, and resources needed to complete project activities is one of the most challenging tasks that project managers must face. Along with the inherent uncertainty associated with managing these activities, additional management uncertainty occurs when there are multiple incremental efforts associated with that project. In order to handle the complication of managing several program increments, each with their own budget lines, portfolio estimating is a method that helps to provide program managers with a fully comprehensive overview of the resource requirements across multiple projects. Program managers can use the portfolio estimate as a tool to help make informed management decisions at the appropriate level and identify how those decisions impact the entire program.2016Management, EVM & Scheduling
Cost Consistency and Completeness as an Impossible ExerciseDavid L. Peeler, Jr.In previous papers, Hilbert’s Problems were used to propose and revisit Hilbert’s Problems for Cost Estimating. This paper employs Gödel’s theorem w/respect to Hilbert’s application onto cost. What can we learn about ourselves as estimators and where can we exert greatest impact with our estimates? Using Godel’s two theorems of undecidability as catalyst, we explore the effect and utility of exacting math and other notions on cost estimates specifically and programmaitics generally.2016Management, EVM & Scheduling
The Performance Metrics Model and StudyDerreck Ross, Haitham Ghannam, Richard C. LeeThe Performance Metrics Model and Study (PMMS) is a comprehensive technique for selecting a performance metric TCPI. The PMMS uses historical data of completed products to determine the best EVM estimation technique for similar products that are in progress. The result is an auditable and repeatable approach that increases the objectivity of an Estimate at Completion. This approach can be applied across any program that has historical data.2016Management, EVM & Scheduling
Fiscal Sustainability of Canada's National Defence ProgramPeter WeltmanThis report provides PBO’s estimate of the fiscal gap between the status quo budget allocations and the cost of sustaining Canada’s status quo national defence forces. Parliamentarians may wish to examine scenarios that will reduce or eliminate the gap between the cost of maintaining the current force structure and the amount of funding being allocated to paying for it.2016Management, EVM & Scheduling
Liars! Why Cost Estimators and Budgeters Inflate and Underestimate Costs!Travis Winstead, Ann HawpeProjects and programs regularly suffer from poor cost estimating techniques that result in insufficient budgets and unanticipated cost overruns. To issue a contract or set a realistic budget, the cost analyst must verify and validate the mountain of cost information and determine a fair and reasonable price for a contractor’s goods and services. This paper will discuss strategies to assist program analysts in reviewing cost estimates and strategies used to support Government negotiations.2016Management, EVM & Scheduling
Multiple Regression: Prediction & Confidence Intervals DemystifiedStacy M. DeanWe all know the value of prediction/confidence intervals as a measure of the certainty in the best fit line for a regression equation; however, once we get past bivariate regressions how to actually calculate these statistics gets a little murky. This paper attempts to de-mystify multivariate confidence/prediction interval calculation as well as provide some fun facts on the origin and history of these infamous intervals.2016Parametrics
How Predictive Analytics is Improving Parametric Cost EstimationAnthony A. DeMarcoPredictive analytics is defined as the use of data, statistical algorithms and machine-learning techniques to identify the likelihood of future outcomes based on historical data. This paper will explore the history and parallels of parametric estimating and predictive analytics. It will highlight how cost management professionals can use current predictive analytics tools, methods and talent to improve their estimates and their success rates.2016Parametrics
The Collinearity Kill Chain: Detect, Classify, Localize, NeutralizeBrian Flynn, Adam JamesThis paper examines the issue of multicollinearity from the fresh perspective of a statistical warfighter. A kill-chain of steps is offered to defeat this illusive foe. Heuristics, chi-squared tests, t-tests, and eigenvalues are the Aegis, Predator, Bradley, and Ma Deuce 50-caliber statistical equivalents used to detect, classify, localize, and neutralize the problem.2016Parametrics
Dialing for Dollars: Improving Cost Estimating Accuracy Through Data MiningBrittany Holmes, James Glenn, Christian SmartThis paper shows the benefits of leveraging obligation and expenditure data from DoD Enterprise Resource Planning (ERP) Systems to supplement cost data from CCDRs or IPMRs. One benefit is that cost trends and funding patterns can be analyzed with more precision because the ERP data is updated real-time vice monthly or annually. Another benefit is that more comprehensive CERs can be created because ERP systems capture all cost vice only contracts with cost reporting CDRLs.2016Parametrics
Develop PRESS for Nonlinear EquationsShu-Ping HuPredicted residual sum of squares (PRESS) and Predicted R2 statistics are commonly used in regression analysis outside the cost analysis industry to determine (1) how well the model predicts new observations and (2) if the model includes more independent variables than necessary. Currently, these two leave-one-out statistics are only available for ordinary least squares. This paper develops these two statistics for nonlinear equations and demonstrates their value in practical applications.2016Parametrics
Including Escalation in Cost Estimating RelationshipsChristopher JarvisPrior to developing a Cost Estimating Relationship (CER) the data should be normalized for effects not explained by the assumed CER model. This typically includes the effects of inflation and/or escalation which has recently generated considerable discussion within the DoD. In this paper we demonstrate two common CERs models and augment them to include an escalation term. Discussion will be made regarding the reliability of the solutions of the augmented models.2016Parametrics
Dangers of ParametricsAndy PrinceWhat if our models are not solving our estimating problems, but instead are the source of our problems? The purpose of this paper is to address this question. We will look at what a cost model is, and what it isn’t. We will examine how cost models appeal to our need for certainty and helps us create a good story for our cost estimate. We will take a look at a simple cost model to see what can go wrong when we trust the model over trusting the data. Finally, we will identify specific actions2016Parametrics
Development of an Additive Manufacturing Cost ModelF. Gurney Thompson, III, Grady NollThis paper discusses our research and development of a cost model for additive manufacturing (AM), also known as 3D printing. Working with academia and multiple AM companies, we investigated cost drivers, collected cost and technical data, derived AM CERs, and developed an approach for including AM components within a larger parametric cost estimate. This paper discusses the research approach, data collection, results, cost model implementation, and future work.2016Parametrics
X-Planes to X-Wings ~ Developing a Parametric Cost ModelSteve SterkIn today’s cost-constrained environment, NASA needs an X-Plane database and parametric cost and schedule model that can quickly provide rough order of magnitude predictions of cost and schedule from initial concept to first flight of potential X-Plane aircraft. This paper takes a look at the steps taken in developing such a model and reports the results. The challenges encountered in the collection of historical data and recommendations for future database management are discussed.2016Parametrics
Integrating Cost Estimating and Data Science Methods in RJosh Wilson, Laura BarkerData science is a growing, interdisciplinary field about the extraction of knowledge from data. It emphasizes the utilization of statistics and advanced analysis techniques to answer questions in a well-documented and reproducible manner. R is a free, open source programming language that is popular among data scientists. This presentation discusses the benefits of using R to supplement traditional cost estimating and data analysis practices.2016Parametrics
AoAs the Right WayAmanda Wilson, David Macdonald, Mariam UzunyanThe follow-ons to two large space acquisition programs from the Space and Missile Systems Center completed the AOA processes last year. They were set apart from past AoAs and AoA-like studies with the leadership and involvement of OSD CAPE and the AFCAA teams, with broad community involvement strengthening the teams. This change resulted in benefits, challenges and lessons learned; various visual tools were created to highlight risks, opportunities, and investment strategies to senior leaders.2016Parametrics
Cost Estimating Relationship Development HandbookJohn Fitch, Adam James, Alfred SmithDeveloping appropriate and well-defined parametric relationships is a foundational requirement for Cost Estimators. Current training for analysts focuses on various ordinary least squares regression (OLS), techniques, but ignores logical decision sequences employed by good analysts applying OLS and omits methods beyond OLS. The CER Handbook includes these techniques in a flow-chart format and guides analysts along each step in developing a parametric relationship.2016Parametrics
Cost Risk for Firm Fixed-Price ContractsDavid Biron, Christian SmartThe terms “cost risk” and “firm fixed-price contracts” seem contradictory. By design the contractor bears all risk under a firm fixed-price (FFP) contract. In spite of this, overruns often occur, and contractors have recourse to a Request for Equitable Adjustment (REA) when cost grows beyond the contract value. We present statistics on cost overruns for FFP contracts, some of which are significant, and show how to model risk for such contracts.2016Risk & Uncertainty
Modeling Prediction Intervals Using Monte Carlo Simulation SoftwareJames Black, Qing Qing "Q" Wu, The use of a prediction interval (PI) is a simple method of quantifying risk and uncertainty for a Cost Estimating Relationship (CER) derived from an Ordinary Least Squares (OLS) regression. Yet, few cost estimators implement PIs in their estimates despite their frequent use of CERs. This presentation will provide a step-by-step tutorial for modeling a PI for an example CER using Monte Carlo Simulation software and will identify the beneficial impact on the coefficient of variation (CV).2016Risk & Uncertainty
Introducing RIFT to Protect Your Uncertain ScheduleNicholas DeTore, Peter FredericThere are industry-accepted methods for allocating cost risk and uncertainty analysis results to detailed WBS elements; schedule results cannot be allocated the same way since duration behaves differently than cost. We present an innovative solution to this issue. The RIFT algorithm calculates a threshold date, for any task or milestone, that if exceeded puts the probabilistic project finish date in jeopardy. RIFT provides a new tangible metric to guide decision makers.2016Risk & Uncertainty
Risk vs. Uncertainty – What’s the difference?Melvin R. Etheridge, Jr.After years of applying “risk”, many in the profession still do not understand the difference between program risk and cost uncertainty in cost estimates. Determining confidence levels goes beyond simply applying the effects of programmatic risks. Simply put, risks have a probability of occurrence, while we know that quantitative inputs have some degree of uncertainty and must be treated as random variables. This paper explores analytical methods in Excel-based cost models using Monte Carlo s2016Risk & Uncertainty
A 'Common Risk Factor' Method to Estimate Correlations between DistributionsMarc GreenbergA ‘common risk factor’ method uses expert-derived risk factors to estimate correlation between two distributions. The premise is to estimate mutual information among risk factors that underlie each distribution. Approximation of such mutual information leverages properties of the joint probability distribution of a unit square. Geometric outputs of each pair of common random variables are compared to estimate common risk factor “intersections” that are, in turn, proxies of correlation.2016Risk & Uncertainty
How Regression Methods Impact Uncertainty ResultsBoyan Jonov, Shu-Ping Hu, Log transformation and weighted least squares are commonly used to develop multiplicative error cost estimating relationships. We objectively compare the two regression techniques and provide a sound defense of log-linear ordinary least squares (LOLS) to counter arguments against its use. We then demonstrate how uncertainty modeling varies substantially based on regression method. Lastly, we establish criteria to use so that LOLS leads to a justifiable uncertainty assignment.2016Risk & Uncertainty
The Zone System of Uncertainty AnalysisJeff McDowellThis paper presents a methodology for selecting cost uncertainty distributions and their dispersion inspired by the photography Zone System. The method seamlessly addresses five groupings of uncertainty across a sliding complexity scale. The distribution shapes and their parameters are drawn from patterns developed from the AFCAA Cost Risk and Uncertainly Metrics Manual (CRUAMM) body of work.2016Risk & Uncertainty
Cloud Total Ownership Costing: Considering the Technologies, Costs and BenefitsDan Galorath, Steven WoodwardCloud abounds with promises of drastically reduced costs, which are all attractive. However, Cloud computing can be ineffective in terms of technology, affordability and total cost of ownership in many situations. The assumption that cloud technology reduces government costs is often flawed. The benefits from cloud computing relating to time-to-market, savings, innovation and agility should be analyzed, providing balanced perspectives for the cloud opportunities presented.2016Software & IT
Data Collection for Agile ProjectsBlaze SmallwoodAgile software development projects produce a unique set of metrics, such as points and velocity, that can provide interesting insight into project progress. However, collecting this data requires specialized mechanisms, since no established standards exist, like SRDRs. This paper will discuss the types of data that are useful to collect for agile software development projects and mechanisms that have been used to collect them for several government projects.2016Software & IT
Apples and Oranges: a Presentation and Analysis of Results of Cloud Cost Calculators and Rate CardsDaniel J. HarperA recent effort for an Army customer examined over a dozen calculators and rate cards for estimating storage and hosting costs for cloud applications. This presentation will provide an overview of several calculators and tools, guidance for cost estimators on interpreting IT-centric inputs, and a discussion of similarities and variation in results. We will also present a cloud complexity plotter which provides a visual tool for explaining cloud cost and complexity drivers.2016Software & IT
A Primer on Agile Software Development for Cost AnalystsEric Hawkes, Cole J. Kupec IIAs Agile software development continues to become more common, cost estimators are compelled to adapt their methods to the Agile approach. This paper begins by explaining what Agile really is and how it differs from the common waterfall methodology. It also presents two case studies that detail how Agile was successfully implemented in a DoD acquisition program. Lastly, the paper proposes four innovative cost estimating methods for an Agile environment.2016Software & IT
The Cost of CyberAnn Hawpe, effrey VothA digital revolution is changing the nature of warfare and the cyber domain is now a top priority for all MDAP and MAIS programs. As the continued advancement of cyber warfare changes the future of land, air, sea, and space over the next decade, the cost community will need to consider the challenges facing defense programs and how to estimate the cyber security requirements within the construct of a reformed acquisition process.2016Software & IT
Maturing the Economic Aspects of Agile Development in the Federal GovernmentJennifer ManringAgile development has emerged in the commercial domain as a leading software development methodology, with growing adoption across federal agencies. This new approach for software development challenges traditional acquisition areas including cost estimation, economic analysis, and system engineering. This research provides a deeper understanding of the economic implications of agile development for the Federal Government and provides insights needed to understand the value proposition of an agile development environment compared with a traditional waterfall environment.2016Software & IT
Using Predictive Analytics for Cost Optimization Across Cloud WorkloadsZachary Jasnoff, David A. CassAs organizations move to the cloud, estimating costs proves difficult, presenting many challenges. Once estimated, cloud costs must be optimized taking into account workloads. Organizations failing to estimate cost based on workloads will either underestimate or over spend on cloud services, often with disastrous results. A framework for optimizing costs (spanning IaaS, PaaS, and SaaS) and determining the cost/benefit between public clouds, private clouds and hybrid clouds will be presented.2016Software & IT
SURF Process Summary & Initial Findings: A Deeper Focus on Software Data QualityNicholas Lanham, Corinne Wallshein, Ranae P. WoodsAs a result of the joint Software Resource Data Report Working Group (SRDRWG), several recommendations were generated by the supporting, multi-agency SRDRWG members to the Cost Leadership Forum. One of which led to the development of the first joint-agency SRDR Validation and Verification (V&V) guide as well as the SRDR User Review Function (SURF) subgroup. This study summarizes the results and data-quality improvement areas identified from 319 V&V comments generated by the SURF team.2016Software & IT
Cloud Solutions - Infrastructure, Platform or Software - Where should you go?Arlene F. Minkiewicz, Ashley HoenigkeCloud computing technologies promise to create some new and interesting challenges for the cost estimating community. How does one determine the effort and costs associated with migration of existing applications to the cloud? What challenges do the various cloud solutions (IaaS vs. PaaS,vs. SaaS) present? This paper presents a case study outlining the migration activities and cost implications for migrating the same application on each of the three cloud solution models.2016Software & IT
Estimating Small Scale Software Integration Efforts with a Large Scale Code BaseJoe BauerEstimating the cost and effort for testing and integration is often one of the most challenging aspects of software estimation. Integration effort depends greatly on the components under evaluation. When relatively small capabilities are introduced into large existing code bases, the integration impact due to the existing code may surprise stakeholders even when requirements are reduced. This presentation highlights software integration modeling for a major defense acquisition program.2016Software Sizing
Alternatives to SLOC in the Modern EnvironmentJeremy GoucherSource Lines of Code (SLOC) based estimating relies heavily on subject matter expert input. However, viable alternatives to SLOC are few. By mapping deliverables to functionality and functionality to effort, the authors propose an alternate technique with reduced reliance on SLOC. Agile development and some recent innovations represent alternatives as well. The authors will present an overview of the topic with examples, required data inputs, and estimating methodology.2016Software Sizing
Agile and Function Points: A Winning CombinationDaniel B. FrenchMany users of Agile have an incorrect belief that projects do not need to adhere to proper project management practices. However, it doesn’t advocate getting rid of metrics or PM processes. The most critical aspect of software development is software size. Function Points can be effectively used with Agile development and this presentation will demonstrate the benefits of using FP in many aspects of project management including estimation, schedule and product backlog.2016Software Sizing
Cost Element Structure for Cloud Migration Business Case AnalysisLorena AguilarIn 2010, the White House released its initial “cloud first” policy. Agencies have since struggled to identify all of the costs associated with cloud migration, and recent audits have criticized agencies for not properly assessing cloud-related costs and benefits. We have developed a cloud migration cost element structure (CES), which simplifies and standardizes cost data collection and estimating. We will present the structure and its applicability to business case and cost benefit analyses.2016Software Sizing
Collaborative Scoring: An Innovative Approach for Sizing Software ProjectsBlaze SmallwoodA major challenge facing government organizations is developing defendable cost and schedule estimates for software development projects, driven by adequately estimating software size. This paper will describe a viable new method for sizing software requirements that utilizes a collaborative scoring methodology similar to those used during agile sprint planning. It has been successfully implemented on several government projects and is a viable option on all software projects.2016Software Sizing
Process-Related Effort and Schedule Estimating Relationships for Software Cost EstimatingNichlas Lanham, Corinne Wallshein, Wilson RosaThe Naval Center for Cost Analysis will present comprehensive, updated findings of software size growth factors, effort estimating relationships (EER), and schedule estimating relationships (SER) with subsets of Department of Defense Computer Software Configuration Item records. This presentation focuses on software size (new, modified, reused, auto-generated, and total code). Subsets include maturity, application and super domain, language, contract type, and operating environment.2016Software Sizing
Health CERs - Using an interdisciplinary approach to estimating the cost of caring for Canada's veteransErin Barkel, Carleigh MalanikReports from the Veterans and the National Defence and Canadian Forces Ombudsmen have brought attention to the prevalence of mental illness among veterans. However, there are no published estimates for the cost of treating this growing population. A key obstacle to this research is that the relevant data is dispersed across several government organizations. We fill this research gap by leveraging actuarial and epidemiological studies of the veteran population to create new cost estimating relationships.2016Government Processes
NASA Commercial Crew Cost Estimating - A Look at Estimating Processes, Challenges and Lessons LearnedLance Cole, Rick BattleTo support annual PPBE budgets and NASA HQ requests for cost information for commercial crew transportation to the International Space Station (ISS), the NASA ISS ACES team developed system development and per flight cost estimates for the potential providers for each annual PPBE submit from 2009-2014. This paper describes the cost estimating processes used, challenges and lessons learned to develop estimates for this key NASA project that diverted from the traditional procurement approach and used a new way of doing business.2016Government Processes
Federal Shared Services: The future Shared-First and challenges facing the Federal Shared Services Providers (FSSPs) and their customersRuth Dorr, Marilyn Fleming, Sarah PopeThis paper will discuss the progress being made in the "Shared-First" approach for sharing of administrative services. Federal Agencies are being asked to execute their missions to ever increasing standards in a resource constrained environment. Administrative shared services have been identified as a way to reduce redundancy and improve efficiency in Federal business solutions. The identification of shared services as a Presidential cross-agency priority along with the designation of several providers for administrative shared services are recent developments in the movement toward shared services within the Federal government.2016Government Processes
Lessons learned from applying cost capability curves on an Air Force AoARobert GeorgiNew direction from Air Force leadership established a requirement for the presentation of life cycle cost versus capability tradeoff analysis for all AoA final reports. This presentation examines the approach that was used for a recent AoA that was a pilot program for the Cost Capability Curve process. When appropriately implemented cost capability analysis can provide significant advantages. There are lessons learned from attempting to implement this approach. We will also discuss examples where this approach does not apply.2016Government Processes
The Navy Modernization Program: Estimating the Cost of Upgrading AEGIS Guided Missile CruisersJeremy GoucherThe Navy is well into its 20 year, $16B (CY10$) plan to modernize 84 AEGIS warships. This cost estimate covers eleven of the planned ships and accounts for maintenance and upgrades to the ship and the combat systems. The estimate leverages recent contract and shipyard performance data and interviews with engineers, resulting in a detailed study and recommended cost savings initiatives. The methods and data in this estimate will assist ship modernization cost efforts across the fleet for the foreseeable future.2016Government Processes
Analysis of Alternatives from a Cost Estimating PerspectiveKirby Hom, Kenny RaglandSubsequent to the Weapon Systems Acquisition Reform Act of 2009, there have been other significant DoD policy changes that require Service sponsors to conduct an Analysis of Alternatives (AoA) prior to Milestone A. "About three-quarters of a program’s total life cycle cost is influenced by decisions made before it is approved to start development" [GAO-09-665]. From a cost estimating perspective, a successful AoA rests with the study approach, overall costing methodologies, key insights from trades analyses, and lessons learned.2016Government Processes
Mining for Cost Estimating Relations from Limited Complex DataMark Jacobs, Shawn HayesNASA’s robotic Earth and space science missions cover a diverse group of projects addressing complex science objectives that include challenging implementation approaches. Progress applying Principle Component Analysis techniques covering project management, systems engineering, mission assurance, integration & test, and spacecraft subsystems is described. Supporting data analysis efforts include a large detailed set of technical and programmatic input candidates that are analyzed to identify the primary spacecraft cost drivers.2016Government Processes
Manufacturing Assembly Plan (MAP) Tool: Bridging the Gap between Performance and the Construction ProcessRichard Lee, Haitham Ghannam, Edward WalshThis presentation explains the four module build plan for the VIRGINIA Class Submarine (VCS) program and provides an in-depth look into labor hours by Section 1/2A, Section 2B/5, Section 6/7, Section 8/9, Final Assembly and Test (FAT), and Post Shakedown Availability (PSA). Further understanding these concepts will not only allow cost estimators and program managers to better understand the current construction build strategy, but it will also aid in better aligning cost estimates with reality at the deck plate.2016Government Processes
It Ain't Easy Being Green... Sustainable Manufacturing with an eye on Cost Avoidance and StewardshipKammy Mann, Jessica BoatwrightThe DoD, one of the largest energy consumers in the world, has committed to sustainability. An important element of sustainable weapon systems acquisition currently not under the purview of the DoD’s strategic plan is sustainable manufacturing. This paper will leverage current research and industry best practices to identify the impact of sustainable manufacturing on cost avoidances relating to human health and environmental benefits. Recommendations will also be provided for complying with the FAR sustainability requirements.2016Government Processes
Business Case Analysis WizardLauren Nolte, Cassie Capots, Marcie Pfeuffer This paper describes the purpose and design of a "Business Case Analysis (BCA) Wizard" that was created for the Air Mobility Command (AMC) Enterprise Learning Office (ELO). As the value of BCAs within the learning transformation at AMC has grown, ELO identified the need to develop a tool that allows "non-cost analysts" to develop high-quality BCAs. This "BCA Wizard" guides users through a standard process of conducting a BCA via a user-friendly interface.2016Government Processes
Masters in Cost Estimating and AnalysisDan Nussbaum, Greg MislickThis presentation provides an update on the all-Distance Learning Masters Degree and/or Certificate Program in Cost Estimating and Analysis (MCEA / CCEA) offered at the Naval Postgraduate School (NPS).2016Government Processes
Methods Used in Pricing and Conveying NASA Assets for Use by Commercial ProgramsJames Roberts, Torrance LambingDifferent agreement types including EUL’s, Use Permits, Land Leases and others are used to convey NASA assets no longer needed by programs such as Space Shuttle and construction of the Space Station. By law, some situations prohibit NASA from pricing facilities in competition with the local market. Costs are estimated either via market-based or actual cost methods, according to circumstances. This paper will explain and clarify how these costs are estimated and pricing is established.2016Government Processes
ATC Zero: Estimating the True Impact of the Chicago Control Center FireKellie ScarbroughFor seventeen days, the Chicago Air Route Traffic Control Center went offline after a contractor set fire to the intricate communications network that controls some of the busiest airspace in the country. Workers scrambled around the clock for seventeen days to restore functionality to the center. This paper seeks to determine the true economic impact of the event, including the costs of time, operations, and environmental impacts.2016Government Processes
Where Does Your Cost Estimate Go?Emily Stafford, Christopher MetzYou have spent several months developing your cost estimate and presenting it to the customer. Now what? This paper will track what happens to your cost estimate after it has been delivered to the customer. It will follow the estimate through the budget formulation process, ultimately becoming a part of the President’s Budget. It will also discuss the cost estimator’s role during the budget process, and the influence of the budget process on future estimates.2016Government Processes
Facilitating Predictive Cost Analytics via Modelling V&VJohn SwarenWe are often asked to “fine tune” estimating software to unique situations. That “calibration” effort is key to Verification & Validation of parametric models. In this presentation, we show an example of using Predictive Analytics to develop a method to predict a Software-estimating model’s inputs as a function of Key Performance Parameters (KPPs). We demonstrate how a calibrated knowledgebase can best be used to V&V an estimating approach reflective of past and future requirement metrics.2016Government Processes
Hyperbolic Discounting and Defense AcquisitionsEric LofgrenCost estimation has largely been approached as an engineering problem which can be solved using data and techniques. While these are undoubtedly invaluable considerations, the broader production process is an inherently open-ended system subject to human interactions which are ill-defined and non-deterministic. This paper explores how behavior and institutions affect cost outcomes through the avenue of hyperbolic discounting, a time-inconsistent model of personal valuations. It will be argued that evidence for hyperbolic discounting in both government agents and contractor management have negative effects on cost estimates and will proffer some solutions based on commitment schemes.2016Government Processes
Cost Estimating Challenges in Additive ManufacturingJoe Bauer, Patrick MaloneAdditive manufacturing possibilities are virtually endless: from rapid prototyping, to printing food, to recreating human tissue. This presentation highlights the challenges of estimating 3D printing processes and offers some solutions to the estimator. Challenges include: amortizing non-recurring tooling across multiple programs, capturing software-centric nature of 3D printing, printing speed, and printing media limitations. This presentation highlights the use of existing cost estimating tools and areas for future development.2016Methods & Models
Long Term Impact of Ship Concepts on Operating and Support Cost AffordabilityEric Buller, Stuart Merrill, Bryan MillerAchieving affordability can be challenging especially once a program enters full rate production. Developing accurate Operating and Support (O&S) cost estimates that reveal tradeoff impacts and sustainment risks provides decision makers with affordability analysis during key acquisition events. We intend to demonstrate through analysis the impact of legacy and new ship concepts on O&S costs and program affordability. We will examine the cost tradeoffs for each ship concept in each O&S cost element.2016Methods & Models
The Peculiar World of Sole Source Contracting; Maybe There is a Better WaySandy Burney, Shawn LarsonIn 2014, 21% of U.S. Government contracts were sole source, accounting for 31% of total contracted value. They typically require negotiation and certification of all cost estimates, which can take up to 18 months. This light-hearted presentation will explore the peculiar world of developing and negotiating sole source estimates, including differences from budgetary and competitive estimating, as well as offer suggestions for achieving fair and reasonable sole source prices for less time and money.2016Methods & Models
Migration of Microsoft Excel Tools to Next Generation Platforms: "Can You Hear the Footsteps"Jeremy EdenAnalysis tools are popular solutions for extended deployments and at the same time an acceleration of platform migrations is occurring to keep up with rapidly changing technology and end user demands. This paper provides specific examples of platform migration challenges, resulting failures, and how they can be best addressed. It will also provide specific best practices for ICEAA member take-away that can be followed to help ensure smoother transitions across platforms.2016Methods & Models
Generalized Degrees of Freedom (GDF)Shu-Ping HuMinimum-Percentage Error/Zero-Percentage Bias (ZMPE) method is commonly used for multiplicative-error models. But ZMPE users do not adjust degrees of freedom (DF) for constraints included in the regression process. This generates misleading ZMPE CER fit statistics and underestimates the CER error distribution variance. Hence, ZMPE results are incomparable with those derived without constraints. This paper details why DF should be adjusted and proposes a Generalized Degrees of Freedom measure to compute fit statistics for constraint-driven CERs.2016Methods & Models
Bottom Up Methods of Estimating Depot Level RepairablesTim Lawless, James Black, Dave GoldbergDepot Level Repairables (DLR) costs comprise a significant share of total Operations and Support costs as they capture the scope of repairing and/or replacing failed hardware items. This paper will provide detailed descriptions of bottom up methods to estimate DLR costs using programmatic technical baseline documentation and acquisition costs as well as supply system requisitions and repair price data sources. Emphasis will be placed on producing data-driven analysis to inform cost estimates and estimate customers.2016Methods & Models
Seven Degrees of Separation: The Importance of High-Quality Contractor Data in Cost EstimatingCrickett PettyThe popular notion that any two people are linked to one another on average via a chain with “six degrees of separation” is based on a relatively small sample. Using massive data sets, researchers have since discovered that the average degrees of separation is closer to seven. This highlights the need for complete and thorough data in any analysis, and cost estimating in particular. Despite its importance the collection of high-quality contractor cost data is challenging. Processes for meeting these challenges are presented.2016Methods & Models
Biometric Analytics Cost EstimatingJoseph SarageThe purpose of this paper is to present cost estimating methods as it pertains to the Biometric Field by which statistical analysis techniques can be applied to better predict the costs and relationships with Biometric data. The paper will explore analytical methods in which Biometric data can be used for trend analysis, forecasting, and strategic decision making within various organizations and applications.2016Methods & Models
Interviewing Subject Matter ExpertsMelissa TeicherAs estimators, we often have to obtain technical information from Subject Matter Experts (SMEs) in order to provide thorough and accurate analysis. But sometimes, getting the information you need is easier said than done. This paper discusses the steps for interviewing SMEs in order to make the most of their time and your time. It explores the benefits of building a cooperative relationship of mutual respect and the “dos” and “don’ts” of SME consultations.2016Methods & Models
The Impacts of Design Change on Reliability, Maintainability, and Life Cycle CostAndreas Viberg, Oskar Tengo When making decisions about a design change to a technical system, it is essential to understand how it affects reliability and maintainability, as that directly impacts both overall performance and cost of operations and maintenance throughout the life cycle. LCC analysis based on modeling and simulation of relevant scenarios is an effective way to gain that understanding. This paper presents an approach for such analysis and illustrates its successful application in a recent project.2016Methods & Models
A Prime Contractor's View Using JCL Tools and TechniquesChip WoodsFor several years now, Lockheed Martin has been integrating cost and schedule data into a single model to perform integrated analysis. This paper will discuss Lockheed Martin’s JCL process, the tools used to develop JCL curves, and how this analysis can be used to shape and steer the program – including modeling funding constraints.2016Methods & Models
Class List of Characteristics: Changing the Landscape of Estimating by Removing BiasPaul BreonCLOC (Class List of Characteristics) is a coding system wherein each digit of the code indicates one of the cost drivers for the product/activity. CLOC gives the ability to “Call a thing by what it really is” and creating a universal language that can be utilized for any purpose. CLOC does NOT include Bias like quantity, cost, etc. These things are used in conjunction with the code to do any kind of analysis needed.2016Methods & Models
Achieving Affordability through high fidelity ROM cost estimatesEric BullerAchieving affordability can be challenging especially if early decisions are made without proper cost estimates. Developing high fidelity ROM cost estimates and identifying capability tradeoffs during the requirements generation phase can be critical to the success of achieving long term affordability. We intend to demonstrate through analysis that preforming high fidelity ROM cost estimates with capability tradeoffs allow decisions makers to informed choices on mission requirements that achieve affordability for future programs.2016Methods & Models
A Regression Method for Sparsely-Populated DatasetsDaniel BarkmeyerA method is presented for including sparsely-populated independent variables in a regression analysis without introducing subjectivity or statistical bias. The method is shown to reproduce the result of a traditional regression method for a fully populated dataset, and to converges to the same result on a sparsely-populated dataset as that dataset decreases in sparseness. Application of the method to develop a new, empirically-derived model for spacecraft testing schedule duration will also be presented.2016Parametrics
Improvement Curves: An Early Production MethodologyBrent JohnstoneLearning slope selection is a critical parameter in manufacturing labor estimates. Incorrect ex ante predictions lead to over- or understatements of projected hours. Existing literature provides little guidance on ex ante selection, particularly when some actual cost data exists but the program is well short of maturity. A methodology is offered using engineered labor standards and legacy performance to establish a basic learning curve slope to which early performance asymptotically recovers over time.2016Parametrics
The NRO CER Analysis ToolDonald MacKenzieThe NRO Cost and Acquisition Assessment Group (CAAG) uses its CER Analysis Tool, CERAT, to inform CER development. CERAT analyzes CERs for sensitivity to influential data points (IDPs). The impacts on estimated costs for a target data point when data points are removed are the primary outputs. The changes in recorded CER constants when data points are removed also help in assessing overall CER "stability." CERAT also provides several other aids to CER developers.2016Parametrics
A Parameter Driven WorldJeffrey PlattenParametric estimating is gaining acceptance. DFARS requires DOD contractors to maintain an estimating system. Parameters are the characteristics, features, or measurable factors that define a product or program. What makes a good parameter? What are the pros and cons of parametric cost estimating? How do you develop and maintain a parametric estimating database/model? What does the future hold, beyond just estimating? How do you gain parametric control over touch labor, support labor, and parts costs to improve affordability?2016Parametrics
Generating a Semi-Automated "Similar To" Basis Of Estimate from a Complex Parametric Hardware Cost Model for AntennasDanny Polidi, David BloomThis paper discusses the development of a "Similar-To" Basis Of Estimate (BOE) generation tool used in conjunction with a Complex Parametric Antenna Cost Model. Starting with the parametric model based on quantifiable sizing mechanisms which are designed to quickly and accurately calculate the "top-down" cost for all engineering and operations disciplines and functions required for antenna development and test, cost is generated, "similar to" programs are identified, and a BOE can be auto-populated.2016Parametrics
Macro-parametrics and the applications of multi-colinearity and Bayesian to enhance early cost modellingDale ShermonThis paper will consider the spectrum of parametric cost models from cost estimating relationships (CER) to micro and finally macro-parametric models. This will lead to a description of the Family of Advanced Cost Estimating Tools (FACET) parametric suite of models and their top-down capability to estimate costs incorporating the application of multi-colinearity and bayesian techniques.2016Parametrics
Using Historical Cost Reports to Derive System Specific FactorsJocelyn Tague, William BanksNon-Recurring Engineering (NRE) is often a cost driver for new, complex systems and must be given high attention during Milestone B cost estimating. Our team developed an approach for developing system specific Cost Estimating Relationship (CER) using actual costs for from combat weapon systems with shared capabilities, resulting in a credible defensible, yet flexible approach that can be replicated for a wide range of estimates.2016Parametrics
A CER Development Process for Spares EstimatingCheryl WilsonThis presentation shows the direction of a group of Spare Parts Estimators and how we are advancing our statistical skills and CER development processes. It shares the progress in our CER development journey attained through education, research, analysis and teamwork.2016Parametrics
Improving Parametric Software Cost Estimation Through Better Data CollectionBob KouryIn 2010 PRICE Systems LLC in conjunction with PEO STRI agreed to work together to improve the data being collected to support software development cost estimation in the command. The paper will compare and contrast the experiences of two other collection efforts with those of original effort. The paper will identify core findings which may be used universally to improve the quality, quantity, and applicability of data being collected.2016Parametrics
Building an Integrated Aerospace Electronics Development Parametric Model from the Ground UpDavid BloomThis paper presents the results of a multiple year effort to create a business specific set of integrated parametric cost models for complex electronics hardware development in such a way that all disciplines and activities that participate in that development process have a piece of the model and a stake in the data collection that drives the model. This "grounds up" approach provides that added benefit of cost allocation guidance once the development is approved.2016Parametrics
Essential Views of the Integrated Program Management Reports (IPMRs)Thomas Coonce, Glen AllemanDuring this session, the authors will describe key views of the IPMRs and other data that will allow a government stakeholder to a) understand the cost, schedule and technical status on applicable contracts, b) investigate the nexus of performance problems, and c) observe likely future problems. The presentation represents a “synthesis” of existing traditional earned value management metrics and other metrics designed to provide an integrative picture to assist in keeping the program “green”.2016Program Management
Meeting Fiscal Constraints: the Evolving Role of Performance MeasurementSteve Green, Kevin Davis, Kurt Heppard Performance measurement tools can reflect the strategic intent of management and help them prioritize initiatives to meet strategic objectives. In fiscally austere times, the Balanced Scorecard can help organizations address these tough decisions. Applying concepts from the Balanced Scorecard to cost analysis and budgeting, this paper develops an integrated aligned system of strategic goals, performance metrics, and cost parameters at a university facing pressure to manage costs. There may be lessons learned for the DoD.2016Program Management
The graph is always greener on the other side: examples of how to interpret and fix bad visual representations of dataStephen KetchamIn my role performing independent reviews of estimates, I’ve seen a quality spectrum of different types of graphs, charts, and forecasts. These visuals summarize key points and remain in decision-makers minds, often providing the convincing point to win funding approval for the project. Unfortunately they can also be poorly crafted, employ scare tactics or even frame the data in a misleading way. This presentation provides bad examples of visuals, and discusses possible improvements.2016Program Management
Seeing What We Need to See vs. What We Hope to See: Visualizing Integrated Cost and Schedule Data for Earned Value AnalysisBrian LeachWhen applied properly, data visualization can serve analysis by showing us what is otherwise difficult to see (and not always what we hope to see). With the emergence of the UN/CEFACT XML standard, integrated cost and schedule data can be shown in exciting new ways. This paper will examine and explore innovative new ways to show integrated cost and schedule data which have practical applications in earned value management system analysis.2016Program Management
Different Incentives in Government ContractingJennifer LeottaThis paper shows stock prices are used as the primary incentive for executives of publicly traded government contractors rather than profit sharing contracts. Looking at the reaction of companies’ stock prices to public and private sector news, this paper shows stock prices do not capture “bad news” for both the public and private sector customers compared to overall market indicators; and would make poor contract incentives. Finally, alternate methods of contract incentives will be explored.2016Program Management
Agile Project Management Controls: Issues with EVM and a Viable AlternativeOmar Mahmoud, Blaze SmallwoodAs more government software programs adopt agile practices, program managers are trying to find effective ways to monitor and control cost and schedule. The challenge is that traditional project control techniques, like EVM, are not well suited to deal with the requirements fluidity and new terms/metrics that come with agile projects. This paper will describe the inherent issues with using EVM on agile projects and offer a recommendation for a more effective solution.2016Program Management
The Psychology of Cost EstimatingAndy PrinceResearch into human psychology has yielded amazing findings into how we process information and how we use information to make decisions. Cost estimators can use psychology and behavioral economics to improve not only our cost estimates but how we communicate and work with our customers. By understanding how our customer’s think, and more importantly, why they think the way they do, we can have more productive relationships and greater influence. The end result will be better decisions by the decision makers.2016Program Management
Cost ManagementGeorges TeologlouCost Management has a key role in project management and is the management trend due to the actual financial and economic worldwide constraints. Cost Management solution was built to be in line with the recommendations of the GAO (US Government Accountability Office) that stated that a large number of Defense projects are at risk of experiencing cost overruns and schedule delays. GAO found that the two major causes of project failure were first an inaccurate initial estimate and secondly poor subsequent cost control.2016Program Management
Using Total Lifecycle Cost as the Basis for Proposal Cost EvaluationRyan TimmThis paper discusses benefits and obstacles of using the TLCC of an entire system instead of the bid price for the purpose of proposal cost evaluation. An example acquisition of a space vehicle (SV) ground system explains the impact to TLCC of portions of the ground system developed by the SV manufacturer, SV operations, external software interfaces, government furnished IT infrastructure, and legacy sustainment until transition. Using TLCC allows higher-level requirements and enables innovative architectures.2016Program Management
A Maturity Analysis of the American and European Cost Community Mark GilmourThe Cost Engineering Health Check (CEHC) is a cost capability benchmarking tool used to identify strengths and weaknesses within organizations costing functions. In 2014 the authors of this paper conducted a CEHC at the ICEAA conference in Denver. That workshop used e-voting to anonymously gather the views of the audience in attendance. This paper presents a summary of these results and contrasts them against data gathered from European workshops, providing a snapshot of the international costing capability.2016Program Management
Presenting EstimatesMelissa TeicherDecision-makers rely on cost estimators to guide them in making the most educated judgments on what is best for the program or organization. A thorough and well-organized presentation can make the difference between an easy decision and a difficult one. This paper discusses how public speaking skills contribute to an effective estimate presentation. It also delivers tips and tricks on how to make an estimate presentation more visually pleasing and easier to understand.2016Program Management
Using Stochastic Optimization to Improve Risk PrioritizationEric DrukerMinimizing the cost of complex programs is critical for government agencies trying to meet their missions in today’s fiscal. Today, identifying cost-savings measures is a manual procedure where an analyst must make an educated guess as to what risks to mitigate. This process is manual and slow. This paper will outline a methodology for using the emerging field of stochastic optimization to automate the identification of cost-savings measures on complex programs.2016Risk
Why Are Estimates Always Wrong: Estimation Bias and Strategic MisestimationDan GalorathThe biggest source of estimation error usually comes from people, either by accident or strategically. This is a disaster because viable estimates are core to project success as well as ROI determination and other decision making. Most people don’t know how to estimate. Those who estimate are often optimistic, full of unintentional bias and sometimes strategically mis-estimating. This paper discusses the issues of estimation bias and strategic mis-estimation as well as how to mitigate these issues.2016Risk
A "Common Risk Factor" Method to Estimate Correlations between DistributionsMarc GreenbergA “common risk factor” method uses expert-derived risk factors to estimate correlation between two distributions. The premise is to estimate mutual information among risk factors that underlie each distribution. Approximation of such mutual information leverages properties of the joint probability distribution of a unit square. Geometric outputs of each pair of common random variables are compared leading to an estimated ‘intersection’ of pair of common risk factors. This intersection is then assumed to be a proxy of correlation.2016Risk
Data-Driven Guidelines for Correlation of Cost and Schedule GrowthSidi Huang, Mark Pedigo, Chris Shaw When running Monte Carlo simulations, one of the most difficult aspects to account for is correlation between the independent variables. Ignoring correlation generally reports overly optimistic analysis results; industry standard suggests that when no additional insight is available, a default value of 0.2 can be used. This study analyzes historical NASA program data to develop data-driven guidelines for correlation. Moving forward, this study can be replicated to other industries and organizations with critical schedules.2016Risk
Risk Mitigation: Some Good News after the Cost / Schedule Risk Analysis ResultsDavid HulettThe results of your cost-schedule risk analysis say you are likely to be overrun on both schedule and budget. Are you are out-of-luck? No. Do not forget the risk mitigation phase. Risk mitigation typically proceeds from a prioritized risk list developed from analysis of the risk and uncertainty-loaded schedule. Complete mitigation of risks is not usually feasible, and uncertainty is usually deemed to be irreducible. Some good news comes from the risk mitigation actions.2016Risk
A Mathematical Approach for Cost and Schedule Risk AttributionFred KuoThe level of sophistication in cost estimate has progressed substantially in the last decade. This is especially true at NASA where, spearheaded by the Cost Analysis Division (CAD) of NASA Headquarters, cost and schedule risks in terms of confidence level have been codified. In this paper, the author attempts to employ the portfolio approach in calculating risk attribution; first applying to a portfolio of cost risks and extend the same concept to schedule risks analysis.2016Risk
Triangular Distributions and CorrelationsJennifer LampeTriangular distributions are often used in estimating cost risk because the math is relatively simple. The concept of an estimate having a lower limit, an upper limit, and a mode is easy to understand. We show the math behind triangular distributions and correlations. What does it mean to say that two risks are 30% correlated? How do risk analysis programs handle triangular distributions and correlations? How can you use random numbers in Excel to generate a Monte Carlo simulation for any defined triangular distribution?2016Risk
The Impact of Selected Assumptions and Core Tenets on Schedule Risk Assessment Results (A Progressive Model Comparison)James QuilliamIn the quest to ensure the sound representation of Schedule risk assessment (SRA) simulations this case study will provide a progressive model comparison of schedule risk assessment assumptions and core tenets. The elements of this approach will focus on the: methodology and tools; the progressive assumptions and core tenets applied; conclusions and lessons learned for practitioners. This will greatly enhance the understanding and confidence that leadership and project teams have in the schedule risk assessment results.2016Risk
Wine as a Financial InvestementMichael Shortell, Adiam WoldetensaeThe purpose of this presentation is to look into wine as an investment and analyze its characteristics as a financial asset. The goal here is to look into vintage wines through time, analyze its profitability and predict how investing in wine could be a good source of diversification. Our objective is to detect the types of wines that are a good source of investment and their associated risk. We aim to identify a good investment portfolio for a wine collector in order to maximize returns and minimize the associated risk.2016Risk
Beyond Correlation: Don't Use the Formula That Killed Wall StreetChristian SmartRisk models in cost estimating almost exclusively rely on correlation as a measure of dependency. Correlation is only one measure (among many) of stochastic dependency. It ignores the effect of tail dependency, when one extreme event affects others. This leads to potential outcomes that do not make sense such as a program with a large schedule overrun but no cost overrun. This method has been widely blamed for the financial crisis a few years ago. The copula method is introduced as a way to overcome these deficiencies.2016Risk
Connecting the Dots: Integrating the risk cube with the POEChristina SnyderDoD guidance instructs program managers to include risk and uncertainty analysis in their cost estimates. Program managers also maintain a risk cube that tracks known program risks and assigns a likelihood and consequence. Risks, regardless of if they are programmatic, technical, or schedule defined, will have an effect on the overall program cost. This paper investigates the linkage between identified risks and POE cost and shows how to translate the risk cube into dollars.2016Risk
Methodology for Integrating Risk mitigation Activities into JCL analysisJames Taylor, Justin HornbackThe intent of this whitepaper is to outline a an improved methodology for capturing risk mitigation activities for both cost and schedule into a Joint Confidence Level (JCL) analysis and assessing mitigation effectiveness.2016Risk
The Continual Pursuit of the One True Software Resource Data Reporting (SRDR) DatabaseRemmie Arnold, Peter BraxtonThis paper will demonstrate the ability for one database to store Software Resource Data Report (SRDR) source data in its original form, as submitted by WBS element and reporting event. This database allows evaluations, interpretations, and annotations of the data, including appropriate pairing of Initial and Final reports; mapping of SLOC to standard categories for the purposes of determining ESLOC; normalization of software activities to a standard set of activities; and storage of previous assessments.2016Software Estimating
Agile Estimation Using Functional MetricsThomas CagleyAgile methods have matured and are now being integrated into many different approaches to the development of software. Estimation has been problematic for all methods, agile to plan based and therefore it tends to be a lightning rod for experimentation and synthesis such as is being described in this paper. Agile Estimation Using Functional Metrics has presented a path for integrating the discipline found in functional metrics with the collaborative approaches found in agile parametric estimation.2016Software Estimating
Put some SNAP in your estimating modelDavid LambertAn effective estimating model is dependent upon having the capacity to accurately size the problem domain for the solution being developed. Function Point Analysis is a widely accepted industry and ISO standard for sizing functional requirements; however, it does not take into account the sizing of non-functional requirements. Learn how the newly released Software Non-Functional Assessment Process (SNAP) can provide a more complete and accurate assessment of project size.2016Software Estimating
Forecasting Software MaintenanceAlea DeSantisProviding a defensible estimate for software maintenance can be complicated. By overlaying historical data with lifecycle events one can derive event driven factors and use trend analysis to forecast future Software Problem Reports. Topics addressed are: common-cause variation, special-cause variation, severity level, complexity level, and decay rate. This method moves away from using a general defect rate calculation to a more complete product-specific calculation.2016Software Estimating
Effective use of Function Points for Analogous Software EstimationDaniel FrenchAnalogous estimation is often used to estimate software projects, especially in the early phases, because it is believed to be quicker and easier than other techniques. However, many organizations do not employ this technique effectively and correctly. One pitfall is improperly applying the technique & basing these estimates on Software Lines of Code (SLOC). This presentation will discuss common mistakes made when using analogous estimates as well as how the use of IFPUG function points can greatly improve this estimation technique2016Software Estimating
NASA Software Cost Estimation Model: An Analogy Based Estimation MethodJairus HihnSoftware development activities are notorious for their cost growth. While there has been extensive work on improving parametric methods there is very little focus on the use of models based on analogy and clustering algorithms. In this paper we will describe the methodology being used in the development of a NASA Software Cost Model using data mining clustering algorithms and evaluate its performance by comparing it to estimates from COCOMO II, a calibrated COCOMO II, liner regression, and K-nearest neighbor models.2016Software Estimating
Estimating Agile Software DevelopmentBob HuntThe use of Agile Software Development continues to grow in large and small Federal programs. These programs often utilize Story Points as a key size metric. This presentation will roovideman update on Agile Cost Estimating techniques and emphasize how to,utilize Story Points as a key size metric.2016Software Estimating
How Much Does Software Maintenance Cost?Cheryl Jones, James Judy, John McGarryTo accurately estimate, budget, allocate, and justify the resources required to meet evolving mission and service affordability requirements across the system life-cycle we must be able to answer the fundamental question: How much does software maintenance cost? This presentation provides a summary of the findings from an ongoing U.S. Army study that is focused on developing improved software maintenance estimation processes, cost relationships, and associated models.2016Software Estimating
Exploring DoD Software Growth: A Better Way to Model Future Software UncertaintyNicholas LanhamThis presentation highlights trends and estimating relationships derived from the April 2014 SRDR data. This analysis provides the estimating community with several new variables to better model future software growth uncertainty. In addition, the analysis includes growth by contract type, domain, and program type as well as introduces a multivariate model to predict final hours. All of the relationships included within this slide set have been derived from the best 400+ data points, were approved by DCARC, and are DoD-specific.2016Software Estimating
Considerations in Data Center Migration Life Cycle Cost Estimating: A Joint Information Environment (JIE) PerspectiveDennis McKeonThe potential benefits of consolidating data centers has led to the creation of initiatives such as the DoD Joint Information Environment (JIE). The goal is to reduce IT infrastructure costs by consolidating system support functions, improve hardware utilization, and lower energy and facility costs. This paper is from the perspective of an organization migrating to a consolidated data center. It will describe the migration steps, cost estimating approach, benefits, risks, and organizational challenges from their perspective.2016Software Estimating
At the Intersection of Technical Debt and Software Maintenance CostsArlene MinkiewiczWard Cunningham introduced the technical debt metaphor in 1992 to ease business leaders understanding of the cost of making poor decisions during software development. While this author has seen many different definitions of technical debt, it is basically an indication of the amount of ‘should fix’ issues that remain in production code. This paper reports on an on-going research effort to understand technical debt and its implications on software maintenance cost and effort estimation.2016Software Estimating
Early Phase Software Cost and Schedule Estimation ModelsWilson RosaSoftware cost estimates are more useful at early elaboration phase, when source lines of code and Function Points are not yet available. This study introduces effort estimation models using functional requirements as a size predictor and evaluates development schedule using estimated effort based on data from 40 military and IT programs delivered from 2006 to 2014. Statistical results show that estimated functional requirements and peak staff are significant contributors and that estimated or actual effort is a valid predictor of development duration.2016Software Estimating
Software size measures and their use in software project estimationHarold van HeeringenUsing the right size measure is extremely important for software cost engineering purposes. However, the number of software size measures is growing rapidly and there is a lot of confusion in the industry with regard to the methods that should be used. The advantages and disadvantages of the major software size measures are discussed and recommendations are given on what size measures should be used to estimate the different types of software systems.2016Software Estimating
Software Size GrowthCorinne WallsheinNCCA will present their findings including cost estimating relationships (CERs) of software size growth using selected subsets. CERs will be compared using initially estimated parameter distributions against CERs using software size. Other parameters include requirements, staff size, hours, and duration. As the 2014 update of the Joint Agency Cost and Schedule Risk and Uncertainty Handbook advises use of standard error for uncertainty bounds, NCCA will compare bounds for recommended CERs, by selected data subsets.2016Software Estimating
NPS and AFIT Masters Degree in Cost Estimating and AnalysisDaniel Nussbaum, Greg MislickThis presentation provides an update on the Joint, all-Distance Learning Masters Degree in Cost Estimating and Analysis offered at the Naval Postgraduate School (NPS) and Air Force Institute of Technology (AFIT).2014Business and Art of Cost Estimating
A Comprehensive CES and BCA Approach for Lifelong LearningKevin Cincotta, Darcy LilleyThe Air Force Air Mobility Command (AMC) Enterprise Learning Office (ELO) mission is to transform AMC into a premier Air Force learning organization, achieve learning through optimum approaches and develop Mobility Airmen into life-long learners who demonstrate institutional Air Force competencies with a positive approach to managing their own learning. In this context, learning has three main components: training, education, and experience. ...2014Business and Art of Cost Estimating
BOE Development: Scope Evaluation and CriteriaMichael Butterworth, Demetrius PradoCurrent basis-of-estimates (BOEs) do not identify succinctly, how the scope of the estimate was derived and how actuals used for build-up, analogous and parametric methodologies fit the technology and product under consideration. We believe that one of the many problems leading to poor BOE development revolves around the lack of identifying the scope of work and understanding the effort needed in particular skill mix to perform the task. ...2014Business and Art of Cost Estimating
Long Term Affordability through Knowledge Based Bid & ProposalZachary JasnoffAccording to a 2013 GAO report, "positive acquisition outcomes require the use of a knowledge-based approach to product development that demonstrates high levels of knowledge before significant commitments are made. In essence, knowledge supplants risk over time." Often times, acquisition proposals that are not knowledge-based introduce significant risk and while seeming reasonable in the short term, cannot sustain long term affordability. ...2014Business and Art of Cost Estimating
What Happens to a Cost Estimate When It's Done?William Barfield, David BachWhat happens to a cost estimate when it is done, or "finished?" Cost estimates are used to support a wide variety of financial decisions. When estimates are done and the decision has been made, is the estimate still useful after the decision, or does it become "shelf-ware?" We surveyed the international cost community to determine how we develop, document, use, and archive our various kinds of cost estimates. ...2014Business and Art of Cost Estimating
Update On The Cost FACTS (Factors, Analogies, CER’s & Tools/Studies) Group – Enhancing KM By Leveraging Enterprise Social NetworkingDaniel Harper, Ruth DorrCost FACTS is a community-driven initiative to bring together the cost estimating community across MITRE, Government agencies, the contractor community, and academia to share reusable, non-restricted cost information and to encourage dialogue throughout the worldwide cost community. Cost FACTS provides an easily accessible repository of reusable cost estimating FACTS, i.e., Factors, Analogies, Cost Estimating Relationships (CERs), and Tools/Studies. ...2014Business and Art of Cost Estimating
Space Shuttle Cost Analysis: A Success Story?Humboldt MandellIn the aftermath of the highly successful Apollo lunar program, NASA struggled for a few years to find a meaningful program which would satisfy long range national space strategies, as well as reflecting the realities of the rapidly changing political environment in the nation. The Space Shuttle emerged from the need to lower the costs of orbital cargo delivery for construction of a space station and enabling Mars exploration, but also was highly constrained by DOD requirements ...2014Business and Art of Cost Estimating
Big Data Meets Earned Value ManagementGlen Alleman, Thomas CoonceWhen the result of an action is of consequence, but cannot be known in advance with precision, forecasting may reduce decision risk by supplying additional information about the possible outcomes. Data obtained from observations collected sequentially over time are common. Earned Value Management is an example where project performance data (BCWP) is collected from the status reports of planned work (BCWS) and a forecast of future performance is needed to manage the program. ...2014Earned Value Management
Don't Let EVM Data Mislead YouSteve SheamerEVM data is a popular data source for cost estimators and for good reason; in theory, it should provide most of the data needed to develop an estimate for a program. For completed programs, it provides historical costs by WBS and for programs that are in work it provides a measure of the work completed, work remaining, and forecast of the work remaining. But during a period of frequent cost overruns, estimates built using EVM data often fail to forecast the extent of program overruns...2014Earned Value Management
Trust but Verify - An Improved Estimating Technique Using the Integrated Master Schedule (IMS)Eric LofgrenIt has long been the wonder of management why the Integrated Master Schedule (IMS) fails to give advanced warning of impending schedule delays. The oft-touted Government Accountability Office (GAO) 14-Point Check for Schedule Quality analyzes schedule health using key metrics, leading one to assume that such a test authenticates schedule realism. Why, then, do practitioners find themselves caught off guard to slips when their IMS appears in good health? ...2014Earned Value Management
A Cure For Unanticipated Cost and Schedule GrowthGlen Alleman, Thomas CoonceFederal programs (DoD and civilian) often fail to deliver all that was promised and many times cost more than estimated and are often late. Delivering programs with less capability than promised while exceeding the cost and planned undermines the Federal government’s credibility with taxpayers and contributes to the public’s negative support for these programs. ...2014Earned Value Management
Unleashing the Power of MS Excel as an EVM Analysis ToolAllen Gaudelli, Steve SheamerWhat do you do if you need to analyze or report on EVM data and you don't have access to (or can't afford) the latest industry software? Nearly everyone has a very powerful analysis and reporting tool on their desktop with the flexibility and capability to consolidate cost, schedule, and risk drivers into a single view. In this presentation, we will show you how to leverage and manipulate the inherent capabilities of Microsoft Excel to build interactive EVM dashboards ...2014Earned Value Management
Design to Cost: Misunderstood and misappliedErin Barkel, Tolga YalkinThe Canadian Department of Defence maintains that concerns over cost overruns are overstated because it adopts a design to cost approach. According to the US Government, design to cost "embodies the early establishment of realistic but rigorous cost targets and a determined effort to achieve them." From the beginning of a project to its completion, "[c]ost is addressed on a continuing basis as part of a system's development and production process." ...2014Earned Value Management
Testing Benford's Law with Software Code CountsChuck Knight, Chris KaldesWhen analyzing a data set, common thinking may lead one to suspect that the leading digit of each data point would follow a uniform distribution, where each digit (1 through 9) has an equal probability of occurrence. Benford's law, to the contrary, states that the first digit of each data point will conform to a nonuniform distribution. More specifically, it states that a data point is more likely to begin with a one than a two, a two more likely than a three, a three more likely than a four, and so on. ...2014Information Technology
Improved Method for Predicting Software Effort and ScheduleWilson Rosa, Barry Boehm, Ray Madachy, Brad Clark, Joseph P. DeanThis paper presents a set of effort and schedule estimating relationships for predicting software development using empirical data from 317 very recent US DoD programs. The first set predicts effort as a function of size and application type. The second predicts duration using size and staff level. The models are simpler and more viable to use for early estimates than traditional parametric cost models. Practical benchmarks are also provided to guide analysts in normalizing data.2014Information Technology
Costs of Migration and Operation in the CloudArlene MinkiewiczAt one level cloud computing is just Internet enabled time sharing. Instead of organizations investing in all the Information Technology (IT) assets such as hardware, software and infrastructure they need to meet business needs; cloud computing technology makes these resources available through the Internet. Cloud computing allows an organization to adopt a different economic model for meeting IT needs by reducing capital investments and increasing operational investments. ...2014Information Technology
How I Continued to Stop Worrying and Love Software Resource Data ReportsNicholas LanhamThis presentation highlights the trends and cost estimating relationships derived from detailed analysis of the August 2013 Office of the Secretary of Defense (OSD) Software Resource Data Report (SRDR) data. This analysis was conducted by Nicholas Lanham and Mike Popp and provides as follow-on analysis to the August 2012 SRDR brief developed and previously presented by Mike Popp, AIR 4.2. ... 2014Information Technology
Mobile Applications, Functional Analysis and Cost EstimationTammy PreussThis presentation will demonstrate how to derive cost estimates at different stages in a project's lifecycle by using function points and the advantages of using an FP based size estimate over a SLOC based estimate. The intended audience is software cost estimators, project managers, and anyone who is interested in software measurement.2014Information Technology
In Pursuit of the One True Software Resource Data Reporting (SRDR) DatabaseZachary McGregor-DorseyFor many years, Software Resource Data Reports, collected by the Defense Cost and Resource Center (DCARC) on Major Defense Acquisition Programs (MDAPs), have been widely acknowledged as an important source of software sizing, effort, cost, and schedule data to support estimating. However, using SRDRs presents a number of data collection, normalization, and analysis challenges, which would in large part be obviated by a single robust relational database. ... 2014Information Technology
Optimizing Total Cost of Ownership for Best Value IT Solutions: A Case Study using Parametric Models for Estimates of Alternative IT Architectures and Operational ApproachesDenton Tarbet, Kevin Woodward, Reggie ColeBecause of a variety of architectures and deployment models, Information Technology (IT) has become more and more complex for organizations to manage and support. Current technology IT system architectures range from server based local systems to implementations of a Private Cloud to utilization of the Public Cloud. Determining a "best value architecture" for IT systems requires the ability to effectively understand not only the cost, but the relative performance, schedule and risk associated with alternative solutions. ...2014Information Technology
Estimating Hardware Storage CostsJenny Woolley, William BlackEstimating Commercial-off-the-Shelf (COTS) hardware storage volume and cost requirements can be challenging. Factors such as storage type, speed, configuration, and changing costs can potentially lead to estimating difficulties. This is especially true when a Redundant Array of Independent Disks (RAID) configuration is implemented. Due to the multiple attributes that can vary within each RAID level, as well as other factors that may influence the total storage volume needed, developing relationships for estimating long-term storage costs can become complicated. ...2014Information Technology
Relating Cost to Performance: The Performance-Based Cost ModelMichael Jeffers, Robert Nehring, Jean-Ali Tavassoli, Kelly Meyers, Robert JonesFor decades, in order to produce a cost estimate, estimators have been heavily reliant on the technical characteristics of a system, such as weight for hardware elements or source lines of code (SLOC) for software elements, as specified by designers and engineers. Quite often, a question will arise about the cost of adding additional performance requirements to a system design (or in a design-to-cost scenario, the savings to be achieved by removing requirements). Traditionally, the engineers will then have to undertake a design cycle to determine how the shift in requirements will change the system. ...2014Information Technology
Lessons Learned from the International Software Benchmark Standards Group (ISBSG)Arlene MinkiewiczThis paper will introduce the ISBSG and the database that are available from the ISBSG. It then provides details of the data driven approach applied to develop these templates - discussing research approach, methodology, tools used, findings and outcomes. This is followed by a discussion of lessons learned including the strengths and weaknesses of the database and the strength and weaknesses of the solutions derived from it. While particularly relevant to software estimators, this paper should be valuable to any estimator who lacks data or has data they are not quite sure what they might do with it.2014Information Technology
Software Maintenance: Recommendations for Estimating and Data CollectionShelley Dickson, Bruce Parker, Alex Thiel, Corinne WallsheinThe software maintenance study reported at ICEAA in 2012 and 2013 continued to progress in 2013 in spite of the high data variability. This presentation summarizes the past years' software maintenance data collection structure, categorizations, normalizations, and analyses. Software maintenance size, defect, cost, and effort data were collected from Fiscal Years (FY) 1992 - 2012. Parametric analyses were performed in depth on available variables included in or derived from this U.S. Department of Defense software maintenance data set. ...2014Information Technology
An Update to the Use of Function Points in Earned Value Management for Software DevelopmentMichael Thompson, Daniel FrenchThis presentation describes the opportunity that was presented to the team and how the recently completed pilot program was developed and implemented to address it. The authors will address how effective the pilot program was as far as identifying and resolving issues, measuring earned value, as well as the challenges and lessons learned with the development, implementation, and sustainment of the FP based EVM process.2014Information Technology
The Federal IT Dashboard: Potential Application for IT Cost & Schedule AnalysisDaniel HarperFederal agencies have experienced a growing demand for rapid turnaround cost and schedule estimates. This need is increasing as the pressure to deploy systems rapidly mounts. The push for Agile SW development compounds this problem. A critical component in cost estimating is the data collection of costs for the various elements within the estimate. Analogous programs constitute a robust source for credible estimates. The problem is how to find analogous programs and how to capture the cost of elements within those programs at a sufficiently detailed level to use in a cost estimate and in a timely manner so that the cost data is still relevant. ...2014Information Technology
Trends in Enterprise Software Pricing from 2002 to 2011Ian Anderson, Dara LoganOne of the biggest challenges in the cost estimating community is data collection. In the Information Technology (IT) cost community, technology is always evolving, while the data capturing it tend to be scarce and more difficult to use in building solid cost models. Fortunately, NCCA learned the Department of the Navy (DON) Chief Information Officer (CIO) has been collecting benchmarking measures, including pricing, since 2002 under the Enterprise Software Initiative (ESI) Blanket Purchasing Agreements (BPAs). ... 2014Information Technology
Estimating Cloud Computing Costs: Practical Questions for Programs Kathryn ConnorCloud computing has garnered the attention of the Department of Defense (DoD) as data and computer processing needs grow and budgets shrink. In the meantime, reliable literature on the costs of cloud computing in the government is still limited, but programs are interested in any solution that has potential to control growing data management costs. ...2014Information Technology
The Agile PM Tool: The Trifecta for Managing Cost, Schedule, and Scope Blaze Smallwood, Omar MahmoudThe growing number of DoD software projects that are adopting an “Agile” development philosophy requires cost estimators to not only adapt the methodologies and metrics they use to estimate software development costs, but also re-think their models to give PMs the information they need to effectively manage these programs. The Agile PM Tool is one manifestation of this trend as it provides a logical, dynamic approach for helping the government effectively manage the cost, schedule, and scope of their “Agile” projects.2014Information Technology
Which Escalation Rate Should I Use?Nathan HonsowetzConducting life cycle cost estimates requires time frames of 10, 20, even 30 years, and with such long time frames it's important to use appropriate escalation indices. Escalation can have a significant impact on cost estimates, especially estimates with longer time frames. However, often cost estimators insert a "standard" escalation index into their models without considering whether that index is appropriate for their estimate. ...2014Life Cycle Costing
Ground Vehicle Reliability Analysis Using the Mean Cumulative FunctionCaleb FlemingThis paper surveys and outlines the fundamental MCF methodologies and explanations detailed in-depth in Wayne Nelson's Recurrent Events Data Analysis for Product Repairs, Disease Recurrences, and Other Applications. Applying the MCF to vehicle maintenance data reveals recurring component failure behaviors, develops new guidelines for interpretation, and assists in data normalization and validation.2014Life Cycle Costing
Cost Overruns and Their Precursors: An Empirical Examination of Major Department of Defense Acquisition ProgramsAlan Gideon, Enrique Campos-Nanez, Pavel Fomin, James WasekThis paper proposes a model of acquisition program future cost for two specific acquisition domains - aircraft and ships - that takes into account the non-recurring developmental costs defined at program approval and each domain's historic tendencies to exceed planned program cost. Technical and non-technical reasons for these observations are discussed. ...2014Life Cycle Costing
System Utilization: An In-depth Method of Modeling and Measuring Military Manpower CostsOmar MahmoudEstablishing defendable cost estimating methodologies for capturing military manpower costs is a key component in any Program Life Cycle Cost Estimate. With a proven and systematic approach to estimating military manpower costs, a program can be confident in selecting a proper course of action among competing alternatives when conducting an EA, appropriately designate their ACAT level, avoid the pitfalls that lead to over/under estimating or double counting costs, and above all obtain a high level of confidence from their resource sponsor and Milestone Decision Authority.2014Life Cycle Costing
Integrating Sustainability into Weapon System Acquisition within the Department of DefenseWalt Cooper, Remmie ArnoldDoD acquisition and logistics professionals use the term sustainment to describe the support needed to operate and maintain a system over its lifetime. In the context of the DoD acquisition process, sustainability involves using resources to minimize mission, human health, and environmental impacts and associated costs during the life cycle. This paper will present a draft version of "DoD Guidance - Integrating Sustainability into DoD Acquisitions," initial findings from pilot studies, and the challenges and road ahead. ...2014Life Cycle Costing
Cost Analysis & Optimization of Repair Concepts Using Marginal AnalysisJustin WoulfeOPRAL is an analytical model for determining the optimal repair locations and spares allocations in a multi-level hierarchical support organization to optimize Life Cycle Cost. With this model, the analyst can either treat repair decisions as fixed and given as input parameters, or using the OPRAL algorithm, evaluate several different repair strategies in order to find the optimal one, considering all aspects of life cycle cost. ...2014Life Cycle Costing
A Balanced Approach to Meeting Fiscal ConstraintsSteve Green, Kevin Davis, Kurt HeppardThe effective and systematic use of cost and budgeting information is a critical component of strategic planning and decision making in most organizations. The Department of Defense's (DoD) current operational environment, scarce resources, and conflicting stakeholder expectations are resulting in extreme fiscal constraints. The result is the need to reconsider missions and goals, reassess priorities, entertain force structure alternatives, and ultimately reduce budgets. ...2014Cost Management
Cost Overruns and Cost Growth: A Three Decades Old Cost Performance Issue within DoD's Acquisition EnvironmentLeone YoungFor the last three decades, the US Department of Defense (DoD) has been encountering program performance issues such as inaccurate and unrealistic estimations for its acquisition programs, and its effort of eliminating cost overruns and cost growth phenomenon has been deemed ineffective as well. This paper examines and consolidates multiple government reports and studies, which pertain to the cost performance of DoD that has been viewed as unsatisfactory and problematic. ...2014Cost Management
Supplier Cost/Price Analyses - Best Practices for evaluating supplier proposals and quotesMike MardesichDuring proposal planning, preparation, and review, an important but often over looked aspect is the evaluation of a supplier's proposal or quote. Requirements in the Federal Acquisition Regulations (FAR) provide for certain visibility into supplier's proposals and quotes, depending on the value of the proposal/quote. Additionally it is critical to note the FAR puts the onus of supplier cost (or price) analysis on the prime contractor. ...2014Cost Management
Innovative Business Agreements and Related Cost & Pricing Methods at NASA in Support of New Commercial ProgramsTorrance Lambing, James RobertsThis paper and presentation, focusing on Kennedy Space Center, will discuss changes and new methods of pricing and estimating the costs of NASA facilities and services to be provided to outside entities for use in new Commercial Space endeavors. It will also give an overview of new NASA regulations and documents that establish policy and guidance for entering into Agreements and how they are priced under the various types of Agreement currently being used at NASA. ...2014Cost Management
The Other RCA: Restaurant Cost AnalysisPeter BraxtonThe Weapon Systems Acquisition Reform Act (WSARA) of 2009 highlighted the importance of Root Cause Analysis, or RCA, but its conduct remains shrouded in mystery. In illustrating the central role of risk and uncertainty analysis in cost, Dick Coleman often made the provocative pronouncement to the effect of "You can't stand outside a restaurant with a menu, and the people you'll be dining with, and a calculator and get within 10% of the final bill, so what makes you think you can estimate a complex multi-billion-dollar acquisition program with that precision?!" ...2014Cost Management
Intelligence Mission Data Cost Methodology GuidebookEugene Cullen, III, Matthew SchumacherIn 2013, Booz Allen Hamilton developed and authored the Intelligence Mission Data Cost Methodology Guidebook (IMD CMGB) for the Defense Intelligence Agency's (DIA) Intelligence Mission Data Center (IMDC). This guidebook is the official costing manual for life-cycle mission data planning, required by Department of Defense Directive (DoDD) 5250.01 "Management of IMD within DoD Acquisitions," and defines costing methodologies, procedures, and processes that result in OSD Cost Assessment & Program Evaluation (CAPE) and Government Accountability Office (GAO) compliant cost estimates that are required by DoD Acquisition Systems. ...2014Cost Management
Achieving Affordable Programs NRO Cost Analysis Improvement Group (CAIG) Support of Cost Driver IdentificationLinda Williams, Pat Kielb, Eileen DeVillier, Jay MillerThis paper will describe the approach taken over the last several months to identify cost drivers and summarize findings that senior leaders found helpful to the decision making process. In summary, as budgets reductions trigger asset affordability reviews, the NRO CAIG has been able to assist in identifying total organizational cost drivers and provide a framework for future cost reduction opportunities.2014Cost Management
Gödel's Impact on Hilbert's Problems Or Cost Consistency and Completeness as an Impossible ExerciseDavid PeelerIn a previous set of papers, the idea of using Hilbert's Problems as a construct to propose and recently revisit the status of a list of Hilbert's Problems for Cost Estimating. This paper similarly employs Gödel's enlightenment with respect to Hilbert's attempts onto the cost estimating community. What can we learn about ourselves as estimators and where can we exert the greatest impact with respect to the use of our estimates? Using Godel's two theorems of undecidability as catalyst, we will explore the effect and utility of exacting math and other motions on cost estimates specifically and programmaitics generally.2014Cost Management
A New Cost Management Policy for Your Organization: An Integrated Approach?Tom Dauber, Woomi Chase, Ken OdomDeveloping a robust Cost Management Policy is a key driver to the success of any organization, regardless of size or industry. The policy should ensure cost control measures that are valid and effective, risks are mitigated, solutions are delivered on time, and profits/ROIs are maximized. The Cost Management Policy should be a systematic approach to managing cost through-out the life cycle of a program through the application of cost engineering and cost management principles. ... 2014Cost Management
Right Sizing Earned Value Management for Your ProjectGordon KranzEarned Value Management (EVM) is a program management tool that provides data indicators that can be used on all programs to enable proactive decision making throughout the program lifecycle and facilitate communication across the program team. Each program has unique attributes that should be considered when determining program management and reporting requirements, including, but not limited to, contract size and type, scope of work, complexity, risk, technology maturity, and resource requirements. ...2014Cost Management

Military Construction Cost Estimating

Nicole Barmettler (Sullivan)An informative presentation on construction cost estimating specifically dealing with military facilities. Within this topic, the author will identify, define, and explain the cost methodologies and cost adjustment factors considered when developing construction cost estimates for general military facilities. Project costs will be exemplified by illustrating a breakdown and walkthrough of the process. The author will specifically discuss the process involved in a five year facility acquisition timeline that is usually required for a typical major military construction effort, which is defined by a project cost exceeding $750,000.2014Methods and Models
Cost and Performance Trades and Cost-Benefits AnalysisSteven IkelerThis paper will discuss the basics of Trades Analysis and the C-BA. It will include what the cost analyst should do to prepare and what information to collect during the Trades Analysis. One observation is that non-traditional Work-Breakdown Structures need to be considered and the analyst should model potential second and third order effects beforehand. ...2014Methods and Models
Lessons Learned from Estimating Non-Standard Equipment Returning from Overseas OperationsMichael MetcalfThis paper explores challenges in estimating the cost of retaining this equipment. Items that present unique challenges include: the type classification and full materiel release processes; repair, reset, and upgrade; short- and long-term storage; and knowledge retention and loss of wartime experience. We will also explore funding challenges; moving targets in number and configuration of retained systems; the transition of O&S from wartime contractor-based to peacetime organic; training; and system disposal and divestiture.2014Methods and Models
Weibull Analysis MethodErik Burgess, James Smirnoff, Brianne WongThe NRO Cost and Acquisition Assessment Group (CAAG) develops time-phased estimates for space systems in support of milestone decisions, budget formulation, and other program assessment tasks. CAAG relies on parametric budget-spreading models based on historical data to provide objective analysis and credible information before contract award or early in a program when there is little or no execution history available. ...2014Methods and Models
Study of Cost Estimating Methodology of Engineering DevelopmentMyung-Yul LeeThe purpose of this paper is to study the actual engineering labor hours for FSED Program and PE/PI project and to create a parametric estimating model in a weapon system, especially cargo aircraft. The C-17 final assembly facility in Long Beach, California closes in 2015. It is worthwhile to review historical data for the C-17 FSED program and PE/PI project, both programs have engineering development program in view of estimating methodology research.2014Methods and Models
Validation and Improvement of the Rayleigh Curve MethodMichael MenderThe presentation begins with a brief overview of the Rayleigh method, followed by an explanation of R-squared and the issues with non-linear functions. We then discuss the method for linearizing the Rayleigh function and then using it for linear regression. We then provide a brief demo of the tool we developed and then move on to a discussion of our results pertaining to the performance of the Rayleigh method. ...2014Methods and Models
Rotorcraft Cost Model Enhancements for Future ConceptsF. Gurney Thompson IIIThis paper will discuss the ongoing research efforts to improve upon existing rotorcraft cost estimation capabilities, from both a cost estimating relationship (CER) update and a model development perspective. We will share our approach, many of our findings, and any lessons learned. Efforts currently underway include data collection, updates to existing cost models and their CERs, adding support for new aircraft types and technologies, and the addition of new analysis capabilities to better understand total ownership cost.2014Methods and Models
Kill Vehicle Work Breakdown Structurennifer Tarin, Christian Smart, Paul TetraultThis paper provides an alternative to Appendix C: Missile Systems for inclusion in MIL-STD-881C, the Department of Defense standard for Work Breakdown Structures (WBSs). The Missile Defense Agency (MDA) produces interceptors that are similar to missiles with the exception of the payload element. Where Appendix C defines the payload element with a limited set of WBS elements, the MDA interceptor payload, referred to as a kill vehicle, includes a large collection of significant WBS elements. ...2014Methods and Models
Meet the Overlapping Coefficient: A Measure for Elevator SpeechesBrent LarsonYou've seen this picture before. . . a plot of two overlapping probability distributions. You may have created one with an overlay chart. Typically this graphic contrasts two cost distributions so as to illustrate similarity, difference or change. However, seldom seen is a number that quantifies the overlap or area shared by both distributions. ... 2014Methods and Models
Excel Based Cost Roll Up MethodMatthew LeezerThis paper will show the user how to create a custom function (sumlowerlevel) using Visual Basic and apply this function in Excel to generate a report that will save time and increase the accuracy of estimates. The method uses the lookup function to capture the cost of purchased components and subsystems and uses the custom function for all make assemblies to roll up the cost of the purchased parts. ...2014Methods and Models
The Role of Cost Estimating in Source SelectionAnnette BarliaThe analysis will focus on the process of developing an IGCE and then utilizing it to evaluate vendor proposals for the acquisition of new technology. It will demonstrate that a strong IGCE facilitates source selection. If the cost estimate is developed the right way, organizations will have more leverage during contract negotiations with vendors, and the acquisition will run smoothly and meet program goals. ...2014Methods and Models
Automated Costing to Identify Opportunities to Reduce Direct Material SpendJulie Driscoll, Dale MartinThis session will cover how technology is automating the costing process through integrating costing solutions with CAD, PLM and ERP; pulling information about components directly from CAD files; and using an intelligent cost engine that evaluates manufacturing routings for feasibility and cost effectiveness. We will look how these solutions enable automated batch costing of components and are used by manufacturers to support cost reduction projects. ...2014Methods and Models
Moving Beyond Technical Parameters in our CERsEric Druker, Charles HuntOne of the frequent criticisms of JCL analysis (integrated cost and schedule risk analysis) has been that the results typically exhibit coefficients of variation (CV) that are orders of magnitude less than those seen in parametric estimates of similar scope. During a recent NASA research task examining how parametrics estimates can be linked to program management artifacts, the research team stumbled upon a characteristic of our Cost Estimating Relationships (CERs) that almost certainly leads our parametric estimates to have higher than necessary CVs. ...2014Parametrics
Using Dummy Variables in CER DevelopmentShu-Ping Hu, Alfred SmithThis paper explains the reasons for using dummy variables in regression analysis and how to use them effectively when deriving CERs. Specific guidelines are proposed to help analysts determine if the application of dummy variables is appropriate for their data set. This paper also demonstrates some common errors in applying dummy variables to real examples. An application using dummy variables in splines (to derive the fitted equation as well as the intersection) is also discussed.2014Parametrics
Bayesian Parametrics: Developing a CER with Limited Data and Even Without DataChristian SmartThis paper discusses Bayes' Theorem, and applies it to linear and nonlinear CERs, including ordinary least squares and log-transformed ordinary least squares.2014Parametrics
Tactical Vehicle Cons & Reps Cost Estimating Relationship ToolCassandra Capots, Jeffery Cherwonik, Adam James, Leonard Ogborn When estimating Operating and Support (O&S), it is reasonable to assume that as reliability increases, consumable and reparable parts ("cons and reps") cost should decrease (less frequent repairs), while as vehicle price increases, parts cost should increase (more expensive parts). Developing a dataset to support cost estimating relationships (CERs) for the Army's Tactical Vehicle fleet is a significant challenge. ...2014Parametrics
Unmanned Aerial Vehicle Systems Database and Parametric Model ResearchBruce Parker, Rachel Cosgray, Anna Irvine, Brian Welsh, Patrick Staley, Praful PatelThis handbook documents the first two years of research sponsored by NCCA and ODASA-CE. With the inclusion of UAS in the United States' (U.S.) military arsenal, the government has a desire to understand the components of a UAS including the air vehicle, GCS and payloads, the development and production process, and the O&S implications of these systems. ...2014Parametrics
Building a Complex Hardware Cost Model for AntennasDavid Bloom, Danny PolidiThis paper discusses the development of a Complex Antenna Cost Model based on quantifiable sizing mechanisms which are designed to quickly and accurately calculate the "top-down" cost for all engineering and operations disciplines and functions required for antenna development and test. ...2014Parametrics
ESA Project Office Cost ModelHerve JoumierThis paper describes the definition and the implementation of a Project Office parametric cost model aimed at defining a reference manpower allocation based on fair judgement and rational fair modelling. It has been developed to improve the cost estimations capability of ESA providing outputs that are used by agencies for comparison with contractors' proposals. ...2014Parametrics
Improving the Accuracy of Cost Estimating Relationship for Software SystemsDavid WangIn this paper, we leverage recently published results on the statistical characterization of schedule and cost risks to analyze the prediction accuracy of CER for software systems. Our analytical analysis and empirical statistical analysis of actual code size growth data suggest that the statistics of code size estimate can also be characterized by fat-tail distributions....2014Parametrics
Hybrid Parametric Estimation for Greater AccuracyWilliam RoetzheimThis talk will discuss hybrid parametric estimation based on HLO catalogs, and give examples of the application and accuracy of this technique within organizations including the State of California, Halliburton, IBM, Procter and Gamble, and multiple top 25 financial institutions.2014Parametrics
Linking Parametric Estimates to Program Management Artifacts (LPEPM)Mike Smith, Ted Mills, John SwarenA common fate of parametric cost and schedule estimates is that they fall into disuse as a Project's own artifacts (e.g. Work Breakdown Structure (WBS), budget, schedule, risk lists, etc.) are created and mature. Parametric estimates typically do not map cleanly to WBS or schedule-derived artifacts, allowing a sense among Project Managers (PMs) ? rightly or wrongly ? that "parametric estimates are fine, but they don't reflect my project." ...2014Parametrics
Impact of Full Funding on Cost Improvement Rate: A Parametric AssessmentBrianne Wong, Erik BurgessThe NRO Cost and Acquisition Assessment Group (CAAG) currently houses data collected from various U.S. Government organizations, including the Department of Defense and NASA. These data points are pooled with NRO data and used in Cost Estimating Relationships for space hardware, which underpin CAAG estimates for major system acquisition programs, aiding in the development of programs and budgets. ...2014Parametrics
Developing R&D and Mass Production Cost Estimating Methodologies for Korean Maneuver Weapon SystemDoo Hyun Lee, Sung-Jin Kang, Suhwan KimIn this research we have attempted to establish a CER development process to meet the current need, and found certain cost drivers for the Korean historical maneuver weapons system data, using Forward selection, Stepwise Regression and R square selection. We have also developed a CER model for production labor costs, using Learning rate which has been generally applied to estimate valid production labor costs. ...2014Parametrics
Excel Based Schedule Risk and Cost EstimatesWilliam EvansThe emphasis on robust cost and schedule estimating solutions has resulted in the creation of multiple solutions for analysts and clients. Excel based integrated cost and schedule risk is only one methodology for solving client problems. Incorporating cost and schedule risk in Excel leads to an increased ability to audit and trace the schedule and cost risk methodology throughout an Excel based PLCCE, improving the confidence and robustness of the estimate. While there are hurdles to the implementation of an Excel based schedule risk solution, when combined with form controls, the benefits to PLCCE auditability and usability are immense.2014Risk
Using Bayesian Belief Networks with Monte Carlo Simulation ModelingMarina DombrovskayaOne of the main aspects of creating a Monte Carlo simulation cost estimate is the accuracy in defining uncertainty and risk parameters associated with the cost components of the model. It is equally important to assess and accurately represent inter-dependencies between uncertain variables and risks, which are measured via correlation. Since oftentimes historical data is insufficient for a rigorous statistical analysis, both probability distribution and correlation are commonly estimated via a subject matter opinion. ...2014Risk
Expert Elicitation of a Maximum Duration Using Risk ScenariosMarc GreenbergThis paper provides with a brief review on a current method to elicit a most-likely commute time, a "practical maximum" commute time and risk factors that contribute to commute delays. This paper continues by showing how these risk factors can be organized into an objective hierarchy of risk factors, leading to the creation of a customized risk work breakdown structure (WBS). ...2014Risk
Quantifying the Necessity of Risk Mitigation StrategiesJames Northington, Christopher Schmidt, Chuck KnightThis paper will begin by highlighting flaws with the current risk management process, walk through the new proposed methodology for risk mitigation, and provide a quantitative example of the process in action using raw data. In the end, the proposed methodology will provide a greater understanding of program risks, a measurement of importance of implementing a risk mitigation strategy, a measurement of the mitigation strategy's subsequent impact, and a quantitative measurement of benefit for Program Mangers to defend their risk mitigation strategies.2014Risk
Improved Decision Making with Sensitivity AnalysisBlake BoswellIn this study, we review common applications of SA methods to project estimation including a description of each method as well as its advantages and disadvantages. Additionally, we explore the topic of Global Sensitivity Analysis (GSA), which is a process for measuring the overall contribution of uncertain model inputs to variation in model outputs and is a popular technique for model validation in engineering and life sciences. ...2014Risk
Affordability Engineering for Better Alternative Selection and Risk ReductionMarlena McWilliams, Bob KouryThis paper will outline the process and steps to how to implement affordability into your estimating environment to understand system requirements vs. system costs and affordability; and provide best value identifying and accepting the most affordable, feasible, and effective system or alternative. The need to evaluate and assign a best value is essential to both the government (DoD) and the contractors supplying systems / alternatives to the government.2014Risk
Risk Adjusted Inflation IndicesJames BlackIt is often observed that Office of the Secretary of Defense (OSD) inflation rates are different than prime contractor specific inflation rates seen in Forward Pricing Rate Agreements/Proposals (FPRAs/FPRPs) and in commodity group composite rates (e.g. Global Insight indices). Yet, it is a standard practice in many cost estimating organizations to use OSD inflation rates for escalating costs in estimates without giving consideration to a range of different possible inflation rates. ...2014Risk
Critique of Cost-Risk Analysis and Frankenstein Spacecraft Designs: A Proposed SolutionMohamed Elghefari, Eric PlumerIn this paper, we present a historical data driven probabilistic cost growth model for adjusting spacecraft cost Current Best Estimate (CBE), for both earth orbiting and deep space missions. The model is sensitive to when, in the mission development life cycle, the spacecraft cost CBE is generated. The model is based on historical spacecraft data obtained from the NASA Cost Analysis Data Requirements (CADRe) database. ...2014Risk
A Next Generation Software Cost ModelJairus Hihn, Tim Menzies, James JohnsonIn this paper we will summarize our findings on effort/cost model estimation and model development based on ten years of software effort estimation research using data mining and machine learning methods. We will then describe the methodology being used in the development of a NASA Software Cost Model that provides an integrated effort, schedule, risk estimate, as well as identifying the changes in the project characteristics that are most likely to improve a given projects cost-schedule performance and risk exposure.2014Space
NASA's Phasing Estimating RelationshipsChad Krause, Erik Burgess, Darren ElliottCost and schedule estimating in support of budget formulation is limited when cost phasing is not considered. As a result, NASA'a Office of Evaluation (OE) Cost Analysis Division (CAD) initiated a review of historic mission funding profiles for the purpose of corroborating current phasing profiles and optimizing future budgeting performance. Actual expenditures by year, technical parameters, and programmatic information were compiled and normalized from NASA's extensive library of CADRe (Cost Analysis Data Requirment) documents for programs since 1990. ...2014Space
NASA Instrument Cost Model (NICM)Hamid Habib-Agahi, Joseph Mrozinski, George FoxThe NASA Instrument Cost Model (NICM) includes several parametric cost estimating relationships (CERs) used to estimate NASA's future spacecraft's instrument development cost. This presentation will cover the challenges associated with creating cost models in an environment where data on previously built instruments is 1) sparse, 2) heterogeneous and 3) book-kept differently by the various NASA centers and support institutions. ...2014Space
The NASA Project Cost Estimating CapabilityAndy Prince, Brian Alford, Blake Boswell, Matt PitlykThe paper begins with a detailed description of the capabilities and shortcomings of the NAFCOM architecture. The criteria behind the decision to develop the PCEC are outlined. Then the requirements for the PCEC are discussed, followed by a description of the PCEC architecture. Finally, the paper provides a vision for the future of NASA cost estimating capabilities.2014Space
Developing Space Vehicle Hardware Nonrecurring Cost Estimating Relationships at the NRO CAAGRyan Timm, Jan SterbutzelThis paper builds on our 2012 SCEA conference briefing that described the NRO CAAG approach to developing Space Vehicle (SV) hardware Cost Estimating Relationships (CERs) for Nonrecurring (NR) engineering. These CERs are developed from the NRO CAAG's cost database of more than 2300 space hardware boxes, and can stand as alternatives to other popular parametric tools, like the nonrecurring CERs in USCM or NAFCOM. ...2014Space
NASA JCL: Process and LessonsSteve Wilson, Mike StellyOur paper will describe JCL implementation and address the creation, implementation, evolution, inherent benefits, inherent issues, its ultimate place among program management's decision-making toolset, and hard recommendations for organizations hoping to wage successful JCL campaigns. Real-world examples will be referenced, including those from the Constellation, Commercial Crew, and Orion spacecraft development programs. ...2014Space
NASA Project Cost Estimating Capability: New Analyses for Spacecraft EstimatingBrian Alford, Frank A. (Andy) PrinceThe NASA Project Cost Estimating Capability (PCEC) is a free, publicly-available parametric tool for estimating the cost of spacecraft and transportation systems for scientific and human exploration of our solar system and beyond. In development since late 2013, PCEC replaces the NASA / Air Force Cost Model (NAFCOM). This presentation will discuss the new overall philosophy and data analysis underlying PCEC, the challenges encountered during development, and provide a high-level overview of the2016 Int'lGovernment Perspectives
OSCAM - US Navy's Operating and Support Cost Estimating ToolStephen CurramOSCAM is a simulation based tool for operating and support cost estimating that has been used by the US Navy for 20 years. It plays an important role in submissions for US DoD procurement milestones. An overview and brief history of OSCAM will be given, with examples of equipment programmes that have used OSCAM. It will conclude with lessons learned, including insights on the type of data required for operating and support cost estimating.2016 Int'lGovernment Perspectives
Master of Cost Estimating & Analysis BriefGreg MislickGreg Mislick will provide an update on the all-distance learning Master’s Degree and/or Certificate Program in Cost Estimating and Analysis (MCEA / CCEA) offered at the Naval Postgraduate School (NPS). Entering its sixth year, the MCEA program has proven to be a success for both the students who take the program and the DoD Services and other US Government Executive Agencies in which the students work. Expert cost estimates are key underpinnings for critical government processes dependent on credible and reasonable cost estimates, from budgeting to cost-benefit and financial analyses.2016 Int'lGovernment Perspectives
NATO Agency Transformation: Creating a Centralized Cost Estimating CapabilityCandace MahalaA new NATO Agency is born, transforming multiple agencies into one, changing culture, and implementing a centralized robust cost estimating capability. This is the journey of how to grow and cultivate cost estimating from inception, integrating into projects as a value added asset, maturing over time and eventually becoming a fundamental part of the culture in the acquisition and execution processes.2016 Int'lGovernment Perspectives
NASA's X-Plane Database and Parametric Cost Model V2.0Steve SterkIn 2015 the idea of creating an Armstrong X-Plane Cost Model was conceived. Initially the data being used was leveraged from Dr. Joe Hamaker’s Quick Cost v5.0 and verified and supplemented by Armstrong’s summer intern Aaron McAtee (PhD). After several peer reviews, it was suggested to go back and take a hard look at the data collected. Our objective is to produce an X-plane database that ensures that all the data is as accurate, traceable and far-reaching as possible.2016 Int'lGovernment Perspectives
Cost Estimate Credibility from a Government PerspectiveHenry ApgarThis original paper describes a search for the definition of "Cost Estimate Credibility." The author documents his review of government-agency estimating handbooks, contemporary textbooks, previous ICEAA papers and journal articles, and the opinions of contemporary experts. The paper concludes with a concise definition which can be used by government and contractor estimators for their cost models and cost estimates.2016 Int'lGovernment Perspectives
The Value of R&D - a Real Options Analysis ApproachJohn Shimell, Neil DaviesThis study sought to apply Real Option Analysis (ROA) to answer the question “what is the value achieved from the MODs Chief Scientific Advisor’s (CSA) £400m annual budget”. Typically this has been reviewed retrospectively via case studies and surveys. This study developed a tool based on an amended Black & Scholes Formula, combined with a Monte Carlo analysis of the “tree” of R&D to determine a predictive value for the R&D spend of the CSA.2016 Int'lManagement
What Does A Mature Cost Engineering Organisation Look Like?Dale ShermonCost Engineering Health Check; How good are those numbers? is Dale Shermon's second book and co-authored with Dr Mark Gilmour. The book is the paper and the presentation will consider what a mature cost engineering organisation looks like? It will cover the essential requirements for a credible and justified cost forecast.2016 Int'lManagement
Using Public Data for Validation and Winning BidsAlexander KingTo win bids, cost forecasting involves more than just creating a reasonable baseline. This presentation will show how public data can be combined and triangulated to ensure our predictions are competitive as well as reasonable. The presentation will show how parametric estimates, past contract data, financial statement data and statistical data can combine to make powerful metrics for productivity and competitiveness.2016 Int'lManagement
OSA: Cost and Schedule Saver or Driver?Victoria Cuff, Brian D. FerschOpen Systems Architecture employs a modular design and utilized consensus based standards for its key interfaces with full design disclosures [Office of the Deputy Secretary of Defense, Systems Engineering, Initiatives]. A challenge with OSA is the ability to continuously design modular systems with obsolete technology and pieces, evolving technological environment and emerging threats. Examining past successes and failures, this study will analyze means to reduce costs and increase success.2016 Int'lManagement
Is it Worth It? The Economics of Reusable Space TransportationRichard L. WebbMuch has been invested in reusable space transportation systems in the belief that reusing system elements is cheaper than expending costly hardware each flight; a view not held by all industry stakeholders. This paper will define the metrics by which stakeholders measure financial "goodness," provide examples of application of the metrics to investment decisions, and offer observations on their potential influence on current and future decisions regarding space transportation system investment.2016 Int'lManagement
The 10+/-2 Factors for Estimate SuccessAndy NolanA study of completed projects across Rolls-Royce revealed the factors that lead to accurate estimates. This has enabled us to develop an Estimate Readiness Level (ERL) assessment method, to determine the likely accuracy of an estimate. Success is no accident and estimates accuracy can be predicted from the outset of a project. This paper will cover the factors for estimate success and the development of the ERL assessment.2016 Int'lManagement
The Signal and the Noise in Cost EstimatingChristian B. SmartWe seek to extract signal and eliminate noise when building models with historical data to predict future costs. One common problem is normalization, which is necessary when making comparisons but can inject noise when used in modeling. We discuss kernel smoothing and distribution fitting as ways to avoid overfitting peculiarities in historical data and the important issues of cross-validation and parsimony as ways to validate models and avoid the lure of overfitting.2016 Int'lModels & Methods
Facilities Cost Estimates Drivers in the Oil and Gas Field DevelopmentLinda Newnes, David Peacock, Ettore Settani, Jon WrightIn aerospace and defence research has focused on estimating the cost of delivering an outcome, e.g. available aircraft. Our research shows that focusing on the product alone, using product reliability/failures does not reflect the cost of delivering an outcome. In this paper we show why this is the case and how the use of input-output modelling and systems engineering can be used. We then demonstrate how the approach can be used in estimating the cost of waste water treatment.2016 Int'lModels & Methods
Successful Cost Estimating with T1-EquivalentsGrady Noll, F. Gurney Thompson IIIThis paper discusses our research and development of a cost model for additive manufacturing (AM), also known as 3D printing. Working with academia and multiple AM companies, we investigated cost drivers, collected cost and technical data, derived AM CERs, and developed an approach for including AM components within a larger parametric cost estimate. This paper discusses the research approach, data collection, results, cost model implementation, and future work.2016 Int'lModels & Methods
Costing Aircraft Availability and 50-Shades of Grey WaterAndrew Langridge, Anna FooteData gathering is often one of a cost estimator’s biggest nightmares for many reasons. Even in cases where data is available, the logistics of data collection can be daunting. This paper discusses a methodology that PRICE has developed to use open source web crawling software to regularly and repeatedly update guidance on commodity prices for Information Technology items with the push of a button. Uses of this methodology and further directions will be presented.2016 Int'lModels & Methods
Outing the OutliersAlan R. JonesInappropriate inclusion or exclusion of data that may or may not be regarded as outliers may make a project unachievable or uncompetitive. This paper looks at some of the issues involved and what tests are available to support an informed decision process rather than relying on a random guess on what is, or is not, an outlier. The paper will compare the performance of these tests against a number of sample data sets.2016 Int'lRisk Analysis
The Way From a Parametric Estimate to a CAD-Driven CalcualtionJoachim Schoeffer, Herbert SpixThe industrial requirements for quick, rough and exact costings (e.g. for budgeting) and the detailed cost break down to the last screw or processor is as old as both methodologies exist. To satisfy both fractions we need different solutions that address both needs. Ideally both scenarios are integrated and interface between each other. In this presentation the author shows a way of integrate parametric and CAD-driven bottom-up.2016 Int'lRisk Analysis
A Parametric Model for the Cost Per Flight HourMichail BozoudisThe parametric estimating technique provides timely cost estimates for "unknown" systems, utilizing cost estimating relationships deriving from historical datasets. This case study describes the development of a parametric model that estimates the cost per flight hour (CPFH). The cost derives as a function of the aircraft empty weight and the engine’s specific fuel consumption. As an example, the F-35A CPFH is estimated under the hypothetical scenario that it is operated by the Hellenic Air Force.2016 Int'lRisk Analysis
Should You Care About How Good An Estimation Process?Alain AbranThis presentation is based on the author’s 2015 book on Software Project Estimation (Wiley & IEEE Press) and illustrates how organizations (including in the car industry) having collected their own data using international standards have built their own estimation models and developed a key competitive advantage through improved software estimation capabilities. Examples will include data from a large European car manufacturer and a small software organization using Agile.2016 Int'lRisk Analysis
CER Issues And SolutionsKurt BrunnerCost estimating Relationships (CERs) are widely accepted as an effective cost estimating tool. There are some issues with CER development and application that should be addressed. This briefing will explore those probable shortcomings and how they may be alleviated with the goal of refining our cost estimating capabilities.2016 Int'lRisk Analysis
Inherent Risk in Spreadsheets (IRiS)Alan R. JonesMany spreadsheets may be certified as being error-free on the day they are tested, but some are more prone to future errors due to the manner in which they have been designed or compiled.2016 Int'lRisk Analysis
Satellite Mass GrowthDonald MacKenzie, Erik BurgessA new study of satellite mass growth on 21 programs shows that equipment type and design maturity are drivers. Unlike current AIAA guidelines, acquisition strategy is shown to be a significant driver, with competitively awarded programs experiencing much higher growth than sole-source awards, even for units with the same design maturity. Overall, mass growth allowance recommended by AIAA for every equipment type is much lower than the actual growth experienced in this dataset.2016 Int'lSpace Systems
QuickCost 6.0Bob HuntQuickCost 6.0 has just been released by Galorath Federal and NASA. QuickCost 6.0 is an all new implementation QuickCost (QuickCost 1.0 was released in 2004). The philosophy behind the model is that accurate cost and schedule estimates of automated spacecraft missions can be obtained parametrically in a quick and easy fashion estimating at the top 11 NASA WBS elements and using only a few input variables. QuickCost 6.0 is based on data from the NASA ONCE CADRe database.2016 Int'lSpace Systems
Hubble Space Telescope and Its Color PhotographsDavid PineThe Hubble Space Telescope has taken thousands of "color" photographs that have excited people the world over. This presentation examines the "Mirror Problem" and then how scientists change the photons being collected by the telescope into the phenomenal photographs that have thrilled the public. Understanding the color photographs begins with understanding what color is, and how Hubble's cameras operate to eventually produce pictures.2016 Int'lSpace Systems
Halfway to AnywhereFabian EilingsfeldA 'tariff zone map' is proposed for space transportation. Using parametric analysis, applying the paradigm of terrestrial public transport, it displays specific energy cost depending on total energy delivered to payloads. Every space transportation system leaves its own distinctive footprint when its performance data is plotted on the proposed map. It visualizes performance and cost trends with an emphasis on affordability, which helps rational discussion of project options and roadmaps.2016 Int'lSpace Systems
The SSCAG LegacyHenry ApgarActive partnership in this ICEAA Symposium marks the final milestone of a spectacular forty-year journey of the Space Systems Cost Analysis Group. SSCAG provided a unique international forum for space-systems cost analysts. Over those years, more than 300 skilled cost estimators, business analysts, program managers, space engineers, and logisticians filled the international venues, at more than 100 scheduled meetings, to deliberate major issues associated with government and commercial development.2016 Int'lSpace Systems
Cloud Solutions - Infrastructure, Platform or SoftwareArlene F. Minkiewicz, Ashley HoenigkeMigrating capability to the cloud comes with several planning and management challenges. How does an organization determine the right capability to migrate and the right platform for the migration? What challenges do the various cloud solutions (Infrastructure vs. Platform vs. Software as a Service) present? This paper presents a case study outlining the migration activities and other implications for migrating the same application on each of the three cloud solution models.2016 Int'lSoftware
Measurement of Software Size: Contributions of COSMIC to Estimation ImprovementsAlain Abran, Charles Symons, Frank VogelezangThis talk presents 1) an outline of the design of the 2nd generation COSMIC method for measuring a functional size of software; 2) industry evidence of its practical value in software project performance measurement and its accuracy in estimating; 3) an account of the method’s full automation, with very high accuracy, for real-time embedded software specified in Matlab-Simulink and developed through a world network of software contractors.2016 Int'lSoftware
The Monitoring and Early Warning Indicators for a Software ProjectChristine GreenThis presentation will focus on controlling by monitoring and early warning indicators on a software project’s scope, estimate and performance - all factors that highly influence the cost and budget.2016 Int'lSoftware
Software Estimating Model Using IFPUG Standard Sizing MethodsChristine GreenA major input for any cost model for software projects are the scope and effort for a software projects regardless of technology or type of project process. IFPUG Sizing Standards methods can be utilized for software estimating model and thereby establish a valuable input for the cost models produced that will include both scope control and effort estimates.2016 Int'lSoftware
Introduction to Software Obsolescence Cost Analysis FrameworkSanathanan RajagopalAlmost every project in Defence sectors has got software with various degrees of complexity and dependencies. Whilst various research and studies have been conducted on system obsolescence and tools developed to cost systems and components obsolescence, no major research has been undertaken to develop a framework to estimate cost of software obsolescence. This paper will discuss the Software Obsolescence Cost Analysis Framework developed as part this research.2016 Int'lSoftware
Applying Earned Value to Agile Development ProgramsBob Hunt, Michael ThompsonA general movement to Agile software development process is one method used to address and potentially resolve these problems. The movement to Agile Software Development introduces several unique problems for EVM assessments. This paper will discuss some key element associated with: Agile/Hybrid Agile software development; Establishing a process for developing the technical, cost and schedule baseline; Applying Earned Value Management to agile programs.2016 Int'lSoftware
TitleAuthor(s)SummaryYearTrack
The [Whole] Truth about ANSI-compliant EVMSMichael NosbischOver the past several years, there have been multiple presentations by government agency representatives and consultants alike that have characterized an EVMS that complies with the ANSI/EIA 748(B) Standard as simply "good project management," and as such should not be that difficult for most successful government contractors to implement. It is true that the basic concepts and principles of EVM can absolutely be likened to sound project management. However, the costly, time-intensive process of implementing, validating, and then maintaining an "ANSI-compliant" EVM system is far and above what any contractor would do if it were not a contractual requirement that is also fully reimbursable. ...2013Earned Value Management
Estimating Cost To-Go Without Stable EVM DataPeter Frederic, Ronald K. LarsonInitial independent estimates for development programs are typically created using parametric methods. As a program progresses, it becomes important to separate sunk cost from cost to-go. In the very early stages of a program, it is acceptable to estimate cost to-go by simply subtracting the reported sunk cost from the parametrically-estimated total cost. However, as the program continues to progress and more significant progress accumulates, we can no longer assume that dollars spent equate directly to technical progress. It is then necessary to understand progress to-date in order to estimate cost to-go...2013Earned Value Management
Time is Money: The Importance and Desired Attributes of Schedule Basis of EstimatesJustin Hornback"Time is Money" is a maxim made popular by Benjamin Franklin. It reflects the long understood importance of schedule requirements and their impact on cost. While the importance of cost basis of estimates (BoE) have gained in popularity among industry communities, schedule BoEs are, at least equally, if not more important as cost, but have yet to reach the same level of understanding of importance within the industry...2013Earned Value Management
Cost Estimation and Earned Value IntegrationSissy Gregg, Michelle EhlingerTraditionally, cost estimators and earned value analysts work separately from one another, focusing on their own individual discipline. We have found synergies by merging these disciplines together resulting in stronger analysis through improved methods, more robust data sets, and new tools. The benefits of cost estimating (CE) and earned value (EV) management integration are realized throughout the acquisition lifecycle and enhance senior leadership decision making and improve acquisition outcomes. This paper presents the proven benefits of CE/EV integration, including more accurate cost estimates, cost estimating relationships, improved program management support, and cross program assessments...2013Earned Value Management
Understanding Requirements for Subcontract EV Flow Down and ManagementMark InfantiDo you use Earned Value Management (EVM)? Do you have a government customer that is requiring you to use EVM and report the data monthly? In either of these situations, if you have a subcontractor (SC), you need to know how they will report to you. You need to have a plan on how you will incorporate their data into your data for a complete program understanding. If your SC reports to you at cost level how do you add the fee to their report? How do you add your companies general and administrative (G&A) overhead to their costs? Do you share your Integrated Master Schedule (IMS) with them or just the key milestones?...2013Earned Value Management
Measuring and Managing Organizational Performance in the GovernmentKellie ScarbroughIn this presentation, the author details the challenges that government agencies face when attempting to measure organizational performance and presents a general methodology for successfully implementing a robust performance measurement program. The majority of government agencies do not operate like private industry and face unique challenges when defining meaningful metrics and collecting performance data...2013Earned Value Management
"The Use of Function Points in Earned Value Management for Software Development
Ben Netherland, Mike Thompson, Dan FrenchIn this presentation, the authors detail their efforts in the development of an EVM methodology for a government software development project utilizing the International Function Point User Group (IPFUG) function point software sizing metric. Traditionally it has been difficult to apply Earned Value Management (EVM) criteria to software development projects, as no tangible value is earned until the software is delivered to production...2013Earned Value Management
New Air Force Integrated Baseline Review (IBR) Process - A Quick Reaction Capability (QRC) PerspectiveJames Ross, Martin Levitan, Christine Bolton, Barbara MeyersSince the early 1990s, the U.S. Air Force has been using Integrated Baseline Reviews (IBR) as a key technique for early assessment of the efficiency and effectiveness of baseline plans, resource allocations, scheduling and costs. Much of the earlier IBR doctrine does an excellent job describing what is required to complete these assessments, but little detail had been written about the specific processes, procedures, steps and techniques required to undertake these assessments. The January 2012 "Air Force Integrated Baseline Review (IBR) Process Guide", developed by SAF/AQXC personnel, is one of the best documents to date to address this concern. A drawback of this document is that the described IBR Process is based primarily on experiences with, and lessons learned from, the Acquisition Category (ACAT) I KC-46 Tanker Program...2013Earned Value Management
Capacity Cost Model: Balancing the Demand for Software Changes Against the Supply of ResourcesChris KaldesA Program Executive Office (PEO) within the Department of Defense (DoD) is responsible for multiple information systems that support the Services (Army, Navy, and Air Force). This PEO is not only responsible for Operations and Maintenance (O&M) of the information systems, but also for implementing system change requests that are made by the Services' Senior Service Representatives (SSRs). The PEO was relatively satisfied with the status quo, but the executives within the PEO recognized that there were three problems with doing business today. (1) Currently, there was no way to scientifically indicate/prove to the SSRs what software changes they could promise...2013Information Technology
Software Maintenance: Recommendations for Estimating and Data CollectionCorinne Wallshein, Bruce Parker, Vanessa Welker, Thomas Harless, Peter BraxtonEntering a period of fiscal austerity, it becomes more important than ever to estimate and consider operating and support (O&S) costs, which represent the lion's share of life cycle cost (LCC) for most platforms, during acquisition. Given the ubiquity of software in today's complex programs, a key component of O&S is software maintenance. This paper will discuss our work to understand the process of software maintenance, data collected and normalized to date, and resultant benchmarks for use in developing and cross-checking software maintenance estimates. It represents an update to "Software Maintenance Data Collection and Estimating Challenges" (Welker, et al., SCEA/ISPA, 2012)...2013Information Technology
Domain-Driven Software Cost, Schedule, and Phase Distribution Models: Using Software Resource Data ReportsWilson Rosa, Barry Boehm, Brad Clark, Joe Dean, Ray MadachyInstead of developing cost and schedule estimation models with many parameters, this paper describes an analysis approach based on grouping similar software applications together called Productivity Types. Productivity types are groups of application domains that are environment independent, technology driven, and are characterized by 13 product attributes. Also consideration is given to the operating environment that the software operates within. Over 196 actual software projects from DoD's Software Resource Data Reports (SRDRs) were fully inspected and analyzed to produce a comprehensive set of Cost Estimation Relationships, Schedule Estimation Relationships, Software Productivity Benchmarks, and Best Practice Data Normalization Guide. Analysis results will be discussed in this presentation...2013Information Technology
Estimating Software Development Costs for Agile ProjectsBlaze Smallwood, Omar MahmoudAddressing the need to more rapidly develop and field capabilities for the warfighter, more and more software-centric DoD programs are transitioning towards an industry trend called "Agile" software development. While "Agile" software development is geared towards producing usable software products more rapidly than traditional waterfall or incremental methods, it also requires more flexibility with managing requirements. The main challenge this has created for program managers is figuring out how to effectively manage scope, cost, schedule, and performance in this flexible, fast-paced development environment in which requirements are more fluid. In turn, cost estimators have been challenged to develop new data collection approaches and estimating methodologies to more effectively estimate software costs for these "Agile" programs...2013Information Technology
Software Maintenance Cost Estimating Relationship Development for Space SystemsRyan TimmMethods development for Space Vehicle (SV) cost estimating has mainly focused on acquisition cost, as acquisition makes up the majority of a SV's lifecycle costs and has high interest from oversight agencies and congress. Tightening budgets and increased space vehicle life expectancy has bolstered the need to further refine methods for estimating SV maintenance costs...2013Information Technology
The COSMIC Functional Size Measurement Method: An IntroductionH.S. van HeeringenThe COSMIC functional size measurement method is a relatively new and not very well known method to measure the functional size of pieces of software. The main advantages of this method over the existing methods (like IFPUG and NESMA function point analysis) is that COSMIC is designed to also measure other types of software than the traditional administrative software. COSMIC can therefore be used to measure real-time, embedded, infrastructure software, as well as administrative software. Another advantage is the continuous scale that is used in the COSMIC method, which makes it possible to accurately measure the difference in size between different functional processes...2013Information Technology
Estimating Real-Time Software Projects with the COSMIC Functional Size Measurement Method and the ISBSG RepositoryH.S. van HeeringenNowadays, software project estimation of administrative software systems has evolved into quite a mature stage. Standardized functional size measurement methods, like IFPUG and NESMA, are used to measure the functionality that the software application is going to offer to the user and this functional size is considered to be the main cost driver for administrative software projects. These methods are suitable for project estimation, because they measure the functional user requirements (which should be described before the project starts), instead of the actual delivered product (for instance source lines of code, which can only be measured after the project finished)...2013Information Technology
Developing A Business Case for CloudCynthia O'Brien, John BellCommercial organizations and their customers have embraced cloud-based data management solutions with astonishing speed. Government agencies are not far behind, spurred on by the Office of Management and Budget’s “Cloud First” Federal IT policy urging agencies to look to cloud-based solutions whenever possible. Government decision makers are moving to cloud solutions not only because of pressure from OMB and end users; they are also intrigued by the promise of dramatic cost savings. But can a decision to pursue a cloud transition be based on the promise of cost savings alone? There have been numerous examples where transitions to cloud have not yielded the 40% or more expected cost savings...2013Information Technology
ODASA-CE Software Growth ResearchKevin Cincotta, Lauren Nolte, Eric Lofgren, Remmie ArnoldFor several years, the Office of the Deputy Assistant Secretary of the Army for Cost and Economics (ODASA-CE) has used a single growth factor to account for size growth in weapon system software development estimates. This factor is invariant to program characteristics and may, for example, lead to excessive growth estimates for large programs, whereas experience suggests that these grow less in percentage terms than their smaller counterparts. Over the past year, ODASA-CE worked with Technomics, Inc. to research improved methodologies for incorporating growth in software estimates...2013Information Technology
Cloud Computing and Big Data - What's the Big Deal?Arlene MinkiewiczAt one level, cloud computing is just Internet enabled time sharing. Instead of Information Technology (IT) organizations investing in all of the hardware, software and infrastructure necessary to meet their business needs, cloud computing makes access to them available through the Internet. On the one hand, cloud computing allows an organization to adopt a different economic model for meeting IT needs by reducing capital investments and increasing operational investments. On the other hand, cloud computing enables the capture, storage, sharing, analysis and visualization of huge amounts of data in multiple formats from multiple media type: Big Data analysis...2013Information Technology
Can DoD Inflation Indices and Discounting Processes Be Improved?Kathryn Connor, James DrydenCurrently the DoD is facing an uncertain budget environment. This will have an impact on what the DoD can spend for acquisition programs and sustainment of major weapons systems. Current practices for inflation and discounting skew program affordability, especially during operations and sustainment. In this presentation, we look at how well current inflation indices and discount rates serve programs today and whether there are strategies to improve the accuracy of these estimates. After examining the experience of several major weapons systems we have identified potential policy changes and strategies for cost estimators to employ on inflation and discounting. We believe that these can improve a program's understanding of long run affordability and potential risks associated with inflation and discounting.
2013Life Cycle Cost
Lessons Learned from the Joint STARS Analysis of Alternatives for Cost and Risk AnalysesDaniel Mask, David StemThe Joint Capabilities Integration and Development System (JCIDS) plays a key role in identifying the capabilities required by the warfighters to support the National Security, Defense, and Military Strategies. The Analysis of Alternatives (AoA) is a documented evaluation of the performance, operational effectiveness, operational suitability, and estimated costs of alternative systems to meet a capability need that has been identified through the JCIDS process. As weapon system complexities and their interdependencies between other systems increase in the collaborative warfighter space, this evaluation of mission systems and technologies has become more and more challenging...2013Life Cycle Cost
Analysis of Large O&S Proposal: Lessons Learned!James LinickRecently, a $20M contract was awarded for weapon system O&S. The contractor's proposal was subjected to technical evaluation and extensive negotiations ensued. Over 100 BOEs were evaluated for multiple CLINs. Proposal cost components were labor, materials and ODCs. This presentation will focus on how a final agreement was reached, focusing on: 1) Proposal construction, what are CLINs, BOE's and how sub-contractor work was credited, 2)
BOE technical evaluations, the delineations and weighting of on-site and offsite work, 3) Forward price rate agreements, what they are, and the difference between them and final negotiated labor rates, 4) Cost to price calculations, how they are performed and what are Fringe, G&A, Overhead, Fee and Cost of Money and 5) How final contract price and weapon system CONOPS cost estimate compared...
2013Life Cycle Cost
The Dynamic Economic Model: A Flexible Approach to Investment Analysis with AnalyticaKevin King, Jason MehrtensGiven the current economic crises facing the United States, government spending is being placed under more intense scrutiny. As a result, major investments are being more closely analyzed and pressure to be cost effective is of the utmost importance. In the world of cost estimation, the scrutiny is translating to increased demand for cost metrics prior to the establishment of clear scope requirements. For MCR, uncovering new tools and strategies to adapt to this type of environment has become a top priority. One way MCR improved versatility while maintaining our valued reliability has been to create economic cost models using Analytica®, a modeling software designed to enhance the capabilities of the modern spreadsheet tool...2013Life Cycle Cost
The Forgotten Costs and Uncertainty Analysis within the DoD Acquisition EnvironmentRoberto Perez, Elizabeth EvansDespite the best efforts of the Department of Defense (DoD) to capture all costs associated with the life cycle of acquisition programs, it continues to be an elusive. A primary reason for this shortcoming in cost estimating is the lack of time spent on estimating operations & sustainment (O&S) costs compared to the time and effort estimating research, development, and procurement costs. The implications for this imbalance are significant given the fact that 60 - 70 percent of an acquisition program's life cycle cost is comprised of operations and support...2013Life Cycle Cost
Life Cycle Cost Estimate (LCCE) AssessmentsCasey Trail, David Brown, Colleen CraigThis presentation summarizes a scorecard tool and criteria that are used to conduct assessments. The criteria, which are based on the GAO Cost Estimating and Assessment Guide, include four characteristics, which are further broken in to 13 criteria and 75 sub-criteria. Lessons learned, and key findings are presented as examples of LCCE characteristics associated with both positive and negative evaluation outcomes. The presentation is of interest to government and contractor cost estimators, as well as anyone from the cost community wishing to leverage best practices and tools related to life cycle cost assessment. Participants with cost assessment experience are encouraged to share their experiences and lessons learned...
2013Life Cycle Cost
Construction vs. Systems Acquisition Cost Estimating:  A Comparative AnalysisTom Sanders, Steve Essig, Tim AndersonThe continued push to improve DOD's acquisition processes includes several thrusts — regulatory streamlining, improving use of incentives, and workforce optimization. These are not new. To deal with these concerns, the acquisition community has a unique opportunity. DOD Cost estimators have recently received a strong "vote of confidence" in the acquisition community; new positions have been created and existing positions have been protected. That is not necessarily so in the construction industry, another large employer of cost estimators. The construction industry has undergone a transformation over the past six years, beginning with the housing collapse in 2006 and enduring through the years of the Great Recession...2013Management
The Joint Integrated Analysis Tool (JIAT)-  Making Data Sharing EasierNiatika Griffin, Melissa Cyrulik, John McGahanThe silver lining in the current DoD budget turmoil is that it provides us with an opportunity to reexamine how we operate and look for ways to be more efficient. In times of belt tightening, improving access and distribution of established resources is imperative. We are challenged to ultimately improve how we share cost resources. For cost professionals, these resources include databases of cost and technical information, libraries of resources, and approved models that can be used rather than re-invented. The Office of the Deputy Assistant Secretary of the Army - Cost & Economics (ODASA-CE) has invested in developing a tool that facilitates secure access to data needed by the cost analyst...2013Management
Strategic Value of the Business Case Analysis (BCA)Eric BullerThe impact of sequestration has become the main topic of discussion in almost all circles in the Department of Defense since the Budget Control Act passed into law in 2011. In response to reduced budget sequestration, the DoD has been interested in exploring potential cost reductions in weapon system operations. In nearly all lifecycle costs, sustainment generally makes up 60% to 80% of the total program costs. In a defense world seeking increased efficiencies and budget savings it is imperative to examine effective sustainment strategies...2013Management
Back to the Big Easy: Revisiting Hilbert's Problems for Cost EstimatingPeter Braxton, Richard ColemanAt the International Congress of Mathematicians at the Sorbonne in Paris in 1900, German mathematician David Hilbert boldly put forth a list of 23 theretofore unsolved problems in mathematics, which subsequently became quite influential in 20th-century research efforts. At the Joint SCEA/ISPA Conference in New Orleans in June, 2007, the authors audaciously emulated Hilbert with a list of 21 problems for cost estimating and risk analysis...2013Management
Net Present Value (NPV): The Basics and the PitfallsKevin Schutt, Nathan Honsowetz One of the essential metrics of an economic analysis is the Net Present Value (NPV) measure. While it is a fairly simple calculation, made even more routine and efficient by modern technology, be wary of pitfalls in its application. This presentation will cover the basic components and methodology of NPV and discuss the pitfalls and easily-made mistakes in its application. These include incorrect discount rate formulas, inconsistent treatment of inflation, incorrect understanding of the discount rate, incorrect application of Excel's "=NPV()" formula (hint: it's not really "Net"!), and including irrelevant cash flows. Come learn about Net Present Value and avoid these potentially career-limiting mistakes!2013Management
Fifteen Undeniable Truths About Project Cost Estimates, or Why You Need an Independent Cost EstimateTim AndersonAll project cost estimates are wrong, some more so than others. It is exceedingly difficult to accurately estimate the cost of a large project (e.g., major defense acquisition project). Reasons for this are manifold, but the primary causes seem to be general uncertainty, insufficient technical detail, unanticipated design/technical changes, funding and schedule perturbations over the acquisition life cycle, other programmatic "difficulties," and otherwise attempting to predict the future with imperfect information. The end result is that project cost estimates tend to be too low, with early estimates being the most likely to be understated...2013Management
Cost Engineering Heath Check — How Good are your Numbers?Dale Shermon, Mark GilmourHigh quality cost estimating gives a business leader confidence to make rational financial decisions. Whether you are a business leader or a cost estimating manager, you have a vested interest in understanding whether you can depend on your organisations ability to generate accurate cost forecasts and estimates. But how can business leaders be confident that the cost information that they are being provided is of high quality? How can a cost estimating manager be sure that their team is providing high quality cost information?...2013Management
Sailing Blind: Data-Driven Estimating at the Concept Phase, a Case Study of Canada’s Joint Support Ship ProjectTolga Yalkin, Erin BarkelCanada's lack of recent experience in naval shipbuilding and the unique nature of the National Shipbuilding Procurement Strategy (NSPS) present unique estimating challenges. A methodology was needed that was at once data-driven and defendable while at the same time able to address shipbuilding realities at a very high level conceptual stage. This paper outlines a case study where PRICE Systems and the Canadian Parliamentary Budget Officer (CPBO) developed a methodology and model to address these challenges for the Joint Support Ship (JSS) project...2013Management
Supplier Cost/Price AnalysesDavid Eck, Todd BishopUnder United States Federal Government contracts, an important but overlooked aspect is the audit and evaluation of a subcontractor's cost proposal. Requirements in the Federal Acquisition Regulations (FAR) provide for certain visibility into subcontractor's cost proposal depending on the value of the proposal. Additionally it is critical to note the FAR puts the onus of subcontractor cost (or price) analysis on the prime contractor. The expectation is that the same level of detail that the prime includes in its proposal should be included as support to a subcontractor's cost proposal. Effectively the prime must perform subcontractor cost analysis in accordance with the FAR...2013Management
National Reconnaissance Office (NRO) Experiences with the Implementation of Will Cost/Should Cost ManagementSissy Gregg, Greg Lochbaum, Erik BurgessThe objective of this presentation is to address how to assess and incorporate contract performance in an Estimate to Complete, for the purpose of determining a reasonable amount of margin and how that margin should be managed and controlled. The discussion will go beyond cost and schedule variance to forecast total program cost including future requirements, other government costs, etc...2013Management
Avoiding Pitfalls When Applying Learning to your EstimateAndrew BusickMost cost estimators understand the basics of learning curves, but we sometimes fail to recognize the implications of the way that we apply learning.
This presentation points out some major pitfalls and mistakes that can be made when dealing with learning. We address both deriving learning curves from historical data and applying learning to a cost estimate. Additionally, we recommend best practices to use when dealing with these issues...
2013Methods and Models I
Cost Effective Analysis: The Role of Discounting in Government InvestingBrandon ShepelakIn order for the government to best fulfill its obligation to efficiently spend tax dollars, it uses various analysis methods to determine which investment to choose. Cost effectiveness analysis is often the preferred analysis method when an investment is not being judged as to the value added but to determine the least expensive spending alternative. According to the Office of Management and Budget (OMB) in circular A-94, an investment is cost effective when, comparing competing alternatives, its cost are the lowest as expressed in present value terms for a given amount of benefits (OMB, 2011)...2013Methods and Models I
Learning Curve Analysis of Small Data Sets - Spacecraft Bus Cost Improvement AnalysisBrian Welsh, James YorkRecent initiatives to implement cost savings across DoD have produced a number of potential opportunities. Within the Space Portfolio one such opportunity is to acquire satellites more efficiently through block buys and limiting production breaks. To quantify these opportunities, we can analyze historical cost experience. This paper describes three approaches used to pool cost improvement data from multiple programs, allowing for regression of all data simultaneously. The approaches are (1) scaling based on weight, (2) normalizing to a common theoretical first unit, and (3) the use of program binary ("dummy") variables...2013Methods and Models I
Building an Analysis Schedule – Lessons Learned from the SGSS ProgramMatt Blocker, David JacinthoConstructing an integrated cost and schedule uncertainty model involves bringing together important program elements: cost, risk, and schedule. The model's core is the schedule; the linking of costs and risks to the schedule is what allows the implementation of time-dependent costs and, ultimately, an analysis that jointly considers cost and schedule. It follows, then, that one of the first, and often largest, difficulties in constructing an integrated cost and schedule model is the development of the schedule backbone...2013Methods and Models I
A Novel Non-Recurring Production CER MethodologyLisa Hackbath, Raymond CovertProduction costs are generally categorized as either non-recurring or recurring. Typically non-recurring costs include tooling and pre-production activities, among others. Cost estimating relationships (CERs) are generally developed first for recurring hardware costs, and then non-recurring CERs are developed as a function of recurring hardware costs...2013Methods and Models I
Data Collection and Analysis Supporting Defendable Cost EstimatesArlene MinkiewiczCost modeling and estimation has a long and interesting history in the Aerospace and Defense industry starting around the time of World War II. All sorts of mathematical and experiential models have been proposed and used over the years to help with bidding, planning, proposing and executing contracts. While general purpose models are useful, more and more industry and government cost professionals are asking for models built with data very specific to their industry and their organization. Unfortunately many organizations do not have the infrastructure, processes or tools for collecting project data efficiently...2013Methods and Models I
Getting (and sharing!) the FACTS: Factors, Analogies, CERs and Tools/StudiesDaniel Harper, Ruth DorrOne of MITRE's corporate values is "People in partnership." MITRE values […] partnership with the government, collaboration within and without..."
We at MITRE have been charged via our corporate goals to "Apply technical and engineering excellence," by bringing to the customer the best thinking possible by "[…] tapping into a deep technical base, both within MITRE and globally, across the breadth of industry and academia." ...
2013Methods and Models I
Aircraft Operation - Cost Models; Lessons Learned!James LinickIn FY10 the Air Force Cost Analysis Agency (AFCAA) asked one of its cost support contractors to study the processes in place for producing specific aircraft cost products and implement improvements, specifically concentrating on format consistency, maintainability and automation. The periodically produced products to be studied were: 1) Aircraft Operation and Sustainment Cost Estimates 2) Cost per Flying Hour Risk Models 3) Jet Engine Maintenance Procedures versus Time on the Wing...
2013Methods and Models I
Implementing a Data Collection Tool to Estimate and Analyze SpendingThomas Brooks, Daniel MaskAs budgets and resources shrink, the effort to better understand spending has become a priority to many decision makers within the Marine Corps. Since Operation Iraqi Freedom (OIF) began in 2003, military organizations have been able to request and receive immediate Overseas Contingency Operations (OCO) funding without following traditional procurement processes. This readily available funding has led to the emphasis of developing and maintaining operational capability in lieu of program estimates and resource management. The lack of quality budget estimates and use of financial tracking mechanisms has made it difficult for Marines to strategically allocate budget cuts based on efficiencies or priorities; more often than not, budget reductions are distributed uniformly across the organization...2013Methods and Models I
Cost Estimating Tips: Learn Tricks in Excel and Best PracticesEric HongThis presentation for the 2013 International Cost Estimating and Analysis Association (ICEAA) Conference is to present Microsoft Excel tricks and best practices for cost estimating. Many of the presentations at the conference are insightful and show the amazing cost estimating products that are being developed, but the knowledge gained from those presentations usually cannot be applied on a daily basis. The goal for this presentation is to provide the attendees efficient and more effective ways of modeling...2013Methods and Models I
The Dashboard Tool: Taking Cost Analysis to the Next Step By Combining Costs with Capabilities to Evaluate COAsJohn KoAlthough life cycle cost estimates can provide a comprehensive view of the costs for a program, cost estimates do not provide the entire picture needed to make sound business decisions because they only focus on the costs for a program. To determine how programs are actually performing, the system's capabilities must be linked with the program's costs to properly quantify how programs meet their requirements with respect to cost. Booz Allen has developed an Excel Based tool called Dashboard that estimates both the program's costs and the capabilities achieved for a program...2013Methods and Models I
Realizing the True Cost of Energy - Keeping the DoD GreenJohn KoIn recent years, the demand for energy as well as volatility in costs of energy for the Department of Defense has prompted individual programs to monitor these costs closely in order to identify new ways to control these costs. In response to spikes in energy costs in recent years, major programs are now evaluating different alternatives to improve fuel efficiency and ultimately reduce the military's dependence on fuel...2013Methods and Models I
Galaxy Charts: Depict and Color Your WBS in a Meaningful WayRobert Nehring, Katharine Mann, Robert JonesFor centuries, we have searched for new ways to display our thoughts and ideas in ways that will allow the viewer to easily digest and understand our point of view. Displaying quantitative data in interesting yet meaningful ways is no different. In fact, discerning how to display your entire Work Breakdown Structure (WBS) succinctly and clearly has proven to be very difficult. One answer to this challenge is called the Galaxy Chart, which shows both relationships and magnitudes on a single chart...2013Methods and Models II
Computing Fully Burdened Costs of Energy - Fuel (FBCE-F)Walt Cooper, Richard LeeThe Defense Science Board, in studies conducted in the early years of the past decade, concluded that the DoD was not according sufficient emphasis to the consumption of energy in the battle space. Based on those studies, DoD leadership has elevated the importance of seeking opportunities to deliver more capable forces that consume less energy. The Joint Capabilities Integration and Development System now requires an energy Key Performance Parameter. Pilot studies have been conducted to establish a framework for the computation of the fully burdened costs of energy (FBCE) that, by statute, must now be used to inform cost, schedule and performance trade decisions in Analyses of Alternatives (AoAs). Based on those pilot efforts, the Assistant Secretary of Defense (Operational Energy Plans and Programs) has recently published a framework for computing FBCE. A detailed and thorough understanding of this framework is essential for cost analysts supporting AoAs...2013Methods and Models II
Estimating Alternatives for Joint Future Theater Lift (JFTL)Robert GeorgiUS military operations in Afghanistan and Iraq are making military transport aircraft work at rates not foreseen just a decade ago. At the same time the existing inventory of transport aircraft are limited in their ability to transport heavy cargo to austere unimproved landing zones where military forces routinely operate. The US Air Force (USAF) with considerable support from the US Army conducted the Joint Future Theater Lift (JFTL) Technology Study (JTS) to consider how best to equip its theater lift fleets beyond 2020 to address this need. The JTS was established by merging the US Army's Joint Heavy Lift (JHL) and the USAF's Advanced Joint Air Combat Systems (AJACS). The JTS analyzes the cost, risks, and operational effectiveness of theater lift technology alternatives that address capability shortfalls identified in the JFTL Initial Capabilities Document (ICD); an essential capability shortfall being the delivery of a combat-configured medium weight armored vehicle (up to 36 tons) into austere, short, unimproved landing areas without ground handling equipment. One of the alternatives included a hybrid airship design...2013Methods and Models II
Ground Radar Expenditure Phasing AnalysisRick GarciaRealistic budget submissions are critical to a program's success. Budget realism relies on a full understanding of the system's requirements so that a comprehensive cost and schedule estimate can be developed that can inform a realistic budget position. In addition to developing a realistic estimate, the budget position must also reflect a realistic phasing of budget dollars across an appropriate schedule; so that sufficient budget dollars are available at the correct time in a program's acquisition lifecycle to assist the program achieve success. Most existing expenditure phasing models are Rayleigh or Weibull-based distributions and were developed on a variety of DoD space and ground programs and are not commodity specific to terrestrial radar systems...2013Methods and Models II
Improving Cost Estimates in a Medical Acquisition EnvironmentMark Russo, Kimberly WallaceEstimating costs for drug development has proven difficult over the years. Difficulties stem from the overlapping of the Department of Defense (DoD) acquisition process and the Food and Drug Administration's (FDA's) review process, small data pool, industry close hold of cost data, the inherent difficulties in the drug development process, and the lack of using best practices in cost estimating. Over the past couple of years the medical cost community has been working towards improving their cost estimates. These efforts included the establishment of a standard drug development work breakdown structure to ensure that data could be collected and compared in a standard way. A medical cost model was then created in order to use the available data to develop program estimates...2013Methods and Models II
Incorporation of Program Priorities in Funding-Constrained EnvironmentsLeon Halstead, Michael Land, Derrick Kuhn, Bryan ShraderThe recent era of declining Federal (and organizational) budgets has given new impetus to the ability to prioritize funding in an efficient and effective manner. While there are multiple methodologies to pursue these budget reductions, there is a tendency for both allocations and reductions of funding to be top-down, i.e. Organization-driven with a less defensible framework to support Program priorities. This approach heightens the risk that Program goals and objectives will be negatively impacted due to underfunding in this manner due to lessened consideration of Program-specific environmental factors...2013Methods and Models II
Quantifying Uncertainty in Early Lifecycle Cost Estimation (QUELCE)Robert Stoddard, Robert Ferguson, Dennis Goldenson, Jim McCurley, Dave ZubrowEarly lifecycle cost estimation continues to grow in importance within both industry and the U.S. Government. In the commercial arena, shrinking business and product development cycles demand more rapid and early cost estimates. Within the U.S. Defense Department, cost estimates are now mandated prior to Milestone A approval within the DoD Acquisition Lifecycle. One of the primary challenges with early lifecycle cost estimation remains the incongruence of input data required by existing cost estimation models and the data available early in the lifecycle. Recent research into capability-based cost estimation offers improvements but ignores much of what may be viewed as program execution change drivers. These program execution change drivers embody the root causes of changes in program performance leading to changes in the cost of the program. As a result, our team developed the QUELCE method as an innovative approach to elicit domain expert knowledge about the possible future scenarios of change driver behavior for a given program...2013Methods and Models II
Sensitivity Analysis in Cost-Benefits AnalysisSteven IkelerThe Army has been using Cost-Benefits Analysis (C-BA) for all new requirements with cost impact for two years. As with the Analysis of Alternatives (AoA), a sensitivity analysis of the C-BA results is required. The C-BAs frequently use weighted decision matrices to combine the benefits into a single Cost-Benefit Index. This greatly simplifies the sensitivity analysis and facilitates allocation of cost to the individual benefits. The author reviewed the sensitivity analysis in 80 C-BAs. This session will address some of the techniques for presenting sensitivity analysis, the basic math behind the sensitivity analysis and allocating cost to benefits and generalizing the results to more complicated problems.2013Methods and Models II
Probabilistic Technology Investment Ranking SystemPeter FredericThis paper describes the Probabilistic Technology Investment Ranking System (PTIRS), a multi-tool cost estimating system. PTIRS is a new cost estimating tool that has been developed for the NASA Environmentally Responsible Aviation (ERA) Project at Langley Research Center. The ERA Project is charged with developing new technologies for commercial transport aircraft that will reduce fuel consumption, emissions, and noise. The PTIRS tool assesses the integrated, end-to-end benefits of a new technology and allows the benefits to be weighed against the cost of maturing, certifying, and ultimately implementing the technology in a generation-after-next (N+2) subsonic transport system. The tool is both deterministic and probabilistic: it allows inputs with ranges of uncertainty described as statistical distributions and it uses Monte Carlo simulation to produce results with statistically described ranges of uncertainty. The tool produces a two part answer: 1) the life cycle cost impact and noise, emission, and performance benefit of implementing and operating the new technology, and 2) the cost and time required to mature and certify the technology to a readiness level appropriate for full-scale development...2013Methods and Models II
Rapid Generation and Optimisation of Ship Compartment Configuration based on Life Cycle Cost and Operational EffectivenessAidan Depetro, Rhyan HoeyIt has been well established that the majority of Life Cycle Cost (LCC) is incurred during the in-service period. Among other factors, this is strongly linked to the design of the ship and the decisions made during the early design phase. In particular, compartment configuration can have a significant effect on LCC. Poorly considered compartment configuration and hull selection can result in hydrodynamic efficiencies which significantly increase energy consumption and hence fuel costs. Associated space limitations, inadequate or non-existent removal routes and other accessibility problems may result in expensive equipment overhaul and replacement procedures, invasive removal methods, longer maintenance availabilities and increased maintenance costs. Current design methods and decision analysis techniques focus mainly on the trade-off between operational effectiveness and acquisition cost rather than LCC...2013Methods and Models II
Data-Driven Estimating - Quantifying Electronics Complexity Using Public DataF. Gurney ThompsonThe parametric cost estimation industry is being challenged to increase the defendability of their estimates, mainly through comparison to analogous data. However, companies are more protective than ever of their high quality cost data. While this data is often obtainable, it usually comes with strings attached (non-disclosure agreements, etc.) that preclude its usage in defending your estimate. At the same time, government and military databases with cost and technical data are being exposed to the public, and while the noise introduced by granularity issues and the mapping process of this data presents its own set of challenges, this trend also contains the key to a solution...2013Methods and Models II
Affordability Analysis: The Role of Process, Cost and ROI Modeling In Improved Program PerformanceDan GalorathAffordability analysis as part of decision making may be the biggest edge of the decade for both commercial organizations and DoD / government organizations. Affordability has been addressed in the past but never to the level it is today. In an IT context companies struggle to increase profits and often view IT as a necessary evil: one that consumes resources rather than contributes to the bottom line. However, IT can be a significant contributor when IT decisions are made after modeling affordability in multiple dimensions.2013Methods and Models II
Applying Cost Analysis to the DoDAFMichael ButterworthCost analysts in the aerospace community today are being challenged to accurately estimate new and on-going program efforts that will be integrated into the Department of Defense Architecture Framework (DoDAF). The DoDAF provides a foundational framework for developing and representing architecture descriptions that ensure a common denominator for understanding, comparing, and integrating architectures across organizational, Joint, and multinational boundaries. It establishes data element definitions, rules, and relationships and a baseline set of products for consistent development of systems, integrated, or federated architectures...2013Methods and Models II
The Role of Value Engineering In Affordability AnalysisQuentin Redman, Robert Koury, Joseph Bobinis, Paul Tuttle, Kevin Woodward, Hein B.A. de JongThe purpose of this white paper is to describe the role Value Engineering plays within the affordability process. The paper is not a step by step "How To Conduct or Execute" Value Engineering (VE) but is a discussion of the context, input, setup, execution hints, and output of Value Engineering in support of conducting affordability analysis and management. As such it is important to understand the concept of affordability within the Systems Engineering paradigm. This paper is designed to provide insights, lessons learned, and suggestions for using Value Engineering in the affordability process...2013Methods and Models II
Adding Process Tailoring to Product Size for Better Cost EstimationDavid Bloom, Chris FudgeGenerally, when one is looking to create a parametric model to predict costs of an activity, then one looks to size relationships associated with that activity. For example, if one were to develop a parametric model for estimating the development of a Field Programmable Gate Array (FPGA), then, because most of the work to complete an FPGA revolves around software code development, building a parametric model around new code, modified code and reused code (with other size relationships) would probably be a good place to start...2013Parametrics
Evaluating Cost Relationships with Nonparametric StatisticsCaleb FlemingParametric statistics are often used to analyze data, evaluate hypotheses, and determine the significance of a given set of inputs. Most commonly, cost estimators use the following parametric testing measures for significance: Pearson's product-moment correlation, z-tests, t-tests, and Analysis of Variance (ANOVA) tests. While each of these tests is subjectively appropriate to evaluate varying scenarios, each test also objectively depends on the validity of a fixed set of assumptions. Though the assumptions differ by test, all root themselves in the general idea that the data is contained within an adequately large sample and follows a particular probability distribution with estimable parameters...2013Parametrics
Fit, Rather Than Assume, a CER Error DistributionShu-Ping HuAnalysts usually assume a distribution (e.g., normal, log-normal, or triangular) to model the errors of a cost estimating relationship (CER) for cost uncertainly analysis. However, this hypothetical assumption may not be suitable to model the underlying distribution of CER errors. A distribution fitting tool is often used to hypothesize an appropriate distribution for a given set of data. It can also be applied to fit a distribution to...2013Parametrics
Developing a Military Aircraft Cost Estimating Model in KoreaSung Jin Kang, Dong Kyu KimA parametric cost estimating model is very useful for the weapon acquisition process. But it requires validated and normalized historic cost data to develop the Cost Estimating Relationships (CERs). Statisticians seek hundreds of data points to assure a good statistical fit between CERs and the supporting data points comprised of cost, technical, and program information...2013Parametrics
Improving Program Affordability Through The Application of Data AnalyticsDavid Wang, Austin LeeIn the current budgetary environment, program affordability is a key concern. Improving program execution and minimizing schedule delays and cost growth are keys to improving program affordability. However, there is a lack of quantitative analysis of schedule risks, and a lack of understanding of the root causes for schedule delays and cost growth. Consequently, most of the common affordability improvement suggestions centered on relaxation of test and verification requirements, and streamlining of engineering processes may not address the true root causes. The effectiveness of these affordability improvement suggestions is uncertain...2013Parametrics
Portfolio Management: A Parameter ApproachAndy NichollsMy paper outlines one high level method of managing a portfolio of future projects from pre-concept into acquisition phases and examines current practices and tool set requirements at the same time; examines the limitations of the current toolset and the possibilities of inclusion of probabilistic risk calculations via use of individual program risk registers...2013Parametrics
On General Purpose Model CredibilityWendy Lee, Evin StumpIn the aero/defense community and elsewhere, cost estimation of proposed large scale hardware projects by prospective bidders or sponsors is often done using general purpose parametric models, i.e., models based on statistical analysis of a collection of historical data. Of interest in this paper are the parametric models that provide estimates of costs of hardware development and production for a wide variety of hardware, especially hardware that may be used in many situations and environments...2013Parametrics
Benefits of Integrating Schedule and Cost Risk AnalysisRafael HartkeProject managers have the arduous task of running their projects within budget and deadlines, all while facing a wide range of risks like prices and labor costs uncertainty, suppliers performance and reliability, technological challenges, weather, currency exchange rates, etc. Thus, assessing the likelihood and impact of risks is of utmost importance to manage not only the project itself, but also the expectations of the various stakeholders...2013Risk I
Modeling the Risk and Uncertainty of Inflation Rate ProjectionsBrian Flynn, Peter Braxton"Experience in controversies such as these brings out the impossibility of learning anything from facts till they are examined and interpreted by reason; and teaches that the most reckless and treacherous of all theorists is he who professes to let facts and figures speak for themselves, who keeps in the background the part he has played, perhaps unconsciously, in selecting and grouping them, and in suggesting the argument post hoc ergo propter hoc." Alfred Marshall...2013Risk I
Probabilistic Mass Growth UncertaintiesDarren Elliott, Eric PlumerMass has been widely used as a variable input parameter for Cost Estimating Relationships (CER) for space systems. As these space systems progress from early concept studies and drawing boards to the launch pad, their masses tend to grow substantially hence adversely affecting a primary input to most modeling CERs. Modeling and predicting mass uncertainty, based on historical and analogous data, is therefore critical and is an integral part of modeling cost risk. This paper presents the results of an ongoing NASA effort to publish mass growth datasheet for adjusting single-point Technical Baseline Estimates (TBE) of masses of space instruments as well as spacecraft, for both earth orbiting and deep space missions at various stages of a project's lifecycle...2013Risk I
Deciphering JCL: How to use the JCL Scatterplot and IsocurvesEric DrukerAs the use of integrated cost and schedule risk analysis (ICSRA) methodologies rapidly expands across U.S. government programs there has been increasing confusion on how to interpret its seminal result: the Joint Confidence Level scatterplot and its associated isocurve. Over the past few years, the following statements have been made about JCL:...2013Risk I
Understanding the Results of an Integrated Cost/Schedule Risk AnalysisJames Johnson, Darren ElliottThe recent rise of integrated risk analyses methods has created a new opportunity for complex projects to understand the dynamic inter-relationship of cost, schedule, and risk. NASA has been implementing Joint Confidence Level (JCL) analysis of cost, schedule, and risk for all major projects since 2009 in an attempt to proceduralize and codify the requirements for an integrated risk analysis product...2013Risk I
Top-level Schedule Distribution ModelsPeter FredericIn the initial stages of a development project, it is sometimes necessary to build a summary-level schedule for planning and budgeting purposes before the day-by-day details of the project are fully defined or understood. However, when uncertainty assessments are performed on schedule networks containing few activities, the distribution forms chosen for individual activity durations can have a significant impact on the overall results. It is therefore important to choose uncertainty distribution forms that accurately represent the behavior of the sub-network of activities represented by each summary activity...2013Risk I
Analysis Schedules: Danger at a Higher LevelJustin HornbackA government organization charged with evaluating the adequacy of schedule and funding of large projects using project provided schedules and cost estimates receives a detailed (~2000 line) schedule capable of analysis with limited modification or summation. Analysis conducted using detailed schedule revealed interesting results when compared with high level schedule used in previous evaluations of the same project when a detailed analysis was not available...2013Risk I
Cost of Mission Assurance for Space ProgramsErik Burgess, Joe Frisbie, Michelle Jones, Chad KrauseStandards and strategies for assuring successful space missions have evolved in recent years, with some programs moving toward traditional government standards and others moving to industry's commercial standards. This has prompted questions from several government leaders about the cost of mission assurance. Although there are decades of engineering studies, test data, and on-orbit failure data to support detailed evaluation of the technical merits of various approaches, there is very little on the cost side of the equation. The National Reconnaissance Office Cost Analysis Improvement Group (NRO CAIG) has begun to address this by focusing on two of the more costly areas of mission assurance—environmental testing and high-reliability electronic parts...2013Risk I
Base Realignment and Closure (BRAC) Savings and Acquisition RiskPeter Braxton, Kevin Cincotta, Richard LeeThe Government Accountability Office (GAO) recently released the study Military Base Realignments and Closures: Updated Costs and Savings Estimates from BRAC 2005 (GAO-12-709R, June 29, 2012). Its appendices contain a wealth of risk data, with initial estimates (2005 BRAC commission) and final costs (Fiscal year 2011 DOD budget) for 175 distinct BRAC initiatives. Applying an innovative method for modeling within-program risk and uncertainty using cross-program data, this paper derives cost growth factors (CGFs) and coefficients of variation (CVs) for BRAC initiatives. Furthermore, the pattern in these data is astoundingly similar to that found in major defense acquisition program (MDAP) data, a strong confirmation of this modeling approach (meta analysis)...2013Risk II
Correlation Matrices RevisitedSteven IkelerIn a cost risk analysis, risk distributions are sometimes assigned at a low level and then combined, most commonly using Monte Carlo simulation. When using this technique, it is important to use a realistic correlation matrix. It is usually impractical to know all the pairwise correlations. This session will address some of the techniques for completing a credible correlation matrix at differing levels of input information and certainty. Additionally, the session will show examples and review the mathematics behind legitimate correlation matrices. Past sessions have discussed these topics and typically deal with correction techniques for expert-based correlation matrices. This session will be different since it deals less with correcting expert based correlation matrices and more with dealing with uncertainty in the correlations or incomplete correlation matrices. It will also address an extension of the problem to a parametric analysis problem.2013Risk II
Joint Cost Schedule Risk and Uncertainty HandbookDuncan Thomas, John Fitch, Alf Smith, Jeff McDowellThe Naval Center for Cost Analysis (NCCA) has created a Cost Schedule Risk and Uncertainty Handbook (CSRUH) that clearly presents simple, well-defined cost risk and uncertainty analysis processes that are repeatable, defendable and easily understood. The CSRUH is based upon the Air Force Cost Analysis Agency (AFCAA) Cost Risk Uncertainty Handbook published in 2007. It updates key processes to address cost risk and uncertainty methods with an emphasis on providing guidance to capture the impact of schedule uncertainty and the risk register while avoiding "double counting"...2013Risk II
Cost Contingency Analysis using PolytopesBohdan KaluznyContingency is defined as a possibility that must be prepared for—an event that may occur but is not likely or intended. In the realm of cost estimation, contingency refers to a reserve above and beyond a baseline estimate that would be tapped if one or more unexpected events led to higher program costs. Ideally, the selection of a contingency amount should be based on achieving a particular level of confidence derived from cost uncertainty and risk analyses, but often a mere rule-of-thumb percentage (e.g. 15% of the baseline estimate) is applied...2013Risk II
Use of the Risk Driver Method in Monte Carlo Simulation of a Project ScheduleDavid HulettIdentifying the root causes of project schedule and cost risk requires that the risk to the project schedule is clearly and directly driven by identified and quantified risks. In the Risk Driver Method the risks from the Risk Register drive the simulation. (As a side note, we find that the Risk Registers are not complete—during the interviews to collect risk data the interviewees introduce important risks that are, surprisingly, missing from the Risk Register.) The Risk Driver Method differs from older, more traditional approaches in which 3-point (low, most likely and high) estimates of the activity durations are applied directly to activity durations...2013Risk II
Robust Default Correlation for Cost Risk AnalysisChristian SmartCorrelation is an important consideration in cost risk analysis. Exclusion of correlation from cost risk analysis results in the de facto assumption that all risks are independent. The assumption of independence leads to significant underestimation of total risk. However, figuring out the correct correlation values between work breakdown structures elements can be challenging. For instance, it is difficult to estimate the exact correlation value between the structures and thermal protection subsystems in a cost risk estimate...2013Risk II
A Step-Wise Approach to Elicit Triangular DistributionsMarc GreenbergAs the federal government acquires less mature, more advanced and more complex systems, there is an ever-increasing burden on the cost analyst to employ methods of eliciting requirements, schedule and cost uncertainties from one of more subject matter experts (SMEs). Arguably, the most common technique a cost analyst uses today to elicit such data is to ask each SME for the lowest, most likely and highest value which, consequently, produces a triangular distribution...2013Risk II
Innovative Business Agreements and Resulting Cost & Pricing Methods at NASA in Support of New Commercial ProgramsJames Roberts, Torrance LambingOn April 15, 2010 President Obama delivered a speech at Kennedy Space Center in which he outlined his new vision for the U.S. space program. Emphasis was placed on enabling the exploration of Space by Commercial entities instead of by the Government. Since that time, NASA’s role has changed in many instances from being a program manager – overseeing development of space launch hardware and conducting space exploration missions – to one of support and a provider of space-related facilities and infrastructure...2013eTrack
Touch Labor Estimate ModelingMichael Yeager, Lyle DavisTo support its mission for the F-35, the production cost estimating team developed touch labor estimate models which include the flexibility to run production rate effects, loss of learning, commonality adjustments, affordability initiative and outsourcing impacts, multiple learning curve break point analyzes, and estimates for touch labor in hours or by realization. Given the scrutiny to the F-35 Lightning II program by the Department of Defense and Congress detailed and accurate cost modeling allows for better budgeting and more credibility within the services themselves and with the American public...2013eTrack
Methods to Analyze Services Portfolio Cost Drivers and EfficienciesValerie Reinert, Virginia WydlerThis presentation provides a statistical methodology used to evaluate the "health" of a services portfolio through performance data analysis and cost analysis, with the goal to identify cost drivers and efficiencies for performance improvement. Managing a Federal government portfolio is a complex endeavor. Portfolio Managers have a major responsibility to evaluate whether their portfolio is operating efficiently and effectively. Thus, they need to use analytical procedures that, when executed, will evaluate the "health" of the portfolio and identify areas for increased efficiency and effectiveness. If the analysis is not data-driven, this leads to a lack of proper business case analysis and cost benefit analysis to weed out bad actors in the portfolio...2013eTrack
Utilization of Visual Basic in Cost Estimating and Analysis Tools - Anyone Can CookJeremy EdenAs collaborative environments become more prevalent in all industries, the cost estimating and analysis industry is no exception to this movement. Increasing amounts of pressure are put upon cost estimators and analysts to develop tools that are low cost, fast, robust in design, have long standing methodologies, do not require proprietary software or licenses, and are and easy to use by both the advanced cost estimator looking for maximum control and the novice simply trying to diligently support the early stages of a development program...2013eTrack
How to Estimate and Measure "the Cloud" and Make COCOMO Cloud EnabledDavid Seaver, John RosbrughCloud and Big Data have become the new buzzwords in the commercial, federal and DoD technology arenas. This presentation will: 1. Define Cloud & Big Data 2. Identify what's new and different with the Cloud and Big data. 3. Identify the cost drivers for cost estimates...2013eTrack
Got a Defense Business System? The effect of the new defense business system policy on cost estimating, tools, and fundingCharles GuThe Department of Defense (DoD) budgets over $7 billion a year for business system investments and has mandated that all information technology systems must be assessed and certified through a formal process before funding can be obligated in FY 2013. This new law, Section 901 of the Fiscal Year 2012 National Defense Authorization Act (FY2012 NDAA), significantly expands the scope of systems requiring certification to include any business system with a total cost in excess of $1M over the period of the current future-years defense program, regardless of type of funding or whether any development or modernization is planned...2013eTrack
Cloud Computing Starter Kit: Cost and Business Case ConsiderationsJennifer Manring, Raj AgrawalCloud computing is a major trend in the information technology (IT) industry that is affecting both commercial companies and Federal Government agencies due to its promise of agility, scalability, and cost savings. Prior to investing in cloud computing, decision makers are asking "what is my return on investment (ROI)?' However, there is no one size fits all answer to this question and caution should be exercised when applying generic rules of thumb and quick assessment cost and economic models. Costs, benefits, and risks must be carefully assessed for each situation...2013eTrack
TitleAuthor(s)SummaryYearTrack
From Bid Package to Detailed EVM Baseline in One Easy StepKaren StiffYour Company has put together a cost estimate and basic schedule along with the technical elements as a bid package for a Request for Proposal for a Government contract requiring Earned Value Management (EVM). The proposals have been evaluated, discussions have been held and now it's time for the award. Congratulations your company's won the contract! Now the manager assigned to the program has the task of setting up an EVM baseline for the program. Why is it so hard to do a detailed schedule and EVM baseline, when you've got the schedule and cost that the company bid? Why doesn't this baseline look in any way similar to the bid package cost and schedule? Should it look similar?...2012Earned Value Management Track
A New EVM Performance Index: The MRPIMichael NosbischIt has long been understood that the accuracy of the cost performance index (CPI) is closely tied to how management reserve (MR) has been historically utilized on a specific project/program. However, a formal assessment of MR usage is still not consistently performed in conjunction with CPI calculation and subsequent analysis. This presentation will offer a potential solution to this issue, by proposing a "Management Reserve Performance Index," or "MRPI." Once calculated, the author will demonstrate how the MRPI can be then used to develop a "risk-adjusted IEAC" that provides a more accurate assessment of a project's or program's overall cost performance.2012Earned Value Management Track
EVM for the Rest of UsJavier SloninskyEarned Value Management (EVM) is the practice of evaluating cost and schedule data to measure productivity and progress throughout the lifecyle of a project. EVM, however, often has the stigma of being overly complex and burdensome to implement in a useful way. This session will discuss best practices in implementing a simple yet robust earned value management solution for organizations that benefit from progress measurement and the ability to forecast cost performance, but don't need to comply with ANSI 748, the EVM standard often used by government agencies and contractors. We'll discuss topics including the integration of required data, developing key performance indicators (KPI) and metrics, application of rules of credit, updating of percent complete, and automated EV reporting.2012Earned Value Management Track
Accepted Standards and Emerging Trends in Over Target Baseline (OTB) ContractsSimon DekkerOver Target Baseline (OTB) projects or programs are those that have run significantly over cost and require formal reprogramming - essentially a complete replanning of the project - in order to help the contractor regain management control over the effort. The OTB process has been well documented and become an established part of Earned Value Management practice. Much of the literature-to-date has focused on OTBs from the contractor perspective, including the steps to take in order to propose and implement an OTB, and the proper channels and occasions for engaging the customer in the process...2012Earned Value Management Track
Successful Implementation of an Over Target Baseline/Schedule from a Government PerspectiveJoseph C. Annunziato, Daniel W. DonaldsonIn recent years, Over Target Baseline/Schedule implementations have been accomplished with varying degrees of success. Using the Over Target Baseline (OTB) and Over Target Schedule (OTS) Handbook as a guide, this presentation will address the proper way to implement an Over Target Baseline/Schedule from a Government perspective. The purpose is to provide both government and industry professionals a general understanding of the processes and decisions that must be considered when implementing an OTB and to assist the earned value management (EVM) community in the understanding and implementation of this important management tool. It will also incorporate examples and lessons learned from recent OTB/OTS implementations...2012Earned Value Management Track
Critical Chain Applied to an MRO - Surface Repair FacilityHoward RainerThe project management methodology of Theory of Constraints, called Critical Chain, is examined for application to Scheduling, Planning and Control (SP&C) for a defense-related Maintenance Repair Operation (MRO). The opportunities for Critical Chain, as an extension of Theory of Constraints (TOC), have been reported for well over a decade. With this proven record comes a proposal for similar application in the SP&C at the Surface Repair Facility (SRF) at Cecil Field, Florida. The SRF has established reporting systems such as Earned Value Methodology (EVM) - mentioned here to distinguish the differences between execution and reporting in SP&C.2012Earned Value Management Track
Integrated Project Management (IPM) - Transforming Data into InformationAndrea Mozzo, Bruce KoontzEarned Value Management (EVM) integrates the cost, technical and schedule performance. EVM is not just tracking Schedule Performance Index (SPI) and Cost Performance Index (CPI), but has been transformed into Integrated Project Management (IPM). IPM involves three key areas: Managing cost, schedule, and technical performance within constraints; initiating effective cost, schedule, and performance tradeoffs when constraints are not achievable; and continually evaluating progress to predict and mitigate problems...2012Earned Value Management Track
WBS Development: Rules, Aberrations and MIL-STD-881C Implementation ChallengesMichael MetcalfIssued on October 3, 2011, Military Standard 881C (MIL-STD-881C) provides the framework for developing Work Breakdown Structures (WBSs) for use throughout the acquisition process, as well as a standard set of structures that fit eleven commodity types. These WBSs are used for management, system engineering, and cost estimation by a variety of stakeholders. Though comprehensive, the MIL-STD does not address some of the complex details encountered when developing a weapon system WBS, and there are many special cases and exceptions that could not be captured in the document...2012Earned Value Management Track
"The Answer is 5": Observations on Cost & Schedule in Small Defense ProgramsBrian Fersch, Wesley Tate, Colleen LeonardCost & schedule overruns have been an enduring issue in government acquisitions. Our largest programs are typically dissected and their flaws illuminated with 20/20 hindsight. DoDs smaller programs, though comprising the majority of the budget and quantity of programs, do not get this same attention. Over the course of the past year this study team has collected historical & current cost and schedule data on numerous programs that has been collected at similar points in each programs acquisition lifecycle. The end result is a database that allowed the team to examine how cost estimates and schedule expectations changed from year to year for the same events...2012Earned Value Management Track
Joint Analysis of Cost and Schedule (JACS) - A New Tool for JCL AnalysisAntonio Rippe, Rey CarpioNASA has recently implemented a policy that requires a program/project to be approved at a 70% Joint Budget and Schedule Confidence Level (JCL) which are to be generated from models that integrate cost and schedule. To comply with the policy, a tool framework is needed that can conduct a risk and uncertainty analysis on schedule logic as well as integrate cost uncertainty analysis to account for phasing and costs that are dependent on overall duration. NASA is currently using several platforms, but has seen some issues in extensibility, cost integration, and overall performance...2012Earned Value Management Track
Optimization of Cost and Performance in Complex Systems Using System Dynamics ModelingMichael Polly, Benjamin HarrisAnalysis of trade spaces between cost and performance is critical to win new contract pursuits in today's highly competitive environment. The overall goal in a program is to design and produce a system that maximizes both performance and affordability. In attempting to balance cost and performance, traditional cost models only allow for experimentation with one variable at a time. This can be very cumbersome on large systems, and inhibits how much of the trade space can be explored under time constraints. Altering one variable at a time also limits the number of cost estimating relationships that can be incorporated into the analysis...2012Earned Value Management Track
Software Maintenance Data Collection and Estimating ChallengesVanessa Welker, Peter Braxton, Wilson Rosa, Corinne Wallshein, Joe DeanEntering a period of fiscal austerity, it becomes more important than ever to estimate and consider operating and support (O&S) costs, which represent the lion's share of life cycle cost (LCC) for most platforms, during acquisition. Given the ubiquity of software in today's complex programs, a key component of O&S is software maintenance. This paper presents the results of a research study co-sponsored by the Air Force Cost Analysis Agency (AFCAA) and the Naval Center for Cost Analysis (NCCA) to collect software maintenance data from government support activities and development contractors to enable high-fidelity cost estimates for software maintenance...2012Life Cycle Cost
Estimating Software Maintenance Costs for U.S. Army Weapons SystemsCheryl Jones, James Judy, Brad ClarkThe emerging defense economic environment is characterized by scarce resources, extended system life cycles, and evolving mission capability requirements. As a result, the Army's investment in software maintenance, sustaining engineering, and operational support efforts is coming under increased scrutiny by decision makers at all levels. Of particular interest is the validity and defensibility of the estimates used to determine, allocate, and evaluate the value of software maintenance funds...2012Life Cycle Cost
How VAMOSC VIEWS Can Help You!Brian Welsh, Walt Cooper, Paul Hardin, Elizabeth Koza, Robert NehringSeveral recent developments underscore the importance of identifying and analyzing operating and support (O&S) costs for weapon systems. This brief will present the VAMOSC VIEWS, innovative graphical displays of O&S cost data that serve as an invaluable tool for the cost analyst. In response to current economic conditions, we are entering an environment of reduced budgets within the Department of Defense (DoD). Legislative initiatives, starting with the Weapon Systems Acquisition Reform Act (WSARA) of 2009, are aimed squarely at increasing the visibility and use of O&S costs in weapon systems decision-making...2012Life Cycle Cost
Software Sustainment: Pay Now or Pay LaterArlene MinkiewiczWith today's budget constraints the Department of Defense (DoD) is likely to build less new systems. In light of this, the systems that we have today will need to be stretched further. This means more time and effort will be going into maintenance. And since many systems fielded today rely on software to deliver much of their mission critical capability; this means more time and money will be devoted to software sustainment...2012Life Cycle Cost
Discrepancy Report Prioritization and Software Maintenance ImpactsJennifer Woolley, Kyle ThomasIn the current budget-constrained environment, programs and organizations throughout the Intelligence Community (IC) are searching for new strategies to reduce costs. Rather than sacrificing new capabilities, maintenance is often one of the first areas of reduction. Many organizations perform developed software maintenance based on the submission of Discrepancy Reports (DRs), which are ranked according to their level of severity/importance. One approach to cutting maintenance costs involves only focusing on higher levels of DRs while leaving lower level DRs postponed or unaddressed. This strategy allows for a reduction in personnel because fewer DRs must be completed...2012Life Cycle Cost
Tactical Wheeled Vehicle (TWV) Fuel Economy Improvement Breakeven AnalysisRaymond KleinbergWith the rising costs of and attention towards fuel usage in the Department of Defense (DoD), efforts are being made to procure more fuel efficient vehicles and modernize the Army's existing Tactical Wheeled Vehicle (TWV) fleet. This research is part of an ongoing effort to help program managers make decisions regarding how much investment can be made into new technologies in order to see benefits within the estimated useful life of individual systems and the fleet as a whole.2012Life Cycle Cost
DCARC Contractor Sustainment Cost Data CollectionSandra B. EnserIn the past, Contractor Sustainment (including Contractor Logistics Support (CLS), Contractor Performance Based Logistics (PBL), and other contractor sustainment efforts) were omitted from DoD centralized O&S databases, or were captured as an annual total with little or no supporting detail. For many programs, Contractor Sustainment costs are a major cost driver. They represent an increasing percentage of operating costs. As DoD faces major budget reductions, valid and timely Contractor Sustainment cost data supports analysis which enables Services to use declining operations funds effectively...2012Life Cycle Cost
Designing a Conceptual Framework for Estimation and Analysis of Total Ownership CostF. Gurney Thompson III, Robert KouryIn recent years, the push for greater efficiency and productivity in Defense spending has yielded an increased focus on affordability analysis. Understanding and estimating Total Ownership Costs (TOC) is key in assessing affordability, and the cost community must adapt to support TOC estimation. This paper discusses the development of a conceptual framework for estimating TOC in support of a broader audience, from the acquisition community to program managers and even as a decision support tool for entities such as Congress, DoD Financial / budgetary community, and G-8 Program Analysis & Evaluation...2012Life Cycle Cost
Life Cycle Costs in CBANiatika Griffin, Ronako CarsonCost Benefit Analysis (CBAs) are of growing importance to the acquisition process. The Vice Chief of Staff for the Army signed a memo dated December 20, 2009 stating that, in this era of constrained resources, Army leaders need to be responsible stewards of government funds. Any new requirement that is unfunded and/or newly expanded program proposals will be accompanied by a thorough CBA. The CBA must identify the total cost proposal to the Army in the form of life cycle cost, benefits that will result, and the bill-payers that would be used to pay for the unfunded or new requirement...2012Life Cycle Cost
Preliminary Study of Palm-Oil Biodiesel Life Cycle Cost Analysis in IndonesiaSidhartha Sahirman, Dr. Saparso, Agus Sarjito, Mokhtar Awang, Lr. Shaharin A. SulaimanRecently, Biodiesel have been gaining popularity and getting accepted as an alternative fuel for diesel engines. In the last few years, Indonesian government has a special interest in the development of this renewable energy, the use of which can help reduce oil imports. Biodiesel has a bright future in this country, as Indonesia is the world's largest producer of crude palm oil (CPO) - a desirable feedstock for biodiesel production. Biodiesel has the potential to become a significant industry sector in Indonesia...2012Life Cycle Cost
Evaluating the Life Cycle Cost and Effort of Project Management for Complex Systems Development ProjectsLeone YoungAcross industries, systems development effort has become more costly and complex in nature over the decades, and the costs of project management that are associated with systems development effort have shown an upward growth as well. The estimation of systems development cost and effort has been studied by many scholars and researchers in several different areas such as hardware, software, integration and systems engineering, but the area of project management cost and effort has had very limited exploration, and the literature is void of project management costing methodologies as of today...2012Management
Modeling R&D Budget ProfilesErik BurgessPrior research sponsored by the National Reconnaissance Office (NRO) and published in the Journal of Parametrics (Summer 2006) has become the principal method for developing space and ground segment budget profiles for NRO programs. Since then, the practice of budgeting to independent cost estimates has been signed into law for the intelligence community. The resulting scrutiny and reliance on budget-phasing models has motivated several improvements that should be relevant to the estimating process for other DoD commodities. The presentation first reviews the mathematical formulation of expenditure profiles that are based on historical data, and how some of the resulting accuracy metrics have been useful in defending annual budgets. New models for space and ground systems are presented. Since the annual budget authority required to support an expenditure stream is heavily dependent on a program's execution rates (i.e., outlay rates), we also present a new approach for estimating what those outlay rates will be. This is a new area of responsibility for most cost estimating organizations, yet its impact on annual budget requests can be significant.2012Management
IT Service-Based Costing: Standardizing Provisioning and Servicing IT ResourcesEmily Jessen, Paul BrownIn today's current economic environment, there is a strong focus on constraining spending of appropriated funds to meet the mission particularly within the Intelligence Community and Department of Defense (DoD). Part of the DIA's mission includes the delivery of IT products and services. In the past, IT services have been provided to non-Core customers for free, but in the new fiscal environment, the DIA does not have sufficient resources to continue to operate in this manner. In compliance with the Economy Act of 1932 and DoD Financial Management Regulations, the DIA has been taking steps to quantify the IT services it deploys in order to recover costs of those services...2012Management
Trying To Do Too Much with Too Little: How Poor Portfolio Management Can Lead to Schedule Delays and Cost OverrunsChristian SmartGovernment organizations often try to accomplish too much with the resources with which they are provided. The genesis of new projects in government organizations often begins with the search to spend idle funds. An organization's leadership may discover that it has a relatively small amount of money that can be used to research a new technology or to begin initial development of a new project. New ideas to improve systems, develop better, newer ones, or to advance scientific understanding abound. In order to sell leadership, project management provides leadership with success-oriented, optimistic projections...2012Management
Case Study of Army Cost Management: Managing Variability in Analytical Cost Products for an ACAT ID ProgramTomeka WilliamsArmy Cost Management requires an understanding of the level of commitment the Department of the Army has made to cost management that goes beyond the Program Management Office (PMO) through, TRADOC, and HQDA levels. By requiring any program that is a new start, or major upgrade of an existing program to submit a Cost-Benefit Analysis based on guidance from the VCSA, is one of the unique cost management products the Army has instituted. This is now the first step to securing a viable and executable acquisition program within the Department of the Army...2012Management
How does a $20B Program Become a $30B Program?  Lessons Learned in Production Cost ManagementSteve Sheamer, Allen GaudelliHow does a $100M ship become a $200M ship or a $75M fighter jet turn into a $125M fighter jet? As we have seen recently with several high profile acquisition programs, systems integrators and cost estimators continue to struggle when estimating the production costs of development systems. This presentation focuses on production-related cost drivers and why contract incentive-fee structures are often part of the problem, then details steps for better management of these production costs...2012Management
NRO Program Assessments - Best Practices and Lessons LearnedGreg Lochbaum, Linda Williams, Lisa Keller, Ken OdomThe National Reconnaissance Office (NRO) performs program assessments for a variety of purposes: the management of acquisition programs; portfolio analysis for resource allocation; and accountability to NRO's oversight organizations - Execution to Oversight. The NRO Cost Analysis Improvement Group (CAIG) recently stood up a Program Assessment group to continuously improve the NRO's program assessment process to become more efficient, consistent and standardized. This presentation provides the NRO's approach to each level of the Execution to Oversight pyramid...2012Management
Capabilities Based Portfolio Analysis (CBPA)Paul Gvoth, Brad EllisTotal cost is no longer the singular driver of defense acquisition program economic decision making. Defense programs and projects must be evaluated according to their contributions to overall monetary and non-monetary Return on Investment (ROI) relative to the capabilities of a Family of Systems (FoS), System of Systems (SoS) or as a part of a portfolio of products that together meet documented operational requirements. This is needed because future defense strategies, including Smart Defence are being driven by budget cuts...2012Management
Using the Tools of Persuasion to "Sell" Your EstimateJennifer KirchhofferWhat is the difference between a good cost estimator and a great cost estimator? Both have robust estimating skills and knowledge, but only great cost estimators are able to garner acceptance and usage of their estimates from even their harshest critics. The ability to "sell" the estimate is key to the utilization of the estimate in the proper context by the decision makers. Our jobs often entail delivering news that our customers and leaders are not anxious to hear. Many factors work against us as we try to garner support and acceptance of the estimates...2012Management
A Case Study in Broad System Analysis: DoD Spectrum Reallocation Feasibility Study, 1755-1850 MHzBrian Wilkerson, Guenever AldrichWith the increasing demand placed on the electromagnetic spectrum by the proliferation of web enabled cell phones, the President has begun an initiative to sell 500 MHz of dedicated bandwidth from the federal government to the wireless industry. The 1755-1850 MHz band was selected as the first band under this initiative for a study of the cost and technical feasibility of relocating DoD assets that use these frequencies. The study was incredibly broad as all services were involved, and the systems affected include Precision Guided Munitions, Point to Point Microwave, High Resolution Video, Multiple Radio Systems, Electronic Warfare, Air Combat Training, Telemetry, Satellite, and Small Unmanned Aerial Systems...2012Management
Cloud Computing: Federal Mandates and the DoDHeather Nayhouse, Eric LumsdenThe Federal CIO has laid out a plan to implement cloud computing in all federal agencies and consolidate 800 data centers by 2015. The Office of Management and Budget has required agencies to default to cloud computing solutions wherever those solutions are secure, reliable, and cost-effective. In response, the Department of Defense has launched a number of pilot programs as well as its own strategic plan to foster their deployment of cloud computing systems. Several case studies of cloud solutions within the Department of Defense are already available. The DoD is analyzing these cases for guidance into a larger push into cloud computing systems...2012Management
Building a DHS Cost Estimating & Analysis Center of ExcellenceKatie Geier, Colleen Craig, David Brown, Kevin CincottaThe Department of Homeland Security (DHS) Cost Estimating & Analysis (CE&A) Center of Excellence (COE) serves as the Department's primary advocate and resource for cost estimating and analysis. Formed in October 2011, the CE&A COE's primary goal is to take a proactive, streamlined approach to improving the Department's resources and skills in cost estimating and analysis. This presentation highlights the roles and responsibilities of CE&A COE and summarizes the work accomplished to date and planned future efforts for accomplishing the following objectives:...2012Management
Social Media ApplicationsMarc Wear, Michelle Ehlinger, David Pearson, Sara WiseThis paper explores the impact of adopting social media tools in the work environment as a means to improve communication, collaboration, and productivity for cost and earned value analysts. Public and private entities have taken steps to adopt social media platforms to benefit their organization; this paper demonstrates how cost analysts can implement these tools to improve methods and processes...2012Management
A Holistic Approach to Multiyear ProcurementsAnn Hawpe, Travis WinsteadMulti-year contracts have the potential to save the Government a substantial amount of money if they are executed on stable programs in an environment with stable requirements and funding. With the current economic pressure for defense programs to do more with less, multi-year contracts are a good option for programs that meet these criteria. The largest amount of savings is the result of procuring a larger quantity of parts than would be procured with a year-over-year contract structure, thus realizing economies of scale. Savings are also captured when the labor hours to develop and submit year-over-year proposals are factored into the calculations...2012Management
Using Treasury Securities to Develop Inflation IndicesC. Tyler Cunningham, Joseph ParisiThis paper examines inflation assumptions in Department of Defense cost estimates. Professional guidance allows and encourages the development of custom inflation indices, but there are few resources detailing the methodologies to develop those indices. We address that need by laying out detailed steps to develop defensible, custom, risk-adjusted inflation indices using techniques that leverage analysis of the spread and volatility between traditional US Treasury Securities and Treasury Inflation-Protected Securities. 2012Management
Job Satisfaction: The Link to Retention and the Correlation to Age, Gender and Organizational PositionDarrin L. DeReusIn an attempt to balance resource availability and workload, leaders in the United States Air Force have attempted to manage the budgets by fluctuating manpower levels. There is minimal research of multiple affiliations (active duty military, government civilians and contractors) and the effects of manpower reductions on the organization. This study collected data on career anchors and satisfaction levels to find the similarities and differences of multiple affiliations in the United States Air Force. The results of this study showed that there were correlations between career anchors and satisfaction scores...2012Management
Building an Agile, Collaborative Environment for Capturing Productivity Based Cost Model DataDavid Bloom, Wanda Grant, Mason WexlerUnderstanding and analyzing productivity within an Aerospace electronics development organization requires continuous oversight and adjustment. For example, in digital electronics, a corollary to Moore's Law says that the capability of digital processor components doubles every 18 months. This would imply that the cost to design a set of digital electronics products to deliver a specific state-of-the-art capability would change (become less) as a function of the ability to incorporate more features on any one component...2012Methods and Models I
Galaxy Charts: The 1,000-Light-Year View of the DataRobert Nehring, Katharine Mann, Robert JonesThe old adage "A picture is worth a thousand words" is one that we have all heard and often try to use to our advantage. This paper applies the principles of visual display of information as advocated by Edward Tufte and others to develop an innovative graphic that will prove invaluable to cost estimators and consumers of their estimates everywhere...2012Methods and Models I
One NASA Cost Engineering (ONCE)Eric Plumer, Mike BlandfordThis presentation is the first roll out of NASA's ONCE (One NASA Cost Engineering) database to the SCEA/ISPA community. Concept and early prototyping of the One NASA Cost Engineering (ONCE) database began several years ago and it has been developed to fully automate the CADRe data in a GUI database allowing for easy search and retrieval of data for the cost community. By automating the retrieval of CADRe data this tool is greatly improving the space cost community's ability quickly acquire the data needed to improve the quality and credibility of these cost estimates...2012Methods and Models I
Defense Cost & Resource Center (DCARC) Executive Overview BriefMike AugustusThe OSD stood up an independent office to manage the CCDR (Contractor Cost Data Report) requirement in the late 1990s. Today, that office is known as the Defense Cost and Resource Center (DCARC), which is part of the Office of the Secretary of Defense (OSD) Cost Assessment and Program Evaluation (CAPE). The original charter has been expanded to accommodate the collection of not only cost data but software metrics data as well, from industry on ACAT 1 Major Defense Acquisition Programs and Major Automated Information Systems (MAIS) programs. Today the Cost and Software Data Report (CSDR) requirement spans over 140 MDAP ACAT 1C/1D programs and a large number of MAIS 1AM and 1AC Programs...2012Methods and Models I
An Approach to Improving Cost Estimating and Budget Integration in Federal ProgramsMichael Noonan, Kristin Jackman, Eric HongOver the past few years, political standoffs in Washington, D.C. have delayed or prevented passage of Federal Appropriations Bills and significantly reduced Federal spending. As Agency budgets continue to shrink, programs must increasingly compete with each other for a slice of dwindling resources. Program and Departmental managers responsible for resource decisions are demanding more detailed and agile resource justifications which can accommodate multiple budget scenarios. In response, program offices require tools that efficiently translate estimated program costs into flexible and defensible budgets...2012Methods and Models I
Enhancing Excel-Based Cost Models with PivotTable ReportingBlaze Smallwood, Omar MahmoudA major challenge facing DoD cost estimators is developing Program Life Cycle Cost Estimates (PLCCEs) that serve as more than just an acquisition reporting "check-in-the-box" document. The ideal is to simultaneously create a dynamic, flexible tool that the program office can use to support their day-to-day costing needs, from answering a wide-range of cost-related data calls to integrating PLCCE outputs into their Program, Planning, Budgeting, and Execution (PPBE) processes. For DoD cost estimators that utilize Microsoft (MS) Excel to create their PLCCE models, a valuable tool that can be leveraged to achieve these goals is Excel's PivotTable reporting capability...2012Methods and Models I
A Canadian F-35A Joint Strike Fighter Cost Estimation ModelBohdan L. KaluznyThe F-35 Joint Strike Fighter (JSF) is a single-engine, stealthy (radar-evading), supersonic multi-role fighter. Canada intends to purchase 65 F-35A-conventional takeoff and landing variant (CTOL) - jets for delivery between 2016 and 2022. In order to secure production, Canada will likely have to commit to procurement as early as 2012. Until recently, Canada relied on the United States (U.S.) JSF Program Office (JPO) for projected costs. In early 2011, the Department of National Defence (DND) F-35A cost estimate came under public scrutiny as a result of a Parliamentary Budget Officer report (Canada's equivalent to the U.S...2012Methods and Models I
Developing Standardized Cost Element Structures for the United States Marine CorpsJeremy EdenThis presentation explains the development of a Cost Element Structure (CES) for use in United States Marine Corps (USMC) Logistics Requirements Funding Summaries (LRFS). Industry and the Assistant Commander for Life Cycle Logistics formed an Integrated Product Team (IPT) to create a new CES. The IPT comprised logisticians and cost estimators who worked together over the course of one year to create the new USMC LRFS CES...2012Methods and Models I
United States Marine Corps Logistics Requirements Funding Summary (LRFS) Cost Estimating Tool (CET) - A Quick Cost Estimator for Logisticians Part IICharles GuSimilar to a Life Cycle Cost Estimate (LCCE), the Logistics Requirements Funding Summary (LRFS) captures Integrated Logistics Support (ILS) related costs for a program throughout its life cycle. The LRFS provides visibility into the logistics requirements of a program which is needed during Program Objectives Memorandum (POM) and budget submissions. Additionally, it helps to plan and quantify requirements, identify and defend funding, and serves as the ILS input to the LCCE...2012Methods and Models I
Modeling Potential Cost Savings by Synchronizing Commercial Derivative Acquisition & Lifecycle ProgramsBrad BoehmkeSynergy is defined as "the combined or cooperative action of two or more stimuli for an enhanced effect" (Razzetti, 2009). As Department of Defense (DoD) system acquisition costs continue to grow and commercial derivative programs becoming more accepted, it is critical that the Air Force look for ways to create cost savings in every possible way. With commercial derivative programs becoming more popular in defense acquisition, there may be acquisition programs running concurrently that have system commonalities and could benefit from synchronizing acquisition efforts. By leveraging commonalities in requirements, the Air Force and the Department of Defense may be able to "obtain cost savings through acquisition and logistics planning" (Poulin, 2010)...2012Methods and Models I
Zero Based Review MethodologyMartha Wells, Peter MeszarosThe days of incrementally funding activities based on past performance are over at DIA. Increased budget pressure throughout the federal government has created an environment for improvements to the way activities are funded. DIA is approaching this increasingly challenging budget environment with a new programming methodology - Zero Based Reviews. The Zero Based Review (ZBR) process incorporates programmatic analysis and activity-based costing to approach budgeting from a right sizing methodology. Rather than reviewing the enterprise based on current structure, the ZBR takes a topical approach to program review to determine if the right resources are being used in the right way for the right mission...2012Methods and Models I
Cost Benefit Analysis OverviewDarrell HamiltonThere are many different requirements to do some type of cost and benefit tradeoff analysis as part of the acquisition process, IT investment decisions, performance based logistics decisions and budget change proposals. Most of them have nuance definition differences and are known by their acronyms as CBA, EA, BCA, AoA, COA and several other terms. Every one of those analysis requirements is aimed at demonstrating or helping others to decide on what is the most reasonable and cost effective decision that needs to be made...2012Methods and Models I
Accuracy Matters: Selecting a Lot-Based Cost Improvement CurveShu-Ping Hu, Alfred SmithThere are two commonly used cost improvement curve (CIC) theories: unit cost (UC) theory and cumulative average cost (CAC) theory. Ideally, analysts develop the CIC by analyzing unit cost data. However, it is common that instead of unit costs, analysts must develop the CIC from lot cost data. An essential step in this process is to estimate the theoretical lot midpoints (LMP) for each lot, to proceed with the curve-fitting process. LMP is generally associated with UC theory, where the midpoint is always within the lot. The more general lot plot point (LPP) term is used in the context of both the UC and CAC theories...2012Methods and Models I
Will-Cost and Should-Cost Management: It's Not Business As UsualZachary JasnoffIn April , 2011 Under Secretary of Defense for Acquisition, Technology & Logistics Ashton Carter issued the Memorandum: "Implementation of Will-Cost and Should-Cost Management". The memo defines implementation of Should-Cost and Will management for all ACAT I, II and III programs and lists "Selected Ingredients of Should Cost Management". Thus, each organization involved with these programs must successfully deal with the challenges or planning, coordinating and managing Should Cost/Will cost programs and have the necessary tools to quantitatively manage them through their life cycle...2012Methods and Models II
DoD Contracts Database and Interactive ToolBrian OcteauProposal Summary Good cost estimates, analyses and assessments should consider all sources of data available. One of those sources - contract data - is often overlooked, and sometimes misused. For many, the collection, compilation, and interpretation of contract data can be an unwieldy and time consuming venture. However, once compiled, the data can be an invaluable resource for cost analysts, providing insight relative to cost, schedule, and technical growth, highlighting data completeness gaps inherent in other data sources, and revealing data trends and/or patterns...2012Methods and Models II
Rapid Business Case Development Using Macro-Based Excel ToolAndrew HutchinsonPurpose: To present a tool that creates business cases based on user-entered customizations such as the start and end year of the investment. With the increasing scrutiny of federal government budgets the development of business cases for future investments will take on added significance. A best practice within industry is the development of business cases to analyze the net present value of potential investment opportunities. The question this analysis answers is how much value will potential initiatives create within your organization and allows for an "apples to apples" comparison of multiple investment opportunities...2012Methods and Models II
An Intuitive Application of Cost Risk Analysis to a LRFSBlake BoswellThis presentation explains the implementation of an Uncertainty Modeling Capability (UMC) into a USMC LRFS. Similar to Life Cycle Cost Estimate (LCCE), the Logistics Requirements Funding Summary (LRFS) captures Integrated Logistics Support (ILS) related costs for a program throughout its life cycle. The LRFS provides visibility of logistics requirements for Program Objectives Memorandum and budget submissions. Additionally, it helps to plan and quantify requirements, identify and defend funding, and serve as the ILS input to the LCCE...2012Methods and Models II
Creating a Framework for Cost Analysis: The Cost Analysis Support Tool (CAST)Eric LumsdenThere are many redundancies encountered when creating new cost models, and many operations that can be automated. CAST is a tool created by TASC that provides a framework to accelerate the process of building a cost model. Estimators are able to execute processes at the click of a button that would otherwise be very time consuming to create from scratch or complete by rote. They are able to transfer over work from other projects to eliminate redundant operations...2012Methods and Models II
Identifying the Cost Capabilities of the DoDAF Architecture FrameworkHolly A. H. Handley, Resit Unal, Andreas TolkThe Department of Defense Architecture Framework (DoDAF) is a key tool used by the United States engineering and acquisition communities to describe the overall structure for designing, developing, and implementing systems. In 2009, version 2.0 of DoDAF was released. This version represented a major shift in focus from output products, or documentation, to the collection of the underlying data that represents the architecture. This version describes a new data model, the DoDAF Meta Model (DM2); the DM2 provides the mechanism needed to collect, organize, and store data. While many pre-defined models reside in the DoDAF, none of them are considered cost models of the system...2012Methods and Models II
EDGARS CB - A New Memory Tool for CostersWilliam BarfieldAre you befuddled by what it takes to do a cost estimate? What kind of information is used and when is it needed so that a defendable cost and benefits estimate can be provided to the investment decision authority? Since beginners and professionals alike rely on tricks-of-the-trade to help get their job done, a new and easy memory aid is presented...2012Methods and Models II
Significant Reasons to Eschew Log-Log OLS Regression when Deriving Estimating RelationshipsStephen A. BookLog-Log Ordinary Least Squares (LLOLS) regression, considered in the 18th and early 19th Centuries as the best (and, in fact, the only) method for fitting nonlinear algebraic relationships of the form y = axb to data sets of (x,y) pairs, has a number of serious defects that make it far from adequate for CER development in the 21st. No other option was available 200 years ago, but the advances in computing power and techniques of statistical optimization available to us today leave no reason to stick with an obsolete method. In the 21st Century, we insist that 21st Century engineering technologies be applied, so why would we continue to develop CERs to estimate them using 18th Century statistical methods?...2012Methods and Models II
Valuation in Cost Estimating: Taking a Page from the Investment Banker's PlaybookKevin SchuttValuation is "the process of estimating what something is worth". It is commonly undertaken in investment banking to support activities such as initial public offerings of company stock and mergers & acquisitions. The need for valuation analysis arose in a cost estimation setting in order to satisfy Office of Management and Budget budgetary treatment criteria, yet valuation techniques in general may be a useful addition to the cost estimator's toolkit, facilitating price analysis and cross-checks of cost estimates. Likewise, cost estimation techniques may inform valuation analysis. While cost estimation generally results in market-independent estimates, valuation provides the viewpoint of what the market will bear. This presentation will provide an overview of four major valuation techniques: Price multiple; Replacement cost; Comparable transaction, and Discounted Cash Flow. A case study in valuation will be presented. 2012Methods and Models II
Cost Estimating Training for Non-Cost EstimatorsJeremy EdenThe effort to improve the accuracy and fidelity of cost estimates is critical given budget restraints. As this effort continues, non-cost estimators (program managers, engineers, logisticians, etc.) are more frequently facing the demand of preliminary cost estimates prior to the development of formal cost estimates. These preliminary estimates are done in order to streamline the cost estimating process and guard against risk for a program in development. While most non-cost estimators have the information and tools available to address early program cost estimating needs, many do not have the skills and experience necessary to make good use of the resources at their disposal...2012Methods and Models II
Just-In-Time Cost AnalysisJohn KoBooz Allen developed and maintains a robust Life Cycle Cost Estimate (LCCE) for the acquisition of the Urgent Universal Need Joint Mine Resistant Ambush Protected (MRAP) vehicle program, the Defense Secretary's top acquisition priority. Building and maintaining the LCCE was challenging because the MRAP Program is unique from other urgent needs Acquisition Category I-D programs in two ways. First, to accommodate the aggressive and accelerated schedule, the MRAP Program conducted an open competition, which required a review of several designs from various contractors in a short period...2012Methods and Models II
Estimating Alternatives for Joint Future Theater Lift (JFTL)Robert Georgi, Bruce PurselUS military operations in Afghanistan and Iraq are making military transport aircraft work at rates not foreseen just a decade ago. At the same time the existing inventory of transport aircraft are limited in their ability to transport heavy cargo to austere unimproved landing zones where military forces routinely operate. The US Air Force (USAF) with considerable support from the US Army conducted the Joint Future Theater Lift (JFTL) Technology Study (JTS) to consider how best to equip its theater lift fleets beyond 2020 to address this need...2012Methods and Models II
Exploring Methods of Conflating Data from Various Data SourcesAshley MosesWhen creating a cost estimate, it is not out of the realm of possibilities that one might be required to combine data from multiple sources to get an accurate estimate. Coleman, Braxton, Druker, et al. have presented research on which methods might be the best to combine opinions given by subject matter experts (SMEs). Additionally, they have also discussed what adjustments are needed to make the estimates provided by SMEs more accurate as studies suggest that SMEs will tend to underestimate the range of possible data values when appropriate feedback is not given to them...2012Methods and Models II
Volatility and Cost EstimatingJennifer LeottaThis paper will examine the properties and uses of implied volatility, stochastic volatility, and historic realized volatility. Further discussion will focus on what applications an assessment of market volatility brings to the field of cost estimating through the application of a volatility range derived for fuel prices at varying intervals over the course of a generic program's life cycle.2012Methods and Models II
Estimating Relationship Development Spreadsheet and Unit-as-an-Independent Variable RegressionsRaymond P. Covert, Noah L. WrightMCR has constructed an estimating relationship (ER) development spreadsheet based on the zero percent bias, minimum percent error (ZMPE) regression technique to help with more credible and efficient development of cost improvement curves and cost estimating relationships (CERs). The CER-development method accommodates linear, power and triad functions with single and multiple technical and dummy independent variables. Furthermore, the ER development spreadsheet may be modified to accommodate other functional forms that may be of value in particular contexts...2012Parametrics
Are Parametric Techniques Relevant for Agile Development Projects?Arlene MinkiewiczAgile software development practices are predicated on the following tenets as introduced in 2001 in the Agile Manifesto [1] - Individuals and interactions over processes and tools - Working software over comprehensive documentation - Customer collaboration over contract negotiation - Responding to change over following a plan2012Parametrics
Comparing Bottom-Up and Top-Down Estimating Approaches in a Custom Cost ModelDonald L. Trapp, Noah L. Wright, Lisa HackbarthDeveloping a cost model involves several steps such as data collection, normalization, cost estimating relationship (CER) development, model building and documentation. Choosing an appropriate level of detail for each CER is an important consideration. This is influenced by several factors such as user needs, data availability as well as the statistical behavior of the data. The cost modeler must sometimes choose to estimate costs for particular work breakdown structure (WBS) elements with either a bottom-up or top-down approach...2012Parametrics
Error Analysis of a Custom Cost ModelRaymond P. Covert, Donald L. Trapp, Noah WrightHow do you know how well a custom cost model performs? If it is based on historical data, then the simplest performance measurement is the model's percent standard error (PSE). The PSE measures a historically-derived model's ability to re-predict its underlying database. In this presentation we will demonstrate an analytic error analysis of a custom cost model using historically-derived cost estimating relationships (CERs) and factors. We will provide a sample model and show how we calculated the PSE of each of the CERs from the data as well as developing the error calculations for each step in the model. We will show how we used the concepts of effective correlation coefficients, propagation of errors, and the calculated variances of the sums and differences of two random variables.2012Parametrics
The Function Point Based Pricing Model: The Price is Right!Daniel B FrenchMany IT organizations face a difficult challenge managing contractual engagements with vendors for application development and maintenance services. Regardless of the pricing model or acquisition strategy, projects come in late and over-budget more often than not. Be it a time and materials, cost plus, fixed price or other contractual arrangement, both the vendor and client face the challenge of providing the functionality desired, minimizing risk, and ensuring that the vendor is able to generate a reasonable profit from the engagement...2012Parametrics
Tactical Missile Bluebook Cost Model DevelopmentDonald L. Trapp, Noah L. Wright, Lisa HackbarthAs part of a quality review process, MCR constructed a cost model based on the cost estimating relationships (CERs) and cost factors contained in the Tactical Missile Bluebook. This effort proved to be valuable in refining the existing CERs and factors and in improving the Bluebook. While building the model, the CER developers found instances where CER input definitions in the documentation needed to be more specific to better serve CER users. When comparing candidate CERs to each other in the regression development process, CER developers relied on error statistics and goodness-of-fit metrics, but these metrics provided an incomplete comparison...2012Parametrics
Tactical Missile Bluebook and Cost Model OverviewRaymond P. Covert, Donald Trapp, Noah Wright, Andrew DrennonMCR recently developed the Tactical Missile Bluebook together with a cost model based on the information contained in it. The Bluebook contains cost estimating relationships (CERs) for tactical missile development and production based on 13 programs and 153 recurring production lots. In this presentation, we show how the Bluebook is organized and how the CERs and factors were developed, including the filtering methods used to determine data points to be included and excluded from CER development, the challenges of building a useful estimating method from this compilation of data, the methods used to develop the estimating relationships, the results obtained, and significant general findings...2012Parametrics
Understanding and Measuring the Impact of Design and Systems Engineering Decisions on AffordabilityZachary JasnoffMethodologies for understanding and measuring the impact of design and systems engineering decisions on affordability In today's data driven cost estimating environment, it is critical to understand the impact of design and systems engineering decisions on cost. It is also important to leverage actual cost data in developing data-driven Cost Estimating Relationships (CERs) that relate engineering design and performance parameters to cost. In addition, the ability to conduct sensitivity analysis on not only the CER, but on the entire system estimated is needed to understand the full impact on Measures of Effectiveness and Measures of Performance...2012Parametrics
Introducing Reliability and Availability Requirements into TOC ModelsEvin J. Stump, Wendy W. LeeTwo important system requirements are reliability and availability. Reliability is the probability of no disabling failures over a certain span of time, while availability is the ratio of the time the system will actually fulfill its operational expectations, to the total time it is expected to fulfill them. Both of these are often included in system specifications...2012Parametrics
Applying Parametric Cost Models as a Predictive Parameter for CMMI ComplianceDavid Bloom, Wanda Grant, Robert Wright, Gary BosworthThe Electronic Engineering Center (EEC) of Raytheon Space and Airborne Systems (SAS) recently breezed through a Capability Maturity Model Integration (CMMI) Level 5 assessment, jumping from an existing CMMI level 3 capability and without needing a separate CMMI Level 4 assessment in the process. Although this success was the result of a lot of hard work by many contributors, the bottom line is that there were very few changes in terms of business processes that were required to obtain a CMMI Level 5 organizational assessment, the hallmark of a learning and optimizing organization...2012Parametrics
Using Method of Moments in Schedule Risk AnalysisRaymond P. CovertA program schedule is a critical tool of program management. Program schedules based on discrete estimates of time lack the necessary information to provide a robust determination of schedule uncertainty and therefore the risk that the proposed schedule will be completed on time. To determine the risk that a proposed, discrete schedule will meet a particular schedule and to find the probable critical paths (i.e., the criticality index), a probabilistic schedule risk analysis (SRA) is performed. SRA is a process by which probability density functions (PDFs), usually triangular, are defined at the task level in an effort to quantify the uncertainty of each task element...2012Risk
Fitting Absolute Distributions to Limited DataBlake BoswellThe choice of probability distributions is a critical component for cost risk and uncertainty modeling. When data is available, distribution fitting techniques, such as Goodness of Fit (GoF) tests and Information Criteria (IC), can be applied to determine distributions that accurately describe potential cost realizations; however, with limited data GoF tests and IC based methods provide little or no insight into the best distribution choice. Therefore, when data is limited it is standard practice in cost risk and uncertainty modeling to solicit expert opinion in the construction of triangular distributions with vertices representing the best case, typical, and worst case scenarios...2012Risk
A Systematic Approach for Empirical Sensitivity Analysis on Monte Carlo ModelsMatt PitlykA key component of risk analysis is the sensitivity analysis performed on the input variables for Monte Carlo models with the goal of determining those variables that cause the most variation in the final distribution and identify the best candidates for risk mitigation plans. While the standard technique of calculating the correlation coefficient between the final distribution and each input distribution is appropriate for linear models, it is not sufficient to accurately identify the largest uncertainty drivers. In the case of the non-linear models it is neither appropriate nor accurate...2012Risk
SAR Data Analysis, CV Benchmarks, and the Updated NCCA S-Curve ToolRichard Lee, Peter Braxton, Kevin Cincotta, Brian Flynn, Ben BreauxTo support the development of better probabilistic cost estimates, the Naval Center for Cost Analysis (NCCA) has championed the development of the S-Curve Tool, which was well received at both the 44th Annual Department of Defense Cost Analysis Symposium (ADoDCAS) in February, 2011 (1), and the joint Society of Cost Estimating and Analysis (SCEA) / International Society of Parametric Analysts (ISPA) conference in June, 2011. This paper presents ongoing research to support both continued improvement of the S-Curve Tool and greater understanding of the nature of cost growth for major acquisition programs; its mean value (risk) and variability (uncertainty); and the components thereof...2012Risk
Enhanced Scenario-Based Method for Cost Risk Analysis: Theory, Application, and ImplementationBryan Flynn, Peter Braxton, Paul Garvey, Richard LeeThere is a growing realization within the cost-analysis community that estimates of cumulative probability distributions of cost, or S-curves, too often understate true, underlying risk and uncertainty. Several organizations cite cases where return program acquisition costs, or actuals, fall at the 99th+ percentile on S-curves estimated years previously. This degree of deviation from the mean is a legitimate possibility for any one acquisition program. After all, there's no such thing as an "average" program. Variation is expected...2012Risk
Real Data, Real UncertaintyAlfred Smith, Jeff McDowell, Lew Fichter, Wilson RosaCentral to any cost risk analysis and model are the uncertainty distributions assigned to point estimates. Ideally, the analyst will have a database of historical cost and technical information that can be used to objectively develop Cost Estimating Relationships (CERs) using approved statistical methods. From that analysis the analyst will be able to objectively define the shape and dispersion of the CER uncertainty distributions. Similarly, with a suitable database or expert opinions to draw upon, the analyst will be able to develop uncertainty distributions for the inputs to the CERs...2012Risk
Diagnosing the Top Level Coefficient of Variation: An Alternative ApproachDaniel J. AndelinThe coefficient of variation (CV), defined as the standard deviation divided by the mean, is a useful metric to quantify the uncertainty inherent in a probability distribution, as it provides a relative (and thereby more intuitive) measure of spread than the variance or standard deviation. In the field of cost analysis, the CV can be used as a diagnostic to determine the level of risk and uncertainty built into an estimate and whether those levels fall within the expected range for a program of a given scope and complexity and at a given stage in its life-cycle...2012Risk
Inflation Cost Risk Analysis to Reduce Risks in BudgetingMichael DeCarlo, Eric DrukerFor any project there is a danger of unanticipated cost growth because inflation rates are extremely difficult to estimate. This presents a significant challenge to estimators. Predicting future inflation rates with some precision is possible, however, when the appropriate analysis is implemented. Even with previous recommendations from a major federal and commercial consulting firm, there is evidence indicating that the government has not been making adequate assessments of inflation rates for future budgeting. Without applying proper attention and techniques to the analysis and prediction of inflation rates, budgets run a higher than necessary risk for increased cost growth due to inflation prediction error...2012Risk
Applying the Pareto Principle to Distribution Assignment in Cost Risk and Uncertainty AnalysisJames R. Glenn, Christian Smart, Hetal Patel, Lawrence JohnsonSignificant effort and statistical knowledge is required in order to complete an accurate uncertainty analysis of a cost estimate. First, a representative sampling of data points must be collected in order to model appropriate distributions for each random variable in the estimate. Additionally, correlation between the random variables in the estimate must be properly assessed and applied. Finally, to generate the cumulative distribution of total costs (i.e., S-Curve) either the joint cumulative distribution function must be computed or an approximation technique such as Monte-Carlo simulation must be used...2012Risk
The Unseen: Statistical Inference with Limited DataTrevor VanAttaObjective measurements of probability are often unavailable, and most significant choices under risk require an intuitive evaluation of probability.' -Nobel Laureates Daniel Kahneman and Amos Tversky. What are the odds of rolling a sum total of seven when tossing two dice? What is the probability of red turning up after a spin of a European roulette wheel? Most analysts, given a little time and a calculator, could answer these two questions with exact precision. For both of these questions, there is only one true correct answer. Such is the nature of probability analysis for questions that are decompositional (all possible outcomes can be determined), frequentistic (the experiment can be repeated an infinite number of times), and algorithmic (the results can be measured with numbers)...2012Risk
Utilizing Optimization Technique to Enhance Cost and Schedule Risk AnalysisColin Smith, Brandon HerzogAdvancements in Monte Carlo simulations have enabled cost and schedule uncertainty analysis to be conducted on large, complex programs which could never be analyzed using conventional methods. Analysts are now able to predict with increasing confidence the budgetary requirements, schedule forecast, and risk mitigation measures on projects at all points during their lifecycle. These advancements in Monte Carlo technologies and methodologies have further lowered the barrier to entry for conducting uncertainty analysis; have reduced simulation times to mere fractions of a second; and have allowed for a comprehensive, integrated cost, schedule and risk analysis...2012Risk
Is My Schedule Ready for Risk Analysis?Mario FountanoThe ultimate goal of schedule risk analysis is the ability to provide project stakeholders with a risk mitigation plan that allows the team to make informed decisions concerning future cost and schedule performance. However, in order to perform schedule risk analysis, a project schedule must first be valid, and provide a clear path to project completion. This study presents the steps our team takes in order to evaluate the readiness of project schedules prior to integrating a risk analysis. Often, project schedules contain flaws in duration and logic, are left un-statused, or were never developed to capture the true nature of project work. The assessment process our team follows helps to build more confidence and transparency into project work. 2012Risk
Joining Effort and Duration in a Probabilistic Method for Predicting Software Cost and ScheduleMichael A. RossThis paper describes a data-driven method for estimating the cost and schedule of software development projects. This method correlates the estimates of cost and schedule such that constraining (perhaps reducing) the cost will impact the estimated schedule and constraining (perhaps compressing) the schedule will impact the estimated cost. This method provides these estimates of cost and schedule that are probabilistic (i.e., provide a range of possible outcomes with associated probabilities of attainment); a capability that is essential to analyzing the impact that affordability and budget constraints have on program cost and schedule and their associated risks...2012Risk
A Closed-Form Solution for the Production-Break Retrograde MethodDarrell Hamilton, Brian GillespieThis article explores and discusses concepts surrounding the multi-step retrograde analysis process for learning curve production breaks that was popularized by George Anderlohr, in his 1969 Industrial Engineering article "What Production Breaks Cost". Mr. Anderlohr based much of his analysis using the cumulative average curve method, but the basic principles have been widely accepted and used to calculate the equivalent calculation using the unit theory learning curves...2012Software/Hardware
Estimation of Expedited Systems Engineering SchedulesBarry Boehm, Dan Ingold, JoAnn LaneA major objective of many organizations is to reduce the calendar time needed for a project to perform its systems engineering (SE) functions, without compromising the resulting product's needed functionality and quality attributes. This paper will present the derivation of a model to estimate the necessary calendar time to perform the project's SE functions as a function of its product drivers (size, domain familiarity), process drivers (maturity, streamlining), project drivers (team cohesion, collaboration technology support), people drivers (knowledge, skills, and agility), and risk drivers (vs. relative needs for speed, quality, and scalability)...2012Software/Hardware
COTS Estimating Metrics for Increased Cost AccuracyJoshua Patapow, Eric TiminskiObserved cost analysis issue was that multiple Commercial Off-The-Shelf (COTS) Hardware/Software cost estimates were significantly higher than recent contract award values. Using data from those contract awards, an analytical approach and metrics were developed to ensure increased accuracy on future estimates. For COTS Hardware (HW) and Software (SW), a metric of the actual contract price as a percentage of the mean online price/list price (Part I) was developed. Additionally, an estimating relationship was developed (Part II) to calculate the cost of annual maintenance support as a percentage of the actual HW/SW initial cost and the annual maintenance support actual contract price as a percentage of a vendor quote...2012Software/Hardware
A Standard Process for Software Code CountingBetsy Legg, Brian EnserSoftware Lines of Code or SLOC has long been one of the critical inputs for measuring the effort and time involved with software development projects. SLOC is by no means the only way to measure the size of a project but it is the most mature measure and is well understood by decision makers. Software cost estimating that relies on SLOC as an input depends on accurate and consistent counts. The problem is that no standard has been universally accepted for defining SLOC. Numerous code counting tools in use throughout the industry rely on interpretations of the definition of a line of code...2012Software/Hardware
Utilization of Visual Basic in Cost Estimating ToolsJeremy EdenAs collaborative computing environments become more prevalent in all industries, the cost estimating industry is no exception to this movement. Increasing amounts of pressure are put upon cost estimators to develop tools that are robust in design, have long standing methodologies, do not require proprietary software or licenses, and are and easy to use by both the advanced cost estimator looking for maximum control and the novice simply trying to diligently support the early stages of a development program...2012Software/Hardware
Cloud Computing - Changing the Way We 'Do' SoftwareArlene MinkiewiczIn 1961 at the MIT Centennial, John McCarthy opined "if computers of the kind I have advocated become the computers of the future, then computing may someday be organized as a public utility just as the telephone system is a public utility&. The computer utility could become the basis of a new and important industry."[1] In 2006, Amazon Web Services was launched providing computing on a utility basis. Since that time the notion of cloud computing has been emerging and evolving...2012Software/Hardware
Software Cost Estimating for Iterative and Incremental Development ProgramsBob Hunt, Jon Kilgore, Jennifer SwartzIterative and Incremental Software Development processes are at the heart of a cyclic software development process. Iterative and Incremental Software Development processes evolved in response to the weaknesses of the traditional waterfall model. Iterative and Incremental Software Development start with an initial planning and ends with deployment with the cyclic interactions in between. The goal is to deliverer what the user needs at the end of the process, not only what was envisioned at the beginning of the process. Iterative and incremental development is an essential part of the Rational Unified Process, Extreme Programming, and generally the various agile software development frameworks...2012Software/Hardware
Estimating for Lifecycle and Product Line AffordabilityJoAnn Lane, Barry Boehm, Ray Madachy, Supannika KoolmanojwongA significant challenge in systems engineering and acquisition is to justify investments in new systems and the evolution of existing systems and product lines. In an era of shrinking budgets, especially in the U.S. Department of Defense, choices are driven by affordability, and not just the affordability of the initial development. Affordability includes development, deployment, usage, maintenance, and retirement/disposal costs. A way to assess or estimate system or product line affordability is through Total Ownership Cost (TOC) analysis...2012Software/Hardware
Learning Curve Analysis with EZQuant - An OverviewMichael MahoneySure you learned how to calculate learning curves by hand while studying for the CCEA exam, but what will you use to calculate them from day to day? This session will provide an overview of the freely available DCAA EZQuant regression analysis shareware. EZQuant takes minutes to learn and is easy to use. Unit and Cumulative Average learning curve calculations and projections are supported. Since DCAA recognizes the software you don't spend time validating the tool. Natural logarithms are fascinating but EZQuant is just fast.2012Software/Hardware
The Importance of Software Cost Estimating Standards among a Diverse CommunityKyle Thomas, Jenny Woolley, William BlackMany cost analysis organizations within elements of the United States Intelligence Community (IC) have developed cost estimation and data collection methods that are based on years of Agency-specific historical programmatic data. These methods have been established over time, effectively creating independent approaches across the Community. While these techniques are valuable, the cost estimation and reconciliation process highlights differences in estimation methods that make analysis across the IC difficult...2012Software/Hardware
The EVP Certification: A Bridge Between SCEA and AACEMichael NosbischIn the 1990s, SCEA and AACE International, formerly known as the Association for the Advancement of Cost Engineering, not only were both members of the now defunct Joint Cost Management Association, but they also maintained a cooperative agreement that intended for members of the two associations to "benefit from increased knowledge of the activities and services available from the other organization." While that document agreement unfortunately did not extend beyond 1995, there has recently been a joint effort to renew a similar type of agreement...2012Informational Sessions
A Structured Approach to CCEA® CertificationMichael Mahoney, Rick Beavers (Presenter)The structured approach to SCEA certification details the methods used by Lockheed Martin Chelmsford to prepare employees for the CCEA examination. In 2010, three of us set out to earn certification at the CCEA level and all three of us passed on the first attempt. The session will introduce a generic schedule for studying the CEBoK modules including the time allotted for each, an approach to taking sample tests, and lessons learned. We'll discuss what worked and what didn't and share tips and tricks.2012Informational Sessions
Touchpoints between the SCEA Cost Estimating Body of Knowledge (CEBoK®) and the Program Management Institute (PMI) Body of Knowledge (PMBoK)Leon HalsteadThe General Accounting Office (GAO), in its publication of the "GAO Cost Estimating and Assessment Guide" in May of 2009, focuses extensively on the importance of Cost Estimation and its contribution to overall project success. While this publication offers specific technical examples of cost estimation and analysis capability, e.g. pre-Program approval cost estimates/baselines, Affordability Analysis, et al, there is less focus on how these technical examples "fit" into the larger Project Management context...2012Informational Sessions

Back To Top

";