Military Construction Cost Estimating (MM-1)
Nicole Barmettler – Cost Analyst, BCF Solutions
An informative presentation on construction cost estimating specifically dealing with military facilities. Within this topic, the author will identify, define, and explain the cost methodologies and cost adjustment factors considered when developing construction cost estimates for general military facilities. Project costs will be exemplified by illustrating a breakdown and walkthrough of the process. The author will specifically discuss the process involved in a five year facility acquisition timeline that is usually required for a typical major military construction effort, which is defined by a project cost exceeding $750,000.
Keywords: MILCON, Military Construction, Facility Acquisition, Timeline, Design
Cost and Performance Trades and Cost-Benefits Analysis (MM-2)
Steven Ikeler – Operations Research Analyst, United States Army
The results from a Cost, Performance and Schedule Trades Analysis are extremely useful when performing Cost-Benefits Analysis (C-BA). A Trades Analysis involves direct participation from all stakeholders and compares the effects of different performance and schedule goals on cost and risk. Trades Analysis products are useful in all cost estimates, not only C-BAs. The Army C-BA is discussed since it provides simple examples. The Army C-BA identifies the most cost-effective solutions from among different alternatives. Every C-BA also includes a cost risk analysis. We will discuss the multiple ways that Trades Analysis enhances the C-BA. For example, it results in new C-BA alternatives that enhance the effectiveness of the C-BA. It also provides essential cost risk and sensitivity insights.
This paper will discuss the basics of Trades Analysis and the C-BA. It will include what the cost analyst should do to prepare and what information to collect during the Trades Analysis. One observation is that non-traditional Work-Breakdown Structures need to be considered and the analyst should model potential second and third order effects beforehand. Another key observation is many participants translate risk to cost on their own with differing results. As a result, the analyst needs to be prepared to explain both cost and cost risk implications and second and third order effects. Ideally the analyst should understand the rationale behind the performance and schedule objectives.
We will discuss an example with obvious second order effects by including a high priority weight limit. The example will demonstrate a cost estimate model that facilitates the Trades Analysis. We will discuss recommendations for conducting the Trades Analysis. The example will explain the results of the Trades Analysis. Finally, we demonstrate how the results inform the C-BA through new alternatives, risk and cost risk information and sensitivity analysis.
Keywords: C-BA, Cost-Benefit, Trades, Tradespace
Lessons Learned from Estimating Non-Standard Equipment Returning from Overseas Operations (MM-3)
Michael Metcalf – Cost Analyst, Technomics, Inc.
Throughout the execution of Operation Iraqi Freedom (OIF) and Operation Enduring Freedom (OEF), the Department of Defense (DoD) purchased myriad items to support the warfighter through non-traditional acquisition processes in order to provide them quickly to theater. These items provided critical capabilities, advantages, and safety improvements that helped reduce casualties. Now that these overseas operations are ending, the DoD must determine what equipment should be retained and how to transition its operations and support (O&S) to peacetime processes. Overseas Contingency Operations (OCO) supplementary funding is expiring, and most actions taken will required funding from the DoD base budget going forward. This paper examines this critical issue.
Our discussion focuses on the return of two families of items to the United States Army (USA): a set of small ground robots under the purview of the Robotics Systems Joint Program Office (RSJPO), and the family of Mine Resistant Ambush Protected (MRAP) troop vehicles. All of these items were purchased as Non-Standard Equipment (NSE) from a variety of contractors, either Commercial Off-the-Shelf (COTS) or using rapid wartime development, and were often maintained during wartime using Contractor Logistics Support (CLS) and program office support. USA desires to preserve the capabilities provided by the small robots and MRAPs, for use in potential future conflicts as well as in current peacetime operations. However, both programs must operate in new affordability and budget environments and establish traditional acquisition and sustainment processes.
This paper explores challenges in estimating the cost of retaining this equipment. Items that present unique challenges include: the type classification and full materiel release processes; repair, reset, and upgrade; short- and long-term storage; and knowledge retention and loss of wartime experience. We will also explore funding challenges; moving targets in number and configuration of retained systems; the transition of O&S from wartime contractor-based to peacetime organic; training; and system disposal and divestiture.
Keywords: Non-Standard Equipment, NSE, Commercial Off the Shelf, COTS, MRAP, rapid procurement, contractor support, sustainment, reset, training
Weibull Analysis Method (MM-4)
Erik Burgess – President, Burgess Consulting Inc.
James Smirnoff – Wyle
Brianne Wong – Consultant, Booz Allen Hamilton
The NRO Cost and Acquisition Assessment Group (CAAG) develops time-phased estimates for space systems in support of milestone decisions, budget formulation, and other program assessment tasks. CAAG relies on parametric budget-spreading models based on historical data to provide objective analysis and credible information before contract award or early in a program when there is little or no execution history available. However, in today’s environment of increased oversight and budget scrutiny, programs are evaluated annually in hopes of finding excess margin and balancing budget risk across the portfolio. The Weibull Analysis Method (WAM) is an improved approach that estimates budget requirements for programs that are already underway by using their actual execution history and by focusing on accuracy in the near years. WAM builds on work previously published by LMI and the Center for Naval Analysis, and is tailored to the satellite acquisition and contracting practices at the NRO, and validated using historical data from 37 completed satellite contracts. This briefing describes the analytical basis for WAM, how its accuracy metric was developed from the historical data, and how it is applied in program assessments. A spreadsheet tool that implements WAM, computes accuracy metrics in each year, and compares the results to a parametric budget-spreading model is also presented.
Study of Cost Estimating Methodology of Engineering Development (MM-5)
Myung-Yul Lee – Estimator, The Boeing Company
Introduction: In July 22, 1982, Initial Full Scale Engineering Development (FSED) contract was awarded to McDonnell Douglas. Along with C-17 Production contract, the Boeing Company had initiated a Producibility Enhancement/Performance Improvement (PE/PI) program and Globemaster III Integrated Sustainment Program (GISP).
The PE/PI effort incorporates new design, modifies the aircraft systems, and updates new technology for the aircraft. The PE/PI projects, therefore, operate separately from the production of aircraft. The outcomes of PE/PI projects applied to the C-17 aircraft are incorporated into the production. In this paper FSED and PE/PI will examine in view of estimating methodology because these programs are involved in engineering development environment rather than Production and sustaining engineering (GSP).
FSED Program in View of Estimating Methodology: The parametric estimating method is one of the most desirable and is a highly creditable estimating method because this method applies same project which was performed or similar project actual hours to estimate the weapon system development cost. For that reason, when a new weapon system requires cost estimate, a discrete estimate method is utilized which is based on the engineering requirement for the new weapon system.
PE/PI Project in View of Estimating Methodology: Along with C-17 Production contract, the Boeing Company had initiated a Producibility Enhancement/Performance Improvement (PE/PI) program. Producibility Enhancements (PE) are efforts used to correct obsolescence and safety/operational deficiencies. Performance Improvements (PI) are efforts used for capability and supportability to increase the C-17 Weapon System performance, or support capability, or decrease the C-17 costs of ownership. The PE/PI effort incorporates new design, modifies the aircraft systems, and updates new technology for the aircraft.
Purpose of the Study: The purpose of this paper is to study the actual engineering labor hours for FSED Program and PE/PI project and to create a parametric estimating model in a weapon system, especially cargo aircraft. The C-17 final assembly facility in Long Beach, California closes in 2015. It is worthwhile to review historical data for the C-17 FSED program and PE/PI project, both programs have engineering development program in view of estimating methodology research.
Validation and Improvement of the Rayleigh Curve Method (MM-6)
Michael Mender – Cost Analyst, Naval Center for Cost Analysis
The Rayleigh Curve is a popular method for estimating both software development and R&D project durations and costs by extrapolating from earned value data available from the early periods of a project. Many of the existing studies supporting the Rayleigh method rely on R-squared as a measure of goodness-of-fit. Because the Rayleigh function is non-linear and not obviously linearized, most tools implementing the method rely on numerical means to fit a curve without linearizing. Because R-squared requires linearity to hold meaning with regard to goodness-of-fit, we questioned the validity of the existing support for the Rayleigh method and sought to conduct our own series of tests. Additionally, we had some concerns about the “freshness” of the data used to validate the method, as many of the original studies were done decades ago and/or used data from projects of a similar vintage. As we had access to data for more recent projects, we additionally sought to test whether the Rayleigh method was still applicable to modern projects.
While doing our research, we uncovered a means by which the Rayleigh function could be linearized, thereby allowing the use of standard linear regression methods. For ease of use, we have taken to calling this method “Rayleigh Regression.” We built a tool using this method, and then proceeded to evaluate its performance over a data set consisting of completed R&D projects. For each project, we generate a series of fitted curves, starting with the first three data points, then iteratively expanding our data set to include successive data points (i.e., the first four, the first five, etc?), through the end of the contract. By doing this, we are able to evaluate the quality of the Rayleigh predictions as more data becomes available to the estimator. We tested the Rayleigh Regression results both by evaluating the fit of the curve within the bounds of each sample, and by evaluating the accuracy of the forecasted cost and schedule with the actuals from each data set.
The presentation begins with a brief overview of the Rayleigh method, followed by an explanation of R-squared and the issues with non-linear functions. We then discuss the method for linearizing the Rayleigh function and then using it for linear regression. We then provide a brief demo of the tool we developed and then move on to a discussion of our results pertaining to the performance of the Rayleigh method. Ultimately, we found that the Rayleigh method is reasonably effective, but that the support some studies have found does not appear to be as certain when the issues pertaining to R-squared are addressed and the method is tested against more current data.
Keywords: Optimization, EVM Analysis, Linear Regression
Rotorcraft Cost Model Enhancements for Future Concepts (MM-7)
F. Gurney Thompson III – Cost Research Analyst, PRICE Systems
The Future Vertical Lift (FVL) program will develop a family of rotorcraft intended to meet the future needs of the U.S. Armed Forces. The precursor programs to develop the needed science and technology improvements are already underway. Much of the U.S. Armed Forces’ fleet of helicopters and rotorcraft are approaching a point where decisions must be made to extend their life, retire, or replace the aircraft. As these decision points approach and new technologies are being created, enhancements are needed to the toolset for future rotorcraft cost estimation.
This paper will discuss the ongoing research efforts to improve upon existing rotorcraft cost estimation capabilities, from both a cost estimating relationship (CER) update and a model development perspective. We will share our approach, many of our findings, and any lessons learned. Efforts currently underway include data collection, updates to existing cost models and their CERs, adding support for new aircraft types and technologies, and the addition of new analysis capabilities to better understand total ownership cost.
Kill Vehicle Work Breakdown Structure (MM-8)
Jennifer Tarin – Operations Research Analyst, Missile Defense Agency
Christian Smart – Director of Cost Estimating and Analysis, Missile Defense Agency
Paul Tetrault – Technical Director, Missile Defense Agency
This paper provides an alternative to Appendix C: Missile Systems for inclusion in MIL-STD-881C, the Department of Defense standard for Work Breakdown Structures (WBSs). The Missile Defense Agency (MDA) produces interceptors that are similar to missiles with the exception of the payload element. Where Appendix C defines the payload element with a limited set of WBS elements, the MDA interceptor payload, referred to as a kill vehicle, includes a large collection of significant WBS elements. A kill vehicle is a guided weapon that utilizes hit-to-kill technology after separation from a boosting vehicle. Often described as “hitting a bullet with a bullet” its purpose is the destruction of a ballistic missile threat and/or a threat re-entry vehicle. MDA’s kill vehicles do not contain any explosives; instead the kill vehicles use kinetic energy from the engagement velocities to provide the destructive forces. Additionally, MDA kill vehicles operate autonomously as short-lived space vehicles. Based on the number of significant WBS elements for MDA kill vehicles, we determined that the current MIL-STD-881C Appendix C Missile Systems Payload WBS is insufficient. An analysis of MDA’s currently produced kill vehicles; Ground-Based Midcourse Defense “Exo-atmospheric Kill Vehicle,” Aegis Ballistic Missile Defense “Kinetic Warhead,” and Terminal High Altitude Area Defense “Kill Vehicle”, was done to establish commonality. As a result, we created three alternatives based on Appendix F Space Systems WBS and Appendix C Missile Systems WBS from MIL-STD-881C. The proposed KV WBS, a hybrid of Appendix F and Appendix C, will support existing and future kill vehicle designs.
Meet the Overlapping Coefficient: A Measure for Elevator Speeches (MM-9)
Brent Larson – Senior Cost Analyst, Infinity Systems Engineering
You’ve seen this picture before. . . a plot of two overlapping probability distributions. You may have created one with an overlay chart. Typically this graphic contrasts two cost distributions so as to illustrate similarity, difference or change. However, seldom seen is a number that quantifies the overlap or area shared by both distributions. This area common to both densities is known as the Overlapping Coefficient (OVL) and is an intuitive, unitless, measure of agreement ranging from zero to unity. The literature reveals that obtaining the OVL from data, such as the output of a monte carlo simulation, can be non-trivial. It typically requires further assumptions and intermediate steps that come with distribution fitting or kernel density estimation. Consideration of large sample theory and some calculus along with exploitation of an existing statistical test, suggest a reasonable approximation of the OVL may be made readily accessible to analysts. This presentation will introduce the OVL, its historical background and applications. It will then demonstrate how to obtain the measure, derived from the totality of both distributions, that quantifies their similarity. Let’s say a year has elapsed between an estimate and its major update. Methods have changed. Data have changed. Tools have changed. Some element uncertainties have been reduced. Some have increased. In the few moments between the 12th and 14th floors you will be able to specifically state the convolution of all that change.
Keywords: Overlapping Coefficient, Empirical Cumulative Distribution Function, S-Curve, KS Two Sample Test, R
Excel Based Cost Roll Up Method (MM-10)
Matthew Leezer – Senior Cost Analyst, Honda Aircraft Company
Corporations are managing the cost of new product earlier in the development cycle, creating a need for timely and accurately product cost roll up of complex systems. These systems can consist of hundreds or thousands of unique parts, beyond the capability of manual management. Many large organizations manage new product development using engineering Product Life Cycle Management (PLM) software; these organizations often utilize Enterprise Resource Planning (ERP) software to manage the cost of production products. As these companies move to manage the cost of new product earlier in the development cycle there is a need to merge the data that exists in these two systems into a meaningful and reliable cost management tool. There are many companies developing solutions to this problem that offer a more robust management capability. The scope of this paper is to detail a Microsoft Excel based method that can quickly and accurately roll up the engineering bill of material costs. This method can also be used to roll up the weight of the system, to validate the results.
This paper will show the user how to create a custom function (sumlowerlevel) using Visual Basic and apply this function in Excel to generate a report that will save time and increase the accuracy of estimates. The method uses the lookup function to capture the cost of purchased components and subsystems and uses the custom function for all make assemblies to roll up the cost of the purchased parts. The end product is a sortable and searchable bill of materials with accurate material cost roll ups at all levels of the BOM. This method can be used to create a costed bill of materials in hours instead of months. This method can be applied to simple assemblies, complex assemblies and very complex assemblies of assemblies.
Once mastered this method can be used to roll up the weight of an assembly, and be used to validate that all parts in the BOM have been accounted for. This method can also be used to roll up the assembly labor hours and result in a total product cost. A comparison of the weight and part count to product description will provide the user with a level of confidence in the roll up.
This report will show two unique methods for rolling up product cost in Excel using data available in PLM systems and ERP databases into a simple excel structure that can be built in minimal time and with minimal effort, but provide a high level of accuracy and efficiency. The results of the two methods can be compared and used to validate the results. Each method has benefits and issues that will be identified.
The Role of Cost Estimating in Source Selection (MM-11)
Annette Barlia – Cobec Consulting, Inc.
One of the most interesting purposes a cost estimate fulfills consists of using it to evaluate vendor proposals. From the development of an Independent Government Cost Estimate (IGCE) to Source Selection, many players come together to ensure the vendor selected will meet program acquisition goals. Numerous decisions must be made: 1) When should the cost team begin to interface with the technical team? 2) How should the cost model be organized, in accordance with the Statement of Work (SOW) or CLIN structure? 3) What level of detail is required, estimating down to the CDRL level? 4) Why should the cost model correlate to the price evaluation tool and the program schedule? This paper will answer such questions.
The analysis will focus on the process of developing an IGCE and then utilizing it to evaluate vendor proposals for the acquisition of new technology. It will demonstrate that a strong IGCE facilitates source selection. If the cost estimate is developed the right way, organizations will have more leverage during contract negotiations with vendors, and the acquisition will run smoothly and meet program goals. Both junior cost estimators looking to understand real-world applications of estimating as well as government program office personnel facing an acquisition will benefit from this analysis.
Automated Costing to Identify Opportunities to Reduce Direct Material Spend (MM-13)
Julie Driscoll – Vice President of Marketing Strategy & Product Management, aPriori, Dresser Rand
Dale Martin – Senior Manager Cost Estimating, Dresser-Rand
This session will cover how technology is automating the costing process through integrating costing solutions with CAD, PLM and ERP; pulling information about components directly from CAD files; and using an intelligent cost engine that evaluates manufacturing routings for feasibility and cost effectiveness. We will look how these solutions enable automated batch costing of components and are used by manufacturers to support cost reduction projects. As part of the presentation, we will review a spend analytics methodology that combines process and technology to identify, analyze and take action on potential outliers for cost reduction, driving significant results in short-time frames. We will also examine a case study of a leading manufacturer where the solution was applied to a program that needed to be cost reduced prior to launch. Wrapping up the presentation, we’ll hear from a cost engineer who uses the solution in his organization who will provide a comparison of automated and traditional costing approaches and a summary of best practice tips.