NPS and AFIT Masters Degree in Cost Estimating and Analysis (BA-1)
Daniel Nussbaum – Visiting Professor, Naval Postgraduate School
Greg Mislick – Lecturer, Naval Postgraduate School
This presentation provides an update on the Joint, all-Distance Learning Masters Degree in Cost Estimating and Analysis offered at the Naval Postgraduate School (NPS) and Air Force Institute of Technology (AFIT).
The Navy and the Air Force are collaborating to meet a need for a distance learning educational program by providing a Master’s Degree in Cost Estimating and Analysis. The two schools have designed and developed the content, have graduated two cohorts, and recently started their fourth cohort. Further information is available at http://www.nps.edu/DL/Degree_Progs/MCEA.asp
The presentation will incorporate details and requirements about the program, achievements to date, research undertaken by current students, and lessons learned from our experience so far in this innovative program.
A Comprehensive CES and BCA Approach for Lifelong Learning (BA-3)
Kevin Cincotta – Technical Director, ICF International
Darcy Lilley – Chief Learning Officer, Air Force Air Mobility Command
The Air Force Air Mobility Command (AMC) Enterprise Learning Office (ELO) mission is to transform AMC into a premier Air Force learning organization, achieve learning through optimum approaches and develop Mobility Airmen into life-long learners who demonstrate institutional Air Force competencies with a positive approach to managing their own learning. In this context, learning has three main components: training, education, and experience. The re-engineering of learning to develop and deploy optimum approaches focuses on all components. AMC ELO is initially focusing on training.
Training is generally represented as only one line within a cost estimate. At most, there are cost elements for Initial Training (as Investment) and Recurring Training (as Opearations and Support). Very little cost research has been published regarding training as the service to be procured. Leveraging prior work done by the North Atlantic Treaty Organization (NATO), the Office of the Secretary of Defense (OSD) Program Analysis and Evaluation (PA&E) Economic Analysis (EA) Guide, and the Department of Homeland Security (DHS) Information Technology (IT) Life Cycle Cost (LCC) Work Breakdown Structure (WBS), we developed a first-of-its-kind comprehensive Cost Element Structure (CES) for training. The CES decomposes training into its core elements, broken out according to 1.0 Investment, 2.0 Operating and Support, and 3.0 Decommissioning. Important distinctions are made between among training hardware and software, and those pieces of hardware and software that are ancillary to core training efforts. Training is a complex endeavor that includes elements of both IT and non-IT acquisitions, so that a comprehensive, mutually exclusive, and completely exhaustive CES?with an accompanying data dictionary?is essential.
Finally, and perhaps counter-intuitively, training (and more generally, learning) is an abstract concept whose outputs are generally not “defense materiel items.” Therefore, the structure is properly service, not product-oriented, and several principles of MIL-STD-881-C do not directly apply.
AMC ELO is using this CES to evaluate several alternatives with respect to re-engineering training of airmen, and their general learning experience, in the context of a business case analysis (BCA).
This paper presents the training CES, conveys its value in the broader context of transforming learning, and outlines an approach for using the CES in the context of a BCA. Finally, preliminary results of the BCA are presented and interpreted.
BOE Development: Scope Evaluation and Criteria (BA-4)
Michael Butterworth – West Coast Cost Analysis Lead, TASC Inc.
Demetrius Prado – Cost Analyst, TASC Inc.
Current basis-of-estimates (BOEs) do not identify succinctly, how the scope of the estimate was derived and how actuals used for build-up, analogous and parametric methodologies fit the technology and product under consideration. We believe that one of the many problems leading to poor BOE development revolves around the lack of identifying the scope of work and understanding the effort needed in particular skill mix to perform the task.
When coming up with a solution to this problem, we must ask ourselves, “How easy is it for evaluators to determine price realism vs. price reasonableness in justifying the estimate?”
We focus on establishing grading criteria for 3 categories during BOE development:
-Hardware
-Software
-Services
The contracting, pricing and cost estimating communities are being challenged with developing defensible Basis of Estimates generated during proposal Task Assessment creation. The Basis of Estimate provides the cost estimating rationale and resulting cost estimate for a specific cost element. They should be prepared by the most technically knowledgeable and skilled at the specific task. These estimates should be subject to an independent review to improve win probability by increasing cost credibility and transparency.
While developing BOE grading criteria, we take into consideration other problems we normally face during estimate creation:
-Grading subjectively vs. objectively
-Lack of documentation
-Time consuming (time limitations)
-Lack of Subject Matter Experts to write Task Assessments
This paper presents a process that solves this problem – a process where someone can create valid, credible and transparent Basis of Estimates by means of 1) understanding the criteria by which BOEs are evaluated (grading scale), 2) providing the criteria by which you identify the scope of the estimate that is being built, and 3) assuring objective criteria to evaluate and grade the Basis of Estimate are used in an independent review. In addition, we will tailor the grading criteria to allow the BOE writer to easily identify the scope of the task assessment and BOE at varying levels, even if the proposal team is faced with limited knowledge and resources.
Long Term Affordability through Knowledge Based Bid & Proposal (BA-5)
Zachary Jasnoff – Vice President, Professional Services, PRICE Systems LLC
According to a 2013 GAO report, “positive acquisition outcomes require the use of a knowledge-based approach to product development that demonstrates high levels of knowledge before significant commitments are made. In essence, knowledge supplants risk over time.” Often times, acquisition proposals that are not knowledge-based introduce significant risk and while seeming reasonable in the short term, cannot sustain long term affordability. These proposals are often based on grassroots engineering judgment and subjective subject matter expert opinion and may be overly optimistic. This paper will present through a recent study with a major defense corporation, the evolution to knowledge based bid and proposal demonstrating best practices using sophisticated toolsets.
Evolving toward long term affordability starts with credible cost estimating at the bid and proposal stage of a project. As the project evolves, it is critical to capture historical information over the course of the program to support a knowledge based bid and proposal system that is accurate and credible for predicating future outcomes. At the heart of knowledge based bid and proposal system is the ability to collect, normalize, analyze data from prior programs. According to the GAO, the building of this knowledge occurs a) when resources match requirements; b) the product design is stable and c) manufacturing processes are mature. In addition, it is critical that any knowledge based bid & proposal system includes both a top down (parametric) and bottoms up (detailed build up) crosswalk comparison and validation of estimates. A crosswalk helps ensure that all programmatic areas are estimated and discrepancies resolved. When taken together, knowledge-based bid and proposals provides a sound foundation for long term affordability while balancing price-to-win and market considerations.
To support long term affordability, estimators require new and sophisticated toolsets to capture and analyze myriads of information to both produce and implement actionable knowledge. These tools provide the ability to quickly capture and normalize large amounts of data, and produce knowledge based findings that lay the foundation for long term affordability.
What Happens to a Cost Estimate When It’s Done? (BA-6)
William Barfield – Executive Cost Estimator Associate, Quantech Services, Inc.
David Bach – Director of Business Analytics, Quantech Services, Inc.
What happens to a cost estimate when it is done, or “finished?” Cost estimates are used to support a wide variety of financial decisions. When estimates are done and the decision has been made, is the estimate still useful after the decision, or does it become “shelf-ware?” We surveyed the international cost community to determine how we develop, document, use, and archive our various kinds of cost estimates. We analyze what happens to completed cost estimates by describing the types that are typically developed, the types of decisions they support, who makes these estimates, the various storage methods, and the (potential) reuse of the cost information. We describe and compile some real-life observations of practitioners and compare them to various cost guidelines (e.g., GAO, DoD, FAA, MOD, CEBoK) that stipulate the maintenance, archival, and disposition recommendations for cost estimates. Survey summary statistics are reviewed and we conclude that a well prepared, documented, and stored cost estimate can be the difference between wasted effort and informed decision making.
Update on The Cost FACTS (Factors, Analogies, CER’s & Tools/Studies) Group – Enhancing KM by leveraging Enterprise Social Networking (BA-8)
Daniel Harper – MITRE Corporation
Ruth Dorr – MITRE Corporation
Cost FACTS is a community-driven initiative to bring together the cost estimating community across MITRE, Government agencies, the contractor community, and academia to share reusable, non-restricted cost information and to encourage dialogue throughout the worldwide cost community. Cost FACTS provides an easily accessible repository of reusable cost estimating FACTS, i.e., Factors, Analogies, Cost Estimating Relationships (CERs), and Tools/Studies. These FACTS may have been originally gathered as the basis for a cost estimate in one agency, but have applicability across many agencies. Cost FACTS members within the MITRE community have also been working to gather basic, non-restricted information from completed cost estimates to build a repository of cost estimating structures factors for use as a basis for future estimates. Since its inception 18 months ago, Cost FACTS has increased its membership exponentially, reaching across all centers within MITRE, government agencies supported by MITRE, as well as others in the industry supporting those agencies. Cost FACTS is currently being recommended in several Government agencies as a tool to assist the cost estimating community and improve cost estimates (e.g., Commerce, DHS, CMS). The Nominees have put over 400 hours into Cost FACTS in FY 2013 alone, gathering “the FACTS,” managing the Handshake site itself, presenting the idea internally within MITRE as well as externally at cost estimating group meeting and professional and society conferences. This is an independent, MITRE employee-initiated effort that has been partially funded through the use of department developmental funds, with a significant amount of work being conducted on an unfunded basis
Space Shuttle Cost Analysis: A Success Story? (BA-9)
Humboldt Mandell – Research Fellow, The University of Texas Center for Space Research
In the aftermath of the highly successful Apollo lunar program, NASA struggled for a few years to find a meaningful program which would satisfy long range national space strategies, as well as reflecting the realities of the rapidly changing political environment in the nation. The Space Shuttle emerged from the need to lower the costs of orbital cargo delivery for construction of a space station and enabling Mars exploration, but also was highly constrained by DOD requirements for cargo mass, cargo size, and once-around cross range.
A key part of the program initiation process was to find a vehicle which would fit the ever-changing budget profiles that were emerging from the Nixon Administration Office of Management and Budget, not only for total cost, but also peak annual funding.
Early in his presidency, Nixon appointed a Space Task Group, chaired by Vice President Spiro Agnew. This body made recommendations for both budgets and program timing for a Space Shuttle, a Space Station, and in some options, a human Mars mission. Nixon chose the single program, the Shuttle.
Cost analysts struggled with (at first primitive) methods to follow the rapidly-changing budgetary constraints and technical configurations, to help the design force to find solutions to the cost and peak funding constraints. In one summer over 40 configurations were costed, forcing rapid changes in the sophistication of the cost estimation techniques.
This paper tells that story, and describes the major success of meeting the original cost and peak funding commitments, and why the success was attained. It also admits to the failure of the estimates for operations costs, and presents the reasons for the failure.