2014 Workshop Cost Management Papers

A Balanced Approach to Meeting Fiscal Constraints (MG-1)

Steve Green – Department Head, Department of Management, United States Air Force Academy
Kevin Davis – Associate Professor and Director of Assessment, United States Air Force Academy
Kurt Heppard – Professor of Management and Deputy for Strategic Planning, United States Air Force Academy

The effective and systematic use of cost and budgeting information is a critical component of strategic planning and decision making in most organizations. The Department of Defense’s (DoD) current operational environment, scarce resources, and conflicting stakeholder expectations are resulting in extreme fiscal constraints. The result is the need to reconsider missions and goals, reassess priorities, entertain force structure alternatives, and ultimately reduce budgets. Linking this cost and budgeting information with financial goals, as well as goals for customer satisfaction, internal processes, and organizational growth and learning is a daunting challenge for many organizational leaders. In many cases, including various DoD organizations, managers have adopted a Balanced Scorecard approach and methodology focused on linking organizational goals and objectives with benchmarks and metrics that are based on cost and budget estimates as well as actual expenditures. This cost and performance driven strategic framework allows organizational leaders and strategic planners to systematically consider and envision cost effective ways to meet the overall strategic goals of the organization including the tough decisions associated with budget cuts. Similar to the DoD, universities and other institutions of higher learning are facing the same increasing pressure to meet the strategic expectations of their many constituencies while effectively managing costs. Since current spending levels may be unsustainable for many universities and institutions, the careful analysis and estimation of costs is of particular importance to strategic planners and decision makers. This paper reviews the current literature as it relates to cost-based strategic planning and strategy mapping at universities and institutions of higher learning. Applying concepts from the Balanced Scorecard approach to cost analysis and budgeting, this paper offers effective cost metrics and performance measures for the critical implementation of strategy maps which link many of the intangible measures of higher education effectiveness with tangible cost budget, and performance measures. The paper develops an integrated and aligned system of strategic goals, performance metrics, and cost parameters based on cost and performance data already collected at many universities and institutions of higher learning. We feel there may be lessons learned for DoD organizations.

Cost Overruns and Cost Growth: A Three Decades Old Cost Performance Issue within DoD’s Acquisition Environment (MG-2)

Leone Young

For the last three decades, the US Department of Defense (DoD) has been encountering program performance issues such as inaccurate and unrealistic estimations for its acquisition programs, and its effort of eliminating cost overruns and cost growth phenomenon has been deemed ineffective as well. This paper examines and consolidates multiple government reports and studies, which pertain to the cost performance of DoD that has been viewed as unsatisfactory and problematic. The commonly accepted cost estimation methodologies and insightful processes and techniques initiated by NASA, RAND and USAF are also presented and analyzed. The research finds that he reliability and accuracy of any cost estimation methods depend heavily on rigid and mature systems designs and low fluctuations of requirement changes, which can only be accomplished through disciplined systems engineering practices. Performing multiple appropriate cost estimations in advance during the pre-Milestone A phase, particularly Engineering Builds method as knowledgeable data on systems designs and requirements become available, can offer a vast amount of benefits of cost savings as well as reduce uncertainty and risk reductions for systems programs. The research results summarize the overall strengths and weaknesses of the current state of the DOD acquisition programs, which is beneficial to the professionals in the defense industry, particularly cost estimators, program and project managers whose programs are constantly faced with financial challenges.

Supplier Cost/Price Analyses – Best Practices for evaluating supplier proposals and quotes (MG-3)

David Eck – Director, Dixon Hughes Goodman
Mike Mardesich

During proposal planning, preparation, and review, an important but often over looked aspect is the evaluation of a supplier’s proposal or quote. Requirements in the Federal Acquisition Regulations (FAR) provide for certain visibility into supplier’s proposals and quotes, depending on the value of the proposal/quote. Additionally it is critical to note the FAR puts the onus of supplier cost (or price) analysis on the prime contractor. The expectation is that the same level of detail that the prime includes in its proposal should be included as support to a supplier’s proposal. Effectively the prime contractor must perform supplier cost/price analysis in accordance with the FAR. The actual scope of the prime contractors review will depend on each particular situation such as type of product or service, dollar value, purchase method, and subcontract type.

Material and subcontract costs are a significant cost element in most proposals. Estimating professionals are challenged to evaluate supplier proposals and quotes and include the best estimate of the supplier prices in their proposals. Inadequate analyses of supplier quotes and proposals may result in your proposal being returned to you by the government, and may result in your company losing a potential contract. This presentation will cover best practices in obtaining adequate information from the suppliers and techniques for evaluating that information. We will present:
• Techniques for obtaining adequate information and proposals from the supplier for evaluation
• Documentation required to support acquisition of commercial items and competitive awards
• Guidelines for evaluating supplier quotes and proposals
• Examples of working papers to document the evaluations of the suppliers’ quotes and proposals
• Best practices for evaluating supplier direct labor, material, forward pricing rates, and other costs
• Techniques for evaluating the technical aspects of supplier information such as direct labor hours and bill of material quantities
• Examples of analytical procedures to use in evaluating supplier information, including sampling techniques, learning curves, and regression analyses
• Examples of information to include in Cost Estimating Manuals

Your presenters have performed supplier cost/price analyses for many years – both with the government and in industry. If you would like to attend a discussion where hands-on experienced presenters communicate real time examples and current best practices to fit your needs, plan on attending.

Innovative Business Agreements and Related Cost & Pricing Methods at NASA in Support of New Commercial Programs (MG-4)

Torrance Lambing – Lead in the Business and Cost Analysis Office, John F. Kennedy Space Center, NASA
James Roberts – Senior Cost Analyst, NASA Kennedy Space Center

On April 15, 2010 President Obama delivered a speech at Kennedy Space Center in which he outlined his new vision for the U.S. space program. Emphasis was placed on enabling the exploration of Space by Commercial entities instead of by the Government. Since that time, NASA’s role has changed in many instances from being a program manager – overseeing development of space launch hardware and conducting space exploration missions – to one of support and a provider of space-related facilities and infrastructure. Especially, since the ending of the Space Shuttle Program in mid-2011, many of NASA’s high-value facilities, which were designed to support Shuttle launches and landings, have become underutilized, and the need was recognized to find alternative uses that support Commercial Space activities.

At the 2011 ISPA/SCEA National Conference, during a time in which NASA was struggling with change to some of its old methods of doing business, the authors presented a discussion of the pricing methods available under the various legislative and legal authorities that allow NASA to enter into Agreements, and related cost estimating considerations. Since that time, NASA has updated and redefined some of its policies and procedures on pricing, as well as its methods of estimating costs of conducting business under these new Commercial Agreements.

This paper and presentation, focusing on Kennedy Space Center, will discuss changes and new methods of pricing and estimating the costs of NASA facilities and services to be provided to outside entities for use in new Commercial Space endeavors. It will also give an overview of new NASA regulations and documents that establish policy and guidance for entering into Agreements and how they are priced under the various types of Agreement currently being used at NASA. (These include Space Act Agreements, Commercial Space Launch Act Agreements, Use Permits, Enhanced-Use Leases, Memorandum of Agreements, etc. and include full-cost, actual cost recovery, direct cost-only, and market based pricing.) The presentation will also provide some insight into the decisions and cost/pricing problems that NASA has faced in making available its facilities, equipment and services under these various Agreements.

This discussion will also work through some actual examples of Agreement pricing and related cost estimating exercises. The presentation will hopefully, provide some valuable insight into current activities and changes at NASA, as well as how costs and pricing policies are being impacted and developed to support this “new way of doing business” for NASA.

The Other RCA: Restaurant Cost Analysis (MG-5)

Peter Braxton – Senior Cost Analyst and Technical Officer, Technomics, Inc.

The Weapon Systems Acquisition Reform Act (WSARA) of 2009 highlighted the importance of Root Cause Analysis, or RCA, but its conduct remains shrouded in mystery. In illustrating the central role of risk and uncertainty analysis in cost, Dick Coleman often made the provocative pronouncement to the effect of “You can’t stand outside a restaurant with a menu, and the people you’ll be dining with, and a calculator and get within 10% of the final bill, so what makes you think you can estimate a complex multi-billion-dollar acquisition program with that precision?!” In a perennially popular training session, Eric Druker uses dinner out with his boss as an evocative example of Monte Carlo simulation. Shamelessly borrowing from those two, this paper presents the accessible analogy of restaurant cost analysis using a readily available source real data, namely the author’s extensive ? much to his wife’s chagrin! ? collection of restaurant receipts, to clearly explicate the principles and conduct of RCA.

RCA aims to separate deterministic from probabilistic causes for variation in cost (usually growth) from initial estimates. More than just a “post mortem,” it seeks to infer lessons learned (or Dr. Tzee-Nan Lo’s more apt “lessons to be learned”), which can possibly be translated into mitigation strategies for future risk management. (In the spirit of the Serenity Prayer, program managers must know which decisions they can make, which external decisions they can lobby to influence, and which factors are simply beyond their, or perhaps anyone’s, control.) Effective RCA requires access to the cost model itself, preferably incorporating uncertainty and as it evolved over time, and the inputs thereto. Thus, we need to know not just what the diners ordered, but the entire menu, capturing the range of possible inputs and outputs. This relates to Dr. Christian Smart’s notion of a progression of conditional estimates. Also essential for RCA are well-defined growth categories and an accompanying order of operations.

The potential for analogies within this framework are virtually limitless. The courses of the meal are life-cycle phases. The basic commodity is the type of meal (breakfast, brunch, lunch, happy hour, dinner), and sub-type is context (family meal, date, group travel, solo travel). The type of restaurant represents the stringency of requirements. Number of diners is quantity, and acquisition strategy is reflected in a la carte vs. prix fixe, and the use coupons or frequent diner programs. And so on.

No analogy is perfect, and the paper will briefly touch on the dissimilarities between the two RCAs. Primarily, restaurants reflect more of a fixed-price environment, where the multitude of meals and diners enables invoking the law of large numbers, with variations in cost priced in to the offerings, assuming financial solvency, which is hardly a certainty in the restaurant business! By contrast, defense acquisition is typified by a cost-reimbursable environment and specialized industrial base often verging on monopoly/monopsony. Still, if this level of insight can be gained by an individual analyst using his own personal data, certainly it is achievable by a well-funded acquisition program.

Intelligence Mission Data Cost Methodology Guidebook (MG-6)

Eugene Cullen, III – Senior Cost Analyst, Booz Allen Hamilton
Matthew Schumacher – Booz Allen Hamilton

In 2013, Booz Allen Hamilton developed and authored the Intelligence Mission Data Cost Methodology Guidebook (IMD CMGB) for the Defense Intelligence Agency’s (DIA) Intelligence Mission Data Center (IMDC). This guidebook is the official costing manual for life-cycle mission data planning, required by Department of Defense Directive (DoDD) 5250.01 “Management of IMD within DoD Acquisitions,” and defines costing methodologies, procedures, and processes that result in OSD Cost Assessment & Program Evaluation (CAPE) and Government Accountability Office (GAO) compliant cost estimates that are required by DoD Acquisition Systems. The guidebook specifically applies these standards to the IMD environment to enable diverse elements across the Defense Intelligence and Acquisition Enterprises to develop and evaluate life-cycle costs for
IMD as a component of DoD acquisition program costs. The objectives of the guidebook are threefold:

• To provide a detailed and consistent costing methodology based upon US government best practices (aligned with the GAO’s 2009 Cost Estimating and Assessment Guide) to be used in preparation and evaluation of DoD acquisition program cost estimates;
• To establish standards that integrate IMD cost estimates into the Defense Acquisition Life Cycle Management System and the Lifecycle Mission Data Planning process; and
• To serve as the principle reference for current and future intelligence cost reporting.

The guidebook outlines commonly used costing methodologies (i.e. analogy, parametric, engineering, expert-opinion and extrapolation from actual costs); explains how to conduct sensitivity, risk, and uncertainty analysis; and identifies O&S cost elements, common cost drivers, trends, and considerations specifically within the IMD life-cycle management process. Further, the guidebook includes several tailored case studies and vignettes, designed to exemplify how IMD cost analysis is being conducted at Intelligence Production Centers, Program Offices, and across five IMD functional disciplines (i.e. Signatures, GEOINT, Electronic Warfare Integrated Reprogramming, Order of Battle, Characteristics & Performance).

To substantiate content and generate support, the Booz Allen team championed an integrated project team (IPT) chartered over a sixth month period to develop the guidebook. The IPT included over 75 participants representing 25 USG agencies across defense, intelligence, acquisition, cost communities, and costing experts. The Booz Allen team led the IPT’s data gathering, analysis, and stakeholder consultations, and reviewed over 150 costing artifacts including cost guidance, policies, and resource material from the DoD Acquisition Community, Service Cost Centers, NASA, and private industry. IPT sessions provided ample opportunity for stakeholders and action officers to raise issues, resolve differences, and de-conflict objectives throughout the content development process. The guidebook creates value for a variety of stakeholders by allowing the DoD to more efficiently manage acquisition program costs related to IMD availability requirements and facilitates capability decision-making based upon a reliable and consistent costing methodology framework.

The IMD CMGB was approved in April 2013, and is available via the Defense Acquisition University (DAU) Portal to DoD Major Defense Acquisition Programs (MDAPs), Major Acquisition Information Systems (MAISs), Intelligence Production Centers, Program Offices, and other organizations qualified to prepare or evaluate IMD cost estimates.

Achieving Affordable Programs NRO Cost Analysis Improvement Group (CAIG) Support of Cost Driver Identification (MG-7)

Linda Williams – Program Manager, Wyle
Pat Kielb
Eileen DeVillier – Associate, Booz Allen Hamilton
Jay Miller – Specialist Leader, Deloitte Consulting

The NRO CAIG has significantly expanded its role beyond independent cost estimating of a large acquisition program to focus on enterprise decision support. One area that is proving invaluable to senior decision makers is cost driver identification.

In an era of looming budget reductions and the possibility of sequestration, the NRO CAIG was tasked to identify cost drivers and provide a framework for additional cost reduction opportunities.

To accomplish this task the NRO CAIG leveraged prior studies and the data collected over 30 years on the organization’s very large space programs. Additionally, the NRO CAIG collected detailed budget data to help focus beyond large acquisition contracts and increase visibility into total organization costs.

This paper will describe the approach taken over the last several months to identify cost drivers and summarize findings that senior leaders found helpful to the decision making process.

In summary, as budgets reductions trigger asset affordability reviews, the NRO CAIG has been able to assist in identifying total organizational cost drivers and provide a framework for future cost reduction opportunities.

Gödel’s Impact on Hilbert’s Problems Or Cost Consistency and Completeness as an Impossible Exercise (MG-8)

David Peeler – Director of Risk Management, Amgen, Inc.

In a previous set of papers, the idea of using Hilbert’s Problems as a construct to propose and recently revisit the status of a list of Hilbert’s Problems for Cost Estimating. This paper similarly employs Gödel’s enlightenment with respect to Hilbert’s attempts onto the cost estimating community. What can we learn about ourselves as estimators and where can we exert the greatest impact with respect to the use of our estimates? Using Godel’s two theorems of undecidability as catalyst, we will explore the effect and utility of exacting math and other motions on cost estimates specifically and programmaitics generally.

A New Cost Management Policy for Your Organization: An Integrated Approach? (MG-9)

Tom Dauber – Principal, Booz Allen Hamilton
Woomi Chase – Lead Associate, Booz Allen Hamilton
Ken Odom – Booz Allen Hamilton

Developing a robust Cost Management Policy is a key driver to the success of any organization, regardless of size or industry. The policy should ensure cost control measures that are valid and effective, risks are mitigated, solutions are delivered on time, and profits/ROIs are maximized. The Cost Management Policy should be a systematic approach to managing cost through-out the life cycle of a program through the application of cost engineering and cost management principles. The policy should include characteristics of a credible cost estimate with a repeatable process, schedule health check metrics, integrated cost and schedule linked to requirements, support approaches with historical and statistically sound data that allows for sensitivity analysis, and the institution of a robust data collection culture. The policy should state explicitly that cost, schedule and risk management activities are to be integrated on a program at a stated confidence level for target, execution and contingency estimates. Mapping the cost estimate and risk register to the schedule ensures that the cost and schedule plans are compatible; risks are identified, and opportunities to mitigate cost and schedule growth are proactively managed. Instituting this policy goes beyond just identifying, collecting, measuring, and reporting information to decision-makers to determine the cost of programs, projects, products, facilities, services and/or systems, it rewards program managers (PM) for cost containment, continuous improvement, and efficiencies. PMs are incentivized to identify, quantify, and manage risks through-out the life cycle of the program. Incentives include both non-monetary to monetary awards. Planning and applying this policy early in the life cycle and periodically through-out the program, ensures decision-makers’ cost surveillance controls are optimized and the impacts of near-term decisions are understood in the long run. This paper describes an approach for this type of integrated cost management policy.

Right Sizing Earned Value Management for Your Project (MG-11)

Gordon Kranz – Deputy Director for Earned Value Management Performance Assessments and Root Cause Analyses, Office of the Assistant Secretary of Defense for Acquisition

Earned Value Management (EVM) is a program management tool that provides data indicators that can be used on all programs to enable proactive decision making throughout the program lifecycle and facilitate communication across the program team. Each program has unique attributes that should be considered when determining program management and reporting requirements, including, but not limited to, contract size and type, scope of work, complexity, risk, technology maturity, and resource requirements. In program management, one size does not fit all; integrated program management techniques, including performance measurement and scheduling can be tailored to what makes sense for each program. In this workshop, participants will discuss various contract types and work scope and share experiences in determining the right fit for their program when applying EVM and measuring performance