3.2.2 A More Flexible Approach to Valuing Flexibility

Procedia Computer Science, 2013

The DoD has frequently demonstrated its ability to procure phenomenal systems; however, these accomplishments are often tarnished by substantial cost and schedule overruns. While defense acquisition policies are continually being revised to address these perennial problems, many believe that a more fundamental source of these overruns is the lack of flexibility in the systems being developed, which tend to preclude effective responses to unexpected events. However, providing justification to invest in flexibility is a tough sell when the measure of value is a military capability or political outcome, as there is no extant method to demonstrate the potential return on investment. This paper introduces a decision tool for valuing the inherent ability of different systems or designs to respond to uncertainty. The proposed tool is essentially a modification of the current life cycle cost model and is premised on the notion that the need for capability changes in a system arises in a stochastic manner that can be incorporated into a continually updated, expected value model presented in terms of total life cycle cost. The cost-based decision tool presented here quantifies the ability of competing designs to respond to these capability changes via a cumulative distribution function (CDF). The design with the most favorable CDF (i.e., the one that is most likely to meet the most likely set of requirements at the lowest expected value curve of life cycle cost) is deemed to be the "best" design.

Research in Engineering Design, 2013

The design of engineering systems like airports, communication infrastructures, and real estate projects today is growing in complexity. Designers need to consider socio-technical uncertainties, intricacies, and processes in the long-term strategic deployment and operations of these systems. Flexibility in engineering design provides ways to deal with this complexity. It enables engineering systems to change easily in the face of uncertainty to reduce the impact of downside scenarios (e.g. unfavorable market conditions) while capitalizing on upside opportunities (e.g. new technology, favorable regulations). Many case studies have shown that flexibility can improve anticipated lifecycle performance (e.g. expected economic value) compared to current design and evaluation approaches. It is a difficult process requiring guidance, and must be done at an early conceptual stage. The literature offers little guidance on procedures helping designers do this systematically in a collaborative context. The paper investigated the effects of two educational training procedures on flexibility (current vs. explicit) and two ideation procedures (free undirected brainstorming vs. prompting) to guide this process and improve anticipated lifecycle performance. Controlled experiments were conducted with ninety participants working on a simplified engineering systems design problem. Results suggest that a prompting mechanism for flexibility can help generate more flexible design concepts than free undirected brainstorming. In turn these concepts can improve performance significantly (by up to 36%) compared to a benchmark designeven though users did not expect improved quality of results. Explicit training on flexibility can improve user satisfaction with the process, results, and results quality in comparison to current engineering and design training on flexibility. These findings give insights into the crafting of simple, intuitive, and efficient procedures to improve performance of engineering systems systematically via flexibility; performance that may be left aside with current approaches. "We created a marvelous technological achievement. Then, we asked […] how to make money on it." (Leibovich 1999) These words from the CEO of Iridium explained the bankruptcy of the largest commercial satellite communication system ever engineered. The 77 Low Earth Orbit (LEO) satellite infrastructure developed for U.S.$4 billions enabled phone calls anywhere on the planet. The design and management processes were centered on very optimistic demand projections. The technology was working beautifully. This led to the rapid deployment of the entire constellation between May 1997 and May 1998 (MacCormack and Herman 2001). This inflexible design and rigid deployment strategy, combined with underestimation of demand for land-based cell phone technology, might have however caused this engineering system to fail. This case demonstrates that the design and management of engineering systems today -e.g. airports, communication infrastructures, real estate projects -need to go beyond considerations of technology alone. Engineering systems are characterized by a high degree of technical complexity, social intricacy, and elaborate processes, aimed at fulfilling important functions in society (ESD 2011). Dynamic socio-technical elements like markets, operational environment, regulations, and technology play a significant role in their success -and failure (Braha et al. 2006). Crucial decisions have to be made in early conceptual phases of the system design, regarding its strategic and long-term evolution. de Weck et al. (2004) revisited the Iridium case after its downfall and suggested a flexible design concept -a flexible strategy and enabler in design -that would have saved up to 20% in expected development cost. The flexibility would have protected the organization from lower demand scenarios by reducing the initial capital expenditure. Simultaneously, it would have positioned the system to expand as needed to capitalize on high demand opportunities. The strategy involved a flexible staged deployment of the constellation, contrasting with

Systems Engineering, 2008

Designing and maintaining systems in a dynamic contemporary environment requires a rethinking of how systems provide value to stakeholders over time. Developing either changeable or classically robust systems are approaches to promoting value sustainment. But, ambiguity in definitions across system domains has resulted in an inability to specify, design, and verify to ilities that promote value sustainment. In order to develop domain-neutral constructs for improved system design, the definitions of flexibility, adaptability, scalability, modifiability, and robustness are shown to relate to the core concept of "changeability," described by three aspects: change agents, change effects, and change mechanisms. In terms of system form or function parameter changes, flexibility and adaptability reflect the location of the change agent-system boundary external or internal respectively. Scalability, modifiability, and robustness relate to change effects, which are quantified differences in system parameters before and after a change has occurred. The extent of changeability is determined using a tradespace network formulation, counting the number of possible and decision maker acceptable change mechanisms available to a system, quantified as the filtered outdegree. Designing changeable systems allows for the possibility of maintaining value delivery over a system lifecycle, in spite of changes in contexts, thereby achieving value robustness.

INCOSE International Symposium, 2007

Designing and maintaining systems in a dynamic contemporary environment requires a rethinking of how systems provide value to stakeholders over time. Classically, two different approaches to promoting value sustainment may include developing either alterable or robust systems. The first accomplishes value delivery through altering the system to meet new needs, while the second accomplishes value delivery through maintaining a system to meet needs in spite of changes. The definitions of flexibility, adaptability, scalability, and robustness are shown to be different parts of the core concept of "changeability," which can be described by three aspects: change agents, change effects, and change mechanisms. Cast in terms of system parameter changes, flexibility and adaptability are shown to relate to the origin of the change agent (external or internal to a system boundary respectively). Scalability and robustness, along with the additional property of modifiability, are shown to relate to change effects. The extent of changeability is determined by the number of possible change mechanisms available to the system as accepted by decision makers. Creating changeable systems, which can incorporate both classical notions of alterability and robustness, empowers systems to maintain value delivery over their lifecycle, in spite of changes in their contexts, thereby achieving value robustness to stakeholders over time.

2015

The modern warfighter operates in an environment that has dramatically evolved in sophistication and interconnectedness over the past half century. With each passing year, the infusion of ever more complex technologies and integrated systems places increasing burdens on acquisition officers to make decisions regarding potential programs with respect to the joint capability portfolio. Furthermore, significant cost overruns in recent acquisition programs reveal that, despite efforts since 2010 to ensure the affordability of systems, additional work is needed to develop enhanced approaches and methods. This paper discusses research that builds on prior work that explored system design tradespaces for affordability under uncertainty, extending it to the program and portfolio level. Time-varying exogenous factors, such as resource availability, stakeholder needs, or production delays, may influence the potential for value contribution by constituent systems over the lifecycle of a portfolio, and make an initially attractive design less attractive over time. This paper introduces a method to conduct portfolio design for affordability by augmenting Epoch-Era Analysis with aspects of Modern Portfolio Theory. The method is demonstrated through the design of a carrier strike group portfolio involving the integration of multiple legacy systems with the acquisition of new vessels.

2011

Department of Defense (DoD) systems are often highly complex, costly and have extraordinarily long life cycles. Due to these characteristics requirements that these systems will need to meet over their life cycle are highly uncertain. To meet future requirements more rapidly at a lower cost requires an understanding of how to manage uncertainty and architecture to make these complex systems more flexible, adaptable and affordable. This paper proposes an alternative approaches to traditional development through managing uncertainty and architecture in an iterative fashion with decision analysis methods. Several specific methods and tools are discussed to include: Influence Diagrams, Design of Experiments, Design Structure Matrix and Target-Oriented Utility. Collectively the approach identifies the component and architectural drivers of cost in military systems.

2012

is a Senior Member of the Technical Staff at the Software Engineering Institute (SEI). During his 15 years at the SEI, his areas of expertise have included data analysis, statistical modeling, and empirical research methods. For the last several years, he has worked with various DoD agencies involved with the acquisition of large scale systems. From 1999-2005, Jim also worked as a member of the Technical Analysis Team for the CERT Analysis Center.

Systems Engineering, 2007

For many engineering systems, flexibility is an important attribute that allows them to adapt to emerging changes. The existence of flexibility can help space systems adapt themselves to internal/external changes, or even take advantage of new possibilities while in space. Given the potential upfront cost of designing an engineering system to be more flexible, there is a need for a comprehensive framework that allows decision-makers to measure the value of flexible systems design in its different dimensions. Based on insights from the flexibility literature, this paper proposes a unified and comprehensive framework for measuring the value of flexibility in space systems based on six fundamental elements through which flexibility in engineering systems can be mapped. While the illustrative case study presented in this paper focuses on the value of flexibility in a space system, DARPA's Orbital Express program, this framework could generally be applicable for many other engineering systems.

Procedia Computer Science, 2012

For decades, the DoD has employed numerous reporting and monitoring tools for characterizing the acquisition cost of its major programs. These tools have resulted in dozens of studies thoroughly documenting the magnitude and extent of DoD acquisition cost growth. Curiously, though, there have been extremely few studies regarding the behavior of the other cost component of a system's life cycle: Operating and Support (O&S) costs. This is particularly strange considering that O&S costs tend to dominate the total life cycle cost (LCC) of a program, and that LCCs are widely regarded as the preferred metric for assessing actual program value. The upshot for not examining such costs is that the DoD has little knowledge of how LCC estimates behave over time, and virtually no insights regarding their accuracy. In recent years, however, enough quality LCC data has amassed to conduct a study to address these deficiencies. This paper describes a method for conducting such a study, and represents (to the authors' knowledge) the first broad-based attempt to do so. The results not only promise insights into the nature of current LCC estimates, but also suggest the possibility of improving the accuracy of DoD LCC estimates via a stochasticallybased model.

The goal of this research is to develop an analytical framework for screening for real options “in” an engineering system. Real options is defined in the finance literature as the right, but not the obligation, to take an action (e.g. deferring, expanding, contracting, or abandoning) at a predetermined cost and for a predetermined time. These are called "real options" because they pertain to physical or tangible assets, such as equipment, rather than financial instruments. Real options improve a system’s capability of undergoing classes of changes with relative ease. This property is often called “flexibility.” Recently, the DoD has emphasized the need to develop flexible system in order to improve operational, technical, and programmatic effectiveness. The aim of this research is to apply real options thinking to weapon acquisitions in order to promote the ability of weapon system programs to deftly avoid downside consequences or exploit upside opportunities.