Cost estimation and cost risk analysis in early design stages of naval projects

André Neumann¹

Abstract

During the design and engineering process for naval projects a comprehensive cost management is required. The overall aim is to ensure the financial feasibility throughout the project with a low financial risk. Cost management combines acquisition costs and in-service costs in order to obtain life cycle costs. In this context methods for cost estimation have to be adapted to the phase of the planning process of a naval vessel. This paper introduces challenges and solutions for the cost estimation in general and describes MTG´s cost estimation approach for conceptual and preliminary designs. Additionally the utilization of cost risk analyses in order to ensure a reliable budget planning and the identification of cost risk drivers will be described.

Key words: Cost Estimation, Cost Risk Analysis, Design and Engineering, VORGES, Risk Minimization, Budget Planning, Program Acquisition Cost, Life Cycle Cost.

Resumen

Durante el proceso de diseño e ingeniería de proyectos navales se requiere un manejo integral de los costos. Esto tiene por objeto garantizar la viabilidad financiera a lo largo del proyecto con un bajo riesgo financiero. Esta gestión de costos combina los costos de adquisición y costos durante el servicio con el fin de obtener los costos del ciclo de vida. En este contexto, los métodos para la estimación de costos tienen que ser adaptados a la fase del proceso de planificación de un buque de guerra. Este documento presenta retos y soluciones para la estimación de costos en general y describe la forma en que MTG ha abordado la estimación de costos para los diseños conceptuales y preliminares. Además, la utilización del análisis de riesgo en costos a fin de asegurar una planificación presupuestaria fiable y se describirá la identificación de factores de riesgo en costos.

Palabras claves: Estimación de costos, análisis de riesgo de costos, diseño e ingeniería, VORGES, minimización del riesgo, planeación de presupuesto, programa costo de adquisición, costo del ciclo de vida.

Date Received: January 1st, 2015 - Fecha de recepción: 1 de enero de 2015
Date Accepted: March 18th, 2015 - Fecha de aceptación: 18 de marzo de 2015

________________________

¹MTG Marinetechnik GmbH. Department of Cost Estimation Services. Hamburg, Germany
............................................................................................................................................................

Introduction

An inherent requirement of every naval project is a timely and economical meeting of the customer’s demand. This implies a nearly exhaustive planning process. Especially in early design phases – and in mid-term planning - risk minimization is emphasized. At the same time the technical and economic feasibility of the procurement plans is paramount.

Reliable cost estimates are necessary for the government to support funding decisions and budget planning. They are also useful for the project management to avoid budget overruns and its consequences (e.g. performance cutbacks, missed time schedules and increased funding), assessing competing solutions and evaluate tenders and select the most efficient solution. Especially in early design phases cost estimation is extraordinarily challenging because:

• detailed and meaningful data regarding the technical design is sparse at this point of time;

• an arguable basis for a comparison of the considered systems is mostly not given (due to technological leaps, e.g.);

• cost data for systems under development are not available, i. e. they are not specified finally;

• estimation of in-service costs is very ambitious for systems which are not introduced into the Navy yet, so historical data for maintenance and repair is not available.

Cost estimations are required to be as precise and reliable as possible which necessitates trustful relationships to suppliers and a lot of experience in the field of cost estimation and planning. Ideally cost estimations are supported by a comprehensive database filled with technical and cost data for naval technology.

Determining cost estimations, based on technical solutions developed in-house, are the main tasks of MTG Marinetechnik GmbH (MTG) in the course of this process. These activities create the foundation in order to provide information and data for phase documents needed during the planning process. This expertise is used by MTG to support the public contracting authority in planning phases and during operation of nearly every project of the German Navy since 1966.

MTG has organized its internal process and work steps in a process model called VORGES (Vorgehensmodell Gesamtentwurf Schiff, comprehensive model for total ship system engineering). This model allows the generation of feasible and affordable solutions based on the customer’s requirements in comparatively short time. Fig. 1 shows the process steps whereas especially ship design and cost estimation are of particular significance. Ship design and cost estimation are interdependent. In the majority of planning processes ship design forms the basis for cost estimation (design-to-requirements). Occasionally a ship design has to comply with a given upper cost limit (design-to-budget). In these cases the portion of fulfilled requirements has also to be determined. Both designs are optimization processes characterized by repeating design loops aiming at an optimal and low-risk solution.

See [1] for any further details concerning the process model VORGES. The rest of this paper focusses on the third step cost management which comprises cost estimation and cost risk analysis. In general terms cost management is the process of collecting and analyzing historical and upto- date data and applying quantitative models, tools and databases to forecast future costs of a project. Cost management helps to translate functional requirements into budget requirements and determine a realistic view of a probable cost outcome which forms the basis for executing the project itself.

During the early stages of the design process several conceptual and preliminary designs have to be generated in a short period of time. Those designs need to be comparable in order to support decision making processes. Requesting prices of potential equipment for the designs implies two problems:

1. Oftentimes internal compliance regulations will not allow suppliers to provide budgetary prices to potential customers without a tender process. Not finalized specifications at the time of the cost request will lead to budget prices that might be fare away from specifications at the time of the tender. This implies the danger that budget prices will be used during the contract negotiations. Also the technical and financial information given by a supplier might be used for the tender documentation and will therefore be given to potential other suppliers, which is undoubtedly not in their sole interest.

2. The process of gathering trustworthy cost data is time-consuming and administratively challenging.

MTG as an independent consultant on the other hand is not a potential customer and can therefore request prices to build up a technical and cost database that also can be used for other designs and the sole purpose of generating comparable designs in a relatively short period of time.

Cost Estimation

Basics

A cost estimate of prospective warships is the most probable total of individual cost elements based on a work breakdown structure (to identify all cost elements), technical parameters (to define the ship to be estimated) and several rates (e.g. labour rates, overheads).

In this context cost estimation is the systematic elaboration of costs based on available data in order to determine the most probable total system costs at the time of estimation. Systematic means a comprehensible, reproducible and widely objective method is used which can be applied to any naval shipbuilding project. Obviously estimations can only rely on data which is available at that moment. Therefore cost estimations always relate to a particular point of time. Regarding that usually a lot of assumptions have to be met – which can change in the course of the project – it is unavoidable to document not only results but especially assumptions and inputs carefully.

Acquisition Costs

Estimating acquisition costs is traditionally the main task of a specialized cost estimation team. This section gives a brief introduction to commonly applied cost estimation methods and the cost estimating process at MTG. Acquisition cost estimations are conducted from a shipyard’s perspective.

a) Estimation Methods

Glancing at the literature one finds multiple estimation techniques which may be summarised as follows [9]:

• Expert opinions
• Analogies
• Parametrical approaches
• Engineering Build Up

Expert opinions may serve as a rough order of magnitude (ROM) if very little information is available and quick statements are required. An “expert” may be an experienced member of the cost estimating team itself or a vendor of a system willing to provide cost information. Asking the vendor directly is advantageous because numbers are typically available but oftentimes it is difficult to understand what is included in the communicated costs.

Using analogies means to perform cost estimation with comparable systems as a basis for the system in question. Analogous estimates always need subjective and elusive adjustments to meet the characteristics of the estimated system. Further downside of analogies is that they rely on a single data point and that it is sometimes hard to identify comparable systems at all. In many cases analogies are the only way to estimate generic systems or prototypes characterised by little definition.

Parametric cost estimation means estimating with mathematical equations which are found in standard software tools or are better based on proprietary models [10]. Parametric equations relate the dependant variable cost to one or more independent cost-driving variables (which may be amongst others: technical parameters, performance characteristics or costs of other elements). The resulting function is a so-called Cost Estimating Relationship (CER). One of the strengths of CERs is that they are an excellent tool for performing rapid sensitivity analysis (“what if”-questions). Oftentimes collecting sufficient appropriate data to generate CERs is difficult and time consuming. Parametric estimation plays an important role at MTG so the idea is picked up again in part (b) of this section.

Engineering build up is a “bottom-up”-approach as it combines single estimates of each cost driving element at a very detailed level into the overall estimate. Usually each cost element causes material and labour costs (e.g. manufacturing the hull, system integration and installation) which have to be estimated separately. If little data is available this approach is similar to expert opinions. Engineering build up is an intuitive and easily arguable approach but it is costly and the many times demanded “what if”-questions are hard to answer in short time.

b) Cost Estimating Process at MTG

The cost estimating process at MTG is organised in the following 7 steps:

1. Start-up of project
2. Define work breakdown structure (WBS)
3. Choose Estimation Model
4. Generate cost estimate
5. Review and validate estimate
6. Perform cost risk analysis
7. Document and present estimate

Step 1 serves as an initiation of the estimation process in order to understand the project’s objectives and its requirements. Technical, programmatic and cost data are collected. Prevalently it is required to adjust the data to account for inflation, learning and quantity. Furthermore the acquisition strategy (competition, consortium) has to be gathered. Typical sources of data are: cost proposals of vendors, historical databases, governmental agencies (the customer), experts and open source (Internet, Jane’s). Additionally necessary assumptions are identified in this step. Amongst other things assumptions are made for inflation rates, hourly wage rates and overheads

The start-up is followed by the definition of the WBS elements in step 2. A WBS is a decomposition of the ship in smaller components or rather functional technical groups like hull structure, propulsion plant, electric plant and so on. A commonly used WBS is the Expanded Ship Work Breakdown Structure (ESWBS) which is promulgated by NAVSEA [11]. The level of decomposition depends on the current stage of the project and the required detail of the results. Obviously the more is known about the project, the more can be detailed in WBS.

The selection of the estimation model in step 3 also depends on the current phase of the project. In order to estimate acquisition cost, a ship design is divided into platform and payloads. To put it simple, platform comprises systems and components needed by the ship to participate in maritime transport. On the other hand payloads are necessary to fulfil the military tasks (e. g. weapons, sensors as well as RAS-equipment and cranes). The methods for estimating costs of platform and payloads differ considerably. Platform costs are estimated by means of a parametric approach, i.e. costs for a specific component (e.g. main engine) are estimated indirectly via a parameter (e.g. power of main engine). The resulting curves called cost estimating relationships (CER) are described below. This method is not applicable for payload estimation because the costs of a payload are typically not dominated by one parameter. Therefore cost estimation of payloads requires intensive cooperation with the suppliers of systems. Expert opinions and analogies are required if contact to the supplier does not lead to plausible results in short time.

Nonrecurring costs are being estimated by means of empirical formulas which are derived from realized projects and include amongst others:

• Software costs
• Logistic support (initial spares, tools and training equipment)
• Documentation
• Detailed design
• Management
• Initial training.

It is without a doubt that there are lot of valuable software tools for cost management available on the market. The tools are used frequently on industry and government side. Therefore MTG has repeatedly tested different applications. According to MTG’s experience the following statements can be made:

• Tools including a cost database can produce results in a very short time. Probably these results are trustworthy for the majority of the users.

• These tools are expensive and therefore cannot be applied economically in small- or mediumsized companies.

• The delivered databases contain projects from different nations and even different industries. Usually naval projects are rare. It is at least debatable whether projects from other nations/ industries can be transferred without difficulty.

• These tools oftentimes require input data which is not available at very early project stages.

• Tools without a database necessitate to build up an own database in order to calibrate the calculations.

• MTG has not found a tool which could be used without customization regarding MTG’s design process.

Based on these findings combined with the longstanding experience using cost estimation tools MTG has decided about 10 years ago to develop in-house tools which are highly flexible and may be adapted precisely to the VORGES process. The VORGES database is a main tool in the methodology and contains technical, cost and risk data which is necessary in order to conduct cost estimations. Therefore MTG’s cost tools have a bidirectional connection to the VORGES database in order to load input data and save results. The database is subject to continuous development which requires immediate modifications of the tools which are not possible with commercial software.

MTG uses two models, one for the very early rough, conceptual design phase (SCEM) and one for the preliminary design phase (GELIMAKO). Both of them are proprietary models which are represented in in-house software tools as no COTS software is available for this kind of estimation task. Both tools apply a combination of the above mentioned methods. The main method of SCEM is parametric estimation with CERs. The data points which were collected in the course of time are connected via polynomial or cubic spline interpolation. Parametric estimation is used for platform cost elements like hull, propulsion and electric plant. Accordingly independent variables like volume of hull, power of main engines, number and kind of propellers, power of electric plant are used. It’s not easy to generate CERs for payloads so expert opinions and analogies are used. Engineering costs for construction, management, proofs and so on are estimated via empirically found formulas. Typical parameters in these formulas are number of yards, number of ships, installation costs of payloads etc. SCEM is usually used for ROM-estimates and “what if”-questions. On the other hand GELIMAKO is a method which uses engineering build up. This implies much more detailed knowledge about the project and the ship. Therefore GELIMAKO also incorporates different currencies, individual inflation per cost element and price adjustment clauses. Bottom-up is simpler regarding the calculation itself but it requires a much higher effort in researching suitable data.

Costs are estimated on per component basis whereas a typical frigate-sized project is made up of several thousand cost elements.

If all necessary data are collected and entered into the software running the model generates the cost estimate at the push of a button. This always goes along with a review and validation of the estimate in step 5. Reviewing means to cross check the overall estimate with historical projects to see if the results are plausible. Implausible results lead to a revision of all input data and one or more re-runs of estimation.

The final result of the acquisition costs estimation comprises of the following elements:

• Acquisition costs of platform and payloads
• Nonrecurring costs
• Contingencies
• Taxes

In-Service Costs

Next to the acquisition cost estimation the inservice costs are the main cost driver during the lifecycle of a naval vessel and are therefore required to elaborate life-cycle-costs. MTG´s studies in collaboration with the German government have shown that personnel costs (crew), maintenance costs and costs for petrol, oil and lubricants dominate the in-service costs of a naval ship.

The annual consumption of petrol, oil and lubricants is estimated based on ship’s operating profile, characteristics of main engines and electrical plants as well as environmental conditions. This is mainly a technical task. Combining the results with actual and forecasted prices for petrol and oil allows deriving the annual costs or the ship’s in-service time. Forecasting of price changes is unavoidable in cost estimation. This applies to all categories of in-service costs therefore MTG’s approach at the end of this section.

Personnel costs are another major factor. MTG considers costs for crew members. Consideration of personnel costs is becoming increasingly important as salaries typically rise more than defense budget.

As the systems aboard the ship become more and more sophisticated highly qualified personnel is needed. Therefore each government has to make efforts to ensure that enough qualified personnel is available in order to operate the ship. One prerequisite for this is an adequate payment. Therefore estimation of personnel costs is a valuable resource for a comprehensive planning of future crews.

The number of crew members and its structure is required to estimate personnel costs. Additionally plausible assumptions concerning distribution of military ranks, age and sex are needed. These data is usually based on each Navy’s experience or statistical authorities. MTG’s calculations then consider basic salaries, accruals for the future (if relevant), additional personnel costs and benefits. These data are available to the public in Germany. If the numbers and the structure of the crew are given (or assumed) a reliable cost estimation is possible. Finally, personnel costs can be rather seen as a bottom-up calculation than estimation.

The technical concept of the ship class as well as administrative rules and operating profiles create the foundation for cost estimation of preventive and corrective maintenance costs. Historical data of the naval ships already in service is combined with expert knowledge in order to perform rough cost estimations. MTG as an independent service supplier has remarkable advantages in data acquisition over competitors. On the one hand data of the navy arsenals which is not available to the market can be incorporated in the estimations. On the other hand suppliers are cooperative as they may find their systems integrated in MTG’s designs. Estimation of maintenance costs is the most challenging task in in-service cost estimations. This is mainly due to the following facts:

• Only large suppliers (e.g. for engines, weapons) focus on in-service costs. Even if this data is available the rest of the ship may be neglected in no case. This requires assumptions to be made.

• Historical data of ships already in service is often difficult to transfer to current systems due to technological changes.

MTG estimates these costs on per component basis using a statistical approach. The cost estimation database contains up-to-date numbers for equipment and military payloads which have been maintained in German yards. These data will be used to forecast preventive and corrective maintenance costs for the future ships. It shall be noted that the following approach allows forecasting the costs for any component, i.e. the component investigated has not to be installed aboard a German Navy ship.

1. Compilation of all available costs for planned and corrective maintenance on a per component basis

2. Compilation of acquisition costs and weights (wgt) for these components

3. Statistical test of the assumption:

4. Statistical test of the assumption:

5. Statistical test of the assumption:

6. Calculation of the coefficient of determination R² for each assumption.

Linear regression methods are used to estimate a straight line for assumptions 3 and 4 leading to an equation of the type:

where:
y = forecast of maintenance costs per year
x = acquisition costs (step 4) ; weight (step 5)
∝ , β = coefficients estimated by regression

Multiple linear regression methods will be applied in order to investigate step 5:

where:
y = forecast of maintenance costs per year
x1 = acquisition costs
x2 = weight
∝ , β , γ = coefficients estimated by regression.

The assumption yielding in the highest R^2 will be used to forecast the maintenance costs of the component under investigation. It is recommended to use this approach only if R 2 >.65. This allows forecasting the maintenance costs for any component with known or estimated acquisition costs and/or weights. If R 2 <.65 for all three assumptions maintenance costs are estimated as a percentage of acquisition costs based on experience. This is on the one hand not the best way but one the other hand a simple and fast method to achieve numbers at all. Additionally the assumption

mostly leads to the highest R² according to MTG’s experience.

Usually in-service costs are estimated for comparing different designs in all ship program acquisition phases and even during the operational phase of a naval vessel. This requires an approved instrument which also considers acquisition costs in order to achieve an overall picture of the life-cycle costs. The next section describes MTG’s approach.

Economic Feasibility Studies

Combining the results of acquisition and in-service costs is recommended when comparative analyses are conducted. In many countries financial regulations obligate the Ministry of Defense to comply with the cost-effectiveness principle. This principle implies cost-minimization i.e. procuring only those goods which are absolutely necessary for closing operational capability gaps. Economic feasibility studies are aimed at finding the most economical solution for a planned procurement.

Oftentimes administrative rules prescribe that economic feasibility studies have to be conducted executed in any governmental procurement having direct budgetary impact. Besides that these studies have to be carried out in advance, during and after a procurement project to allow for an efficiency review.

In Germany any major investment project requires the net present value (NPV) method to be applied. NPV is commonly used for budgeting to analyse the profitability of an investment or project. Despite the fact that a naval project is not a classical investment in order to earn financial profits (naval projects do not generate financial revenues) and in absence of better methods NPV can be applied in this context. Compared to the simple and static methods which sum up the annual payments this is a capital budgeting method which considers the moments of cash flows in order to calculate the time value of money. The NPV is the sum of all present values of cash outflows discounted to a fixed point of time.

where:
C₀ : net present value
−a : initial investment
e_t : cash inflow in period t
a_t : cash outflow in period t
i : discount rate
L : decommissioning / sale

This allows an objective comparison of life-cycle costs of alternative designs to the greatest possible extent in very early design stages.

Fig. 2 shows a comparison of military ships which solely induce cash outflows (but no inflows). Compared to the typical presentation of net present values the sign has been changed in order to achieve positive values. This means lower net present values are more economic. In this case it is interesting to compare designs V1 and V3. V1 has significant higher acquisition costs than V3 but is less expensive during the operational phase which leads to a lower NPV beginning in the year 2037.

Price Escalation

Another important aspect - especially when estimating in-service costs - is to respect price escalation. This is mainly due to the quite long in-service times of naval ships (probably more than 30 years) and the obvious change of prices in such a long period. Besides applying a simple percentage change per year MTG generally uses a sophisticated approach which is based on time series analysis. Time series analysis may be used in many different fields of applications which led to a great amount of literature. Analysis of time series is a complex field of applied economic research and a full coverage of this topic is beyond the scope of this paper. Therefore MTG’s approach is explained here in short terms only (see [2] for a detailed description and further references).

In broad terms a time series is a set of observations y_i of an indicator Y at uniform time intervals. MTG uses price indices as basis for the cost forecasts.

Fig. 3 shows a typical time series which represents any exemplary material price index (Y = exemplary consumer price index, y_i = index value observed in i):

This approach requires the user to find suitable price indices to be forecast by means of escalator clauses. It therefore is assumed that a nation has some kind of statistical office or census bureau in order to search for suitable price indices.

Given this, forecasting of prices is based on the following main steps:

1. Identify suitable price indices for cost categories to be forecast (e.g. oil price index for changes in petrol and oil prices).
2. Analyse and forecast price indices.
3. Build escalator clauses for each field to be forecast (usually personnel costs, maintenance costs and petrol/oil costs).

In order to analyse and forecast prices the following classes of methods are commonly used ([8]):

1. Simple: average, naive, random walk
2. Sophisticated: decomposition, exponential smoothing, ARIMA processes.

The choice of the method is heavily dependent on the price index to be analysed. Error measures help to identify the most suitable method for each use case.

The above mentioned escalator clause (step 3) can be written in general terms as follows:

where:
P_t : price at t
P_t+n : price at t+n
∝ : percentage of price not to be escalated
Mi_t : price index of i-th material M at t
Mi_t+n : price index of i-th material M at t+n
β_i : weight factor of i-th material
Lj_t : price index of j-th labour input L at t
Lj_t+n : price index of j-th labour input L at t+n
γ_j : weight factor of j-th labour input

and:

The degree of detail, i.e. the number of material and labour indices is up to the user requirements and finally depending on available time and data. It is recommended to search for labour and material indices reflecting the underlying work breakdown structure of the project. This probably allows distinguishing between mechanics, electronics, software, energy and different labour types.

MTG has developed escalator clauses for the main in-service cost categories (see section (3)) which are typically applied in life-cycle cost estimations of conceptual and preliminary designs.

Applicability of MTG’s Cost Estimation Approaches in other nations

In order to transfer the methods of acquisition cost estimation to other countries it is definitely worth to think about productivity as the average labour rates differ significantly between countries.

Productivity may be used to measure performance of yards and it is simply defined as the amount of output achieved for a given amount of input. In this context input is primarily material and manpower (labour).

In order to incorporate productivity at least two approaches are conceivable:

1. Use economic indicators like gross domestic product (GDP) per employee or GDP per hour worked

2. Use yard indicators which are specific to the considered yards. This may be a steel labor rate (hours needed to work with 1 ton of steel) or hours needed to produce 1 ton of ship, e.g.

Economic indicators are available to the public for almost every country but obviously this leads to a macroeconomic level of analysis. Individual yard indicators would probably lead to more meaningful results are often difficult to determine.

The above mentioned approaches to estimate petrol and personnel costs can be transferred without difficulty. Of course data concerning operational profiles and crew structures/salaries have to be adjusted to the respective country/navy. Transferring methods to estimate maintenance costs requires an intensive discussion due to strong dependence on national procedures.

Price escalation methods can also applied in other nations. Of course, German price indices must be replaced by local ones. If, for whatever reason, this is impossible macroeconomic values like consumer price index (CPI) or producer price index (PPI) may be analyzed and forecasted for reasons of simplicity.

Cost Risk Analysis

Basics

Cost estimation especially in early project phases requires a multitude of assumptions which in addition are often prone to changes in the project’s progress. Given this, it becomes apparent that cost estimations inevitably contain various uncertainties, e.g. in way of cost estimation methods itself, in time schedules, economic terms and conditions, as well as changes to requirements just to name a few. This means: The singular outcome of a conventional estimation just produces one of the possible cost results. This implies the need for a model which allows to quantify these uncertainties and hence to assess the inherent cost risks of the project. A cost risk analysis (CRA) is a powerful instrument which provides the project management with the possibility to achieve a better transparency in cost estimation and with it the probability to adhere to the limits of a given budget.

A CRA enables the user of the model to quantify uncertainties and hence to identify the cost risks of a project. The magnitude of uncertainty around an estimate is valuable information for the decision maker. Much more detailed information on a variety of topics – that cannot be assessed with a conventional estimation – can be obtained by means of a CRA, including but not limited to the following:

• Cost margins and confidence intervals;
• Probability of exceeding the budget by a certain amount;
• Identification of cost elements that chiefly contribute to the project uncertainties (“risk drivers”, e.g. technology with a low technology readiness level (TRL)).

These results allow the project manager to plan budgets and to derive measures for risk reduction on a quantitative and objective basis.

The remainder of this chapter is organised as follows. Section (2) defines essential terms for the further understanding of the paper. Additionally typical sources of uncertainty are mentioned. In Section (3) exemplary probability distributions are introduced as an idea in order to model uncertainty. Furthermore the impact of correlation on a cost estimate’s risk result is examined. Two commonly applied risk simulation methods, Monte Carlo and Latin Hypercube Sampling, are described briefly in section (4). Running the simulation leads to several results which are shown and interpreted in section (5). Section (6) illustrates a concept to “buy” additional certainty for the project. The chapter closes with the most important conclusions in section (7).

Risk and Uncertainty

c) Definitions
It is important to distinguish between the terms risk and uncertainty. Glancing at the literature a lot of different definitions of the term risk can be found dependent on the contemplated field of activity. In the context of CRA the following definitions are commonly accepted [3]:

• Risk: A chance of loss or injury. Risk is the probability a given budget will be exceeded.

• Uncertainty: Is the indefiniteness about the outcome of a situation. The uncertainty of a cost estimate is modelled as a basis for estimating the risk associated with a specific budget.

Given this, a CRA is a process of quantifying the uncertainties associated with the cost model [3]. As a conclusion one might find that CRA should better be called cost uncertainty analysis as uncertainties are analysed and not risks. In fact this is also a term in use.

d) Sources of Uncertainty
Firstly, the sources of uncertainty depend on the cost model. Generally speaking almost every input variable in the model might be considered as uncertain. However, the more uncertain inputs, the more effort is required for performing CRA.

The following bullets categorise typical sources of uncertainty in 4 groups (the list is not exhaustive) [4]:

• Economic: inflation rates, wages, overheads;
• Technical: New/not introduced technologies, obsolescence;
• Programmatic: Changes in requirements, changes in quantity, uncertainty about budget;
• Estimating: Estimating methods itself, learning curve assumptions.

Usually acts of God, strikes and bankruptcy are neglected as it is nearly impossible to model the associated uncertainties.

Modelling Uncertainty

a) Overview
It was quite easy to identify sources of uncertainty in the last section but it is definitely more complicated to model uncertainty quantitatively as an input to CRA. First of all there are at least two general ways of simulating uncertainty: Inputs-based and outputs-based. Using outputs-based simulation the analyst applies uncertainty directly to the results of the cost model (the estimated costs). This paper will focus on inputs-based simulation which means that uncertainty is modelled by means of statistical distributions that have to be assigned to each uncertain input. Regarding this, one of the main tasks of the analyst is to find suitable distributions and to specify its shape. In the sense of a recap the main characteristics and terms of probability distributions which are necessary for CRA are given in the next sub-section.

b) Probability Distributions
A probability distribution function (PDF) describes the likelihood of all possible outcomes in a given situation. These outcomes, with values that change from trial to trial are called random variables. P(X=x) is used to indicate the probability that a random variable X takes a certain value x. It is necessary to differentiate between discrete and continuous random variables. A discrete random variable can only take a finite number of possible values x [5]. Running through the set of possible values X the probability always adds to one:

Values which are not included in set of possible values of X always have probability zero.

On the contrary a variable is said to be continuous if it can take an infinite number of values in a given interval. In this case it is not allowed to just sum up the singular probabilities; instead an integral over the function f(x) (which models the behaviour of X) is needed:

A further characteristic (which is used later in this paper) of continuous functions is that the probability that X falls into specified interval [α, β] is given by:

In the context of CRA discrete functions might be used, for example, to model uncertainties of input variables like number of ships or number of involved yards. In contrast to this continuous functions can be applied to model uncertainties of variables like weight or power.

Another important concept which is helpful in this regard is the so-called cumulative distribution function (CDF). A CDF states the probability that X takes a value less or equal to u. That is, if f(x) is the PDF of X then

can be found as the CDF. To put it simply, a CDF is the integral over a PDF up to a certain value u. Often it is not easy to integrate the PDFs but luckily the CDFs can be found in the relevant literature for commonly applied distributions.

c) Finding Suitable Shapes
As stated above, finding adequate distribution shapes to model the uncertainty under investigation is a comprehensive task. In general there are three different approaches to select a distribution:

• Use historical data to fit the distribution;
• Distribution is suggested by underlying physics;
• Choose simple distribution with arguable limits.

Most of the times the first approach is not applicable as enough historical data is not available in naval shipbuilding. The second option might be used if sufficient data points are not on hand but the analyst has a reasonable feeling to shape a distribution. The third one is a kind of last resort as a consequence of a significant lack of data [4].

For example a triangle distribution can be considered as simple in that sense. A triangle distribution is defined by only three values: Lower bound, most likely and higher bound. Actually triangle distributions are favoured by analysts because the point estimate can be entered as most likely value. In addition to that the analyst often has a feeling for lower and upper bound values based on his experience.

Just to give another example, one can imagine the durability of some kind of assembly as an input in the underlying cost model. Possibly historical data of that assembly is available which is shown in Fig. 4. Looking at the graph it becomes obvious that a log-normal distribution probably fits the data points.

Of course, there are lots of further distributions defined in the literature. It is recommended to constrain the selection of distributions to the following ones in the context of CRA [3]:

• Normal (quite easy to use but symmetrical shape is sometimes unrealistic)
• Log-Normal (always > 0, positively skewed)
• Triangle (easy to use, understand and communicate)
• Uniform (use if very little is known, every value has the same likelihood)
• Weibull (popular because of remarkable diversity of possible shapes)

d) Example: CER Uncertainty
A very popular tool for conducting cost estimations are Cost Estimation Relationships (CER) which are mostly based on parametric equations. CERs relate the dependant variable cost to one or more independent cost-driving variables (which may be amongst others: technical parameters, performance characteristics or costs of other elements). The data points which were collected in the course of time may be connected for example by regression analysis, polynomials or cubic spline interpolation to obtain a mathematical function.

To give another illustrative example, consider the power of a diesel engine as its main cost driver for the sake of simplicity. Furthermore a positive correlation of cost with power is assumed. This might lead to a CER depicted by the middle line in Fig. 5.

Disregarding the uncertainty would result in the estimated costs P for the engine based on the input E (power). In reality this CER holds twofold uncertainty:

• The necessary power might change so it is not certain, especially in very early design stages;

• The position of the CER itself may vary in an area which is indicated by the dotted lines which act as lower and upper bound.

Both uncertainties have to be represented by a suitable distribution. The example in Fig. 5 uses normal distributions for both uncertainties.

If input uncertainty is considered the input power may vary in the interval [A, B]. The probabilities of each possible value within A and B are described by the corresponding normal distribution. Finally, adding the uncertainty in the position of the CER results in the combined uncertainty which is illustrated by the possible values in the interval [C, D] for the engine costs.

e) Correlation
Usually a lot of CERs are utilised to estimate the singular components of a ship. If two (or more) CERs rely on the same parameter (imagine the power of diesel engine is also used to estimate costs for exhaust pipes) they are correlated. A change in the power value leads to a change in both cost estimates. This is called implicit or functional correlation as it is introduced through the cost model itself. In this case no further correlation needs to (but can) be considered [3].

On the other hand there are often situations in which cost elements are “tied together” by other means (explicit correlation). For example, this would be the case if two (or more) components are made from the same material and the material price is uncertain. In the majority of these cases the relationship is not perfect but varies from time to time [6]. Explicit correlation also arises if uncertainty is applied to outputs. Obviously no functional correlation is present in that case.

Correlation has an impact on the spread of the resulting distribution: the higher the correlation, the wider the spread

Without accounting for correlation, adding multiple distributions will result in too low variances of the final PDF which makes it artificially too narrow. This fact can easily be demonstrated mathematically. Let σ_i and σ_j be the standard deviations of the i-th and the j-th cost element, respectively. Additionally, the total cost variance σ² expresses the spread. Using ρ_ij as correlation coefficient for elements i and j gives:

Omitting correlation, i.e. letting ρ_ij = 0 implies the last addend to become zero and therefore a smaller total variance.

Knowing all this, the analyst now faces the problem to apply correlation by a certain amount, i.e. it is required to find reasonable values for ρ_ij. “Measuring” this kind of correlation is an important task and, especially if several hundred cost elements are used (which is quite common), another real challenge in CRA. In the absence of better data it is even recommended to enter a “default” correlation coefficient for each combination of cost elements. This issue is complex and beyond the scope of this paper. Please see [3] and [6] for further details.

Monte Carlo Simulation Method

Assuming that suitable distributions are found for all uncertain input variables the analyst’s next task is to start simulation in order to estimate the distribution of the total results.

Monte Carlo is a stochastic simulation method relying on a large number of random experiments. This methodology is used to estimate the probability of certain outcomes by running multiple trials using random variables. Determining π by dropping needles on a floor is probably the mostcited application of this method in the last 70 years. This shows that Monte Carlo is a manifold approach which can be applied in various fields of interest.

Coming back to CRA the general workf low of Monte Carlo is illustrated in Fig. 6 (based on [4]):

In this example four uncertain input variables (economical, technical) are shown in which the uncertainty is modelled by triangle and normal distributions (the ‘…’ shall illustrate that there are usually a lot more uncertain variables). For each trial run a point is selected randomly based on the corresponding distribution. Doing this for all input variables leads to a first point estimate. After repeating this for thousands of trial runs finally a PDF is shaped. As a consequence of the central limit theorem the mean of a large number of random variables is normally distributed which is also shown in Fig. 3. Finally a CDF can be derived (its interpretation is given in section 5).

Results

a) Total Cost Histogram
Fig. 7 displays an exemplary histogram which is the first result of CRA. This distribution was shaped based on about 7,000 iterations. The diagram shows the ship’s estimated costs on the abscissa and the respective frequency categorised in 20 intervals (bins).

As already illustrated in Fig. 3 it can be seen clearly that the results approximate a normal distribution.

A couple of results may be derived from that distribution. First of all it can be stated that the total cost range which is indicated by leftmost and rightmost result. Furthermore the analyst can identify the 90%-range, i.e. there is a 90% probability that the costs fall within that range. Of course, other statistical values like mean or median can be computed easily.

b) S-Curve
What every analyst wants to know is the probability of overrunning the point estimate or a specified budget. Such interpretations can be made best using the CDF or S-Curve which is – as stated in section 3.2 – the integrated PDF. An exemplary S-Curve is shown in Fig. 8.

Besides the S-Curve itself there are three markers in the figure. All of them can be interpreted in the same way. Using the project’s budget as an example the graph shows that there is a 30% probability of being on budget, so there is a 70% chance of overrunning it. Additionally, the table presents the percentiles and its respective cost values which also may be interpreted in this way.

c) Risk Drivers
Cost risk drivers are those distributions that produce the highest variability in the risk adjusted cost estimate. Usually the coefficient of variation (CV) is used to identify the risk drivers. CV is a ratio or percentage that divides the standard deviation of a distribution by its mean. This value is accepted as a metric which characterises the spread of the CDF. A higher CV denotes a wider spread of the S-Curve and therefore a higher risk in the component under investigation.

It is important to understand that this is idea can only serve as an indication for risk rather than as an exact computation. Bearing this in mind, it is recommended that the analyst focusses on the top five or ten risk drivers and considers their risk equally.

It is important to understand that this is idea can only serve as an indication for risk rather than as an exact computation. Bearing this in mind, it is recommended that the analyst focusses on the top five or ten risk drivers and considers their risk equally.

• objective and transparent budget planning
• definition and initiation measures to reduce financial risks
• higher confidence in the cost estimation results (usually conventional cost estimates underestimates real project costs).

Modelling Uncertainty

At this stage the analyst knows how to model uncertainty and how to assess the results of CRA. At least the project management is interested in ways to “buy” additional certainty for project. In general there are two ways of gaining certainty. The first one focusses on the reduction of risk to increase confidence (e.g. using proving technologies instead of new ones or investing in risk reduction efforts). The second possibility is to make use of additional funding in order to reduce uncertainty. Regarding the S-Curve (Fig. 5) the main steps to determine an amount of funds needed to raise confidence are [3]:

• Calculate point estimate
• Calculate CDF by simulation
• Choose a desired confidence level
• Use S-Curve to obtain an value on the abscissa based on chosen confidence level
• Compute the amount of funds as the difference between chosen confidence level and point estimate.

Summary

Naval vessels are technically complex and sophisticated systems. Therefore one of the core issues concerning the planning of a naval ship is to identify requirements and translate them to technical solutions. Another important aspect for any project manager is that the technical solutions are affordable in line with the budget or given cost limitations. Therefore MTG has developed a cost management procedure in order to take up the challenge of estimating costs and cost risks for naval projects in very early design stages enabling governmental decision making processes and budgetary planning. Accordingly this paper has introduced cost estimation and cost risk analysis in general terms and also summarized MTG’s approaches.

It should be noted that cost estimation is challenging and difficult especially in early project stages. Cost estimating requires a lot of experience, valuable contacts to industrial suppliers and governmental facilities, comprehensive databases and supporting software tools.

Another key finding of this paper is that every cost estimation model is prone to uncertainty by nature. This is why MTG has established a process for cost risk analysis which is a valuable concept to advance an existing cost estimation process and achieve higher transparency when dealing with cost estimates.

In the course of MTG´s cost estimating process the acquisition costs, in-service costs and cost risk analysis will be elaborated in order to ensure low risk and financial feasibility throughout the planning, acquisition and operational phase of naval vessels.

References

1. NEUMANN, A., ‘VORGES – A Procedural Model For Total Ship System Engineering Developed By MTG Marinetechnik’, ColombiaMar 2013, 2013.

2. RUDIUS, M., ‘Forecasting of prices – An introductory guideline using simple and sophisticated forecasting techniques based on time series analysis’, MTG internal paper, 2014.

3. US AIR FORCE, ‘Cost Risk and Uncertainty Analysis Handbook’, 2007.

4. TECOLOTE RESEARCH, Inc., ‘NAVSEA Approach to Cost Risk Analysis’, 2009.

5. SCHAEFFER, R., MULEKAR, M. and J. MCCLAVE, ‘Probability and Statistics for Engineers’, 2011.

6. SMITH, A. and S. HU, ‘Common Errors when using Risk Simulation Tools’, ISPA/SCEA Conference, 2005.

7. MCKAY, M., R. BECKMANN and W. CONOVER, ‘A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output From a Computer Code’, Technometrics, 1979.

8. BOX, G., G. JENKINS and G. REINSEL, Time Series Analysis - Forecasting and Control, Wiley, 2008.

9. NATIONAL AERONAUTICS AND SPACE ADMINISTRATION (NASA), ‘Cost Estimating Handbook’, 2002.

10. INTERNATIONAL SOCIETY OF PARAMETRIC ANALYSTS (ISPA), ‘Parametric Estimating Handbook’, 2008.

11. NAVSEA, ‘Cost Estimating Handbook’, 2005.