This article is the third in our series on the subject. Click to read issues one and two.

There is also a full report, available from BVR, entitled “Monte Carlo Simulations:  Advanced Techniques” which includes chapters by the author of this piece, Jason Andrews, alongside A&M colleagues Dave Dufendach and Neil Beaton.

Introduction

After understanding when and how to apply Monte Carlo simulations for valuation purposes, it is important to be able to interpret the statistics of the results of the simulation and conduct diagnostics using those statistics to ensure the simulation is performing as expected. In this article we will provide a summary of some key statistics to consider and how to interpret those statistics, as well as how to use statistical analyses to conduct diagnostics on the simulation to ensure it is performing within expectations.

Understanding Key Statistics and Conducting Diagnostics

During the preparation of the analysis/model to be used in the Monte Carlo simulation, the user should have some expectations of the performance of the simulation and results, and then identify and design diagnostics that will facilitate a statistical analysis of the results.

In understanding statistics for any Monte Carlo simulation, it should be reiterated that within the simulation, each trial is of equal weight (if a certain outcome is more probable than others then that outcome will occur in more trials than others). Thus, the statistical analysis is performed on the entire dataset of the outcomes from all trials within the simulation with each outcome given equal weight.

The following is a description and summary of how to interpret some key statistics that may be relevant when performing a Monte Carlo simulation:

Mean – The mean of the results, in most cases, is the conclusion to derive the input into another calculation (i.e., discrete cash flow when simulating financials) or the estimate of value; therefore, this is the most critical statistic for valuation purposes (but not the only).

Median – In certain instances, the median may be considered a more meaningful indication of the “average” of a distribution than the mean, given that it is less skewed by outliers. In the context of a Monte Carlo simulation, the median can be helpful in understanding the distribution of the results. As an example, in a unimodal distribution if the mean is less than the median, this indicates that the mean is not in the middle of the distribution, but instead the distribution is skewed to the left. Additionally, certain accounting guidance, such as determining the average time to vesting for market-based awards, may require the use of the median of the results of a particular outcome.

Minimum/Maximum – The minimum and maximum are helpful to understand the potential range of outcomes as well as to ensure the simulation is not producing illogical results (e.g., the value of a restricted stock award or option should never result in a negative value or security with a fixed payoff should not have results exceeding the fixed amount).

Standard Deviation – The standard deviation is helpful to understanding the general distribution of the results; a larger standard deviation indicates a wider distribution of results. The expectation regarding the standard deviation of any outcome should be consistent with the underlying assumptions (e.g., higher expected volatility of stock price should correspond with a higher standard deviation of outcomes) and complexity of payoff structure, vesting, etc.

Kurtosis – The kurtosis is the measure of the extent the distribution of the results is peaked or flat. The notable value is 3.0, which indicates a standard normal distribution. A kurtosis higher than 3.0 indicates the results are peaked and concentrated at the mean and less than 3.0 indicates the results are relatively flat at the mean.

Skewness – The skewness statistic provides a numerical representation of what any observer of a distribution chart would be able note. A skewness of 0 indicates a symmetrical distribution of results, while a positive value indicates a log-normal or skewed to the left distribution.

In addition to analyzing the results of a key outcome(s) to derive the intended value estimate, the statistical analysis can be leveraged for other outcomes within the simulation to interpret the performance of calculations and understand the results. Leveraging the prior post’s example of the application of a Monte Carlo simulation for the valuation of an rTSR award, we can provide several of potential examples of diagnostics that could be conducted for such an analysis.

One outcome of the rTSR simulation that would be of interest to analyze is the number of shares vesting and/or the rank of the subject company’s stock price return. A simple solution would be to track the rank and/or number of shares vesting in each trial; however, the statistical analysis of the rank or number of shares vesting would not necessarily provide a clear understanding of the frequency of the various vesting thresholds (i.e., rank of return) being achieved. Alternatively, a secondary calculation could be performed which would result in a value of 1 when a certain rank is achieved and 0 if not; the resulting mean of all trials would provide the probability of that rank being achieved.

Another diagnostic that is often helpful to perform when preparing a valuation of an equity security or derivative using a risk-neutral framework (i.e., geometric Brownian motion) is to calculate the present value (discounted at the risk-free rate) of the payoff of a standard European stock option (maximum of 0 and future stock price less exercise price) then compare the mean of the results to the value indicated by a standard Black-Scholes-Merton option pricing model with the same assumptions, which would provide some reassurances that the simulation of the stock price is behaving as expected and/or a sufficient number of trials has been selected. Alternatively, the behavior of the stock price simulation can be assessed by comparing the mean of the results of the present value of the future stock price at maturity in each trial to the beginning stock price; the theoretical difference should be zero.

While it may be tempting to prepare the Monte Carlo simulation and just pull the mean from the results to derive the estimate of value without further analysis, it has been our experience that a more detailed review of the statistics and advanced consideration of potential diagnostics can provide assurances that the simulation is performing as expected and allow an analyst to provide insightful explanations of the results that may be invaluable when discussing with stakeholders.

This article originally appeared in a BVR Special Report.