1. A fresh approach
The Model Audit
Financial models are at the heart of major
finance transactions. We believe the standard
approach to auditing these models is flawed.
Our fresh approach offers tangible benefits for
both senior funders and shareholders.
2. A flawed process
A typical financial model audit carried out by an Documentation – what you don’t know can
accountancy firm will consist of the following steps: hurt you
• a review of model logic and calculations (including tax A key role of the auditor is to check the model to project
and accounting compliance); and finance documents prior to financial close. The
• a review of the model to supporting documentation; and following are typical failings:
• a report on model issues. • key documents are produced late in the transaction,
sometimes finalised only hours or minutes before close,
These steps are performed iteratively, culminating in the a review is either then rushed or does not take place at all;
production of an opinion letter confirming model integrity
which is issued on financial close. • documents which are erroneously considered as not
material, receive scant attention;
Our experience, drawn from the practice of both auditing
and being audited, combined with discussions with • this work is often undertaken by analysts with little
shareholders and funders, reveals a level of dissatisfaction commercial experience and nuances which impact the
with each stage of this typical process. robustness of an operational project company are
missed.
Model logic review – missing the unknown
unknowns Reporting – more is less
Model auditors typically check model logic in two ways: After each iteration of review, the model auditor will
produce a list of issues for the attention of funders and the
1. They run spreadsheet mapping tools to check for model model builder. When we send off a bid model to an auditor
consistency. which has been subject to internal review, we are often
2. They check the spreadsheet ‘cell-by-cell’, ticking as they surprised by the weight of the tomes received in return!
go to confirm correctness. In nearly all cases, closer inspection reveals that no
The first of these techniques, spreadsheet mapping, is commercially significant errors have been found. Instead,
certainly necessary to checking the consistency of any comments consist of a combination of a download from
complex spreadsheet. However, there can be a tendency whichever software package has been thrown at the model
for over-reliance on this technique, and in some cases this and a list of general ‘observations’ which can be applied to
is thought to be a sufficient test in itself. It must be any model review. While superficially a large audit report
remembered that these tools only check for may suggest rigour, in reality it is more likely to reflect an
consistency and poor referencing, they do not actually indiscriminate approach, lacking sufficient focus on
analyse any logic. correcting commercially significant errors.
The second of these techniques, ‘cell-by-cell’ review, is not Model opinion – ‘Insurance’ or ‘Assurance’
particularly satisfactory. As a typical model may contain In our view, there is an important problem with the way
over 5,000 distinct and often complex formulas the task of that the role of the auditor is viewed.
checking each is a daunting one. In reality, the main
drawback to this sort of checking is that it is proven to be Funders typically view the opinion letter as providing
far too boring for the human brain! Errors will be missed. insurance, believing the auditor will pick up the tab if errors
Research has suggested that cell-by-cell reviews can be are subsequently found. Also, as an opinion letter is a
expected to capture around 70% of errors. This is actually condition precedent for the credit agreement they also
a fairly good success rate – but you may have sleepless tend to view the audit as an important piece of ‘admin’.
nights considering the unknown 30%! The criteria for selecting an auditor then becomes a
straight comparison of who will offer the highest ‘cover’
Another issue with the ‘cell-by-cell’ process is that it is time for the lowest price.
consuming and therefore costly. There is no ‘short-cut’ in
such a process which will enable an early view to be given The auditor is also very aware of potential liability and
on model accuracy. This has historically meant that model funders requirements, and this may lead to them running
reviews are only carried at the last possible moment and the following business model: offer the minimum liability
that there has been no appetite to procure an independent caps required for market entry, reduce the costs of
review at an early stage in the competitive process. This delivering the actual audit, employ a good lawyer to draft
leads to the unfortunate position that commercially terms and conditions and finally, caveat his opinion as
significant errors either jeopardise a project or the strongly as possible!
consequences must be taken on the chin by shareholders.
3. A fresh approach
So far, we have looked at a number of areas where Scope of work – a stitch in time…
the current model audit process might disappoint.
There is a clear case for rethinking the scope of the
More positively, here are some suggestions for
auditor’s work.
improvement.
In the world of Competitive Dialogue, it becomes
Methodology – it is rocket science! increasingly important for reliance to be placed on the
In order to look for improvements to the standard financial model at an earlier stage of the transaction.
‘cell-by-cell’ approach it is useful to look to other Requesting some level of sign off from an independent
industries. When NASA commissions its complex source as to model logic correctness should be a
models and needs to be really sure its computations requirement of final tenders. This is not to say that a full
are correct, it does not ask one firm to model and audit is required, but as has been outlined above an early
another to audit. Instead, it will give the same job to consistency check can be carried out on which an
two sets of developers and see if they come up with independent reviewer can give a letter of comfort. This
the same answers. This rigorous approach would check need not be prohibitively expensive and, as the
historically have been viewed as too expensive in the auditor is completing work which will be needed later if
project finance market. the bid is successful, fee structures could be tailored to
minimise cost impacts during the bid stage.
However, the increasing prevalence of ‘template’
models and the standardisation of financial structures Also, a robust approach should be taken to the selection
mean that it is now possible for an initial model of a of auditor and agreement to the detailed scope of work,
relatively complex scheme to be constructed in a including:
matter of hours. This opens the way for the ‘NASA • expectations for scope of document review;
approach’ to be used in model audit. A commercially • timescales for reporting;
astute auditor independently creates a shadow model • approach to sensitivity review;
using project data inputs, the outputs of the shadow
• details of the commercial experience of the analysts
model are then compared to the model under test.
who will be undertaking the review; and
Any discrepancies can be fully interrogated, and the
‘errors’ in either shadow or test model corrected. • agreed processes for ensuring final auditor sign off of
completed document review.
This method has three very important advantages to
the standard approach. Firstly, by challenging the On this last point, by ensuring that the auditor’s opinion
analyst to create a replicated model you retain his letter is matched to the list of final documents, and
interest – this is crucial to ensuring a job well done. making this an explicit condition precedent, many
Secondly, the integrity of the review is clearly potential issues can be avoided.
demonstrated; if the analyst achieves a different Finally, all parties should place the importance of ensuring
answer, the discrepancy is clear – and it needs to be model correctness high on their priority list. While having
explained. Thirdly, an early indication of the accuracy recourse to the auditor’s potential liability has a role in
of the model under test can be given, this can be used ensuring rigour, it is in everyone’s interests that such
to inform commercial decision making. remedy is never required.
Mazars' fresh approach
Mazars’ fresh approach offers tangible benefits when compared to standard offerings.
• We are confident that our replication methodology is more robust – we offer a separate, not subordinate,
liability to shareholders to demonstrate this.
• We are prepared for the Competitive Dialogue process – we will undertake reviews prior to bid submission
and on a contingent fee basis to ensure the robustness of your bid.
• We tailor our scope of work to your requirements – you will find our fees are highly competitive when
compared to standard offerings.