Data may not lie, but it sure can conceal…

Desk Performance

Nov 24

In our Insights blog from September, we explored the value of independence when considering a review of dealing desk performance, leveraging the events surrounding a well-known historical incident to make some hopefully valuable points. In this post I follow-up on the topic by sharing some insights into how a simplistic acceptance of a firm’s existing performance and/or risk data, justifiably derided as the ‘tick-and-flick approach’, can lead to downstream problems.

Lee Baker

Lee Baker is an accomplished data scientist, software creator, CEO of ChiSquared Innovations, and author of at least fourteen books on how to work with data. His almost perfectly-named firm has a vision to “produce statistical analysis systems that take you ‘from data to story’ with as little fuss as possible.”

I was drawn to Lee’s earliest work in 2017 when his hit, Truth, Lies & Statistics: How to Lie with Statistics seemed to confirm a former boss’s suspicion that when Powerpoint was used in business the content was nine-tenths deception. But Baker’s clever rearrangement of a quote often attributed to Mark Twain was also catchy; though Twain borrowed from Benjamin Disraeli who is reputed to have remarked that: “There are three kinds of lies: lies, damned lies, and statistics.”

Truth, Lies & Statistics is an easy and fun read, and with chapters like “Pirates Caused Global Warming” (debunking the Post Hoc fallacy) it’s like a crib book for the statistically devious. But while Twain and Disraeli risked painting actual statisticians in a bad light, Baker mounts a partly-convincing defence, turning his sights on those who might use statistical tricks to “hoodwink and otherwise dupe the unwary.”

What Baker’s works reveal is that there are lots of ways data can be manipulated or misrepresented for gain or deception.

The data challenge

In modern finance the average firm produces quite staggering amounts of data, and there are many ways it is ‘manipulated’ for good or not-so-good, but with the rather obvious rider that it is overwhelmingly intended to be for good. For those faced with the daily consumption of Fantasia-inspired bucketloads of information it can be useful to ask if you truly understand what the information is telling you? But we also suggest you ask: what is it not telling you?

At Martialis, our view is that practically none of the data ‘manipulations’ used to create 1st and 2nd line reporting were ever devised to “dupe the unwary” as Lee Baker put it. In truth, they’re purpose-built, specifically designed to guide and inform, and firms spend millions trying to ensure this. Despite this, our experience has shown that even the most perfectly presented reporting, containing data of the highest quality (clean), can still dupe both the wary and the unwary, and this can create problems if left unchecked.

What has led us to this seemingly contentious conclusion?

It would be easy to answer this by simply reminding readers that humans are fallible, but there are often deep-seated reasons why data can cause problems, which we broadly catalogue as:

Complacency, inertia and variability of experience;
Misinterpretations, and/or misunderstandings;
Misadventure, not exactly “the dog ate my report,” but variants that might surprise, including reporting-line and staffing changes, particularly of management;
Information overload, a not uncommon problem in a heavily regulated industry; and
Lack of reporting completeness and/or modernity.

And it’s this last category that I will expand on today: the lack of completeness and/or modernity in the 3rd-line space, drawing on recent work with clients and the directions we have been taking in our Dealing Desk Review practice.

Lessons from 2004

In my September blog-post I referred to a difficult moment in 2004-finance that shocked dealing desks across Australia and elsewhere (and upon which information can be readily found with any search-engine). What we know from the incident is that dealing staff sought and found ways to conceal unauthorised activity. We also know that senior dealing management’s interest in the underlying activities of the desk proved insufficient, and there were elements of complacency, misinterpretation of data, and some information-overload thrown in for good measure (regulators did eventually note some 800 individual limit breaches had been concealed; so, data was likely piling up somewhere).

However, and this is key; the reporting at the firm which suffered the eventual loss did not produce red-flags of sufficient magnitude at high-enough points on the management-hierarchy to back-up those brave risk management professionals who did alert management to a suspected desk-out-of-control problem.

At this point I will re-emphasise the point I made in my September blog: an independent review of the flags that were being raised in 2003 and ‘04 would likely have saved millions.

Bigger lessons still

Through relatively simple research, we found that since Y2k there have been at least 47 dealing-desk losses of greater than US$100m magnitude, including the Australian loss of our blog. What’s striking is that a clear majority of these were experienced by well-regarded and major global financials.

Of these:

Total losses (of the 47) amounted to U$101.6B at mid-2021 values,.
The largest amounted to U$11.4 billion (2008).
A great many more (that we couldn’t research) comprised losses of less than US$100 million.
Of losses (by the amount lost):

o 78.7% were generated by firms that had a Big-4 audit relationship;

o 39.3% were generated by a Systematically Important Bank; and

o 10.4% involved non-linear derivative products (somewhat surprisingly)

Not all involved fraudulent dealing activity.

Which has, at least in part, encouraged us to expand on the client-driven demand for Martials approach to data-analysis completed in prior years in this specialist field.

What can we deploy (now)

Having commenced desk reviews in 2019, we have steadily grown the areas of this interesting capability, with focus on three broad fields:

1. Discovery – developing a proprietary base-line view of the desk on which the need for deeper review can be determined:

a. Type of desk

b. Unusual or atypical attributes – returns/risk/other

c. Peculiarities versus norms/peers

2. Desk Conformity, with/to:

a. The implied or present regulatory environment

b. Industry standards

c. Market best-practice

d. Policy

3. Desk Alignment, with/to:

a. Strategic intent, e.g., sources of revenue and/or risk, type of desk

b. Adjacent businesses, as stand-alone or as a component with

c. The degree of complementarity

Answering interesting questions

Does the desk generate actual outcomes consistent with its mandate and stated strategy?
Does the desk demonstrate sustainability from a range of Martialis proprietary standpoints?
Are there elements of performance/risk/funding that warrant deeper review?
In generating revenue, is the desk conforming with regulatory and industry standards and expectations?
Does the activity entail supervisory corner-cutting, individual or entity overreliance, or elements of questionable activity?

In the process, we catalogue and flag areas where a desk may be adrift from industry best-practice standards and methods. Our focus is on the alignment to your strategy and how this compares with similar desks at other firms.

We look at the performance over a number of years and your assumptions of revenue sources against the actual revenue in the Discovery phase. Our experience suggests assumptions and performances are often poorly correlated and a detailed investigation can improve business performance.

Ross Beaney