Shadows in the Comprehensive Assessment
di Emilio Barucci, Roberto Baviera, Carlo Milani

Feb 22 2015
Shadows in the Comprehensive Assessment <small><small><I>di Emilio Barucci, Roberto Baviera, Carlo Milani </I></small></small>

When dealing with the Comprehensive Assessment (CA) the key question is: was the CA really comprehensive? But why this question is so important?

In our precedent columns on this site, evaluating the determinants of the shortfall of the CA, we have showed that there is some evidence of double standards: a different approach with respect to credit activity and financial assets, a different approach with respect to core and non-core countries, a penalization of non diversification-home bias effect, the fact that national discretion has played a role (Barucci, Baviera and Milani, 2015a, 2015b). One can discuss at length whether such double standards should be there and the possible rationale, one can also claim that an analysis of the adjustments (instead of the shortfall) may contain more (or complementary) information about the assessment made by the ECB. However, having a non-comprehensive CA can be simply dangerous; in this contribution we investigate this point.

An instance could clarify this point which is crucial whenever one analyses banking regulation issues. Let us suppose that regulator focuses just on one class of risk (e.g. credit exposure) neglecting the others when analyzing two banks (A & B) with the same business structure; let us also imagine that bank A is more clever in balance sheet management techniques allowed by the most advanced financial engineering tools (securitization without a real risk transfer to third parties, off balance sheet vehicles, level 3 assets, etc..). Clearly, in such a case bank A could pass more easily than bank B any “exam” from the banking supervision authority: this could be a tremendous incentive for regulatory arbitrage increasing financial products and balance sheet complexity.

One of the main objectives of CA is to enhance transparency (European Central Bank, 2014a). We observe two main limits in the exercise, which are related to the use of an incomplete information set and to some features in the methodology that has been followed by the authority.

With respect to the first issue, two key elements, probably not adequately analyzed in the CA, are level 3 assets and off-balance sheet items. As it is well known, both elements have played a crucial role in the current financial crisis.

Level 3 assets should have been analyzed in detail during work block 8 in AQR phase 2 (e.g. see figure 1 at page 10 in European Central Bank, 2014b). They amount to €178 billion (bln) for the 130 banks that underwent the AQR. The Fair Value (FV) analysis has divided level 3 assets in two sets: Non-Derivative assets (NDL3) and Derivatives (DL3). The FV of NDL3 for the banks in the panel amounts to €83 bln; the AQR impact has been of €1.4 bln (1.5% of NDL3 assets). ECB declared that “the vast majority of positions in-scope for independent revaluation were priced using cash flow discounting methodologies” and “the main differences in value were identified as a result of an increase in spreads used for cash flow discounting” (see page 97 in European Central Bank, 2014a). On this evaluation approach we observe the following: the valuation revision seems to be inadequate for assets that present scanty liquidity or that could involve rather complex modeling features (e.g. dependency structure for defaults in securitization assets), furthermore impacts on FV seem quite limited considering the characteristics of these assets. The results of the CA are even more astonishing considering the Derivatives’ set: the total impact is about €0.2 bln. If we consider that more than 60% of this amount comes from a single bank (Banque Populaire Caisse d’Epargne) the impact for the other banks is negligible. Let us recall that this set includes the most complicated derivatives with some parameters that cannot be calibrated through market data. For these assets, changing the valuation method can significantly affect the value. The AQR on this points does not appear to be deep enough also considering that some banks in the panel have created and commercialized some of the most complicated derivatives in the market place. Furthermore, we notice that differently from the credit part of the CA, there has been no statistical projection outside the selected portfolio.

Off-balance sheet items are taken into account by the CA only via the Total Assets according to Capital Requirement Regulation (TACRR) and the related leverage ratio. Unfortunately, the interpretation of the regulation does not appear adequate to enhance balance sheet transparency. An example helps to clarify the point. We refer to a bank whose off-balance sheet exposures amount to €200 bln (four times bank’s CET1), the bank’s TACRR is equal to €1440 bln,  the bank’s “Total Assets based on prudential scope consolidation” (the basic measure of total assets in the CA) is €1580 bln. Curiously enough, the measure of total assets including off-balance sheet items is smaller than the basic measure of total assets. Analyzing in detail the bank’s balance sheet, we observe that the decrease is due in particular to nettings for about €400 bln and a set of “adjustments” (e.g. the supervisory volatility adjustments approach) for further €160 bln. This example shows that it is not straightforward to isolate off-balance sheet items in the CA database, however their size is remarkably more significant than the total AQR credit provisions that, for the previously mentioned bank, amounted to €0.2 bln. In conclusion, the off-balance sheet represents an opaque area in the CA. A major UK bank has recently admitted an incorrect CET1 in the ST exercise due to a not-correct inclusion of Deferred Tax Assets (DTA) in the ST, (Fleming and Dunkley, 2014). The rules on DTA are quite simple; if a major bank can impact significantly its results just with improper calculations on DTA, the possibility of miss reporting on complex off-balance sheet structures that involves a significantly larger amount of funds can be really relevant.

From a methodological point of view, CA results are likely to be incomplete on Probabilities of Default (PD) and Losses Given Default (LGD) of banks’ counterparties. It seems that these parameters have been adjusted in the AQR process for the CVA part, but not in the computation of risk weighted assets according to IRB models (the only exception is related to the so called “Join-up”, see also page 35 of European Central Bank, 2014a). The AQR has analyzed in detail the Credit Value Adjustments (CVA) in banks’ balance sheets. On a CVA that amounts to more than €11.4 bln, the impact has resulted in a 27% increase. The most of the impact is related to the PD and the LGD parameters used in the CVA that the supervisor considered to be inadequate (see page 98 of European Central Bank, 2014a). We recall that PD and LGD of banks’ clients are also the main ingredients in IRB models for risk weighted assets computation: there is no rationale for the different approach to these parameters on CVA and IRB computation. This fact also explains why market considers a more reliable measure of risk the leverage ratio with respect to Common Equity Tier1 (CET1) ratio (Barucci, Baviera and Milani, 2014).

As already mentioned, CA appears to be more focused on banks’ credit risk than to market risk: market risk that has been analyzed mainly for the level 3 component and this analysis appears to be (at least) incomplete. Potentially, this fact may negatively affect future transparency and balance sheet strength. As a matter of fact, financial engineering allows modifying one kind of risk into the other via balance sheet management techniques; techniques that generally increase the complexity either within the balance sheet or using complicated off-balance sheet structures. A biased assessment by the supervisor could spread the use of such regulatory arbitrage techniques with very negative drawbacks.

Figure 1. Volatility vs CET1 shortfall/excess


 Source: Barucci, Baviera and Milani (2014).

In a recent analysis, Acharya and Steffen (2014) have questioned the ability of CA in capturing real risks: they find a negative relation between the shortfall and a market risk measure represented by SRISK, the market risk measure that they consider.

In Barucci, Baviera and Milani (2014) we find that at least a part of the risk – perceived by the market – is caught by CA capital deficit, although evaluated on a risk weights-based threshold, since we show a positive relation between capital deficit and volatility. In order to evaluate the capability of CA in capturing bank risk, Barucci, Baviera and Milani (2014) match the capital deficit that emerges from CA scaled by the common equity Tier 1 with a market risk measure represented by the market volatility, restricting their analysis on listed banks. In Figure 1 is reported the capital deficit, taking into account the 5.5% CET1 ratio threshold in the adverse scenario stress test (the capital deficit could be also negative in the case of banks that have a level of CET1 ratio higher than the threshold), in percentage of the CET1 by the end of 2013, and the historical volatility of stock returns, for the 41 listed banks in the EBA sample, for the period January-October 2014. The relation is positive, this fact shows that the CA was able to capture market risk.

Summing up, the most evident consequence of CA is the requirement of capital increases for some banks (mostly concentrated in Italy, Greece and Cyprus) and this fact has strengthened the European banking sector as a whole. However, in our opinion, there are some shadows in the CA exercise since we are in presence of a methodology with several pitfalls.



  • Acharya, Viral and Steffen, Sascha (2014) Benchmarking the European Central Bank’s asset quality review and stress test: a tale of two leverage ratios, VOX, november 2014.
  • Acharya, Viral, Engle, Robert, Richardson, Matthew (2014) Testing macroprudential stress tests: the risk of regulatory risk weights, Journal of Monetary Economics, 65: 36-53.
  • Barucci, Emilio, Baviera, Roberto and Milani, Carlo (2014) Is the Comprehensive Assessment really comprehensive?,
  • Barucci, Emilio, Baviera, Roberto and Milani, Carlo (2015a) What are the lessons of the Comprehensive Assessment? A first evidence,
  • Barucci, Emilio, Baviera, Roberto and Milani, Carlo (2015b) Further evidence on the Comprehensive Assessment ,
  • European Central Bank (2014a), Aggregate report on the comprehensive assessment, available on
  • European Central Bank (2014b), Asset quality review Manual Phase 2.
  • Fleming, Sam and Dunkley, Emma (2014) RBS ordered to call in auditors after admitting stress test error, Financial Times, November 21.

I commenti per questo post sono chiusi