Comparing Pension Plan Operating Costs: Correcting the Fraser Institute's Faulty Analysis and Conclusions

The Fraser Institute recently published a paper titled “Comparing the Costs of the Canada Pension Plan with Public Pensions Plans in Ontario”.ⁱ Based on a comparative cost analysis of six Canadian public sector pension organizations, the authors reached three conclusions that “should be sobering for the proponents of large public sector pension plans”:

1. “The paper finds no systematic relationship between a pension plan’s size and its [operating] costs”.
2. “In fact, there may be diseconomies of scale for large public sector pension plans because of the complexity of implementing their investment strategies….”
3. “These more aggressive strategies raise costs.”

Using the databases of the premier benchmarking organization in the global pensions space, this paper refutes the first two conclusions of the Fraser Institute paper, and places the third in a constructive context.

CEM Benchmarking Databases Indicate Significant Scale-Economies

Toronto-based CEM Benchmarking Inc. was founded in 1991. Today it is globally-recognized for the quality and size of its databases which measure ‘value-for-money’ in both the investment and member services/administration functions of pension organizations. It monitors over 500 pension organizations across the world, with aggregate assets exceeding $8 trillion.

As part of its client services, CEM designs and executes research projects using its databases, including on the question of economies of scale in both the investment and member services/administration functions. These studies have produced clear evidence of scale economies in both functions:

1. Investment costs as a percent of assets decline by 0.16 percentage points for every 10-fold increase in assets. So for example, on average, the investment costs of $100 billion pension plans are lower by 0.16 cents per dollar of assets than the costs of $10 billion pension plans with the same asset mix, and 0.32 cents per dollar of assets lower than the costs of $1 billion pension plans with the same asset mix. This finding is based on an international sample of 449 pension plans with assets values ranging from $100 million to $1 trillion. In terms of the statistical significance of the measured relationship, its ‘t-value’ was -14.6 (any ‘t-value’ below -2.0 or greater than +2.0 indicates the measured relationship is very unlikely to be due to chance).
2. Member services/administration costs per member also declines with size as fixed costs are spread over a bigger and bigger base. For example, based on a sample of 63, on average, a 200,000 member system has a $57 per member economy of scale advantage relative to a 100,000 member system. A 400,000 member system has a $29 per member advantage relative to the 200,000 member system. Once again, with a ‘t-value’ of -2.3, the strength of the statistical relationship exceeds the -2.0 value cut-off. 

Significant scale economies are only part of the story. Do they translate into greater ‘value-for-money’ for plan members?

Yes, they do. On average, based on a sample of 6666 annual observations, the pension plans in the CEM database outperformed their investment benchmarks by an average of 0.58% per annum before costs, and 0.16% after costs during the 22 year period 1992-2013 (‘t-value’ +17.8). Bigger funds performed better than average on a net basis, primarily because of their economies of scale advantage. Net value added increased by 0.08% for every 10-fold increase in assets due to lower investment management costs (‘t-value’ +2.0).  For more details, see the article “Value Added by Large Institution Investors Between 1992â€�2013” by Alex Beath on the CEM website.

Where Did the Fraser Institute Study Go Wrong?

So where did the authors of the Fraser Institute study go wrong? We identify three reasons: 

1. Too-small sample size: A simple rule of thumb in statistics is sample sizes should exceed 20 before it becomes reasonable to draw inferences from measured statistical relationships. The authors based their conclusions on a sample of six. The CEM findings were based on samples of 449, 63, and 6666 respectively.ⁱⁱ
2. Non-comparable data: Investment cost data in annual reports of pension organizations tends to be incomplete, inconsistently reported, and not fully comparable. Reported costs are incomplete because many funds do not include full private asset management fees, management fees paid to underlying ‘fund of funds’ managers, costs associated with real estate operating subsidiaries, and costs that may be netted from returns. On a best-effort basis, CEM attempts to standardize investment cost information before performing any analysis.ⁱⁱⁱ
3. Faulty interpretations: For example, the primary driver of member services/administration costs is the number of members in the plan, not the asset size of the plan. On the one hand, the authors acknowledge that the CPP’s member services/administration cost is low when compared on a per member basis. On the other, they also express this cost as a percent of CPPIB’s assets. This makes no sense as the CPP is only 17% funded. Thus CPP assets per member are a small fraction of the other five plans in the study (i.e., about $10,000 per member versus an average of $246,700 per member for the other plans). This means that when the CPP’s member-driven costs are reported as a percent of assets, they are unfairly inflated by a factor of 25 times. As another example, the authors assert that the “high returns earned by the CPPIB’s assets will not benefit its members”. This is incorrect: high returns reduce the cost of providing CPP benefits for both current and future CPP participants.

Generating ‘Value-for-Money’ Tomorrow

Finally, the authors suggest that the good investment results being reported by the six Canadian public sector pension organizations in recent years should be treated skeptically because private assets allow management some discretion in how these assets are valued. CEM research can give comfort here because its databases cover much longer time periods, thus minimizing the impact of ‘valuation discretion’ decisions. We already noted above that, on average, pension funds in CEM’s database have outperformed their benchmarks over the 1992-2013 period, and that bigger funds have outperformed smaller funds.^iv

As to their skepticism that higher-risk/cost private markets investing will pay off in higher long-term investment returns (after costs and risk adjustments), the authors fail to mention that Ontario Teachers’ has actually achieved this, generating a statistically significant net excess return of 2%/yr. over the course of the last 25 years.^v In addition to scale, OTPP’s documented success drivers have been organizational autonomy, mission clarity, good governance, and ability to insource its private markets investment programs. With OTPP’s proven long-term success record, these success drivers have been adopted by other Canadian pension organizations, and increasingly, by pension organizations outside Canada as well.^vi 

In Conclusion

In principle, we have no quarrel with an organization like the Fraser Institute insisting that Canada’s public sector organizations produce measurable ‘value-for-money’ for their stakeholders. However, to be credible, it should do so with studies that employ large, high-quality databases, sound analytical methods, and that draw logically- and statistically-valid conclusions from the study results.  

Keith Ambachtsheer is Director Emeritus of the International Centre for Pension Management at the Rotman School of Management, University of Toronto. He is also President of KPA Advisory Services Ltd, and a co-founder of CEM Benchmarking Inc. 

David Dupont is a Senior Analyst for CEM’s Investment Benchmarking service. He is a CFA charter holder and has a degree in Mathematics and Philosophy from the University of Toronto.

Tom Scheibelhut is the Managing Principal of CEM Benchmarking Inc.

Endnotes:

1. Philip Cross and Joel Emes (2016) ““Comparing the Costs of the Canada Pension Plan with Public Pensions Plans in Ontario”. Fraser Institute.
2. The biggest reason for differences in costs of similarly sized funds such as the ones used in the study is differences in asset mix. The authors are right to point out that the costs of large funds can grow faster than the savings from economies of scale because they are more likely to invest in private markets. This shift in asset mix can increase costs more than the savings from economies of scale because private assets are more costly to acquire and manage. Historically, funds have been rewarded for making this shift. For example, CEM’s research shows that private equity outperformed public market alternatives during the period 1996-2012 when implemented in the most cost effective manner. This outperformance ranged from 3.52% per annum for internal implementation and 0.28% per annum for investing directly in limited partnerships. Size opens up a larger opportunity set of assets to invest in cost-effectively.
3. We agree that more effort should be spent by the global pensions sector to produce cost data on a fully comparable basis. CEM is working co-operatively with a number of international organizations to that end.
4. This positive record contrasts sharply with that of the retail mutual sector, both in Canada and abroad. As logic would suggest, the average mutual fund underperforms its benchmark by its MER (management expense ratio), which can easily exceed 2%/yr. in Canada. See, for example, French (2008), “The Cost of Active Investing”, The Journal of Finance.  
5. See Ambachtsheer (2014), “The Case for Long-Termism”, Rotman International Journal for Pension Management, Fall. See also Taylor (2015), “These Canadians Own Your Town”, Fortune Magazine.
6. See The Economist (March 3, 2012), “Maple Revolutionaries”.