Quality Control: How to check for errors in your investment performance

Sean P. Gilligan
Author
May 5, 2021
15 min
Quality Control: How to check for errors in your investment performance

Recent investment performance calculation mistakes at Pennsylvania Public School Employees’ Retirement System (“PSERS”) have highlighted the importance of quality control reviews and raises questions about where risk exists, how these risks can be mitigated, and what role independent verifications should play in the quality control process.

What happened at PSERS?1

An error in the return calculation for Pennsylvania’s $64 billion state public school employee retirement plan has had serious implications for its beneficiaries and those involved in the calculation mistake.

In 2010, the plan, which was already underfunded, entered into a risk-sharing agreement where employees hired after 2011 would pay more into the plan if the return (average time-weighted return) over a specific time period fell below the actuarial value of asset (AVA) return of 6.36%.

In December 2020, the board announced that the plan had achieved a return of 6.38%, a mere 2 basis points above the minimum threshold. But in March the board changed its tune, announcing that the calculation was incorrect and the 100,000 or so employees hired since 2011 (and their employers) should have actually paid more into the plan.

What’s worse is PSERS also announced that the FBI is investigating the organization, although details of the probe have not yet been released.

According to PSERS, a consultant, that had calculated the return, came forward and admitted to the calculation error. But the board also said that it is looking into potential cover up by its staff. From what we know, at least 3 independent consultants were involved in providing data used for the calculations, calculating the returns, and verifying the returns. So, with all these experts involved, how could this happen and what can your firm do to avoid a similar situation?

Key issues to address in an investment performance quality control process

Firms should develop sound quality control processes to help identify errors before results are published. Often these processes either do not exist or are insufficient to identify issues. Following a robust quality control process that considers the key risks involved and then finds ways to mitigate these risks greatly increases the accuracy of presented investment performance.

Although we do not yet know the cause of the errors found in the PSERS case, we can highlight a few primary reasons errors occur in investment performance reporting. Primarily, errors found in published performance results are caused by:

  • Key Issue # 1 – Issues in the underlying data (e.g., incorrect or missing prices, unreconciled data, missing transactions, misclassified expenses, or failing to accrue fixed income)
  • Key Issue #2 – Mistakes in calculations (e.g., manual calculations that fail to match the intended methodology)
  • Key Issue #3 – Errors in reporting (e.g., publishing numbers that do not match the calculated results)

A robust quality control process should specifically address all three of these areas.

Considerations when designing a robust quality control process

Key Issue #1 – Issues in the underlying data

As they say, garbage in, garbage out. It is important to ask and address questions confirming the validity of data before it is used to calculate performance. Specifically, consider how the data used in the calculations is gathered, prepared, and reconciled before completing the calculations. Is there any formal signoff from the operations team confirming that the data is ready for use? Has a review of the data been conducted by an operations manager prior to this confirmation being made?

While deadlines to get performance published can be tight, taking the time to ensure that the underlying data is final and ready to use before performance is calculated can prevent headaches later on.

The following is a list of issues to look for when testing data validity:

  • Outlier performance – Portfolios performing differently than their peers may indicate a data issue or that the portfolio is mislabeled (i.e., tagged to a different strategy than it is invested in).
  • Differences between ending and beginning market values – Generally, we expect a portfolio’s market value at the end of one month and the beginning of the next month to be equal (unless using a system where external cashflows are recorded between months and differences like this are expected). Flagging differences can help identify data issues.
  • Offsetting decrease/increase in market value – Market values that suddenly increase or decrease and then return to the original value may have an incorrect price or transaction that should be researched.
  • Gaps in performance – A portfolio whose performance suddenly stops and then restarts may have missing data.
  • 0% returns – The portfolio may have liquidated and may no longer be under the firm’s discretionary management.
  • Very low market values – The portfolio may have closed and is only holding a small residual balance, which should be excluded from the firm’s discretionary management.
  • Net-of-fee returns higher than gross-of-fee returns – Seeing net returns that are higher than gross returns could indicate a data issue unless there are fee reversals you are aware of (e.g., performance fee accruals where previously accrued fees are adjusted back down).
  • Gross-of-fee returns and net of-fee returns are equal – If gross-of-fee and net-of-fee returns are always equal for a fee-paying portfolio, it is likely that the management fees are paid from an outside source (paid by check or out of a different portfolio). The returns labelled as net-of-fee in a case like this should be treated as gross-of-fee returns.

Key Issue #2 – Mistakes in calculations

Mistakes happen, but there are ways to reduce their frequency and impact. First, you’ll want to consider how manual your performance calculations are as well as the experience of the person completing the calculations.

Let’s face it, Excel is probably the most widely used tool in performance measurement, especially for smaller firms. While many firms likely find Excel to be a user-friendly tool for calculating performance statistics, it has its limitations. Studies have shown that up to 90% of spreadsheets contain errors and spreadsheets with lots of formulas are even more likely to contain mistakes. Whether it’s not properly dragging down a formula or referencing the wrong cell, fundamentally, the biggest problem is that users do not check their work or have carefully outlined procedures for confirming accuracy.

Although this may seem obvious, having a second set of eyes on a spreadsheet can save you from the embarrassing headache of having to explain errors in performance calculations. It is even better if this review is a multi-layered process. Having someone review details as well as someone to do a high-level “gut-check” to make sure the calculations and results make sense can reduce this risk. Depending on the size of your firm, this may be easier to accomplish with a third-party consultant, where you serve as a final layer of review.

Having this final “gut-check” can help prevent avoidable errors prior to publication. We find that this final “gut-check” is best performed by someone who knows the strategy intimately rather than a performance or compliance analyst, as these individuals may be too focused on the calculation details to take a step back and consider whether the returns make sense for the strategy and are in line with expectations.

If you use software to calculate performance, you can significantly reduce the risk of manual error, but due diligence should still be performed from time to time to manually prove out the accuracy of the calculations completed in the program. This does not need to be done every time but should be conducted when introducing a new software system and any time changes are made to the program.

Key Issue #3 – Errors in reporting

It may seem silly, but many performance reporting errors come from transposing strategy and benchmark returns in presentations or placing the return of one strategy in the factsheet of another. Therefore, it is important to consider how the final performance figures make it from the system or spreadsheet into the performance presentations. Are they typed? Copy and pasted? Or are the performance reports generated directly out of a system? It’s not enough to complete the calculations correctly, the final reports must also be accurate, so adding a step to review this is crucial.

A similar review process to the one described above can really make a difference, but ultimately, understanding the vulnerabilities of your performance reporting will help you design quality control procedures that address any exposure.

Calculations completed by external performance consultants

Whether performance is calculated internally or by a third-party performance consultant, the same key issues should be considered when designing the quality control process. Due diligence should be done on the performance consulting firm to evaluate the level of experience the firm has with calculating investment performance and what kind of quality control process they follow prior to providing results to your firm. This information will help you determine what reliance you can place on their procedures and what your firm should still check internally.

For example, outsourcing performance calculations to an individual or single-person firm likely necessitates a more in-depth review since this individual would not have the ability to have a second set of eyes on the results prior to providing them to your firm. However, even larger performance consulting firms with robust quality control processes may not have intimate knowledge of your strategies, meaning that, at a minimum, a final “gut-check” should be done by your firm prior to publication.

Reliance on independent performance verification firms to find errors

Many firms that hire performance verification firms rely on their verifier to be their quality control check; however, this may not be a good practice for a variety of reasons. If this is a common practice at your firm, you may want to check the scope of your engagement before relying too heavily on your verifier to find errors.

Verification is common for firms that claim compliance with the Global Investment Performance Standards (GIPS®). But even firms that claim compliance with the GIPS standards and receive a firm-wide verification are required to disclose that, “…Verification does not provide assurance on the accuracy of any specific performance report.

This is because verifiers are primarily focused on the existence and implementation of policies and procedures. While their review may help identify errors that exist in the sample selected for testing, it specifically does not certify the accuracy of presented results. While the verification process is valuable and often does turn up errors that need to be corrected, regardless of the scope of your engagement, a robust internal quality control process is likely still warranted.

Firms that are not GIPS compliant may engage verification firms for various types of attestation or review engagements like strategy exams or other non-GIPS performance reviews. In these situations, the scope of the engagement may be customized to meet the needs (and budget) of the firm seeking verification. A clear understanding of exactly what is in-scope and specifically what the verifier is opining on when issuing their report is key.

Situations where the engagement entails a detailed attestation tracing input data back to independent sources, confirming that calculations are carried out consistently, and verifying that published results match the calculations, allow for heavy reliance on the verifier as part of your quality control process.

Alternatively, when the scope merely consists of a high-level review confirming the appropriateness of the calculation methodology, a much more robust internal quality control process should be applied.

Knowing the scope of the engagement your firm has established with the verification firm is an important element in determining how much reliance can put on their review and findings, which can then be incorporated into the design of your own internal quality control procedures.

Key take-aways

Mistakes happen in investment performance reporting, but a robust quality control process can greatly mitigate this risk. Understanding the risks that exist, designing processes to test these risk areas, and understanding the role and engagement scope of all consultants involved are essential items in designing a quality control procedure that work for your firm – and hopefully one that will help you avoid situations like what happened with PSERS.

If you are not sure where to begin, we have tools and services available to help. Longs Peak uses proprietary software to calculate and analyze performance. Our software helps flag possible data issues and outlier performers and also produces performance reports directly from our performance system.

In addition, our performance consultants are available to work with your team to help identify potential vulnerabilities in your performance reporting process and can help you develop better quality control procedures, where needed.

Questions?

If you would like to learn more about our quality control process or any of the services we offer (like data and outlier testing) to help improve the accuracy and reliability of investment performance, contact us or email Sean Gilligan directly at sean@longspeakadvisory.com.

1 For more information on PSERS, please see this article from the Philadelphia Inquirer.

Recommended Post

View All Articles
Key Takeaways from the 2025 PMAR Conference
This year’s PMAR Conference delivered timely and thought-provoking content for performance professionals across the industry. In this post, we’ve highlighted our top takeaways from the event—including a recap of the WiPM gathering.
May 29, 2025
15 min

The Performance Measurement, Attribution & Risk (PMAR) Conference is always a highlight for investment performance professionals—and this year’s event did not disappoint. With a packed agenda spanning everything from economic uncertainty and automation to evolving training needs and private market complexities, PMAR 2025 gave attendees plenty to think about.

Here are some of our key takeaways from this year’s event:

Women in Performance Measurement (WiPM)

Although not officially a part of PMAR, WiPM often schedules its annual in-person gathering during the same week to take advantage of the broader industry presence at the event. This year’s in-person gathering, united female professionals from across the country for a full day of connection, learning, and mentorship. The agenda struck a thoughtful balance between professional development and personal connection, with standout sessions on AI and machine learning, resume building, and insights from the WiPM mentoring program. A consistent favorite among attendees is the interactive format—discussions are engaging, and the support among members is truly energizing. The day concluded with a cocktail reception and dinner, reinforcing the group’s strong sense of community and its ongoing commitment to advancing women in the performance measurement profession.

If you’re not yet a member and are interested in joining the community, find WiPM here on LinkedIn.

Uncertainty, Not Risk, is Driving Market Volatility

John Longo, Ph.D., Rutgers Business School kicked off the conference with a deep dive into the global economy, and his message was clear: today’s markets are more uncertain than risky. Tariffs, political volatility, and unconventional strategies—like the idea of purchasing Greenland—are reshaping global trade and investment decisions. His suggestion? Investors may want to look beyond U.S. borders and consider assets like gold or emerging markets as a hedge.

Longo also highlighted the looming national debt problem and inflationary effects of protectionist policies. For performance professionals, the implication is clear: macro-level policy choices are creating noise that can obscure traditional risk metrics. Understanding the difference between risk and uncertainty is more important than ever.

The Future of Training: Customized, Continuous, and Collaborative

In the “Developing Staff for Success” session, Frances Barney, CFA (former head of investment performance and risk analysis for BNY Mellon) and our very own Jocelyn Gilligan, CFA, CIPM explored the evolving nature of training in our field. The key message: cookie-cutter training doesn't cut it anymore. With increasing regulatory complexity and rapidly advancing technology, firms must invest in flexible, personalized learning programs.

Whether it's improving communication skills, building tech proficiency, or embedding a culture of curiosity, the session emphasized that training must be more than a check-the-box activity. Ongoing mentorship, cross-training, and embracing neurodiversity in learning styles are all part of building high-performing, engaged teams.

AI is Here—But It Needs a Human Co-Pilot

Several sessions explored the growing role of AI and automation in performance and reporting. The consensus? AI holds immense promise, but without strong data governance and human oversight, it’s not a silver bullet. From hallucinations in generative models to the ethical challenges of data usage, AI introduces new risks even as it streamlines workflows.

Use cases presented ranged from anomaly detection and report generation to client communication enhancements and predictive exception handling. But again and again, speakers emphasized: AI should augment, not replace, human expertise.

Private Markets Require Purpose-Built Tools

Private equity, private credit, real estate, and hedge funds remain among the trickiest asset classes to measure. Whether debating IRR vs. TWR, handling data lags, or selecting appropriate benchmarks, this year's sessions highlighted just how much nuance is involved in getting private market reporting right.

One particularly compelling idea: using replicating portfolios of public assets to assess the risk and performance of illiquid investments. This approach offers more transparency and a better sense of underlying exposures, especially in the absence of timely valuations.

Shorting and Leverage Complicate Performance Attribution

Calculating performance in long/short portfolios isn’t straightforward—and using absolute values can create misleading results. A session on this topic broke down the mechanics of short selling and explained why contribution-based return attribution is essential for accurate reporting.

The key insight: portfolio-level returns can fall outside the range of individual asset returns, especially in leveraged portfolios. Understanding the directional nature of each position is crucial for both internal attribution and external communication.

The SEC is Watching—Are You Ready?

Compliance was another hot topic, especially in light of recent enforcement actions under the SEC Marketing Rule. From misuse of hypothetical performance to sloppy use of testimonials, the panelists shared hard-earned lessons and emphasized the importance of documentation. This panel was moderated by Longs Peak’s Matt Deatherage, CFA, CIPM and included Lance Dial, of K&L Gates along with Thayne Gould from Vigilant.

FAQs have helped clarify gray areas (especially around extracted performance and proximity of net vs. gross returns), but more guidance is expected—particularly on model fees and performance portability. If you're not already documenting every performance claim, now is the time to start.

“Phantom Alpha” Is Real—And Preventable

David Spaulding of TSG, closed the conference with a deep dive into benchmark construction and the potential for “phantom alpha.” Even small differences in rebalancing frequency between portfolios and their benchmarks can create misleading outperformance. His recommendation? Either sync your rebalancing schedules or clearly disclose the differences.

This session served as a great reminder that even small implementation details can significantly impact reported performance—and that transparency is essential to maintaining trust.

Final Thoughts

From automation to attribution, PMAR 2025 showcased the depth and complexity of our field. If there’s one overarching takeaway, it’s that while tools and techniques continue to evolve, the core principles—transparency, accuracy, and accountability—remain as important a sever.

Did you attend PMAR this year? We’d love to hear your biggest takeaways. Reach out to us at hello@longspeakadvisory.com or drop us a note on LinkedIn!

ColoradoBiz Names Longs Peak’s Jocelyn Gilligan, CFA, CIPM as a Gen XYZ Top Young Professional
Longs Peak is pleased to announce that Partner and Co-Founder, Jocelyn Gilligan has been named a GenXYZ Top Young Professional by ColoradoBiz Magazine. As ColoradoBiz states, “They’re uncommon achievers, whether as entrepreneurs, CEOs, nonprofit leaders, visionaries critical to their companies’ success or, in some cases, all of those roles. This year’s Top 25 Young Professionals figure to continue making a difference professionally and in their communities for years to come.”
March 14, 2023
15 min

Longs Peak is pleased to announce that Partner and Co-Founder, Jocelyn Gilligan has been named a GenXYZ Top Young Professional by ColoradoBiz Magazine.

As ColoradoBiz states, “They’re uncommon achievers, whether as entrepreneurs, CEOs, nonprofit leaders, visionaries critical to their companies’ success or, in some cases, all of those roles. This year’s Top 25 Young Professionals figure to continue making a difference professionally and in their communities for years to come.”

Jocelyn grew up in Boulder, CO and graduated from the University of Colorado. She started her career at Ernst & Young in New York City where she worked on their Financial Services Transfer Pricing Team. She transferred with EY to their office in Shanghai and then eventually to Hong Kong. Jocelyn left EY as a Manager and relocated back to Colorado where she and her husband started a family. Soon thereafter, Jocelyn and Sean founded Longs Peak out of a small one-car garage in their home in Longmont, CO. Now running a thriving team of 14, Jocelyn has weathered the ups and downs of entrepreneurship. She credits a lot of their success to their amazing team and the community of entrepreneurs they live near and network with (Longs Peak is an active member of EO (Entrepreneurs Organization)).

Jocelyn is a voting member of the PTO at her children’s school and a member of Women in Investment Performance Measurement, a group recently founded to support women in the investment performance industry.

About ColoradoBiz’s Top 25 Young Professionals

The 13th annual Gen XYZ awards is open to those under 40 who live and work in Colorado — numbered in the hundreds, making for difficult decisions and conversations among judges, as always. Applications were judged by our editorial board based on career achievement, community engagement and their stories of how they got to where they are now.

About Longs Peak

Longs Peak is a purpose and values-driven company. It is our mission to make investment performance information more transparent and reliable—empowering investors to make better, more informed investment decisions.

At the onset, we were looking to help smaller investment managers by giving them access to professional performance experts and tools typically only available to very large firms. We know that our work enables emerging managers to compete with the big guys and helps facilitate their growth. We strive to be our clients’ most valued outsource partner and to be known for our exceptional client service. We know that providing exceptional client service means that we must first create a culture that lives by the ideals we are trying to create for our clients. A place where incredibly talented individuals are empowered to put their best work into the hands of clients that truly value what we do. As a firm, we recognize that our greatest asset is people – both those we work with and those we work for. We continue to evolve into something that represents the needs of both of these groups and hope someday a GIPS Report is provided to every prospective investor in the world.

SEC Clarifies Marketing Rule: Gross-of-Fee Returns Allowed Under Certain Conditions
The investment management industry has spent significant time grappling with the SEC’s Marketing Rule and the question of whether gross-of-fee returns can be presented without corresponding net-of-fee returns in certain cases. Many firms have invested resources in trying to allocate fees to individual securities and sectors in an effort to comply. However, the SEC has now issued two FAQs (March 19, 2025) that provide much appreciated clarity on extracted performance and portfolio characteristics. The key takeaway? It is possible to present gross-of-fee returns without net-of-fee returns—if certain conditions are met.
March 27, 2025
15 min

The investment management industry has spent significant time grappling with the SEC’s Marketing Rule and the question of whether gross-of-fee returns can be presented without corresponding net-of-fee returns in certain cases. Many firms have invested resources in trying to allocate fees to individual securities and sectors in an effort to comply. However, the SEC has now issued two FAQs (March 19, 2025) that provide much appreciated clarity on extracted performance and portfolio characteristics. The key takeaway? It is possible to present gross-of-fee returns without net-of-fee returns—if certain conditions are met.

Extracted Performance: Gross Returns Can Stand Alone Under Specific Criteria

Investment advisers often present the performance of a single investment or a subset of a portfolio (“extracted performance”) in marketing materials. Historically, the SEC required both gross and net performance to be shown for such extracts. The new guidance provides a pathway for firms to display only gross-of-fee extracted performance, provided the following conditions are met:

  1. The extracted performance must be clearly identified as gross performance.
  2. The advertisement must also present the total portfolio’s gross and net performance in a manner consistent with SEC requirements.
  3. The total portfolio’s performance must be given at least equal prominence to, and facilitate comparison with, the extracted performance.
  4. The total portfolio’s performance must be calculated over a period that includes the entire period of the extracted performance.

If these conditions are satisfied, the SEC staff has indicated they will not recommend enforcement action, even if the extracted performance is presented without corresponding net returns. This is a notable shift, as it allows firms to avoid the complex and often impractical task of allocating fees at the investment or sector level.

Portfolio and Investment Characteristics: Net-of-Fee Not Always Required

Another common industry question has been whether certain portfolio or investment characteristics—such as yield, volatility, Sharpe ratio, sector returns, or attribution analysis—constitute “performance” under the marketing rule, and if so, whether they must be presented net of fees.

The SEC’s latest guidance acknowledges that calculating these characteristics net of fees can be difficult and, in some cases, may lead to misleading results. As a result, the staff has confirmed that firms may present gross characteristics alone, without net characteristics, if they meet the following criteria:

  1. The characteristic must be clearly identified as calculated without the deduction of fees and expenses.
  2. The advertisement must also present the total portfolio’s gross and net performance in a manner consistent with SEC requirements.
  3. The total portfolio’s performance must be given at least equal prominence to, and facilitate comparison with, the gross characteristic.
  4. The total portfolio’s performance must be calculated over a period that includes the entire period of the characteristic being presented.

As with extracted performance, these conditions help ensure that the presentation is not misleading, reducing the risk of enforcement action.

Bottom Line: A Practical Path Forward

This updated SEC guidance provides much-needed flexibility for investment managers, allowing for the presentation of gross-of-fee returns in a compliant manner. Firms that clearly disclose their approach and follow the specified conditions can reduce compliance burdens while still meeting investor protection standards. While this does not eliminate all complexities of the Marketing Rule, it does offer a practical solution that allows for more straightforward and meaningful performance reporting.

For firms navigating these changes, ensuring clear disclosures and maintaining compliance with the general prohibitions of the rule remains critical. Those who align their advertising materials with these guidelines can now confidently use gross-of-fee performance in a way that is both transparent and in compliance with regulatory requirements.

Questions?

If you have questions about calculating or presenting investment performance in a manner that complies with regulatory requirements or industry best practices, we would love to talk to you. Please feel free to email us at hello@longspeakadvisory.com.