If you’re assessing your regulatory risk in light of recent developments, your approach to monitoring and supervision is an obvious place to start. Well-run licensees understand that effective reviews do more than identify deficient files. Considered collectively, review results can identify repeated weaknesses across advisers, products, strategies and processes, revealing broader compliance risks that no individual review result adequately captures.
Licensees fall when they ignore, or fail to recognise, that fact.
The reality is that one incomplete replacement analysis may be a distinct, manageable problem: a simple oversight, a failed check, an unusual process failure, or a one-off ‘brain explosion’.
But twelve incomplete replacement analyses aren’t usually twelve unrelated accidents, but they aren’t necessarily evidence of systemic failures.
After reviewing over 24,000 files, we believe effective and responsible governance requires licensees to take a more nuanced approach. Unfortunately, too many advice-review programs still treat each file as a self-contained event. The file is reviewed. Findings are recorded. The adviser responds. Remediation is completed. The file is closed. Then the process starts again.
That approach may identify individual advice failures. It may even produce reassuring statistics about how many files passed. What it may not tell the licensee is what keeps going wrong.
What should an advice review actually tell you?
Section 912A(1)(ca) of the Corporations Act requires an AFS licensee to take reasonable steps to ensure its representatives comply with the financial services laws, subject to specified exceptions.
ASIC’s RG 104 gives practical expression to that obligation by focusing attention beyond individual compliance failures. It asks licensees to consider both how they prevent failures from recurring and how they identify and address “systemic compliance failures or other trends in compliance issues”.
A file review is evidence about an individual piece of advice. A series of file reviews is potentially evidence about the effectiveness of the system producing that advice.
Those are different things.
An adviser failing to document why they rejected an alternative strategy adequately may require coaching. If several advisers make the same mistake, the problem may instead sit in the advice process, template, training, supervision or control design.
The individual finding tells you what happened. The pattern begins to tell you why it’s happening.
Why did July make this more important?
Our July 2026 Regulatory and Compliance Update identified that regulators and licensees are increasingly using aggregated information to identify concentrations, outliers and emerging risk.
ASIC’s product-governance interventions, FSC Standard 31, AFCA complaint information and other developments pointed in the same direction: regulated businesses increasingly have access to information that can expose concentrations, outliers and recurring behaviour before individual failures develop into widespread harm.
The July Update made the AFSL implication explicit: advice-review analytics should move beyond single-file pass rates and identify recurring deficiencies by adviser, product, advice type, lead source and control category.
This reinforces our fundamental belief that advice-file monitoring can’t stop at individual pass/fail outcomes. The value lies increasingly in connecting those individual observations and understanding what they say about the system producing the advice. The qualification to this approach is that more data only becomes useful when it changes supervision, controls or decisions.
Can you recognise this in your review program?
- the same finding keeps appearing after coaching or remediation;
- failures cluster around one adviser, product, strategy, referral source or process;
- management receives pass rates but cannot see recurrence or root cause;
- review findings cannot readily be reconciled with complaints, incidents or breaches; or
- nobody can evidence whether previous remediation changed subsequent behaviour.
When Does a File Finding Become a Pattern?
One deficient advice file is a finding; three similar files might be a pattern, but ten might still tell you less than one file that exposes a defective process affecting every client.
That’s the problem with treating advice-review results as simple numbers.
No universal law says three incidents make a pattern, five make an issue systemic, or ten make it material. We can simplify matters for convenience, but research from auditing, criminology, psychology, high-reliability organisations, and international regulators points to a more sophisticated approach; pattern, systemic failure, and materiality are different questions.
A numerical trigger is an escalation mechanism, not a conclusion. Its purpose is to require someone to investigate recurrence, concentration, cause, and exposure before closing another apparently isolated finding.
Let’s break this down by recognising that a pattern simply suggests that observations are recurring, concentrated or unusual. The “systemic” lens asks whether these issues share an underlying cause that could affect other cases. Materiality asks whether the actual or potential consequences are sufficiently important to require intervention.
Effective advice reviews need to consider all three aspects.
When does an error become a pattern?
Humans, as any astrologer or gambler will confirm, are pattern-seeking creatures. We instinctively impose order on randomness and see patterns in unrelated instances. Casinos thrive on this tendency.
So it’s tempting to infer patterns from the frequency of events, but that can be misleading. Seven recommendations of the same investment product might suggest adviser preference, but those seven clients might have had similar needs best served by the same product.
Three disclosure failures might indicate a problem. Or they might have been found because the review deliberately targeted an adviser already considered high risk.
The better question isn’t “how often did this happen?” but “how often did it happen compared with how often we should reasonably expect it to happen?”
Criminologists have wrestled with this issue for decades. Research consistently shows that crime clusters. A relatively small proportion of places, offenders and victims account for a disproportionate amount of crime. Recent research reviewing 35 years of crime-concentration studies found that around half of recorded crime occurred on approximately 4.5 per cent of street segments. Earlier systematic reviews reached the same broader conclusion: concentration matters because averages conceal where the underlying risk actually sits.
Advisers aren’t criminals, and we need to exercise caution here, but the application to advice supervision is obvious.
A 5 per cent failure rate spread randomly across an adviser population tells you something. The same number of failures concentrated in one adviser, product, advice strategy, referral source or client cohort tells you something quite different.
Concentration can reveal risk that aggregate numbers conceal, and that’s why an overall pass rate can provide false comfort.
Do three make a pattern?
Yes. No. Maybe. Can you repeat the question?
The only defensible answer to that question is “not necessarily”.
As I understand it, statistical disciplines generally reject arbitrary numerical rules in favour of identifying observations that depart from an expected baseline. The same principle should apply to advice reviews, but few licensees have determined their expected baseline.
If we focus on deviation rather than frequency, two occurrences can justify investigation if they share an unusual characteristic, but one occurrence can justify investigation if it is sufficiently anomalous. Ten occurrences may tell you very little if you don’t understand the baseline, denominator, the sampling method or the expected rate.
Audit may provide us with a particularly useful model. ASA 530 requires auditors to investigate the nature and cause of deviations or misstatements. More importantly, an auditor seeking to dismiss an identified error as an anomaly must obtain a high degree of certainty that it isn’t representative of the wider population.
It effectively reverses the evidential burden.
Instead of asking, “We only found this issue once. Why should we think it’s systemic?”, the question becomes: “We found this issue once. What evidence tells us it’s isolated?”
When does a problem become systemic?
This is where frequency becomes even less useful because “systemic”doesn’t mean numerous. It means that the cause extends beyond the incident already identified; the impact is broader, or the issue more widespread, than the identified failure point.
AFCA’s approach illustrates the distinction quite well. AFCA describes a systemic issue as one likely to affect consumers or small businesses beyond the complainant, and says firms should consider whether multiple complaints, or a single complaint, indicate a possible systemic issue. A system failure, defective process, incorrect contractual interpretation or recurring conduct may expose a wider population even if only one affected person has yet been identified.
Advice reviews should apply the same logic.
One defective Statement of Advice template used across the business may immediately become systemic. One error in an automated fee calculation may affect hundreds of clients. One review may establish that a mandatory control doesn’t exist.
Conversely, ten similar errors committed by one adviser might principally indicate an individual competence, conduct or supervision problem rather than a business-wide control failure.
Recurrence isn’t a prerequisite for systemic weakness, nor does recurrence prove one. It’s simply evidence that you need to investigate whether there’s a common cause and a broader impact.
Why do repeated small failures matter?
Audit offers another useful lesson. ASA 450 distinguishes matters that are genuinely inconsequential from matters that are merely individually small. Auditors accumulate identified misstatements and assess their effect individually and in aggregate. The standard expressly warns that “clearly trivial” isn’t another way of saying “not material”. Nature and circumstances matter as well as size.
That should resonate with anyone reviewing advice and, in fact, it’s a fundamental element of risk-based compliance.
A missing explanation of alternatives may have little apparent impact on one otherwise sound file. But if that’s repeated across twenty files, it may demonstrate that advisers aren’t properly considering alternatives, the template doesn’t require the analysis, training hasn’t worked, or reviewers have repeatedly identified a weakness without management fixing it.
Psychology and high-reliability research make repeated low-level failure more concerning. Organisations can gradually normalise deviance. Departures from expected practice become accepted because previous departures didn’t produce obvious harm. What was once regarded as an exception becomes “how we do things here”. High-reliability organisations deliberately resist this tendency by treating small failures and near misses as information about the health of the system.
There’s a tendency for some licensees and advisers to dismiss observations or trivialise non-compliance because, well, the world didn’t end when you found it last time. They take comfort from the fact that ASIC hasn’t intervened and complaints haven’t increased. It’s reasonable, although dangerous, to think “We’ve seen this before, and nothing happened” when a responsible gatekeeper should instead be asking “We’ve seen this before. Why is it still happening?”
When does the problem become significant?
Significance is another separate judgement.
A problem doesn’t become significant simply because it occurs a prescribed number of times. Nor should we confuse the broader governance concept of significance with the specific statutory test that applies to reportable situations.
There are really two related questions.
First, is the issue significant from a governance perspective?
That requires management to consider matters such as severity, potential consumer harm, the number of people exposed, recurrence, concentration, duration, vulnerability, control significance, detectability, reversibility and whether previous remediation has failed.
A single deliberate act of dishonesty may warrant immediate escalation. So might a small systems error that could affect thousands of clients. A recurring documentation deficiency may also become important if it demonstrates that a supervisory control isn’t operating effectively.
Second, does the issue trigger the statutory significance test under s 912D of the Corporations Act?
That is a separate legal assessment.
Some breaches of core obligations are deemed significant under s 912D(4). Where a breach isn’t deemed significant, s 912D(5) requires a licensee to consider factors including the number or frequency of similar breaches, the effect on the licensee’s ability to provide the financial services covered by its licence, and the extent to which the breach indicates that its compliance arrangements are inadequate.
The reportable situations regime expressly makes recurrence relevant to significance. ASIC also identifies monitoring for systemic issues and the number and frequency of similar breaches as an important function of a comprehensive incident and breach register. Repetition may also indicate a continuing underlying systemic problem.
That means apparently minor failures cannot always be assessed independently. A breach that looks insignificant on its own may look very different when management can see the same problem has occurred repeatedly.
What should advice-review data actually show?
Once you recognise this, it changes what a useful advice-review program should produce.
For example, knowing that 92 per cent of files “passed” tells a Responsible Manager very little unless they can also determine:
- What findings keep appearing? (Recurrence);
- Are they disproportionately associated with particular advisers, practices, products, strategies, referral sources or client groups? (Concentration);
- Are they increasing, decreasing or unchanged? (Trajectory);
- Do they reflect individual conduct, training, templates, technology, processes, incentives or control design? (Cause);
- Who else could be affected even though no problem has yet been detected? (Exposure);
- Do complaints, incidents, breaches, product data or other monitoring point in the same direction? (Corroboration);
- What was done? (Remediation);
- Did that response stop the problem recurring? (Effectiveness).
The UK’s Financial Conduct Authority essentially applies this approach to complaints. Its recent review emphasises granular data, root-cause analysis, identification of recurring and systemic problems, action to prevent recurrence and subsequent testing to establish whether the solution worked.
ASIC reported that 55% of reports in its initial reportable-situations analysis identified staff negligence or error as the sole root cause, including reports involving previous similar breaches or grouped breaches. ASIC questioned whether licensees were adequately identifying the underlying causes of repeated staff error.
“Human error” often describes what happened, but it isn’t necessarily the root cause. Unfortunately, in our experience, some licensees treat it as the final word on the topic.
Be careful with percentages.
There’s another trap that’s difficult to avoid
The reality is that many advice-review programs are deliberately “risk-based”. They select advisers, products, strategies or transactions because they present greater risk.
That makes them useful for finding problems, but not necessarily statistically representative of the whole advice population.
Think about the specific nature of targeted reviews. If 20 per cent of a deliberately high-risk sample fails, it doesn’t necessarily follow that 20 per cent of all advice provided by the licensee would fail.
Generalising from specifics is poor reasoning. Thankfully, audit sampling standards explicitly recognise a similar problem. Reasonable conclusions about a population ultimately depend on whether the sampling method provides a reasonable basis for drawing them.
The practical response isn’t to abandon risk-based monitoring but instead to use a detected pattern as a reason for targeted validation.
Find the signal. Define the potentially affected population. Then test whether the signal persists.
How should licensees set escalation triggers?
Numerical thresholds still have value, but they should trigger investigations, not drive automatic conclusions.
A practical framework might treat:
- one occurrence as requiring classification and consideration of broader exposure;
- two similar occurrences as requiring comparison of causes and common characteristics;
- three similar occurrences as a presumptive pattern requiring documented analysis;
- any unusual concentration as requiring investigation regardless of absolute numbers;
- any repeat finding after remediation as requiring escalation;
- any control-design failure capable of affecting a wider population as potentially systemic immediately; and
- any high-consequence event as requiring immediate significance assessment.
To be clear, there’s nothing scientifically magical about three occurrences. It’s not an optimised algorithm; it’s a rule designed to stop repeated weak signals being dismissed one file at a time.
Technology such as [complye] can make this easier by structuring review findings so licensees can analyse recurrence, concentration, cause, exposure and remediation rather than leaving observations trapped inside individual review reports.
But technology shouldn’t decide what the pattern means; that is, and should remain, a management responsibility.
What should Responsible Managers and boards receive?
Boards and Responsible Managers don’t need another spreadsheet containing hundreds of individual review findings. They need enough information to understand where risk is changing.
That means reporting should identify:
- recurring deficiencies and their frequency;
- concentrations by adviser, practice, product, strategy or other relevant factor;
- changes over time;
- links to complaints, incidents, breaches and other monitoring data;
- root causes;
- remediation undertaken;
- whether remediation was subsequently tested; and
- matters requiring changes to supervision, processes or controls.
The file is the starting point.
Recognising complexity doesn’t change the fact that advice reviews remain essential for assessing individual advice. Nor does it obscure the reality that their greater value comes from what those individual observations reveal when aggregated and considered together.
A good monitoring program therefore doesn’t simply ask how many files failed.
It asks:
- Is this happening more than we would expect?
- Where are issues concentrated?
- What’s causing it?
- Who else could be affected?
- How serious could the consequences be?
- And did our response prevent it from recurring?
Individual reviews still matter; it just isn’t always where the most useful signals sit.
But the review isn’t finished when the file is closed.
Advice reviews remain an essential way to assess individual advice quality, but that’s only one function. Taken together, review results can identify deteriorating behaviour, ineffective controls, emerging product risks, recurring process failures and problems that no individual file adequately reveals.
That’s where file review becomes supervision, and supervision becomes governance.
If your review findings are still trapped in individual reports, consider whether [complye] can give you the structured evidence needed to track recurrence, accountability, remediation and subsequent testing.
Further reading
Frequently Asked Questions
Not necessarily. An aggregate pass rate describes the outcomes of the files reviewed; it does not, by itself, establish that the licensee’s supervisory arrangements are effective.
A 90% or 95% pass rate may conceal failures concentrated around one adviser, practice, strategy, product, referral source or control. The sampling method also matters. Where reviews deliberately target higher-risk files, the result should not automatically be extrapolated across the entire advice population.
A Responsible Manager should therefore ask what sits behind the percentage: which findings recur, where they cluster, whether their incidence is changing, what causes them and whether previous remediation prevented recurrence. RG 104 reinforces this broader supervisory perspective by asking how licensees identify systemic compliance failures or trends and prevent failures from recurring.
Recurrence alone does not automatically determine reportability, but it can materially affect the assessment.
The governance question should first identify the underlying obligation, cause, affected or potentially affected population and consequences. The separate statutory assessment under s 912D then needs to be undertaken where relevant.
Importantly, licensees should not assess repeated incidents in artificial isolation. ASIC says a comprehensive incident and breach register helps a licensee monitor systemic issues and the number and frequency of similar breaches. ASIC has also specifically criticised cases where repeated problems were attributed merely to staff negligence or error without sufficient investigation of the underlying cause.
Practically, recurring findings should therefore trigger both root-cause investigation and consideration of whether the information changes an existing reportability assessment.
Yes. A systemic problem does not have to occur repeatedly before it warrants investigation as systemic.
The critical question is whether the underlying cause or potential effect extends beyond the identified case. A defective template used across an adviser population, for example, could expose many clients even if the weakness was first discovered in one reviewed file. Similarly, a defective automated calculation or missing mandatory control can have a broader potential impact immediately.
AFCA applies comparable reasoning to complaints: its guidance says firms should consider whether multiple complaints or a single complaint indicate a possible systemic issue, with the focus on whether other consumers or small businesses may be affected.
The governance response should therefore define the potentially exposed population before assuming that a single detected instance is isolated.
A defensible conclusion needs more than the fact that reviewers found the problem only once. The licensee should consider the denominator, sampling method, affected process, adviser or practice, product or strategy, possible common cause and whether other monitoring sources point to the same weakness.
That can require targeted validation outside the original review sample. For example, if one file exposes a defective replacement-analysis template, management could identify every adviser using that template and test an appropriate group of affected files. Complaints, incidents and breach data may provide additional corroboration.
The practical objective is not to prove that every isolated error is systemic. It is to obtain enough evidence to explain why management concluded that the problem is confined — or why broader investigation and remediation are required.
Technology can identify signals and organise evidence, but management still needs to determine what those signals mean.
Structured review data can make recurrence, concentrations, trends, root causes, exposure and remediation easier to identify. That matters because observations buried in individual reports are difficult to aggregate or test over time. ASIC similarly notes the value of comprehensive registers for monitoring systemic issues and the number and frequency of similar breaches.
But an alert that three files contain the same deficiency is evidence for investigation, not a regulatory conclusion. Management still needs to consider the sample, denominator, cause, consequences and affected population.
The useful distinction is simple: technology can tell management where to look. The licensee remains accountable for deciding what the evidence means and what response is required.
