Can an Algorithm Spot a Child at Risk?

Protecting children is non-negotiable. But scrutinising the technology we use to do it is part of protecting them. SAFE says machine learning can connect warning signs humans might miss. We look at what is publicly known about the system, what remains opaque, and why getting the context right matters as much as connecting the dots.

In this editorial

 

There are few technology problems where getting the answer wrong matters quite as much as this one.

Australian company SAFE is using machine learning in child protection, combining structured child-risk assessment with technology designed to help early childhood educators identify children who may be at risk of harm.

SAFE says its technology has been trained on 12 years of field data from every Australian state and territory. Its current website says data relating to 68,943 children has been assessed across 1,064 services. [Source: SAFE]

The premise is compelling.

An educator sees a bruise. Another notices a change in behaviour. Something concerning is disclosed in fragments over time.

Individually, those observations might mean little.

Together, they might reveal something nobody has recognised.

SAFE describes a system that captures observations, connects signals across time, prioritises risk and puts concerns before child-protection experts for review. SAFE itself describes the platform as “decision-support infrastructure” rather than a replacement for human judgement. [Source: SAFE Product]

Protecting our children is non-negotiable. How we use technology to do that is absolutely open to scrutiny.

Nobody seriously argues against identifying children at risk earlier or giving educators better tools to protect them.

The question is whether a particular system does that accurately, proportionately and transparently — and what safeguards exist when the information feeding it is incomplete or wrong.

SAFE is attempting to solve a very human problem: sometimes everybody sees a small piece of what is happening to a child, but nobody sees enough of the picture to realise that child needs help.

There is an uncomfortable inverse.

Sometimes those pieces, without enough context, can create a picture that isn’t actually true.

Five observations may be independent corroborating signals. They may also be several interpretations of the same underlying event or misunderstanding.

Connecting the dots is powerful. But only if the dots belong to the same picture.

Technology capable of doing that well could potentially save a child’s life.

That makes understanding how SAFE reaches its conclusions important.

Before preparing this analysis, DigitalReviews Network put detailed questions to SAFE’s public relations representatives covering how the machine-learning model weighs and validates observations, false-positive and false-negative rates, how contextual or subsequently corrected information affects risk assessments, whether families can challenge or correct information, how SAFE interacts with human judgement and mandatory reporting, and the retention and future use of child-safety data.

Those questions were sent on 11 August, with a follow-up on 13 August. At the time of publication, we have received neither an acknowledgement nor a response. The invitation for SAFE to respond remains open.

SAFE isn’t just keeping notes

SAFE isn’t simply a digital notebook for recording incidents.

Its current product material describes a system that turns observations into structured records, identifies patterns and maintains what SAFE calls a “Living Child Record”. [Source: SAFE Product]

SAFE says signals can be connected across time, people, rooms and services. Its current material also says participating organisations retain governance over their own records, with combined information handled subject to permissions and governance controls. [Source: How SAFE Works]

Those are meaningful safeguards.

But this remains a more consequential proposition than recording an incident.

It is decision-support infrastructure.

The potential value is obvious. So is the need to understand how those safeguards work in practice.

What exactly does the machine see?

HubHello’s publicly available SAFE material describes an assessment containing 41 questions intended to identify the level of risk and potential forms of abuse or neglect.

It says those questions were developed in consultation with specialists in child protection, trauma and case management. Machine-learning and language-processing technology then analyse the assessment and produce a risk prediction rating. [Source: HubHello SAFE]

There is an important qualification.

That HubHello material contains older references, while SAFE says its technology has continued evolving. SAFE’s current history describes a progression from labelled classifier training and threshold-based risk flagging through structured decision inputs, machine-learning probabilities, triage and expert review. We therefore cannot establish from public information that the current 2026 implementation is identical to the older HubHello workflow. [Source: SAFE Our Story]

But it raises an important point.

The questionnaire itself is part of the decision-making process.

Anyone who has completed a workplace psychometric assessment has encountered the principle. Such assessments aren’t simply collections of neutral questions. They are designed to elicit information about particular characteristics or predictors.

There is nothing inherently wrong with that. It is how a properly designed assessment instrument can work.

But if an assessment is specifically designed to identify risk of abuse or neglect, its design and validation matter.

How much subjective judgement is involved? Are some responses weighted more heavily? Does the assessment actively solicit information that might contradict a concern as well as information supporting one?

The fact that machine learning subsequently analyses those answers does not by itself make the resulting prediction objective.

The algorithm doesn’t have direct access to reality. Many of its inputs are observations and structured information that have already been selected, recorded or interpreted by people and systems.

Public information gives us some understanding of what goes in and what comes out.

SAFE’s own history provides more detail about its technical development, including labelled training data, classifier training, analysis of true and false positives and threshold-based risk flagging. [Source: SAFE Our Story]

That’s useful information.

But the part that matters most for independent evaluation — how particular observations are weighted, combined and translated into a risk prediction — remains opaque from the outside.

What does 96.2 per cent actually mean?

There is an eye-catching number attached to SAFE.

The Child Abuse Prevention Service’s 2021 annual report said SAFE had developed a 96.2 per cent accuracy rate “at predicting at-risk children”, adding that the figure had been verified by CAPS. [Source: CAPS Annual Report 2021, p.8]

HubHello and CAPS continue to publish the 96.2 per cent figure. [Source: HubHello SAFE] [Source: CAPS]

More recent SAFE material instead refers to approximately 96 per cent accuracy in behavioural pattern recognition. [Source: SAFE, 30 June 2026]

SecurityBrief Australia reported on 5 August 2026 that, according to SAFE, the platform had recorded a 97 per cent accuracy rating in government pilots. [Source: SecurityBrief Australia]

These figures may represent rounding of the same underlying metric. They may reflect different datasets, validation exercises or definitions of accuracy.

Without the methodology, we cannot tell.

Nor can we determine exactly what constituted an “at-risk” child in the original validation.

Was SAFE predicting a subsequently substantiated finding of abuse or neglect? Agreement with an expert reviewer? A historical child-protection decision? Something else?

We would also like to see measures illuminating false positives and false negatives.

A simple hypothetical illustrates why.

If 97 out of every 100 members of a population didn’t have the condition a model was attempting to detect, a useless model classifying everybody as negative would still achieve 97 per cent accuracy while identifying none of the positive cases.

That is not a claim about SAFE’s performance.

It demonstrates why a headline accuracy figure isn’t enough to independently evaluate a safety-critical predictive system.

The underlying validation methodology matters.

Rubbish in, rubbish out

Artificial intelligence can be extraordinarily powerful.

It doesn’t make poor information good.

I’m an engineer by background, and there’s an old problem here that predates machine learning.

Give me two data points and I can draw you a line. Give me a framework that determines which data points I collect, and I can potentially make that line tell a very particular story.

AI doesn’t magically solve that problem.

If the inputs are incomplete, unrepresentative or wrongly interpreted, sophistication further down the pipeline cannot magically restore the missing context.

In engineering shorthand: rubbish in, rubbish out.

Consider technology reviews and support forums.

When a product develops a fault, people go online to complain. When the same product works perfectly every day for three years, almost nobody posts a daily update saying so.

Look only at complaints and you can develop a distorted picture — not because the complaints are false, but because exceptional negative events are disproportionately recorded.

The stakes are vastly higher here, but the underlying data problem is worth considering.

If an educator records a concerning interaction at pickup, does the system meaningfully represent the hundred unremarkable pickups surrounding it?

If a child misses a meal, is the ordinary pattern represented as well?

Then there is the person recording it.

An educator at the end of a difficult day is still human. Another educator might interpret the same ambiguous interaction differently. The same person might describe it differently with additional context tomorrow.

The machine isn’t directly observing the child or family. At least part of what it analyses is another person’s representation of what occurred.

That creates a chain:

event → human observation → human interpretation → recorded information → machine interpretation → risk prediction → human review

Every transition is an opportunity for context to be added — or lost.

Machine learning may process information consistently, but consistency at the end of the pipeline doesn’t automatically correct variability at the beginning.

That raises another validation question: inter-rater reliability.

If the same hypothetical case were given to multiple educators, how consistently would they complete the assessment, and would SAFE classify the child consistently?

Validating a classifier is one thing. Validating the entire decision pipeline is another.

Context can change the picture

Take something as simple as a bruise.

Some children collect bruises with astonishing efficiency. Other bruising can be an important indicator that a child may be experiencing harm.

The observation matters.

So does the context.

Withdrawal, communication difficulties or emotional dysregulation could similarly form part of a concerning pattern. They could also have explanations unrelated to abuse or neglect, including developmental differences or emerging mental-health needs.

SAFE’s current material itself frames possible outcomes broadly: monitor, support, refer, escalate or report. That suggests the system’s intended role is wider than simply pushing concerns towards mandatory reporting. [Source: SAFE Product]

That reinforces the importance of interpretation.

Sometimes the question may be:

“Is this child at risk of abuse or neglect?”

Sometimes it may be:

“What does this child need?”

Those aren’t necessarily the same question.

Corroboration or repetition?

SAFE’s central proposition is that individually small observations can become significant when considered together.

Older HubHello material explicitly says assessments that do not individually reach the mandatory-reporting threshold can, when combined with other observations, increase the assessed risk level. [Source: HubHello SAFE]

That can be enormously useful.

It also makes the provenance and dependence of observations important.

Consider a high-conflict family dispute in which one person repeatedly raises concerns about another.

Those concerns may be legitimate. They may be mistaken or lack context. In some circumstances they may be deliberately misleading.

A child-safety system obviously cannot disregard a concern simply because parents are in dispute.

But persistence isn’t the same thing as independent corroboration.

If repeated concerns conveyed to educators are subsequently recorded as separate observations or assessment inputs, the system may see an accumulating pattern.

We have found no evidence that SAFE currently mishandles this scenario.

It is simply a governance and modelling question any system designed to accumulate weak signals needs to address.

Five genuinely independent observations pointing in the same direction may be highly significant.

Five records ultimately arising from the same disputed event are something different.

SAFE says its cross-service data handling is subject to governance controls. What isn’t apparent publicly is how its risk model accounts for the provenance and dependence of individual signals. [Source: How SAFE Works]

A child-safety system designed to connect weak signals therefore needs to distinguish an accumulating body of evidence from accumulating repetitions of the same underlying information.

Erring on the side of safety isn’t consequence-free

There is an understandable instinct in child protection to err on the side of safety.

If the choice is between investigating something innocent and failing to identify serious abuse, the latter possibility is horrifying.

But false positives aren’t harmless.

I know that from experience.

I know what it is like for a family to be subjected to reports that could not be substantiated, to struggle to challenge information when the reporting party is protected, and to discover that another report can begin the process again.

I’m not presenting my experience as evidence that SAFE gets these decisions wrong.

It explains why I don’t regard “erring on the side of safety” as the end of the discussion.

There is also an important distinction in the decision pipeline.

A SAFE risk flag is not itself a mandatory report, let alone a finding that abuse has occurred. SAFE says expert review may lead to monitoring, support, referral, escalation or reporting. A statutory child-protection response sits further downstream again. [Source: How SAFE Works]

But error at an earlier stage can still matter because these stages inform what happens next.

A risk assessment can eventually be downgraded. Context can emerge. A case can be closed.

In a child-protection process, a later correction does not necessarily erase the consequences of an intervention that has already occurred.

An unsubstantiated report should not automatically be treated as false or malicious. People must be able to raise genuine concerns without fearing consequences simply because an allegation cannot ultimately be proved.

But the reliability and provenance of signals still matter.

If later evidence changes the interpretation of an earlier concern, the risk model should reflect that context too.

The subject accumulates signals. The reliability of those signals deserves equivalent scrutiny.

What happens when the information is wrong?

What opportunity does a parent have to provide context, challenge an observation or correct something factually incorrect?

If an observation is subsequently explained, does that explanation become part of the Living Child Record?

Does the risk prediction change? Are downstream conclusions recalculated?

These aren’t unusual questions to ask about children’s information.

The OAIC’s current exposure draft for Australia’s Children’s Online Privacy Code includes dedicated provisions for access to and correction of children’s personal information, as well as a right to request destruction, subject to exceptions including safety, legal proceedings, statutory retention and enforcement requirements. [Source: OAIC Exposure Draft]

We are not asserting that those provisions apply to SAFE. Whether the eventual Code applies is a legal question. The OAIC has said the Code is to be registered by 10 December 2026. [Source: OAIC]

There is also existing privacy guidance associated with the earlier HubHello implementation.

HubHello’s privacy policy says it will take reasonable steps to destroy or permanently de-identify personal and sensitive information when no longer needed for the purpose for which it was disclosed, while allowing retention to meet legal record-keeping obligations. The same policy provides mechanisms for correction and requests for access. [Source: HubHello Privacy Policy]

However, that policy was last revised in February 2022, while SAFE says it moved towards a standalone registration and independent deployment pathway from 2024. [Source: SAFE Our Story]

We therefore cannot establish from public material what current SAFE-specific retention and deletion rules apply to its Living Child Record or machine-derived risk information.

Persistence needs to work in both directions.

A system designed not to forget concerns must be equally good at remembering when those concerns were explained or contradicted.

Otherwise persistence risks becoming persistence of error.

Human in the loop is not the end of the question

SAFE says technology surfaces signals while people make decisions, with child-protection expertise brought into the review process. [Source: SAFE Product]

That is an important safeguard.

But “human in the loop” doesn’t automatically resolve every concern about algorithmic decision support.

If a reviewer sees a machine-generated risk prediction before independently considering the observations, could that framing influence their interpretation?

What happens when professional judgement and the machine prediction disagree?

A computer doesn’t need to make the final decision to influence one.

There are humans at both ends of the pipeline: one interprets at least some of the original events and supplies information; another interprets the system’s output. SAFE’s current material also contemplates other structured signals such as attendance patterns and participating systems, so human narrative is not necessarily the system’s only input. [Source: How SAFE Works]

Professional experience is valuable, but experts remain human. They work with incomplete information, finite time and complex family dynamics.

Those pressures aren’t hypothetical. The Victorian Auditor-General found that child-protection workload pressures could prevent timely updating of records and reduce data quality. Current Victorian workload-management procedures also continue to provide formal review mechanisms for workload and capacity. [Source: Victorian Auditor-General] [Source: Victorian Child Protection Manual]

That doesn’t establish anything about SAFE’s reviewers.

It demonstrates why human review should be understood as part of the decision system rather than assumed to eliminate weaknesses preceding it.

What was the model trained and validated against?

SAFE says its technology has been trained on 12 years of field data from every Australian state and territory.

Its own history describes earlier work involving labelled training data, classifier training, true and false-positive analysis and threshold-based risk flagging. [Source: SAFE Our Story]

Older HubHello material separately says individual SAFE assessments were analysed against health and welfare datasets involving more than 230,000 children and mandatory-reporting guidelines across Australian jurisdictions. [Source: HubHello SAFE]

We don’t know how those populations and datasets relate, so we shouldn’t assume they are the same thing.

But we still need to understand what constituted the ground truth.

What did a positive label represent?

A substantiated finding? A report? An investigation? An expert assessment?

If historical human decisions formed part of that ground truth, how were inconsistencies or biases in those decisions accounted for?

If a model learns from the outcomes of an imperfect human system, it can potentially learn some of that system’s patterns as well.

That doesn’t mean SAFE has done so.

We don’t have enough public information to know.

Which is precisely why transparency around training and validation matters.

Security isn’t the same as governance

HubHello’s publicly available material says its SAFE implementation used AWS-hosted data, encryption and password protection, with case information limited to authorised child-protection personnel. [Source: HubHello SAFE]

Those are important security controls, but that material relates to the older HubHello implementation rather than independently establishing the architecture of standalone SAFE today.

SAFE’s current material separately describes permissions and governance controls around cross-service information. [Source: How SAFE Works]

Security and governance answer different questions.

Security asks:

Who can access the information?

Governance also asks:

What are legitimate holders of that information allowed to do with it, for how long, and what happens when it changes?

To be clear, DigitalReviews Network has found no evidence that SAFE uses children’s information for unrelated secondary purposes, and we are not suggesting that it does.

What we want to understand is whether the current system’s technical, contractual and governance controls prevent information collected for child protection from later being used for incompatible secondary purposes.

For information this sensitive, that distinction matters.

This isn’t an argument against AI in child protection

None of this establishes that SAFE is unsafe, inaccurate, biased or badly governed.

And we have found no evidence that SAFE has caused the hypothetical harms considered in this article.

Our concern is whether publicly available information is sufficient to understand how the current system prevents them.

The problem SAFE is trying to solve is undeniably important.

Child abuse can remain hidden precisely because individual observations seem insignificant. People miss warning signs. Organisations lose information. Staff leave. Records become fragmented.

Technology may help us do better.

Machine learning could genuinely make children safer by recognising longitudinal patterns an individual educator may never see.

Precisely because those capabilities are meaningful, the methodology deserves scrutiny commensurate with the consequences.

To properly assess SAFE, we need more detail about its current assessment and validation, what its accuracy figures mean, false-positive and false-negative performance, consistency between assessors, provenance and dependence of repeated signals, correction mechanisms and current data-retention arrangements.

Those aren’t reasons to reject SAFE.

They’re the information required to understand it.

How this story came to us

SAFE came to our attention through an unsolicited media email from its public relations representatives, Third Hemisphere.

The email highlighted recent Sydney Morning Herald coverage of SAFE and concluded:

“A national safety infrastructure with children at its heart, like SAFE, is exactly what is needed to protect our greatest assets, in our opinion.”

The underlying Sydney Morning Herald story was considerably more nuanced. It discussed SAFE’s potential while also raising questions about privacy, consent and the fact that observations feeding the system are human judgements.

Those questions caught our attention.

Public relations exists to advocate for clients, and there is nothing unusual about presenting a client’s technology positively.

But the more consequential a technology becomes, the less comfortable we are assessing it solely through the outcomes its proponents want us to see.

So we started looking more closely.

We asked SAFE’s representatives

DigitalReviews Network wanted SAFE’s perspective before publishing this editorial.

On 11 August, we contacted Third Hemisphere with detailed questions covering the machine-learning model, validation and error rates, contextual information, parental contestability, mandatory reporting, data retention and potential secondary use.

We followed up on 13 August, making clear that we didn’t expect immediate answers to detailed technical questions and simply wanted to establish whether Ruby O’Rourke or SAFE’s technical team was interested in responding.

At the time of publication, we have received neither an acknowledgement nor a response.

That silence does not establish that SAFE lacks answers to any of the questions raised here.

Nor can we establish whether our questions were passed from Third Hemisphere to SAFE.

It simply means DigitalReviews Network hasn’t been given those answers.

The invitation remains open.

If SAFE provides substantive responses, we’ll update this article and, where appropriate, correct or revise our analysis.

We would genuinely like to know the answers.

Because a system capable of recognising that a child is in danger before everybody else sees it could be extraordinarily valuable.

But SAFE isn’t describing a small experiment.

It describes itself as decision-support infrastructure designed specifically to preserve and connect signals humans might otherwise miss. [Source: SAFE Product]

That deserves a commensurate level of transparency.

SAFE isn’t a complete black box, but the part that matters most for independent evaluation — how observations are weighted, combined and translated into risk — remains opaque from the outside.

Protecting our children is non-negotiable.

Understanding the systems we entrust with protecting them should be too.

---

Sources