INTHEBLACK May 2026 - Magazine - Page 38
F E AT U R E
“Training data can under-represent certain industries,
business models, geographies, risk scores or forecasts.”
DR DIMITRIOS SALAMPASIS, SWINBURNE UNIVERSITY OF TECHNOLOGY
accuracy when it comes to AI. Accuracy
is objective (for example, the percentage
of correct predictions), whereas fairness
is highly contextual.
O’Callaghan explains that the context
in which the system is deployed matters
when deciding what a fair decision looks like.
“The definition of fairness in a hiring tool
may differ from that in a medical diagnostic
or an audit system.”
UNCERTAIN OUTCOMES
In accounting and finance workflows,
algorithmic bias can potentially lead to
errors and unfair decisions. This could
include inaccurate risk assessments, unfair
credit evaluations, inaccurate financial
reporting and fraud detection, or an
over-reliance on automated outputs,
according to industry research.
When financial decision-making is
automated at scale, any distortions embedded
in the underlying data can be amplified
across thousands of decisions, according
to Dr Dimitrios Salampasis, associate
professor in emerging technologies
and fintech at Swinburne University
of Technology.
In risk assessments, errors and biases can
lead to distorted risk classification, uneven
investigations or compliance burdens, which
can result in false positive or false negative
outcomes. “You may have training data which
actually under-represents certain industries,
business models, geographies, risk scores
or forecasts,” explains Salampasis, who is
a member of CPA Australia’s
Digital Transformation Centre of Excellence.
There is also the risk of inaccurate
financial reporting. “When you have biased
algorithms, this might create issues when
estimating key accounting elements,”
he says. “You may also have risk in relation
to materiality in financial reporting.”
In practice, an AI model could misclassify
transactions, skew risk scores or distort
38 INTHEBLACK May 2026
sampling, influencing which items are
considered material and potentially altering
outcomes. Furthermore, a lack of critical
insight into AI outputs and opaque “black
box” systems can hinder transparency,
undermine trust, and make it difficult to
assess potential discriminatory outcomes.
“An over-reliance on these automated
outputs creates issues when it comes
to human professional judgement,”
Salampasis adds.
BIAS-MITIGATION STRATEGIES
Recognising bias is just step one. The next
challenge is developing safeguards to
mitigate bias in workflows. Practical
strategies include diverse data sourcing,
algorithm audits, stronger governance
and AI education.
Structural and historical factors can
be embedded in data, therefore diverse
data sourcing is important to help limit
skewed outcomes.
However, algorithmic bias mitigation is
not a one-time technical fix, but an ongoing
governance responsibility, O’Callaghan says.
Organisational leaders need to be involved
in decisions around the definition of fairness,
not leave it to engineers and technical leads
alone. This definition also needs to be revisited
over time.
“It is not good enough for just the
technical people to try to make the system
fair,” he says.
Algorithm audits are also necessary to
limit “model drift” — when the AI moves
from its original parameters and provides
inaccurate outputs — and test the
underlying assumptions.
Evaluating models is essential, O’Callaghan
says. “You can measure different definitions
of fairness and see how the system performs
against them.” However, there are always
trade-offs and competing criteria on questions
of fairness, balance and bias.
“You cannot satisfy all definitions of fairness
