INTHEBLACK October/November 2024 - Magazine - Page 38
F E AT U R E
“Bringing an LLM in-house means you do not have
subscription fees and the APIs, so you are only paying
for what you are using or what you are really needing.”
JANNAT MAQBOOL CPA, ARTIFICIAL INTELLIGENCE RESEARCHERS ASSOCIATION
While building an LLM from scratch
is cost prohibitive for most organisations,
customising existing foundational models,
such as Microsoft’s Azure OpenAI Service or
Google’s Gemini, may help drive efficiencies
and enhance productivity across a workforce.
BUILD YOUR OWN
Research from Accenture shows 85 per cent
of c-suite leaders are focused on increasing
generative AI investments in 2024.
Customising LLMs may be an appealing
solution, because they may be able to bridge
the gap between generic AI capabilities and
specialised, industry-specific tasks.
Dr Jake Renzella, lecturer and co-head
of the computing and education research
group in the School of Computer Science
Engineering at the University of New South
Wales, says there are two main ways to do it.
“The first is by fine tuning a foundational
model,” he says. “For instance, you can expose
it to some examples of tasks that you want it
to be better at and then train it.”
In an accounting context, imagine you have
some financial data that contains a common
type of error. If you have many examples
where that error occurs in a real dataset, you
could show them to the LLM and train it to
identify them.
The other common way to customise
an LLM is through retrieval-augmented
generation (RAG). RAG involves exposing
a foundational model to documents or data
from your organisation that it is unlikely to
have been trained on.
“You are basically saying to these LLMs,
‘Hey, here is a bunch of information about
our organisation to draw on when you are
doing your tasks’,” Renzella says.
CUSTOM CONVENIENCE
Creating a customised LLM presents huge
benefits for those who get it right.
Data integrity is one of them, says
Jannat Maqbool CPA, executive director
of the Artificial Intelligence Researchers
38 INTHEBLACK October/November 2024
Association and a member of CPA Australia’s
Digital Transformation Centre of Excellence.
“You can ensure the data is from
trusted, verified sources,” she says. “That
means you can have a really good go
at eliminating bias and improving the
reliability of the model.”
Maqbool notes that customised LLMs can
also present cost benefits.
“Bringing an LLM in-house means you
do not have subscription fees and the APIs,
so you are only paying for what you are using
or what you are really needing,” she says.
“You may also have a competitive
advantage,” Maqbool adds. “You can
align it more with market conditions and
compliance requirements, as well as your
clients and stakeholders.
“The stability and quality mean that you
can be a lot more responsive as well, because
it is part of your infrastructure, and you can
optimise the model.”
ESSENTIAL PREPARATION
Before considering a customised LLM for
your organisation, Renzella says there are
three key things to assess.
“The first is do you have the right training
data and the right volume, because you might
need tens of thousands of examples, which
might seem a lot to some organisations, but
in the scale of LLMs, that is a drop in the
ocean,” he says.
“The second is whether your dataset is
representative – if you are trying to teach
a model to do a task, do you have lots of
different examples that cover the full scope
of that task?”
The third, says Renzella, is about having
the right capability.
“You could have the data, but has it been
thoroughly reviewed for quality control, and
do you have the right people in-house, such
as software engineers or machine learning
engineers, who can put the data in the right
format in order to successfully fine tune or
build a RAG model?”
