
By Fuad Salimzade, AI Engineer at Adclear
“Is there a risk warning in this ad?” is one of the simpler questions we ask when reviewing a financial promotion. We were using a large language model (LLM) to answer it, often as part of a much longer question about whether the promotion met a policy requirement.
When TypeSafe released Jev, we wanted to see how a decision model would handle some of those individual checks. Cost got us interested. On input tokens, the price was as little as 1/200 of the cost of a comparable LLM. What convinced us to keep experimenting was how well the model performed on some of the examples we found hardest to classify.
Where decision models fit in a compliance review
A decision model can answer yes or no, choose from a list you supply, or give a score within a range you define. That maps quite neatly onto some of the work we do at Adclear. Identifying which product a campaign promotes is a choice. Checking whether a risk warning appears is a yes/no question. Assessing how legible it is lends itself to a scale.
The distinction matters when you look at an actual promotion. A warning can be present in a video and still be difficult to read. If we ask one broad question about the warning and get a “no”, we have to work backwards to understand what failed. Asking about presence and legibility separately gives us an answer for each. It also means we can run independent checks at the same time.
There’s still a role for a frontier LLM here. Working out how a policy applies to a particular claim, or what to change when an unusual case falls outside the examples we’ve seen, calls for more reasoning. The engineering work is in deciding which parts of that review we can express as precise questions.
How we tested the same rule across images, PDFs and video
We started with 10 to 15 difficult examples per problem and labelled each one before running the model. We wanted to know whether it could recognise the same policy issue in different formats. A rule might stay the same across a static ad, a PDF and a video, but the model has to find the relevant evidence in a different place each time.
One of the most useful changes we made was to mark the section we wanted the model to assess. Its confidence increased when we pointed it to the relevant part of the promotion. That gave us two things to refine together during testing — the wording of the question and the evidence we supplied with it.
The small initial test helped us decide which checks were worth pursuing. We then replayed typical production runs to calibrate the confidence threshold for accepting an answer.

What happens when you break a compliance review into smaller decisions?
Financial promotion compliance involves hundreds of rules across regulators, jurisdictions and internal firm policies. Reviewing a single promotion can mean checking everything from the presence and prominence of risk warnings to the accuracy of financial claims and the suitability of particular statements for a target audience.
Historically, we'd approached many of these checks by asking an LLM broader questions about whether a promotion complied with a particular policy. That works well when interpretation and contextual reasoning are required, but we began to question whether every part of the review needed that level of reasoning.
Take a relatively simple question: Is a risk warning present in this promotion?
We don't necessarily need an LLM to interpret an entire policy to answer that question. What we need is a reliable answer about whether something is present, followed by a separate assessment of whether it meets the relevant requirements.
Decision models are particularly well suited to this kind of task. They can return a yes-or-no answer, select an option from a predefined list or produce a score within a specified range. Each decision comes with a confidence value that we can use when determining how to handle the result.

For example, we can ask whether a risk warning appears in an advertisement, which financial product the promotion concerns and how legible the warning is. Each question isolates a different part of the review, giving us more specific information about what the model has identified.
That distinction becomes particularly useful when something fails. A risk warning might technically be present but displayed so briefly in a video that it's almost impossible to read. If we ask one broad question and receive a negative answer, we still have to work out what caused the failure. Separating presence from legibility allows us to identify the issue much more precisely.
It also means we can run independent checks in parallel, rather than waiting for a single model to reason through every requirement sequentially.
Applying decision models to financial promotions
The next question was whether this approach would hold up across the different types of content Adclear reviews.
Financial promotions come in many formats, from static social media advertisements and website copy to PDFs and videos. The underlying regulatory requirement may remain the same, but the evidence needed to assess compliance can look very different.
We experimented with asking decision models to evaluate individual rules against promotions in different formats, returning a structured answer and confidence score for each check.

One of the most important findings was how much the context we supplied influenced the model's confidence.
When we gave the model a broad question about a promotion, it had to work out which part of the content was relevant before making its decision. We found that explicitly marking the section we wanted it to assess improved its confidence.
This led us to experiment with two things together: how precisely we framed the question and how we presented the evidence the model needed to answer it.
We started with small datasets of around 10–15 difficult examples for each problem, labelling the expected answers before testing. We deliberately included examples that had proved challenging, because we wanted to understand where the approach would struggle rather than simply demonstrate that it worked on straightforward cases.
Once we'd identified promising use cases, we replayed typical production runs to calibrate the confidence thresholds at which we would accept the model's answers.
The experience reinforced something we've found repeatedly in our research: the way a problem is structured can have as much influence on the result as the model selected to solve it.
From hundreds of policies to the ones that actually matter
One of the more interesting applications of this approach is identifying which policies are relevant to a particular promotion before asking an LLM to review them in detail.
Consider a hypothetical promotion that needs to be assessed against a library of 240 policies. Depending on the financial product, jurisdiction, claims and format, only a subset of those policies may actually apply.
Rather than passing all 240 policies to an LLM for detailed consideration, we can break the task into individual questions: Is this policy relevant to this promotion?
Those questions can be evaluated independently and in parallel. The resulting decisions allow us to narrow the set of policies that need further attention.

In the example above, 240 individual relevance checks identify nine policies that require closer review. The LLM can then focus its reasoning on those nine policies instead of processing the entire library.
This is where the combination becomes particularly interesting. Decision models can handle narrowly defined classification and filtering tasks, while frontier LLMs remain valuable for interpreting complex requirements, understanding unusual cases and reasoning about what a promotion needs to change.
For us, the opportunity lies in understanding how to allocate different parts of a compliance review to the models best suited to performing them.
What did our experiments show?
We measured the performance of decision models against frontier LLMs, focusing on three areas: cost, response time and accuracy.
The latency difference was substantial. In our tests, the decision model returned a median response time of 0.3 seconds, compared with 20 seconds for the frontier LLM. At the 95th percentile, the decision model completed in 1.5 seconds, compared with 43.5 seconds for the LLM.
Cost was similarly compelling. At the prices represented in our benchmark, decision model input tokens cost $0.07 per million, compared with $3.25 for the frontier LLM, with no additional charge for the decision model's output.

While these improvements were encouraging, speed and cost would mean very little if the models couldn't make reliable decisions.
We therefore evaluated accuracy using our own labelled benchmarks, measuring both precision and recall where applicable. Precision tells us how often a positive result is correct, while recall measures how many of the actual positive cases the model successfully identifies.
The results varied by task, but several stood out. When identifying whether a request was still open, the decision model achieved 0.96 precision and 0.93 recall. On a task involving matching two statements, it achieved 1.00 for both precision and recall.
We also tested whether the model could identify which policy a piece of feedback related to. The correct policy was ranked first in 93% of cases and appeared within the top five in 100% of cases in our labelled benchmark.
These are results from our own datasets and experiments, rather than universal performance guarantees. The quality of the question, the evidence provided and the confidence threshold all affect how reliably a decision model performs.
What interested us most was the precision we could achieve when those factors were carefully controlled. On some of our defined tasks, we were seeing near-perfect precision while substantially reducing the time and cost required to produce an answer.
Of course, high precision alone doesn't tell us whether the model catches every genuine issue. Recall remains equally important, particularly in compliance, where missing a relevant problem can have significant consequences.
What we've learned about building with decision models
Our experiments have changed how we think about certain parts of the financial promotion review process.
The most useful lesson has been to look more closely at the work we're asking a model to perform. A compliance review might look like one complicated reasoning task, but when you examine the individual steps, many can be expressed as more precise questions.
That doesn't mean every review should be reduced to a series of yes-or-no answers. Financial promotion compliance depends heavily on context, and some decisions genuinely require the more sophisticated reasoning of a frontier LLM. The challenge is knowing where that reasoning is necessary and where a more specialised model can deliver the same result more efficiently.
We're continuing to explore that distinction as part of our research and development at Adclear, testing different approaches against real-world review requirements and examining how model performance changes across policies, content formats and levels of complexity.
For anyone experimenting with decision models themselves, I'd recommend starting with a single question you can reliably label. Test it against examples you already know are difficult, experiment with the context you provide and measure precision and recall separately before expanding the approach.
What began as an experiment into whether decision models could match the accuracy of LLMs ultimately made us reconsider how we structure the problem itself. And for an engineering team working on financial marketing compliance, that's a useful finding in its own right.


