An LLM on a marketplace’s critical path: three years in production, the same architecture

Artificial intelligence · Trust & safety · LLM · Node.js · PostgreSQL

In a marketplace, the chat between those who buy and those who sell is essential. Before buying, a customer wants to clarify the details of the offer, check whether it fits their needs, ask for a variation or a different date. But the same chat is also the door through which the sale walks out: a phone number, an email address or a “find me on Instagram” is enough, and the negotiation continues elsewhere, with no commission for the platform and none of the protections the platform offers both parties. The terms of use forbid exchanging contact details before purchase, but a written rule is not enough to enforce it.

For a marketplace where customers buy offers from independent sellers, we designed and built the system that enforces this rule. It evaluates every message before it is delivered and, at its core, uses a language model that reasons about the writer’s intent, not about the form of the text. It has been in production since March 2023: we put it on the message delivery path a few weeks after the new generation of language models had become available to developers through an API.

Handing a revenue-critical flow to a technology only a few weeks old looks like a gamble. It would have been one if we had built the system around the model. We built it the other way around: the language model is a replaceable signal, inside a system that decides, stores and explains every outcome regardless of which model sits behind it. That choice is what allowed the filter to go through more than three years of production, two model families and a fine-tuning experiment without rewriting the evaluation pipeline.

Why keywords are not enough

The first tool that comes to mind is a list of forbidden words and a few regular expressions for emails and phone numbers. It works against people who do not try to hide, and in fact it is still one of the system’s signals. But anyone who wants to take the negotiation off the platform learns fast, and there is no practical limit to the creativity with which contact details can be disguised. Every new rule invites a new variant, and the list keeps growing without ever closing the problem.

Then there is the opposite mistake, and it is just as costly. In a purchase conversation numbers are everywhere (prices, quantities, dates, times), and so are links to resources that help people decide. A rule broad enough to catch every disguised contact detail would also block the questions that lead to a purchase. A filter that hinders legitimate conversations hurts the marketplace as much as one that lets contact details through.

So the right question is not “does this text contain a number?”, but “is the writer trying to be contacted outside the platform?”. It is a language understanding problem. In early 2023 language models had only just become capable and cheap enough to ask that question of every single message, and we decided to use them right away, on a business-critical flow, instead of waiting for the technology to settle. Precisely because it had not settled, though, everything else in the system had to be designed not to depend on it.

The check happens before delivery, not after

The second fork in the road concerns when the check happens. The most convenient path is after-the-fact moderation: the message goes out immediately, a background process analyzes it and, if needed, removes it. It adds no latency to sending and keeps the model off the critical path. Here, though, it is useless: a phone number seen for a few minutes is a phone number already copied, and a removal always comes after the damage. That is why we chose the more demanding path: the check runs synchronously, inside the send request, before the message reaches the recipient.

Where it lives matters too. A filter written into the chat endpoint protects that endpoint, and the day the product adds a new way to write to the other party, someone has to remember to replicate it. Our check instead lives in the domain services that create messages: every path through which a user writes to the other party in a conversation uses the same evaluation. A new touchpoint between buyer and seller inherits the filter instead of reimplementing it.

One detail makes the mechanism reliable, not merely functional: secondary channels must also respect the filter’s outcome. The email notification does not start from the resolver, but from a domain event emitted only for delivered messages and only after the transaction commits. A held message, or a rolled-back transaction, therefore cannot generate an email containing it: the secondary channel cannot let through what the chat has held back.

The model votes, but does not decide alone

Once a language model is available, the temptation is to hand it the whole decision. That would be a design mistake. Some information should not be inferred from the text because it is already known with certainty: whether the buyer has completed the purchase, for instance, is something the database tells you, not the tone of the message. And a system in which a single judgment decides everything cannot be corrected one piece at a time. The classic alternative, a chain of if statements that grows with every new case, after a few years no longer tells you which condition decided the outcome.

We chose a third path: a set of independent rules, each looking at the message from a different angle and returning a contribution to a shared score. The rules run in parallel, so the synchronous check waits for the slowest one, not for the sum of all of them, and the sum of the contributions decides the outcome.

Signals do not all carry the same weight, nor do they all point the same way: some push toward blocking, others toward delivery. Context, in particular, can override content. Once the buyer has completed the purchase, exchanging contact details is no longer an attempt to bypass the platform but the natural way to deliver what was bought. Conversely, the window in which a conversation stays open is tied to the order lifecycle: beyond a certain phase it no longer has a reason to exist.

In practice, the scoring model makes the system easy to evolve and to defend. Adding a rule means adding a term to the sum, without touching the others. Every decision can be broken down, item by item, into the contributions that produced it. And term lists are managed from the back office for each individual seller or buyer: when operators spot a recurring behavior on an account, they act immediately, without waiting for a release.

The prompt is the policy, written as a specification

The heart of the system is the rule that queries the language model, and the typical weak spot of such a rule is the prompt. A model asked only to “find contact details” tends to see them everywhere: a time, a price, the name of a widely used app. Our prompt is not a generic question: it is the platform’s policy written in natural language, with the same precision as a specification. It describes the model’s role, what is forbidden and, above all, what is explicitly allowed.

That last part is what makes the difference in production. A model that is told what is normal in a purchase conversation can tell a practical question from an attempt to leave the platform. Defining what is legitimate precisely is what makes it possible to be strict about what is not, without hindering the conversations that lead to a purchase.

For forbidden cases, the prompt describes, as general patterns, the ways information can be hidden. It does not list strings to look for, which would bring back the limits of keywords: it teaches the model to recognize the pattern, not the individual variant. Nor does it require the contact detail to appear in full: a recognizable intent is enough. It is a strictness the system can afford precisely because what is legitimate is defined just as precisely.

The rest of the contract makes the answer usable by a system. The model answers with a binary verdict, in a fixed format the code interprets unambiguously. The temperature is zero, so that the same message gets as stable a judgment as possible. Messages arrive in any language, and multilingual understanding is one of the reasons a language model is the right tool: there is no need for a dictionary of rules for each language.

Finally, the prompt lives in the application’s configuration, not in the code. Changing the policy means editing a text, subject to the same review as any other change, without touching the filter’s logic.

Three model setups, one architecture

The filter was born in March 2023 on a general-purpose language model, and within the first few weeks the prompt took the shape described above: a fixed-format verdict with a short justification, the platform’s context, the allowed cases.

In early 2024 we tried the path many consider the natural evolution for such a specific task: fine-tuning, with two variants of the same model trained specifically for it. In April we chose the general model with a richer prompt. A trained model freezes the policy at training time, while a prompt is updated with an ordinary configuration change whenever the policy evolves or a new disguising technique emerges. For a problem where the rules change more often than the models, flexibility is worth more than specialization.

The choice also proved right for a more structural reason. A trained model is a variant of a specific base model and shares its lifecycle: providers regularly retire older models, and when a base model is retired, the variants trained on it stop being served as well. Sticking with fine-tuning would have meant retraining at every generational change. With the prompt, in 2025 we moved to a newer-generation model and enriched the instructions with new disguising patterns: a more capable model makes better use of more detailed instructions, and the switch required nothing else.

The reason every step was so light is the rule’s contract, which has stayed the same since the first weeks: a message in, a verdict and the model’s raw answer out. Model name, temperature and prompt live in configuration: a model change is a configuration change. The rest of the pipeline never noticed. When a better or cheaper model arrives, adopting it is a product decision, not a project.

A held message does not disappear

Blocking a message raises a problem that is usually underestimated: what to do with it. The most obvious answers are both wrong. Deleting it destroys the only evidence of what happened: support can no longer explain the decision, and there is no way to verify whether the filter was right. Making it vanish silently, letting the writer believe it was delivered, produces conversations that break off for no reason, frustration and support tickets.

Our choice is that a held message is kept intact, invisible to the recipient by construction and explained to the person who wrote it. It is saved unchanged, together with its evaluation, and marked as not delivered.

“By construction” means that the visibility rule lives in the read model, not in the interfaces. The query that reads conversations, shared by both applications, returns a held message only to the person who wrote it. The same applies to the conversation previews in the message list. If the filter were left to the screens, every new screen would be a chance to forget it; this way, showing the message to the recipient would mean bypassing the query itself.

The writer, on the other hand, sees clearly what happened. In the buyer app the message is marked as not sent, with an explanation: the seller’s contact details come with the purchase, and customer care is available for anything else. In the seller app the message appears struck through, with a warning. Anyone who had a legitimate question can rephrase it, and everyone is reminded that the platform protects both parties precisely because the transaction happens on it.

Every outcome can be reconstructed, item by item

A decision made partly by a language model is indefensible if only the final verdict survives. Faced with a user asking why their message did not arrive, support can only answer “the algorithm decided”. And faced with a change to the prompt, no one can say whether it improved things or just moved the errors elsewhere.

That is why every message carries the full breakdown of its own evaluation, in a JSONB field on PostgreSQL: each rule’s contribution and, for the model-based rule, the request sent and the response received. The total score is only the summary; the explanation is always behind it.

A filter that reads conversations between buyers and sellers processes personal data by its very nature, and every evaluation carries part of it. That is why privacy has been a design requirement from day one, on a par with the filter’s effectiveness: what is sent to the model, what is stored and who can access it are decisions made with user protection as a constraint, not checked after the fact.

In the back office, held messages appear flagged in the conversation, with their scores, and support can check exactly why they were held. The same information makes it possible to assess the effect of a policy change on real messages, rather than on contrived examples, and to notice when a new disguising technique is emerging. That is what makes it safe to evolve the prompt: every change is measured against real traffic.

There is an operational side too. Calls to the model provider appear in the same OpenTelemetry traces as the rest of the backend, and latency and errors are observed with the same tools used for the rest of the platform, not in a separate console.

The same policy, a different measure for each channel

The chat is the main channel for exchanging contact details, but not the only one. And the measure that works in the chat, holding the message, does not apply everywhere: a review, for example, is public text and cannot be left pending. That is why we applied the same policy with a different action for each channel, instead of forcing the same mechanism everywhere.

In the sellers’ back office, the buyer’s phone, email and address are visible only once the order is fully paid; until then, placeholder data is shown. Direct contact thus comes with the completed transaction, outside the chat too. In reviews, disallowed terms are masked in the published version, while the original text remains available to operators, and when the rules change, existing reviews are reprocessed automatically. Finally, sending messages is protected by an anti-bot check, which keeps automated clients out of the chat.

Hold, mask, defer or reject: each layer does the right thing for its own channel. Together they make exchanging contact details before purchase an awkward path, rather than an obstacle to get around once.

What it takes to put an LLM on the critical path

For a marketplace, the commission on each sale is the business model, and disintermediation erodes it silently, one conversation at a time. The filter protects that value without sacrificing what makes the platform useful: buyers and sellers keep talking freely about everything they need to decide, and exchange contact details when the time is right.

The method applies beyond this project. In a system that uses a language model, the model is the part that changes fastest and the one you control least: new versions come out, old ones are retired. That is why we do not treat it as the center of the architecture, but as a component behind a stable contract, and we design everything that has to last (where the check happens, how signals are combined, where the policy lives, what is stored and traced) so that it does not depend on it. That is what allowed us to adopt a technology in its first weeks, on a revenue-critical flow, and to watch it improve for more than three years without rebuilding the system.

Work on the prompt continues with the same approach the system has grown with from the start: making the policy more precise while leaving the architecture that enforces it unchanged.