AI compliance pilot at Google
A Vertex AI pilot that re-approved wrongly flagged items, for a Google advertising product.
The short version
- What it is
- An AI pilot that rewrote product ads a rules-based checker had blocked by mistake. It also said why each one was blocked.
- Who for
- A large online marketplace and its sellers, on a Google advertising product.
- What I did
- I found the cause, designed the AI workflow and wrote the prompt. I built a first version in a spreadsheet, then got product, sales and safety teams behind it.
- What happened
- 78% of fixed items were re-approved automatically. Under 1% were approved against the rules. I won a 2024 Risk Taker award for it.
Pilot results, 2024. One seller, English only, every item checked by hand.
- What I’d change
- Bring the safety team in on day one. They saw a risk where we saw revenue.
Two minutes? Read the five lines, then the results.
Start with the problem: good products blocked, and nobody knew why.
Start with the workflow and its guardrails.
A Google advertising product. The client and its sellers stay unnamed.
The problem: good products, blocked
A rules-based checker blocked ads it should have allowed. Sellers could not see why.
The client was a large online marketplace. Warnings stacked up on its account. It risked being cut off from advertising on the product.
Its honest sellers were in the dark. Was it the image? The words? The rules caught real products, and revenue went with them.
Some of the cost had no number. Unfair blocks were straining the relationship between Google and the client.
The kind of item a keyword rule catches
- A firework-shaped chandelier
“Shooter” or “explosive” in the text.
- An ink cartridge
“Cartridge” read as ammunition.
- A hanging egg chair
“Hanging” or “swing” matched to harmful-content rules.
- A children’s book called “Tough Cookie”
The blurb read as a sensitive theme, or “cookie” as web tracking.
Illustrations of the kind of item that gets caught. Not taken from the client’s data.
What I built: an AI that explains itself
A better rule would not fix this. It needed judgement about real products. I proposed a small test to prove AI could do better than the rules.
- 1. A sheet of blocked itemsTitles and descriptions the rules had refused.
- 2. Vertex AI and one promptThe prompt condensed Google’s ad policy.
- 3. Four answers per itemA new title, a new description, the reason, and a confidence score.
- 4. A sheet for checksA person checked every result.
- 5. Back to the ad productFixed items went back through its own checks.
What it did
- Rewrote titles and descriptions to meet the policy.
- Said why each item was blocked, so sellers could learn from it.
- Gave a confidence score, to help tell honest sellers from bad actors.
- Ran from a spreadsheet, so we could test in days, not months.
Guardrails
- One seller and one language, English, so every result could be checked by hand.
- No items blocked for offensive or harmful reasons. The tool never saw them.
- Trust and Safety had the final say on what went further.
I linked the client, two product teams, directors in gTech, and Trust and Safety, who were the gatekeepers.
Results
What each number means, in plain words.
| Measure | Result | What it means |
|---|---|---|
| Re-approved automatically | 78% | Of items with a new title or description, 78% passed the ad product’s own checks. |
| Precision on false positives | 99% | Of the items the tool approved, 99% had been blocked wrongly in the first place. |
| Unsafe approvals | Under 1% | Under 1% of re-approved items broke the advertising rules. |
| Client revenue | $6-10M | The client’s revenue at stake from wrong blocks, as a range. |
What happened next, and the lesson
The pilot passed to Trust and Safety. They used it in their own research into false positives, and towards their own targets. That is where I won the 2024 Risk Taker award.
The tool worked. It also exposed a conflict: helping sellers sell, against keeping the platform safe. Product teams saw a revenue saver. Trust and Safety feared bad actors could use it, if it went public, to learn how to beat the checks.
What this does not show
The honest edges of a pilot.
- One seller and one language. Results at a larger scale are unknown.
- The revenue figure is a range, not an exact number.
- It stayed a pilot. Scaling it was the next step I was pushing for.
What I proposed next
- Proposed, not built
Rebuild approvals for titles and descriptions with AI first. - Proposed, not built
A seller reputation score built from the confidence score, to flag bad actors and fast-track good ones.
Shipping AI where a mistake is costly?
Tell me what you are working on. I read every message.