A free daily word game has one job above all others: the scores have to feel fair. The whole point of Letterly is that obvious answers score low and clever ones score high, with a leaderboard that actually means something. So when a player flagged that some genuinely daft answers were scoring full marks, it went straight to the top of the list.
The examples were not subtle. An apple counted as a vegetable. A type of gas counted as a gemstone. A handful of made-up words sailed through for a hundred points each, purely because nobody else had typed the same nonsense. On a game built around originality, a junk answer that nobody else picks looks like a stroke of genius to the scoreboard. That is the worst possible way for this to fail.
The checker was the problem
Every answer that starts with the day's letter gets checked by an AI model that decides whether it really belongs to its category. After digging through a full day of real answers, the honest finding was that the model itself was wrong. It was the cheap, fast option, and on the close calls it kept saying yes to real words that simply are not members of the category. This was not a glitch or an outage. The checker was confidently grading bad answers as good.
The fix was to move to a stronger model. Before touching anything live, I replayed the exact bad answers from that day against both the old and the new one. The old model caught seven of the ten worst offenders. The new model caught all ten, and just as importantly did not start rejecting the genuine answers either. That is the test that matters: stop the nonsense without punishing real players.
Free does not mean free to run
Here is the part nobody mentions about free apps. Every one of those answer checks costs a fraction of a penny. On something with no revenue, you genuinely sit there and weigh up whether to pay for the better model. The numbers turned out kind: a few pence a month at the current size, with plenty of room to grow before that changes. The real sting of the free tier was never the money, it was the rate limit. When a lot of people played at once, the free allowance ran out and the checker quietly gave up and waved everything through. Paying lifts that cap and closes the gap.
The unglamorous bit was the cleanup. Months of past answers had already been graded by the weaker model and remembered, so the same mistakes would keep repeating forever. Every one had to be re-checked. The first sweep over-corrected and tried to bin real answers too, including a few famous names and some perfectly good spellings, so it took a slower, more careful second pass to rescue those and only remove the genuinely wrong ones. Fiddly, repetitive, and completely invisible to anyone playing. Which is rather the point.
Free to play does not mean free to keep honest. But a few pence a month to make sure the boards can be trusted is the easiest call I have made all week.