In short
For the product
- One pick in four, or almost none. That is how often the weakest and the best models picked something that broke what the collection was about.
- The priciest was not the best. Cost per deck spanned 33×. The most expensive model, Claude Opus 5.5, came fourth by judged score; GPT-6 Luna was among the best at $0.13.
- Personal feel may be the remaining gap. The best models rarely miss the basics. They may differ in how personal the picks feel, but no shopper has checked that score.
For engineers
- Errors start early. On the collection we traced, the weak model misread which items the shopper had saved: two runs added black, two loosened white to “neutral”, and their picks broke the brief in 20, 12, 12 and 8 of 20.
- More thinking didn't save a small model. No reasoning setting brought it close; a strong model on three of the four steps scored about the same as the strong model throughout.
- Token price can mislead. GPT-6 Luna, among the cheapest per token, made 2.5× as many calls as GPT-6.1 Terra.
- Know what the score is. It is the whole agent's result, judged from product text by Anthropic models. It shows which models keep a collection's garment and colour, not how much they know about fashion or how the picks look.
What the shopper sees
Every night the agent builds a deck of 20 picks for each of a shopper's collections. In the morning the shopper swipes through it in the app.
- Every night, a new deck of picks for each collection.
- Save or dismiss. The shopper saves a pick into the collection or dismisses it; picks left alone count too.
- Steer. One question about where the next deck should go.
- Overnight, the agent rereads the collection, the saves and the dismissals, updates its account of the shopper's taste, and builds the next deck.
One collection over seven weeks
Here is that loop for one real collection, a shopper's everyday wardrobe, between 24 September and 10 November. Each column is one night's deck of 20. We leave out the shopper's name and the collection's title.
- 1 24 Sept “Contemporary Scandinavian minimalist wardrobe”
- 2 25 Sept “Tactile everyday casual capsule”
- 3 3 Oct “Scandinavian architectural minimalism”
- 4 8 Oct “Scandinavian architectural tailoring”
- 5 17 Oct “Refined Scandinavian everyday smart-casual”
- 6 21 Oct “Sculptural Scandinavian tailoring & expressive shirting”
- 7 26 Oct “Tactile Scandinavian minimalism”
- 8 7 Nov “Tactile cold-weather everyday”
Over seven weeks the shopper kept building the collection, from 22 to 113 items, often adding many at once. They opened 15 of the 48 decks and saved 25 picks, one to five at a time, without dismissing any. The agent's account of their taste followed: it started from a broad Scandinavian minimalist wardrobe, narrowed to architectural tailoring as the first saves came in, moved to smart-casual pieces and expressive shirting in late October, and turned to cold-weather everyday clothes in November.
The question
Fashion interest is hard to read. A collection called “white spaghetti dress” states three hard constraints, a colour, a garment and a strap (thin spaghetti straps), and implies softer ones: a length, a fabric, a price level, the brands this shopper tends to save. A collection called “Everyday” states almost nothing, and its purpose has to be inferred from the items in it. The evidence is a short title and a handful of items.
Our Discover Daily agent reads that evidence every night and picks 20 items for each of a shopper's collections. Which model it runs on is a decision we face whenever a new model ships, and general benchmarks say little about this domain. Production metrics are too slow to tell models apart, so we replayed the agent with each model, scored the picks, and split fit to a collection into five parts:
- Role
- What the collection is for, usually a garment category or a job: dresses, cold-weather footwear, office wear.
- Defining traits
- Hard constraints the name or the items make obvious: a colour, an occasion, a season, a fabric, a garment detail. A perfect dress in the wrong colour for a colour-defined collection is a miss.
- Taste
- Soft preferences in the shopper's saves and dismissals: brands, silhouettes, style.
- Price
- The price level of the collection and the shopper's saves.
- Personalisation
- Whether the deck as a whole reflects this shopper rather than generic items for the collection name.
The setup
We kept the agent exactly as it runs in production: four steps that write a taste report, write the collection's brief (a short summary of what the collection is for and which traits its items must keep), search the catalogue with tools, and curate the final 20. We replayed it against a snapshot of 16 real collections from 10 shoppers and the live catalogue, and changed only the model. We also checked the replay against production: replaying the deployed configuration came out close to its real decks on average, in mean cost ($0.090 against $0.094), retries (44% against 50%) and judged quality (1.91 against 1.84), though single decks for the same collection differed by up to 2.4 points.
| Model | Vendor | Reasoning |
|---|---|---|
| Gemini 3.1 Flash Lite | default, low, medium | |
| Gemini 3.5 Flash Lite | low | |
| Gemini 3.8 Flash | low (reference) | |
| Mixed: 3.8 Flash interprets, 3.1 Flash Lite searches | low | |
| GPT-6 Luna | OpenAI | medium |
| GPT-6.1 Terra | OpenAI | low |
| GPT-6.1 Sol | OpenAI | low |
| Claude Haiku 5.5 | Anthropic | low |
| Claude Opus 5.5 | Anthropic | low |
Every pick was scored 0 to 3 for one question: would this shopper see this item in this collection's deck and think it belongs? Judges flagged picks that broke the collection's role or a defining trait, clashed with the shopper's taste or missed the price level, and scored each deck for coherence, variety and personalisation. The judges were Claude Opus 5.5 agents working blind, one per collection; Claude-built decks were scored by a second judge model, Fable 5.2. A person then compared unlabelled pairs of decks, with photos.
Results
Three OpenAI models lead, and they are the only configurations clearly better than Gemini 3.8 Flash on the same collections. Claude Opus 5.5, Gemini 3.8 Flash, Claude Haiku 5.5, Gemini 3.5 Flash Lite and the mixed pipeline form a second group. Every Gemini 3.1 Flash Lite configuration sits well below. Sixteen collections do not resolve the order within the groups. The same grouping holds on the 11 customers' collections alone. Claude Opus 5.5's place is the least settled: its judge comes from the same vendor, and on a dozen votes the person ranked Claude-built decks lower than the judges did.

Keeping to the brief is the main separator
Breaking the scores down shows where models differ. The Gemini 3.1 Flash Lite configurations broke the collection's role or a defining trait in 16 to 25% of their picks. Gemini 3.8 Flash, Gemini 3.5 Flash Lite, Claude Haiku 5.5 and the mixed pipeline did so in 4 to 8%, and the OpenAI models in at most 0.9%. Claude Opus 5.5's 2.5% comes from the second judge, who gave Gemini 3.8 Flash 2.5% too, so those two are level. Strong models are distinguished less by more perfect picks than by almost no misses. These breaks are constraints not kept, the wrong garment or the wrong colour, measured on the whole agent. They are not a test of fashion knowledge.

Judges rarely flagged a pick as clashing with the shopper's taste, 1.6 to 6.6% of picks, but they saw only a few of each shopper's saves and dismissals, so this study says little about taste. The weak model also skipped part of the job: Gemini 3.1 Flash Lite asked the shopper its steering question in 1 to 3 of 16 decks, and every other model in all of its decks or all but one.
Personalisation may separate the strongest models
Among models that rarely break the brief, deck personalisation still ranged from 3.44 (GPT-6 Luna) to 4.19 (GPT-6.1 Sol), with Claude Opus 5.5 at 4.12 from the second judge. Luna keeps to the brief as well as Sol but builds less personal decks; that gap is the only one among these models that is clear of noise. For the highest-scoring models, fit is close to what a text rubric can measure, and personalisation is where they may still differ. No shopper has checked this score.

Where the colour is lost
The agent's step reports let us follow one collection, white strappy dresses, through every step. Every run read the same evidence: the collection, and the shopper's reactions to earlier decks. All three saves were white dresses, from a deck of white dresses shown from 1 to 12 November. The newest deck, with black, brown, blue and grey pieces among the white, had no saves yet; it had been out for a few hours. The first step reads that history and writes a taste report. The second writes the brief, whose “Must have” line every later step works from. Figure 4 shows both, word for word, and the 20 items each run picked.
What every run read
“white spaghetti dress”: 30 items, 3 saved from earlier decks











+18Shown 1 to 12 November: 3 saved
Shown from 13 November, hours before the runs: none saved
- Gemini 3.1 Flash Lite, no retry fix Brief adds black 0.00
Taste report: “The palette is intentionally quiet—relying on off-white, cream, and classic black …”
Brief, must have: “Spaghetti straps (essential), minimalist column or bias-cut midi/maxi silhouette, neutral color palette (cream, ivory, optic white, black), and lustrous or tactile fabrics (satin, viscose, fine linen-blends).”
- Gemini 3.1 Flash Lite Brief drops colour 1.05
Taste report: “… the user's lack of saves suggests this was too restrained. I am correcting this by … expanding the palette into deep black …”
Brief, must have: “Spaghetti straps, bias-cut or fluid body-skimming silhouette, clean/minimalist design, midi or maxi length.”
- Gemini 3.1 Flash Lite, low reasoning Brief loosens to neutral 1.60
Taste report: “The palette should remain neutral (white, cream, ecru).”
Brief, must have: “Spaghetti straps, bias-cut or streamlined column silhouette, neutral color palette (white, cream, ecru), accessible price point (under 120 EUR).”
- Gemini 3.1 Flash Lite, medium reasoning Brief loosens to neutral 1.75
Taste report: “First run — no prior hypothesis … The palette is strictly neutral (optic white, cream, ecru).”
Brief, must have: “Spaghetti-strap detailing, solid neutral palette (optic white, cream, ecru), and a fluid, minimalist silhouette (satin, linen-viscose, or clean cotton).”
- GPT-6 Luna Brief keeps white 2.75
Taste report: “None of the 20 items in the latest set was saved, while the three earlier saves were white/off-white thin-strap dresses … I’m narrowing the hypothesis to that literal color-and-strap fit.”
Brief, must have: “White, cream, or off-white color; dress silhouette; slim or spaghetti straps; a wearable, feminine shape, from flowing to gently fitted.”
- Gemini 3.8 Flash Brief keeps white 2.75
Taste report: “… every black, brown, and blue slip was ignored. The user strictly requires white, off-white, and cream spaghetti strap dresses …”
Brief, must have: “Slender spaghetti/string straps (sleeveless), strictly optic white / off-white / ecru / cream colorway, fluid or relaxed feminine dress silhouette (satin, fluid viscose, cotton broderie, or lightweight lace).”
The evidence points one way: every save was white. GPT-6 Luna and Gemini 3.8 Flash read it that way and narrowed the brief to white. Two Flash Lite runs kept or added black in the taste report, and their briefs followed: one required black, the other dropped colour. 20 and 12 of their picks were misses. The other two loosened white to “neutral” and chose beige, sand and grey dresses, sleeves and a nightdress for 12 and 8 of their 20 picks. So the weak model's errors began before the brief, in reading what the shopper had saved. We traced only this collection step by step.
Configuration did not close the gap
Gemini 3.1 Flash Lite scored 1.80 at low reasoning, 1.94 at its default and 2.12 at medium, at about the same cost. Only the gap between medium and low is clear of noise, and medium still sat 0.40 below Gemini 3.8 Flash. Using 3.8 Flash for the taste, brief and curation steps, with Flash Lite only for search, scored 2.42: not distinguishable from 3.8 Flash throughout, at 63% of its cost. That design changes three steps at once, so it does not show which of them matters.
Is it the prompt, or the model?
A fair objection is that a better prompt would rescue the small model. Every model got the same instructions, word for word. They were written while Gemini 3.8 Flash ran the feature, and three OpenAI models beat 3.8 Flash with them, so they are not simply tuned to one model. The first step's instruction already asks for the comparison that settles the white-dress case:
“What do the not-kept items have in common … that the kept ones don't? That difference is your correction signal.”
The stronger models made that comparison. None of the four Flash Lite runs did: two kept or added black, two loosened white to “neutral”. More reasoning helped it only a little, from 1.80 to 2.12.
The instructions do leave openings. They never say that a trait in the collection's name is fixed. They warn that sharing a colour is not reason enough to pick an item. They count every unsaved item as a wrong guess, even one the shopper has not seen yet, and require a correction, and the first step sees product views from anywhere in the app. A prompt that closed these might lift Flash Lite; we have not tested one. If it did, it would show that the small model has to be told what the larger ones work out, and each such rule tends to be written only after a shopper has seen the failure.
Would an ensemble close the gap?
Another objection is that several cheap runs combined would match one strong run. Our data put a ceiling on this. Suppose a perfect selector, using the judges' own scores, kept the best of the four Flash Lite decks for each collection. The mean would be 2.46: level with Gemini 3.8 Flash, but 0.28 below GPT-6 Luna (95% interval 0.10 to 0.47) and 0.36 below GPT-6.1 Sol, for four decks' cost, about $0.32 against Luna's $0.13.
Pooling the four runs' picks, about 75 different items per collection, and keeping the 20 the judges scored highest gives 2.82, level with Luna and Sol. So the cheap model's searches do find enough good items. What it lacks is telling them apart, and an ensemble needs a selector that can; in practice that is a stronger model. Our mixed setup is that design: 3.8 Flash for the taste, brief and curation steps and Flash Lite for search, at 2.42.
Cost per finished deck

Median cost per deck ranged from $0.05 (Claude Haiku 5.5) to $1.51 (Claude Opus 5.5). The frontier, the models that no other model beats on both cost and score, runs through Haiku 5.5, GPT-6 Luna ($0.13), GPT-6.1 Terra ($0.27) and GPT-6.1 Sol ($0.56). Token price alone was a poor guide, as one case shows: Luna has the lowest price of the OpenAI models but made a median of 50 model calls per deck against Terra's 20, and was the slowest at 328 seconds. Luna and Terra also had many search queries rejected for naming a field the catalogue index does not have; sending them again added about 4% to their calls and did not change their scores. Opus paid 80% of its cost in prompt-cache writes under our cache settings, and one Sol run cost $11.75 against a $0.56 median.
Can the judges be trusted?
Two runs of the Opus judge agreed exactly on 88% of 420 items (quadratic weighted kappa 0.955, where 1 is perfect agreement). Opus and Fable agreed exactly on 82% of 1,900 items and within one point on 99%. On decks built by Claude models, the Opus judges were slightly more generous than Fable, by 0.04 to 0.10. That is within noise, but enough to turn Claude Opus 5.5 from a tie into a win, which is why Claude decks are reported with the second judge's scores. Both judges are Anthropic models, so a judge from another vendor is still to do.

The person agreed with the judges on 9 of 12 pairs where the judges saw a clear gap and was at chance on close ones. The pairs were unlabelled, but he had seen the overall results and the app named the models after each vote. In the six votes where the judges preferred a Claude-built deck he agreed once; that is too few votes to settle, and both judges are from Anthropic. On single items the person was much stricter about visual style: several white slip dresses the judges scored 3 got 0 or 1, because the photos showed a cut or fabric the collection did not have. Text-only judges can check stated traits against product text. They cannot check what only the photo shows.
Beside classic recommenders: our view
This section is our view, not a result: we ran no classic recommender in this study. Most recommenders learn taste from behaviour: collaborative filtering, embedding retrieval and rankers trained on what shoppers clicked and saved. Once trained they cost a tiny fraction of a cent per recommendation. An agent like ours costs $0.05 to $1.51 per deck of 20 and takes minutes, so it has to earn its place.
It works before there is data. Within Discover Daily alone, the sampled shoppers' last two months showed 26,122 picks and produced 197 saves, 0.75% of those shown. A collection created today has no history at all. A language model can act on a new collection's stated intent on the first night.
Similarity is not a constraint. In an embedding space we would expect a black slip dress to sit close to a white one. The brief makes the colour a rule. It is not either/or: the agent's search already runs on a keyword and vector index.
Taste lives in details, and many are only in the photo. Shoppers are picky about small things, the width of a strap or the sheen of a fabric, and a few saves say little about which of them matter to one person. On the white-dress collection, the person rejected items the text-only judges accepted because of details only visible in the photos. The agent has a tool that sends product photos to a vision model; the models used it in 5 of 191 decks, and the judges never saw images. Visual taste is untested here.
Every surface is training data, and item features are cheap. Each time the app shows a set of items, in search, browse, the community feed or a deck, what the shopper opens, saves or dismisses is an example to learn from. A model trained on all of it could score an agent's deck and tell us whether judged fit predicts what shoppers do. A language or vision model can also run once per item and describe it in fine detail: strap width, neckline, fabric, fit. Those features turn a collection's hard traits into filters, and much of what this study scores, keeping to garment and colour, could move into them.
For a startup, speed and cost decide. An agent on a hosted model gave us a working, explainable recommender without training data or ranking infrastructure, and this evaluation compared 12 configurations in a day for about $73 in the agent's model calls. At $0.13 per deck, 100 new decks a day cost about $390 a month, far less than the engineering time the classic route needs. The hybrid of item features, an engagement model and embedding retrieval is worth revisiting once volume makes the per-deck cost the bigger bill.
What we take from it
- Read every step's report, not only the picks. On the collection we traced, the weak model's error began in how it read the shopper's saves, two steps before any item was picked. The short report each step writes shows where.
- Mix models by step, not by runs. More thinking time did not rescue a weak model, and even the best of four of its decks stayed below the strong models. A strong model on three of the four steps scored about the same as using it everywhere, for 63% of the cost.
- Price the whole task, including the worst run. The OpenAI model with the lowest token price made the most calls per deck.
- Check the judges against each other and against a person. Use a judge from another vendor, and give the judges the photos: text cannot show how a garment looks.
All results
| Configuration | Score [95% CI] | Δ vs 3.8 Flash | Won-lost | Breaks brief | Personal | Cost |
|---|---|---|---|---|---|---|
| GPT-6.1 Sol | 2.81 [2.71, 2.90] | +0.29 | 12-2 | 0.9% | 4.19 | $0.56 |
| GPT-6.1 Terra | 2.80 [2.72, 2.88] | +0.28 | 12-2 | 0.0% | 3.94 | $0.27 |
| GPT-6 Luna | 2.73 [2.62, 2.85] | +0.22 | 13-2 | 0.9% | 3.44 | $0.13 |
| Claude Opus 5.5 † | 2.63 [2.50, 2.76] | +0.06 | 7-7 | 2.5% | 4.12 | $1.51 |
| Gemini 3.8 Flash (production) | 2.58 [2.40, 2.74] | +0.06 | 8-5 | 4.1% | 3.69 | $0.52 |
| Gemini 3.8 Flash (reference) | 2.52 [2.37, 2.66] | reference | 5.3% | 3.75 | $0.38 | |
| Claude Haiku 5.5 † | 2.46 [2.35, 2.58] | −0.09 | 6-8 | 6.7% | 3.27 | $0.05 |
| Gemini 3.5 Flash Lite | 2.46 [2.23, 2.67] | −0.06 | 9-6 | 6.9% | 3.31 | $0.11 |
| Mixed: 3.8 Flash + 3.1 Flash Lite | 2.42 [2.29, 2.55] | −0.10 | 7-7 | 8.1% | 3.38 | $0.24 |
| Gemini 3.1 Flash Lite, medium reasoning | 2.12 [1.84, 2.36] | −0.40 | 2-11 | 15.9% | 2.69 | $0.09 |
| Gemini 3.1 Flash Lite | 1.94 [1.54, 2.32] | −0.58 | 4-11 | 23.4% | 2.56 | $0.08 |
| Gemini 3.1 Flash Lite, no retry fix | 1.91 [1.49, 2.27] | −0.61 | 4-11 | 19.7% | 2.44 | $0.08 |
| Gemini 3.1 Flash Lite (production) | 1.84 [1.47, 2.17] | −0.68 | 1-14 | 23.4% | 2.44 | $0.08 |
| Gemini 3.1 Flash Lite, low reasoning | 1.80 [1.44, 2.13] | −0.72 | 0-13 | 25.0% | 2.25 | $0.07 |
Score: mean judged item score, 0-3. Δ: paired difference from Gemini 3.8 Flash on the same collections. Breaks brief: picks that break the collection's role or a defining trait. Personal: deck personalisation, 1-5. Cost: median per new deck at list prices. † Scored by the second judge model (Fable 5.2), with Δ against Gemini 3.8 Flash as that judge scored it (2.57).
Limits
The score is the result of the whole agent judged from product text. It mixes reading the collection with tool use and catalogue tags, so it is not a test of fashion knowledge on its own. One run per configuration on 16 collections, so differences under about 0.15 points are not resolved. The collections were not drawn at random: five are staff's, and one of those is where we first saw the failure that started this study. Reasoning settings are not comparable across vendors, and each model was tested in our agent as built rather than one tuned for it. The judges read text, not photos, all of them are Anthropic models, and two judge models share the results table. One person, an author, cast the votes. Fifteen of the 16 collections are women's fashion. The method, statistics and appendices are in the paper. Questions? Write to engineering@favora.ai.
Who did what
Claude Opus 5.5 did the work: the investigation, the replay and judging setup, every experiment and analysis, the primary judging, the voting app, the figures, the screenshots, the code and the text as first written. Fable 5.2 was the second judge. Two further AI reviewers, both Fable 5.2, then checked and edited the report: one audited the numbers against the experiment records, and one read only the report and narrowed its claims.
Alexander Viala Bellander directed and reviewed. He spotted the failure in his own white-dress collection, asked for the model comparison and its rubric, added the OpenAI models and the Claude models (with a second model judging them), had the voting app built and cast its 33 votes, and chose the production model. On the write-up he set the framing (the method rather than our incident, takeaways an outsider can use), steered the comparison with classic recommenders (engagement models trained on every surface, per-item feature extraction, what a startup can afford), corrected errors such as treating thin straps as a soft trait, and required real screenshots, anonymised data and references that were searched for and read. He wrote no code or text and has not read the references. He is accountable for this report.
The paper's 35 references come from a logged literature search (188 queries across Semantic Scholar, arXiv and the web), and Claude agents read every one in full. An earlier draft that cited papers from memory was discarded.