AI shopping only works if you give up your privacy
As I wrote when OpenAI launched its Atlas browser, AI models simply don’t know your preferences well enough to shop on your behalf. The technology firms envision a future where we type “buy trainers” and they arrive at our door, but they don’t account for all the different types of trainers - brands, colours, styles, and other even tinier details that inform consumer choice.
When we shop, we make thousands of subconscious decisions. Not only would these be exhausting to describe in a prompt - we’re often unaware of them ourselves. Our preferences are expressed in desires we can’t accurately explain. This creates a problem: either people won’t use AI shopping, or conversion rates will plummet as AI presents products that miss the mark.
Profiling as a competitive advantage
AI’s usefulness in online retail requires us to sacrifice the privacy we fought for after scandals like Cambridge Analytica. User profiling that currently delivers tailored adverts will become a core part of AI platforms. The barrier is removed if the user doesn’t need to describe the specifics of what they’re looking for, because the AI assistant already knows what they like.
History has taught us that users are often more than willing to sacrifice privacy for convenience
Worse still, I fear that Big Tech may use AI as justification for invasive tracking. They might pressure users - “the future doesn’t work without your data - you’ll be left behind” - but history has taught us that users are often more than willing to sacrifice privacy for convenience of their own accord.
And they’re already laying the groundwork, directly explaining why this kind of profiling is needed. Nicholas Thompson, CEO of The Atlantic, recently had lunch with Sam Altman and asked him about Google’s challenge to ChatGPT:
“He mentioned that their biggest mode as he believes it is memory… This came up in the context of the question of whether Google Gemini has surpassed them… Their belief at OpenAI is that they have an advantage because the last couple of years they’ve been the market leader… and there’s the memory - your ChatGPT knows a whole bunch more about you and can answer questions more efficiently… The reason he has confidence that they’ll maintain their lead over Google is more about the memory than anything else.”
Next-level lock-in
Even if you're on board with giving up your personal data for easier shopping, there's an interesting aside here. Based on what Altman's saying, AI users could be subject to vendor lock-in even more intense than the Apple ecosystem, as switching to a new provider will mean starting from scratch and receiving low-quality answers until its model learns more about you.
Could you opt out, and just continue shopping in the traditional way? Yes, for now. But just as there was a time when you missed parties if you weren’t on Facebook, and you now pay more in Tesco if you don’t have a Clubcard, there may come a time when the best deals are accessible only to AI agents.
The choice won’t be presented as one. As in the privacy scandals of the 2010s, AI profiling will be positioned as the default, with FOMO pushing acceptance. By the time users realise what they’ve traded away, AI assistants will know them too well to abandon. As we saw with the decline of social media, it’s a lot harder to give something up once you’re locked in.