Consumers who experience discrimination are significantly more likely to rely on AI-based recommendations than on human advice, driven by the fear of negative judgment and embarrassment, according to a study. While users view algorithms as neutral alternatives, experts warn that biased systems risk worsening the problem.
Shopping for dinner on a budget of 20 euros with a gluten-intolerant guest might involve asking a shop assistant, or it could mean consulting Hopla, the AI-powered shopping assistant launched by French supermarket chain Carrefour. Behind these ordinary choices lies a growing psychological shift in how consumers decide whom to trust when seeking guidance.
Why Discriminatory Experiences Drive Consumers Toward AI Advice
Decisions involving services ranging from car dealerships to financial consultancies often expose shoppers to unfair treatment. Studies show that certain groups face less favorable conditions in areas such as access to credit or insurance. Beyond economic consequences, these experiences trigger stress, undermine fairness, and alter future behavior.
Market interactions across restaurants, banks, and healthcare platforms sometimes feature differential treatment based on relevant commercial criteria, such as a VIP customer receiving premium service. However, discrimination occurs when service providers differentiate customers using irrelevant and illegitimate factors like skin color, ethnic origin, or gender. This mistreatment can remain hidden or unintentional while causing genuine psychological harm.
When individuals face unfair treatment or social judgment over irrelevant personal attributes, they often experience embarrassment. This social emotion directly influences subsequent choices, leading people to limit human interactions and seek refuge in systems perceived as neutral and free from bias.
Algorithm Aversion Shifts to Algorithm Appreciation After Unfair Treatment
Extensive documentation shows that people typically prefer human advice over AI systems—a well-established phenomenon known as algorithm aversion, which persists even when algorithms perform objectively better. Yet, research highlights that this preference reverses when consumers face discrimination on platforms and offline.
Victims of public place discrimination are significantly more likely to turn to AI-based recommendations rather than human advice, even in subsequent domains completely independent of the original discrimination. This reversal transforms algorithm aversion into algorithm appreciation. Field studies demonstrate this shift clearly.
Researchers conducted a real-world field study where confederate agents assessed loan applications on a university campus using nearly identical questionnaires. One version introduced an additional question asking participants to self-assess their skin color to evoke discriminatory feelings. Although all applicant requests were subsequently rejected, participants who answered the skin-color question reported feeling discriminated against during phase one.
During phase two of the data collection, participants were offered a choice between a human assessor and an AI assessor. A significantly higher proportion of those who felt discriminated against in the first phase chose the AI system, viewing it as devoid of emotions and biases. Opting for AI allows consumers to minimize the risk of feeling judged or treated unfairly once again.
The Hidden Risks Behind Perceived Algorithmic Neutrality
Compared to human beings, algorithmic systems are frequently perceived as more neutral, more objective, and less prone to prejudice. The emotional mechanism powering this shift is the desire to escape negative social judgment. Yet, the underlying reality remains far more complex than consumers realize.
Algorithms can harbor biases depending on the data and training they receive. When AI systems themselves carry biases, they risk exacerbating the exact discrimination that consumers tried to escape by avoiding human advisors. While public perception treats machines as objective arbiters, organizations deploying these tools must confront whether their systems truly operate without prejudice.
