 |
|
|
|||||
Its the last week of Black History Month (BHM) and its clear Americans are over performative values. Trite BHM-inspired merchandise sits on retailer shelves untouched while media is abuzz covering the artistry, activism, and symbolism of Bad Bunnys Super Bowl halftime show. The signal is clear: consumers are looking to brands for real solutions to real problems, not products that commodify culture. Most companies build everything from advertising to AI for the “average user,” but in doing so, they react to rather than lead markets. Strategic leaders look to growth audiencesunderserved groups who are the fastest-growing demographicsas lead users. They are the “canaries in the coal mine” because they navigate the highest levels of systemic friction, making them the first to experience “average” design failures. What does championing these lead users look like at a communications, product, or systems level? It looks like Elijah McCoy automating engine lubricationan innovation bred from the friction between his engineering degree and the menial labor he was forced to perform, thus creating the real McCoy quality standard. It looks like Jerry Lawson changing the economics of the gaming industry by inventing the video game cartridge that divorced its hardware from its software. And it looks like emergency medicine becoming a global standard after being piloted by the Pittsburgh Freedom House Ambulance Service who, in the face of medical bias and systemic unemployment, also redefined emergency care as a public right. Drawing from their lived experiences in underserved groups, these pioneers didnt just solve problems; they mastered environmental friction. Today, that friction also manifests in algorithms. Championing growth audiences as lead users means ensuring they are critical AI system “stress testers.” When we fail to design for them, we allow AI data, development, and deployment to default to obtuse “averages” that can frustrate or drive away valuable customers. Three recent examples highlight issues and opportunities. Relying on ‘Data Infallibility’ versus Lived Realities In this Infallibility Loop bias, a brands AI trusts a data sourcelike a flawed GPS coordinate or outdated government mapas an absolute truth, even when customers provide contrary evidence. This is a digital echo of historical redlining: a systemic refusal to see humans over faulty data. The Experience: A Black homeowner in an affluent area is penalized by an AI that confuses her address with a property in a different town, automatically forcing unnecessary flood insurance onto her mortgage and increasing the payments. Despite providing human-verified deeds and highlighting known GPS errors, the AI blocks her incomplete payments and triggers automated credit hits. A resolution only came months later after the consumer filed state-level servicer complaints. The Fix: Prioritize Dynamic Qualitative Data Collection. Design should allow real-time, contextual evidence to override static, biased datasets. True brand innovation requires systems to yield to the experts: their customers. Leveraging ‘Data Intimacy’ while Neglecting Situational Accuracy This trust paradox occurs when brands use private data, but fail to combine situational data, making personalization feel like needless surveillance. The Experience: During Januarys recent record-breaking New York snowstorm, a customer called a national pharmacys location in her neighborhood to make sure they were open. The AI-powered interactive voice response (IVR) recognized her number, asked for her birthdate, and greeted her by name. Yet, after performing this exchange, it provided a “default” confirmation that the store was open when asked. Without a car, the customer braved life-threatening conditions on foot only to find a handwritten note on the door indicating it had closed due to the storm. The Fix: Add Good Friction. A term coined by MIT professor Renee Richardson Gosline, “Good Friction” requires that when external context (like a Level 5 storm) conflicts with standard scripts, the system pauses and verifies first. Prioritizing ‘Recency’ But Erasing Loyalty Recency bias in algorithms weights the last data point more heavily potentially resulting in algorithmic erasure. The Experience: A 20-year elite status customer calls an airline, only to be greeted by the name of his niece (a nonmember relative for whom he recently booked a one-off ticket) and then is erroneously deprioritized in the automated journey as a nonmember. In many “growth audience” and immigrant households, economics are multigenerational and communal, with a single “lead user” facilitating purchases for extended family. This airline systems “memory” was shallow, seeing only the most recent transaction and ignoring a decades-long relationship because a reservation shared the same contact number. The Fix: Focus on Holistic Design. AI must be weighted to recognize the arc of the customer journey, ensuring that loyalty isnt erased by a single data point or the nuances of communal purchasing. To be sure, bad data is a universal problem, but the lack of situational intelligence in our AI systems hits growth audienceslike Black consumersfirst and hardest. Because these audiences represent a disproportionate share of future consumption and have the most “cultural common denominators,” their frictions are diagnostics for markets writ large. We arent just solving for a niche by championing them as lead users, we are adopting more rigorous, empathetic, expansive, and effective standards that solve real problems for all people.
Category:
E-Commerce
At hundreds of Burger King restaurants across the U.S., theres a new invisible worker whos tracking which ingredients are in stock, analyzing daily sales data, and checking in on whether employees are saying Thank you and Youre welcome. Its an AI assistant named Patty. According to Thibault Roux, Burger Kings chief digital officer, the voice-activated chatbot is designed to help employees and managers handle tasks that might usually require pulling out a computer or consulting with an instruction guide. Patty began showing up at select locations about a year ago, and is now in a pilot phase at approximately 500 Burger Kings. Its expected to roll out to the rest of the chains U.S. locations by the end of the year. On a day-to-day basis, Patty has an array of functions, from letting a manager know if a store is low on onions to helping an employee build a new burger. But it has another role thats raising quite a few eyebrows: analyzing Burger King locations based on friendliness by tracking employees use of key phrases like Welcome to Burger King, Please, and Thank you. Online, commenters are concerned that this functionality is a slippery slope toward 1984-style employee surveillance. In an interview with Fast Company, though, Roux clarified that Patty is not being used to analyze individual employees performance, and is instead imagined as a kind of coach. It’s truly meant to be a coaching and operational tool to really help our restaurants manage complexities and stay focused on a great guest experience, Roux says. Guests want our service to be more friendly, and that’s ultimately what we’re trying to achieve here. Patty, are we running low on Diet Coke? Technically, Patty is the chatbot version of Burger Kings assistant platform, which collects data from operations including drive-through conversations, inventory, and sales, and then uses AI to analyze patterns in that data. For now, Patty operates on a customized model from OpenAI, though Roux says the technology is flexible enough that it could integrate with another partner in the future (like Anthropic or Gemini) depending on the companys needs. For managers and employees in stores, Roux says Patty operates similarly to something like Siri. Patty is activated by a small button on the side of an employees headset, and they can ask it direct verbal questions related to their specific storelike recent sales figures or inventory updatesas well as more general company information, to which the bot will provide a verbal answer. If you’re looking to clean the shake machine [you can ask Patty] the procedures to clean it, Roux explains. Or we have a lot of limited-time offers, and sometimes they can be cumbersome to remember. You can easily tap into Patty and be like, Hey, remind me, does the new build maple bourbon barbecue have crispy jalapeos? Patty can also reach out to employees directly if it notices a pattern of interest. For example, if Patty thinks a specific store is out of lettuce, it might ping a manager to confirm. Once its received confirmation, it can mark lettuce as sold out on that locations app and websitea process that previously would have required human intervention. Roux says franchisees and regional managers can decide how they want Patty to reach employees with information, whether its through their headsets or via a text message (though the tech is programmed explicitly to never interrupt a worker during a customer interaction). Insights from Burger Kings Assistant platform also live outside of employees headsets. Managers can check information from the tool on an accompanying website or app. For example, Roux says, when a district manager is visiting a new store, they might ask Patty on the app, What are the top three guest complaints at this location this week? or What are their top missing items? In an interview with Fast Company writer Jeff Beer earlier this month, Burger King President Tom Curtis said the assistant platform has already led to some significant menu changes. Curtis explained that the AI tracked all the times that team members said Im sorry, we dont have that and linked them back to a common denominator: apple pie. In January, Burger King brought back its apple pie for the first time since 2020. Were in the idiocracy version of 1984 Pattys more straightforward uses, like helping managers access sales data and check inventory, seem fairly predictable in the context of fast food. Where Burger King is really pushing Pattys use cases, though, is with its friendliness metric. In an interview with The Verge on February 26, Roux said Patty would recognize phrases like Welcome to Burger King, Please, and Thank you, and then give managers access to data on their locations friendliness performance based on those keywords. Mere hours after that piece went live, a thread in the subReddit r/technology on Patty had already amassed more than 15,000 upvotes and nearly 3,000 comments. Common refrains from users include comparing the technology to the surveillance state in George Orwells novel 1984, labeling it authoritarian and dystopian, and accusing Burger King of employee surveillance. “This would be criticized as being cartoonishly unrealistic in a sci-fi movie 10 years ago,” one user wrote. Another added, “We’re in the idiocracy version of 1984.” When asked about this response, Roux says the data from employees conversations is anonymized, and that none of these friendliness metrics will be used for grading or assessing individuals. Further, he adds, Patty will not directly instruct employees on what to say or how to say it. Instead, data on friendliness will be shared with managers, who can use it for face-to-face coaching with their teams. Still, its unclear exactly how Patty is quantifying friendliness. In a video explanation of the feature, a manager is shown asking the bot, Is there anything that needs my immediate attention? to which it responds, The teams friendliness scores this morning were the highest this week. In an email to Fast Company, a Burger King spokesperson said, In select pilot locations, weve explored using aggregated keywords, including common hospitality phrases, as one of several signals to help managers understand overall service patterns. The tool is not used to score individuals or enforce scripts. Burger King did not respond to ast Companys request for clarification on how friendliness scores are calculated. So far, Roux says hes seen growing interest in Patty from franchisees, with several managers making specific requests for future add-ons. A lot of our franchisees . . . and regional general managers are very competitive, so they want to know, Hey, how do I compare to other restaurants? Roux says. I think that’s something that we’re going to be rolling out. In fact, we were looking at some of the designs earlier this week with the franchisees. So this is only the beginning.
Category:
E-Commerce
Recently, Grok AI faced criticism after users found it was creating explicit images of real people, including women and children. Although xAI has now implemented some restrictions, this incident revealed a serious weakness. Without safeguards and diverse perspectives, girls and women are put at greater risk. The dangers artificial intelligence poses to women and girls are real and happening now, affecting their mental health, safety, healthcare, and economic opportunities. Last fall, a mother discovered why her teenage daughter’s mental health had been deteriorating: It was a result of conversations with a Character.AI chatbot. She’s not alone. Aura’s State of Youth Report, released in December, found that parents believe technology has a more negative effect on girls’ emotions, including stress, jealousy, and loneliness51% compared with 36% for boys. Thats unacceptable, and we need to do better. The risks extend beyond mental health. OpenAI recently reported that more than 40 million Americans seek health information on ChatGPT daily. As AI in healthcare expands, the consequences of biased training data can be dangerous. AI models that are trained predominantly on male health data produce worse outcomes for women. For instance, an AI model designed to detect liver disease from blood tests missed 44% of cases in women, compared with 23% in men. Uneven playing field In the workplace, AI is not leveling the playing field. Despite laws prohibiting discrimination, AI-powered hiring tools have repeatedly caused concerns about bias, fairness, and data privacy. A study published by the University of Washington found that in AI resume screenings, the technology favored female-associated names in only 11% of cases. These failures reflect who is building our technology. Women make up just 22% of the AI workforce. When systems are designed without women’s perspectives, they replicate existing inequities and introduce new risks. The pattern is clear. AI is failing girls and women. Pivotal moment This could not come at a more pivotal moment in the job market. A quarter of the roles on LinkedIns latest list of the 25 fastest-growing jobs in the United States are tech-related, with AI engineers at the top. Decisions about how AI is designed today will shape access to jobs, healthcare, education, and civic life for decades. It is critical that women play an active role in developing new AI tools so that inequity is not baked into the systems that increasingly govern our lives. Young women are not disengaged with AI. Research conducted last year by Girls Who Code, in partnership with UCLA, found that young women are deeply thoughtful about the dual nature of technology. They see its potential to advance healthcare, expand educational access, and address climate change. They are also aware of its dangers, such as bias, surveillance, and exclusion from development. This isnt blind optimism. Instead, it offers a perspective that is often missing in todays AI development. Creating technology is an exercise of power and holds great responsibility. Since girls are often the most affected by AIs failures, they must be empowered to help lead the solutions. Women like Girls Who Code alumna Trisha Prabhu, who developed ReThink, an anti-bullying tool, exemplify this. Latanya Sweeney, recognized as one of the top thinkers in AI, founded Harvards Public Interest Tech Lab. Their achievements demonstrate the potential when women lead in tech development. Smart steps If we want safer, more responsible AI systems, three steps are essential. First, computer science education should integrate social impact. Coding cannot be taught in isolation from its consequences. Students should learn technical skills alongside critical analysis of how technology shapes communities and lives. This approach produces results. For instance, one Girls Who Code student utilized the skills she learned to create an app called AIFinTech to help immigrant families manage their personal finances. Second, women must be represented in AI development and governance, particularly those from historically underserved communities. They need seats at the tables where AI systems are designed, tested, and regulated. This means ensuring gender diversity on AI ethics boards and that government AI committees are representative of the demographics most affected. Finally, how we evaluate artificial intelligence needs to evolve. Today, AI is assessed by efficiency, accuracy, and profitability. We must also evaluate health, equity, and well-being, especially for girls and young women. Before an AI system is deployed in a high-stakes environment such as healthcare, it should be required to pass tests for gender bias and demonstrate that it does not produce disparate outcomes. New York City, for example, requires employers that use automated employment decision tools to undergo an independent bias audit annually. We do not have to accept AIs flaws by default. We are witnessing AIs impact on girls in real time, and we must seize the opportunity to change course while the technology is still being shaped. When girls are given the chance to lead in AI, they will build safer systems not just for themselves, but for everyone.
Category:
E-Commerce
All news |
||||||||||||||||||
|
||||||||||||||||||