Beware of bias at bat: when AI benches the wrong players | Opinion

Listen to this article

This year, I took on the task of coaching two Little League teams. It is fun, exhausting, informative, and provides great preparation for thinking on the fly with hostile witnesses (because let’s be honest — a team of 8 three-year-olds is filled with irrational takes).

To keep things equitable, I continually rotated players through the various field positions via a snake-style rotation. It was a flawless system, or so I thought, until I learned that one family was unable to show up for certain weekday games. The impact was that their son and daughter never got to play first base. I never realized this until the kids brought this up to me during the last game. While my actions were not deliberate, the algorithmic system had an unintended negative result.

Similarly, in mid-May a federal judge cleared the way for a nationwide class action against Workday, finding that its AI-driven hiring tools may have unfairly screened out applicants older than 40. The case sent a clear message: employers and AI vendors (and Little League coaches, in my experience) can face liability for algorithmic bias — even when the discrimination isn’t deliberate. Following is a brief review of the facts of the case, why the ruling matters, and next steps for employers.

Facts

The plaintiff, a Black man over age 40 who lives with anxiety and depression, applied to more than 100 jobs at companies using Workday’s AI hiring tools (i.e., personality and cognitive tests) — and was rejected every time. The plaintiff sued Workday and the judge allowed the case to go forward on the question of whether Workday’s AI recommendation system had a disparate impact on applicants over 40.

Why the ruling matters

This lawsuit is one of the first major challenges to AI-based hiring practices under federal discrimination laws. It shows that employers can still be held liable if vendor tools unfairly reject protected groups, even if the bias is not intentional. Moreover, courts may treat screening systems as a “unified policy” even when different employers use the tools differently.

Next steps

When confronted with potentially disparate impact in my Little League lineups, I had to make quick adjustments on the fly. Similarly, employers need to be nimble and proactive to limit exposure related to similar AI hiring tools. Here are three steps your company should be taking:

1. Create AI governance framework that sets guardrails on how your company is going to use these algorithmic tools. Moreover, the crux of this is who watches the watchmen. Make sure you have some oversight of how to monitor the outcomes of your AI. This means you should provide training to teach employees when to override algorithmic rankings, and to audit results for desired (and nondiscriminatory) outcomes.
2. Keep clear records of hiring choices and the reasons behind them. Be cautious with tools that rely on vague or unexplainable scoring systems (i.e., the candidate is a “good fit,” a “good guy,” or a “personality hire”).
3. Ask vendors for documentation showing how their systems are tested for bias; if they balk, look harder. Insist on contractual assurances like data transparency and nondiscrimination. Ask about each vendor’s approach to bias mitigation or whether the vendor even conducts any bias audits.

As AI steps up to the plate in the hiring game, companies cannot sit back and hope the system plays fair. As in Little League, everyone deserves a chance to play first base — and it’s up to coaches (here, the companies) to make sure the rules are clear, the tools are tested, and no one gets left on the bench.