The First Hiring-Algorithm Lawsuits: What New York's AEDT Audits Actually Caught

• Also see: The space opera renaissance
• Also see: Creatine, magnesium, omega-3
• Also see: Let the states keep writing AI law

In 2021, New York City passed a law that sounded procedural and turned out to be historic. Local Law 144, the automated employment decision tools law, said that any employer using software to substantially assist hiring or promotion decisions must first commission an independent bias audit, publish a summary of the results, and give candidates ten business days of notice that a machine will score them. Violations carry penalties of 500 to 1,500 dollars per day. Enforcement began in July 2023. By 2026, roughly 200 audit summaries had been published, the first lawsuits invoking the law had been filed, and Colorado had gone further with a statute that reaches well beyond hiring.

The audited tools are the quiet infrastructure of modern job hunting: resume screeners that rank applicants, video interview analyzers that score speech and facial movement, game-based assessments that infer traits from play, and internal promotion engines that shortlist existing staff. Vendors long claimed these systems reduce human bias by applying the same rule to everyone. The New York audits tested that claim with a specific yardstick, the impact ratio, and the results were messier than the marketing.

An impact ratio compares selection rates between groups. If 50 percent of male applicants advance and 40 percent of female applicants advance, the ratio is 0.8. Under the Equal Employment Opportunity Commission four-fifths rule, a ratio below 0.8 flags adverse impact worth investigating. Local Law 144 requires auditors to compute these ratios by sex, by race and ethnicity, and at the intersection of both, using the same scoring thresholds the employer actually uses. The audit does not grade the model in the abstract. It grades the deployed system, with its cutoffs, on the employer’s own applicant pool or a representative test set.

What did roughly 200 published audits actually catch? Three patterns stand out. First, most tools passed on the headline numbers and failed somewhere in the margins. A screener could show impact ratios above 0.85 for broad categories like male versus female, then drop to 0.65 or lower for a smaller intersectional group such as Black women over 40. Second, thresholds mattered more than algorithms. The same model at a top-10-percent cutoff and a top-30-percent cutoff produced different disparity stories, which means employers who bought a certified tool and then cranked the cutoff tighter were operating a different system than the one audited. Third, a meaningful minority of employers chose withdrawal over publication. Rather than post a marginal audit, they narrowed the tool out of scope, added nominal human review, or quietly stopped using it for New York applicants.

type: bar
title: NYC AEDT audit outcomes published by 2026
x: Passed, Marginal, Failed, Withdrawn
Count: 120, 45, 20, 15

The chart above stylizes the public corpus: about 120 clean passes, 45 marginal results with at least one subgroup near or below the four-fifths line, 20 clear failures, and 15 withdrawals where a posting or audit vanished after scrutiny. Exact counts shift as PDFs come and go from career pages, which is itself a finding. Transparency worked, but only where someone kept the receipts.

The lawsuits followed the audits, not the other way around. Plaintiffs lawyers did what the law’s drafters hoped they would do: they read the published tables. Early complaints allege failure to publish any audit while continuing to use covered tools, failure to give the required ten-day notice, and reliance on vendor assurances instead of an independent audit of the deployed configuration. A second wave targets the substance behind the paperwork, arguing that continued use of a tool with known sub-four-fifths impact ratios for protected groups supports disparate-impact claims under city, state, and federal antidiscrimination law. None of the marquee cases has yet produced a definitive appellate ruling on how much an AEDT audit proves in court, but motion-to-dismiss fights have already established that an audit showing disparity is discoverable, admissible context, and expensive to explain away.

Vendors sit at the center of the liability question. Most employers did not build these models. They licensed them, configured a cutoff, and trusted a sales deck that said independent testing was the vendor’s job. Local Law 144 places the legal duty on the employer deploying the tool in New York, not on the out-of-state vendor that sold it. That mismatch created a predictable scramble: employers demanding audit rights, re-testing clauses, and indemnification in procurement contracts, and vendors offering standardized audit packets that describe a reference deployment rather than the customer’s actual use. The lesson for buyers is blunt. A vendor packet is marketing until an independent auditor reruns it on your applicants, your thresholds, and your hiring funnel.

Colorado SB 205, the 2024 law on algorithmic discrimination in consequential decisions, expands the frame beyond hiring. Where New York asks about one decision type in one city, Colorado asks about consequential decisions across employment, education, financial services, housing, insurance, and essential government services. Developers and deployers of high-risk systems must exercise reasonable care against foreseeable algorithmic discrimination, complete impact assessments, provide notices to affected people, and, for deployers, stand up governance programs with annual review. Enforcement runs through the attorney general rather than a public database of audits, with rulemaking filling in assessment templates and rebuttable presumptions. In practice, Colorado trades New York-style sunlight for broader scope and a duty-of-care standard that plaintiffs and regulators can invoke across sectors.

The two models complement each other. New York showed that forced publication moves markets: tools with embarrassing ratios get tuned, re-thresholded, or retired. Colorado shows where the law goes next: from hiring to housing and credit, from published tables to documented risk management, from city penalties to statewide attorney general action. Employers operating nationally should assume that a hiring audit done for New York will be requested by a Colorado regulator, an EEOC investigator, or a plaintiffs lawyer asking about a lending or leasing model built by the same vendor on the same platform.

What should a responsible employer do now? The compliance checklist below distills what auditors, employment lawyers, and procurement teams converged on by 2026.

flowchart TD
    A["Employer deploys hiring tool"] --> B["Is it an AEDT"]
    B -->|"yes, screens candidates"| C["Commission bias audit"]
    B -->|"no, human decides"| D["Document human review"]
    C --> E["Check impact ratio above four-fifths"]
    E -->|"passes audit"| F["Publish audit and give 10-day notice"]
    E -->|"fails audit"| G["Fix vendor model or withdraw tool"]
    F --> H["Monitor and re-audit yearly"]
    G --> H

First, inventory every tool that substantially assists a decision, not just the ones labeled artificial intelligence. A rules-based filter that auto-rejects applicants without a keyword counts if nobody meaningfully reviews the rejects. Second, demand an independent audit of your configuration, not the vendor’s reference case, and require impact ratios disaggregated by sex, race, and intersectional groups at your real cutoffs. Third, validate that every reported ratio sits above four-fifths with a margin you can defend, and treat anything between 0.8 and 0.9 as a monitoring trigger rather than a clean bill of health. Fourth, publish the summary where candidates actually look, keep the full audit on file, and build the ten-day notice into the application flow rather than a footer link. Fifth, document human review where you claim it. A rubber stamp does not take a tool out of scope under New York guidance; the reviewer must have authority and time to override, and overrides should be logged.

Sixth, fix procurement. Require vendors to disclose training data, validation results, and known limitations, to cooperate with auditors, to notify you of material model changes, and to accept re-testing when you move thresholds. Push liability where the knowledge lives. Seventh, align with Colorado early even if you only hire there. Use the same impact assessment template for employment, run it for any credit, housing, or education model you deploy, and assign an owner for annual review. The attorney general standard is reasonable care, which in 2026 means written assessments, not vibes.

Critics argue the regime is theater: small penalties, easy scope games, audits that can be gamed by choice of test data. There is truth in each charge. Some employers plainly re-labeled automated screening as human decision-making without changing who decides. Some auditors worked from vendor-supplied samples with limited power to detect intersectional harm. Penalties of 500 to 1,500 dollars per day sting a small business but read as a line item to a national retailer. Yet the counterfactual matters. Before Local Law 144, almost nobody published anything. Candidates had no notice, no ratios, and no language for a complaint. After it, there is a public corpus, a plaintiffs bar that reads it, vendors competing on published fairness metrics, and a second state law carrying the logic into lending and housing.

The deeper shift is evidentiary. Hiring discrimination used to turn on emails and testimony about who said what in an interview room. Increasingly it turns on tables: selection rates, ratios, confidence intervals, cutoff memos, and version histories. That favors employers who keep records and punishes those who outsourced judgment to a black box and kept no notes. The four-fifths rule is not a legal safe harbor by itself, but a documented ratio above it, recomputed after every material change, is the difference between a company that can explain its hiring and one that can only describe its vendor.

New York did not solve algorithmic bias. It made it legible. Colorado will test whether legibility scales from one city’s job market to every consequential decision a model touches. Employers who audit the vendor, hold the line above four-fifths, give real notice, and write down the human part of the loop will survive that transition. Those waiting for Congress to preempt it all should reread the last three years: Washington talked, the states wrote, and the lawsuits learned to read.

Three principles for creating safer AIPlays from YouTube when you press play

---

By OMMAIS: Muse Spark 1.3 using Cline’s cloud API