The New York City Council passed a bill that requires HR vendors selling AI-driven hiring screenings to undergo a yearly bias audit starting January 1, 2023. The bill is designed to ensure that hiring tools don’t use biased algorithms or standards when they select applications to move to the next step of the hiring process.
The bill also requires companies to disclose the use of AI screening tools to applicants, allows candidates to request an alternative screening process, and requires companies to disclose to candidates the type of data collected, how it is used in the screening process, and how long that data is retained.
The bias audit would presumably be run by a third-party company that would test how the algorithm reacts to several sets of applications where details have been engineered to trigger biased reactions, although details of what the bias audits required by the city would entail were not included in the bill.
How do AI hiring tools work?
Automated hiring tools screen application materials for keywords that match a pre-set hiring criteria, and reject applications that don’t meet these criteria. The tools are meant to help recruiters sift through piles of applications for suitable candidates quickly.
Recruiting and applicant tracking software vendors that employ screening AI programs often market these tools as unbiased. The vendor’s reasoning is that a computer doesn’t have to fight against the unconscious bias a human recruiter might bring based on associations they have with personal details like the candidate’s name, education, or previous employment.
The bias inherent in the system
Humans build AI, and humans build the tools to audit the AI. All humans hold biases, whether conscious or unconscious, and those biases are bound to leak into the data that is used to train the AI. No artificial intelligence system will be completely without bias, and as Protocol.com points out, critics have already mentioned the bill includes provisions for race and gender bias, but doesn’t include protections against age or disability.
The rest of the nation is watching
New York City is by far the most populous city in the US, and this provision would affect any company who employs those living in the city limits. In addition to adding a burden to HR teams within the NYC area, it could affect thousands of companies located outside of the major metropolitan area whose employees live in the city.
While this is far from the first of these bills to be proposed or even enacted in the US in the past year — Illinois enacted a similar bill governing the use of AI in video interviews — the implementation of the bias audit and data collection portions of the bill could help mold similar legislation around the country.
Top Human Resources Software Recommendations
Get our best stuff.
