United States: How Employers Can Prepare For NYC's Regulation Of Artificial Intelligence Tools

Key Takeaways:

• New York City Local Law 144 ("NYC Law 144"), which regulates the use of certain artificial intelligence (AI)-driven hiring tools in certain hiring and promotional decisions, will begin being enforced on July 5, 2023.
• NYC Law 144 applies to all employers and employment agencies, no matter where they are located, that hire employees who reside in New York City.
• Beginning July 5, 2023, covered employers will be subject to penalties if they fail to comply with the law's requirements, which include commissioning an independent bias audit; publishing the results of that audit; and disclosing the use of the AI-driven hiring tools to candidates and employees.
• Covered employers should evaluate their use of these AI tools and ensure compliance with NYC Law 144.

On July 5, 2023, the New York Department of Consumer and Workplace Protection ("DCWP") will begin enforcing NYC Law 144, which regulates the use of AI-driven hiring tools, referred to as Automated Employment Decision Tools ("AEDT"), in certain hiring and promotional decisions. AEDTs come in a variety of forms, ranging from tools that screen resumes for certain words to programs that evaluate a candidate's personality traits. NYC Law 144 applies to any AEDT that "issues [a] simplified output, including a score, classification, or recommendation, that is used to substantially assist or replace discretionary decision making for . . . . employment decisions that impact natural persons."

Although the law took effect earlier this year, the DWCP will begin penalizing employers who fail to comply with the law's requirements on July 5, 2023, issuing fines between $500 and $1,500 per day of noncompliance. To prepare for enforcement, employers should evaluate their use of AEDTs to determine whether they use an AEDT to make employment decisions that are covered by Law 144. If so, the employer must commission an independent bias audit; publish a summary of the results; provide notice to applicants and employees of the AEDT's use and functioning; and provide notice that affected individuals may request an accommodation or alternative selection process.

Does NYC Law 144 Apply to You?

NYC Law 144 applies to all employers and employment agencies, no matter where they are located, that hire employees who reside in New York City. There is no exemption in the law for small businesses.

Covered employers should evaluate their AEDT use and whether it implicates NYC Law 144 by consulting with counsel and their AEDT vendor to answer three threshold questions:

Although NYC Law 144 covers a broad range of technology, its coverage is limited to decisions about hiring and promotion. This includes screening and selecting for interviews candidates who have applied for a specific position or promotion as well as scoring them for a promotion or hiring. However, using an AEDT to aid in decisions regarding compensation, termination, workforce planning, labor deployment, benefits, workforce monitoring, and likely even performance evaluations, are beyond the reach of the law. NYC Law 144 also does not apply to AI-tools used to identify potential candidates who have not yet applied for a job.

The final regulations issued by DWCP specify that the use of an AI-tool is only within the scope of NYC Law 144 where the AI-tool's output – which may be a score, classification or recommendation – is used to "substantially assist or replace discretionary decision making." The rules clarify that the output must be used in one of three ways to be covered by NYC Law 144:

1. as the sole criterion in making the employment decision, with no other factors considered;
2. as a criterion that is given more determinative weight than any other criterion; or
3. to overrule conclusions derived from other factors including human decision-making.

Any other use of an AI-tool's output – for instance, where that ranking, score or classification is merely an equal part of the decision calculus for an individual or where a human decision maker can overrule the AI-tool's recommendation – would appear to exempt that tool from the definition of an AEDT under the statute. Accordingly, an employer using an AI-tool in hiring or promotional decisions in some other way does not need to comply with the law's bias audit requirements.

Importantly, however, employers should know that determinations of whether the use of an AI-tool makes it an AEDT subject to NYC Law 144's requirements will be a fact-specific determination that is subject to challenge by candidates and government agencies. Employers will need to be prepared to defend challenges to their conclusion that their use of an AI-tool is exempt from the law. Employers should make sure they understand exactly how these tools are being used in hiring and promotion decisions and should document the use of scores or ranking in their selection process so that they can demonstrate, in practice and on paper, their AI-tool use falls outside any of the law's covered uses.

Per NYC Admin. Code 20-870, a tool is covered by NYC Law 144 only if it is derived from "machine learning, statistical modeling, data analytics, or artificial intelligence," which term is defined, in the final regulations, to mean a group of mathematical, computer-based techniques that:

1. generate a prediction, meaning an expected outcome for an observation, such as an assessment of a candidate's fit or likelihood of success, or that generate a classification, meaning an assignment of an observation to a group, such as categorizations based on skill sets or aptitude; and
2. for which a computer at least in part identifies the inputs, the relative importance placed on those inputs, and, if applicable, other parameters for the models in order to improve the accuracy of the prediction or classification.

This definition restricts the law's reach to those predictive tools that employ the specific algorithmic techniques that comprise machine learning. As a practical matter, this means, for example, that where a test applied to candidates is designed by a human who identifies the necessary inputs and the relative importance of each input, the tool is likely outside the definition of AEDT and thus not within the scope of the law. For example, if an employer has a human programmer design a test to determine whether a candidate is likely to perform well in a particular position, and the test measures criteria X, Y, and Z as determined by the employer, and the employer decides the test will weight X as the most important criterion, the test involves enough human discretion that it is likely not an AEDT under the definition in the law. Conversely, if the tool is programmed to autonomously adjust the weighting of criteria based on prior data sets and outputs, such that the basis for its output may "evolve" to give more weight to criterion Y than X, it may well be a covered AEDT.

What to Do if NYC Law 144 Applies to You?

After determining that the use of an AI-tool is covered by NYC Law 144, employers must comply with four requirements before they can use or continue using the AI-tool:

1. Commission an independent bias audit. If an AI-tool is already in use, this audit should be performed as quickly as possible, and must be performed annually. This audit must be performed by "independent auditors," defined as objective individuals or groups who are not and have not been involved in the use, development, or distribution of the AEDT; have not at any point during the audit been employed by the employer, vendor, or AEDT developer; and who have no direct financial interest or material indirect financial interest in the use of the AEDT or the vendor that developed or distributed it. The bias audit will provide data about the selection rate of candidates in a number of categories (e.g. sex, gender, race).
2. Publish a summary of the bias audit results. Employers must post in a clear and conspicuous manner on their websites the following information about the bias audit: (i) the date of the most recent bias audit; (ii) a summary of the results, including the source and explanation of the data used; (iii) the selection (or scoring) rates and impact ratios for all categories; and (iv) the distribution date of the AEDT.
3. Provide notice to applicants and employees of the tool's use and functioning. Employers must provide candidates for employment or promotion with notice 10 business days before use of the tool: (i) that an AEDT is being used in assessing and evaluating the candidate; (ii) the job qualifications and characteristics the AEDT will use in its analysis; (iii) if not disclosed elsewhere on its website, the AEDT's data source, type, and the employer's data retention policy and (iv) a notice that candidates may request an alternative selection process or reasonable accommodation.
4. Provide affected individuals a way to request an accommodation or alternative selection process. Employers must offer either an alternative selection process (ASP) or reasonable accommodation to those employees or applicants who request it. The "reasonable accommodation" appears to describe an ADA requirement to protect applicants with disabilities. However, for non-ADA based requests for an ASP, the DWCP regulations explicitly state that "[n]othing [in these regulations] requires an employer or employment agency to provide" an alternative selection process. In sum, it would appear that employers are not required to do more than provide a mechanism for the ASP opt-out request, and there is apparently no requirement that these non-ADA related ASP requests be granted. Notably, the law does not impose any standard or decisional method for decisions about whether to grant an ASP in either circumstance. Employers should also consult with counsel about the potential for discrimination claims or disputes based on refusing to provide ASP for those who request it.

Although the DWCP final regulations clarify many of the requirements of NYC Law 144, the law's definitional criteria (both legal and technological) are complex, and employers using AI screening tools that may fall under the law would be well advised to consult counsel to discuss whether they are covered and, if so, what they may be required to do to ensure compliance.

The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.