Expert Guidance in a Fragmented AI Regulatory Landscape
Employment tools that include some application of Artificial Intelligence (AI) are becoming a standard part of how organizations hire, promote, pay, and manage their workforce. At the same time, a growing patchwork of new state and local requirements is taking shape along with the professional standards and federal laws that have long governed all selection tools (i.e., AI-enabled or traditional).
Whether your organization just began to explore the use of AI in Human Resource (HR) practices, is mid-way through vendor selection, is already live, or is facing a specific challenge to a decision made, DCI's Industrial/Organizational (I/O) psychologists can become a partner to navigate the evolving landscape of AI regulations.
Why DCI?
For decades, DCI's team of I/O psychologists and labor economists has specialized in conducting expert reviews of a wide-variety of HR processes, including AI-enabled selection tools. We have extensive experience partnering with internal and external counsel who frequently leverage our work when they are providing their clients with legal advice.
Our approach to consulting on AI-enabled selection tools is grounded in a combination of long-standing professional standards and more contemporary research and literature associated with AI advancement. The former includes the SIOP Principles on the Validation and Use Of Personnel Selection Procedures (2018), which are focused on assessment quality, reliability, job-relatedness, and specific conceptualizations of bias and fairness; the latter includes broader frameworks and state and local laws, which add additional considerations such as transparency, explainability, and human-in-the-loop design.
Our Consulting Services for AI-Enabled Selection
No single service or expert review fits every tool, job, or organization. What the tool does, where it sits in the selection process, how it's used to make decisions at your organization and for your jobs, all shape what a meaningful expert review may include. For this reason, context will drive which DCI services are most valuable for your organization. Below are exemplar services our clients often find valuable:
We can help your organization assess readiness, explore use cases, consult on governance standards, evaluate and validate tools, and monitor use after launch.
We can evaluate the tool's design, development, job-relatedness, fairness, potential sources of bias, and explainability against relevant long-standing and contemporary standards.
We can review inputs to the AI-based assessment and check for factors such as job relevance, potential for measurement bias, and other qualitative aspects. If the tool is intended to leverage job information, we can help assess the quality of that information, including how closely it tracks the actual job
We can conduct adverse impact analyses to identify subgroup differences in outcomes for a dataset consisting of local employment decisions at the appropriate unit of analysis (e.g., by job title) as prescribed in professional standards.
In some cases, it may be possible to design a study to measure the relationship between scores from an AI-enabled selection tools and on-the-job criterion measures.
DCI can partner with counsel to assist clients in responding to requirements for AI-enabled selection tools that are subject to compliance with a growing list of state and local statutes. For example, New York City, California, Colorado, and Illinois have taken different approaches to compliance requirements and other jurisdictions are likely to follow. The European Union’s AI Act adds another layer that impacts international organizations. For laws with clearly defined requirements, like New York City's Local Law 144, we can provide the specific “bias audit” and summary of results the laws define.
