Blab AI
Federal AI Rules Leave Vulnerable Beneficiaries at Risk, Study Finds
In April 2025, the Office of Management and Budget released two memoranda—M‑25‑21 and M‑25‑22—that set new rules for how federal agencies use and purchase artificial intelligence. The guidance requires agencies to appoint Chief AI Officers, establish governance boards, manage risks in high‑impact systems, disclose vendor information, and guard against vendor lock‑in.
According to the 2025 Federal Agency AI Use Case Inventory, 3,611 AI deployments span 56 agencies, with 445 classified as high‑impact. The Department of Veterans Affairs reports more than 200 uses, while the Social Security Administration relies on AI in many service and benefits processes.
However, the new rules do not automatically ensure fair treatment for the people receiving benefits. In programs that provide food, medicine, and housing, an automated decision can have life‑changing consequences. Low‑income families, people with disabilities, older adults, and those with limited English proficiency are the most vulnerable. A delayed notice or a miscalculated eligibility decision can mean missed groceries, skipped medication, or unpaid rent.
Past failures underscore the stakes. Between 2013 and 2015, Michigan’s MiDAS unemployment system flagged roughly 40,000 workers for fraud. A state audit later found a 93 percent error rate. The system processed accusations quickly, but workers had to fight slowly to clear their names. In 2016, Arkansas’s ARChoices Medicaid algorithm set in‑home care hours, and some applicants saw their care reduced by about 43 percent. Lawsuits highlighted problems with fairness, notice, and design.
A Brookings analysis of high‑impact federal AI systems in 2025 revealed that more than 85 percent lacked key risk safeguards. The analysis suggests that compliance with OMB guidance does not automatically translate into protection for beneficiaries.
The research argues that agencies must conduct civil‑rights audits before and after deploying high‑impact AI in benefits programs. The audits should address five questions:
1. Data quality – Does the system rely on data that misrepresents the people it affects? 2. Unequal outcomes – Are denial rates, fraud flags, delays, or appeals different across race, disability, age, language, or income groups? 3. Extra burden – Does the system shift more paperwork, verification steps, or digital requirements onto applicants? 4. Clarity – Do people understand why a decision was made and what they can do next? 5. Human judgment – Can staff override the system, and are they trained and supported to do so?
If an agency cannot answer these questions, it should not claim that compliance equals protection. The research notes that civil‑rights law—Title VI of the Civil Rights Act, the Americans with Disabilities Act, Section 504 of the Rehabilitation Act, and constitutional due‑process guarantees—already applies to automated benefit decisions. Courts have found that systems lacking clear notice and fair process can violate due‑process rights.
The author recommends that procurement contracts require vendors to disclose how systems work, what data they use, and how they are tested for fairness. The larger challenge is institutional: AI in benefits administration is both a public‑service decision and a civil‑rights decision.
The study calls for agencies to adopt civil‑rights audits as a standard practice, not an after‑thought. By doing so, agencies can ensure that the new federal AI rules protect the people they serve.
The research was conducted by Albert Nii Noi Okwei, a doctoral researcher in public policy and administration at the L. Douglas Wilder School of Government and Public Affairs, Virginia Commonwealth University. Okwei is affiliated with the Research Institute for Social Equity and is a 2026 Sara Miller McCune Scholar at the ICPSR Summer Program at the University of Michigan.
Key Takeaways - Federal AI guidance requires governance and risk management but does not guarantee fairness. - Past failures in Michigan and Arkansas show that algorithmic decisions can produce large, unequal harms. - A Brookings study found that most high‑impact systems lack essential safeguards. - Civil‑rights audits before and after deployment can identify data quality, outcome disparities, burden, clarity, and human‑override issues. - Compliance with OMB rules is necessary but not sufficient; protection requires additional checks. - Procurement contracts should mandate vendor transparency and fairness testing. - The new rules create an institutional responsibility for agencies to protect beneficiaries.
The research underscores that the federal AI framework must be paired with civil‑rights safeguards to ensure that benefits decisions serve the people they are designed to help.
According to the 2025 Federal Agency AI Use Case Inventory, 3,611 AI deployments span 56 agencies, with 445 classified as high‑impact. The Department of Veterans Affairs reports more than 200 uses, while the Social Security Administration relies on AI in many service and benefits processes.
However, the new rules do not automatically ensure fair treatment for the people receiving benefits. In programs that provide food, medicine, and housing, an automated decision can have life‑changing consequences. Low‑income families, people with disabilities, older adults, and those with limited English proficiency are the most vulnerable. A delayed notice or a miscalculated eligibility decision can mean missed groceries, skipped medication, or unpaid rent.
Past failures underscore the stakes. Between 2013 and 2015, Michigan’s MiDAS unemployment system flagged roughly 40,000 workers for fraud. A state audit later found a 93 percent error rate. The system processed accusations quickly, but workers had to fight slowly to clear their names. In 2016, Arkansas’s ARChoices Medicaid algorithm set in‑home care hours, and some applicants saw their care reduced by about 43 percent. Lawsuits highlighted problems with fairness, notice, and design.
A Brookings analysis of high‑impact federal AI systems in 2025 revealed that more than 85 percent lacked key risk safeguards. The analysis suggests that compliance with OMB guidance does not automatically translate into protection for beneficiaries.
The research argues that agencies must conduct civil‑rights audits before and after deploying high‑impact AI in benefits programs. The audits should address five questions:
1. Data quality – Does the system rely on data that misrepresents the people it affects? 2. Unequal outcomes – Are denial rates, fraud flags, delays, or appeals different across race, disability, age, language, or income groups? 3. Extra burden – Does the system shift more paperwork, verification steps, or digital requirements onto applicants? 4. Clarity – Do people understand why a decision was made and what they can do next? 5. Human judgment – Can staff override the system, and are they trained and supported to do so?
If an agency cannot answer these questions, it should not claim that compliance equals protection. The research notes that civil‑rights law—Title VI of the Civil Rights Act, the Americans with Disabilities Act, Section 504 of the Rehabilitation Act, and constitutional due‑process guarantees—already applies to automated benefit decisions. Courts have found that systems lacking clear notice and fair process can violate due‑process rights.
The author recommends that procurement contracts require vendors to disclose how systems work, what data they use, and how they are tested for fairness. The larger challenge is institutional: AI in benefits administration is both a public‑service decision and a civil‑rights decision.
The study calls for agencies to adopt civil‑rights audits as a standard practice, not an after‑thought. By doing so, agencies can ensure that the new federal AI rules protect the people they serve.
The research was conducted by Albert Nii Noi Okwei, a doctoral researcher in public policy and administration at the L. Douglas Wilder School of Government and Public Affairs, Virginia Commonwealth University. Okwei is affiliated with the Research Institute for Social Equity and is a 2026 Sara Miller McCune Scholar at the ICPSR Summer Program at the University of Michigan.
Key Takeaways - Federal AI guidance requires governance and risk management but does not guarantee fairness. - Past failures in Michigan and Arkansas show that algorithmic decisions can produce large, unequal harms. - A Brookings study found that most high‑impact systems lack essential safeguards. - Civil‑rights audits before and after deployment can identify data quality, outcome disparities, burden, clarity, and human‑override issues. - Compliance with OMB rules is necessary but not sufficient; protection requires additional checks. - Procurement contracts should mandate vendor transparency and fairness testing. - The new rules create an institutional responsibility for agencies to protect beneficiaries.
The research underscores that the federal AI framework must be paired with civil‑rights safeguards to ensure that benefits decisions serve the people they are designed to help.
