New York’s Use Of AI For Family Policing Summarizes The Last 30 Years Of City Politics

Davene Roseborough still can’t believe the number of foster homes she lived in that had locks on the refrigerators and cabinets. At one spot, Roseborough was barred from eating dinner if she wasn’t home by 6:00 p.m. That place didn’t last long, though. She was bounced from home to home from the time she entered foster care at the age of eight to the time she aged out at 18. Those years were characterized by rampant physical and sexual abuse, for which her foster parents received a $1,200 a monthly stipend. By the time she had her own child, she exemplified the kind of mother scrutinized by the very family policing system that raised her—Black, former system involvement, survivor of abuse, mental health struggles. Sure enough, her son was eventually taken from her and placed in the foster system for five years, where he also experienced abuse.

The Administration for Children’s Services, the New York City agency tasked with investigating allegations of child abuse and neglect, has been algorithmically encoding this kind of profiling since 2018. Before DOGE lived and died, before ICE contracted Palantir to supercharge mass deportation, before ChatGPT even launched, ACS was already using AI. What started with a single predictive analytics score meant to identify children at a high risk of severe harm has since grown into a six algorithm suite of surveillance, which operates with very little public oversight.

Yearly AI disclosures required by city law don’t include detailed information about what data the algorithms are trained on, or how exactly they are used by caseworkers. Despite the lack of transparency, what is evident is that the algorithms function as an expensive way to perpetuate the status quo—encoding extant and extreme human biases in the family policing system into algorithmic ones, and enabling a policing agency to claim it has the technical capacity to oversee the welfare of New York’s nearly two million children.