AI governance & safety
AI fails quietly. Governance is how you notice.
I'm Daniel Foley, a healthcare systems and data analyst within Providence, moving into AI governance and safety. My focus is the analyst's side of the work: reviewing AI systems before go-live, setting the guardrails they run inside, and monitoring them long after launch.
01Focus
Four moments where AI gets governed.
Most AI risk doesn't sit inside the model. It sits in the handoffs: vendor to customer, model output to human decision, launch day to month eighteen. These are the points I study, and the question an analyst has to answer at each one.
Before go-live
Review
Is this system fit for the people it will actually affect?
Intended use, local validation, a risk tier, and a named owner. A lot of failures are visible here, if someone checks.
- NIST AI RMF: Map
- EU AI Act: risk tiers
- CHAI playbooks
- FDA CDS guidance
At go-live
Guardrails
What stops a bad output before it does harm?
Alert thresholds, human sign-off points, disclosure to the people affected, and a way to switch the system off. Decided in advance and written down.
- NIST AI RMF: Govern
- EU AI Act: human oversight
- CA AB 3030
After go-live
Monitoring
Is it still working, and for whom?
Performance decay, data drift, gaps between subgroups, and alert burden, each tracked against a threshold someone agreed to.
- NIST AI RMF: Measure
- EU AI Act: post-market monitoring
- Calibration
- Subgroup performance
When it fails
Investigation
Why did it fail, and what stops it happening again?
Root cause across data, model, workflow and human factors, written up so the fix outlives the incident.
- NIST AI RMF: Manage
- EU AI Act: incident reporting
- Incident review
02Review queue
Case reviews, written the way an analyst would file them.
Each review takes a documented AI failure, or a risk the law already names, and works it like a ticket: what happened, why, how bad, which controls would have caught it, and what to monitor next. Published sources only, and every claim cited. The first three come from healthcare, where I know the systems best. The method carries over to any sector.
- R-001Predictive
Epic Sepsis Model: the external validation gap
An independent validation at Michigan Medicine found a widely deployed sepsis model discriminated far worse than reported (AUROC 0.63 against 0.76–0.83). It missed 67% of sepsis cases while alerting on 18% of hospitalized patients.
What local validation and monitoring would have caught the gap before go-live?
The Epic Sepsis Model's area under the ROC curve (AUROC), as reported by Epic and as independently measured. Published data from the study cited as this review's source. 0.50 is a coin flip; 1.00 is perfect. Source AUROC Epic's documentation, Reported by the developer 0.76–0.83 Conference paper, Co-authored with Epic 0.73 Independent validation, Michigan Medicine · 38,455 hospitalizations 0.63 (95% confidence interval 0.62–0.64) Source: Wong et al., JAMA Internal Medicine, 2021 (opens in a new tab)
- R-002Generative
AI-drafted replies to patient messages
California's AB 3030 requires a disclaimer on AI-generated clinical messages to patients, unless a licensed clinician reads and reviews the message before it's sent.
What makes that clinician review real oversight rather than a rubber stamp?
- R-003Payer
Algorithm-assisted prior authorization
California's SB 1120 requires that medical-necessity decisions in utilization review be made by a licensed clinician, not by an algorithm on its own.
What does a compliant human-in-the-loop look like inside a utilization-review workflow?
Every review follows the same eight steps.
Method
- 01The ticketTwo or three lines, the way the incident would land in an analyst's queue.
- 02System cardWhat the model does, its inputs, where it sits in the workflow, and who acts on its output.
- 03What went wrongThe published evidence, cited.
- 04Root causeData, model, workflow, or human factors. Usually more than one.
- 05Risk ratingSeverity × likelihood, and who carried the risk.
- 06ControlsWhat would have caught it: local validation, a monitored metric with a threshold, human sign-off.
- 07Framework mapNIST AI RMF function, sector guidance, and applicable law.
- 08RecommendationGo, fix, or stop, plus the monitoring plan.
I'd welcome a critical read from people who do this work. Drafts are available on request. Get in touch.
03About
Field science, then healthcare systems. Next, AI governance.

I came to AI governance the long way round: fisheries biology in the Colorado Rockies, science media for NOAA and National Geographic, then a year running IT and video aboard the deep-sea research vessel E/V Nautilus. Field science taught me that a number is only as good as the way it was collected.
Since 2023 I've worked in healthcare within Providence. At Pacific Neuroscience Institute I analyze patient data and build Tableau dashboards on Epic EHR data for a busy neurosurgery practice. At Ayin Health Solutions I test configuration changes and software releases on a proprietary health-plan platform, automate requirements work with Python and SQL, and write the system documentation our software vendors review.
That work taught me where healthcare software tends to break: less often in the code than in the rules, the data, and the handoffs between people and systems. AI multiplies all three, in healthcare and everywhere else. That's where I'm taking my career next: reviewing AI systems before they reach people, putting guardrails around them, and watching them after launch.
I'm not a clinician or a lawyer, and I'm not trying to be either. Clinical and legal judgment belongs to the people trained for it. The work I want is making sure the AI systems people rely on deserve their trust.
Path
- 2024 – now
System Configuration Engineer · Ayin Health Solutions (Providence)
Release testing and QA on a proprietary health-plan platform; Python and SQL automation; system documentation reviewed by vendors.
- 2023 – now
Patient Data Analyst · Pacific Neuroscience Institute (Providence)
Tableau dashboards on Epic EHR data; scheduling, authorizations and post-op tracking across eight surgical subspecialties.
- 2025
Full-Stack Engineer (Intern) · Haven AI, via the Climatebase Fellowship
Python and FastAPI services for an AI climate-resilience platform.
- 2022 – 23
Lead IT / Video Engineer · Ocean Exploration Trust, E/V Nautilus
Kept 30+ video and telecom systems running for a 40-person science crew; live broadcasts to 500+ students a month.
- 2020 – 23
Science Educator (freelance) · NOAA, National Geographic, Discovery and others
Made science media for audiences who aren't scientists.
- 2019
Fisheries Biologist · Colorado Parks and Wildlife
Spawning surveys across 30+ lakes in the San Juan Mountains, analyzed in R.
Education & certificates
- B.S. Biology, University of St. Thomas
- Climatebase Fellowship, Cohort 8 (honors)
- Google Data Analytics Certificate
- Analyst Builder certificates: MySQL, Excel
Earlier technical work
Before turning to AI governance, I built geospatial data systems. Three still show how I work.
California Wildfire Risk API
A PostGIS and FastAPI service over 22,810 CAL FIRE fire perimeters, with a documented 0–100 risk score, a test suite and CI. The lesson I kept: if you can't explain a score, don't ship it.
NYC Urban Accessibility Index
Spatial ETL joining five public datasets into one composite index, at neighborhood and H3-hexagon resolution.
SDG 15 Global Deforestation
SQL cleaning and analysis of 2000–2020 forest-cover data, published as a Tableau dashboard.
04Now
What I'm working on
Updated
- Reading the NIST AI Risk Management Framework and its Generative AI Profile, alongside the Coalition for Health AI's governance playbooks.
- Outlining R-001, a governance review of the Epic Sepsis Model's external validation.
- Having conversations with people who review, monitor and investigate AI systems. If that's you, I'd value 20 minutes.
05Contact
If you work on AI governance, I'd like to hear how you do it.
I'm looking for 20-minute conversations with people who review, monitor or investigate AI systems, and for analyst roles on AI governance, risk and operations teams, in healthcare and beyond.