Daniel Foley

AI governance & safety

AI fails quietly. Governance is how you notice.

I'm Daniel Foley, a healthcare systems and data analyst within Providence, moving into AI governance and safety. My focus is the analyst's side of the work: reviewing AI systems before go-live, setting the guardrails they run inside, and monitoring them long after launch.

Four moments where AI gets governed.

Most AI risk doesn't sit inside the model. It sits in the handoffs: vendor to customer, model output to human decision, launch day to month eighteen. These are the points I study, and the question an analyst has to answer at each one.

  1. Before go-live

    Review

    Is this system fit for the people it will actually affect?

    Intended use, local validation, a risk tier, and a named owner. A lot of failures are visible here, if someone checks.

    • NIST AI RMF: Map
    • EU AI Act: risk tiers
    • CHAI playbooks
    • FDA CDS guidance
  2. At go-live

    Guardrails

    What stops a bad output before it does harm?

    Alert thresholds, human sign-off points, disclosure to the people affected, and a way to switch the system off. Decided in advance and written down.

    • NIST AI RMF: Govern
    • EU AI Act: human oversight
    • CA AB 3030
  3. After go-live

    Monitoring

    Is it still working, and for whom?

    Performance decay, data drift, gaps between subgroups, and alert burden, each tracked against a threshold someone agreed to.

    • NIST AI RMF: Measure
    • EU AI Act: post-market monitoring
    • Calibration
    • Subgroup performance
  4. When it fails

    Investigation

    Why did it fail, and what stops it happening again?

    Root cause across data, model, workflow and human factors, written up so the fix outlives the incident.

    • NIST AI RMF: Manage
    • EU AI Act: incident reporting
    • Incident review

Case reviews, written the way an analyst would file them.

Each review takes a documented AI failure, or a risk the law already names, and works it like a ticket: what happened, why, how bad, which controls would have caught it, and what to monitor next. Published sources only, and every claim cited. The first three come from healthcare, where I know the systems best. The method carries over to any sector.

  1. R-001Predictive

    Epic Sepsis Model: the external validation gap

    An independent validation at Michigan Medicine found a widely deployed sepsis model discriminated far worse than reported (AUROC 0.63 against 0.76–0.83). It missed 67% of sepsis cases while alerting on 18% of hospitalized patients.

    What local validation and monitoring would have caught the gap before go-live?

    The Epic Sepsis Model's area under the ROC curve (AUROC), as reported by Epic and as independently measured. Published data from the study cited as this review's source. 0.50 is a coin flip; 1.00 is perfect.
    SourceAUROC
    Epic's documentation, Reported by the developer0.76–0.83
    Conference paper, Co-authored with Epic0.73
    Independent validation, Michigan Medicine · 38,455 hospitalizations0.63 (95% confidence interval 0.62–0.64)

    Source: Wong et al., JAMA Internal Medicine, 2021 (opens in a new tab)

    OutliningTarget Jan 2027
  2. R-002Generative

    AI-drafted replies to patient messages

    California's AB 3030 requires a disclaimer on AI-generated clinical messages to patients, unless a licensed clinician reads and reviews the message before it's sent.

    What makes that clinician review real oversight rather than a rubber stamp?

    Source: California AB 3030 (2024) (opens in a new tab)

    PlannedTarget Feb 2027
  3. R-003Payer

    Algorithm-assisted prior authorization

    California's SB 1120 requires that medical-necessity decisions in utilization review be made by a licensed clinician, not by an algorithm on its own.

    What does a compliant human-in-the-loop look like inside a utilization-review workflow?

    Source: California SB 1120 (2024) (opens in a new tab)

    PlannedTarget Spring 2027

Every review follows the same eight steps.

Method

  1. 01The ticketTwo or three lines, the way the incident would land in an analyst's queue.
  2. 02System cardWhat the model does, its inputs, where it sits in the workflow, and who acts on its output.
  3. 03What went wrongThe published evidence, cited.
  4. 04Root causeData, model, workflow, or human factors. Usually more than one.
  5. 05Risk ratingSeverity × likelihood, and who carried the risk.
  6. 06ControlsWhat would have caught it: local validation, a monitored metric with a threshold, human sign-off.
  7. 07Framework mapNIST AI RMF function, sector guidance, and applicable law.
  8. 08RecommendationGo, fix, or stop, plus the monitoring plan.

I'd welcome a critical read from people who do this work. Drafts are available on request. Get in touch.

Field science, then healthcare systems. Next, AI governance.

Portrait of Daniel Foley

I came to AI governance the long way round: fisheries biology in the Colorado Rockies, science media for NOAA and National Geographic, then a year running IT and video aboard the deep-sea research vessel E/V Nautilus. Field science taught me that a number is only as good as the way it was collected.

Since 2023 I've worked in healthcare within Providence. At Pacific Neuroscience Institute I analyze patient data and build Tableau dashboards on Epic EHR data for a busy neurosurgery practice. At Ayin Health Solutions I test configuration changes and software releases on a proprietary health-plan platform, automate requirements work with Python and SQL, and write the system documentation our software vendors review.

That work taught me where healthcare software tends to break: less often in the code than in the rules, the data, and the handoffs between people and systems. AI multiplies all three, in healthcare and everywhere else. That's where I'm taking my career next: reviewing AI systems before they reach people, putting guardrails around them, and watching them after launch.

I'm not a clinician or a lawyer, and I'm not trying to be either. Clinical and legal judgment belongs to the people trained for it. The work I want is making sure the AI systems people rely on deserve their trust.

Path

  1. 2024 – now

    System Configuration Engineer · Ayin Health Solutions (Providence)

    Release testing and QA on a proprietary health-plan platform; Python and SQL automation; system documentation reviewed by vendors.

  2. 2023 – now

    Patient Data Analyst · Pacific Neuroscience Institute (Providence)

    Tableau dashboards on Epic EHR data; scheduling, authorizations and post-op tracking across eight surgical subspecialties.

  3. 2025

    Full-Stack Engineer (Intern) · Haven AI, via the Climatebase Fellowship

    Python and FastAPI services for an AI climate-resilience platform.

  4. 2022 – 23

    Lead IT / Video Engineer · Ocean Exploration Trust, E/V Nautilus

    Kept 30+ video and telecom systems running for a 40-person science crew; live broadcasts to 500+ students a month.

  5. 2020 – 23

    Science Educator (freelance) · NOAA, National Geographic, Discovery and others

    Made science media for audiences who aren't scientists.

  6. 2019

    Fisheries Biologist · Colorado Parks and Wildlife

    Spawning surveys across 30+ lakes in the San Juan Mountains, analyzed in R.

Education & certificates

  • B.S. Biology, University of St. Thomas
  • Climatebase Fellowship, Cohort 8 (honors)
  • Google Data Analytics Certificate
  • Analyst Builder certificates: MySQL, Excel

Earlier technical work

Before turning to AI governance, I built geospatial data systems. Three still show how I work.

  • California Wildfire Risk API

    A PostGIS and FastAPI service over 22,810 CAL FIRE fire perimeters, with a documented 0–100 risk score, a test suite and CI. The lesson I kept: if you can't explain a score, don't ship it.

  • NYC Urban Accessibility Index

    Spatial ETL joining five public datasets into one composite index, at neighborhood and H3-hexagon resolution.

  • SDG 15 Global Deforestation

    SQL cleaning and analysis of 2000–2020 forest-cover data, published as a Tableau dashboard.

What I'm working on

Updated

  • Reading the NIST AI Risk Management Framework and its Generative AI Profile, alongside the Coalition for Health AI's governance playbooks.
  • Outlining R-001, a governance review of the Epic Sepsis Model's external validation.
  • Having conversations with people who review, monitor and investigate AI systems. If that's you, I'd value 20 minutes.

If you work on AI governance, I'd like to hear how you do it.

I'm looking for 20-minute conversations with people who review, monitor or investigate AI systems, and for analyst roles on AI governance, risk and operations teams, in healthcare and beyond.