Published 28 October 2026

Who Checks the AI Now? A Regulatory Digest for Healthcare in the EU, Germany, Austria and Switzerland

Two healthcare professionals review a clinical AI performance trend. One examines declining results with a magnifying glass, while the other holds a regulatory digest beside an unassigned staff badge.

AI doesn’t announce when it gets worse.

Take a deterioration score that went live on a medical ward after a careful local validation. Eighteen months later the software is unchanged. Two blood tests are now ordered at a different rate, some inputs arrive late, and the night team overrides the alert more often than it used to. The ward is illustrative; the changes in test ordering and inputs come from published studies. No error message appears, and no single case looks like a serious incident.

Illustrative ward · deterioration score · software unchanged

Go-live · validated locallyMonth 18
  1. Two blood tests are ordered at a different rate
  2. Some inputs arrive late
  3. The night team overrides the alert more often

No update. No error message. No case that looks like a serious incident. Its performance slipped anyway.

Each person in that picture is doing their job, and the manufacturer keeps monitoring the tool. Today’s rules in Germany, Austria and Switzerland say who reports a serious incident, but not who in the hospital or practice watches for slow drift. The same gap opens in a practice when the scribe’s notes need more correcting each month, or the phone assistant books an urgent call as routine. The EU AI Act adds oversight and monitoring duties for high-risk AI from December 2027 or August 2028, and Switzerland has no such rule yet.

Where the checks happenEU, Germany, Austria, Switzerland
AI tool

from the manufacturer

Market entry

CE marking, where required

Open question:

On site, once in use

Point of care

doctors, nurses, pharmacists, practice teams

The manufacturer keeps monitoring after market entry. Once it is in use, no role on site is named yet to watch its performance.

Key takeaways

Serious incidents have a reporting route; today, a slow loss of accuracy has no named owner on site in Germany, Austria or Switzerland. Naming one per tool is a leadership decision you can take now.
AI literacy is already an EU duty for any AI tool: if your hospital, practice or field team in Germany or Austria uses a scribe, chatbot or phone assistant, it includes you.
Switzerland: neither the AI Act nor the new EU liability rules are Swiss law, though both can reach Swiss companies selling AI into the EU.
AI software placed on the market or put into service after 9 December 2026 is a product under new EU liability rules; the liability of the people using it stays national.


Model drift: how an unchanged AI tool gets worse

Model drift is a gap between how an AI tool performed when it was validated and how it performs now. It starts with a change in the patients and data the tool receives. The model itself can stay locked while a new scanner, camera or analyser arrives, a protocol changes, something elsewhere in the chain is updated or the patient mix shifts. The decline can be gradual, or a sudden step after an update.

A study published in PLOS Digital Health in July 2026 followed four clinical AI systems that nobody changed for 18 to 24 months.[1] At first the tools still ranked patients about as well as before, but their calibration slipped: a predicted 20% risk no longer reliably meant 20%. Missing or delayed inputs tracked that decline more closely than changes in the patients. In a Toronto study, shifts in admissions and in how often two blood tests were ordered weakened a mortality model for some patient groups, and a monitoring method that needed no outcome data flagged those shifts.[2]

What slipped first · illustrative

Ranking held. Calibration slipped.

Ranking

Who is at higher risk than whom?

At validation

Later

✓ Same order, at first

Calibration

Does a predicted 20% still mean 20%?

Observed now about 27% here (it can drift either way)

Illustrative, not data from the study.[1]

About the two studies

The PLOS Digital Health study looked back at four systems in one large healthcare organisation, with ethics approval in Saint Petersburg: two acute-care prediction models, an outpatient decision-support tool and a radiology imaging model. The models, their thresholds and their intended use did not change; the imaging model later also showed a larger drop in its headline score (AUROC).[1] The Toronto study was retrospective and covered 143,049 internal medicine inpatients at seven hospital sites between 2010 and 2020; the blood tests were BNP and D-dimer.[2]

Neither study measures how often this happens, and neither comes from Germany, Austria or Switzerland. Together they show that a model nobody has touched can drift, and that the drift can be detected. One clinician sees one case at a time, while drift shows only across many, and the manufacturer’s post-market surveillance depends partly on what sites report. Spotting it takes someone whose job is to look at the pattern.

One tool · one year · illustrative

One case, one night

Alert overridden: “not clinically indicated”.

✓ Looks reasonable

Share of alerts overridden, month by month

Month 1Month 12

overridden   followed each column = one month’s alerts

Each case looks fine. The trend shows only across many.

Rising overrides can also mean a noisy alert nobody trusts. Either way, someone should look.


What the EU AI Act already requires, and what comes later

The EU has already given part of this job to the organisation using the AI. Since February 2025, any organisation using an AI tool, from a practice’s phone assistant to a diagnostic tool, has had to address its staff’s AI literacy (KI-Kompetenz) under Article 4 of the AI Act (KI-Verordnung). After a July 2026 rewording, organisations must take measures to support it but need not guarantee any individual’s level. National supervision began in August 2026, though Austria has not yet designated its authority.[3]

For high-risk AI (Hochrisiko-KI), the duties start later, on dates that Regulation (EU) 2026/1744, the “Digital Omnibus on AI”, pushed back in July 2026 (timeline below).[4] From those dates, Article 26 requires the deployer, the hospital or practice using such a system, to assign human oversight to people “who have the necessary competence, training and authority, as well as the necessary support”. The deployer must also monitor the system and, if it sees a risk, inform the provider or distributor and the authority and suspend its use. Manufacturers keep their post-market duties under the MDR and IVDR, which the Omnibus did not change.

EU only · AI Act and liabilityStatus 26 Oct 2026

Two dates moved. The oversight duty stayed.

2 Feb 2025 Applies now

AI literacy

For any AI tool, scribes and chatbots included. Organisations must take measures to support it; no individual level is guaranteed (since Jul 2026).[3]

9 Dec 2026 Liability

Software is a product

AI included, if placed on the market or put into service after that date. Germany: law passed. Austria: not yet transposed.[10][12]

Aug 2026 (moved to)2 Dec 2027 High-risk

Uses such as emergency triage

High-risk uses listed in Annex III. The hospital names people for oversight and monitors the tool.[4]

Aug 2027 (moved to)2 Aug 2028 Devices

AI medical devices and IVDs

Class IIa–III devices and class B–D lab tests (IVDs) that need a notified body. Ask your vendor which class yours is. The same duties for hospitals and practices.[4]

Pending: a Commission proposal (Dec 2025) would apply the AI Act’s high-risk rules to AI medical devices only through the MDR/IVDR, so the August 2028 duties could change.[5]

EU only · for your legal team: Tools already in use may stay outside these duties

Under Article 111(2), systems on the market before a cut-off date are caught only after a significant design change, so a vendor update can bring a tool into scope. Whether that cut-off is the new dates or still 2 August 2026 is not settled, nor is which date applies to triage software that is also a certified device. Systems intended for public authorities must comply by 2 August 2030; whether a publicly owned hospital counts is for your lawyers.[3] A December 2025 Commission proposal would apply the AI Act’s high-risk rules to medical devices only via the MDR/IVDR, so the August 2028 duties could change; the European Parliament aims to adopt its position in early 2027.[5] Under Article 88 MDR, the manufacturer’s trend report covers any statistically significant increase in the frequency or severity of non-serious incidents (or expected side-effects) that could significantly affect the benefit-risk analysis and have led or may lead to unacceptable risks.[6] The reworded Article 4 (KI-Kompetenz) reads: providers and deployers “shall take measures to support the development of AI literacy of their staff”.[3]


Germany, Austria and Switzerland: a route for incidents, no role for drift

Germany · Austria · SwitzerlandStatus 26 Oct 2026

Three countries, three starting points.

Germany, Austria and Switzerland: AI authority, device incidents, practices and checking accuracy on site (status 26 October 2026)
CountryAI authorityDevice incidentsIn a practiceChecking accuracy on site
Germany Bundesnetzagentur since 29 Jul 2026 (KI-MIG); devices: existing authoritiesIn facilities that regularly employ more than 20 people, a device-safety officer (Beauftragter für Medizinproduktesicherheit) coordinates. Every professional user reports to BfArM (PEI for certain IVDs)Device-safety officer only if more than 20 people are regularly employed; the KBV counts the practice as the deployer (Betreiber)No role named yet
Austria Not yet designated as of Oct 2026 (deadline was Aug 2025); the AI Act applies anyway; RTR KI-Servicestelle informs onlyTechnical safety officer in hospitals; every professional user reports to the BASG, in hospitals via the medical directorWhoever uses the device reports directly to the BASGNo role named yet
Switzerland No AI law yet; consultation draft (Vernehmlassung) due end 2026; devices: SwissmedicVigilance contact in hospitals; every professional user reports to supplier and SwissmedicWhoever uses the device reports directly to supplier and SwissmedicNo role named yet

Sources: KI-MIG § 2[7] · Commission list, 7 Sep 2026; RTR[8] · Federal Council, 12 Feb 2025[9] · incident routes[6]

In all three countries, a serious device incident has a set route: a named contact in hospitals, and a duty on every professional user to report.[6] For a medical device, worse performance also counts as an incident under the MDR and the aligned Swiss MedDO. If the non-serious ones rise significantly enough to affect the benefit-risk balance, the manufacturer must report that trend. Users must report only the serious ones, and none of the named roles includes looking for a slow decline on site.

Device incidents vs slow drift · Germany, Austria, Switzerland

A serious incident

Professional user

in a hospital or practice

Authority

BfArM · BASG · Swissmedic (CH: and the supplier)

✓ A set route

set by law in all three countries

In hospitals: Germany · the device-safety officer coordinates. Austria · reports go via the medical director. Switzerland · the vigilance contact reports.

A slow decline in performance

Many small signs

overrides, corrections, late inputs

No role named on site

none of the named roles includes looking across them

From the small signs, partly via user reports:

Manufacturer’s trend report

a legal duty (Art. 88 MDR); depends partly on what sites pass on

The first lane has a route set by law. The second has no one named on site yet.


Liability for AI moves before oversight does

Under the EU’s new Product Liability Directive (Produkthaftungsrichtlinie), software, including AI systems, is a product when placed on the market or put into service after 9 December 2026. Anyone who substantially modifies it outside the manufacturer’s control and then puts it into service can be treated as its manufacturer, and that can include a hospital or practice.[10] So ask your vendor which local settings stay within its intended purpose. The proposed AI Liability Directive was withdrawn in 2025, so the fault-based liability of doctors, nurses, pharmacists and hospitals stays with national law.[11]

Liability · two tracks · EUStatus 26 Oct 2026

The product’s liability changes. The people’s stays national.

The product · EU directive

Placed on the market or put into service before 9 Dec 2026: the old law still applies

Placed on the market or put into service after 9 Dec 2026: software, including AI, is a product

Substantially modify it outside the manufacturer’s control and put it into service, and a hospital or practice can be treated as its manufacturer.[10]

The people · national law

Unchanged

Fault-based liability of doctors, nurses, pharmacists and hospitals stays with national law.[11]

Germany: New law applies from 9 Dec 2026.

Austria: Not yet transposed; whether stand-alone software counts under § 4 PHG is disputed.

Switzerland: The directive does not apply, and there is no AI-specific liability law.[12]

On a night shift

Medical associations in all three countries ask physicians to check AI output critically. Yet residents, nurses, pharmacists and practice assistants often act on the same output first, at triage for example. Whether any of them can make that check on a night shift is an organisational question.

Liability by country, and what medical associations say

Germany’s new product liability law (Produkthaftungsgesetz) completed its passage through parliament in October 2026, with the Bundestag vote on 8 October and the Bundesrat on 16 October; it applies from 9 December 2026, and products placed on the market before then stay under the old law. Austria has not yet transposed the directive, due by 9 December 2026, and its current law (§ 4 PHG) covers movable physical things and energy, so it is disputed whether stand-alone software counts. Switzerland, where the directive does not apply, has no AI-specific liability law.[12] The withdrawn AI Liability Directive would have lightened the burden of proof for people harmed by AI.[11] The German Medical Association keeps responsibility with the physician, who checks AI output for plausibility, and the 2026 Deutscher Ärztetag called for physicians not to be held liable for errors made by AI used as intended.[13] Austria’s Medical Chamber says final responsibility must stay with the physician, and the Swiss Medical Association (FMH) says that neither marketing claims nor certifications can replace a critical check of every language-model output.[14]


Where your team can start on AI oversight: three questions and a first pass

None of this needs to wait for a deadline. For tools already in use there may be none, and in Switzerland there is none yet, so whether anyone checks them is a leadership decision. The less regulated the tool, the more question 3 matters: many scribes, evidence chatbots and phone assistants are not medical devices at all, so nobody is named to notice if their output gets worse.

On site · once in useFor your agenda

Device incidents have an owner on site. Performance doesn’t, yet.

Who checks the AI now?

1. Can the people using it check its output before acting on it?

EU: Supporting their AI literacy is already a duty (Art. 4 AI Act, since Feb 2025).

Switzerland: No legal duty yet; for language models, the FMH asks for critical checking and documented use.

  • Brief everyone who uses each tool in working time, night staff, new residents and practice assistants included: what it does, where it goes wrong, what to check first (were its inputs complete?) and whom to tell.
  • For pharma and MedTech field teams: can the colleague who meets the physician check what the AI assistant prepared before the visit?

2. Which of your tools count as high-risk?

EU: Examples: emergency triage, class IIa–III AI devices.

Switzerland: Is it a medical device, and in which class? Ask the vendor.

  • List the AI tools in use, including scribes, chatbots, phone assistants and AI inside the prescribing or record system.
  • Ask each vendor: is it a medical device, could it be high-risk, which logs can we export, which local settings stay within its intended purpose, and does the phone assistant tell callers it is AI (Article 50)? Put log export and advance notice of updates in the next contract.

3. Who checks it stays accurate, month after month?

  • Name one person per tool, not the most junior user, and give them authority to pause it, an agreed fallback, rostered time and the vendor’s safety notices. In a hospital, that may be a hospital pharmacist for a medication-safety tool or a nurse expert for an early-warning score, reporting to the board once a year; in a small practice, it may be you.
  • Agree the signals with your team, and staff representatives where you have them: a triage category the nurse changes, a CAD mark the radiologist dismisses, an interaction alert overridden on almost every order, a scribe note corrected every day, an urgent call the phone assistant booked as routine and, for clinical scores, missing or late inputs. Count them per tool from existing logs, never per person or as an extra form. A rising override rate can mean drift or a noisy alert nobody trusts; either way, someone should look.
  • Look again after every update, new scanner, camera or protocol change. Send concerns, with the counts, to the manufacturer or distributor, via your device-safety officer, technical safety officer or vigilance contact where there is one, and tell the team what came back. Serious incidents still go to BfArM, BASG or Swissmedic.

If you use the tool

If you notice something, telling the person named for the tool is enough; if nobody is named, that is your answer to question 3.

Owner: A named person, with authority and support.

Illustrative logs · already on record

Ward score · one night

02:14Alert · score 7 · overriddenoverride
02:31Blood result arrived after the score was calculatedlate input
03:48Score calculated with respiratory rate missingmissing input
04:20Alert · score 6 · overriddenoverride

Phone assistant · one morning

08:05Caller asks for an urgent appointment · booked as routinere-booked by staff
08:22Repeat prescription request · booked
08:47Booked as routine · practice assistant moved it to same dayre-booked by staff
Person named to review these logs Not yet named

Illustrative entries. Signals like these are counted per tool, never per person.

On the ward we started with, the late inputs and the rising overrides were already on record. Looking at them together was nobody’s job, and no route for a concern had been agreed. The signs of drift may already be in your logs; what is missing is someone named to read them.

In the commentsYour turn

Which question would you put first?

1

Checking the output

2

Finding the high-risk tools

3

Naming an owner

Any of the three is a sensible start: a practice, residency programme or field team might begin with 1, a hospital with 2, a ward with 3. A number is enough; no need to name your hospital or tool.

Version 1 · published [date] · status as of 26 October 2026 · next scheduled check: 10 December 2026 [keep only if someone owns it]. This digest is not legal advice. If the position in your country has changed, tell us; we will update it and record the change below.

Changes to this digest

[Publish date] · First edition; status as of 26 October 2026.


References

[1] Kopanitsa G. Validation is not enough: longitudinal evidence of post-deployment fragility in clinical AI systems. PLOS Digit Health. 2026;5(7):e0001534. https://doi.org/10.1371/journal.pdig.0001534

[2] Subasri V, Krishnan A, Kore A, et al. Detecting and remediating harmful data shifts for the responsible deployment of clinical AI models. JAMA Netw Open. 2025;8(6):e2513685. https://doi.org/10.1001/jamanetworkopen.2025.13685

[3] Regulation (EU) 2024/1689 (AI Act), consolidated text of 27 July 2026, Articles 2, 4, 26, 50, 111 and 113, https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX%3A02024R1689-20260727; European Commission, AI literacy: questions and answers (supervision from 2 August 2026), https://digital-strategy.ec.europa.eu/en/faqs/ai-literacy-questions-answers

[4] Regulation (EU) 2026/1744 of 8 July 2026 (“Digital Omnibus on AI”). OJ L, 24 July 2026. https://eur-lex.europa.eu/eli/reg/2026/1744/oj/eng

[5] European Commission. Proposal to simplify the MDR and IVDR, COM(2025) 1023, 16 December 2025, https://health.ec.europa.eu/document/download/25e7ea7c-cab3-40cf-86d9-d11f5e7744d8_en?filename=md_com_2025-1023_act_en.pdf; Council of the EU, progress report 9801/26, 5 June 2026, https://data.consilium.europa.eu/doc/document/ST-9801-2026-INIT/en/pdf

[6] Incident routes. Germany: § 6 MPBetreibV, https://www.gesetze-im-internet.de/mpbetreibv_2025/__6.html; § 3 MPAMIV, https://www.gesetze-im-internet.de/mpamiv/__3.html; KBV, PraxisWissen KI (December 2025), https://www.kbv.de/documents/infothek/publikationen/praxiswissen/praxiswissen-ki.pdf. Austria: §§ 40–41 MPG 2021, https://www.basg.gv.at/gesundheitsberufe/medizinproduktebetreiber. Switzerland: MedDO Arts 4(2), 66(4) and 67(2), https://www.fedlex.admin.ch/eli/cc/2020/552/en; Swissmedic, vigilance, https://swissmedic.ch/swissmedic/en/home/medical-devices/reprocessing—maintenance/vigilance-mep.html. EU: Regulation (EU) 2017/745 (MDR), Arts 2(64) and 88, https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32017R0745

[7] Gesetz zur Marktüberwachung und Innovationsförderung von künstlicher Intelligenz (KI-MIG), BGBl. 2026 I Nr. 223, § 2. https://www.gesetze-im-internet.de/ki-mig/BJNR0DF0B0026.html

[8] European Commission. Market surveillance authorities under the AI Act (list updated 7 September 2026), https://digital-strategy.ec.europa.eu/en/policies/market-surveillance-authorities-under-ai-act; RTR KI-Servicestelle, FAQ, https://www.rtr.at/rtr/service/ki-servicestelle/faq/FAQ.de.html

[9] Swiss Federal Council. Media release, 12 February 2025, https://www.admin.ch/gov/en/start/documentation/media-releases.msg-id-104110.html; Federal Office of Justice, https://www.bj.admin.ch/de/kuenstliche-intelligenz

[10] Directive (EU) 2024/2853 on liability for defective products, Art. 8. https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32024L2853

[11] European Parliament. Legislative train: AI Liability Directive (withdrawn). https://www.europarl.europa.eu/legislative-train/theme-a-europe-fit-for-the-digital-age/file-ai-liability-directive

[12] Gesetz zur Modernisierung des Produkthaftungsrechts: Bundesrat Drucksache 584/26 (Bundestag resolution of 8 October 2026), https://dserver.bundestag.de/brd/2026/0584-26.pdf, and Bundesrat decision 584/26(B), 16 October 2026, https://dserver.bundestag.de/brd/2026/0584-26B.pdf; Faegre Drinker, Transposing the EU’s new Product Liability Directive: a Member State progress report (2026), https://www.faegredrinker.com/en/insights/publications/2026/6/transposing-the-eus-new-product-liability-directive-a-member-state-progress-report; Austria: § 4 Produkthaftungsgesetz, https://www.ris.bka.gv.at/NormDokument.wxe?Abfrage=Bundesnormen&Gesetzesnummer=10002864&Paragraf=4, and Nationalrat motion 1071/A(E), 14 October 2026, https://www.parlament.gv.at/dokument/XXVIII/A/1071/imfname_1783436.pdf; FMH, Rechtliche Überlegungen zu generativer KI, https://fmh.ch/themen/ehealth/trends-neue-technologien/generative-ki/rechtliche-ueberlegungen.cfm

[13] Bundesärztekammer. Stellungnahme “Künstliche Intelligenz in der Medizin” (February 2025), https://www.bundesaerztekammer.de/fileadmin/user_upload/wissenschaftlicher-beirat/Veroeffentlichungen/KI_in_der_Medizin_SN_neu.pdf; 130. Deutscher Ärztetag, press release, 15 May 2026, https://www.bundesaerztekammer.de/presse/aktuelles/detail/auswirkungen-von-ki-anwendungen-in-der-medizin-in-klinischen-studien-ueberpruefen

[14] Österreichische Ärztekammer, press information, 10 October 2024 (final responsibility with the physician), https://www.aerztekammer.at/presseinformation/-/asset_publisher/presseinformation/content/id/2825945; Österreichische Ärztezeitung, 26 May 2026 (checking AI-generated content critically), https://aerztezeitung.at/2026/oaz-artikel/politik/e-health-und-digitale-medizin-mehrwert-im-fokus-ki-war-gestern/; FMH, Empfehlungen zum Umgang mit grossen Sprachmodellen (LLMs) im ärztlichen Alltag (web page, accessed 26 October 2026), https://fmh.ch/themen/ehealth/trends-neue-technologien/generative-ki/empfehlungen-der-fmh.cfm

Arthur Boritzka, Chief Product Officer @ Elumity