AI doesn’t announce when it gets worse.
Take a deterioration score that went live on a medical ward after a careful local validation. Eighteen months later the software is unchanged. Two blood tests are now ordered at a different rate, some inputs arrive late, and the night team overrides the alert more often than it used to. The ward is illustrative; the changes in test ordering and inputs come from published studies. No error message appears, and no single case looks like a serious incident.
Illustrative ward · deterioration score · software unchanged
- Two blood tests are ordered at a different rate
- Some inputs arrive late
- The night team overrides the alert more often
No update. No error message. No case that looks like a serious incident. Its performance slipped anyway.
Each person in that picture is doing their job, and the manufacturer keeps monitoring the tool. Today’s rules in Germany, Austria and Switzerland say who reports a serious incident, but not who in the hospital or practice watches for slow drift. The same gap opens in a practice when the scribe’s notes need more correcting each month, or the phone assistant books an urgent call as routine. The EU AI Act adds oversight and monitoring duties for high-risk AI from December 2027 or August 2028, and Switzerland has no such rule yet.
from the manufacturer
CE marking, where required
On site, once in use
doctors, nurses, pharmacists, practice teams
The manufacturer keeps monitoring after market entry. Once it is in use, no role on site is named yet to watch its performance.
Key takeaways
Model drift: how an unchanged AI tool gets worse
Model drift is a gap between how an AI tool performed when it was validated and how it performs now. It starts with a change in the patients and data the tool receives. The model itself can stay locked while a new scanner, camera or analyser arrives, a protocol changes, something elsewhere in the chain is updated or the patient mix shifts. The decline can be gradual, or a sudden step after an update.
A study published in PLOS Digital Health in July 2026 followed four clinical AI systems that nobody changed for 18 to 24 months.[1] At first the tools still ranked patients about as well as before, but their calibration slipped: a predicted 20% risk no longer reliably meant 20%. Missing or delayed inputs tracked that decline more closely than changes in the patients. In a Toronto study, shifts in admissions and in how often two blood tests were ordered weakened a mortality model for some patient groups, and a monitoring method that needed no outcome data flagged those shifts.[2]
What slipped first · illustrative
Ranking held. Calibration slipped.
Ranking
Who is at higher risk than whom?
At validation
Later
✓ Same order, at first
Calibration
Does a predicted 20% still mean 20%?
Observed now about 27% here (it can drift either way)
Illustrative, not data from the study.[1]
About the two studies
The PLOS Digital Health study looked back at four systems in one large healthcare organisation, with ethics approval in Saint Petersburg: two acute-care prediction models, an outpatient decision-support tool and a radiology imaging model. The models, their thresholds and their intended use did not change; the imaging model later also showed a larger drop in its headline score (AUROC).[1] The Toronto study was retrospective and covered 143,049 internal medicine inpatients at seven hospital sites between 2010 and 2020; the blood tests were BNP and D-dimer.[2]
Neither study measures how often this happens, and neither comes from Germany, Austria or Switzerland. Together they show that a model nobody has touched can drift, and that the drift can be detected. One clinician sees one case at a time, while drift shows only across many, and the manufacturer’s post-market surveillance depends partly on what sites report. Spotting it takes someone whose job is to look at the pattern.
One tool · one year · illustrative
One case, one night
Alert overridden: “not clinically indicated”.
✓ Looks reasonable
Share of alerts overridden, month by month
overridden followed each column = one month’s alerts
Each case looks fine. The trend shows only across many.
Rising overrides can also mean a noisy alert nobody trusts. Either way, someone should look.
What the EU AI Act already requires, and what comes later
The EU has already given part of this job to the organisation using the AI. Since February 2025, any organisation using an AI tool, from a practice’s phone assistant to a diagnostic tool, has had to address its staff’s AI literacy (KI-Kompetenz) under Article 4 of the AI Act (KI-Verordnung). After a July 2026 rewording, organisations must take measures to support it but need not guarantee any individual’s level. National supervision began in August 2026, though Austria has not yet designated its authority.[3]
For high-risk AI (Hochrisiko-KI), the duties start later, on dates that Regulation (EU) 2026/1744, the “Digital Omnibus on AI”, pushed back in July 2026 (timeline below).[4] From those dates, Article 26 requires the deployer, the hospital or practice using such a system, to assign human oversight to people “who have the necessary competence, training and authority, as well as the necessary support”. The deployer must also monitor the system and, if it sees a risk, inform the provider or distributor and the authority and suspend its use. Manufacturers keep their post-market duties under the MDR and IVDR, which the Omnibus did not change.
Two dates moved. The oversight duty stayed.
Pending: a Commission proposal (Dec 2025) would apply the AI Act’s high-risk rules to AI medical devices only through the MDR/IVDR, so the August 2028 duties could change.[5]
EU only · for your legal team: Tools already in use may stay outside these duties
Under Article 111(2), systems on the market before a cut-off date are caught only after a significant design change, so a vendor update can bring a tool into scope. Whether that cut-off is the new dates or still 2 August 2026 is not settled, nor is which date applies to triage software that is also a certified device. Systems intended for public authorities must comply by 2 August 2030; whether a publicly owned hospital counts is for your lawyers.[3] A December 2025 Commission proposal would apply the AI Act’s high-risk rules to medical devices only via the MDR/IVDR, so the August 2028 duties could change; the European Parliament aims to adopt its position in early 2027.[5] Under Article 88 MDR, the manufacturer’s trend report covers any statistically significant increase in the frequency or severity of non-serious incidents (or expected side-effects) that could significantly affect the benefit-risk analysis and have led or may lead to unacceptable risks.[6] The reworded Article 4 (KI-Kompetenz) reads: providers and deployers “shall take measures to support the development of AI literacy of their staff”.[3]
Germany, Austria and Switzerland: a route for incidents, no role for drift
Three countries, three starting points.
| Country | AI authority | Device incidents | In a practice | Checking accuracy on site |
|---|---|---|---|---|
| Germany | Bundesnetzagentur since 29 Jul 2026 (KI-MIG); devices: existing authorities | In facilities that regularly employ more than 20 people, a device-safety officer (Beauftragter für Medizinproduktesicherheit) coordinates. Every professional user reports to BfArM (PEI for certain IVDs) | Device-safety officer only if more than 20 people are regularly employed; the KBV counts the practice as the deployer (Betreiber) | No role named yet |
| Austria | Not yet designated as of Oct 2026 (deadline was Aug 2025); the AI Act applies anyway; RTR KI-Servicestelle informs only | Technical safety officer in hospitals; every professional user reports to the BASG, in hospitals via the medical director | Whoever uses the device reports directly to the BASG | No role named yet |
| Switzerland | No AI law yet; consultation draft (Vernehmlassung) due end 2026; devices: Swissmedic | Vigilance contact in hospitals; every professional user reports to supplier and Swissmedic | Whoever uses the device reports directly to supplier and Swissmedic | No role named yet |
Sources: KI-MIG § 2[7] · Commission list, 7 Sep 2026; RTR[8] · Federal Council, 12 Feb 2025[9] · incident routes[6]
In all three countries, a serious device incident has a set route: a named contact in hospitals, and a duty on every professional user to report.[6] For a medical device, worse performance also counts as an incident under the MDR and the aligned Swiss MedDO. If the non-serious ones rise significantly enough to affect the benefit-risk balance, the manufacturer must report that trend. Users must report only the serious ones, and none of the named roles includes looking for a slow decline on site.
Device incidents vs slow drift · Germany, Austria, Switzerland
A serious incident
Professional user
in a hospital or practice
Authority
BfArM · BASG · Swissmedic (CH: and the supplier)
✓ A set route
set by law in all three countries
In hospitals: Germany · the device-safety officer coordinates. Austria · reports go via the medical director. Switzerland · the vigilance contact reports.
A slow decline in performance
Many small signs
overrides, corrections, late inputs
No role named on site
none of the named roles includes looking across them
Manufacturer’s trend report
a legal duty (Art. 88 MDR); depends partly on what sites pass on
The first lane has a route set by law. The second has no one named on site yet.
Liability for AI moves before oversight does
Under the EU’s new Product Liability Directive (Produkthaftungsrichtlinie), software, including AI systems, is a product when placed on the market or put into service after 9 December 2026. Anyone who substantially modifies it outside the manufacturer’s control and then puts it into service can be treated as its manufacturer, and that can include a hospital or practice.[10] So ask your vendor which local settings stay within its intended purpose. The proposed AI Liability Directive was withdrawn in 2025, so the fault-based liability of doctors, nurses, pharmacists and hospitals stays with national law.[11]
The product’s liability changes. The people’s stays national.
The product · EU directive
Placed on the market or put into service before 9 Dec 2026: the old law still applies
Placed on the market or put into service after 9 Dec 2026: software, including AI, is a product
Substantially modify it outside the manufacturer’s control and put it into service, and a hospital or practice can be treated as its manufacturer.[10]
The people · national law
Unchanged
Fault-based liability of doctors, nurses, pharmacists and hospitals stays with national law.[11]
Germany: New law applies from 9 Dec 2026.
Austria: Not yet transposed; whether stand-alone software counts under § 4 PHG is disputed.
Switzerland: The directive does not apply, and there is no AI-specific liability law.[12]
On a night shift
Medical associations in all three countries ask physicians to check AI output critically. Yet residents, nurses, pharmacists and practice assistants often act on the same output first, at triage for example. Whether any of them can make that check on a night shift is an organisational question.
Liability by country, and what medical associations say
Germany’s new product liability law (Produkthaftungsgesetz) completed its passage through parliament in October 2026, with the Bundestag vote on 8 October and the Bundesrat on 16 October; it applies from 9 December 2026, and products placed on the market before then stay under the old law. Austria has not yet transposed the directive, due by 9 December 2026, and its current law (§ 4 PHG) covers movable physical things and energy, so it is disputed whether stand-alone software counts. Switzerland, where the directive does not apply, has no AI-specific liability law.[12] The withdrawn AI Liability Directive would have lightened the burden of proof for people harmed by AI.[11] The German Medical Association keeps responsibility with the physician, who checks AI output for plausibility, and the 2026 Deutscher Ärztetag called for physicians not to be held liable for errors made by AI used as intended.[13] Austria’s Medical Chamber says final responsibility must stay with the physician, and the Swiss Medical Association (FMH) says that neither marketing claims nor certifications can replace a critical check of every language-model output.[14]
Where your team can start on AI oversight: three questions and a first pass
None of this needs to wait for a deadline. For tools already in use there may be none, and in Switzerland there is none yet, so whether anyone checks them is a leadership decision. The less regulated the tool, the more question 3 matters: many scribes, evidence chatbots and phone assistants are not medical devices at all, so nobody is named to notice if their output gets worse.
Device incidents have an owner on site. Performance doesn’t, yet.
Illustrative logs · already on record
Ward score · one night
Phone assistant · one morning
Illustrative entries. Signals like these are counted per tool, never per person.
On the ward we started with, the late inputs and the rising overrides were already on record. Looking at them together was nobody’s job, and no route for a concern had been agreed. The signs of drift may already be in your logs; what is missing is someone named to read them.
Which question would you put first?
Checking the output
Finding the high-risk tools
Naming an owner
Any of the three is a sensible start: a practice, residency programme or field team might begin with 1, a hospital with 2, a ward with 3. A number is enough; no need to name your hospital or tool.
Version 1 · published [date] · status as of 26 October 2026 · next scheduled check: 10 December 2026 [keep only if someone owns it]. This digest is not legal advice. If the position in your country has changed, tell us; we will update it and record the change below.
Changes to this digest
[Publish date] · First edition; status as of 26 October 2026.
References
[1] Kopanitsa G. Validation is not enough: longitudinal evidence of post-deployment fragility in clinical AI systems. PLOS Digit Health. 2026;5(7):e0001534. https://doi.org/10.1371/journal.pdig.0001534
[2] Subasri V, Krishnan A, Kore A, et al. Detecting and remediating harmful data shifts for the responsible deployment of clinical AI models. JAMA Netw Open. 2025;8(6):e2513685. https://doi.org/10.1001/jamanetworkopen.2025.13685
[3] Regulation (EU) 2024/1689 (AI Act), consolidated text of 27 July 2026, Articles 2, 4, 26, 50, 111 and 113, https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX%3A02024R1689-20260727; European Commission, AI literacy: questions and answers (supervision from 2 August 2026), https://digital-strategy.ec.europa.eu/en/faqs/ai-literacy-questions-answers
[4] Regulation (EU) 2026/1744 of 8 July 2026 (“Digital Omnibus on AI”). OJ L, 24 July 2026. https://eur-lex.europa.eu/eli/reg/2026/1744/oj/eng
[5] European Commission. Proposal to simplify the MDR and IVDR, COM(2025) 1023, 16 December 2025, https://health.ec.europa.eu/document/download/25e7ea7c-cab3-40cf-86d9-d11f5e7744d8_en?filename=md_com_2025-1023_act_en.pdf; Council of the EU, progress report 9801/26, 5 June 2026, https://data.consilium.europa.eu/doc/document/ST-9801-2026-INIT/en/pdf
[6] Incident routes. Germany: § 6 MPBetreibV, https://www.gesetze-im-internet.de/mpbetreibv_2025/__6.html; § 3 MPAMIV, https://www.gesetze-im-internet.de/mpamiv/__3.html; KBV, PraxisWissen KI (December 2025), https://www.kbv.de/documents/infothek/publikationen/praxiswissen/praxiswissen-ki.pdf. Austria: §§ 40–41 MPG 2021, https://www.basg.gv.at/gesundheitsberufe/medizinproduktebetreiber. Switzerland: MedDO Arts 4(2), 66(4) and 67(2), https://www.fedlex.admin.ch/eli/cc/2020/552/en; Swissmedic, vigilance, https://swissmedic.ch/swissmedic/en/home/medical-devices/reprocessing—maintenance/vigilance-mep.html. EU: Regulation (EU) 2017/745 (MDR), Arts 2(64) and 88, https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32017R0745
[7] Gesetz zur Marktüberwachung und Innovationsförderung von künstlicher Intelligenz (KI-MIG), BGBl. 2026 I Nr. 223, § 2. https://www.gesetze-im-internet.de/ki-mig/BJNR0DF0B0026.html
[8] European Commission. Market surveillance authorities under the AI Act (list updated 7 September 2026), https://digital-strategy.ec.europa.eu/en/policies/market-surveillance-authorities-under-ai-act; RTR KI-Servicestelle, FAQ, https://www.rtr.at/rtr/service/ki-servicestelle/faq/FAQ.de.html
[9] Swiss Federal Council. Media release, 12 February 2025, https://www.admin.ch/gov/en/start/documentation/media-releases.msg-id-104110.html; Federal Office of Justice, https://www.bj.admin.ch/de/kuenstliche-intelligenz
[10] Directive (EU) 2024/2853 on liability for defective products, Art. 8. https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32024L2853
[11] European Parliament. Legislative train: AI Liability Directive (withdrawn). https://www.europarl.europa.eu/legislative-train/theme-a-europe-fit-for-the-digital-age/file-ai-liability-directive
[12] Gesetz zur Modernisierung des Produkthaftungsrechts: Bundesrat Drucksache 584/26 (Bundestag resolution of 8 October 2026), https://dserver.bundestag.de/brd/2026/0584-26.pdf, and Bundesrat decision 584/26(B), 16 October 2026, https://dserver.bundestag.de/brd/2026/0584-26B.pdf; Faegre Drinker, Transposing the EU’s new Product Liability Directive: a Member State progress report (2026), https://www.faegredrinker.com/en/insights/publications/2026/6/transposing-the-eus-new-product-liability-directive-a-member-state-progress-report; Austria: § 4 Produkthaftungsgesetz, https://www.ris.bka.gv.at/NormDokument.wxe?Abfrage=Bundesnormen&Gesetzesnummer=10002864&Paragraf=4, and Nationalrat motion 1071/A(E), 14 October 2026, https://www.parlament.gv.at/dokument/XXVIII/A/1071/imfname_1783436.pdf; FMH, Rechtliche Überlegungen zu generativer KI, https://fmh.ch/themen/ehealth/trends-neue-technologien/generative-ki/rechtliche-ueberlegungen.cfm
[13] Bundesärztekammer. Stellungnahme “Künstliche Intelligenz in der Medizin” (February 2025), https://www.bundesaerztekammer.de/fileadmin/user_upload/wissenschaftlicher-beirat/Veroeffentlichungen/KI_in_der_Medizin_SN_neu.pdf; 130. Deutscher Ärztetag, press release, 15 May 2026, https://www.bundesaerztekammer.de/presse/aktuelles/detail/auswirkungen-von-ki-anwendungen-in-der-medizin-in-klinischen-studien-ueberpruefen
[14] Österreichische Ärztekammer, press information, 10 October 2024 (final responsibility with the physician), https://www.aerztekammer.at/presseinformation/-/asset_publisher/presseinformation/content/id/2825945; Österreichische Ärztezeitung, 26 May 2026 (checking AI-generated content critically), https://aerztezeitung.at/2026/oaz-artikel/politik/e-health-und-digitale-medizin-mehrwert-im-fokus-ki-war-gestern/; FMH, Empfehlungen zum Umgang mit grossen Sprachmodellen (LLMs) im ärztlichen Alltag (web page, accessed 26 October 2026), https://fmh.ch/themen/ehealth/trends-neue-technologien/generative-ki/empfehlungen-der-fmh.cfm
