A healthcare AI readiness assessment should run in stages: screen the AI's clinical documentation across a case cohort, then have local clinicians review it before any bounded proof of concept (POC). When institutions consider agentic AI for a bounded clinical-workflow pilot, the question underneath every pitch is how to screen it across the dimensions that matter without mistaking a benchmark result for production or clinical readiness.
V1 reports machine-side structural and operational screening of twelve foundation models on one synthetic-regenerated Indonesian MCU corpus; local expert adjudication, clinical suitability, and production readiness are outside V1.
A demo cannot establish whether a healthcare AI is suitable even for a bounded clinical review or POC. It shows one good report and says nothing about repeatability, completeness, traceability to documented findings, or review burden across a cohort. Those properties need structured evidence, followed by local expert judgment.
We developed a staged screen for AI-generated clinical documentation in an Indonesian medical check-up (MCU) benchmark. The machine-side stage checks structural and operational properties; outputs that clear it still require local clinical review before a bounded POC can be considered. The benchmark does not certify a system for production or clinical use. The full V1 report is here.
What the method is
The method answers two questions in order. Does the output meet a structural and operational screen so it can advance to clinical review? Then, do local experts judge it clinically and operationally suitable for a bounded POC in their setting? Each stage narrows what a person has to inspect; none replaces production validation or clinical governance.
How the evaluation is staged
A scope freeze fixes the workflow, prompt, schema, and model so results are comparable. A machine-side gate then checks structural and operational properties: valid and complete output, required fields, a schema-valid fitness-for-work field, zero defined machine-critical errors, and repeat-stability under the frozen test. Passing these checks means the output may advance to clinical review; it does not establish that the fitness decision is clinically correct or that the system is ready for production.
Running cleanly is only half the screen. The machine-side rubric checks predefined structural and operational properties and signals related to five clinical and workflow dimensions; it does not establish clinical correctness. Local experts must interpret the evidence and decide whether an output is clinically correct and suitable for their setting. The published, weighted rubric has 24 individually-scored criteria and three safety blockers that auto-fail an output, for 100 points:
The clinical & workflow rubric — how each output is scored (100 pts)
- Safety & EscalationCritical-value flagging, scope boundaries, and no hallucinated clinical data — with three criteria that auto-fail an output.25 pts
- Evidence & AccuracyEvidence grounding, correct reference ranges, risk classification, and no internal contradiction.25 pts
- CompletenessRequired fields, full finding coverage, no orphan findings, and follow-up timelines.20 pts
- Context AwarenessDemographic and occupational (K3 / Hiperkes) context, with no context hallucination.20 pts
- Communication & UsabilityLanguage register, structured and parseable output, and signals relevant to review. Actual reviewer effort and time require local measurement.10 pts
The bands are explicit: any safety blocker auto-fails the output; otherwise 80 or above with no critical issue is a POC candidate, 70–79 is monitor, and below 70 is remediation.
The rubric uses HealthBench as a methodological reference for clinician-authored, case-specific criteria rather than as proof that Micromeet's rubric is valid. Its clinical thresholds draw on a layered reference stack that still requires local expert review:
- Recognized clinical parameters — WHO diabetes definitions, WHO hypertension guidance, and the WHO Expert Consultation BMI action points for Asian populations, alongside local laboratory reference ranges.
- Indonesian occupational-health context — UU No. 1 Tahun 1970, PER-02/MEN/1980, and Permenaker No. 5 Tahun 2018 inform parts of the occupational-safety and worker-health context (K3 / Hiperkes). Each rubric criterion still needs an explicit mapping to the applicable source, institution SOP, and intended use.
- Institution SOP and local expert interpretation — each institution’s own reporting, referral, and sign-off rules, with disputed cases adjudicated by local clinical and occupational-health reviewers.
The method is informed by WHO guidance, the NIST AI Risk Management Framework, medical-device evaluation practice (IMDRF / GMLP), and clinical-AI reporting standards (DECIDE-AI, CONSORT-AI, TRIPOD+AI). These references do not validate the benchmark by themselves. A no-self cross-review — where the reviewer is never the model that produced the output — surfaces disputed cases for local expert adjudication outside V1.
Why a multi-model benchmark
For this benchmark, the workflow, prompt, schema, and rubric are held fixed while the foundation model is the variable; V1 compares twelve models from different vendors. The purpose runs deeper than scoring each model.
A workflow that only one model can pass is fragile. Models get deprecated, repriced, rate-limited, restricted in a region, or regress on an update — all outside your control. If a clinical workflow depends on a single model, any of those events can break it.
The goal is broader than crowning a winner: identify whether several models clear the same structural and operational screen under fixed test conditions. That produces a shortlist for local clinical review and a bounded POC, not a portfolio of production-qualified models. Testing across vendors can reduce early dependence on one provider, while production suitability, switching, integration, security, and change control still require separate validation.
What it’s worth to you
For an institution, the method turns an AI decision from a demo impression into evidence you can act on:
- You shortlist what to examine next using repeat runs across a synthetic-regenerated case cohort, so the decision is not based on one polished demo.
- You turn the results into concrete entry and acceptance criteria for a bounded POC; production and clinical acceptance require additional local evidence.
- You surface high-risk and disputed cases for local expert review; the institution defines the required review scope and measures reviewer effort and time in its own workflow.
- You compare models and vendors on one inspectable rubric — in your own language, terminology, and rules.
- You reduce early single-model dependence by taking more than one screened candidate into local review, without assuming interchangeability in production.
How to use it
The rubric is the core, and it belongs to the institution: what “correct” means depends on your check-up packages, your fit-to-work rules, and your SOP. There are two ways to run it:
- Your clinical, quality, and occupational-health owners write the rubric; the AI provider reviews it and flags anything unclear or unfair; then the agent is tested against it.
- You start from our reference rubric and adapt it to your scenario.
Either way, local experts must adjudicate the disputed cases before the result can support a clinical verdict, and the vendor must not grade its own work.
Put the evidence in context
Review the intended task, source information, human checkpoints and observed results together. A benchmark or sandbox result has a defined scope; it does not establish suitability, approval or performance for every institution.
Read the full explanationCare Workspace connects healthcare work such as appointments, reviewed reporting and follow-up. Micromeet Cowork supports team work with digital employees and reusable Skills, including meeting summaries, decisions and task handoffs.
Read the full explanationHow we can help
We can step in at whatever point you’re at:
- Share the method and the reference rubric from this benchmark.
- Help you adapt the rubric to your scenario and SOP.
- Run the staged evaluation on the models or agents you are considering.
- Support institution-led clinical review, with a responsible clinician and local reviewers adjudicating disputed cases under the institution's approved rubric.
Our first application of the method: Indonesian MCU
We applied the method to Indonesian MCU reporting across a twelve-model slate from different vendors, with the workflow held fixed and only the model varied.
V1 result — models that cleared the machine-side gate
All 12 models were covered in both cohorts. Clearing the gate indicates structural completeness and repeat-stability under this frozen test. It does not establish clinical correctness or production readiness.
Several models cleared the gate in both cohorts. That creates more than one candidate for the next step: local clinical review and, where the institution approves, a bounded POC. It does not qualify a provider for production or establish that models can be switched without further validation.
A machine-side pass is a structural and operational screen under the frozen V1 conditions. Clinical correctness is a separate verdict reached through the rubric and local experts, and production readiness would additionally require institution-specific workflow, security, integration, performance, monitoring, and governance validation. V1 is a pilot-scale signal on one synthetic-regenerated corpus under one fixed prompt; we keep it bounded by publishing the protocol, anonymizing lower-tier model names, and separating its machine-side results from local expert adjudication, which is outside V1.
The agent under test is MCU CoPilot — Micromeet AI for MCU — which drafts structured medical check-up reports from documented inputs for a doctor to review. Specialist findings — ECG, chest X-ray, audiometry, spirometry — are carried as reported by the responsible clinician. MCU CoPilot prepares drafts from documented inputs and configured rules; it does not autonomously interpret raw diagnostic images or signals, and it does not make or approve final clinical or fitness decisions. This is Micromeet — healthcare AI infrastructure for continuous care: clinical judgment stays with clinicians; the AI does not diagnose or sign.
Beyond MCU
V1 applies the staged pattern to MCU. The pattern may inform evaluation of other healthcare-AI documentation tasks, but the rubric, thresholds, data, and expert review must be adapted and validated locally for each institution, workflow, and market. V1 demonstrates the procedure on one synthetic-regenerated Indonesian MCU corpus under one fixed prompt; it does not establish transferability, doctor-supervised readiness, or production readiness. Local expert adjudication of disputed cases is outside the V1 results.
If you are weighing AI for your own clinical workflows, we are glad to share the method and the reference rubric, and to help you adapt and run it. Start with the full V1 report →
If your institution produces MCU reports, see how MCU CoPilot structures MCU evidence and prepares clinical synthesis and report drafts for physician review. Clinical judgment stays with clinicians; the AI does not diagnose or sign.
FAQ
What is the Indonesia MCU Healthcare AI Agent Readiness Benchmark? It is an open benchmark that runs Micromeet's staged structural and operational screen before local clinical review and a possible bounded POC. V1 holds one Indonesian medical check-up (MCU) reporting workflow fixed — the same prompt, schema, and rubric — and runs it across twelve foundation models from different vendors. It does not certify production or clinical readiness. The full V1 report is available on Micromeet's benchmark page.
How is this different from a model leaderboard? A leaderboard ranks raw model capability. This benchmark screens one fixed workflow configuration and routes clinical judgment to local expert review; it does not declare the system ready for doctor-supervised or production use.
What does a “pass” mean? A machine-side pass means the output cleared a structural and operational screen under the frozen test. It may advance to local clinical review and a possible bounded POC. It does not establish clinical correctness or production readiness.
Who owns the rubric? The institution. We provide a reference rubric and the method; local experts decide what is correct for the institution's scenario. V1 reports machine-side results only; local expert adjudication of disputed cases is outside V1.
How does V1 make the evaluation traceable and independent? V1 publishes its protocol and rubric, identifies the corpus as synthetic-regenerated, anonymizes lower-tier model names, and prohibits self-review in cross-reviews. Local expert adjudication of disputed cases is a separate step outside V1.
How does Micromeet help institutions evaluate healthcare AI? Micromeet shares the method and the reference rubric from this benchmark, helps adapt the rubric to an institution's own scenario and standard operating procedures, runs the staged evaluation on the models or agents under consideration, and brings clinical review — with local experts adjudicating the disputed cases. The same clinical-review principle applies to the agent under test: MCU CoPilot drafts structured medical check-up reports for a doctor to review. Clinical judgment stays with clinicians; the AI does not diagnose or sign.
About Micromeet AI — Micromeet provides healthcare AI infrastructure for continuous care, within institution-approved workflows. MCU CoPilot prepares structured medical check-up report drafts for authorized doctor review. The institution decides which steps run automatically and which need a person's confirmation; clinical judgment stays with clinicians, and the AI does not diagnose or sign.
V1 reports an automated, machine-side evaluation across synthetic-regenerated Indonesian MCU cases. Local expert adjudication of disputed cases is outside V1. The full method, pass gate, rubric, and references are in the V1 benchmark report.
