A Clinical Decision Calibration Study

What This Is

Threshold is an anonymous, interactive clinical scenario tool designed to measure where urgent care providers draw the line between continued workup and emergency department transfer.

Providers walk through realistic patient cases, entering their clinical reasoning at each decision point in free text — no multiple choice, no right answers. The tool responds to each decision with updated clinical information, advancing the case realistically regardless of the path taken. Every scenario ends with the patient receiving appropriate care.

Why We Built It

Urgent care medicine sits at a productive but sometimes uncomfortable boundary. The decisions made at that boundary — when to continue a workup, when to involve a specialist, when to transfer — are rarely discussed openly, and almost never measured.

As a result, providers at the same institution may have substantially different thresholds for the same clinical situation. Some of that variation reflects appropriate clinical judgment. Some of it reflects assumption, habit, or uncertainty about what the institution expects and will support.

Threshold was designed to make that variation visible — not to punish outliers, but to give every provider a clear picture of where their peers stand, and to give the institution a data-informed basis for guidance development.

How It Works

Providers access the tool via a unique single-use link — no login, no account. Two brief demographic questions are collected at the start: provider type and years of experience. No name, no identifier.

Each case is structured around a series of clinical nodes. At each node, the provider reads an updated clinical picture and enters their assessment and intended next steps in free text. The tool processes that input, advances the case to the appropriate next state, and presents new clinical information. Providers who pursue aggressive workup encounter a different path through the case than providers who transfer early — but all paths converge on the same clinical outcome.

At the end of each scenario, providers see a brief resolution: what happened to the patient, and what was ultimately found. There is no score. There is no verdict.

What the Data Captures

At each decision point, provider input is classified into one of five categories:

• Continue WorkupOrders tests, medications, or observation without disposition decision.
• Request ConsultSeeks specialist input or attending review.
• Initiate TransferInitiates or discusses emergency department transfer.
• Discharge with Follow-UpPlans discharge with outpatient follow-up arranged.
• OtherInput unclear or does not fit above categories.

For each scenario, the primary endpoint is the transfer distribution by node — what proportion of providers initiated transfer at each clinical juncture. Secondary data includes provider type and experience level as aggregate filters.

All raw free-text input is stored alongside the coded classification and is available for qualitative review by the QI team.

What We Can Conclude

When a substantial majority of respondents converge on a transfer decision at a particular clinical trigger, that convergence represents peer-generated consensus on the standard of care for that situation in an urgent care setting.

A pre-specified consensus threshold of 70% or greater is used. Where that threshold is met, the finding supports development of institutional transfer guidance — guidance that can be cited as reflecting the collective judgment of providers themselves, not top-down policy.

With approximately 40 Crystal Run urgent care providers and optional participation from regional Optum NY/NJ urgent care colleagues, the study is powered for descriptive consensus analysis. If regional participation extends the sample toward 80–100 respondents, findings are interpretable with sufficient confidence for formal QI reporting.

Aggregate findings will be shared with all participating providers. No individual session data will be attributed or disclosed.

What We Cannot Conclude

Threshold is designed with honesty about its limitations:

• It measures stated behavior, not observed behavior.Provider responses reflect how they reason through a scenario — not a verified record of what they do in practice. Correlation with actual transfer rates and test-ordering patterns requires separate analysis of clinical data.
• It cannot establish that any individual provider is practicing outside the standard of care.The data is anonymous by design. It produces population-level findings, not individual assessments.
• Scenario selection shapes findings.Cases were chosen because they have defensible bright lines — points at which a reasonable majority of urgent care providers would consider transfer. Cases without clear bright lines were excluded. The tool measures consensus on specific, deliberately selected clinical situations, not urgent care practice in general.
• Social desirability bias cannot be fully eliminated.Branching free-text scenario design reduces but does not eliminate the possibility that providers respond as they believe they should, rather than as they would in practice. Findings should be interpreted alongside, not instead of, operational data.
• Sample size limits subgroup analysis.With 40 providers, findings by experience level or provider type are directional, not definitive.

Study Parameters

Study Team