Operating Point

This page aims to highlight the role of the “operating point” selected by AI developers when building a new model, while also providing some baseline results for the National COVID-19 Chest Imaging Database (NCCID). The operating point allows AI developers to adjust the threshold used by AI models to make a binary decision. This means an AI tool could be made more conservative by giving a positive answer even when unsure, and vice versa. A Receiver Operating Characteristic (ROC) curve can be plotted to represent the relationship between the True Positives (TPs) correctly detected by an AI model against the True Negatives (TNs) in function of the operating point. Typically, the operating point in healthcare applications for automated diagnosis would be set by AI developers so it guarantees that as many symptomatic patients are detected earlier in the healthcare pathway while keeping the false positives as low as possible to keep triage efficient and avoid any unnecessary stress for the patients.

Below, an AI model has been trained on the chest x-rays stored in the NCCID to predict whether a patient was COVID-positive or negative, using polymerase chain reaction (PCR) tests as a reference. This AI model has purely been built as a prototype for research and is not meant for real-world deployment. The model was tested using data from 4,872 patients (3,457 COVID-negative and 1,415 COVID-positive patients) held out from the NCCID training data.