#Outset medical logo manual#

For reference, WUSM’s manual approach for identifying patients at risk for PTHrP had an F1 score of 0.6, making the algorithm a significant improvement over standard practice. The winning team, Team Kagglist, achieved an F1 score of 0.9 with their predictive model. For the final attempt, each team ran their predictive model through the private dataset.

Each time a team submitted a predictive model for an attempt, they used the resulting F1 score to improve-or “train”-the model. Organizers used a second, private dataset to judge the algorithm’s effectiveness.įrom May to June 2022, 24 teams ran a total of 395 iterations of their predictive models through the public dataset. Overfitting would mean the algorithm worked well on the initial data but failed if applied to new data and was not generalizable. Organizers set up the competition using Kaggle, a popular online platform for machine-learning modeling and contests, and selected F1 score (the harmonic mean of sensitivity and specificity) as the performance metric (Figure 1).Ī major challenge for the teams was developing a predictive model that achieved high accuracy without overfitting it to the public dataset (the practice dataset). “With real patient data, the process is going to be more engaging and more interesting.”

“We wanted the competition to be grounded in something that was realistic because it’s a better way to learn,” he said. Using real clinical data was a big deal because most machine-learning competitions use synthesized datasets, Zaydman noted. A successful predictive algorithm would help laboratorians quickly and accurately identify potentially inappropriate PTHrP test orders by predicting whether laboratory data available at the time of order already suggest an abnormal PTHrP result.Ĭompetition participants formed teams and used securely shared real, deidentified clinical data from PTHrP orders at WUSM to build their predictive algorithms. Excessive PTHrP testing can lead to expensive, unnecessary, and potentially harmful procedures, including invasive laboratory testing to locate a possibly nonexistent cancerous tumor. The problem: Clinicians often order it for patients with low pretest probability. The purpose of the competition was to see if a machine-learning approach could better predict test outcomes compared to the traditional manual approach many clinical laboratories use, reviewing calcium and PTH results to identify potential inappropriate parathyroid hormone-related peptide (PTHrP) orders.Ĭlinicians rely on PTHrP measurement to help establish a diagnosis of humoral hypercalcemia of malignancy-a rare form of cancer that causes, among other things, high levels of calcium in the blood. That’s the outlook we brought to the call for submissions to the PTHrP results competition.” Leveraging Real-World Data “Ultimately, we want to achieve better outcomes for the patient and the institution. “With data analytics, we’re trying to bring more quantitative rigor to lab medicine-as you’d see in other disciplines, like finance,” said competition organizer Mark Zaydman, MD, PhD, an assistant professor of pathology and immunology at WUSM. Through this competition,AACC and WUSM aimed to engage the community of laboratory medicine practitioners in a fun and friendly online environment where they could practice their data analytics skills, learn from each other, and see how others approach problems on the data-driven side of laboratory medicine. Louis (WUSM), to introduce AACC’s first machine-learning challenge, the Predicting PTHrP Results Competition, at the 2022 AACC Annual Scientific Meeting in Chicago. As part of this effort, AACC worked with the informatics section in the department of pathology and immunology of Washington University School of Medicine, St. AACC has focused on enabling members to lead at the forefront of this evolution in healthcare, using the wealth of laboratories’ data to create better medical outcomes for patients.Īs a pillar in the association’s strategic plan and through the actions of the Data Analytics Steering Committee, AACC has formed a three-pronged approach for data analytics education, data access, and community building. Advances in machine learning and data analytics are transforming the field of laboratory medicine.

Outset medical logo