An ACTES user interacting with the program

Clinical Research Coordinators in the Emergency Department at Cincinnati Children’s are using ACTES, an artificial intelligence tool, to identify eligible subjects from Electronic Health Records.

More than 70,000 patients visit the Emergency Department at Cincinnati Children’s each year. Visits are unplanned, conditions are complex, and demographics are varied. There is no predictability. It is the job of Clinical Research Coordinators (CRCs) to identify which of these patients may be eligible for clinical trials. Could artificial intelligence help?

A team of researchers from Cincinnati Children’s, led by Yizhao Ni, PhD, of the Department of Biomedical Informatics, is testing a new computerized solution in this busy clinical environment. ACTES—short for Automated Clinical Trial Eligibility Screener—uses artificial intelligence to identify patients who may be good study candidates.

There are eight full-time CRCs on staff in the Emergency Department. As patients are admitted, information is entered into Epic, an Electronic Health Record (EHR) system. CRCs review each patient’s chart, looking for details associated with eligibility criteria for open clinical trials.

These details can be found in structured data (vital signs, medications, procedure orders) or unstructured notes from nurses and physicians (chief complaints, signs, symptoms). At times, CRCs must track down clinical staff to ask questions about a patient’s condition or treatment.

If the patient is an eligible candidate for enrollment, the CRC will approach the patient’s family to talk with them about the clinical trial. The CRC will then either enroll the patient or remove them from the list if the family declines.

In the Emergency Department, CRCs have a short window of time to complete these steps—the average patient stay is just 3.4 hours.

ACTES is designed to streamline this process by leveraging natural language processing and machine learning technologies. The system extracts structured data from the EHR and identifies unstructured information from clinical notes. ACTES then matches this content with eligibility criteria to determine patients’ suitability for clinical trials.

In a study published in JMIR Medical Informatics, ACTES reduced patient screening time by 34 percent and improved patient enrollment by 11.1 percent compared with manual screening. The system also improved the number of patients screened by 14.7 percent and those approached by 11.1 percent.

“ACTES has a lot of potential in the Emergency Department,” says CRC Noura Barazi. “Because we see high volumes of patients, it can be overwhelming for a CRC to screen every single patient that comes through the ED. ACTES puts all the information in one place, so we can filter through only what pertains to active studies.”

When CRCs first open ACTES, they see a long list of information. But when they select an active study, the system populates a list of patients who meet its eligibility criteria. CRCs can mark patients as eligible, ineligible, approached, enrolled, or declined—following each step of the enrollment process.

The system’s design is the result of close collaboration between the research team and CRC staff. “We learned a lot from interviewing our users and understanding their needs,” says Ni. “The question we started with was, ‘How can artificial intelligence improve, rather than add a further burden to, clinical studies?’ We found that the best way to leverage an AI solution is to meet the users’ needs.”

Since ACTES curates the 15 data fields most commonly reviewed by CRCs, users can quickly find information that is relevant to their work. They can also search for specific keywords without reading through nurse and physician notes.

“We’re less likely to miss a patient who could benefit from participating in a study,” says CRC Gena Koutsounadis. This is particularly true in studies with multiple eligibility criteria, including HealthyFamily, a study that aims to reduce children’s exposure to secondhand smoke. “A patient’s chief complaint during their visit could be fever, but the notes mention asthma, which is much easier to identify in ACTES.”

Sometimes, the system recommends patients who are not actually eligible for a study, a sign of the imperfection of AI technologies. “However, our human-factors engineering design turns imperfection to an advantage,” says Ni. “We display information in a way that improves users’ perception of false positive results, saving time on further screening those candidates.”

Thanks to machine learning technologies, ACTES improves its recommendations with every enrollment. The system continuously evolves by analyzing an ever-growing pool of patient data.

“To see ACTES improve over time has been neat,” says CRC John Witry. “With machine learning and artificial intelligence, it will get even better as time goes on.”

For more information on ACTES, contact

Finding Clinical Trial Candidates at Cincinnati Children’s: A Collaborative AI Solution