USF uses AI to detect pain in newborns

Babies born prematurely are often too weak to cry, leaving no way for them to convey pain. University of South Florida researchers believe artificial intelligence can speak on their behalf.

An interdisciplinary team from USF Health, the USF Bellini College of Artificial Intelligence (AI), Cybersecurity and Computing and their partners at Tampa General Hospital (TGH) are utilizing technology and algorithms carefully crafted over nearly a decade to detect pain in vulnerable newborns. 

The continuous, real-time monitoring system interprets physical movements, facial expressions and respiratory rates and alerts neonatal intensive care unit (NICU) nurses. Dr. Yu Sun, the groundbreaking project’s principal investigator, said the overarching goal is to manage pain before it requires prescription narcotics. 

“You want to minimize the pain because the pain will affect the brain development of the infants,” Sun said. “On the other side, you do not want to give a lot of opioid-based pain medication to the babies.” 

Opioids also impact brain development, lead to dependencies and cause “many other problems.” Nurses can provide intravenous Tylenol, “one level down,” if they detect pain before it becomes too severe. 

Approximately one in 10 babies needs critical care in the NICU, according to the Health Care Cost Institute. Pre-term newborns are often incredibly small and lack fully developed lungs needed to cry. 

NICU nurses typically rely on periodic observations and a one-to-10 scoring system, both of which can vary widely, to assess pain. “They’re certainly dedicated to their work, but they just can’t be there all the time,” Sun said. 

What was once mild discomfort can quickly spike in between checks. Sun said the only solution then is to provide opioid-based medication, and “that’s where our research comes in.” 

The non-invasive, low-cost AI system uses a GoPro camera mounted inside incubators to record video of physical movements and facial expressions. Another camera and sensors monitor heart and respiratory rates. 

“We combine that data into a deep-learning model,” Sun explained. “We usually have a stress code of four. So, if you reach a certain pain level, the system will send an alarm to the nurse or care team … to take a look at the baby and decide what to do.

“The system is basically to provide an extra eye on the baby and send the alarm.” 

Dr. Yu Sun, the project’s principal investigator, analyzes algorithms.

USF researchers began collecting data in 2010. They subsequently started building computer models with funding from the National Institutes of Health (NIH). 

Sun, a computer science and engineering professor specializing in AI healthcare applications, noted the interdisciplinary team has secured multiple patents and written several papers “reporting the accuracy and results of this.” The next step is a randomized clinical trial. 

Partnering NICUs at Stanford University Hospital and Virginia’s Inova Hospital, in addition to TGH, provide infants’ data before and after surgeries. Families must first consent to participate in the study. 

“Because this alarm is sent to the nurses, we don’t want a lot of false alarms,” Sun said. “Data collection and model building are one thing. Having a design that works well in NICU settings, in the critical flow, is different. They’ve got a lot of other things to do. They will throw this away if there are too many false alarms.”

Researchers need to achieve at least 90% accuracy. The system was at 85% before the latest grant. 

Sun said dim lighting presented a challenge, and the team needed a night vision-equipped camera. Babies turning their heads away was another hurdle to overcome. 

Sun is confident the researchers will achieve their 90% goal within the next year, potentially transforming treatments in NICUs nationwide. He also believes “there’s a lot of room to make their incubators smarter.” 

Healthcare providers are now “much more open” to using the new technology and contemplate additional use cases, Sun said. “I think they are quite excited.” 

He stressed the importance of building trust with care teams over time. Sun said the Bellini College is adept at conducting need-based research, which requires close collaboration with healthcare providers. 

“We’re not looking to replace nurses,” Sun emphasized. “We provide a tool so we don’t need nurses at every bedside, all the time. We rely on nurses to finally look at the baby to make a decision, and physicians decide the treatment protocol.” 

USF offers some collected data to other researchers by request, with safeguards to protect patient privacy.