A new age of voice assistants is emerging, allowing us to control and query products using natural language. The medical industry, one of the most stringently controlled markets, is where this type of hands-free interfacing can create the most value.
Why Hands-Free Medical Voice Assistants? Patients confined to wheelchairs, beds, or immobilized in some way require a simple and intuitive means to control their environment, request help, or seek information. Hands-free usage can also improve productivity and capability for doctors. For example, surgeons whose hands are occupied, and eyes are focused on the patient may want to command and control devices or access data from machines.
Meeting Stringent Privacy Requirements. The Health Insurance Portability and Accountability Act (HIPAA) places strict requirements on patient privacy and the use of Protected Health Information (PHI), such as medical condition or treatment details. Recordings of patients must adhere to these stringent privacy requirements. Authorization and consent are required before making such recordings.
One way to maintain confidentiality and comply with HIPAA is by not making recordings. An on-device speech recognition system can perform this function without recording or sending information for transcription or analysis. Many medical devices are experiencing the benefits of Sensory’s TrulyHandsfree technology for wake words and small vocabulary command and control. Sensory’s TrulyNatural Speech-to-text STT can run on a simple PC or mobile device, offering complete on-device transcription and natural language interfacing with Natural Language Understanding (NLU).
Real World Use Case: Sonosite Voice Assist. In challenging environments like emergency rooms or during sterile procedures, hands-free on-device voice recognition helps to provide advanced care safely and efficiently. Sonosite’s Ultrasound Voice Assist offers medical professionals the ability to adjust ultrasound devices from across the bedside—completely hands-free. By incorporating voice recognition technology, Sonosite is redefining ease of use for point-of-care ultrasound, ensuring that doctors can maintain focus on the patient while still controlling essential equipment effortlessly.
Key takeaways from doctors using Sensory’s On-Device Speech Recognition:
“It makes an easy-to-use machine even easier.” – Christopher Peabody, MD, MPH (Emergency Medicine)
“That’s super responsive” – Patrick Ockerse, MD (Emergency Medicine)
“You could imagine this enabling a more hands free, kind of sterile environment” – Christopher Peabody, MD, MPH (Emergency Medicine)
“This could be further away, and you could still have pretty good control. This way you get more access to the patient, and still have total control of your ultrasound. I think thats really cool.” – Patrick Ockerse, MD (Emergency Medicine)
“I think this is so clearly something that we need.” – Cameron Murray Baston, MD MSCE (Critical Care)
“It’s surprisingly fast and responsive.” – Patrick Ockerse, MD (Emergency Medicine)
“Sometimes voice activation complicates things, but I feel like this one simplifies it.” – Patrick Ockerse, MD (Emergency Medicine)
Intelligent Wake Words? By training a wake word on a specific user’s voice, an intelligent medical assistant can respond differently to doctors and patients. For example, a patient might receive responses in more layman’s terms, while a doctor might prefer more technical explanations. A patient can also be restricted by their voice to control only certain devices or access specific information.
User-Defined Wake Words. With Sensory’s custom wake-word approach, an end user can easily and quickly create their own wake-up phrase. This could be particularly useful in a multi-user room, where one patient’s control of their bed settings might accidentally activate another patient’s device in the same room.
Hybrid Solutions. Edge or embedded voice systems are great for domain-specific assistants and command-and-control applications. However, sometimes a patient or doctor may require out-of-domain information.
For example, a bed-confined patient might want to get a stock quote, check the weather forecast, or find information about various topics through a cloud-based solution. A patient might be bored and want access to online quizzes or jokes for entertainment. A doctor might need off-device information from an on-premise or private cloud that summarizes patient history or lists common side effects of a drug they’re unfamiliar with.
In all these scenarios, Sensory’s on-device speech recognition solution can hand off to the on-premise data center or a cloud-based large language model, enhancing the user experience with greater knowledge and bandwidth. Sensory’s cognitive arbitration model can make an on-the-fly decision about how a query should be answered (whether on-device or through a handoff).
This hybrid approach creates an on-device private assistant that can control and respond within a limited set of domains while offering access to a broader range of information when needed for doctors and patients.
Next Steps. As medical voice assistants continue to evolve, they offer a new level of convenience while maintaining privacy in healthcare settings. From enhancing patient care to empowering doctors with hands-free control, the benefits are clear. To learn how Sensory’s on-device speech recognition solutions integrate with your medical device or healthcare application, contact us today.