Under the Hood: A Healthcare Executive's Guide to AI Buzzwords Pt. 2
Editor’s Note: We’re excited to continue a new monthly series to demystify healthcare AI concepts, empowering leaders to make informed decisions. This month continues our series providing a history and clear explanations of common AI acronyms and terms.
Previously on “Under the Hood,” we started unraveling the buzzword overload in vendor marketing and pitch decks. We discussed machine learning with an emphasis on natural language processing (NLP) and computer vision (CV).
This month, we’ll explore neural networks, deep learning, and transformers: the building blocks of generative AI and the rapid advancements seen over the last several years.
Let's start with neural networks, the core building blocks of modern AI. These computational models loosely imitate the brain's structure, using interconnected nodes (neurons) to process and transmit information.
Geoffrey Hinton, the “godfather of deep learning,” described neural networks as, “…models of neurons. They have connections coming in, each connection has a weight on it, and that weight can be changed through learning. And what a neuron does is take the activities on the connections times the weights, adds them all up, and then decides whether to send an output. If it gets a big enough sum, it sends an output.”
On their own, single-layer neural networks can recognize basic clinical patterns like determining if a patient's vital signs exceed predefined thresholds or separating fetal ECG from a composite ECG of both maternal and fetal ECGs. However, many healthcare problems involve more complex patterns that are difficult to capture with just a single computational step.
Deep learning addresses this complexity by stacking multiple layers of neurons, each refining insights from the previous layers. This multi-layer—or “deep”—architecture helps models interpret complicated, real-world healthcare data like identifying early-stage tumors from medical imaging or predicting patient deterioration from subtle lab result changes.
Around 2010, two key developments propelled deep learning forward, rapidly accelerating progress and practical impact:
Researchers (including Hinton) realized that graphics processing units (GPUs), the specialized computer processors historically used mostly for video games, could train neural networks. This new processing power significantly sped up and reduced cost barriers, unlocking faster iteration and the ability to generate increasingly deeper networks.
Web 2.0 generated massive labeled datasets—especially images—transforming deep learning from niche experimentation into a practical toolkit. Models like AlexNet, which revolutionized image recognition in 2012, and Facebook’s DeepFace in 2014 showcased this capability vividly.
Traditional neural networks and deep learning models effectively analyze structured data and uncover complex patterns in images or text. However, even deep neural networks struggle with long context sequences, which are required for tasks like understanding clinical notes, conversations, or lengthy patient histories.
In 2017, everything changed when Google published “Attention Is All You Need.” This paper introduced transformers: not the cartoon robot vehicles of the 1980s, but instead, a groundbreaking neural network architecture that handles sequential data exceptionally well, whether it’s text, time series, or patient trajectories, by employing what’s called self-attention.
Unlike previous models, self-attention allows transformers to weigh each data element dynamically against every other. This capability captures nuanced relationships across long distances in a sequence. Transformers excel at modeling context and nuance. This is the reason systems like GPT-4 can now generate coherent clinical summaries, engage in human-like dialogue via voice agents, or interpret complex medical literature with striking precision.
Next month, we’ll detail generative AI and large language models (LLMs), connecting the fundamentals and applications now transforming healthcare from revenue cycle management to clinical workflows.