Melissa Mulholland was 16 weeks pregnant with her second child when her doctor noticed something unusual in an ultrasound scan. It was a rare fetal condition called posterior urethral valves, PUV, and it meant her son wouldn’t survive the womb without medical intervention.
She was fortunate to have a doctor skilled in detecting the condition and intervening to address it, and the good news is that her son, Conor, is now 5 years old.
But the experience left Mulholland thinking about the families who aren’t so lucky to have such expert health care. She wondered if technology could be a solution. She’s not an engineer, and doesn’t have a technical background, but she works at Microsoft, so she’s familiar with the latest technologies in her role working with the company’s cloud customers and partners.
She asked a question that not a lot of people would ask: could artificial intelligence help?
The answer was yes. What happened next was a case study in the potential of machine learning and other forms of artificial intelligence to transform health care and other industries, particularly as Microsoft, Amazon, Google and others work to make the tools more accessible.
Melissa Mulholland tells us her family’s story on this episode of the GeekWire Health Tech Podcast. Listen above, or subscribe in your favorite podcast app, and continue reading for edited highlights.
Melissa Mulholland: I work at Microsoft, and I help partners with their skilling and their enablement globally. It means helping them build technical capability in the market so they can have successful businesses with us. I’m really fortunate because I get to work with partners every single day, discovering their best practices, and how they are building out their ecosystem with us. I write and curate these playbooks. Think of them as comprehensive guides on how to build a business: how to build a business on AI, IOT, cloud migration, etc. So they are very technology focused, but then we put them into real terms: how do you think about going out and securing customers? How do you think about hiring and skilling the right people to ultimately build a sustainable business long-term with us?
Todd Bishop: Are you technical by nature? Are you an engineer by training?
Mulholland: Not at all. I’m just super curious. The thing about working at Microsoft is it’s enabled me to really build a whole set of knowledge that isn’t anything that I was taught in school. By foundation, I have a sales, marketing and business background, but I happen to be really excited about how we can use technology to shape lives. At the end of the day, we have one life to live, and so if we can use technology in ways to help the world, to help others, it’s extremely meaningful. I’m curious, more than anything else.
TB: Tell me about your family.
Mulholland: I have two children, Conor, who is 5 years old, and Emma, who is 6 years old. Two kids, and a wonderful husband who’s from the Pacific Northwest. My first born, Emma, is a typically developing child, successful in terms of my pregnancy. But my son Conor is a complete medical miracle. That’s why I smile when I say he’s turning 5. It’s quite a remarkable thing to be thinking about as he really shouldn’t be here today. In 2014, we discovered that we were pregnant with our second child, and about 16 weeks into the pregnancy we learned that he would not survive the womb due to a condition called posterior urethral valves, PUV.
At the time, we were really fortunate because we just coincidentally happened to be at a clinic that was high-risk in Bellevue, Wash. We met a world-renowned doctor just by the off chance that he happened to be in the office that day. He confirmed that my son wouldn’t be able to make it without medical intervention, and there was a very high-risk surgery that they could do if his kidneys were viable for saving. This was Father’s Day weekend, and we had to make a call. Do we want to take the chance of intervening, or do we want to just say we can’t make it?
We knew that we had up until 28 weeks to viably terminate if we had to go that direction. But as a mother, I psychologically knew I had to do whatever it could take to save his life. We decided to see if his kidneys were functioning enough that we could actually intervene. Thankfully they were. We intervened at 16 weeks by essentially diverting the urine, because he had a blockage. Essentially, imagine never being able to void, which inhibits lung development, all sorts of things.
With that intervention, we had less than a 2 percent chance of survival. I’m so grateful to say that he survived today. But what struck me during the whole process was looking at how technology was really the factor in discovering the issue. I became extremely motivated to see how can we leverage technology and my role at Microsoft to be able to help others in these types of scenarios.
TB: But you and your husband were fortunate enough to be at this clinic in Bellevue, Wash., where they were able to spot this, because it was one of their specialties.
Mulholland: Well, one, we had a phenomenal radiologist. Typically you don’t have a radiologist doctor do your ultrasound scans. In most scenarios you have a technician. I’m not dismissing technicians, they’re fabulous. They do great work each and every day, but this doctor knew what he was looking for, and was just highly skilled. I happened to have a doctor that knew what he was looking for, combined with a doctor who was surgically equipped to do extreme medical intervention. He’s world-renowned for this, and I feel like I got the luck of the draw. How in the world I got the odds of being able to be at the right place at the right time for a condition that today, one in 8,000 boys have, but oftentimes, it’s not caught early enough. The outcome is, if you can’t catch it early enough, oftentimes most kids won’t survive.
TB: You have a glimpse into all of this artificial intelligence through your work with the company’s partners. You know that lots of data can provide patterns. It sounds like that led you to some big-picture thoughts about whether this kind of detection could be automated.
Mulholland: Yes. About six months post all of this occurring, we came out with an AI API through what we call Cognitive Services. It’s custom vision, the ability to detect and recognize objects. You can train a machine learning model where we build the algorithms, and a partner or company can build upon those APIs. At the time, we had just released this, and I was really excited about the ability to take two different objects and be able to train a model. I thought, “Well, couldn’t we do this with ultrasound images?” Especially in the case of my son’s ultrasound scan, the bladder image is very much the shape of a balloon. It has a little bit of a tail at the end. A normal bladder is going to look round, as you would expect. Very quickly, we were able to pull open source images and take an image of this condition including my son’s scanned image, and we were able to train a model in less than an hour, costing 16 cents.
TB: When you say “we” in this context, is this literally you at your computer? Did you pull in some colleagues, or others? Tell me about the process that you went through.
Mulholland: Great question because I should clarify that. As we talked about earlier, I don’t have, by default, a technical background. … In this case, I met a company called InterKnowlogy. Tim Huckaby, the founder of the company, had really gotten me excited about using AI.
Tim Huckaby, founder and executive chairman of InterKnowlogy: The amazing thing about Conor’s story is this rare syndrome, PUV, is easily caught by physicians if they’re trained to look for it. But you can’t train all physicians to look for it. So when it’s missed, it’s 100 percent fatal. …
I said to Melissa, ‘If you can get me the ultrasound of Conor, I’ll gather up a bunch of publicly available ultrasound scans and pictures of PUV and then I’ll create a machine learning model and build you a little proof of concept. I’ll prove to you that this type of tool, this machine learning thing can truly save lives.” It’s not going to replace doctors, but it’s the type of thing that could help doctors make diagnoses.
I’m 57 years old now, and started my career at Microsoft, working on server product teams. So I’m way beyond my production programming years. But I got so excited about this that I found myself trying to figure this problem out at 3 o’clock in the morning. And it’s been a long time since I’ve pulled an all-nighter, scratching my head on how to pull this off. In the middle of the night, I had built this machine learning model and built a little wrapper around it so that it could be tested.
And sure enough it was working great with an incredible confidence level. In computer vision you always get some form of score or confidence, meaning if I’m doing a facial recognition, 78 percent confidence that’s Tim Huckaby, or in some of the stuff my company does, like weapon recognition, 67 percent confidence that’s in AK-47 in that crowd, you know, that type of stuff.
So I was getting unbelievable precision (in detecting PUV). I sent her a note in the middle of the night and I said, you gotta see this. So I demoed it for her next day and she’s basically like, “You are frickin’ kidding me.” And I’m like, “I know, right?” So we built a little test harness around it so that it could be demoed publicly. And next thing I know, a whole bunch of things happen.
Number one, I’m doing a keynote demo of it at this huge Microsoft conference. And then I find out that Satya Nadella himself, the CEO of Microsoft, is telling the story of this thing. If you put this technology simply in the process by which a physician looks at ultrasounds, the radiology process, it would flag, “Hey, I’m 80 percent confident this is PUV, look at it.” We don’t replace doctors. These are just tools to help them.
Mulholland: Together we were able to train a model to 99 percent accuracy in less than an hour.
TB: After you and Tim went through this process, what happened next?
Mulholland: We started to share our story because I got so inspired by how simply we could leverage a technology Microsoft already had. Yes, Tim is very technical and, yes, he has an incredible set of developers, but in this case, they didn’t build the technology. The technology was already there. We started to share our story around the world and really motivate other companies to start building out healthcare-based businesses.
In the realm of AI, it is so important to have very targeted scenarios, because it is really the art of the possible. You can think of limitless ideas around scenarios that you can land with customers, but you really need to make sure that you have things that are going to resonate out there. We started to share these with other companies, and it got me so excited, I actually ended up writing an entire book on AI, which is an artificial intelligence playbook.
TB: Obviously, what you did is a key demonstration of the potential of AI. Do you have hopes long-term that this could be something that might be brought to the world of healthcare in a way that it would be used by ultrasound technicians?
Mulholland: I sure hope so. I think healthcare is so deficient of technology day in and day out. We have so many opportunities to make the experience really positive for both the patients as well as, in this case, the mother of a patient, through technology. There are companies around the world that are doing similar scenarios. There’s one company that has actually built, through machine learning models, the ability to detect specific types of cancer because in that specific market, they can access data. \I think it’s definitely emerging.
The key is for companies to really be thoughtful about, what is the intention that they’re trying to serve and being really pragmatic around where they can actually build solutions because, more often than not, companies don’t actually make money, because they don’t have a long-term viable customer pipeline that they can sell into. So my hope is that, yes, you’re starting to see AI be used around the world. The other day, I was flying and they uses AI through same technology we were just discussing, custom vision. It detected my image. It mapped to my passport. I didn’t have to check in to the flight. I didn’t even have to show my passport because it recognized me. We’re seeing more and more of that emerge.
TB: It’s interesting because, in the example you just gave, which I believe was Delta Airlines, they’re doing that at Sea-Tac Airport and other places where they’re using image recognition to confirm identity. That’s one of the things that brings up concerns about AI in areas like privacy, but at the same time, you don’t hear as many stories about people using it for good, and it sounds like that’s largely your focus.
Mulholland: I am so passionate about it because I think companies have to take an ethical stance. We at Microsoft are really standing behind that because we know that at the end of the day, we want it to be trusted.
TB: Tell me about your son. What’s his life like? What do you feel comfortable sharing?
Mulholland: While he’s a complete medical miracle, our story did not end with birth. He had over 12 surgeries from birth to 2 years old, and as an outcome, he’s severely developmentally delayed. It’s a great point that you bring up because what I often don’t share is that he’s on the autism spectrum, and in working with one partner actually based in France, they came up with a solution leveraging AI to help their employees’ children be able to learn how to speak, and I got so excited about it because it resonated to my son, who is just now learning how to speak.
Sometimes, with kids with autism, they may have low verbal skills, and he is certainly one of those children, and in this specific solution, they created an application that basically teaches the child. You take a picture of an apple and, for example, if you hit apple, it’ll say apple back. One of the key ways children learn how to speak is through images. It’s actually teaching him how to speak.
His life has been completely shaped by technology, and I’m sure in many more ways to come in the future, but he’s a very sweet, loving boy. I’m so grateful that we’re blessed with him today.
TB: What advice would you give to other people with non-technical backgrounds, people from marketing, other parts of businesses, about the accessibility of artificial intelligence and these advanced forms of technology? Could anybody team up with a developer like you did and come up with a solution, or are you in a unique position because of where you are inside Microsoft?
Mulholland: Anyone can do it. My biggest piece of advice would be just go out there and learn, read. I learned so much by just reading about use cases, or stories in the news and media. Stay super curious, and you can teach yourself so much without actually having any technical background at all. The more I learn, the better I am at the job that I’m able to do because I can think and discover ways of using solutions, technical solutions, in ways to overcome challenges in the workforce. That’s the era it is today. We all have to learn.
TB: Microsoft CEO Satya Nadella is very public about the fact that his own son has a disability, cerebral palsy. I can imagine that there’s just natural empathy coming straight down from the leadership to others who not only might have disabilities themselves working inside the organization, but those who have close family members whose disabilities they help to manage.
Mulholland: It’s so true. I think he’s extremely empathetic as a leader, which really has cultivated the culture over the past three years. You can see that cultural transformation and shift. When I shared my story with Satya, because I couldn’t help myself, he emailed me back immediately, and he said, “You’ve inspired me,” but what I really was inspired by was how he took just my story and shared it across the company because it, to your point, it reminded him of his scenario of his son. He said, “Imagine back then if I had access to this type of technology, the things I could’ve done to help my son,” and so it’s a very personal connection with him. With it being so open, that’s why I’m always open to sharing my story. I think it’s so important to be empathetic to the world that we live in. We all have struggles in our daily lives, and I think it makes your story that much more relatable.
