Eisenberg: Hi, I'm Kate Eisenberg, and welcome to this special mini podcast series. This is the first episode of three highlighting the team behind Dyna AI. I am the senior medical director for Dyna AI. I’m a family physician, epidemiologist, and informaticist, and I've been involved with this team and with building Dyna AI from the start. So over the past couple of years, we've worked across disciplines to build this clinical decision support platform grounded in evidence shaped by real world clinical expertise, and this series is a chance to reflect on that work, what we've learned and celebrate the team that's gotten us here. So to kick things off today, I'm joined by two of our clinical leaders who have been absolutely instrumental in shaping Dyna AI from the beginning. I'm going to let them introduce themselves, starting with Diane Hanson.

Hanson: Great, thanks, Kate. Glad to be here. I'm Diane Hanson. I work as the chief nursing officer and editor in chief here at Clinical Decisions at Upskill. Been involved in the Dyna AI project from the beginning, specifically supporting team members who were participating on the team itself, but also came in at a time when we needed to scale our root cause analysis group and our clinical quality program and got deeper involvement in really giving us a direction on how to scale and bring more people into the work team to assure clinical quality. So it's been a great journey with that program.

Eisenberg: Fantastic. And, Alan, go ahead.

Ehrlich: Hi, I'm Alan Ehrlich. I'm the executive editor for DynaMed. I'm also an associate professor of family medicine and community health at the UMass Chan Medical School in Worcester, Massachusetts, and I've really been involved with this from the very beginning. I think I was brought in primarily, at the request of our editor in chief, Peter Oettgen, as the, my title should have been Chief Editorial Skeptic. And I was, you know, I was really brought on to make sure that whatever came out, if anything came out at all, it was of high quality and something that clinicians could rely on. And so that was the perspective I came to start this journey with.

Eisenberg: Okay. Thank you both, and thank you so much for taking the time to join today. And I would also just reflect that the two of you each have made just an incredible contribution to this tool. You come from different clinical backgrounds, and I'm interested, starting with Alan, you described yourself as the skeptic, the chief skeptic. Can you describe how your clinical background and your editorial role shaped your approach to this work?

Ehrlich: Sure. So when I said I was a skeptic, one of the things that I did initially was look at what was being produced by some of the tech folks, and I was harshly critical of it because it was not something that any clinician would want to use. And that's more a reflection, perhaps, of my lack of understanding of the tech development process, where they start out with something simple that may not necessarily reflect the final goal, but they build off of that, and they certainly built off of that with feedback from people like myself and Diane and others, and you, Kate, as well. For me, the main thing was that I was fixated on making sure users have something of high quality and high reliability. So just the fact that it was reliable isn't good enough if what it's producing is not going to be helpful to people in the moment. And obviously the converse is also true. If it's producing something of a very high quality that's wrong, well, that's a problem too, and I think that's something we've all seen with AI in various forms, whether it be medical or otherwise. This notion of the hallucinations, where it looks perfectly proper English, it's well cited, except that when you trace down to the originals, you realize there's no they're there. Those were the main areas of focus for me, to make sure that what we were producing is something I'd want to use in clinical practice, and that I would be comfortable using with my residents, showing them how to get answers very, very quickly to clinical questions.

Eisenberg: Oh, so you were bringing a couple hats there, that educator and teacher hat, in addition to your clinical role?

Ehrlich: Exactly.

Eisenberg: Diane, I wonder, you know, you have that nursing background as well as your leadership role. I'm wondering, you know, what are your thoughts on this same kind of journey, on making the tool usable and accessible to folks with similar backgrounds to you?

Hanson: I think as a nurse, you know, what you do is you understand kind of the needs of that discipline in the different roles that nurses play in the health care system, and then anticipating or understanding what are the curiosities, what are the information needs at all different parts of that workflow or that professional service delivery? And so really understanding that allows you to think through what are the queries that we're going to see and what would be the intent behind those. And I think we did a fair amount of work on the front end to try to understand intent and understand who we were serving in the health care industry. And then as a nursing leader, nursing leaders are looking for tools that they can trust so that care providers are not going outside and finding tools in the market that are giving them answers. And much like what Alan was saying, but there's really nothing behind the answer to substantiate what they're bringing forward. So, for instance, going out to Google and just googling answers to clinical questions, knowing that we have a, we have information that's been highly vetted and is evidence-based, being able to put a AI layer on top of that as a discoverability tool for that great content allows you to speed delivery. So nursing leaders are looking for ways to support point of care providers and doing their very best job maintaining quality and also getting speed for their answers. So I think, you know, bringing that view into the project work as part of the team, I think was an important lens to bring in for nurses.

Eisenberg: I agree, and I know for myself, having those different perspectives, the physician perspective, the nursing perspective, in addition to more technical and product-focused perspectives, has been a really fantastic learning journey to be on and to understand how each of those perspectives then can contribute to a stronger tool and a stronger outcome. I'm wondering, you know, speaking of speed, obviously there's always trade offs. There's always different priorities. Diane, how do you think we as a team navigated some of those tensions between moving quickly, clinical quality, meeting user needs and really making sure that we're fully investing in our clinical evaluation and governance program, as the organization has really backed us in doing.

Hanson: Yeah, I think that clinical precision was really important, obviously, for the clinical editorial team, just as Alan had mentioned, right. Coming in a little bit skeptical, but I think the team did a nice job at understanding we do have to bring something to market to have an impact on those who we’re trying to serve here, and we need to bring something to market so that it can be tested so we can learn. And we've been on this journey. So being able to let some things go, knowing that we're going to circle back around and that we have, at the same time, we're building a key structure to support clinical quality and putting governance around that, and also then putting processes and people in place with clinical backgrounds to support the precision that we need at the same time we're developing and moving forward. So I think the team had really good, honest conversation. If you think about product management, product development, the clinical editorial team all coming together, setting targets, but then putting the necessary processes in place to assure quality. I think we did a nice job kind of balancing that, always having to do enough planning, but also take action or you never have an end product.

Ehrlich: I would agree with that. I think that there was never a time when I felt we were sacrificing quality for the sake of getting something out, but one thing that we did, which I think was necessary, was at times we would realize certain problems we had to solve were too difficult to do it on the timeline that we wanted. And so one of the techniques that we used was to create these guardrails where we might say, we will not answer these types of questions because we don't feel we can do them in a high quality fashion, and that the ability to wall off certain types of problems and then proceed with the main idea, I think, has helped us advance much more rapidly than we otherwise would have if you had to solve each problem as it arose.

Eisenberg: That's a great point. And in the background, the technology and the market have both evolved so quickly that I think that approach has actually worked very well, because it may be by the time we pick that problem up again, the technology is better or the market's more ready and we can move much faster. So I think that's a really important point you're making. We're focusing on celebrating the team. Alan, I wonder if you would reflect on something Diane has done during this project that has impressed you just because, you know, we've all brought so much effort and so much heart to this work.

Ehrlich: Well, the thing that has impressed me the most about Diane is her fantastic organizational skills. I myself view myself as sort of an ideas person, and I get big picture stuff I think pretty good, but I can get bogged down in a lot of details and having someone who's very well organized, who understands there's a lot of work we need to do in terms of, for instance, testing the output, addressing problems. Diane mentioned the root cause analysis. She's been extremely active in making sure that happens in an organized and efficient fashion. Her organizational skills have been absolutely essential for us to move forward in a way that makes sure we're not leaving anything behind in terms of issues that need to be addressed. Even if we're going to say, okay, we'll deal with this later, it's clearly demarcated and then she makes sure that we as a group circle back to it and everything does get addressed. And deciding what are the things that we can't move forward until we tackle, and what are the things that are, you know, nice to have but not must haves. She's been really helpful in making sure we all have clarity on those types of issues.

Eisenberg: Yeah. Agreed. Thank you. I think in terms of, you know, moving that work forward, I think we've all relied on Diane's organizational skills and ability to draw those distinctions. Diane, same question to you. What's something about Alan that has impressed you during this work?

Hanson: I just love this question. So thanks for putting that out there. I love it when we can appreciate each other, and we don't often take time out to really acknowledge what people bring as far as skill sets. So Alan just digs in. He gives his all to the things he's been asked to show up to do. And he's very collaborative. One of the things I really appreciate about Alan, he may have a different opinion than you, but he honors your opinion and he wants to hear it. And he acknowledges that if he's building on your opinion, he acknowledges you as an individual. He also does his homework to really understand the issue and he’s not afraid to bring his voice. And there's been so many times through this process where we've needed to have good, strong voices at the table. And so if you have somebody who's sitting there and we're doing really important work and they won't bring their voice, they're just there to represent a team or a discipline and they don't speak, then it really doesn't help us move forward. So Alan always speaks. He always brings his opinion in a way it can be heard. It's highly credible. He always explains the why behind his opinion. He just doesn't have an opinion without something to back it up. And that's what I mean, he goes and he does his homework and he comes prepared. So I just always appreciate that. And I think that's just highly respected in any work team. So thanks, Alan.

Ehrlich: Thanks very much for those very kind words. And thanks, Kate. Yes, it was nice for you to ask us to say nice things about each other. It’s a wonderful thing. We don't do that enough as Diane said.

Eisenberg: We don’t, and we, I mean, we work together so closely every day for such a long time, I think. And we are all bringing different things. And as a clinical team, I think we've really been quite cohesive about what the needs are and what we think is important. And, you know, I really have learned so much from you both. It is fun to take a moment to appreciate that.

Hanson: And Kate, you've done a, you've done a great job too, Kate. You've been out there in the industry, leading with our work with CHAI and putting our governance structures in place and having that bigger picture and kind of keeping us focused on moving forward in the right way and bringing that voice as well, when we've needed to hear key messages, whether it's from the market and something shifted or, you know, just something that you're questioning, like you're very good at keeping what's most important out front. So I always appreciate your voice as well.

Eisenberg: Yeah. Thank you so much. All right. We have just a few minutes left, so I'm going to ask you to each think of one word you would use to describe this journey so far, and then we can wrap up. Alan, I'll go to you.

Ehrlich: Sure. The one word I would use for my experience in participating on this project is exhilarating. We started this at a time when AI was just breaking out into the news. I, for one, had no idea whether this was really gonna be something that could be applied in a medical setting or not. And to be part of the team from the get go, learning as we went, seeing that what we were doing was really cutting edge for our field. And we could see that when you and I went to the HIMSS conference and we presented and it was very clear there wasn't anybody who was further along than we were in this journey. And to me, that was really exciting to be part of something that was really setting the bar for what others would come to later. And this is also reflected in some of the work you've done with CHAI where a national body comes together and is looking for what kind of policies and procedures and exemplars can be put out there. And they're looking to us to show that. And so for me, that's been really, really exciting to be part of that and to see the results of the fruits of our labor.

Eisenberg: Thanks, Alan. Diane, how about you?

Hanson: Yeah, I think my one word would be intentional. I think coming into the project with a good intention for what we were trying to do and bring to the market and how we're intentionally trying to help point of care providers and then intentional design. We were very intentional about our design and how we brought things together, and how we tested and iterated on our collaborative work to to help with our Dyna AI tools. So intentionality and intentional design, I think, is really important when you're trying to bring something to help the health care system and the disciplines we work with.

Eisenberg: Absolutely. It can't be haphazard or reckless when you're working in health care. Thank you both so much. This was a fantastic conversation and we will wrap with that.

Ehrlich: Thanks for having us, Kate.

Hanson: Yeah, thanks. Appreciate it, this was fun.