Resources
Nov 19, 2021 |
1h 6min
|
3.4k views

Andy Choens | R in Epidemiology | Posit

Connecting Primary Care Providers to their own data
Presentation by Andy Choens

The Electronic Health Record, or EHR, is a digital replacement for traditional paper medical charts. And while today's EHRs are an effective replacement for digital records and have increased clinical data sharing, they do not provide the level of comprehensive clinical decision support envisioned by early proponents.

Acuitas is a consulting firm that works with providers to unlock the hidden value in their EHRs. From helping providers manage chronic diseases, succeeding in value-based payment plans, and responding to crises such as Covid 19, Acuitas helps providers succeed by connecting the data in the EHR to claims, Hixny (the local HIE), and other crucial data sources.

To help providers "connect the dots", Acuitas provides custom reports inside of and outside of the EHR. Some of these reports are patient-specific. Others are at the level of their practice or specific populations of interest such as patients with diabetes.

Acuitas is able to do this by using an agile, dynamic process and a collection of largely open-source tools including R and Python.

image: thumbnail.jpg

Transcript#

This transcript was generated automatically and may contain errors.

Yeah, the topic that I'm going to speak to today is connecting providers to their patient data. And as I'm watching people, I've got the chat pulled up and I'm watching people saying, someone just said that they were from France. I saw Nigeria, Canada, my old hometown of Atlanta, Georgia.

I'm up here in upstate New York, but yes, I am a rambling wreck from Georgia Tech. Thank you for joining us today. Given the fact that, obviously healthcare is a highly, we all have, healthcare is the same everywhere in that we're all always treating people. We are trying to make people healthy. Yet the business of healthcare is highly localized.

And I was very much, when I was putting the slides together, I will confess I was sort of thinking of the concept of US national level thing. So I'm gonna try to be very conscientious as I'm going through some of these topics and I start throwing out terminology because epidemiology, healthcare, et cetera, is just rife with terminology. So I'm gonna try to not drown everyone in lingo.

Also when I just have to, because it's healthcare and we need more acronyms, I will try to make sure that I am backing up and kind of trying to at least give where that acronym comes from.

I wanna also start with connecting providers to their patient data. So what am I talking about there and who rather? Specifically, I'm talking about in today's discussion, ambulatory providers. A lot of times when we start talking about healthcare, a lot of the papers you'll read, et cetera, they're about hospitals. Providers, a doctor in a hospital doing a surgery, providing us a care, providing an intervention.

Well, the world that I live in is actually the other side of the coin. It's when the patients go home. I work with providers all over upstate New York that are providing care in the ambulatory setting. It's a mix of both primary care and specialist care. But the point of it is that all the providers that I work with are what we call ambulatory providers, which means you can ambulate in, you talk to the doctor and you ambulate out. This is not inpatient. This is not emergency department care.

This is going in for your annual physical, trying to maintain and deal with chronic diseases like chronic heart failure, what have you. This is not the stuff that we always see on the news in the evening. You know, you watch your local news, it's like, oh, the emergency department is reporting this or that. That's a little bit different than what we're talking about here today.

About Acuitas Health

All right, so with that, what is Acuitous Health? The terminology we like to use is that we are a population health services company. We help providers make that transition to, in America, we are making the transition from a world of fee-for-service. So you, patient walks in, patient is given some care, it is charged, health insurance pays for it. The American healthcare system is moving away from this fee-for-service model into this value-based payment model, which is more about trying to codify and document the conditions that people are dealing with and provide a mechanism to empower providers to really treat people holistically and to try to make sure that we're actually dealing with all of the needs that patients have.

This is a sea change in how healthcare is provided because most medical office models are based around the idea of, have patients come in, we will do something for them. We give them an X-ray, we do whatever. The patient walks out, we get paid. Value-based payment is a completely different world. And so what Acuitous does, and I've got a picture up on the screen of prior to COVID-19, the office that we occupied. Hello, River Street in Troy, New York. And that is the Rice Building in Troy, New York. It's a lovely building that goes back to the 19th century, which I have not been inside of for more than a few minutes at a time and longer than I care to admit.

Across the bottom there, so who are we more broadly? We are this independent population health services company. We work with and are owned by CDPHP, which is a regional healthcare provider here in upstate New York. And one of our service vendors that we work with very closely that I need to make sure I acknowledge their role in what we do is Health Catalyst. So those are the three logos that you see across the bottom because what we are doing, the position that Acuitous sort of occupies is trying to get everyone to work together. And I'll talk about that more in just a moment.

Meet the team and technology stack

But before I do, okay, so who am I? I like to joke that I am the pirate captain. I lead the data science team here at Acuitous Health. I personally have a background in program evaluation and program improvement. I've worked in areas such as ambulatory care, child welfare, elder care, and other human services domains. Rumor has it, although this is just a rumor and remains to be seen, but rumor has it I am a fan of the 18th letter of the English alphabet.

And yes, I did actually choose this picture very, very, very consciously. So on Slack and in Zoom, I use a picture of the pirate captain. If you are fans of British humor, you know this show. If you don't know this show, you need to know this show. Go watch it, it's hysterical. Charles Darwin has never been so much fun.

But the pirate captain only succeeds. He only survives literally to the end of the show because of his crew. So the reason I chose this picture is because, sure, I'm the one you're talking today, but this isn't really my work. The thing that I wanna make sure that I'm really transparent about is that this is the work of a large group of professionals that I have the pleasure of working with. And they have come together to help us all create these really interesting ways of trying to connect providers to their data across EHR, claims, HIE systems, all of these different data sources that tend to live in their own silos and their own bubbles. It is this team, not me, that created this. And it is this team that is going to be able to make this carry forward into the future.

this is the work of a large group of professionals that I have the pleasure of working with. It is this team, not me, that created this.

All right, so technologies, like I said, I did wanna make sure, I throw their logo in here a couple of times that we do work with a company called Health Catalyst that helps provide some of our backend services. In terms of technologies used, we use a lot of R, we use a lot of Python, we use a little bit of JavaScript, and don't tell anybody, we still have some Gleck.

But all jokes aside, we also do, we are customers with RStudio and we use RStudio Connect. And we also use, if anybody really wants to know all the dirty laundry in the backend, we use Microsoft MS SQL Server for our backend database driver. So those are sort of the suite of technologies that we use. Oh, I didn't put the logo up here, but we also have what we call the report server, which runs Docker instances. So those are sort of our suite of technologies.

When we started Acuitous, we were very purposeful about trying to not have a single technology or not have a single language that we would try to implement everything in. We wanted to recognize that data science is a rapidly growing and changing field. And that the reality is that we should be trying to use the best tool for the job. And while people are certainly welcome to argue about whether or not we have done that in all instances, these were the best choices for us at the time.

And we have been trying to make sure that we are empowering the people on our team. Remember I said, it's really about that team. So we were trying to make sure that we were empowering the people on that team to use the technologies that best fit that situation and their needs at that moment. So that's why we have tried to make sure that we don't do, I've worked in other places where everything has to be done in SPSS or everything has to be done in SAS or everything has to be done in R. And we have taken the opposite approach where we say, let's just try to figure out what the best tool for the job is and then use that tool. So there's some limits on that. I don't think we have anything implemented in Erlang yet, but give us time, give us time.

About Acuitas: facts and context

All right, so some fun facts about Acuitous. Acuitous, hence the logo, we live in this sometimes uncomfortable space between insurance companies, AKA payers and your doctor, the provider. I tend to use the terminology of payer and provider, but when I say payer, I mean medical insurance companies. And when I say provider, I mean your doctor or the practice sort of more generally. It is our job to help both succeed.

We are currently importing, just for some context in terms of what we are dealing with, we import EHR data from 14 different vendors. And we do so by a variety of means, including ODBC. If you are familiar how EHRs are implemented here in the US, the Consolidated Clinical Document or CCDA, and just good old fashioned CSV. We are actually supporting over 200 practices in the Capital District and upstate New York region. And we work with 11 different payer groups. And we do all that as a company that is going to turn five next June. So we are a very young company. We move very fast.

We break things from time to time, but what we are doing is trying to provide a highly fluid group of services to a very diverse group of providers to help them, like I said, succeed and thrive in this value-based payment regime. And so we live in the middle, which is sometimes uncomfortable because it's difficult living in the middle between two different groups that actually have a lot of the same common interest. They don't always see themselves as having all of the same common interest. So we live in the middle, helping everyone understand how actually providers and payers really do have a lot more overlap than they have in terms of differences of interest.

Provider tools and data sources

Some context around the provider tools that we're going to talk about. Let's think about, so I started this saying that we were going to connect providers to their data, and that is what we do. So let's think about what those data sources really are. You have EHRs, which are these marvelously complex relational data sources that have all of the patient history, the amount of data that is, and by the way, these screenshots, no, I pulled these down from the vendor's websites. So if there's PHI in this, not my fault.

These are not screenshots from our clients. But an EHR is this incredibly complex thing that is, it's basically a filing cabinet for everybody's healthcare data. And then you have the practice management system, which deals with things like billing and upcoming appointments, helping, you know, you call them up and say, hi, I'd like to see my doctor. And they say, okay, so we have an appointment available at next Tuesday at two o'clock, can you do that? That's what that practice management system is. In most practices, these two systems live independently of one another. They're connected, there's, you know, obviously the practice management system can tell the EHR there's an appointment, so there has to be connectivity.

Yet at the same time, they are actually separate systems. And then other groups of data that exist. Here in the United States, we have what we call health information exchanges. A health information exchange I work with most often is called HICSNY. And a health information exchange sends out what we call an HL7. I know so much lingo, I hit healthcare sometimes. So an HL7 message is just a fancy way of saying XML. And it is an XML message that has some information about an event or a patient that gets sent out over yield interwebs. And so that information goes out and usually gets dumped into the EHR and notifies the provider of some event or some thing. Again, all that's going into the EHR.

And then of course, the other source of data that providers are having to deal with increasingly are claims. So a patient, you know, the practice management system, the EHR know what's going on here. But there's this bigger picture of when you start trying to talk about a holistic picture of what someone is medically, there's this larger picture of what are other providers doing when there are, you know, DAA signed and all this legal stuff gets taken care of. How can we leverage the incredible power of claims data, the incredible power of these health information exchange systems? And all of the data providers have historically collected in their EHR and practice management system.

And the reality is that that big holistic picture is what I'm referring today to as population health. And there isn't really a stock tool. You can't go buy a system for this. And in fact, I'm not even sure at this point you can because the needs are so regional that are so dependent upon what that practice is doing at that time. Here in the United States, the Department of Health and Human Services comes out with a new medical model roughly every three minutes or so that providers may or may not decide to get involved with. So the complexity of healthcare just within the United States, let alone the rest of the world, but just within the United States is mind bogglingly complex. So that's why companies like Aqeulus have been starting up is to help support providers in all of this.

Population health tools

So now, of course, the obvious question is, okay, so you say that you're supporting people with this. So what does that mean? So I'm gonna take you through what I'm gonna refer to as some population health tools. These are some tools that we have provided for our clients. Now, I've already said that this is obviously very tricky because there's all these different data sources. They don't necessarily wanna talk to one another. So effectively, what I've introduced you to is the Gordian Knot of modern medicine. So we're gonna be a little bit Alexander the Great here, and we are going to cut that Gordian Knot, and there's only one way that I know of to do this, and that's by getting everybody at the table and getting them involved.

So when I show you these tools, these are not things that were made in the back office somewhere, and some people, the data science team got together, and they're like, oh, this is a cool idea. That's not how we do things. We are not providing a product like Microsoft where we work on it for a couple of years, and then shove it out, and we put it in the store, and we ask people to buy it. We have contracts with providers. They need services from us. What happens in terms of how we actually get to these tools is the providers or the payers come to us and say, hey, there's this need. We need to be able to communicate around this.

We need to be able to help people navigate this specific space. Maybe we need to help with transitions of care. So when people leave the inpatient setting, how do we make sure that that works? How do we make sure that the primary care provider is following up with that patient?

And just if you think about that, that transition of care, you have a patient leaving the hospital, and they're gonna go home. And it would be ideal for their primary care provider to follow up and provide contact, you know, check in with them. Do we need to talk about your medications? How are you feeling? Do you need to come in and see me? In fact, let me go ahead and schedule that for you. That's what we're talking about around transition of care.

And it's really hard because what's happened traditionally is EHRs get this message, and it gets dumped into the EHR, and it may or may not generate an alert for the provider. And there's more people at an actual practice than just the doctors doing work. They have a support team around them.

Introducing the Hospital Activity Monitor

So I wanna introduce you to the first tool that's going to help us cut this Gordian knot is the Health Hospital Activity Monitor, or HAM.

Now, you will note that the picture that I have on the screen at this point is of a lump of ham, and not the tool, which gets us into this really funny thing about healthcare is I can't show you a lot of what we do directly because it's literally chock full of PHI. And I don't really want to violate those federal laws, so I have to be a little bit careful here. I do have some other screenshots, but I just wanna acknowledge the pink elephant in the room of we're doing things that we can't directly show.

So what is the Hospital Activity Monitor? HAM, as we call it, and you'll see that a lot of our, there's a joke with everything. We have to have our fun.

The HAM report is a daily email that goes out. It is actually provided to the sites as an Excel file. And in that Excel file are gonna be a whole bunch of tabs. And those tabs have things like, here's the list, doctor, you know, whoever. Here are the list of your patients who, within the last 24 hours, were admitted to the hospital. So here they are and what the admit reason is.

These are all patients that have consented to having their information exchanged over the HIE, the Health Information Exchange. They have consented to that. This doesn't just happen by magic. They have to consent.

If they do consent and they allow their information to be sent out over HCME to the provider, we pick it up and we give the doctors a report, say, hey, these are your patients that have been admitted inpatient in the last 24 hours. We also have a list of, here are the patients who were discharged from the hospital in the last 24 hours. So they've gone home. Somebody should probably follow up with them.

And again, there's context. So you can see in terms of, you know, what type of stuff we're providing. Some of it's really boring. The MRN, the name of the date of birth, it just helps people know who we're talking about. But if you go in and you look at some of the event information, well, where are they going? Where were they admitted to or discharged from?

So, you know, hospitals that we have up here in the Capital District would be places like Albany Med or St. Peter's. This is, you know, what I mean by an admit facility or discharge location. Why were they inpatient in the first place? What was that primary reason? What was the discharge date? If there is one, that's obviously only on the discharges. And for context, for the provider, again, driven by the HIE, how many recent ER visits have they had? How many recent inpatient visits have they had?

Because if someone who has an inpatient stay and you can look and go, wow, in the last year they've had 70 visits and now they're inpatient, that's a very different patient than someone who just went in, you know, overnight. You know, we have to try to gauge what is the severity of the patient's needs. So we try to provide a snapshot picture of what's going on.

The provider still has the EHR. If they have further questions, they can go diving through the incredibly rich dataset that is the EHR. This is a snapshot that allows the provider, a nurse, a care manager to make rapid decisions about which patients really need to be prioritized for follow-up upon that transition of care, which patients need to be followed up while they're actually in the hospital, what have you.

And all of this thing is about really providing patient connectivity between the ambulatory primary care provider and the patient when they're going in and out of the hospital. So that is, well, a whole lot of ham.

The Chronic Conditions Report

Another tool that we have, which we call the Chronic Conditions Report or CCR. And this is another Excel file that is generated and sent out. And both of these, so the, actually back up, the Hospital Activity Monitor is actually a Python script and the CCR is an R script. And these run on what we refer to as our report server, which is our Docker powered system for emailing out large numbers of Excel files that we developed internally.

The, this is again, an Excel file, like I said, and this is something that we send out every other week to a site. And so if you are familiar with an EHR, one of the things that you would know is that EHRs are, I kind of compare, oh, you can't see this because it's in front of me, but I have a filing cabinet in front of me. They are a patient filing cabinet. All of the information gets put into the right little, you know, folder inside the EHR for that patient. And it's a marvelously rich data source, but EHRs are not good at providing the 30,000 foot view of what's going on.

So if you take off in a plane and you're flying over, you know, wherever, and you look down, you have a completely different view of the countryside than you do if you are driving down the road. And I would say the EHR is like driving down the road and using, you know, Google to get you from point A to point B. And this chronic conditions report, this is what you see when you get to the plane, because this is full panel data.

EHRs have, obviously, full panel data. And when I say full panel data, what I mean is all of the patients at the site. So EHRs have that, but when you use an EHR, you are looking at just my data or just your data. You're not looking at that 30,000 foot view.

The CCR is that 30,000 foot view. So we break it down and we say, okay, so how many patients have diabetes at your practice? And of those who do, what was their last A1C value?

Because providers need a way of saying, hey, you know, I'd like to be able to do this new intervention. I've got this new technology or I've got this new care manager who's joining my practice. And we wanna really be able to target people who have an A1C over nine. Well, this CCR, what we like to say is this democratizes that data because without having to come to us or to come to anyone else, the provider now has the ability to say, all right, who are all my patients who have diabetes with an A1C over nine?

Well, this CCR, what we like to say is this democratizes that data because without having to come to us or to come to anyone else, the provider now has the ability to say, all right, who are all my patients who have diabetes with an A1C over nine?

They have the ability using this Excel file to literally make that list. And then they can hand that off to their care manager and the office say, please, let's go provide some services for these patients. It makes it rapid. It makes it so that they have full transparency into what's going on.

And you can see the entire list of things that we are providing in terms of the types of chronic conditions and labs that we target on this. And we're adding more slowly. It also provides a high level view of what are those conditions. You can see a risk assessment at this particular provider, at this particular 10, they do what we call provider risk assessment. Provider puts patients into risk levels of one, two, three, and four based on clinical criteria. Patients who are in level four are referred for care management.

And that CM active is, are they active in care management? Hicksony consenting. So we, you know, can we share their data over Hicksony? Do they have diabetes? This is how many. And an EHR isn't really good at this because that's not really what it was designed to do. And see, I can finally show a screenshot because that says a PHI. But this is the kind of data that an EHR is not well equipped to provide to a provider.

The pre-visit planning document

The next one here, and I've just got one or two more slides here before I wrap up, but this is what we call our pre-visit planning document. So Jane Doe is not a real patient. She is not a real person. So this is a dummy document, but it is emblematic of what our pre-visit planning tool is.

Again, like I was saying, an EHR is super complicated and the EHR provides all the detail, all of their A1C's values, all of their history. So we developed this pre-visit planning tool to help providers get ready to see you.

So hopefully in the day or two, the week, what have you, before your appointment, when you go in to see your provider, they sit down and they do a chart review. They go through and they think about who you are medically. They prepare themselves. And because of our role of trying to help providers succeed in value-based payment contracts, some of the space here is occupied by information that will help that provider know what things they should be doing.

So for example, down here, we have some lists of cervical cancer screenings and this, you know, in that case, there's something for them to do. But we have a list of things that these are coming, these measures are eCQM measures, for those of you in the United States. These are measures that the doctor should be considering about how am I going to fill on this patient? This particular patient is apparently very complicated, but this is a little over the top in terms of an example, but it gives you an idea of what type of information is on it, how we're trying to help providers identify opportunities to close gaps in care.

It goes down and has, you know, an ER and inpatient stay. So again, we're integrating data from the EHR, from HICSNY, from claims. All this is getting dumped into this one single document that your primary care provider can review and try to mentally get ready to provide you with the best possible care at that point of contact in the office. They can sit down and they can see the orders and they can say, oh, well, did you follow up on that? What have you.

Then if the provider has further questions, again, they still have that EHR. They can still go to that and they can open that up and go looking through and go, okay, so I see that your last A1C was really high. What's a 12? That's not good. But they can go into the EHR and say, okay, you had 112. Everything else was pretty good and you had 112. That's different than a patient that's got lots of 12s.

12 is a not good A1C. So they still may have to go to the EHR to really kind of put that full picture together, but we're providing a snapshot of information that really helps the provider get their head in the right space so that when they walk in to see you, they are ready, they know what's going on, and they have some context for where you are medically.

The last tool I've got I wanted to talk about just because this is an R in epidemiology thing. Kind of seems like we need a Shiny application shown off somewhere in this thing. So this is a Shiny app that we made, and it's also, I wanted to use this as an opportunity to highlight something that's not EHR-driven. A lot of our tools are, because we are connecting providers to their data, EHR-driven. Of course they are.

This appointment trends application is not. In fact, there is almost no EHR data in this thing whatsoever. Also fun, because that means I can do screenshots. But what this is really being driven by is by that practice management tool. You've got appointments coming up. You've got appointments showing up or not.

If you would all recall, back in March of 2020, the state of New York shut itself down.

So what this tool came about was over about a weekend after COVID-19 hit, and New York City and the New York State just a day or two later started shutting down. Providers started calling us and saying, I know that people are not going to show up for their appointments. Sorry, I didn't mean to do that. I know patients aren't going to show up for their appointments. I know that my numbers are decreasing of appointments, but I don't have a limitability to wrap my head around what that means. I don't know how bad.

We were able to very rapidly put together a tool. It started off as a flex dashboard that we emailed the people, and later we were able to put it together as a shiny application that now lives actually in our network.

And that purple line is 2020. And you can see the fall off the crater there is right there as we go into March of 2020 when COVID-19 causes New York State to shut down hard to deal with the epidemic. And the green line up there is what it should look like from 2019. So a little bit of a difference.

This tool also was able to be adapted to help providers understand, okay, I know my in-person numbers are crashing. That's what that screenshot is. But they were also implementing remote care. All of a sudden, kind of wild, you could go to your doctor on Zoom and have an entire medical visit where you didn't even have to be in the same room with your provider. And all of a sudden, we shoved medicine into the 21st century.

And the practice sites were working really hard to implement these technologies, and they were honestly doing a really good job. What they didn't know was were they plugging the hole? They said, I know my in-person numbers are down. Am I plugging that hole with these remote visits? And so this tool actually has the ability to let us go in and track. And of course, it varies by the site, and it varies by the type of care provided. But we were able to start helping providers understand, yes, your in-person visits are way, way down. This is to the extent that you're able to make that up using remote technologies, remote visits, et cetera, doing phone calls, what have you, to really try to make up that hole, make up that gap, plug that hole.

This was a tool that came about, I wanna say the initial version of this, we put together literally in one weekend. And we were able to get this out. And by the end of the year, it is now a standing application that some of our practice sites continue to use here in 2021 to understand what their numbers are looking like trending over time, because simply put, appointments mean revenue.

because simply put, appointments mean revenue.

So while one blip depends, how much revenue depends on what the appointment is, your appointments are going up, revenue will eventually follow.

So just one more application that we were able to do to help them see that 30,000 foot view of what their practice looked like.

All right, so that was my presentation.

Thank you so much for everything. I put a link up there. I made the slides that I just went through publicly available there on my GitLab page. And there you go for everyone who needed it, your obligatory R code junk.

Thank you so much, Andy. I really appreciate your energy as well.

I am the pirate cat. You definitely are.

While I go over to the Slido for some of the questions, do you mind copying and pasting the slides into the chat or just that link there? Yeah.

And just a reminder, if anybody joined a little bit later, we do have a Slido link where you can ask questions to or upvote other questions. But Andy, one of the most upvoted questions so far is, are there R packages that help with EHR data wrangling?

EHR data wrangling and tools

Good question. So that is something that I skimmed over when I went through the discussion around technologies. There it is. I skimmed over that. So what we actually do, I did say that the technology provider that we use to help us do what we do is Health Catalyst. What we actually wind up doing is Health Catalyst provides a lot of R backend services in terms of ETL. And they have a set of tools that they have developed, which we use to actually help us manage all of that complexity.

And so what we actually do, we describe our data warehouse as a layered cake. And that first level of, well, we technically do ELT, but we extract load and then later we do the transformations. We use the Health Catalyst tools to actually do that initial export out of the EHRs into the data warehouse. So no, we are not using any R packages for that. There are really good R packages to help you with diagnosis groupings and things like that that are useful, but they're very well made B tools but we are not using those at this point.

Handling large data sets in R

The next one, Andy, is about large data. How do you deal with large data sets greater than a hundred gigabytes using R? I tend to use SAS to do data wrangling since R is limited by RAM size. Is there a way to solve this in R?

So I guess the first question that I would address with that is, so if you have very, very large data, a hundred gigabytes is pretty big. What are you trying to do with that? So the solution that I would suggest if you are doing some kind of a machine learning model would be different than if you are trying to do some sort of summarization.

Because if what you are trying to do is say, so practice management data is something that tends to scale up into the OMG scale because just think about how many appointments a provider has in a day. I'm talking to a provider like one doctor and there's often more than one doctor at a site and we're working with over 200 sites. So our practice management data is large, it's deep.

I don't usually need to do a machine learning model off of the full cohort of that data. So what we actually do and what I would recommend if this is a relevant opportunity for you is to use a database server. Because yes, R is memory bound. And, but if what I really need, for example, going back to this, our appointment trend is going up or down. That calculation of how many appointments are happening in a day, that is not actually being calculated in R. I'm actually offloading that through dbplyr onto the database server and it's calculating the number of appointments per day.

R is only in this, in the instance of this particular application, it's having to graph a number of, you know, about 50K. It's an integer and this is the monthly view. So there's only, you know, there's two lines, there's only 12 months in a year. So at most 24, 48, you know, data points. So hundreds and hundreds of thousands of appointments, but I can slim it down to 48 data points and R is really, you can graph that on a pocket calculator.

and R is really, you can graph that on a pocket calculator.

So if you can use a database server, I would recommend that. If you can't, there are using virtual machines. Other jobs that I've done where I needed to do machine learning models on very, very large data sets, I used virtual machines where I could actually have an arbitrary amount of RAM, but there you have to have the budget to pay for it.

Thanks, Andy. And I meant to say this before, but if we're getting to anyone's question and you want to jump in and add any more context or follow up, feel free to unmute yourselves. It should let you do so. And if anyone has issues with that, let me know.

But Abdullah, I know you had put your name there and that was your question. So I just wanted to see if you had any follow up too.

Can I add something? Sure, Manju. Thank you. Hey, Andy, just wondering about this EHR data, is that accessible to anybody or you have to go through Health Catalyst for something like that?

Accessing EHR data and privacy

So the EHR data, to help me answer your question, where are you on the globe? So I can tell you what, let me just answer this. I'm in Chicago. Okay, you're in Chicago, okay. So you're in the same healthcare regime as I am in terms of federal laws and such. So in order to get access to the EHR data, what we actually wind up doing, the very first step that we have to go through, and now I'm like, gosh, which some of the folks on the team that are doing this day to day were here to help me with this. But the very first step is what we have, what you would have to do is get a DAA. You actually have to get permission to access that data.

So obviously in America, we have what we call PHI laws, which very tightly control who can have access to private health data. And those laws are very important. And if you violate them, there are very significant consequences. May I suggest you do not violate those? So the very first step is to get a DAA with the practice. And then once we have that DAA with the practice and the doctor has to say, Acuitous Health, I am licensing Acuitous Health to be a third party vendor and they're going to work with me to manage my EHR data. Then we reach out to their vendor, Allscripts, Cerner, whoever. And we start working with them and the details here kind of depend on who the vendor is and how that particular site has that tool set up. And we start working with them on an information exchange so that we can access that data.

So that's where we started getting into, it might be an ODBC connection, it might be a flat file, what have you. And then we are able to pull that down. Our data is, it is in the cloud, but it's in a PHI grade, the expensive grade of all VMs, because no, we can't just let people access that data because obviously it's private. We have information that people have the right to have privately and keep private. So that is data that, hence why I had to be sort of careful and make silly CCR jokes and ham jokes throughout this presentation, because the actual spreadsheet of the hospital activity monitor, I didn't want, if I had put it up here, it would have been two thirds, obfuscated out, I would have had to have like made it really hard to read so it wouldn't have been much fun to look at. So I felt like it was better to approach this, but no, we have to be very careful around healthcare and data exchange within this world, which is good, but it also makes everything complicated. Did I answer your question?

Yes, you did. Thank you so much, Andy. I hope I wasn't rambling. I wasn't mute. Thanks, Andy.

Jocelyn, I see you had just asked a question in the Zoom chat and I wanted to pass the mic over to you if that's okay to ask that live.

Tracking patient medical history across systems

Yes, thanks, Rachel. So my question is how do you track a patient's medical history when you look at their service chart? Because during the years, a patient could change their plans and have multiple patient ID or insurance plans ID. So how do you track their medical history?

That is a really good question, actually. So inside our platform, we actually use an MDM tool called MultiView. And what MultiView does, and no tool of this nature is perfect. We have tuned ours to a place that we're very comfortable with, but is it perfect? Nope. Will it ever be? Nope. But what this does is it allows us to connect. So I mentioned that we work with, I believe, we work with 11 different payer groups, multiple different provider groups.

And so you mentioned the complexity of, I went from Blue Shield to CDPHP insurance. For example, at some point in the past, I used to have Blue Shield. Right now I have CDPHP. I used to have Blue Shield. Right now I have CDPHP insurance. So I have a new provider. I have a new payer ID on that. Some number with Blue Shield, that was a record number with them. And I have a new record number with CDPHP, which is obviously not the same. And yes, we, up to a point, are able to track people across those because we use this MultiView that'll go in and look at first name, last name, date of birth, gender, and a whole bunch of other criteria to try to guess that Andy in this data set is the same Andy as in this data set.

We also have to be very careful about that because there are very complex rules around when we can combine that data and share it and when we can't. Again, because we have to be very conscientious around privacy. So anytime we start doing things that, any process that starts to walk across multiple data sets, we have to stop, take a deep breath, and really, really, really think it through from a legal and privacy standpoint to make sure that we're not doing something inappropriate.

But yeah, we use a tool called MultiView to help us say, the other part that can be fun, that can be complicated, is you as an individual patient over the course of your life will have multiple providers. Just imagine as a simple example, a kid who goes from having a pediatrician to eventually a family medicine practitioner. And that could be in multiple EHRs. So we're also able to, on behalf of the payer groups that we work with, map that progression across different EHR systems so that we can say, hey, this is how that particular healthcare for this patient who has had your insurance for all those years has been developing.

But again, that's something that both has both technical and legal and privacy aspects to it.

Thank you very much.

Other Shiny apps and internal tooling

Awesome. I'll take us back to the Slack for the next, or I'm sorry, the Slido for the next one. I'm gonna ask one that I'm particularly interested in the answer of, Andy, and it's, are there other quick tools you've had to make in Shiny like the Appointment Trends app?

Oh, gosh, yes. We generate Shiny apps weekly.

Weekly? So things that we are doing in Shiny right now, on a technical side. So I mentioned that we use that Health Catalyst platform and they have a number of tools that go through and do that ETL and all those jobs. We actually have been starting to develop our own Shiny applications to help us manage the data warehouse so that we can have better insight into what's running, how long do those things run? We have a series of Shiny dashboards that we built that go beyond what the Health Catalyst platform provides out of the box to help us meet our needs.

Related to that, not actually Shiny, but it's kind of fun internally. We also use Slack probably too much. And so if you happen to work someplace that use Slack, you understand the emoji hell that we've all been living in in the last 18 months. But the other thing that we do is that we have our markdown reports. And for example, some of those are about our ETL jobs. We actually have channels for our dev and prod environments. And one of the members of our team who's absolutely phenomenal, she wrote an R Markdown script that runs every morning and it puts out a list of, these are your jobs that failed last night. You need a human to go look at them. And that is something that gets then dumped into, nobody actually looks at it on, the R Markdown generator is actually ignored. What we use is the side effect of it that gets shoved into Slack. And we're able to read that. It's in our communications platform and it's from an R script that's running on RStudio Connect, but it gets dumped into Slack, which is where we live moment to moment. And it says, hey, these ETL jobs failed and a human needs to go figure out why and fix it. And every morning, what we call the IO team looks at that and starts figuring out how to make our ETL spend better.

And then in terms of the providers, we have a number and I couldn't go through them all in part. I was told to keep it to about half an hour. And we have so many shiny apps. We have things that are helping people with financial analytics. We have things that are helping folks with care management. We have a whole bunch of different things that we have been doing that kind of try to walk that gamut of data sources that we deal with. So yeah, shiny is a very important part of what we do as our static R Markdown files.

That's amazing. Thanks, Sandy.

Toyin, I know you had a few great questions that you had copied over to the Slido and just wanted to pass the mic over to you to have you ask those, if that's okay.

Certainly. Thanks, Rachel. Yeah, one question that came to my mind when you were speaking about, I think it was the CCR and identifying special populations as whether or not you ever incorporate data from remote patient monitoring devices. Is that something that any of the sites that you support use?

Not yet, but it is something that I have been having multiple conversations with. A couple of the provider sites that we work with

that are underneath a single 10, their CIO and I have been having conversations about that. And somewhere here, just to see if I can find it.

I actually got, I don't remember the brand that they're talking about implementing, but I actually bought some of the devices so I could start playing with the API. Because yeah, we are trying to figure out how to do that and start actually doing that.

And that is something that is in the next, oh, two to five years, I think is going to be extremely exciting in this ambulatory space. I mean, everything, you know, this is a Garmin. A lot of people have Apple watches, just this. Like, the other day I was having a really, really awful day and the watch kept telling me that my blood pressure was up, which the end result of that was I threw the watch across the room and told it to shut up.

But I didn't need an RAPI to tell me that I was stressed, but being able to track that, like being able to track heart rate and blood pressure data and be able to start integrating that into EHRs and be able to connect all those pieces.

The other thing that is personally very interesting to me is CGM data. So continuous glucose monitoring data for diabetes. My wife is a type one diabetic. And so she is a user, a patient, using a continuous glucose monitor. And to be able to integrate that more holistically with her provider's EHR is something that I think is really, really interesting. And if I manage to help be part of providing that, that'll be very, very cool.

But I would love to be able to provide that because the doctor's gonna run an A1C and say, oh, so Mrs. Cohen, it looks like blah, blah, blah. But to be able to then, and right now he has to go to a completely separate system. He has to log into a separate system and he can't put those two pieces together. So yeah, I think to be able to connect those is where ambulatory care has to get too quickly to really take it to the next level of patient care.

to be able to connect those is where ambulatory care has to get too quickly to really take it to the next level of patient care.

Sadly, I'm not doing that yet.

Outbreak monitoring and COVID follow-up

Thanks, Andy. Another anonymous question was, are you monitoring for outbreaks like respiratory viruses or micros that could spread easily in a hospital setting? And what would be your latency on that?

So we are not doing that because like I was saying earlier, what we are really focused on is ambulatory care. So that would really be more of an emergency department. They would have that kind of data. Eventually through the HIE, we would get that, but no, that would be something we would tend to just rely on the local emergency departments to use, to actually track because our providers aren't really doing that kind of testing.

And the only way that we would even get that kind of outbreak data would be via the HIE because ambulatory providers aren't doing just routine testing or that kind of stuff. We are doing some stuff to help the providers manage patients with COVID. So patients who have had COVID and have tested positive, especially in that transition of care circumstance where they're being discharged from an inpatient state due to COVID, we are highlighting that for providers based off HIE messaging so that they know and can provide appropriate transition of care and follow-up care for those patients.

And then we're also monitoring the transition of care and follow-up care for those patients. But that type of community outbreak monitoring is pretty, it's out of scope for what we really do.

Practice datasets and open source EHRs

Thank you. Nicole, I'll turn it over to you for some of the other questions, especially because you have the HIE background and will say things much better than me. I don't know about that, Rachel, but sure.

Andy, there's a little bit of a conversation that's been spanning the chat about practice datasets and how to get your hands on a dataset where you can play around with R and do some epi-related work, but not have to access PHI. Do you know any datasets or any sources that people could use?

So we're timing on that one. So in another part of my life, I'm an adjunct professor at the Albany College of Pharmacy and Health Sciences. And I have been wanting to answer that for my students because what I've been wanting to do in the spring semester is actually provide them with an EHR that's all fake data. So, Nicole, I think you guys had said that you were gonna put out a Slack channel. Yes.

I am blanking on the name of the EHR, but I will actually put out something on, I'll put something into that Slack channel because I found an open source EHR that you can spin up. And one of the cool things that they actually have is they have a SQL script that would, in a MeSQL database would actually allow you to generate it's anonymized data so that you can actually have your own private EHR to kind of play with and experiment with. And it would be running in a database server that you would control.

So that, yeah, that's actually something that I'm actively pursuing. So once I can tell that story, I'll try to work on that here in the next couple of nights and I'll try to put like out some instructions for how to do that. Because yes, that is definitely possible.

Awesome, thanks, Andy.

Yeah. I just muted someone, but I think someone was trying to ask a question and the audio was going in and out. Was there a question that someone wanted to ask? I can go back to the Slido if not, but I'll give a second in case that person wants to come back and try again.

Robert, I think there might be a issue with your audio, but if you want to put it into the chat, we can ask it too. Or we can go to one other from the Slido first too. Yeah, I'll ask a Slido question and we'll see if that question comes through.

Feature engineering for diagnosis codes

What kind of features might your team engineer about a single diagnosis? For example, if someone had an ICD-10 code associated with diabetes in the last three years?

So if someone, what would we, hit me with the what would we do part again. What would- I got sucked down into details. The question went away, but it was- Sorry, I was marking it as answered as we went through. But it was what kind of features might your team engineer about a single diagnosis? For example, if someone had an ICD-10 code associated with diabetes in the last three years?

Okay. So what feature, if we're, by using the word feature, I'm assuming that we're talking about machine learning models at this point, which is awesome. The answer to that is going to be really dependent upon what the, obviously, what is the question that we're trying to answer?

Some things that we have done, for example, and diabetes is a feature in this model, is things like, what are the odds, or a machine learning model trying to predict the risk of readmission to an inpatient stay after discharge? So we have a readmission risk model, and chronic diseases play a part in that model because people with lots of chronic diseases are more likely to be very obviously sick, and therefore, more likely to readmit. So that would be an example of something where we would use something like a diabetes diagnosis, and even more importantly, things connected to their A1c, because that would help us understand if it was a diabetes-related inpatient stay or not. So things like that would be used in a model to help predict something like a readmission risk.

Other things that we have been starting to work on are trying to, for example, understand disease progression. So you, for example, the person asking the question started with a diagnosis of diabetes. If you have diabetes, then the question is gonna become, what types of neuropathies are you likely to develop? So what types of complications are you likely to develop? And when, how long is that progression going to look like?

The other side that we're a little bit more active on is actually diabetes becoming the output, the consequence, is trying to go, so here are patients with pre-diabetes. Of all those that we know who have pre-diabetes, who are the most at risk of becoming diabetics in the next three to five years? So it's, I know I'm flipping the question on its head, but that is this idea of disease progression being going up to diabetes and actually being formally diagnosed with diabetes instead of just pre-diabetes.

And then if you have diabetes, initially you would have, if you developed diabetes in that regard, you're a type two. And so you'd have uncomplicated type two diabetes. And eventually, as the disease progresses, those patients are gonna start developing, it can impact your kidneys and lead to chronic kidney disease, things like that. So there's opportunities in modeling around who is most at risk for things, for consequences of that with diabetes. And any chronic disease, be it, you know, asthma, what have you, can have, you know, side effects that are holistic and systemic.

Measuring success and Shiny adoption

Thanks, Andy. As we get to the top of the hour here, I know some people might have to jump. So I just wanna have Nicole give another plug for the Slack channel as well, because we can answer some of the unanswered ones there. Yeah, I'll join that Slack channel. So I'll be happy to continue the conversation.

Yeah, so I'm going to drop the link in the Zoom chat here. And I think we'll be able to share it after the fact as well, somehow. But any questions that didn't get answered or that you think about after the fact, feel free to throw those in the Slack channel. And Andy has told us that he will tell us more about the practice dataset question there too. So I'm dropping this link. Oops, not the Zoom link. I have to find the Slack link. I'm dropping the Slack link in the Zoom chat now. And we can continue any discussions over there.

Awesome, and if anybody wants to unmute themselves and ask a question live now, I also wanna give everyone the opportunity to do so. I could take unmuting yourself as raising your hand.

Hey, David. Hey, Rachel. Andy, a really interesting presentation. Thank you. So what's a little unclear to me is how do you measure how successful your activities or your products or shiny dashboards are?

Oh, good question. And that one is ultimately a per... I have a really simple, there's a cop-out answer I could say. Well, the people that asked us for it tell us they like it. That's a cop-out answer though.

A better answer is, so with, and this is actually connected to why we have been trying to convert more of our tools to shiny. So if I send out, here, I was earlier. Apparently I drew a little bit of my screen. I didn't even notice that. Sorry about that.

So we have that idea of that CCR report or the HAM report. Those are email reports. And you're actually right in that, or I suspect you're right, in that I don't know if someone opens up the CCR. Once it goes out as an email, assuming it doesn't get rejected, and we do track that so that we can follow up with like, hey, why is this email address not working anymore? We don't know if someone opens it.

So we are reliant on, so I represent the data science team at Acutis. We do have what we call a practice intelligence team that has constant communication with our sites. They have regular touch points with them and whatnot. They talk about the tools. So we get some feedback that way. But one of the things, and one of the big driving reasons for why we've been trying to go more and more in terms of using shiny is because there we can better actually say, hey, this is how many people actually used this. This is how many people were able to go in with that.

In other cases with things like following up on transitions of care, we've actually been able to do some very simple, starting to look at analyses where we start saying, can we bend that curve and start increasing the number of primary care visits that happen after a discharge? But a lot of it is going to be what I would call a process measure where we're trying to look at, you know, people utilizing the tools emotionally and whatnot. And shiny is way, way, way better for that than an emailed Excel file is.

That's really interesting because as somebody who goes to a doctor, I would have thought the answer might've been, there's like an improvement or at least not a degradation in, you know, some key lab tests, you know, for patients who have specific kinds of conditions.

And depending upon what the intervention is, that might be, like I said, we were doing a number of different things, but you know, if we are, for example, I know there are a couple of providers who are targeting diabetes. So there we would be able to use A1C as a proxy for any kind of an impact.

But in other cases, you know, because, and this is sort of the funky part about this, that CCR, one of the terms that I use is that the point of it is to democratize the data. And one of the, honestly, in a way, unfortunate side effects of that is I don't always know or have transparency into how providers are using it. So I don't know necessarily to look.

So a doctor can get that chronic conditions report and go, oh, I got this thing, I'm gonna target diabetes. I don't hear about that, but I don't even know to follow that up because in order to get that list of patients they want to go do, you know, work with, they don't have to ask me anymore. I've almost, in a way we've cut ourselves out of the loop. And so they can go do that thing and we don't even know to look. So like monitoring lab values across all lab values, across all sites, looking for signals is guaranteed to be, you know, lead you down the path of false signals.

the point of it is to democratize the data. And one of the, honestly, in a way, unfortunate side effects of that is I don't always know or have transparency into how providers are using it.

So yeah, in a way they would have to come to us and say, hey, you know, we'd like to work with you on a program, you know, improvement plan and evaluation to see if this is effective. And then absolutely what you're saying is true. If someone comes to us and says, I'd like to do this, can you help me track the outcome? Then we will start looking at appropriate lab values and might be proxies for improved patient health.

But they could also do something without even telling us.

Andrea, I see that you had unmuted us as well and I wanted to give you a chance to ask your question.

I can't hear you yet, but if you have an issue with the audio, feel free to use the chat too.

Nicole, do you see any other questions that we haven't really touched upon yet?

Can I ask a followup to the diabetes modeling question? Sure.

So there were two sort of potential applications you said, like, for example, readmission risk or the likelihood of complications given the fact that they have diabetes or some other ICD-10 code, fill in the blank. If you know that one of the most important predictors for that is some kind of metric that let's say comes from a blood panel or white blood cell counts or something along those lines, is there an implementation of this model and sort of like an iterative loop where the physician who sees this patient will know or have an action item to get the blood panel to best predict whether readmission is highly likely for a patient at a later date? You know, is there a feedback loop sort of to make sure patient outcomes improve?

Yeah, that is something that we could implement. That would most likely already be driven, unless we came up with some sort of a novel lab, you know, correlation, that's probably already going to be driven natively by the EHR, in which case we would just offload that to the EHR to drive, you know, getting that lab completion and all of that, getting those results into the EHR.

But no, I mean, what you're saying absolutely makes sense. One of the things, you used a word that is worth highlighting, which is feedback loop. And one of the things that we are very conscientious about doing anytime, if it's a tool or a model or anything else, is involving the provider. So not only would they, you know, a provider ask us for some sort of a model or something like readmission risk, but we would also have those people involved in the development of the model, because what we have found is not only does that increase the trust in the model, it also gets people to really start using it, because anytime you have a model, it's a model, it's not reality, it's not perfect. And having those providers involved from the very beginning, in terms of that feedback loop of talking about what this model is, helps them get the understanding of, well, when is the model right? When is the model likely a little bit off? You know, what are the weaknesses of that model? So having a feedback loop on any data science project back and forth between those domain experts and the people doing, you know, the data science and the technical side is critical.

So having a feedback loop on any data science project back and forth between those domain experts and the people doing, you know, the data science and the technical side is critical.

Normalizing R in SAS and SPSS environments

Thanks, Andy. I know we're about 10 minutes over, so I just want to be respectful of your time, especially if you have to run to another meeting too. Yeah, I do have a one that I need to get to here at some point.

Okay, so thank you so much. I know there's a few other unanswered questions, is it okay if we just ask you one last one, which was, I see this one, I've been upvoted. How do you suggest our users normalize our use when so many people are using SAS or SPSS?

Fun question. I have been accused of doing this. Our advocacy is something I have been accused of before.

So what I would actually say is, so previously I worked at the New York State Department of Health, and we had an organization within that, it was called EBCOP, which was Epidemiology and Biostatistics Community of Practice, you know, because there needed to be another acronym in healthcare. And within the bounds of EBCOP, what I was able to start doing was starting a conversation around both R and ultimately also aversion control, but trying to like start talking about how it could be used, ultimately having a new, a language that's a little bit different than say SAS, if you're in a SAS shop or an SPSS shop or what have you, having ultimately a champion who is in some form of a leadership role, you know, start identifying, I would say if you are in a leadership position, then you have the opportunity to start pushing that conversation. Otherwise it helps to have someone who is in a leadership position, who is open to the idea of, you know, maybe we can do some things that are different. Maybe we can do some things that are, you know, can broaden our horizons here in terms of the technologies that we're using and get that champion.

And we were able to use this idea of EBCOP as sort of that proxy for that champion. And we started getting people to talk about it. We started holding workshops on doing R and how to get people up to those skill levels and working with the Department of Health IT to get R installed on people's workstations so that they can actually start using R, which is how we were able to make some headway there in a place that has a lot of SAS users.

But yeah, I think having a process where you have both a champion and a process to start enabling people are both gonna be the two critical steps for that.

That's awesome. Thank you so much, Andy. I know you have to run to your next meeting, but really appreciate you sharing your insights with us and answering everyone's questions. I'm excited to see that Slack channel grow too and we can answer some of the unanswered ones there too. Yeah, thank you for the good questions. Really good questions.

Thank you all for joining as well. Have a great rest of the day.