February 9, 2021


Line Dempsey: Welcome to our podcast, Regulation Matters: a CLEAR conversation. Once again, I’m your host, Line Dempsey. I am currently the chief compliance officer with Riccobene Associates Family Dentistry here in North Carolina, and I'm also the Chair of the National Certified Investigator Training Committee with CLEAR.

As many of you may know, the Council on Licensure, Enforcement and Regulation, or CLEAR, is an association of individuals, agencies and organizations that comprise the international community of professional and occupational regulation. Our podcast is a chance for you to hear about the current topics in our regulatory community. And today, I'm joined by Dr. Elizabeth Carter. She is the Executive Director for the Virginia Board of Health Professions and the Director of the DHP Health Care Workforce Data Center. We're so glad to have you with us, Liz.

Elizabeth Carter: Hello. I'm glad to be here. Thank you.

Line: Yes. And thank you also for our listeners for joining us today. The topic of today's conversation is specifically Virginia’s Sanction Reference Points Program. But I think what's great about this is that it touches on a couple of areas that have seen a lot of discussion lately in some of our CLEAR events recently. You know, it's certainly an example of using data to inform policy. And it's based on ensuring fair treatment and reducing bias in regulation, certainly an issue that a lot of boards are looking at right now. So for our listeners that maybe don't know what the Sanction Reference Points Program is, could you give a brief overview of that for us?

Liz: OK, thank you. Bear with me; it's a bit of a mouthful. It began 20 years ago. But in essence, the Virginia Sanction Reference Points Research Program really is an application of research methods that were used to develop, update and maintain the criminal justice system sentencing guidelines. We basically borrowed heavily from them. So it is tailored for professional licensing in our disciplinary arena. We have separate sanction reference point manuals that have point factor based scoring sheets that have been prepared in consultation with each of the licensing boards.

And the SRP's are really designed to ensure fair treatment because they are based on factors that the respective board has determined should be taken into consideration,and it excludes all other factors. So at the heart of both, with the criminal justice system’s version and our own, is a desire to leverage empirical, objective, data-driven approach to help ensure that fair treatment.

They start basically by determining the factors that those decision makers and other experts consider to be the most relevant in making your sanctioning decisions. And then we obtain the data to gain the objective empirical information that will detail those factors that are actually associated with your past case decisions. So, you do statistical analyses. Researchers then share the results with the decision makers and ask them to determine if there are what we call extra-legal or unwarranted factors that should not continue to play a role. And once they've told us what shouldn't be there, we'll take what's left of the factors that should be there, and then we can develop operational definitions of those factors and scaling of point values for offense and respondent factors. And then the scaling is based on their statistical weights. And the point is basically they mirror severity. Their instructions for scoring and comparisons are then used against the sanctioning threshold and prepared in a manual with the scoring sheets reflecting some common categories for cases for that particular board. So like, standard of care may have a separate sheet, then inappropriate relationships, frauds and so forth.

And I can talk all day about what we did. But I do want to share with you, if you go to the Department of Health Professions website, you will see each of the boards’ manuals are there. And our main website is www.dhp.virginia.gov. Click on any of the licensing boards, and under their guidance documents, you will see a manual. So it's better to actually see a live manual. I think you can do that in your own time - when you're bored, middle of the night and wanna read something, you can go look at that, but that's available to you. Thank you.

Line: So, what was the impetus for the program? It seems like the most glaring thing would be consistency. And from that how, how are these sanction reference points developed?

Liz: Yes. Absolutely. Well, I think first I need to answer and say something about the Department of Health Professions and the Board of Health Professions. In Virginia,we have the Department of Health Professions as an umbrella agency. It houses 13 health professional boards, plus some other programs, including Health Care Workforce Data Center, and it also has a Board of Health Professions. And that Board has members from each of the licensing boards—they're all governorate appointees—and then we have five citizen members. But they have several powers and duties, one of which is their primary role is to conduct independent research and make recommendations to the Governor or Secretary or director of our Department, General Assembly and so forth, on matters that pertain to health professional regulation and agency performance.

And there's one little section in the code that really is the impetus for everything here. We are to also periodically review the investigatory, disciplinary and enforcement processes of the department and the individual boards to ensure the public protection and the fair and equitable treatment of health professionals. Now, I've been with the Department of Health Professions since 1990 in one capacity or another, have been a board exec for Optometry and Veterinary Medicine. But in 2001, I became the Executive Director for the Board, and I actually looked at my code—because we're supposed to do that—and I thought, ‘Oh, wow. How do you do that? How do you do all those things to ensure public protection, the fair and equitable treatment of health professions?’ The Board of Health Professions is advisory only. And I knew through the years, there are thousands of cases that come to us each year and hundreds of proceedings. Some of them are actually being held at the same time; again, we’re in that umbrella agency, so we've got lots of hearings that are going on. So how in the world can you—the staff at your hearing is one third of Board of Health Professions staff? How can you be at all of those things? How could, you know, determine where fairness is going?

I mean, so for years, actually since the 1970s, the Board of Health Professions has reviewed the disciplinary project, the number of cases that we had, and the outcomes of those cases in terms of violation/no violation. That's easy. We have a database that had all kinds of information that we just have, sort of standard, their case categories and all that. But when you really look at that second statement, it talks about fairness. So, what criteria do you use for fairness? I mean, you can kind of look to see if something has been appealed to the courts, but I knew there was something better out there.

And also, we're starting to get complaints from the agency, from the media. Different consumer groups, complainants themselves, are going to fuss if they weren't found in violation, and respondents may be upset about, you know, how they've been sanctioned. You hear complaints of something being too lenient or too harsh, and worst of all is inconsistent, like why did Mr. Jones get a fine of $50 where Ms. Smith has to take extra CE and a fine and, you know, all these other kinds of things that you would have in your sanctions. So, that's it; it's kind of hard to answer that question.

For us, the board and the board members are basically the ones who make the decisions about the findings of fact, conclusions of law and the sanctions, and then the orders are written based on their judgements. So, you know, it's not like we have hearing judges and that sort of thing or a little bit of case law or anything to go by. So it really is, you know, case by case. So it's not fair to them. I knew from being a board exec from optometry and veterinary medicine, what would often happen is, like, well, you know, they're very clear on what the violation was, very clear on your law and regulations and all that, but, you know, they'd look at staff in a closed session and go, ‘Well, what kind of sanctions should we impose?’ They look at us, and I'm going, ‘Oh, you can't do that.’ You know, you can’t ask us what we've done in the past. Well, there's a lot of things wrong with that— it's inherently biasing; your memories are going to be subjective and they provide no proof that the sanctions that are imposed relate to the type or severity of misconduct.

We just didn’t have case histories like you have in the criminal justice system. And on top of that, more and more attorneys are involved in a lot of our proceedings. And so, you know, they see when you go into closed session, that's ex parte communication that's going on. So they don't even have a chance to argue for the sanctions against us. So, all of that kind of came together. And the Board of Health Professions just said, ‘OK, we need to really start looking at this; this is what our law is.’

And so they asked us to develop a work plan. And I'm gonna read it; I have to do that. I know, I don't wanna be scripted and everything, but I'm gonna say this because they told us we had to ‘provide an empirical systematic analysis of board sanctions for offenses and to derive reference points for board members and an educational tool for respondents and the public.’ That's a tall order. And we said, ‘OK, let's see where we can find; where is this out there?’ And there's nothing in our regulatory literature that I could find similar. There are some boards that had some basic recommendations for certain kinds of fines or something like that for CE and those sorts of things. But nothing really systematized, like it was in the criminal justice system. So thank goodness we had them. And because we're actually doing research and we're not a university, we had to have the help of our AG's office because they sort of serve as our institutional review board. And they allowed us to go through and set these systems up. They gave us the go-ahead to do that, but not to use them in a formal hearing.

We also worked inside with our internal staff, the other Board execs, and they all recommended for any sanction system to be successful, it's gotta be really developed with complete board oversight. Each individual board had to be involved. It's got to be value neutral. It's got to be grounded in a sound data analysis, and it's got to be voluntary. So those are the parameters under what we have, and that's what kind of got us going.

We were very fortunate. Also, I think serendipity plays a lot in what we have been doing with this as well. We already had Visual Research, Inc. help us with some of our own statistical analysis when you had to get into complex multivariate analysis of whatever policy question would come along. That contractor actually would help us, so they already were available to us. And as fate would have it, one of the chief folks that are involved in that company had actually worked with the criminal justice system when they were developing the sentencing guidelines back in the 1980s. So, we were really lucky to say, ‘OK, let's try to work our way through that.’

Line: I guess, on a practical side of things, how do the licensing boards actually use this tool during a disciplinary process? And I think you mentioned before that you're not able to use it for formal hearing. Is that correct?

Liz: We're not able to use it for a formal hearing. I mean, it would be doable. It's very easy. Everything we do is so transparent. It was just out of an abundance of caution; it was something new, and so, I left it alone. It's worked well for us. About 5% of our cases end up at a formal hearing anyway, so, you know, the majority of our cases where you find probable cause (and that's about 20% of all the cases overall), you can certainly use the sanction references, and it's worked very well for us.

But yeah, the boards were engaged in this. We didn't just kind of go off and do our little statistical analyses alone. We met with the respective board, representatives from each of their boards, their staff, and we had some sort of a qualitative little survey that we went through saying, what factors do you think? Where should we start to look at this? Because we didn't have the literature to go by. We can't use the criminal justice literature; we had to look at what was going on our own world. And, you know, they gave us, basically, over 100 factors for us to start looking at. And it allowed us to have sort of a historical portrait for respective boards. We started with medicine first. They have a lot of cases. And, again, most of their cases were standard of care and those kinds of things, but they had a very complex array of different kinds of case categories and levels of severity to start out with. And so we're able to basically work with them to figure out and just do all the statistics and see what we come up with. And we were able to identify some factors that had some heavy weights in terms of if you could predict what the sanctioning would be based upon what the case involved.

And so we took all that back and then you meet with the Board. You do it during your whole board meeting. ‘Here's what we found, guys.’ And I said with Board of Medicine first, we found a lot of factors that they said ‘this is great.’ And the whole board looked at everything. But then we found some factors that were not things that they'd need to consider, unwarranted factors. And that's where we made our normative adjusted judgements. Those are considered those extra-legal factors, just like the justices dealt with. So things like the age of the practitioner; it mattered. You could hold all those other factors constant, and you were more likely to lose your license if you're an older practitioner. And nobody knew that, because they’re case by case, you don't know this stuff until you see it. We found out that if you were a woman, you were more likely to lose your license, and if you didn't have an attorney present during the preceding, you are more likely to lose your license, holding all those other variables constant. So of course, boards said, ‘Oh, please throw that out. No, we don't want those biasing factors.’ So we've excluded those. And we were able then to develop manuals that provide instructions and worksheets that they use, that use those point values of the remaining factors. And they're both offense and the respondents scores. And we do training to the boards and so forth. So that helps you to establish it.

And then basically, every month we go back to each board. All 13 of our boards now have their own sanction reference points systems, and they're based on their own factors. And I'll have to say this; just kinda funny. You can laugh if you like. You know, it matters a lot to dentistry, for example, the number of teeth involved; but psychology, they don't care about that. So, you really do have to tailor it to the individual board. Because these tools also need to evolve over time, we monitor it every month, so we look to see if there's agreement with what the recommended sanction is. And if there are reasons for departure, we ask the board to fill that out. It is voluntary; they're not forced to do what we recommended as the range of sanctions, but we ask them, ‘If you don't use that, tell us why not.’ You know, there may be ameliorating, there may be mitigating factors, and we will also monitor that. We feed that back to the board. So it's an ongoing, dynamic data-fed process.

Line: That's an interesting point. Obviously you guys have been doing this for a little while, and you mentioned, you know, kind of tweaking things along the way. I guess, what kind of benefits are you seeing from using this?

Liz: Oh, absolutely, the benefits are that we can train new board members. They are on par with everybody else, basically, whether you're an older practitioner or not. Some of our boards have agency subordinate panels, and what these manuals do is basically say, ‘these are the factors that the Board considers important in your sanctioning,’ so you know that upfront. We use this to train our investigators. So that actually helps them to know, ‘Hey, we need to know what happened on this particular day and the level of harm,’ which we have scales for. They can actually look at that and see what they need to be finding evidence of as opposed to just a lot of other factors that don't matter to the Board. So, that's been a huge help for them.

And it's basically just having a sense of it being in their control. Each board can do that if they need to change factors. There are new professions that come up and so will amend it then. So again, it's their tool; it's each board's own tool. The big benefit that they see is having that as something to rely upon. It's their own case history that they can have at their fingertips. They don't have to go through all these musty, old case law books like the judges do. They actually do have that for them, so that's the biggest thing.

The other thing about it, too, as I mentioned earlier, before, when the press was fussing like, what kind of sanction did this person get versus the other, what are you all doing? The boards had no way to defend themselves. If they use the sanction reference points, you can go ‘Here, look at this, member of the press,’ ‘Look at this, opposing attorney.’ It's transparency; it explains the why behind the sanctioning decision that was made.

We also have done a follow up on this as well to see after 10 years into it, we were doing it. We looked at consistency, as you were mentioning a little bit earlier. We wanted to make sure that the sanctions that were imposed were proportional to the offense and that everything was kind of neutral. So we basically got a way to defend; you have a way to ensure fairness, and they can explain themselves with this. So, it helps those new board members, old; the boards are no longer defenseless.

Also, these manuals are online. We do training for the bar; we do training for the public, anybody who's interested in it. You now know why a Board might have made a sanctioning decision that they did. So that's the biggest benefit that I see.

Line: Yeah, that's really good. I'm sure, you know, even that practitioner can take a look at those things to kind of understand kind of where they're headed when they've made mistakes along the way.

Liz: Well, let me interject real quick. That's true.

Line: Absolutely.

Liz: That's another benefit. We have found that attorneys will say, ‘if my client were guilty of this. . .’ and they'll look at it. And often what we found anecdotally is that they said, ‘Can I do a pre-hearing instead of us go into a proceeding?’ We’ll stipulate to this because they basically do know what the general sanction will be for their clients. So that has helped us.

Line: Right, that’s helpful. Well, you know, I mentioned earlier, and I certainly know anecdotally that a lot of boards are looking at using data to make informed decisions, and also reducing bias. Obviously that’s a big thing in regulation right now. But if other boards were interested in developing this sort of tool, are there some existing sources of data that most boards likely already have to help them get started in this?

Liz: Oh, absolutely. Again, I think it never hurts to find out from the wisdom of those that are practicing. Just ask them what their own thoughts are about what their sanctioning culture is like. What are those things that they should look at? But, you know, it'd be great if you already have your cases in a database that talks about the case category, for example, standard of care or CE, or whatever it was. If you’ve got that already there, that's great. If you have some measure of severity, you know, for cases where you had to do a summary suspension, for example. You know, that data should be in your system. If you don't have it on a computer, you do have all your orders, your minutes, the case files. The data are there; it just needs to be systematized.

And again, we're happy to share with everybody, you know, how we have been able to pull data from those other sources as well and use those to help us figure out what the outcomes of the cases are looking at those factors. So you've got it. It may be in written form, but you have it. So that's what I'd like to share.

Line: Perfect. Would a board or agency need to actually bring in some technical expertise if they're trying to go this route?

Liz: I would recommend it. It would not hurt if you did. You're certainly welcome (you can't just use our manuals because they are based on our own individual board's data) but the methodology, we're happy to share with anybody. You might need to be bringing in somebody who could do some statistical analyses for you, or actually even doing the data collection part if you have everything in paper. You might be able to reach out to your local universities. There may be consultants that are out there that can help you with it. But, you know, we're always a resource. I'm happy to help anybody kind of get going in this direction.

Having this information has really been a huge benefit to us. Just like I said, your boards can certainly defend themselves with this. Everybody kind of knows where the ballpark is when they're going to have their sanctions. And it does seem to cut down on all these requests to go forward with more proceedings. So, you know, I think it's worth the effort. And obviously, I'm very happy to help you with that, but I think you are going to need somebody to do the statistical analyses for you.

Line: Gotcha. So, any other resources that you would recommend?

Liz: I would. I know we’re in our own community of regulators for professions, but it might not hurt to reach out to your own state or province’s criminal justice research agencies; almost all of them have them. And again, since we borrowed heavily from them in the first place, they may be able to have some insight into your own state or province’s own sentencing guidelines. And I think that would be a big boon to set up that relationship with one another. Again, we're not always looking at criminal offenses, but that method just makes sense. And I can tell you, from looking at all kinds of resources out there now, the expectation of having data and letting that help you be objective in what you're doing—that's the future for anything where we're doing administrative or criminal law. So, I think setting up just a chat with them wouldn't hurt.

Line: Makes good sense. Well, I think this has been a great discussion. So, I want to thank you, Liz, for your time and being part of this CLEAR podcast. Any last words you'd like to share?

Liz: Absolutely. You know, collaboration is the key. You need to work with the licensing boards themselves. You're helping them to learn what their own data looks like. And, you know, it's just an effective tool to support fair and equitable and defensible sanctioning decisions. And it just makes it transparent for everybody. It's neutral; it’s consistent; it’s proportional. Of course, it's my baby and I'm very proud of it, and I think it's just the way to go. Thank you.

Line: Absolutely. Thank you. You know, it's always wonderful to learn about programs in differently regulated agencies and examples of how these things and these types of issues can be addressed. So I really do want to thank you for speaking with us today.

Liz: Well, thank you for having me.

Line: And also, we want to thank our listeners for tuning in and joining us today. And we'll be back with another episode of Regulation Matters: a CLEAR conversation very soon. If you're new to the CLEAR podcast, please subscribe to us. We're available on Podbean, Apple Podcasts, Google Podcasts, Stitcher, Spotify and TuneIn. And we're newly available on Alexa for Amazon Echo devices, iHeartRadio, Player FM, Listen Notes, and now Pandora. If you enjoyed this episode, please leave a rating in our comments in the app. Your reviews help us to improve our ranking and make it easier for new listeners to find us.

Feel free to visit our website at www.clearhq.org for additional resources, as well as a calendar of upcoming online programs and events. Finally, I want to thank our CLEAR staff, specifically Stephanie Thompson. She is our content coordinator and the editor for our program. Once again, I’m Line Dempsey, and I hope to be speaking to you again very soon.



The audio version of this podcast episode is available at https://podcast.clearhq.org/e/sanction_reference_points/



Home  |  About  |  Membership  |  Resources  |  CLEAR Learning  |  Awards  | Events
Join  |  Contact  |  Login  |  eNews Signup  |  Communities by CLEAR  |  Privacy Policy

Copyright © 2020 CLEAR
Please read the following disclaimer.  |  
Website by Nicasio LLC and CLEAR
108 Wind Haven Dr., Ste. A
Nicholasville, KY 40356
(859) 269-1289
Fax: (859) 231-1943
Powered by Wild Apricot Membership Software