Shine On, Data-Quality Influencers: A Date with Data in Idaho

A Date with Data

Reach out to us if you want to access Podcast resources, submit questions related to episodes, or share ideas for future topics. We’d love to hear from you!
You can contact us via the Podcast page on the IDC website at https://ideadata.org/.

### Episode Transcript ###
00:00:01.52 >> You're listening to "A Date with Data," with your host Amy Bitterman.

00:00:07.34 >> Hey, it's Amy, and I'm so excited to be hosting "A Date with Data." I'll be chatting with state and district special education staff who, just like you, are dealing with IDEA data every day.

00:00:19.50 >> "A Date with Data" is brought to by the IDEA Data Center.

00:00:24.49 >> On this episode of "A Date with Data," we are going to continue to explore what it means to be a data quality influencer and think about the different roles that each of us play that could have a significant impact on the data that we're collecting, analyzing, reporting and eventually using. I have with me Alisa Fewkes, Data and Reporting Coordinator, and Debi Smith, Special Populations Coordinator, who are both with the Idaho Department of Education. Welcome. Can you each share a little bit about yourself and tell us what you do?

00:00:56.62 >> Hi, I'm Alisa Fewkes, and I am the Part B Data Manager here for Idaho, so that means that I work with all of the EDFacts reports for special education. I also provide technical assistance and support to all of our districts and charters in the state or their state reporting, so going through providing data for Child Count, going through and troubleshooting data issues or questions. I also am the lead for our state performance plan annual performance report.

00:01:40.32 >> Hi, and I'm Debi Smith, and as Amy said, I'm the Special Populations Coordinator here at the Idaho State Department of Ed, and so I oversee all the charter schools in the state of Idaho as well as the correctional facilities and residential facilities in Idaho. I also oversee what we refer to as GSFR, which is our General Supervision File Review, LEA determination and significant disproportionality in the state of Idaho.

00:02:14.85 >> Great. Thank you so much. So both of you are heavily immersed in data in everything you're doing, I would imagine. So to start out, can you just tell me what it means to be a data quality influencer?

00:02:27.72 >> For me, it's really about going through and trying to get folks down at the local level and our stakeholders to understand what the data means, why having quality data is important and what that information shows you and how to go through, analyze your information and work towards making decisions based on that information. But you have to first start with that data quality because if you have bad data, you can't make good decisions off of it.

00:03:13.65 >> Yeah, and I guess I would say in my role here, I work, as I mentioned, around significant disproportionality and determinations and file review, and so when I'm working with LEAs, one of the things that we talk about with good data is just really at sort of that root cause, doing analysis of what your data looks like and stepping back even to prior to special education. What does it look like within your particular LEA when it comes to interventions? How are you deciding which kids get intervention and which kids don't? What data are you looking at, and do teachers in general ed, even before it gets to special ed, understand what that data means and how they can react to the data that's in front of them?

00:04:07.84 >> So really having that high quality data hopefully to start with so that you understand the data, can use it to really drill down and make decisions and serve students, even students who are in special education, students who aren't, but it sounds like that's a lot of what you are talking about what it means being a data quality influencer.

00:04:29.56 >> Yeah, exactly. Yep, and making sure that even though it's special ed data we're looking at, sometimes it's not always special ed people that need to be at the table. It's really the whole team that needs to look at the data and really be making some decisions about it.

00:04:47.06 >> Absolutely. Everyone is a data quality influencer, not just in special ed, but, yeah, across the SEA and beyond.

00:04:54.23 >> Yep. Mm-hmm.

00:04:55.20 >> So given kind of your take on what it means to be a data quality influencer, can you tell me about what both of you are doing in your state, in Idaho, to really influence your data?

00:05:06.72 >> Well, because I work a lot with that front-end of how the data is coming in, I work a lot with our IT team and those folks who are actually doing the assistance with the school districts and charters, the LEAs, do create data validations to discuss with them, how are we supposed to see this data? How can we ... What little pieces of training do we need to develop for our LEAs so they can understand how they're supposed to put that information in and in some cases why certain scenarios don't make sense? Going through and really we keep on bringing up this line of information that your data tells a story, and you want your story to make sense. If it doesn't, here are the different steps along the way, those validations, those warnings and errors, and what they're really telling you. So I do focus a lot more on how that data information is coming into the state.

00:06:26.93 >> Yeah, and I would say as far as being an influencer, I work with Alisa a lot, and because I am not a data person, that's not part of my role, oftentimes it's confusing to me, and so I always sort of play the role, like if it's confusing to me, I'm guessing it's confusing to some of the other directors and teachers out across the state. So I guess I sort of see my role as being that guinea pig that says, "Alisa, I need you to explain that to me in a different way because that doesn't make sense to me." So I think we work well together, kind of play off each other so that we can make our presentations to our school districts and our charter schools more understandable so that when we're done, hopefully they understand what the data means, how to interpret it, and one of the things we do in Idaho, and it's Alisa and one of our other coworkers, Kailey Bunch-Woodson, is a data drill-down, and really we go across the state, present the data, all kinds of data, and it's just a great opportunity to both teach schools how to read and analyze data but also teach me, like my first time I went to a data drill-down I learned a lot about data and the importance of good quality data and how to read data.

00:07:55.24 >> Yeah, and one of the great opportunities that that data drill-down training provides us is that opportunity for district and charter staff to really look at their data, compare it to state or regional information, and when we first started, usually that first comment if people were going to have a reaction to the data is go, "This isn't my information." Well, this is the information you provided us, and having that honest conversation of, this is what it ends up looking like and this is how it impacts you as it goes into those different calculations, whether it's those LEA determinations, whether it's significant disproportionality, all of those different pieces connect in together, and so having that honest conversation of, "Here is what your system is looking like. How can we focus in on different areas and really drill in what other information do you have at your district or in your schools that's going to help inform you and turn around some of those situations?"

00:09:22.42 >> It sounds like those data drill-down meetings are very powerful tools and support that you're providing in your districts. Is it ... Do you gather multiple districts together for it, or is it really more one-on-one with individual districts?

00:09:37.54 >> Well, we have developed it so that districts can come and have one-on-one meetings with us later on after the drill-down. It is really about having those regional ... those districts or charters in the region come together. We try to intentionally put like districts or charters in the same space so that if they're willing they can have those conversations about, "Here is what we're doing in our district," or "Oh, we're having something else different going on. We actually have, say, a high rate of OHI," while this other district that's in their same region is showing high numbers in SLD. What are the differences? What are your differences in policies, practices and procedures that might be impacting that identification process? Those type of things. We also have conversations about graduation or drop-out rate or also assessment data. So if one district in the area is having really high performance for their students with disabilities, that gives those other districts that are in that same region the opportunity to really go in, ask these questions and develop more relationships with their peers to try to identify maybe potential areas that ... and processes that they can use to improve their data and improve their results for students with disabilities.

00:11:33.34 >> Yeah, that's fantastic. So they can learn from each other, share what's working, maybe what isn't working necessarily and some of those best practices. Can you share any challenges that you've encountered to being a data quality influencer, and if you've been able to address them, how have you done so?

00:11:51.33 >> I would say ... This is Debi. So again, with significant disproportionality, I'd say one of the challenges that we have faced is when we have LEAs who are significantly disproportionate around discipline because often when we're talking to the LEA, we're talking to the special ed director primarily, and more often than not, it's not the director who's directly involved in issuing the discipline. It's usually the high school or middle school principal. That tends to be where a lot of the discipline occurs, so really that can become challenging because it feels like we're telling the special ed director, "Hey, you guys are having an issue with discipline," and maybe they don't have a lot of influence on what's happening at that high school or middle school level. So it's really trying to again bring that team together, the general-ed population, those high school, middle school principals, superintendents and really talk about what discipline looks like, and I would say that the quality of data that we get around discipline isn't always accurate, and so one of the things that we've really tried to talk to districts about is doing sort of a quarterly, if you will, review of your discipline data ...

00:13:18.93 >> Hmm.

00:13:19.05 >> ... because waiting until the end of the school year to go back and look at all the discipline that you've issued during a school year, it's really hard to remember, right?

00:13:28.89 >> Yeah, it's too late.

00:13:29.72 >> Now, was Johnny really in in-school suspension, or was he on a time-out based on a behavior plan written in his IEP? It's those kind of scenarios that we're hearing about, and so it's like, get in there quarterly while it's still fresh and really scrub your discipline data so that when you're submitting it, you know that it's accurate.

00:13:49.38 >> That's a great thing to keep in mind for other states out there.

00:13:53.93 >> Well, and probably one of the areas that I see is that as you're going through, folks are providing that demographic information, that you really do need to have communication across your system because the people who are doing data entry aren't necessarily those ones that have direct interaction with the student. We see this especially around program exit information. You have a student walking with their graduating class, but they're going to be coming back for services the following year. It's really just the student is going through and doing that event with their peers, so they have that interaction, but they're coming back. Well, the registrar sees that the student walked with their class members, automatically records that student as graduated, met regular requirements, while those program folks know that that student is going to be coming back to receive services again in the fall. So having that communication, and we've built in different validations, but validations are really just going to be able to show you what's outside of the norm. Those conversations, relationships and internal processes are much better and more effective at creating change.

00:15:37.25 >> Yep. That all rings very true. We were just not too long ago at the Interactive Institute in Nashville, and we had a virtual Interactive Institute. Was there anything you learned or took away from the Interactive Institute that you think will help in your data quality influencer roles?

00:15:54.91 >> In this last year, we really started boosting up our amount and different versions of information that we're providing out to the public for our stakeholders and educational partners. One thing that really resonated with me was in the Interactive Institute that you really need to focus on those motivators for your audience, and what motivates me may be very different than what motivates those stakeholders and educational partners and that I really need to focus and tailor that information to the audience, and if you're focusing in on putting your information so that that audience is part of the solution and part ... one of the heroes of that data story, that they're a real influencer on that data, then that will have a much more positive impact and influence on that data audience, and that's one of the areas that I really rely on other folks in our office, Debi, those folks who are really more program-related because I can look at a data set and go, "Oh, these are the different things that are happening. Here is what these numbers in this spreadsheet are telling me," but that isn't the majority of our audience. The majority of our audience isn't necessarily that data literate or informed about what the data is even about. So it needs to be very clear, concise and to the point.

00:17:56.55 >> Yeah, and I'm going to just piggyback off of one of the things that Alisa said when she talked about ... She used the word story, and that was a big takeaway that I got from the Institute was really kind of telling the story, so relating the data back to a powerful story because that's what your audience really remembers. They don't necessarily remember the exact numbers, but the story that you can tell around the data that maybe tugs at their heartstrings or just plants a seed that they don't forget. That was powerful for me so that when we're going out and we're meeting with our school districts that if we could share a story about another district or a student that is powerful, it helps them to really, I think, at least have more buy-in about the data and how it can impact, even if it's just one student. That was a big takeaway for me is being able to tell a story.

00:19:02.13 >> Yep, so that it really hook the audience in ...

00:19:04.84 >> Yep.

00:19:04.98 >> ... and make them interested and want to take action and really sticks with them, and something you had mentioned a little while ago, too, was the partnership that the two of you have and being able to bounce ideas and be a sounding board and know when there's times maybe when this is a little too technical or this needs to be explained differently, being able to work together in that way I think sounds like a real strength of what you all are doing.

00:19:30.92 >> Yeah, I am always learning things from Alisa when it comes to data, and I've just learned so much since I've been here. And I think it's a real powerful relationship that we have.

00:19:44.23 >> Yeah.

00:19:45.44 >> We have a great team across the board, and that's been one of our areas of focus really recently is really establishing a lot stronger ties of, "Oh, here is how my information and my work connects with your work," and connecting the dots across ... system so that we can help better support each other because my strengths are more in data. I have not been more in that program area or in the district working with the students. Debi is much better at being able to tell that story. So bouncing ideas off of each other, relying on each other's strengths is very important.

00:20:43.12 >> Yeah, definitely. What do you have coming up next in Idaho as data quality influencers? What are your future plans?

00:20:53.14 >> Well, we mentioned the data drill-down. That typically takes place in the fall of the year. So I think starting conversations around what that data drill-down will look like, if there's more data elements that we want to add. We are just getting ready to have webinars in August for determinations, so having LEAs join us to talk about their determinations, especially those that we in Idaho identify as level three, meaning they need more support. So really digging into what their data looks like, and one of the requirements that we have in Idaho if you are in a level three, meaning you need more support, is that they write a 3-year SMART goal, and it's a goal of their choosing based on a data element that they feel like needs the most work. So we are starting to do some of that work, and, yeah, just connecting with the LEAs once school is back this fall.

00:21:57.85 >> Yeah, and really what right now in large part of my focus is going through and recapping, okay, here is the information that we have out. What has been good? What needs edits? Going through all of that guidance and training information so it's ready for the following year. I just got finished doing a whole new batch of validations and everything with our IT folks because every year we go through and re-evaluate. What are the areas that we had the most problem or data issues with? For Debi and I, what areas in our internal processes can we make better? So looking at addressing those LEA questions better, just post having it out and the information posted so that folks can easily access that information. That reduces the time and effort for districts and charters going through trying to find the information. Also reduces our time and effort in going through and addressing questions so that we can really dig in and focus to those areas and specific needs of districts and charters better.

00:23:32.42 >> So it sounds like you very intentionally kind of when the school year wraps up, take some time to evaluate, how did things go? What changes do we need to put in place? What areas do we need to emphasize in the next year? That makes a lot of sense.

00:23:45.88 >> Mm-hmm. Yeah, and we actually do that after ... really after each of our collections is go through as a team and discuss, "Okay, what worked really well for file review? What needs some enhancements?" And we do go through on all of our different applications we use and have a regular recap of the collection. What worked? What didn't? What would be nice for moving forward? And then we have conversations with our developers, and it is pretty well on a regular annual basis that we do that.

00:24:27.82 >> Yeah, one of the things that we looked at this year, and I'm sure not unique to Idaho, but sort of the idea around staff retention, realizing that we have a lot of directors who are leaving or left this spring not to return in the fall. So we sort of looked at that and shifted what would be presentations in late spring, early summer, we shifted to fall knowing that a lot of our audience might be brand-new directors, and so we need to make sure we're hitting those folks right out of the gate with the information so that they have it and we're not relying on the former director to share that information with them because sometimes it just doesn't happen. So we took a look at that too and sort of shifted gears on how and when we're presenting our information.

00:25:17.44 >> Great. Is there anything else that I didn't ask you that you want to mention?

00:25:23.75 >> I'll say that as a new participant to the Institute, I was telling Alisa that again I'm not a data gal, but it was really one of the best conferences I've gone to. I just ... I learned so much and walked away with lots of ideas, and I felt like it was just a great Institute. So I just want to thank all of you guys for putting on such a wonderful conference.

00:25:49.19 >> Great. I'm so glad to hear that. It was wonderful to see you both in person and be together in real life.

00:25:56.26 >> Yeah.

00:25:58.90 >> All right. Well, thank you both so much. You shared such incredibly rich information, and I have so many follow-up questions that I'm thinking about along the way, but we'll maybe catch up again on another episode later on. So thank you for taking out the time to do this with me.

00:26:18.30 >> Yeah, thank you, Amy.

00:26:19.13 >> Yeah, thank you, Amy.

00:26:21.29 >> To access podcast resources, submit questions related to today's episode or if you have ideas for future topics, we'd love to hear from you. The links are in the episode content, or connect with us via the podcast page on the IDC website at ideadata.org.