Episode 20: Qualitative Research in a Digital World

Qualitative Conversations

In this episode, Alexandra Panos interviews Jessica Nina Lester and Trena Paulus about doing qualitative research in digital words. With the onset of the COVID-19 pandemic, the need for digital methods and strategies has never been stronger. This conversation addresses important practice and theoretic questions for approaching digital inquiry.
Digital Tools for Qualitative Research - Trena M. Paulus, Jessica N. Lester, Paul Dempster
Doing Qualitative Research in a Digital World - Trena M. Paulus, Jessica N. Lester
The following includes the transcript of the talk. (please excuse minor transcription errors)
Alexandra
Hello there and welcome to qualitative conversations a podcast hosted by the qualitative research special interest group of the American Educational Research Association. I'm Alexandra Panos an assistant professor of literacy studies and an affiliate faculty member in research and measurement at the University of South Florida. I also have the pleasure to serve as program coach with Cassie Quigley for the call SIG. And I'm delighted to be here with doctors Trina Paulus and Jessica Nina Lester to talk about the role digital tools play in qualitative research. Dr. Paulus is a professor in the Research Division of Family Medicine and director of undergraduate research and creative activities, as well as an affiliate faculty member with the Applied Social Research Laboratory at East Tennessee State University. Dr. Paulus's scholarship is primarily in the area of methodological innovation, especially as it intersects with new technologies. Dr. Lester is an associate professor of inquiry methodology in the School of Education at Indiana University Bloomington. Her scholarship focuses primarily on discourse and conversation analysis, disability studies, and more general concerns related to crop qualitative research. Dr. Paulus and Dr. Lester have co authored with sage, the 2014 book digital tools for qualitative research, and an exciting new and in press volume, titled doing qualitative research in a digital world. Thank you both so much for joining it today and sharing your time and energies. Thanks for inviting us. Yes, thank you so much. I'm truly excited to learn from you both and really just want to dive right into our conversation if that's okay. And I wanted to start with
Alexandra
a question that situates us in the here and now, given the shifts that have happened worldwide over the last 10 months with the covid 19 pandemic? Can you share a bit about what from your perspective, this really means for qualitative researchers? And how digital tools might play into this?
Trina
Yeah, so it's kind of this been this weird experience of being in the right place at the right time, or being in the right place at the wrong time? I don't know. But you know, COVID-19 is impacted all researchers in significant ways, for sure. And, you know, we had started writing this new edition of the book,
Trina
fully revamped book that's coming out shortly, a couple of years ago, actually. And then when COVID-19 hits sage really asked us to try to wrap it up, because researchers really needed some guidance for how to basically do their research in a new way.
Trina
And so how do we make sense of those spaces? How do we look at online interaction as a source of data as qualitative researchers, you know, we are interested in the human experience and understanding it as qualitative researchers, and that is now completely emeshed with, you know, doing business, doing education doing everything online. So, you know, there's new opportunities here, even though you know, most people have been, you know, there there are researchers have been doing this for a while, we all kind of have to consider how online spaces might be treated as a source of data, how our experiences are different.
Trina
And so while I think kind of one of the first things people want to know is how do we do interviews in zoom? It's, it's more than that. It's that we're all now spending lots and lots of time in zoom. So how can we understand what's happening there? So we've got digital tools, digital spaces, and also the digital space as a phenomenon in and of itself. Those are just a few of my initial thoughts. Jessica, what do you think?
Jessica
Yeah, I mean, I think that one of the things that's really helpful to keep in mind and in this moment, even though there's this you know, it feels like such a significant and even forced shift for qualitative research, I do think it's helpful to remind ourselves that this move to doing qualitative work and online spaces is not particularly new. And there's really a vast body of literature that we can draw upon to support us and offer some guidance to the questions that we're facing. And even really provoke us to think more about what it means to do qualitative research and be a qualitative researcher in a, you know, a historical moment where we're not just using technologies, but we're living through them even as researchers, and we're making meaning with them. So I think that's something for us to kind of use as a way to frame this particular moment. And that there's resources that we can turn back to, but that also, these are important questions that we should be asking ourselves about our work, and about what it means to make sense of meaning making in a space where technologies are really intersecting with everyday life.
Alexandra
Thanks, yeah, it's, it's, um, it's a lot to process. And I appreciate your point about that. Sometimes it does feel for Steven F, for we're all going through it. And it makes me think a little bit about design, designing scholarship right now designing qualitative scholarship right now. So I wonder if you can speak a little bit about how you work through as a qualitative researcher having to adjust your expectations about a research project, when what you'd hope to do isn't currently feasible. Given realities? Yeah, I
Jessica
can speak to that a little bit. You know, one thing that I often talk with about this question in relationship to students coming to me and saying, I can't do what I hope to do, particularly in this, this given moment, what are my options, and I think it is helpful to think about that, you certainly can return back to your original design and think about if there are ways that you can transform some of those methodologies and methods in engaging with digital tools, that you could digitize some of them, for instance. And then also, it is possible that you actually have to go back to the drawing board. And one of the things that you could potentially engage with is really thinking about designing a study, from the get go, that really engages in with digital spaces. And what that might mean is that you expand definitions of data. And this can be really exciting. And you can engage with new kinds of data that you hadn't even envisioned engaging with before. And so I think, you know, there are those two pathways to think about turning back to that original design and potentially, in some way, digitizing that original design or really rethinking it. And I think that this is an okay thing. I think that, you know, part of part of methodology writ large, is that it's always in the making. And so right now, our methodologies are in a really real way intersecting with technologies. And so what that means is that methodology is being remade. And that's, there's something also really both challenging, and also, potentially really exciting about kind of that moment. So at the same time, I think what that also means is, as we think about re envisioning a study, that we also have to keep in mind that the technologies that we engage with are, of course, not neutral. And they are, of course, you know, always fraught with consequences, including, you know, political consequences and equitable access. And so this is certainly something that we also want to set with as we think about our design, and particularly as we think about redesigning a study.
Alexandra
It's so helpful. I wonder if there are theoretical perspectives that have you found particularly helpful for conducting this kind of Digital Research and turning to to the stat sitting with process that you spoke of?
Jessica
Yeah, I can speak
Jessica
to this, this idea of theories that have informed our work, and I think I'll just share a little bit of a story of how we're coming to think with theory now. So. So first off, so just in general, if you were to engage with the literature around technology and qualitative research over the last, you know, about decade, you would probably conclude that qualitative researchers have historically kind of held on to this view of what's often referred to as technological determinism. So that's this orientation that humans are essentially passing And therefore, they must adapt to changes that technology forces upon them. So this particular view is one that often assumes that it's intrinsically best and most efficient technology will be adopted regardless of the context. And so this particular view is one that when Trina and I wrote the our first book, around digital tools and qualitative research, we really explicitly wanted to counter this perspective that was in the methodological literature. And in some ways, we did this implicitly, but what we really argued for was thinking about the ways in which we as qualitative researchers could really use technology to do things that we wanted it to do. So we didn't position ourselves that's passive. And so within this viewpoint, then qualitative researchers could be thought of as kind of retaining control of qualitative data analysis software, for instance, and not assuming that the software would control the study. So technology, and from this perspective, would be theorized and viewed as not just instrumental, but really positioned as what a human qualitative researcher can use it to do. But after we were wrote our first book on this topic I ran across a book that Katherine Adams and Terri Lynn Thompson had written, which was titled, researching a post human world interviews with digital objects. And in their book, they engaged with new materialistic post qualitative perspectives, and really offers some interesting ideas about how we as qualitative researchers are really intermingled intertwined with digital technologies. And as a read, I saw some references to our book. And so I immediately texted Trina and Katrina, someone has something to say about our book. And, and ultimately, it was a critique, it was a critique of our view of technology that we crafted in the first book was really not engaging and a full way with the ways in which technology really can be conceptualized and theorize as being co researchers with us that there's a dialectic. And so in our newest book, we really take up this critique and have begun to think more with you've been realistic ideas of technology, as well as some of the critical theories related to technological use. And we have found that to be really helpful, and generative, and pushing our thinking about how technologies are co researchers with us and are entangled with us in the process that is then something that we have to really think carefully about, and think about the implications of the tools that we use, and the ways that we're engaging in meaning making.
Alexandra
That I love hearing the story of how these texts evolve over time one, it also makes me wonder, the about the ethical and privacy considerations is something that's important when you do that kind of Digital Research when you are intermingled. So I wonder if you could share a little about that.
Trina
Yeah, I can talk a little bit about that. Because it does these issues come up a lot. In the work that I do. And the researchers that I work with these, I'm primarily looking at Digital spaces, online communities and online groups as a source of data. So I think the first thing to think about is that, you know, we always are dealing with ethical issues and privacy considerations when we're doing qualitative research as qualitative researchers, you know, we're often looking very deeply and intensely at people's lives. And so it's always different than if you're giving a survey or doing you know, lab based research is different kinds of ethics. The good thing is that there's actually been a lot of scholarship around ethics and Digital Research for many years. In fact, when we were writing this book, I couldn't believe the proliferation of entire texts on digital ethics that appeared since we wrote the first book in 2014. So there's a lot of guidance out there for sure. A couple of the things that issues that come up frequently for me in in when I'm giving talks and talking with people about it, is the issue of digital traces. And the fact that so much of us are so much of our lives now even before the pandemic world lived on the internet and in the cloud and with mobile devices. And as we go through the world, we're live leaving digital traces everywhere. And whether or not those should be treated as a source of data is the is one of the big issues, right? Who owns those traces? And whose permission Do you need to get to look at those as a data source? institutional review boards often are getting better about having policies around this, but they don't always know how to guide researchers. And sometimes, even though technically, the IRB approves the study, because they don't consider looking at online discussion posts as human subjects research because they're publicly visible, just because they say it's not human subjects research may not automatically mean that it's ethical to look at online discussion posts as a source of data without telling anyone. So I think that thinking about who owns these spaces, who's interacting on these spaces, who has access to them? I think, you know, there, there are no hard and fast rules, because the landscape is changing all the time, right? Like we have Tick Tock now, and we didn't even have that before. So how do we think about Tick Tock as a source of research data. But a few things to think about is, you know, I'm working in a medical school context now. So I hear a lot more about you know, the do no harm, first, do no harm mandate. And so you want to first be sure you're not harming or putting anybody at risk, whether that's an entire online community that no one would have known about until you wrote a paper about it, or if it's about an individual who was posting under their real identity, about a very sensitive illness online, and you bring attention to them inadvertently, or, or on purpose. really thinking about that is, of course, the basis and the fundamental issue around ethics. I've been also thinking about privacy. Do they expect that that this community is private? You know, is it really just for insiders? How can you respect people's privacy? At what point do you need to get informed consent, which may be very difficult when people are in online communities not as themselves, but under an assumed identity assumed identity? How do you navigate that? And so keeping identities private, protecting the data? If you do store? If you do treat it as data, then how can you make it as hard to track down the original sources possible? And then if you do do that, is it changing the essence of the data, or the essence of those online interactions, so that it actually might impact how you interpret it? So those are things that we have to struggle with? how sensitive are the topics that people are talking about? And again, you know, just trying not to put people at risk. So I think the good news, like I said, is that there are lots of case studies. The one of the best sources for guidance around this is the Association of internet researchers, they're actually on their third version of their ethics guidelines that came out, I think the most recent one came out this year.
Trina
And they really, you know, cover, it's all on a continuum. And they give a lot of holistic advice in terms of things to think through. And what I will always say on this topic, really is that if you can do research with people instead of on them, these issues are going to be much easier to navigate. If you want to look at an online community, get in touch with whoever's in that community and see what kinds of topics they would like you to study, and what would they like to know more information about, so that you're actually working in collaboration with the people that you want to understand better?
Alexandra
Thank you for that for the wonderful resource. And then just the plugs, work with folks to
Trina
think about their community. Yeah.
Alexandra
So important, simple, and, and really effective, I
Alexandra
think.
Alexandra
So I want to turn a little bit towards method here. And I'm wondering what tips you have for that data collection process online? Or how to think about additional sources of data to look for once you move research into those online spaces.
Trina
Yeah, and I can talk a little bit about that, too. I think, you know, one of the first things to ask is, okay, where are people talking about the issues that I'm interested in? So your research question like and just to give you an example of one of the earliest cities I did that was outside of an educational context. I'd collaborated with Dr. Mary Alice, Varga who's at the University of West Georgia now, and she's one of her areas is grief counseling. And she was really interested in why people choose to go to grief support groups or not, when you've suffered a loss, you know, you know, you're you're advised to kind of get support, but sometimes it's hard for people to go to grief counseling either individual or in a group. But we discovered or she discovered that there Actually a lot of online grief support groups out there. And she was really curious about why are people going online to get support when we have all of this in person counseling. And so we were able to analyze an online grief support group to kind of understand how people constructed their grief in those spaces. And those findings then could speak back to how people were doing grief counseling offline, you know, so in a pandemic, you may only have access to these online spaces, because so much of our in person services are no longer operating. If you think about any kind of social human experience, phenomenon, social science topic that you're interested in, think about where people talking about it, and just do some investigating. And we make the distinction in the book between naturally occurring or pre existing sources of data, which are things that are already out in the world, Reddit forums, tick tock, lots of online support groups for people who have specific illnesses. And they're just grassroots efforts or they're supported by a certain professional organization, travel blogs, and forums, Google Groups I hear is a huge source for parents trying to school their kids in the pandemic, there's all these neighborhoods and friend groups, setting up Google Groups, test text message threads. There's lots of places where people are talking electronically, and they have been for years, but now especially there's electronic conversations going on, that might give you insight into how people are talking about things without people talking to a researcher directly. research agenda generated data is when I decide to go interview someone, they're talking to me as a researcher, so they'll give me you know, their thoughts on things up to a point up to what they're willing to disclose to a researcher that they may not know that well. So that's important data. But what's really interesting is to see how people are going about their lives in these spaces without researcher intervention. And that can give us some really interesting insights that we wouldn't get otherwise.
Alexandra
That's, that's really interesting to think about all of those spaces that we're all contributing to right now.
Trina
Exactly.
QR SIG AD
Right. So that's really interesting. The qualitative research special interest group was established in 1987. To create a space within the American Educational Research Association. For the discussion of ethical, philosophical and methodological issues in qualitative research. We invite you to consider joining the qualitative research thing today. for members of a era the annual fee for joining qualitative research special interest group for regular non graduate student members is $10. And the annual fee for graduate students is $5. As members of the QR SIG, you will gain access to a network of fellow qualitative scholars, as well as our many activities ranging from mentoring opportunities to our podcast series to updates and news related to recent qualitative publications and jobs. Please visit the American educational research associations website at www dot att era dotnet to join the qualitative research SIG today.
Alexandra
I guess something else I'm thinking about is that this idea that much might be lost when doing online interviews, interviewing is such a staple. For us as qualitative researchers, I think, do you have any thoughts or tips for enriching interview data beyond the recorded audio when we're working with digital tools?
Jessica
Yeah, I can, I can speak to this. And I think a useful place to start in response to this is actually to flip the script a bit on this. So rather than assuming that, you know, much is lost, I really prefer to think about it as just being different. I think it's really important to keep in mind that, you know, historically, face to face interviews, in qualitative research have really rested on some pretty notable assumptions about what it means to do qualitative research and about what participants should be doing and how they should be doing it. I have a favorite paper that was written by two critical disability studies scholars, Stephanie Kershaw and Margaret price in I think, was 2017. And their paper was was focused on thinking about how we can center disability and qualitative interviewing. And one of the things that they noted was that interviews writ large, and they were speaking both to those conducted in face to face contacts and as well as online, but that they really rely upon normative conceptions of body mind. So people, you know, ask a verbal question. And participants are expected to respond in a particular way. And we assume that language given in a particular way, and shared in a particular way is how it will occur. So it's a very normative assumption about even meaning making. So I say that because I think it's really important that we're reflective and careful about orienting to interviewing, as it has always been done as the only or even best way to capture making sense of people's lives and experiences. You know, we do know from some groups of people that this really has not been their experience of this method and crush on price speak to that a little bit from their own experience. Um, so then, if you are conducting interviews in a virtual space, I do suggest that you, you know, really orient to it as different. And certainly, there are important considerations, some of which are similar to face to face interviews, and others that are really unique to the particular technologies that you're using. You know, so like, an example of this might be, you know, if you want to consider whether videos will be turned on or off, and what does this mean for things like rapport building, or even how participants might experience a researcher viewing their private spaces. So, you know, corresponding price. They also argued that, even though you know, there has been this writing, and kind of argument from some researchers that we need to consider the significance of digital interviewing methods, because they do create access. And some people, some participants prefer that kind of interviewing space. They even pointed out in their work that even in these digital spaces, there can be this over reliance on kind of a normative body mind way of thinking about interaction. So I think in general, the real key is just to be critical, regardless of kind of the the modality that the interview is taking place in. And so as a starting point, I always encourage folks to, to your number one turn to your participants, to invite them to share with you ways that they can share their experience and their lives. In these digital spaces. They might have ideas, first, you know, ways that they want to do screen sharing, or even apps that are really useful in their own lives for sharing how they're going about living their lives. And also, you know, there is, again, a really nice body of literature that you can turn to, to get some guidance. You know, beyond Kershaw on prices article, Janet salmons has written a lot about online interviewing. And I also think it's useful to turn to some of the critical methods writing in the disability studies community that has really problematized interviewing, and both face to face and online spaces, and also highlighted how, you know, we never want to rest easy with being armed with a bunch of methods, literature, but the real importance of turning back to our participants to really help us understand better how we can collect data that allows us then to make sense of meaning making
Alexandra
you for, for talking through that and flipping that script, I think it's so important, and I just learned so much. And I want to turn now to your point about what the data is how we how we cope, how we collected or generated. So to think about technology, I guess I'm wondering if there are any, you know, specific particular platforms, technologies, devices that you have found particularly beneficial, and that you use when you're doing digital research? Yeah, I
Jessica
mean, you know, one of the challenges is that there are so many. And so of course, it depends on the the study and the nature of the project that you're working on itself. In our in our new book, one of the things that we have throughout is, is more than 40 vignettes, so on the ground researchers that describe their work and the specific tools that supported their work and that they engaged in. And so I think one of the ways to learn about what's out there is is really to engage with the writing of on the ground researchers who are are working across a range of disciplines and therefore asking really different kinds of questions that lead them to engage with different technologies. But again, you know, it really does depend on on the study and the nature of the project project. So if, for instance, I'm working with Instagram data, there are particular applications that I would use As I would download and format my image base data, versus when I'm working with interview data collected via zoom or another video conferencing platform. So it really does depend on the design of the study. And this is something that we've described in our writing as being part of you're generating your own Digital Research workflow. So in my own work, regardless of the project, I typically use qualitative data analysis software, specifically, I'm an atlas ti user, and recently have begun to delve into learning and using max q da. I mean, I use qualitative data analysis software really to manage and organize the entirety of my research study, including things like my literature review, and also using various features within a package to write up some of my my early findings within the package itself. So in this way, I, I personally orient to qualitative data analysis software, as kind of being like the the One Stop Shop that supports many of the aspects of my digital workflow. And I think what's important is that we we all individually spend time really not just designing our study, but thinking about where it intersects with our own Digital Research workflow, and identify ways that that can support the the work that we're interested in pursuing.
Alexandra
It's really helpful. I love the idea of a digital workflow and just having that be part of a study design and thinking through it that way. And I guess another sort of staple for us as qualitative researchers is transcribing data. I wonder about your preferred methods for that process? I know there's there's a lot of literature around transcription. But what what are you guys seeing right now?
Trina
Yeah, so that is, a whole chapter of the book is about innovations in transcription, because this is one of the areas that has really changed a lot since we wrote the 2014 book, in part because of just the leaps and bounds that auto transcription, artificial intelligence supported transcription tools, what they're able to do now such as Trent temi, otter AI, there's a lot of them out there, and they are getting better and better all of the time. And and just as an example of that, for the for people using zoom, you may have noticed that if your institution subscribes to it, you will actually get an automatically generated zoom transcript, which is phenomenal, if you're doing your interviews in zoom. And I would say this is actually where online interviews are hugely advantageous over face to face because there's an automatically generated transcript at the end. Now, granted, we all know that you have to make edits. But compared to what this used to look like the edits, if it's good sound quality, standard English or standard version of whatever language you're speaking in. If the conditions are right, the transcript can really be amazing. So for, you know, video conference type interviews, you know, if there's an automatically generated transcript, that's definitely a great place to start. This summer, I actually used Trent for the first time as a first pass to transcribe some patient, patient interviews, the health care providers, students in the health professions, were interviewing standardized patients. And I had a bunch of video data. So I ran it through Trent as a first pass it automatically it timestamps that automatically you can edit the transcript within Trent. It's a great data storage, it's all cloud based. So you do have to get IRB approval, and we didn't have any HIPAA data, HIPAA protected data, so it worked for us. But I do think that looking into some of these AI based services is definitely worth it as a first pass, if you're not actually conducting the interviews in zoom, another really good tool is ink scribe i en que se RIBE. It also lets you timestamp because what that means is if you can synchronize your recording with the transcript that is just so beneficial as a qualitative researcher so that you're not just relying on the words, you can actually click anywhere in the transcript and listen to that interaction again. And so Jessica and I both do conversation analysis and discourse analysis and other language based analytic methods were how people speak is as important as what they say. And so the technology, the ability to not just rely on the written transcript, but to be able to go back and listen again to how something said that's just been invaluable. And so I think we do need to really think about transcription as part of that overall Digital Research workflow. And there are cases in which some of the qualitative data analysis platforms We'll support that as well. So if you're using the Mac version of Atlas ti, I think that you can actually transcribe within that software. And the same with in vivo and Max q da, they provide the ability to do synchronize transcripts. So it's definitely worth thinking about how that's going to be integrated into the whole process. And, Jessica, I'm not sure if you wanted to add something on this one, too.
Jessica
Yeah, I was just gonna also say that is the one of the things that I think is really interesting about new technologies as they relate to transcription is, I think it's a really vivid example of how you can see innovations and technologies shaping how we think about method and methodology. And so what I mean by that is, you know, many of the qualitative analysis software packages now allow us to do import in a fairly fluid way. sizable video based data sets, though I work with a lot of video based data in my own scholarship, and they're relatively large data sets. So working with, you know, upwards of 100 plus hours of interactional data. And one of the things that these new innovations have really pushed to the fore is questions around do we even need to be transcribing our entire data set? And why are we transcribing our entire data set? What might it mean to leverage things like directly,
Trina
directly
Jessica
analyzing with the tools that are embedded within qualitative analysis, software packages, or video, and then we're selectively transcribing our data. And these questions, of course, become really pertinent when you're working with large datasets and just thinking about transcribing, which has been the norm and conversation, analytic work and much of discourse analysis as well. You transcribe everything and you transcribe everything, using transcription systems that are really, really intense, and take an extensive amount of time. So there's this time issue, but then there's also what's I think, arising is questions around why are we doing what we're doing. And I think if we trace across time, we'll see that there is a lot of methodological shift that happens in relationship to technological innovation. I mean, even if we just think about interviewing, how we collected interview data has radically shift as the development of recording technologies came to be, and then a refinement of those. And so I think right now, a really compelling and provocative and important question that I do hope that we, as a community, spend time really wrestling with is what is the place of transcription? And what might it mean to think about transcription differently at the intersection of technology and our methodological practice.
Alexandra
But I'm really thinking a lot about what you just said, I'm gonna send you a message after. Um, so I guess another question that I'm wondering about is the tools what tools are you using for storing all this massive amounts of data and these big files, etc, in both an accessible yet also secure way?
Trina
Yeah, you know, my biggest recommendation there is to use whatever your university is supporting, because you don't want to get in so like, at my institution, it's OneDrive, right Microsoft product, and, yeah, it may not be like the easiest, most accessible in terms of, you know, efficient way because the I don't necessarily like the way that the navigation is set up. But my institution has it, it's secure, it's supported IRB, are okay with it. Everybody that I'm working with in my institution can access it. Theoretically, people at other institutions should be able to access it as well. And so, you know, I think that really sticking with what your institution supports is a good first way to think about that. I do want to say that if you are thinking about long term storage of data, you need to consider a qualitative data repository, especially if you are willing for other researchers to have access to your data for reuse. to Oregon, that's another good reason to use qualitative data analysis software, because actually, all of your data is stored in that platform. That's how it's organized. And so you've got the software package, organizing your data, then you've got the original files, you know, also maybe in OneDrive, or what other whatever other system that you're using, that can keep it all very manageable. That's, you know, and then you know, that there's, there's the There's the password protected things with, you know, sharing files in certain ways that I do think you have to think about. But one nice thing about being in a secure cloud based service, like one drive supported by the institution is you don't have to think about it as much as you used to have to when everything was stored on individual computers, or hard drives, or zip drives and jump drives, and then you had to think about how you were going to password protect each file, and then how you were going to send it in secure emails to your collaborators. So you know, just look into what your your university supports. And I do realize that's a privileged position. If you're not working at an institution that provides something like that, then you do have to kind of think through all of those steps that that we did before we had these services...
[End of the transcript]