Andrew Holter on When the Data Doesn’t Fit

In this episode, Dr. Ben Butina interviews Dr. Andrew Holter about the surprising findings in his recently defended dissertation on whether self-efficacy, grit, and job satisfaction predict turnover intention among new registered nurses. Dr. Holter also discusses his background in public safety and how IO psychology is applicable in his role as a fire department administrator.


This transcript is AI-generated and may not be completely accurate. Please do not quote myself or any of my guests based on this transcript.

[00:00:00] Ben Butina, Ph.D.: Hello and welcome to the Department 12 Podcast where we talk about everything I-O psych. I’m your host, Dr. Ben, Butina, and joining me today is Dr. Andrew Holter. Congratulations on your recent dissertation defense, Dr. Holter, how does it feel?

[00:00:13] Andrew Holter, Ph.D.: I don’t know what to do now with all my time.

[00:00:15] Ben Butina, Ph.D.: I remember that feeling pretty well.

We’re gonna come back to your dissertation in a minute. But I wanted to ask about your background. I know you work in the field of public safety. How did you end up.

[00:00:26] Andrew Holter, Ph.D.: I really started out here in high school. I got involved in local emergency services. I became a volunteer firefighter, I’ve been attached to some shape or form ever since.

So although my interests academically have shifted over time I still find. Even right now working for a municipal government as the administrator of a fire department. But I’m certainly able to do a lot of IO psychology things in this role and in this space where there’s just not that many of us doing that kind of work.

[00:00:55] Ben Butina, Ph.D.: What don’t we understand about that world?

[00:00:57] Andrew Holter, Ph.D.: It’s sort of culturally [00:01:00] encapsulated to itself. Being a, a firefighter myself we tend to think of ourselves as this unique breed that, regardless of any other background factor. Once you become a firefighter, you’ve like joined some exclusive.

And, you know, secret things now and you’re part of this association and I think it’s tough for the fire service in general to accept things that are different. You know, the common saying is that hundreds of years of tradition unimpeded by progress and, and so now as IO psychology does become a growing field in the United States, and there’s a lot of good things that, that we do and know that we can bring to the fire service.

Just trying to be able to communicate the value of those academic findings or those research based findings to the fire service is probably tougher or just as tough as it is in any other setting or field because of our culture of tradition. We kinda know better than [00:02:00] anybody else on how to do everything related to being a

[00:02:02] Ben Butina, Ph.D.: firefighter.

Somewhat similar it seems like, to the military in that sense that regardless of your identity going in, you pride yourself joining this, elite group.

[00:02:12] Andrew Holter, Ph.D.: military comparison is, pretty accurate. The fire service agencies are structured like a paramilitary organization.

They’re very top. Chief officers and, and a whole bunch of ranks in between to the frontline staff. A lot of chain of command structures and things like that. So it’s perhaps a lot more unique than a lot of agencies in the private sector are. Absolutely.

[00:02:32] Ben Butina, Ph.D.: So we’re here today to talk about your dissertation, which was titled, and I love this title because it’s so, To self-efficacy, grit, and job satisfaction.

Predict turnover, attention amongst new registered nurses. I love a title like that because I know what I’m getting and I’ve read your dissertation. It delivers what it says on the tin. congratulations on that. But let’s start with some basics for new listeners.

We’re trying to figure out whether three different variables predict, turnover, intention. [00:03:00] What is turnover intention?

[00:03:01] Andrew Holter, Ph.D.: That’s a great question. And when I started this, I struggled with defining turnover and turnover intention. It’s really a tough thing to measure. I was able to find one validated measure out there that did measure turnover intention and it was short which is good. Mm-hmm. We certainly don’t like giant long surveys. Yeah. People tend to quit ’em and so, That turnover intention was really looking at a, a scale that assessed the likelihood of whether somebody would or would not quit their job in the next six months.

[00:03:38] Ben Butina, Ph.D.: Hmm, interesting. Why was turnover intention more manageable than measuring turnover?

[00:03:43] Andrew Holter, Ph.D.: I would have into actual turnover decisions, but being able to find those people who have left their employers and ask , Hey, you left. Why? Right. To me, it, it looked and sounded and felt like more of a qualitative [00:04:00] design.

And if I was still in school, I would be doing that right now. And, and asking that exact question of people who have had the, the lived experience. Being a, a new nurse and leaving their employer, and I’m really after that quest to find out what things didn’t go right for them.

[00:04:17] Ben Butina, Ph.D.: Turnover, intention, it means what it says we have to operationalize it in a particular way.

And it sounds like for this study, it was this, Would leave in six months kind of thing. Job satisfaction is, I think to most listeners it is what you think it is. And, and self-efficacy is, more or less the belief that I can do something.

So I have high self-efficacy for, you know, running a podcast and I have low self-efficacy for you.

[00:04:42] Andrew Holter, Ph.D.: Brain surgery. Yeah, that’s, that’s perfect. One researcher put it, it’s the, I think I can construct, but it’s definitely specific.

[00:04:53] Ben Butina, Ph.D.: And so we’ve got self-efficacy, we’ve got job satisfaction. We know what turnover intention is, and now we get to [00:05:00] grit. Grit’s, one of the, the predictor variables you explored, and it’s kind of a controversial construct, isn’t it?

[00:05:05] Andrew Holter, Ph.D.: It has been yes. And a much newer on the scene than the.

 Can you

[00:05:10] Ben Butina, Ph.D.: Share a little bit about what you found in your lit review understanding that you probably got started a little before, you know, this controversy was in full swing, but, what did you find out about it? What is the argument against grit, so to speak?

[00:05:21] Andrew Holter, Ph.D.: There’s definitely a lot out there now today about.

 The validity and then the two subscales that make up the, the grit construct as the scale stands today which is a a passion construct that there’s a passion towards that goal or whatever that individual is working towards. And then there’s a persistence or perseverance that regardless of obstacles or challenges that come my way, I’m gonna persevere through them

There’s been a lot of debate over whether it truly is two subscales. The scale itself has some issues with questions that might be asking the same thing numerous times. There’s been a lot of holes shock it since the [00:06:00] time that I started down the road of using that and getting that study approved by the IRB at the college.

[00:06:06] Ben Butina, Ph.D.: I share research on LinkedIn usually once a week, usually around Thursday. It is something I like to do every week as part of my own professional development. And it’s kind of fun to share stuff on LinkedIn because you get so many perspectives.

One of the perspectives I got when I share an Andrew’s dissertation was this kind of pushback against grit. My thought about it is that, you know, the main area controversy, at least as far as I can tell, is, is grit truly its own construct or is it a combination of constructs we already know about, like conscientiousness and so on.

For the purposes of your study, I think it’s not terribly relevant because like, hey, let’s say grit isn’t really its own thing and it’s really conscientiousness in something else, or, or grit isn’t what we thought it we’re, we’re still measuring something. Even if that’s something isn’t, you know, the, the variable name that we’ll use five years from.

Or 10 years from now, whatever it is that we think grit is measuring. It did have a [00:07:00] relationship with turnover intention. So I guess let’s start there. You had a lot of hypotheses, but what was your hypothesis related to grit and turnover? Intention?

[00:07:08] Andrew Holter, Ph.D.: Yeah, so I, my thought process was that grit would act as a personal resource and ins.

Workers who were subjected to, pretty stressful work environments and obviously that of being a new nurse a new registered nurse at that would be a, a well documented stressful work environment. My thought was The grittier, these individuals were, the less likely they would be to have turnover, intention.

They would be resilient mm-hmm. And things like that. Especially in healthcare, in public safety. Resilient. Resilient. So kind of saw grit as as a crosswalk. Type construct to like measuring that resilience, how strong somebody would be to persevere and, continue in a workplace.

That was my hypothesis, is that grit[00:08:00] you know, would have a relationship with turnover intention. It would’ve been an insulating one we’re people with grit would be less likely to have that turnover attention.

[00:08:11] Ben Butina, Ph.D.: Your hypothesis though was that it would be a positive relationship that, that when a person has that resilience and, and they have some passion for their position in this case as nurses, that it would be associated with a reduction.

And turnover intention, because this is a person who’s, you know, they’ve got the resilience, they have the passion for what they’re doing. This is the kind of person that we think is just gonna stick with it. So let’s go back to the moment you reviewed your data for the first time. Where, where were you, were you at, at your house, at your apartment, at school?

Like how paint the scene.

[00:08:42] Andrew Holter, Ph.D.: I was at home and the data collection had been process


The sample size needed to make it all work out. And so, you know, I was super relieved. You know, I [00:09:00] felt great. That

[00:09:03] Ben Butina, Ph.D.: was your first, that your first clue that something was about to go wrong. You felt great. You ready? Look

[00:09:08] Andrew Holter, Ph.D.: at your data. A pinnacle moment because it had taken so long to work up to this. And so I, I had used SurveyMonkey so I downloaded that information, everything

Excel. I was just gonna simply score the assessments. Mm-hmm. And you know, I wasn’t even looking yet. The progression analysis or, or putting it into spsf which is what I used for the analysis of it. Mm-hmm. Just simply scoring and immediately I could start to see some interesting things in scoring.

I could see that, you know, the group itself, the sample was largely dissatisfied with their work. I had. Relatively unremarkable scores for things for the, the other variables like grit and self. Nothing really stood out to me. So you, I converted it over and put[00:10:00] its, and the regression and. I must have scored the assessments wrong.

Like, I

[00:10:07] Ben Butina, Ph.D.: So your first reaction is, okay, this can’t be right. Like, oh yeah, you must have

[00:10:12] Andrew Holter, Ph.D.: screwed something up. Yep, absolutely. I reloaded everything probably several times. I rescored everything several times. I went back and read the rules for the scoring. You know, what did I do? I convinced that there was something incorrect somewhere. And unfortunately after probably like 30 times it, I just kept getting the same result. How

[00:10:34] Ben Butina, Ph.D.: are you feeling

[00:10:35] Andrew Holter, Ph.D.: at this time?

Mixed emotions. Yeah. I was I didn’t know. I didn’t know if something had gone wrong and it was gonna be ok?

[00:10:44] Ben Butina, Ph.D.: But eventually, I assume, you know, after you’ve checked and double checked everything you say, okay, well, This is what the data is telling me.

Like how did you try to make sense of that?

[00:10:52] Andrew Holter, Ph.D.: I tried to look for any other studies out there that would’ve found some kind of similar result, something that I [00:11:00] could lean on to support what I had found. If for no other reason in that immediate moment to validate my me and like, like how did I, how did I get this so wrong?

Like, what, what was I thinking? You know, I wrote so many pages about this and, you know, I must have had no idea what I was talking about or thinking or, or whatever. And finally, I spelled it out and I sent it off to my. committee chair. Mm-hmm. And I said, can you just, can you look at this?

I’ve been all the way around this thing a hundred times and like, this is what, this is what I got, this, it is what it’s, . And thankfully was like, this happened, you’re okay. The results are the results. You don’t touch anything. Don’t try to manipulate anything.

This is what you found. And yeah, she’s like, if nothing else, it’s probably even more significant than what you were trying to prove

[00:11:49] Ben Butina, Ph.D.: yeah. I’m glad she said that because I was thinking the same thing. When you had first shared the results of this dissertation, that was my first thought as well, which was, well, if. [00:12:00] Did predict, turnover, intention, be like, okay, great. You know, that’s a nice, neat little package. Good for him, good for the dissertation, all that. But this is actually quite a bit more interesting and it got some really interesting conversation going on LinkedIn as well about like why

it just seems like common sense would say those with higher levels of grit , would stick it out longer, have lower turnover intentions. And, your data showed the reverse, right?

[00:12:25] Andrew Holter, Ph.D.: Obviously the results were, very significant and . The real big relationship in that whole regression model was grit. You know, I going into it job satisfaction was gonna be the, the pretty predictable outcome and the, the fairly stable construct in there.

And I wanted it to be there so that I had something that kind of grounded the whole thing. Something to compare the other two predictors to in that study. And so job satisfaction acted like it was supposed to. And Yep. As people became more dissatisfied, they had higher levels of turnover, [00:13:00] intention but even that relationship was not as strong as the one between grit and turnover intention.

 That too, just added to the surprise element of it all. You know, it wasn’t just a small connection there. It was as strong as you could see in a regression model like that. So that just added to the panic I had at first and, and then the seeking validation from somebody somewhere that I hadn’t messed something up.

But the more I really dove into it and tried to find other research and tried to think about it and I lay that out in, in chapter five of the discussion section of the dissertation, where, you know a person’s grit is aimable and what they’re passionate about and what they’re persistent about is not necessarily their.

And certainly in a post covid era which is when this data was collected people I think were at that point of starting to reevaluate things in their life and maybe look at work a little bit differently. So these folks who had high levels of were saying to themselves you know what, I don’t, maybe I don’t need to be here, [00:14:00] or I can do better.

Yeah. If my workplace is this hostile or this unsatisfactory I can move on and I’ll be okay.

[00:14:07] Ben Butina, Ph.D.: Once you start thinking about possible explanations, then after a while they all start to sound kind of normal too. Like, oh yes, of course. But it’s only through work like yours that we get there.

I, I think as your, your committee, Mentioned, if this had come up with the relationship in the expected direction, it would’ve been one of those things that people read about it and they say, yeah, whatever. Like, we really need science to know this. Everybody knows that this is why we do the science.

Sometimes we get surprising outcomes. And this one was definitely a big surprise for you. And it probably felt like you had a, a model in your mind of how you were gonna finish your dissertation and, and I’m guessing a correlation in, completely the wrong direction was not part of that plan.

[00:14:48] Andrew Holter, Ph.D.: Oh, for sure. I mean it required even a reexamination of the lit review and you know, I had at sort of, at the conclusion, The writing I [00:15:00] was looking for other source material that had found some competing thoughts about grit and how it could be applied in this type of situation.

Whereas the bulk of the research out there is that grit is a great predictor of successful outcomes and positive outcomes, and. Certainly from the perspective of the employer turnover is not a desired outcome. We wanna retain folks as much as they can. . So it definitely required a a pivot there.

 As I pulled myself away from it and looked at it a little bit more objectively, It, made a lot of sense, like, well, yeah, grittier people are gonna see themselves through a bad working situation, and that’s gonna mean leaving more often than not probably.

[00:15:43] Ben Butina, Ph.D.: I’ll be really curious to see if anybody picks up the thread and does some follow up on it. I wanted to ask you, you know, this is a scenario that I think just about everyone who starts graduate school when they know they have to write a thesis or a dissertation that they dread this, they fear it.

And as somebody who has been [00:16:00] through. I wondered, is there any advice you would give to, to students who live and dread of what if the, what if the data really doesn’t

[00:16:08] Andrew Holter, Ph.D.: fit? Yeah. I, I think you, you learn something no matter what. And, that is the goal of becoming a researcher.

 If you’re chasing this career pursuit in the academic sense, and you’re doing that work, You’re becoming a researcher and you’re creating new knowledge, you’re becoming a creator of information rather than a, a consumer or even a commentator of other existing information.

And so in that sense, regardless of what your results are your results are always meaningful to the body of knowledge that’s out there. It doesn’t matter what the results are we’re gonna learn something here.

[00:16:44] Ben Butina, Ph.D.: I’m sure it was genuinely terrifying for Andrew for a little while. But he’s hit on something that is just really genuinely exciting in our field. So Andrew, thank you so much for taking the time to talk today. I really appreciate this opportunity and thank you so much for the advice and the experience

[00:16:59] Andrew Holter, Ph.D.: you’ve.[00:17:00]

Thank you Ben, for the opportunity. I appreciate it. And hopefully just like anything else, hopefully it helps someone out there facing the same situation. So, if

[00:17:07] Ben Butina, Ph.D.: folks would like to get ahold of you to ask you a question or see if you’re interested in, an opportunity working in, public safety or with fire departments what would be the best way for them to get ahold.

[00:17:20] Andrew Holter, Ph.D.: I am super accessible on social media. I’m a pretty avid user on LinkedIn. And I, I think it’s a great space for folks in, in IO to connect with other IO professionals and to share our, what we know with non IO professional which is certainly something of, value to , the working world right now.

So yeah, I, people can easily find me on LinkedIn or, or even Instagram or Twitter whatever, whatever the flavor of the day is. And it’s at Andrew Holter, at Dr. Andrew Holter on any of those platforms.

[00:17:55] Ben Butina, Ph.D.: Thank you so much, and congratulations once again Dr. Holzer. Thank you.