Growing Up Comm

Dan Andrew 0:02
ICA presents.

Welcome to Growing Up Comm., a production of the ICA Podcast Network. In this series we talk about topics primarily relevant to students and early career scholars. I'm your host, Dan Andrew, and a member of ICA's Student and Early Career Advisory Committee, also known as SECAC. I'm a lecturer at the University of New South Wales in Australia, where I teach advertising, marketing comms, and a range of other different communications and media units. Today, I'm speaking with Dr. Nick Bowman about article reviewing. Nick is an Associate Professor at the SI Newhouse School of Public Communications at Syracuse University. His research focuses on the use and effects of interactive and immersive media, with specific interest in social media, video games, and Metaverse technologies. Welcome, Nick.

Nick Bowman 1:01
Hey, thank you for having me. I'm excited that this conversation.

Dan Andrew 1:04
Fantastic. I'm going to start with what might seem as a very obvious question, it will help us frame the conversation in a lot of people's head. So, what is the purpose of peer reviewing?

Nick Bowman 1:16
The general notion of peer reviewing goes into a deeper academic principle of value, if you will, that the community owns scholarship and the academic community at large is best equipped to evaluate any one study coming out of the academy. When we do a study, when we do a piece of scholarship, we presume we did it correctly. We presume that it makes a contribution within our academic areas. We're proud of our work and we should be. Of course, we are not always the best candidates do evaluate our own scholarship. And so the peer review process, at the very least, is meant to be a check and balance. Could other peers in your field, in your area, with similar expertise, read your scholarship and arrive at a similar conclusion? Now, it doesn't always work that way in practice, and I can tell you as a reviewer and as an editor, I do feel that it works more often than it doesn't. If we can all agree that a given manuscript is going in the right direction, the hope is that when that work is published, there's a lower likelihood that it's flawed, inaccurate, unethical. Broadly speaking, it's our chance to make sure the scholarship we're producing is going to make its impact today and tomorrow.

Dan Andrew 2:32
So for students, and particularly those who are early-career, there's often a feeling of that imposter syndrome or don't have the expertise or haven't earned the right to review or critique the work of other people, particularly people who may assume have a lot more experience than you. So what would you say to SECAC?

Nick Bowman 2:52
It's easy for me to say this in the position that I sit in: we're either all imposters or we all belong here. As editor, one of the hardest things I have to do is make a decision between scholars that are my senior in expertise and tell them no. And you always wonder if you're making the right decision when you click that button. I point that out to say, we're all there. And I might even go so far as to say that hint of imposter syndrome keeps us honest. There's humility and then there's imposter syndrome. And I always ask people to make that distinction. You should be humble in the face of scholarship. You should be bold enough to ask your own questions and critique it, but humble enough to realize you may not always be right. In fact, in my notes for best practices, I always tell my reviewers that you're one person who's right, but you're not the only reviewer. I hope that's seen as a freedom. It's okay if you make a couple of mistakes in your review, because there's going to be other reviewers, and there's going to be an editor, and there's a process around you. And a good process should correct those things.

As to whether or not you're qualified to review, I think it's a fair question to ask. So on the one hand, if we're saying peer review, you might make the argument what does "peer" mean? And earlier in my career, I used to be very much of the opinion that I discouraged my graduate students from reviewing for journals and encouraged them to review for conferences. I still somewhat hold that opinion. It's less about qualification, it's more about time balance, frankly. You're giving service to a field at the review level for journals when you still have other things you have to do to become a full-fledged member of that field. And I worry about the time commitments and the pressures involved there. For conferences, I think the graduate student and early-career reviewers are essential because a lot of the submissions are also from those sorts of people. So you are really a peer reviewer. In fact, a good conference chair will oftentimes try to calibrate those things. When I was the chair for Game Studies, I would have more graduate student, early-career reviewers on SECAC authored papers, for example.

For journals, I don't think I can provide a clean answer. I can tell you with my journals, I tend not to invite graduate students. And I mention graduate students in particular, it's mostly a matter of worrying about power differentials and time commitments. I think it can be easy to get trapped in the neverending cycle of reviews on top of the other responsibilities. What I've preferred in the past is to work with co-reviewers. So in other fields, it's very common for people to do a co-review, where you review something alongside a faculty member. It's often a conversation with advisors when it's graduate students. When it's early-career, post-PhD, I think it's essential. It's one of the best ways to get involved in your fields. And when in doubt, ask the editors. I think it's one of the best things or pieces of advice I can give is, when in doubt ask the editors directly.

Dan Andrew 5:54
So what are the benefits of it - of getting involved?

Nick Bowman 5:57
There are lots of benefits to getting involved in the peer review process. The one that I've seen in my career is a chance to see what folks are working on. And I don't mean to take their ideas, please don't do that. But it's a good way to see what folks are working on, what types of methods are being done, what are some of the newer theories coming out. If you're feeling impostor syndrome, it's a chance to see the relative quality of the work being produced. Most of us are our own harshest critic. And we tend to obliterate our own ideas and our own scholarship before we ever bring them to the public attention. Especially early in my career, I would read manuscripts and it would make me feel a little better about the work that I'm doing. Like, "Oh okay, well, work of this type is getting submitted for publication in this top journal, maybe I should submit my work there as well." There's the mystique some journals have and then there's the reality of the process. So it helps you see what's being out there, what's presentable.

It is a chance to demonstrate your analytical skills. If there's one thing we don't get to do very often, it's demonstrate our thinking. When we've published, we've already thought, we've already written, it's already out there. And that's great. When we present at conferences, we've already thought. Rarely is the case that people external to your career, get to see you thinking. And in the peer review process, in those private dialogues that go back and forth between the author and the reviewers, that are read by the editors, it's a chance to practice and to make your thinking skill more visible, for example, to an editor. You get to be involved in the field. Many of us feel lost. This happened a lot during COVID when we didn't have some of those social reinforcements that reminded us. It's a chance to be involved in the field broadly. So reviewing helps us sort out our professional identities. It can help us feel like we are part of a broader community. And I think over the last couple of years, we've been reminded about the importance of feeling like you're part of something larger.

I'll tell you what it's not for, it's not a chance to earn points. I think a lot of people either implicitly or explicitly think, "If I do some reviews for this journal and then my paper comes across a couple of weeks later, they'll somehow help me with the paper." It's important to talk about. And so I always tell people try to avoid the notion that it will give you a favor.

Drawback to reviewing, it can reinforce gatekeeping practices. Especially with early-career scholars who might be protecting their identity or protecting their ideas. There can be a motivation, whether it's implicit or explicit, to block other works. That can get really nasty really fast. And I've seen it happen before. Sometimes you're just nervous about your ideas. The imposter syndrome affects you in negative ways. And then somebody else says something about your work and suddenly, there's this like feeling to protect it. And you have to be careful with those kind of things.

For conferences, one of the major benefits is having a hand in shaping the proceedings, having a hand in helping the scholarly community sort the ideas in such a way that when we go to these events, we're going to take something home from them. So when I review conference papers, I often talk about what the presentation might look like. And I try to frame it in terms of where it's going.

And then for journals, for people who do want to get involved on the editorial process, reviewing is the path towards editorial board memberships. That's where those things play a role. I know that's not how it's always been but that's how it's increasingly becoming. Especially with services like Publons that actually track your reviews and quantify those, for better or for worse. For some departments, for some programs, this is service obligation. And then so there is a tangible benefit in terms of demonstrating that you're part of your fields, that you're being recognized as an expert in your field. All of this is to say that no one should ever feel like they have to review. That's a bit of a paradox. But saying no, is completely okay.

Dan Andrew 9:56
If people are interested, where would they begin? How do you reach out to your preferred journals or journals that you're reading?

Nick Bowman 10:03
I'll start at the conference level, just to help folks distinguish the two. Because first and foremost, I always recommend for early-career scholars to get involved with conferences. It's a bit more of a low-pressure, high-intimate space. The reviews tend to be a little shorter. The review structure is different. So, a good conference review is as long as the abstract of the paper. That's getting involved in program planners, business meetings. I've never met a conference that will turn down a reviewer, ever. That's actually where I got my start. I was reviewing for Game Studies at ICA before it was even a division and they were just getting started. For journals, usually, they'll find you. And that's a bugger thing to say because it introduces social biases and complications. What I mean is, oftentimes through your conferences and through your participation in the scholarly community, folks will eventually find people they're looking for that have an expertise in a given area, which also makes people feel left out when they're not invited. Because it's presumed to be a slight or some negative evaluation.

It's okay to email an editor to make yourself available. I'll tell you, if you do it, it might seem strange, only because folks don't do it very often. And in that case, it's just a matter of being tactful and upfront, "Hey, I'm a new scholar, and I'm trying to get involved in the field. And I was wondering if there'd be an opportunity to review papers at some time." Probably clearly stating your credentials. A frank email is more than sufficient to get your name on the roster. If you don't get a lot of invitations right away, it's not necessarily you. You have to remember that we have an editorial board of 80 to 100 people, they're the primary reviewers. Then after them, there's ad hoc reviewers, and that list is the entire field. Sometimes ECRs will feel bad because they haven't gotten an invitation in nine months. And I'm like, that's just because I haven't gotten anything that I need help on. So it's not necessarily a sign of anything nefarious. Reaching out with a frank email, I think, is the only strategy that removes a lot of the strangeness of academia. Keep it short. Be upfront with why you want to do it. I can't imagine an editor getting mad at you for volunteering your services. And a good editor might even walk you through a bit of their expectations.

Dan Andrew 10:03
What are your tips for reviewing?

Nick Bowman 12:17
As much as I loathe word counts, it does help folks, especially new to the academy, calibrate expectations. Conference reviews are shorter. It's an in-progress paper. It's going to be talked about at an event. It's probably going through revisions. It may already be under review somewhere else. Keep it short, keep it high level, keep it forward-looking. If you feel like it should be rejected, be direct in these things, "Here are the things that I think are fatal flaws."

For a journal article, a little longer. I usually suggest a section per section of the paper. So literature review, methods, results, discussion, probably having an abstract-length comment for each one. It's okay to go long. But I don't want folks to feel like length is better. Sometimes it's just rambling. And again, it's time commitment. A journal review does have more impact than a conference review. Respect the work. This sounds very basic, you know, almost paternal. But what I mean is, your assessment as the reviewer shouldn't presume the author is wrong. It's an assessment. It's possible that you could be wrong. And you have to allow for that in your comments. And so oftentimes, I'll write things where I will state that something is, I think, incorrect. I'll justify why it's incorrect. And that is so critical. It is on you to present the counter-evidence. I do a lot of statistical analyses. So I might say, "I don't like how you did this factor analysis. I think there's an error. Here is the specific error, in my view. Here's a citation that supports my view. Here's how I would recommend you do it differently." And then I'll review the rest of the paper, presuming that I was incorrect. Because you could argue, "Well, your stats are wrong, your conclusions are moot." But then you could also be wrong. And so I might even say in my review, "Presuming that I am incorrect on my analysis of your stats, here are some things I noted in your review." This can be hard for lots of people because it's easy to tell somebody they're wrong. But just like a good professor gives feedback, you have to give feedback as the reviewer, for no other reason than if you want to be taken seriously by the editor or by the authors. You can't just say, "Dan, you're wrong," because then the authors can say, "I'm not though." You have to respect the work. You have to be detailed.

On that note, a really critical piece of advice that I give is to mark your comments as major or minor. It really helps you as the reviewer have a bit of freedom to be more nuanced. There might just be something that you don't understand, but it doesn't sink the paper. You might note, "I don't notice anywhere in your manuscript where you include your ethics review." That's probably marked as minor because they can probably address that. The minor/major markings help you give a more nuanced review. They help direct the author's attention. They also present an opportunity for dialogue. How does it feel when someone gives you a list of 50 things you did wrong? And to the author, each one seems equally important. Then the author gets mad because they feel like they're being nitpicked.

This is gonna sound like I'm being picky, but I ask reviewers not to ghost the editors. And what I mean by this - academics have a habit of just disappearing when deadlines get under pressure. When it's a high-pressure, variable deadline job, we start to feel guilty, nervous, we don't want to be judged, for various reasons, we just disappear for long periods of time. And in the face of discussions around equity and inclusion and recognizing people's work-life balance, the best thing to do is just communicate. It's just a simple email, "Hey, I'm going to be a month late with this. Hey, I can't come back to this manuscript." It reduces the uncertainty for both parties. And ultimately, it's the thing that keeps us moving. I know not all of us have time for that quick email. But I always tell folks, "Don't hide from the pressure, just discuss it." I've never told a reviewer, "You're fired." Or I've actually told reviewers, "Hey, if you haven't started, don't worry about it. Just let me know, I'll release you from the obligation and I'll move forward. No harm, no foul." And this is the part of the review process that I think especially early-career scholars don't quite understand. When you agree to review a paper, you're implicitly agreeing to review it until it's published or rejected. And that's not always clear. So if a paper gets revise and resubmit, revise and resubmit, you might be on that train for a while. Honor the request, or tell us just right up front.

Which leads to the most important best practice: say no. It is completely acceptable. You get your call up to the majors, that first invitation to review for the Journal of Communication, and it feels good. That's okay. It's great. You should feel good. It means that your name was recommended, or your email was honored, or for some reason, we want your feedback on this because you're an important member of our community. You can also tell us no. There are so many reasons why we can't do things. What I would recommend, when you do say no - be proud, don't hide it. If you are worried at all about ruffling feathers, which you should not be, recommend others. And I say that because oftentimes as editors, it really helps to know other people with similar expertise to you that might also be quality reviews. It's also a great way for us to overcome biases in the review process. You might take that moment to throw out reviewers that don't get recognized. Maybe they're not in a North American or Western European nation and so sometimes their reviews get overlooked. Or maybe they're scholars you've met at conferences who were just really competent, but for various reasons, haven't felt comfortable talking to journals and being very loud. You might want to tell them you recommended them before you do it. But giving us two or three names is quite helpful. I know it's a little bit of labor, but it goes a long way in helping the system continue to move forward.

Dan Andrew 18:29
That's some really great advice there. Particularly like the importance of saying "no". We definitely have that feeling of wanting to get noticed. It's easy to get caught up in the admin side, and the other things, forgetting that it's our research that's gonna get us noticed. I've only had one paper published. And it's only been referenced a couple of times, but it's been referenced by people I've referenced. And you realize, that's much more important - that I'm involved in this dialogue, in this conversation, I'm contributing to the greater knowledge. But at the same time, having helped reviewed a lot of conference papers, and journals, it actually helped me gain confidence in my own writing. Partly seeing other people's writing going, "Oh, if that's getting in, I can do a lot better than that."

Nick Bowman 19:19
Focusing on who you are as a scholar is the part that's going to make peer review the most rewarding, the most functional, and the most likely to help you become a peer reviewer. There's lots of things in the field I'll never review because they would never possibly invite me. In fact, if anything you want to be known in a relatively narrow expertise, especially in the beginning. If that's your dissertation, or your thesis, or that narrow band. And then start to broaden it out, maybe as you go through your career and you develop expertises and you become more broad. You worked in advertising for 15 years. You could go look at media management papers, but then the question is, do you want to? Is that what you are now? And I was a sports writer before I was an academic. I don't review a lot of sports journalism manuscripts. Other folks have a deeper understanding of it than I do. And it's not going to help me craft my narrative for who I am.

Dan Andrew 20:13
Thank you very much for your time, Dr. Nick Bowman.

This episode of the Growing Up Comm podcast series is a production of the International Communications Association Podcast Network. Our producers are Jabari Clemons, Kate In and Sharlene Burgos. Our executive producer is DeVante Brown. If you'd like to learn more about today's guests and their work, please check the show notes in the episode description. Thanks for listening!