0:00:05.200,0:00:09.840
I'm going to be talking about a study on 
the perceptions of, and the impacts of,

0:00:09.840,0:00:15.200
destructive criticism in software code 
review, and I'm hoping that I can convince you

0:00:15.200,0:00:18.640
to be more careful about not giving 
destructive criticism in your code reviews.

0:00:20.960,0:00:28.560
So I think you'll all know what software code 
review is - it's where developers are reviewing

0:00:28.560,0:00:35.040
other people's source code contributions.
And this tweet illustrates a potentially

0:00:35.040,0:00:40.480
problematic viewpoint about software code review, 
and it says that software code review can be

0:00:40.480,0:00:46.960
honest or nice, pick one, so indicating that 
you can't have both honest and nice code review.

0:00:49.360,0:00:54.160
And the goal of my study was to check if, 
you know, what really are the perceptions of

0:00:54.160,0:01:01.120
destructive code review, which is code review 
that is both inconsiderate and non-specific.

0:01:01.920,0:01:06.320
And i wanted to see, you know, if 
there were different perceptions around

0:01:06.320,0:01:12.480
these honest - destructive criticism - those that 
aren't considering the nice parts of code review.

0:01:13.920,0:01:19.280
And so to do this we used an online 
questionnaire - we used vignette style questions.

0:01:19.280,0:01:24.800
So basically we developed these hypothetical 
scenarios and we asked developers - our

0:01:24.800,0:01:30.800
participants - to look at a code snippet, imagine 
that they've just submitted this code snippet

0:01:30.800,0:01:37.120
for review, and that they got some feedback.
And so some of the developers saw constructive

0:01:37.120,0:01:43.120
version of the feedback and other participants 
saw destructive version of the feedback, and all

0:01:43.120,0:01:49.120
of the participants saw two different scenarios, 
so one with constructive and one with destructive.

0:01:49.120,0:01:53.200
And we asked them a series of questions on 
how they would react to the feedback - how

0:01:53.200,0:01:56.720
they perceived it - and I'm going to 
show you some differences between the

0:01:56.720,0:02:02.960
constructive and the destructive perceptions.
And so here you see these two graphs.

0:02:02.960,0:02:10.000
The first at the top is showing reactions 
perceptions of constructive criticism and

0:02:10.000,0:02:16.080
the bottom is showing the destructive criticism.
So the first three questions that we asked around

0:02:16.080,0:02:19.440
the feedback that they received 
in these hypothetical scenarios

0:02:19.440,0:02:24.240
was, would this feedback help improve your 
code, is it valid, and is it appropriate?

0:02:25.840,0:02:30.480
And you see at the top - the constructive 
criticism - the graph is mostly green.

0:02:30.480,0:02:33.760
So green indicates the participants 
are agreeing with these statements.

0:02:34.640,0:02:38.240
So at the bottom you see a lot more gold, 
which is indicating that our participants

0:02:38.240,0:02:41.760
are disagreeing with these statements.
So participants see destructive

0:02:41.760,0:02:47.440
criticism particularly as less appropriate.
They did agree that it was mostly valid criticism

0:02:48.000,0:02:51.920
but that it wouldn't help them improve their code.
They were disagreeing that it would improve their

0:02:51.920,0:02:56.880
code even though the feedback is valid because 
it's presented in an inappropriate manner.

0:02:59.760,0:03:04.720
We also asked them how motivated 
they would be to continue working.

0:03:04.720,0:03:09.040
And so here you see at the top we've got again 
constructive and at the bottom destructive.

0:03:09.040,0:03:13.360
And we see that overwhelmingly our participants 
are motivated to continue working on the task

0:03:13.360,0:03:17.840
with the developer and on the project 
after receiving constructive criticism.

0:03:19.040,0:03:24.560
On the other hand for destructive criticism we 
see that participants report being much less

0:03:24.560,0:03:29.280
motivated to continue working, particularly 
for the developer, but even on the task and

0:03:29.280,0:03:33.040
on the project as a whole.
So if you have this culture

0:03:33.040,0:03:37.520
of destructive criticism, what we learned 
from these two slides that I just showed,

0:03:38.240,0:03:43.760
you know, people are less likely to improve their 
code, even if they're getting good critiques of

0:03:43.760,0:03:47.760
the code, if it's not presented in the right way.
They're less likely to actually improve the code

0:03:48.480,0:03:52.800
and they're also less likely to stay 
and continue working on the project.

0:03:52.800,0:03:58.160
So we've shown that destructive criticism 
doesn't have the best impact on your team.

0:03:59.760,0:04:05.120
We also looked at demographic differences so we 
asked participants some demographic questions

0:04:05.120,0:04:08.640
and we saw some differences.
So particularly women, they

0:04:13.520,0:04:16.320
perceived destructive criticism - 
less motivated to continue working

0:04:16.320,0:04:20.720
particularly with the developer, but also on the 
project after receiving destructive criticism.

0:04:21.680,0:04:25.120
I think this finding is particularly 
important because we know that

0:04:25.760,0:04:30.880
women are less than 25 percent in big 
tech companies, and so this could have

0:04:30.880,0:04:36.160
implications on retention of women if you have 
a culture of destructive criticism on your team.

0:04:37.600,0:04:41.280
Another interesting difference that 
we saw is, less experienced developers

0:04:41.280,0:04:47.200
were less likely to call out this bad behavior.
So we asked the participants to also,

0:04:48.160,0:04:52.720
you know, write a response - if you were to 
to write something in response to the feedback

0:04:52.720,0:04:56.960
that you were given, what would you write?
We saw that our more experienced developers

0:04:56.960,0:05:02.320
were more likely to say, you know, this feedback 
is not appropriate, it should be written in a more

0:05:02.320,0:05:08.080
considerate manner, whereas our less experienced 
developers were actually more likely to apologize

0:05:08.960,0:05:14.960
for writing the bad code and to agree with 
the feedback even when it was destructive.

0:05:14.960,0:05:18.320
So again this can have important 
implications for retention on your teams.

0:05:20.400,0:05:23.840
Okay, we also asked about the 
frequency of destructive criticism.

0:05:24.400,0:05:28.720
We asked participants how often 
they gave destructive criticism

0:05:28.720,0:05:32.720
and also how often they received it.
And you can see that our participants

0:05:32.720,0:05:38.080
reported to receive much more destructive 
criticism than they reported to give.

0:05:38.080,0:05:43.760
So 22 percent of participants said that they 
received some inconsiderate feedback in the

0:05:43.760,0:05:50.080
last year, whereas only one of our participants 
or one percent said that they gave destructive

0:05:50.080,0:05:55.920
criticism - inconsiderate destructive criticism.
Now this can be for a variety of reasons,

0:05:55.920,0:06:01.280
but one of them could be that people just don't 
know when they're giving destructive criticism.

0:06:01.280,0:06:06.960
So we need more awareness mechanisms to know how 
our feedback is being received by the recipient.

0:06:09.120,0:06:12.240
And finally we asked about general 
opinions of destructive criticism,

0:06:12.880,0:06:15.360
and this one's interesting because 
we see some differences here.

0:06:15.360,0:06:20.880
So the first two questions ask, destructive 
criticism is harmful and destructive criticism

0:06:20.880,0:06:25.200
will cause a negative reaction for this recipient, 
and these are lines A and B on the chart.

0:06:26.000,0:06:29.280
And you see that overwhelmingly the 
participants agree with these statements:

0:06:29.280,0:06:34.560
destructive criticism is not great, right, when 
you ask it generically, but then when you ask

0:06:34.560,0:06:40.400
a slightly more nuanced question - so question C 
says, when receiving code review comments I don't

0:06:40.400,0:06:45.040
mind getting inconsiderate feedback as long as 
the feedback helps to improve the code quality.

0:06:46.400,0:06:51.120
And here we see a much bigger difference.
So there's a good number of people that are

0:06:51.120,0:06:55.280
disagreeing with this statement but still a 
significant number of participants that agree

0:06:55.280,0:07:01.040
with this statement: they're happy to be abused 
and get inconsiderate feedback as long as it's

0:07:01.040,0:07:05.200
going to improve the code base.
We saw quite similar

0:07:06.000,0:07:10.240
comments in our open-ended responses.
So we asked some open-ended questions about

0:07:10.240,0:07:17.200
opinions as well, and we saw some comments to 
these questions along the lines of, like, I don't

0:07:17.200,0:07:22.720
mind getting inconsiderate feedback, I just prefer 
direct wording and easier to parse feedback,

0:07:22.720,0:07:28.000
whereas other participants said things 
like, it always needs to be constructive,

0:07:28.000,0:07:31.600
you really need to have considerate 
feedback every time you give feedback.

0:07:33.200,0:07:37.760
And so what are the big implications 
that I'd like you to take out of this

0:07:37.760,0:07:42.160
is how you treat people matters.
So if you have a culture of destructive criticism

0:07:42.160,0:07:46.640
people are less likely to improve the code and 
they're less likely to stay on your project,

0:07:46.640,0:07:50.400
so your code quality will suffer.
I also think you need to have

0:07:50.400,0:07:53.760
pathways for feedback.
So we saw that much less

0:07:53.760,0:07:58.800
people reported giving inconsiderate feedback, so 
we might need ways to help people understand when

0:07:58.800,0:08:04.800
they are giving inconsiderate feedback so that 
they can improve in how they give their feedback.

0:08:05.600,0:08:09.360
I also think that there's not a 
one-size-fits-all approach to code review.

0:08:09.360,0:08:15.440
So you saw that there were differences in opinions 
in how destructive criticism can be accepted, and

0:08:15.440,0:08:21.360
some people prefer direct feedback whereas others 
would rather have feedback that helps them learn.

0:08:21.360,0:08:26.160
So considering the needs of the person that you're 
giving the feedback to is also really important.

0:08:26.160,0:08:30.720
And finally you have to be careful with what is 
made public: some of the comments - the open-ended

0:08:30.720,0:08:35.120
comments - also pointed that destructive 
criticism is particularly harmful when it's

0:08:35.120,0:08:40.080
being made in a public venue, so considering the 
type of feedback that you're giving and whether

0:08:40.080,0:08:44.960
it really is appropriate to be transparently 
shared across the entire team or if it needs to be

0:08:45.520,0:08:48.080
discussed in private with the person 
that you're giving the feedback to.

0:08:48.720,0:08:53.680
And thanks again, I'm very happy to discuss 
this study, my contact information is here.

0:08:53.680,0:08:57.680
Thanks to all of my co-authors on 
this paper - Sanuri, Peter, Isabelle,

0:08:57.680,0:09:06.000
Lola, and Emerson - and thanks 
to Google for funding this work.