So once again it's a pleasure to be here Greg, thank you for inviting me, it's a great opportunity to be in this event.
So today I will talk about a report that I conducted with some colleagues from Federal University of Amazon, and it's about padding and negotiation and software estimates.
So it's well - it's well known in software engineering, stakeholders take the effort needed to execute a task, as well a project, or to to predict the value that can be delivered to the customer, so they can have the outcome that they are expecting.
The estimators, they bring this perspective about the factors that would affect somehow the estimates, looking for improving the accuracy when they are discussing how much effort and how much time they will need to put - they will need to take to develop what they - what they need for the next release.
It's important to understand here that this estimation process, it happens in social settings, and as everything that I do, it involves something related to behavior and social aspects inside of software engineering process.
So why is this social?
Because it involves not only people working on that but also the context where they are, they are immersed in..
So the literature about estimates is extensive.
So we can find things from the 70s until like papers from last week talking about software estimates.
And what we found when looking into the literature is that there is a broad range of factors affecting estimations, from in fact from early estimation, so at the beginning of the project they have early estimation - estimates coming in, and this is taken into account when they are defining the estimates for the next release or the next sprint, whatever you - you call it, until things related to changes in requirements, change in scope, the project flexibility, etc.
And this goes through a lot of - a lot of stuff that happens in the project, like when when you're estimating, you're executing, performing this - estimation, this estimation process.
And in this point, we have a bunch of social things happening, contextual things happening, which includes for example pressure from high management, from clients, from the PO.
It includes the team itself, how is the relationship between the team's expertise, seniority where these people are coming from, so there are multiple kinds of factors affecting the way that people estimate software.
To better understand this phenomenon in a more recent fashion, we conducted some observation studies and a couple of interviews - a few interviews with people from different countries, different companies, different teams.
We took like three months observing four different teams during their estimate sessions, and our goal was to understand how software estimates were used to establish common commitments regarding the business values.
So we just wanted to understand like, what's - what's tying these two things - the estimates and what is being delivered, what has been - what the team is being committed to do.
And it ended up that we found out that estimate negotiation and padding were key things happening in the process.
So it was quite impressive the number of things that we could tie to negotiation and padding during this process.
So what we see is that - what he saw is that the teams that we analyzed and the people that we talked to after, because we talked to many more people from three different companies later on, is that all of them follow some kind of agile practices, like planning poker-based estimates, where people come with estimates from the team, so they discuss the task and each of the members comes up with an estimation, and they discuss to - to come to a consensus.
Okay, and each of the teams has a different kind of resolution strategy when you have, like, disagreements in terms of estimates.
But at the end of the day, what what we understood is that the PO and team lead just want to know the arguments that they need to have to discuss it with the - the next layer, with higher management or the client.
So they are seeking to understand how they can build commitment bringing the arguments that the developers are bringing - are bringing them.
And interestingly some people do not even argue.
Some people, depending on what are the other people that are estimating and what the PO has to say, they don't even disagree, they just gonna say, okay, I can change my my estimates if this is better.
So sometimes what we saw is that there is a a kind of power relationship going on.
It's like okay, we need to deliver this by next week, so if we don't - if we don't change the estimates we'll not be able to do.
Or sometimes there's a very senior person who's like, okay, this is quite hard and I know that it's hard and some some other people who are more junior, they will just accept, so without arguing against or in favor of something.
So the way that estimators defend is quite important because this is what is giving the - the ammo for the next layers, and we saw that, at the end of the estimation process, we talked it to the POs and other layers to understand, like, what was going on.
And we saw that there was kind of a mismatch, and each layer have their own goals with what was going on in the estimation process.
So one thing that we noticed, that they played by the book when we were talking about project management.
As you know that risk management is something important, and in order to better manage the risk, it's common to bet a little bit.
So we have a contingency buffer, sometimes, like, as you can see here in the examples, when we cannot define the feature very well, when there are some hidden aspects that we don't know about, it's a different API that we need to consume, all right, we don't know which what kinds of problems we have to face in this release.
So this is a common - and this is completely understandable.
However what caught our attention is that we are doing more than that.
For every - everything that we talked to and observed, we could see that we are doing different - different things that result in padding and quite often - and what we are doing is actually hiding details from the next level.
So for example we found a couple of cases where people just said like, okay, we could not finish what we promised in the previous release in the previous sprint, so we'll just add some more time here to this task so we are able to finish something else.
Or we need to adjust whatever - whatever it is in the UI from the previous feature that they delivered, so let's just add something here so we can do that.
Now we need to set up a new server, so why not adding some time to this - this and these tasks.
And also things relate to related to technical debt, for example, or improving the overall quality, maintainability of the process.
So it was quite usual to hear like, okay, let's do this because we need to refactor a couple of artifacts, so let's just add some time here add some effort there.
What - what is behind that, like, everybody is hiding from the next level just because they need to deliver something that is not a direct kind of benefit or business value that can be easily shown to the customer.
So what I've learned here is that padding is used in the right way in many many cases, but it's sometimes used as a social way of hiding things.
So people just lie informally to make things fit because they know that there are other things that they can do and everybody who is listening here, who is watching to this talk should be like, okay, we do that it's common, yeah it's common, but it's kind of a bad smell if we think about the process - of the software process.
And accuracy is in harm when you think about that because we don't know exactly what we are doing, and we don't know exactly how our team performs.
What do we need to do?
We need to sell everything as business value, and this needs to come from the bottom.
So we need to expose the reasons as estimators, as developers, say, like what are we doing, are we refactoring, are we improving maintainability, what are we doing?
And in the next level, like PO level, team leader level, we need to describe these reasons that developers gave us as somehow customer value.
So in the next release this will be something better, we will improve our performance, the - the product will have a better performance.
And for the next the last level of management, when we're in touch with the client, we need to educate them - that there are some things that are not exactly new features, that are not exactly direct benefits to the product as they see, but it's something related to the process that will make it better from that moment on.
I hope you liked that and thank you very much again Greg and Mike for inviting me.