Interview: Ben Armstrong

What is distributed software engeneering,  and what makes the teams work?
Of course, one may argue that the right toolset – witch is the mind – is all you need to get a good product. But don’t forget the leader!

Whenever something goes wrong you need to fix them, and as my interview with Jeffrey Snover somewhat two years ago pictured – it’s never a walk in the park.

On my quest for more knowledge and still to get my academic papers done, I reached out for Ben Armstrong. This awesome guy is not only the program manager lead for the Hyper-V team. He is also concidered as the virtual-PC-guy!

About Ben

Ben is entitled “Program manager lead” at Microsoft where he have been working on virtualization for more than a decade – making him more senior of expertise than his seniors (my words, not his).

You might want to follow @VirtualPCGuy at Twitter

The interview found place somewhere online 16th of november 2015

---

• So, you’re in Australia?

Yes, I’m at ignite Australia pretty much all this week, it’s actually Tuesday thru Friday and its Monday night here.

• You are traveling A LOT, like, a lot. How is your experiences with distributed teams? 

It’s funny that you mention this. I am on the road right now, and I actually have a new employee starting tomorrow, and I won’t actually get to see him for two weeks. I had coffee with him just before I left. Also, on the topic of distributed teams inside Microsoft we multiple teams that are globally distributed. So for instance the core Hyper-V team is based in Redmond, but the team that delivers Hyper-V replica is based out of the India development center, the team that does a lot of our Linux support is based out of Shanghai.

Hey, at the end of the day, here is where we want to be.

– Ben Armstrong

A bunch of things comes to my mind on this, and the biggest thing is having a really clear understanding and conversation about what success look like at the end of the day. You know, when you have distributed team, you can’t work in a mode where you can assume that, if someone gets blocked on something – they are going to be able to contact you – or, have a conversation with everyone involved.

You need to set people up so that they can go and solve problems themselves, and you need to have conversations where everyone agrees: “hey, at the end of the day, here is where we want to be. Here is the high-level goal that we have and how we get there – we’ll figure that as we go along”.

 

• So the “end of the day” is more like “at the end of the development process”?

Yes, well you know, one of the biggest challenges we got and we spend a lot of time trying to set-up the people and teams like this. You can’t have a situation where every step of the way, people have to check-in on each other and say “hey, am I doing the right thing here? Am I doing the right thing there?”. You need to be able to set people up so that you can say, “hey, in four weeks’ time, we all want to be in this place”.

A great example of this is this; I have a fellow on my team. I won’t say what he’s working on, it’s a secret project, but he’s working on a project, a challenging project for him, and before I left I sat down with him and we discussed what steps he is going to be working on, and what things he was going to be working on next. But, one of the things I was very clear with him about was: When I get back, the primary way I am going to measure whether you were successful or not Is by talking to two of the key people that you work with and seeing if they believe you’ve done a good job.

So if you go a week from today and talking to these two people and they believe you need to change what you’re working on, then that’s the right thing to do. 

• How about culture? Does people get offended by this?

This is the really interesting one for me. Just dealing with the cultural and social aspects of working in environments like that. People undervalue the importance of the social connection. So one of the things I talk to people about is the importance in an environment like this, to schedule time to sit down with people and just have casual conversations so that they can understand what motivates you and understand what is important to you as a person outside of individual projects.

• That sounds like a dream-team to me! So you do travel a lot, and I understand that you are a program manager. First of all, can you explain what a program manager is and how you manage your role as a program manager and assure that things are getting done in these distributed teams?

So first, I’m actually a program manager lead which means I manage a team of program managers. The program manager position at Microsoft is a more engineering- focused version of product managers at other companies. So part of the program manager job is spending time looking at the business, the marketplace, talking to our customers and trying to understand what problems people are facing and features and functionalities we should be building to go after. But then, program managers are also more on the engineering side, responsible to design those features. Often going as far as laying at the user experience and defining API’s and user flows, and we are also responsible for helping just drive the day-to-day management of the project.

So as a program manager lead, I manage a team of project managers and my primary job is coaching them and helping them to understand how to best achieve their job. So one of the things that I always talk to my people about – and this is actually something I work really hard on when I’m travelling – is, the most important thing is to always communicate, to always be in contact.

• Can you be more specific?

In fact I got two examples of this: I’m in the middle of a five-weeks of being on the road and traveling and doing different things. I’m talking to customers and interacting with people, every night I’m writing up e-mails, I’m sending them to the team saying “hey, these are the people that I’ve met with, and here are the discussions that we had”, and I’m also e-mailing my team saying “hey, what questions do you have, wanting me to be asking customers? What information can I be getting for you?”, and just keeping that conversation going.

The other is, even though I’m on the road, my team is constantly sending me copies of their work and I always make time to review and give feedback. If they are on the right path and where they should be going, so just making sure that even when I’m on the road there is a continuous conversation and communication happening.

• So your experience is – you come from a programming background right? Your primary skill is programming (Ben acknowledging with a yes), so tell me a little bit more about that.

So I started out as a developer and developed on a bunch of different systems. I actually started out primarily doing database development, and being a developer, you are very much focused on the technical problem and a lot less focused on the human side of things. Over time I got into the program management position because what I find most interesting is not actually the core software itself but more how what we do with the software affects people.

When people use software, they don’t actually read the text on the screen.

– Ben Armstrong

You know I spend a lot of time chatting to people about the fact that, as a developer and as an engineer, you look at computers as very cold – logical things, but the reality is that people use computers emotionally. We don’t use them logically. We get angry at our computers, we get happy at our computers, we talk to our computers. Even when they don’t have voice recognition.

These days I drive many developers crazy about this, but, one of the classic things that I point out when we’re developing software is: By in large, when people use software, they don’t actually read the text on the screen.

It’s fascinating! You get developers who aren’t used to focus on human aspect of it, and if there is a technical problem, their go-to solution is to “we’re just gonna put a lot of text on the screen, that tells the user exactly what they need to know”, and the reality is if you study how people actually use computers, people will try and figure out what button they should be hitting intuitively – without reading any text on the screen. They’ll only read the text on the screen if they genuinely can’t guess what the right thing to do is. We have had countless examples of that over the years, you know, people doing strange things because they just haven’t read the text on the screen.

But it’s lots of stuff like that I really enjoy. I like understanding how people use our software and understanding what we can do to make it better.

• …You just explained my experience with my camera! But, what you are saying is, communication in teams that is distributed is like a continuously conversation that goes on and keep that going?

Yup! You know, the other thing to highlight and this is actually not so much with my immediate team, but with the teams we have in India and China. It’s really important to account for cultural differences. A number of years ago I got to travel and visit the teams in China and India, and in both places I got to spend a couple of weeks actually in their offices. Just sitting with them and understanding the different cultural norms. Because when you are not aware of that, it’s easy to have accidental miscommunication. People just believe that words have been interpreted one way or another.

Just being aware of the cultural differences is really important. One of the simple tools that I love for that and witch I use all the time is, if I’m talking with someone or I’m e-mailing with someone and I believe that we’ve just agreed on something, I actually the time to explicitly say; “OK, just so that we’re clear – we just agreed to do is A-B-C” and just spill it out.

Often when I’m dealing with a new team, or even a new person from a culture that I’m not used to, I have about fifty percent hit rate on that. And it’s fascinating. You know I’ve had new members to the team who came from a different countries and with a different background, and I’ve feel that we have had a great constructive conversation. 

I get to the end and I say like; “just so we’re clear, we’ve just agreed, we’re going to do A-B-C” and the person looks at me and It’s like “n-noo? That’s the opposite of what I just said”. So just being like really aware of those differences is really important.

• You got an example of cultural misunderstanding that you got from first-hand experience?

I got so many! In fact, when I was working on the feature dynamic memory, which was a feature in Windows Server 2008 R2 Service Pack 1, one of the developers I worked with was a fellow from Germany and it took us months. You see, I’m from Australia, he was from Germany, it took us months to figure out how to communicate and it was really funny because people who knew both of us would say that we’re both really intelligent guys, we had a strong background but time and time again this fellow would sit there tell me about the approach he was going to take, and if I sat and tried to tell him back what I just heard – he would be like “no, that’s not what I said at all!”.

Iit took us a bunch of time to just work through that and understand each other’s communication. 

Conversely I mentioned we have the team in India. The Indian culture has a very different view on the chain on management and communication than the American. So in the American culture, if I’m working with a partner team and we make a decision where we say “okay, we’re going to go down this path”. If my manager then comes in and disagrees with me, I’m going to respond and say “actually, no. We’ve already made this commitment to this partner team and if you, manager, have an issue with that, you need to go and talk with the partner team because this is the agreement we have”. In the Indian culture however, if you’re in that situation and your manager comes in and discourage them, then the manager’s right.

You go back to the partner team and say, “sorry! You know I know we agreed to this, but we don’t agree to it anymore”. And that’s not a positive or negative, it’s just the way the culture is. But if you’re not aware of that, both sides can very easily offend the other side without realizing it.

It is important to be able to tell the difference between someone being careless or inappropriate versus someone who is just behaving the way you would in that culture.

 

• So I got kind of a follow-up question to that one. Some claim that distributed teams are functioning worse. You mentioned culture, why does distributed team sometimes function worse, or when is this the case?

So when do things function worse? Let me thing through this, I want to give the answer right here (Ben pauses for a couple of seconds for the first time!). One of the things I find hardest in a distributed environment is resolving conflict when it comes up. One of the things I often talk to people about is “it’s really hard to get out of a fight on e-mail. It’s really easy to get into fights on e-mail, but it’s really hard to get out of a fight on e-mail”.

it’s really hard to get out of a fight on e-mail.
– Ben Armstrong

I chat to people about a lot about this and say; “Look, you can use e-mail to discuss, to plan, to coordinate – as soon as there is a disagreement – if you can’t resolve the disagreement two or three e-mails. It’s time to stop the e-mail and set up a phone call”.

I’ve seen things go pretty badly where people are using e-mail, and e-mail is terrible at conveying emotion and intent, and I’ve seen situations where we have developers who both have the same common goal and are practically yelling at each other over e-mail.

Just getting them on a phone call, getting them in a conference call and take the time to highlight “hey, we all want to do what’s right here”, and we actually all agree at a high level what’s right and just kind of get people to calm down. That I one of the hardest things. Once conflict is in the open – and conflict does happen because we’re human – once the conflict happens, getting people to calm down, come to the table and negotiate is tricky when people can’t – you know – when it’s not easy to just walk down the hallway and chat to someone.

• From a system developer perspective, that was not what I expected to hear, but from a psychological or behavior thing – it kind of makes sense…

…and the reality is, it’s one of the things I chat about –  even for system developers – one of the things that we often overlook is if I or another engineer puts together a design proposal where we say that we want to solve this problem. We want to solve it using the following techniques and the following approaches. The reality is, and you can’t avoid this, is that some of your own ego is tied up in that proposal.

If people don’t like it’s not that they don’t think any less of me, it’s that they have other ideas.

– Ben Armstrong

As soon as people criticize it, it’s very hard to not see that as a criticism of yourself. It’s a skill to be able to say; “you know what, it’s just an idea, and if people don’t like it’s not that they don’t think any less of me, it’s that they have other ideas”. But it is, I would say the vast majority, of conflicts that I see on the engineering team – it’s so more a conflict of ego than of actual engineering design or intent.

Thank you Ben for your willingness to share your knowledge and experience with us. Please stay tuned for more news, articles and interviews in the time to come.

– Alex