Interview: Jeffrey Snover

This is the everyday life and the reality for millions of developers around the world. Though, not all of them develop ICT systems.
Some of them might design and create your new car, airplane or shoes for all I know. However, collaborating and coordination the flow of information is of the most importance.

The main reason why I called Jeffrey in the first place was this acedemic paper I and two fellow students had, regarding systems developement. Together, we got these amazing answers, and we would like to share them with the rest of the world. I hope that this interview can provide you and your team some new ideas and insight.

Not only “what there is that could go wrong” – but also give you an idea how to solve what have already started to break apart.
I could not think of anyone better for this topic than the keynote speaker from NIC (Oslo, Norway) in 2013, Jeffrey Snover.

About Jeffrey

Jeffrey Snover is entitled as «Distinguished Engineer» at Microsoft and is the inventor of what we today know as Windows PowerShell.
Jeffrey’s current role is the lead architect for Windows Server and System Center Datacenter, where he in his role is responsible for the overall architecture for both products.

You might also want to follow @jsnover at Twitter

Again, I would like to give a special thanks to Arlen Espeland, for her teamwork during the interview, and in the transcription of the interview.

 

• Prior to develop PowerShell you met some internal and organizational resistance.
  We heard you mention something about having to take a demotion during your keynote at NIC in 2013.

Oh yeah, so what happened was, I was the lead architect for all the management technologies and products at the company, and there was a team that was funded to basically have a UNIX-replacement shell. Unix has a shell we should have a shell, and so they were looking to port ”k-shell” or ”Bash” or something to the system. And at the time I had been experimenting with some technology and I said ”no, no, no, you’re doing it the wrong way and it’s not going to work, so here is the way we should do it”. However they did not understand what I was saying, they were kind of odd ideas, so I tried to explain repeatedly. In general, I am a pretty good explainer, but no matter how many hours I explained it, you know, these folks sort of weren’t getting it.

So I thought, “well, okay, I’m doing something wrong” and I said, “Let’s just stop talking for a while, and I’ll go write you a demo”. So I got to work and I wrote a ten- thousand line demo over a period of not very long, and it basically had all the core concepts of PowerShell in it. I than returned to these people and said: “Okay, now, here is what I mean”, and then I’d show them, and their eyes would light up and they ask “what about this?“, and then I showed them that, and their eyes would light up.

So, anyway, by that time I was just totally hooked, and it was very, very clear to me that I was onto something extraordinarily powerful and, frankly, I’d say it was the best idea I’ve ever had and so I wanted to pursue it.

So at the time I, I went to my boss and I said, “I think this is something we should go do”. She replied, “Well it doesn’t really make sense in our organization”. I didn’t care to much and I said, “I understand that, but you know what, I want to’ go do this”, where she replies again “well,  you’re a very senior guy, and this is kind of like, it’s not a strategic thing, there’s is no way we can support a, a guy at your level doing that work, because, its sort of a “checkbox” item“. However, I did not think of it as a “check box” item, I thought this was and a long story short I accepted a demotion to go work on this.

 

• So Jeffrey, you were developing PowerShell, in fact, you are actually the inventor of PowerShell.
  How long time did it take from planning to developing, starting and to actually having a deployable application?

Yeah that was actually quite a long time. We had quite a few obstacles we had to overcome. Not many of them were technical. There a few technical, but not many of them, most of them were organizational, funding and then cultural.

So let’s see I wrote the, the «Monad manifesto» in 2002, and I think our first version of PowerShell shipped in either 2005 or 2006, so it was quite a long time. A part of that was that we were trying to do the development in India. This did not work at all. The project wasted eighteen months on that, and then affectively had to restart the effort, from scratch. You see, none of the code was usable. It was a mess.

Therefore, and we had the «Longhorn» mess, which caused a lot of delay. In general it took quite a long time, but there weren’t any particularly good technical good reasons for that.

 

 

• You mentioned screwing up eighteen months of work. How many people were working on the project for the different periods? How many people were working in India, how many people were working in the States, how many people were working the project “home”?

Oh forgive me, but I am pretty bad at these numbers but let me try.

What we did was originally – and the reason why the project was funded is that somebody wanted to have a replacement-shell, but it was like “not a big deal”. What really happened was we were trying to fund and grow our Indian development center. One of the things we do at Microsoft is we try to find the best talent in the world, and so we wanted to create an environment where we could get this great programmers out of India and not have to make them move to Redmond to work for us.

So we were looking for projects, and it was really funded that way, it’s like, we got a group of people there, and I think there were probably, twelve to fifteen developers there, and then, we were kind of driving and defining it here with maybe five people.

Again, it did not work. The time zones were all broken, I had asked a question and they would get it twelve hours later. They would respond, turned out that I would not get the respond for another twelve hours. It turned out they did not understand the question so I had to clarify the question. It was just horrible. So when we tried to get very, very formal, we did all the things that you are supposed to do, but in end, it just did not really work out particularly well.

So then, when we reset the project I think we ended up with, again, about ten to fifteen engineers here. We got the engineers and test people where there is usually about a “one-to-one” ratio. We also got the PMs, you know, varies.

I am not sure how many PMs we have… Order of like five.

 

• In other words, you had quite a large group actually invested in this project?

Absolutely! It is a major investment for the- the company, it continues to be.

 

• Do you know anything about how many people are working on PowerShell as we speak?

You know, actually I don’t. I like to say that with PowerShell we are digging our way out of a thirty -year hole. Which is to say for thirty years the company went and did everything “non command-line”, everything was GUI-based. Now kind of undo that, and it takes quite a long time.

 

I like to say that with PowerShell, we are digging our way out of a thirty -year hole. 

-Jeffrey Snover

 

Also, at some point the teams were saying “Hey, we got to grow, we got to grow“, this task is too big for us! I kept trying to convince them “no, no, no my friends, that’s not how you solve this problem“. The problem you can never have a big enough team to solve the problem. “What you have to do is you have to figure out how to make the team not go from ten or fifteen, to twenty or thirty, you need to figure out how to make it go to two hundred and three hundred, so figure that one out my friends”.

The answer is, it’s all about leverage, so what we did was we institutionalized some guidelines and some requirements. In return, now all the teams have to implement commandlets, we got other organizations adding the functions etc.

To get to your question. The point is, it is hard now to go and draw a line around the PowerShell team. I mean, I still have the PowerShell core engine team, but it’s spread out.

So it’s really hard to say.

 

 

• Were there any particularly coding, hardware or software issues that you encountered?

No, not really. There is a very sophisticated technology in there (referring to PowerShell) and the use of the object adapter, “the type system”, et cetra, but no, actually, there weren’t. The technology was straightforward. However, that said, I want to give credit where credits do, there is sort of basic architecting and block diagram of things, and then when it came to the language itself, make sure whenever you get involved in these projects, one way to screw them up is to have multiple people all expressing their opinion about the user experience.

So throughout my career I have always tried to find the person that I trusted, told what it is I wanted out of them, and then told everybody else: “Hey, you got an idea, run it by that person, what he say goes“. That is how you get a consistent point of view. So I did that and in this case there were two people – by the way they worked as one – they did just a fantastic job and that was Jim Truher and Bruce Payette. Honestly, these people gave me something that was at least ten times better than what I was hoping for.

 

“Hey, you got an idea, run it by that person, what he say goes“.

– Jeffrey Snover

 

To be honest, I had pretty modest expectations, I was really after an automation engine, great command lines et cetera. What Bruce and Jim delivered was just a fantastic language that spans both simple interactive use and all the way up to system programming.

They just hit that all out of the ballpark.

 

• Let us return a little bit to the development process of PowerShell. Are there any particular developing methods that were used to develop PowerShell, for example XP (extreme programming), SCRUM, or did they decide for themselves what worked out best for them?

We have gone through a number of phases. At first was the traditional waterfall phase – I hated that! At Microsoft, we have some very, very, very heavy processes and honestly they don’t particularly match well with to figuring things out. You know, experiment, prototype, et cetera, so it was pretty difficult. Then at some point, we switched and got on the agile programming, the SCRUM- model, and that worked for a while, but again, whenever you match up with the, the windows environment, right? Windows is the biggest show on earth. Right?

You cannot screw Windows up, if you do something that screws Windows up, you are screwing up a multi-billion dollar a year industry, and so that is why we do things the way we do them. Schedules really matter, time tables really matter and being able to say; “I’m gonna’ deliver you this functionality at this dates, so that you can take advantage of it so you can tell somebody else when you’re gonna deliver your functions”.

All those things are super- important.

So right now we really do have kind of a hybrid- model, where there are times where we do SCRUM, and other times where we do more waterfall planning.

When I started, it was a very clear separation between developers, testers and PM. The developers wrote the code, and then the testers tested it. In my mind that made no sense at all. I mean, the functional code and I said “hey no, we’re gonna’ do this, the developers have to write unit tests for their code”. Then I realized “oh, hey guys. The unit tests have to be done at the same time as the code”. They were thinking they were going to write all the code and at the end of the project, they would write the unit tests.

Oh no, you have to do it at the same time. You have to check them in – together – OK? Nothing gets in unless it has unit tests. Then I had another fight that said “woah – wait, now you have to run the tests, before you check in the code”.

OK fine… Then “No – the tests have to work before you check in the code. You can’t check in the code unless the test works”. Anyway, we had a very strange culture, at the beginning of 2002-2003. Now, we changed the culture and strives to make sure that the developers are responsible for the feature quality, and the test- personnel is more focused on scenarios, performance and things like that, and now the whole company is moving to this model.

 

“Windows is the biggest show on Earth”.

– Jeffrey Snover

 

• For how long does a typical iteration lasts?

Now we are a Windows component, so we follow the Windows- schedule. Therefore, whenever, you see, for the 2012 release, it was a three-year schedule.

I have to tell you, that was tough. We produced a fantastic amount of work in three years, and then, right after, they changed and said “The next one. It’s gonna’ be in a year”. We did a years’ worth of work and each time we release, they look out and decide what the next schedule’s goanna be.

So, that is a bit of a challenge.

 

• What is your experience in developing software from scratch?
  Are there any particular issues you must take in consideration? Any “heads-up”?

Actually, I have worked on all sorts of projects. PowerShell obviously was from scratch and lots of things I’ve done, have been from scratch. However, I have also taken over projects that were, in general pretty bad shape and turned them around. So I have dealt with both. And there are really quite different problems. So starting from scratch, I think most people have the problem of, looking at a blank sheet, and knowing how to begin.

 

“There is a fine line between agile and random”.

– Jeffrey Snover

That is really quite a tough task. We have visionaries that tells us “ooh, we’re gonna have flying cars, and we’ll have computers that know what you want to do and don’t”, you know visionaries. However, it is a completely different skillset to say, “Here is the first step”. I got nothing, here is step number one, then here is step number two, and then here is step number three: That is, is a big challenge.

One of the jokes we have around here is that there is a fine line between agile and random. You know some of the folks who do agile stuff; it is just sort of random walk along the code path.

 

• Let us back up a little bit. You mentioned difficulties with having the engineering team in India and so on, I assume that the time zones, the presence, and the proximity is important. How important is the collaboration tool for development. In general, what is the most important?

First I would like to say we have a number of projects that is successfully working in India, and in general what happens is that – you put your finger right on it, it’s that their given enough freedom, that they can run for periods of time without any synchronization.

What does not work is where you require intense synchronization. So what I experienced, and the reason why we were unsuccessful was:

• One; the team that we hired was junior, and I was very senior.
• Two; they did not understand (uh) management and administration scenarios, so we could not have an efficient conversation. I talked about things and they did not get the basics of how system administrations should be done.
• Three; It was a new technology, .NET, and,
• Four: The approach the architecture is taking was a very radical departure, from anything anyone have ever done, so we were not able to have a short- hand conversation. Everything was a long conversation.

That was one critical thing, and then it was one very funny thing about culture. At the time, and it is probably still the case. India was a very status- oriented society. Where status really matters. The hierarchy really matters, so the fact that I was a partner architect I ended up having from my word, it was like the word of God, and I do not work that way at all. My idea is my job is not to come up with the best ideas. My job is to ship the best ideas, so if you have a better idea than what I have, you got to’ tell me because if it is better – that is what we are going to go with.

 

• Finally, what do you think is the most crucial for a successful development team to succeed?

Oh, let me see. I used to work for Tivoli, and we had a joke that a well-caffeinated engineering- staff was in our best interest the business.

Honestly, so I have worked many different companies. Companies where people had liquor in the office, and then I worked at companies where people got fired once the CEO walks though the parking lot and found booze in someone’s backseat of a car, had them looked up and had them fired.

There is something to be said for all those things, pros and cons, but honestly, I think it is in my view it is simpler than that. Engineers are special people. We are people that love to create and be useful.

Okay, let me tell you a story. When Microsoft called me up, Microsoft recruited me. I did not want to’ work for Microsoft – at all. I did not like their software, I did not like their approach, so what happened was an executive, who I knew and really quite liked called me, I returned the call and he said, “Hey Jeffrey, I hear that you are not interested in coming. I would really just like to talk with you, so would you please come talk to me?”. So I then came here, and what I found was that people was just awesome. So working with awesome people improves everyone’s game.

“Engineers are special people that love to create and be useful”.

– Jeffrey Snover

What really convinced me was the opportunity to come here and have an impact. You see, I worked at other companies where I get the software out and it affects thousands, tens of thousands – maybe hundreds of thousands of people. If you are lucky maybe a few million people. However, at Microsoft we have the opportunity to take our ideas and ship them, at a billion people. It affects the lives of a billion people, and I’ll tell you that there is nothing that compels you to get out of bed, get to work and deliver something great than the fact that if you come up with an idea, and you pursue it, you can have an impact on that many people’s lives. Therefore, I think that is the heart of it.

Free coke is great, I like free coke but, but that is not as important. Honestly, if I had crappy offices, and crappy parking, and a crappy cafeteria, but you still had the opportunity to, meaningfully affect that many people’s lives I would definitely do it!

What it takes projects to succeed? Well, if you look at your business at hand and take a look at the bookstore. You will find these shelf’s with magazines “how to succeed in business” What you will see is that some of these will say that you have to do “a”. Then you will see all the companies that do not utilize “a”, and they still succeed. So then you will see around the books that say, “the way to succeed is “not a””, and there are no real rules in how to succeed.You will have to take it from the human perspective.

 

On behalf of Arlen and myself, we would like to give our greatest thanks to Jeffrey and his willingness to share his knowledge and experience with us. Please stay tuned for more news, articles and interviews in the time to come.

– Alex