Search This Blog

Tuesday, December 17, 2013

Trust in Agile Environments

The author Frank Crane wrote “you may be deceived if you trust too much, but you will live in torment if you trust not enough.” It is not easy to let go of fear and doubt. Yet, unless you decide to live as a hermit crab, you must release some degree of fear and doubt if you are to function at all in modern society. If you are working as part of a software development team, you must have a great deal of trust in your fellow developers, testers, managers, business analysts, and other team members. Functioning well on any agile team requires a high level of trust.

Types of trust

It may help to define trust by describing what it isn’t. There are two basic types of mistrust; functional and dysfunctional. The latter is an extreme form of mistrust that can approach pathology – feelings of persecution, conspiracy theories, and excessive thoughts on the actions of others. Although distrust has traditionally been viewed as a bad thing, there can be some valuable benefits of mistrust; this is called functional mistrust. For example, you certainly wouldn’t loan your car or lend money to just anyone. We have all had experiences where we misjudged others, only later to be exploited. A certain level of functional distrust allows us to set boundaries, and also to prevent excessive group cohesion which may hinder good decision making.

However, a deficiency of trust can lead to many adverse effects: lack of cooperation, lower employee satisfaction, loss of commitment, and in its extreme form retribution and openly hostile behavior. It is when trust becomes this dysfunctional that we must break the cycle and analyze, understand and treat its root causes.

There are two basic kinds of mistrust: calculus based and identification based. Calculus based distrust is “confident negative expectations of another's conduct grounded in impersonal, arms-length transactions where the overall costs of maintaining trust are expected to outweigh the benefits of maintaining trust.” This type of mistrust is basically a cost-benefit analysis, a calculation of the return on investment of personal relationships. We see this all the time in our daily lives: we might trust the neighbor kid to mow our lawn, but we wouldn’t let them touch our computer. Establishing boundaries, setting expectations, crisply defining roles and keeping the lines of communication open will help to temper this type of mistrust.

Identification based distrust is “confident negative expectations of another's conduct grounded in perceived incompatibility of closely-held values, dissimilar or competing goals, and/or a negative emotional attachment.” In this type of relationship, we see little in common with one another. The other individual or group in such a relationship may even be seen as an adversary. This type of mistrust can be the most damaging as it is extremely personal. Although we would love to minimize our interaction with people who do not hold the same values as we do, we are often forced into situations where we must. On these occasions, using a third party as an intermediary can often help facilitate communication.

Signs of mistrust

Nosce te ipsum. Begin by examining yourself and your motives. Are you a mistrusting person? Here are some signs to look for:

  • Constantly needing to explain and justify your own personal behaviors, viewpoints, and preferences, when others don't need to hear it
  • Having a reputation as "unusually conservative," and being uncomfortable with risks and major change
  • Chronically saying "No" to new situations or invitations
  • Compulsively “mind-reading” others: assuming “I know what you’re really thinking, feeling, or wanting”
  • Fear of the unknown

The second place to look for mistrust is in the environment. Are you in a place that has a low level of trust? Here are some signs to look for:

  • Lack of enthusiasm following announcement of new initiatives by senior leaders
  • Increasing complaints about policies and practices controlled by senior leaders
  • Managers beginning to question decisions and actions of senior leaders
  • Increased grumblings by groups of employees
  • Morale issues
  • Increased mention of senior leaders in exit surveys
  • Active resistance to leader initiatives and change efforts
  • Workplace gossip
  • Nothing can get done without the boss’s approval, or conversely, nothing can get done unless employees go behind the boss’s back
  • Performance reviews are not done on time
  • Face to face communication takes second seat to instant messaging, email or text
  • Too many meetings
  • High turnover

How can you improve trust in your respective teams? Let’s start by examining how trust is broken before we look at how we can build it.

Breaking Trust

It is far easier to break trust than it is to build it. Trust is something we all value, but it is exceedingly fragile: it takes months and years to build, but only seconds to destroy. Here are some sure-fire ways to destroy trust in an organization:

  • Withhold feedback and information – Don’t tell teams exactly what is expected of them. Don’t tell individuals exactly what is expected of them. Don’t give feedback on their performance. Make them question whether they are doing a good job. Don’t give them information they need to get the job done. Hoard information on the premise that people are not competent to understand it. All of this sends a strong message that people are untrustworthy.
  • Dismiss evidence of the team’s capacity – Despite objective evidence of the team’s capability to deliver, impose requirements on them that don’t take it into account. Then, punish the team when they inevitably fail.
  • Micromanage –Based on the assumption that people are basically incompetent, or less competent than you are, micro-manage their work to make sure they stay on track. Which, inevitably, they may not, because of the overhead of the micromanagement.
  • Act and speak inconsistently.
  • Seek personal rather than shared gain – when others come to believe that your own agenda is based on personal advantage, your motives immediately become suspect, and trust breaks down.
  • Lie - Or perhaps worse, tell half-truths - no one trusts a liar.
  • Be closed-minded – don’t actively listen to people. Assume you know better than they do, and that your ideas are always right. Based on this thinking, make all decisions yourself and don’t allow your subordinates to make them; or worse, reverse the decisions they do make. Don’t be open to new ideas and new ways of thinking about problems.

Building and Growing Trust

So how can we go about building trust in an organization? Here are some ideas:

  • Establish and maintain integrity - Do what’s right regardless of the personal risk. Value and practice honesty always in your communication with others.
  • Be consistent and predictable - Keep your commitments. Deliver what you say you will when you say you will. Meet your service level agreements. There should be no surprises at any milestone. Be respectful of other people’s time when conducting meetings.
  • Consider all employees as equal partners – Americans have always been very independent. Our culture celebrates individual heroes. We are far more egalitarian and less autocratic than any other culture on the planet, and when autocratic systems are opposed on us, we tend to rebel. By treating everyone in a company as equal partners, all focused on the same mission and vision, you cannot help but engender a trust based environment.
  • Consult with the others before you make commitments on their behalf – before signing others up for work, consult with them. Listen to the doubters; they will raise red flags and project risks you need to track.
  • Communicate – Make sure the team exactly knows your needs. Communicate often. Be transparent. Be open about changes you are making. Actively seek feedback on everything. Better to over-communicate than communicate too little.
  • Establish boundaries and test them – Have well defined (written) roles and responsibilities for everyone on the team, and actively communicate when tasks come close to the borders of those roles. Know exactly what is expected from every role and every discipline at every phase of a project.
  • Take responsibility – Apologize if it’s your fault. Own up to your mistakes and work to improve.
  • Be empathetic to those around you – Actively listen to others. People trust good listeners.
  • Develop and demonstrate competencies – Always be continually improving in your chosen profession. You will more likely be trusted if you are seen as the “go-to” person in a feature area.

Trust in software development

In a traditional waterfall software development environment, we see the symptoms of the lack of trust: they are document heavy, senior management defines the designs and architecture as opposed to experienced software architects who know how to do the job better, and managers own the command and control as opposed to the teams building the software. For every task there is a methodology that is simple, neat, and wrong, and waterfall is one of those methodologies. Inexperienced managers will often ignore history and fall back to this less agile method because they believe all of the up-front work will save them, or because they don’t trust the teams to build what they actually want.

Agile teams are different. One of the basic precepts of an agile environment is trust and teamwork. Agile teams craft optimal solutions over a period of time, and the people who manage them need to create optimal environments in which they can function. Agile teams iteratively maximize value.

There are basically two different opinions on the subject of trust in an agile environment. The first states that you must initially trust the team to create the software in the first place. The second believes that trust is created out of the environment of the agile team; that trust is built by consistently and reliably delivering incremental value that is visible to stakeholders. Personally, I think that both have to happen: teams have to start somewhere, so they must be imbued with at least some level of trust in order to function. By then functioning well and iteratively delivering on commitments, trust is also iteratively built up.

Trust, But Verify?

We’ve all heard this saying. It was a signature phrase of Ronald Reagan when discussing relations with the Soviet Union. I think this axiom is oxymoronic: the Compact Oxford English Dictionary defines trust as acceptance of the truth of a statement without evidence or investigation. If you have the need to verify that something was done, then I submit that there wasn’t much trust in the first place. But that doesn’t mean we don’t check our work with appropriate QA measures. People make mistakes, and it’s QA’s job to protect our customers from those mistakes.

At the end of the day, trust is a choice. As such, it will always be accompanied by risk. A willingness to take risk with others makes a powerful statement to them. Lao Tzu said, “He who does not trust enough, will not be trusted.” Begin by trusting yourself, then others. We don’t really have much choice if we are to function at all, but the rewards will always outweigh the risk.

Tuesday, June 4, 2013

Agile Milestones


I’ve been engaged in a lively debate at work about the role of milestones in an agile environment.  How and when should they be used?  Is the definition of a milestone in an agile environment any different than in a traditional project?  From the outset, I want to go on record as saying milestones are important, but doing them incorrectly can lead us down a slippery slope away from agility.
We take the concept of a milestone from the numbered markers placed along roads, which provided reference points to travelers, and date back to the Roman Empire.  It is a convenient metaphor to apply this concept to a project management, but when applied to a software development effort, the metaphor sometimes tends to break down.  In the Project Management Book of Knowledge (PMBOK), a milestone is defined as “a significant point or event in the project” (p. 136).  This definition doesn’t really help us in the discussion, though, as further interpretation of the definition often leads to disagreement and confusion.  The PMBOK gives us a little more information, though, when it further describes one attribute of a milestone: whether it is mandatory (i.e. contractual) or optional.  I think we can use this key differentiator to help guide our discussion.

We must first realize that a traditional, waterfall view of projects, which have an “on time/ in budget/manage to the plan” approach, requires deterministic activities to be successful.  A great deal of modern project management still successfully uses this approach to these types of projects; construction is a good example of this.  We can see this in the approach to risk management.  Risk management in construction is fairly well understood – for example, actuarial models exist for most types of construction projects.  Plans for deterministic projects are generally static, the focus is on controlling cost, and the amount of time it will take to complete the project is, for the most part, predictable.

However, software development is a non-deterministic activity: it is a process of change, invention and design, not one of manufacturing.  It has “unknown unknowns” at the start of a project, some of which can never be fully understood until project completion.  Edison filled more than 40,000 pages of notes and tested over 1600 materials before he had a light bulb that would last just 40 hours.  Tesla had been trying to solve the puzzle of a rotating magnetic field for years.  One day while walking through a park in Budapest, reciting Faust, the solution presented itself to him in a blinding flash of insight.  Imagine asking Edison if his project to invent the light bulb were “on time”, or asking Tesla when he would complete the invention of the AC induction motor.  They would have probably scoffed at you.

Fortunately, software development is not completely non-deterministic as it is not purely invention.  We don’t have to re-invent databases, recursion or Unix; we stand on the shoulders of giants such as Jim Gray, Edsger Dijkstra and Dennis Ritchie for these and countless other advancements in computer science and engineering.  We can leverage this knowledge and re-use tools and design patterns when developing new software, and so not have to completely re-invent the wheel every time we develop a new system.  Good developers know a broad array of these design patterns, know how to apply the right tools and technologies to solve business problems, and more broadly align IT with business needs – one of the primary functions of the CIO (or in some organizations, the CTO).

However, software development projects remain at least partially non-deterministic, and they cannot be fully managed with traditional project management methods.  Success must often be defined as delivering as much value as possible in a reasonable timeframe.  Plans for non-deterministic projects are volatile, the focus must be on making tradeoffs between value and cost, and the amount of time it will take to complete the project is either unknown or only known within some degree of uncertainty.

Are milestones in software projects then useless?  I don’t think so.  To understand whether a milestone has value or not, as my friend Kenny Rubin said, we need to look at the economics.  Ask: what is the economic value of meeting the milestone?  What is the cost of being early or late?  It might be helpful to look at two different examples.

Example 1: Milestones with time-based economic impact
In the first example, imagine that a team needs to complete features in a software project for a demonstration at an upcoming annual convention.  If the software is late, the company will lose an opportunity to demonstrate new product features and corporate technology progress to a large audience.  Conversely, if the product is completed too early, then there is also value lost in terms of opportunity cost.  Figure 1 illustrates a simple value curve based on time.  One can of course argue the shape and slope of the curves based on a number of factors: a contract might specify that if a deliverable is late, penalties may be incurred; as well, having the software completed late may still add value (just not as much as if it were delivered on time), but the point is that maximum ROI is achieved by (just) meeting the milestone.


Figure 1: milestone based on economic value

Example 2: Milestones with no time-based economic impact

In the second example, let’s assume an operations team would like to have a tool developed to automate some part of the IT operational process.  Not having the tool is costing the company operational expenses in terms of manual processes that need to be run by hand every month.  In this example, the company is continuously paying the operational cost of the manual process, which for the sake of argument we can assume does not change over the short term.  Figure 2 illustrates this cost curve.  Given that there are probably many other projects queued up on which the team could work, and the team executes them based on the ROI of each project, as long as the development team is reliably producing value at a good, sustainable rate, what is the purpose of having milestones at all for this type of work?
Figure 2:  Milestone ROI with no time-based variability
There are many old-school arguments against this, but the most typical goes like this:  “How do we know if a team is meeting its commitments?”  This sounds good on the surface, but if we analyze it deeply we find an insidious belief: that meeting constraints is more important than delivering value.  The undercurrent of this belief still pervades a large part of the software industry, and agilists must constantly be on their guard against it, and work to educate others about it.  This is not easy; I have struggled to do this for over a decade.  I have not always been successful.
There is a simple reason this thinking is insidious: incentives work.  If the primary incentive for a software development team to meet delivery dates, then they will do their very best to make sure those dates are met.  But the upshot is that, in their quest to lower risk and do everything possible to meet dates, several things happen.  First, estimates become inflated.  What would have been estimated at 2 days now might take two weeks.  Let’s be clear: developers aren’t lying or being underhanded when they do this - they are simply reflecting the non-determinism of their craft back into their estimates in an attempt to meet the success criteria given to them.  Second, Parkinson’s Law starts to operate: “work expands so as to fill the time available for its completion.”  Focusing on constraints instead of value will cause developers who complete tasks early to continue to add features, bells and whistles until time runs out.  We call this gold plating.  Again, developers aren’t being underhanded when this happens.  Who wouldn’t want to say “not only did I get the work done on time, but I went the extra mile and added X!”

When we incent value instead of constraints, this does not happen.  Developers give the best estimates they can and strive to deliver as much value as possible in a given time frame.  The difficulty, of course, is measuring value.  Admittedly, it is much easier to look at a calendar and ask whether a team met a date than it is to do the really hard work of measuring how much value was delivered.  Personally, I think it’s worth it.  Business members of teams need to participate in the process, roll up their sleeves and do the hard work of calculating business ROI.