Saturday, September 23, 2017

Agile Coaching for Automation Success

If you'd like to discuss how I or a colleague can help you with Agile and automation adoption in your organisation (and avoid "the Agile Trap" discussed below) send me an email: dan@skillfire.co

Introduction

The central thesis of my talk on Test Automation and Agile Coaching (slides below) is that success in automation is 80% mindset, skill and culture and only 20% processes and tools.

My claim echoes the first of the four core values of the Agile Manifesto, that "we have come to value Individuals and interactions over processes and tools".



Backtracking a bit, why is automation a key to Agile success? Simply because in Agile (and its descendant, Continuous Delivery) we seek to deliver quality software to our customers in a constant stream of small, continuous batches. To do so without compromising quality we must constantly run test suites to quickly detect (and then remediate) any regressions. Running and re-running test suites is a task much better suited to computers than humans, who get bored by repetitious work, make mistakes, and are relatively slow and expensive.

Now, if you buy that argument, you may proceed to the head of the class and skip the rest of this article. Presumably, your automated test suites match the pyramid profile on the right, and you're good to go!

AgileTestingPyramid2.jpg

You have successfully made the argument that speed comes from quality, your developers understand that writing unit tests enables fast feedback loops and helps them sleep at night, and you are presumably delivering software into production every couple of weeks or faster. (For example Amazon, as of 2014, deployed new code on average once per second.)

However, for most organisations, their testing profile looks more like the ice-cream cone situation on the left, and this has likely come about because the prevailing mindset and culture within the organisation is resistant to the logic and many successes of Agile software development, which is perceived as deeply threatening and irrational.

A precedent: the Copernican revolution

The situation is not unlike the stand-off in the early 1600s between Galileo and the Church. Galileo championed the idea that the Earth revolved around the Sun (rather than vice versa), a theory previously advanced by Copernicus. While not being burned at the stake, Galileo was forced to recant his claims and live out his life under house arrest, despite having convincing evidence (essentially incontrovertible scientific proof) for his claims.

f-d66117b7502d406bfdd8864adfff4d28a49693c8aebd239d82baa9a54IMAGE_THUMB_POSTCARDIMAGE_THUMB_POSTCARD.1

Now, the important point here is that Galileo's Copernican view was deeply counterintuitive on a naïve reading, while the Earth-centric view is fairly intuitive: from the standpoint of most earthlings the Earth does feel immovable, and it is correct that the moon revolves around the Earth, and it appears that the Sun and planets similarly move around our fixed planet.

To appreciate that the Copernican model is superior requires an understanding of the limits of the Earth-centric model, and how the counterintuitive aspects can be reconciled.

Have human beings improved at integrating new(ish) ideas that go against the conventional, societal wisdom? It doesn't look like it to me!


* * *

Today there are many such mental shifts required to master Agile and Lean ways of thinking and acting. One example is valuing flow over busy-ness: the illusion that if we are busy at work we must be making a solid contribution. Henrik Kniberg calls this phenomenon the Resource Utilization Trap and has made a brilliant 5 minute video to dramatise the common misunderstanding.

The Agile manifesto lists four mental shifts out of the box, including the main subject of this post: valuing individuals and interactions over processes and tools.

Here's a critical mental shift I went through early in my software development career: the shift from manual testing after writing a program, to writing self-testing code in advance. Writing test code first is a bit more expensive in the short-run, after all you are writing more code, but it leads to better designs and pays off spectacularly as your code-base grows ...


My Story: How I Learned to Write and Love Self-Testing Code

When I switched from academia to software development in 1997 — making this something of a 20th anniversary blog post — I was appalled by the sheer amount of time I spent in the unsatisfactory activity of finding and remedying defects. Design and programming new features seemed to be value creating exercises, but owing to my imperfections as a programmer (and I am by no means a slouch) I was routinely spending about half my time finding and fixing bugs, both defects of design and implementation: i.e. the crash the program type.

Worse, fixing one bug might break other parts of the program, and I wouldn't find out about this for a considerable period of time. This built up a background stress of not knowing whether the programs I worked on actually worked.

The less said about how I felt about finding defects in other programmer's code the better!

Now, being a hitherto competent academic I figured that there must already be a better way and started searching the existing literature, and soon discovered Design by Contract. I immediately tried it out using my existing, mandated, substandard tools (cough: Visual Basic 5 & 6) and was pleasantly surprised to find that the time I spent debugging plummeted by about 90%! I still made mistakes, but I could now find and fix them really fast. Because I made a relatively small investment in up-front self-tests, when something went awry the computer was helpfully pointing me to the location of the defect, often giving additional insights into the nature of the defect. As a bonus, the added discipline helped clarify the design of my programs, making them more readable, maintainable, and well-structured.

In essence Design by Contract involves documenting assumptions with assertions (simple code that can be read by a human, and executed by the program). Here's an example, written in pseudo-code to transfer money between bank accounts:

define transfer-funds (from, to: Account; amount: Money)
    require
        non-negative-amount: amount > $0.00
        sufficient-funds-in-from-account: from.balance ≥ amount
    do
       // code to transfer the funds
    ensure
        correct-amount-sent: from.balance = old from.balance – amount
        correct-amount-received: to.balance = old to.balance + amount
    end

In this example the require clauses tell the computer to halt the computation if a zero or negative amount is to be transferred, or if there is an insufficient amount of funds in the from account. In either case blame can be correctly levelled at the calling code.

On the other hand, the ensure clauses check that the correct amount of money has been subtracted from one account and placed in the other account. If one of these conditions are violated, the local code (shown in orange) is to blame: it didn't deliver on its promise.

The net effect of systematically following this discipline and more advanced aspects of Design by Contract is to make highly readable and code that is self-checking. Bugs are found fast, and usually easily remedied.

Do you notice that the code that does the work is omitted? With well-chosen assertions the job of the code is make sure the assertions all pass when given valid arguments. Interesting! The purpose or why is contained in what's expected (required) and what's done (ensured).

Occasionally there is a defect in the assertions themselves, but in practice these tend to be fairly straightforward to find and fix. 
Another objection that is sometimes raised is that these checks (especially the ensure checks) can be quite computationally expensive. While there is some truth in this, computers are now so fast that a lot of checking is quite affordable, and there are trade-off strategies that can be brought into play when these expenses become a problem: typically slower, more thorough checking against a test suite to flush out problems early, and lighter use of expensive self-tests in production.

Back to the story: I was overjoyed by my new practice of this more effective way of programming. I was having more fun, had become more effective, enjoyed the aesthetic improvements to my programs and had far more confidence in their reliability. [A few years later, when I learned about the similar, but different Test Driven Design / Development (TDD) I was happy to find a second, related technique that gave similarly positive results. There are others in the family.]

But what was interesting (and again, appalling) was that my colleagues did not follow my lead. It was all too much trouble. The expression "bondage and discipline" may have been used. Sadly, this reaction is not unusual in the programming profession.

And don't think management was of any help: at the time they didn't "get it" either. Have you ever tried persuading non-technical management to permit pair-programming (let alone mob programming)? These qualitatively beneficial and (net) cost-effective techniques suffer from being counter-intuitive and only make sense from a more sophisticated perspective.

Mindset, Skill, and Culture

So why can't people just see? When you find a better mouse-trap why doesn't the world sit up and listen?

I urge you to reflect on your own experiences of when you have been wrong. We've all been there! A tightly held belief that you've foolishly clung to, is ripped away ...

The problem is, none of us are omniscient, and there are other beliefs that we continue to foolishly hang onto, but which are they? When someone argues against your existing belief, how do you distinguish the sage from the charlatan?

Inevitably track record comes into play. If I have reason to trust someone's level of discernment I'll be more likely to contemplate their challenging idea. If not, not so much. A useful rule of thumb is that "extraordinary claims require extraordinary evidence".

Mindset

A very useful concept courtesy Prof Carol Dweck is Growth Mindset vs Fixed Mindset. Growth and fixed mindsets are attitudes to one's ability to learn something. A person with a fixed mindset believes strongly in nature over nurture: either I can or can't do that thing: learn to dance, do calculus, whatever. By contrast a person with a growth mindset believes that they can learn through study, practice and application.

The key point is this: if I have a fixed mindset about something I'll just believe that my level of achievement is basically set, so why bother improving and learning? That's why some Agilists have been known to refer to the growth mindset as the Agile Mindset. It really is a key to continuous improvement.

Skill

A growth mindset puts you in the right frame of mind to learn, but deep skill takes time to develop. Daniel Goleman (of Emotional Intelligence fame) makes the point that most amateurs learn until they hit about the 50 hour mark and then plateau, while world-class performers and athletes continue to learn and improve under the guidance of a Coach, who helps them mix things up and break through those plateaus to increasing levels of performance.

Pablo Casals, the great cellist, when asked in his eighties or nineties why he still practiced for so many hours a day, replied: "Because I think I am making progress".

Of course, Pablo Casals owned a fabulous cello (the tool), but without his amazing skill very little of value would have emanated from the instrument. Forced to choose, I'd much rather listen to Pablo Casals playing an inferior cello than an amateur playing on a Stradivarius!

Culture, and the Agile Trap

We can see how coaching and deliberate practice coupled with a growth mindset leads to ever-increasing skill, but without a supportive culture you're dead in the water.

Increasingly I am witnessing what I call the Agile Trap. Here's how it works: by introducing pretty basic Agile techniques from typically Scrum or Kanban, a team or teams will get an immediate 10% to 30% bump in performance. That's huge by normal standards, but well short of the Agile promise of twice the value in half the time (a 400% uplift). The trap is to go, "Goody: let's push through 10% to 30% more work from here on, and slap ourselves on the back". The bad news is that even that uplift will drop off as people burn out.

The smart move is to reserve all or most of that capacity for continued re-investment. Learn UX methods so you're building the right stuff, and how to automate so that you're building it right.

Toyota has risen to become the pre-eminent car manufacturer based on a culture of innovation, experimentation, and continuous learning over decades. My good friend, Sydney-based coach, Dave Martin expands eloquently on this point in his article on Sustainable Pace for Organisations. Too many leaders announce victory before putting in the hard yards. They think that the end goal is all there is, and forget that the journey of learning and improvement is long and involved.

Real Learning and the 70 : 20 : 10 Rule

One of the problems that I've observed in conventional corporate training is that it seems to work as a check-the-box exercise, with too much weight placed on formal training, and insufficient attention paid to integrating and internalised new skills in the workplace.

The real test of what you've learned is to notice what happens when the going gets tough. Do you make use of the new skills, or fall back onto old ones? To truly incorporate new skills requires not just conceptual understanding — which we have seen cannot be taken for granted — but also repetition and habit-building. For example, it took me over a month to re-learn how to tie my shoes after it was pointed out to me that I was tying them wrong. And to remain at the forefront of shoe-lacing tying technique — and in the spirit of continuous improvement —I will need to learn how to tie my shoes again!

At time of writing the standard approach is to send employees off on a standardised two-day course and then expect them to return with skills raring to go. There are several problems with this model:
  1. Attendees usually only a retain a fraction of the material
  2. Contextualisation is left to novices
  3. There is typically little coordination between the follow-up activities and the training.
A superior model is to gradually integrate new skills into the workplace in smaller modules, with coaching support. The 70 : 20 : 10 rule is a rule-of-thumb for this: for every hour of formal training, plan on two hours of on-the-job coaching / mentoring on real tasks with the new skills, and then about 7 hours practice with lighter touch supervision to embed the skills.

This is the approach I have recently been taking as a consultant / trainer / coach, with promising results.

Conclusion

So there you have it. You can't buy automation (and Agile) success, but you can invest in it through training, coaching, bringing in people with the right skills and backing them to the hilt. Of equal importance is fostering a growth mindset with respect to skill and leadership, with coaching and mentoring for leaders and teams to support learning.

To have any longevity, organisations will need to develop and maintain a culture of excellence and continuous improvement, by becoming true learning organisations. These entities will of course make full use of tools and technology, selecting and apply them intelligently and effectively by dint of their peoples' skill.

By contrast, organisations that try to buy the tools and processes while neglecting the human skill, mindset and culture will be as competitive as a monkey playing a cello! 

We're on an exciting journey to change the world of work (and ultimately the world) for the better. And we're barely getting started!

Friday, April 15, 2016

The #NoEstimates Game

Introduction

The #NoEstimates game is a group activity about the power of using data to forecast. It gives practitioners and stakeholders a better way to understand and allow for the effects of variation on their project planning efforts.

The game uses historical data about team performance to not only forecast the duration of a future project, but also uses the data to quantify the uncertainty around that forecast. This is a big step up from relying on a rule-of-thumb and an even bigger improvement on wishing that forecasts are exact!

This activity does not teach you everything that you need to know about #NoEstimates, but it does form an excellent jumping off point, with the simulation and surrounding discussion giving participants a real feel for the spirit of #NoEstimates.

Yes: dice-rolling will be part of the activity!

Once you've played the game, the next step is to start using the simple and convenient Skillfire project forecasting app on your actual projects. In fact, the #NoEstimates game doubles as training in the use of the app.

Please: If you want to use the output of the app to discuss project deadlines, don't just present the results to your stakeholders: play the game with them first!

Credits: This work derives from Adrian Fittolani's approach to Monte Carlo forecasting of projects. The Skillfire forecaster was a collaboration between myself and Tim Newbold.

Objectives

Participants will learn
  1. That uncertainty in project duration is unavoidable for any team (or process) that has variable output
  2. How to roughly estimate the degree of uncertainty for a new project based on historical data
  3. How to use the Skillfire project forecasting app and interpret its output
Bonus: Playing the game leads nicely into a broader discussion of the value and limitations of estimation and forecasting, and how other Agile and allied #NoEstimates techniques can be used to deliver value and coordinate effectively in the presence of uncertainty.

Duration

  • 20 to 45 minutes (depending on length of discussion)

Equipment

  • Whiteboard and markers
  • Lots of six-sided dice: one per participant
  • Each participant will need a pen and paper

Procedure

1. Introduce the Historical Data

In a typical #NoEstimates approach, rather than trying to estimate the size or duration of individual chunks of work (here we talk about stories, but these could be tasks, items, whatever) we assume an experienced, stable team that knows how to decompose work into small pieces

For those used to working in story points, this corresponds to a team that is skilled enough to split everything into (say) 1 to 3 point stories. At this point, perhaps we can stop bothering estimating, but have to keep slicing thinly. For those work in time-based estimation, a similar state of affairs is achieved when no item takes longer than (say) a person-day to complete. 

Now ...
Consider two teams, team P and team V. Both teams delivered their most recent project (both consisting of 48 stories) in 6 sprints, but that their delivery patterns differ significantly. 

Whiteboard

SprintTeam PTeam V
1
8
5
2
8
9
3
8
14
4
8
5
5
8
5
6
8
10
Total48 stories48 stories

Questions for the group / discussion

  1. Does your real-life team (or teams) resemble Team P or Team V?
  2. What are the full names of Team P and Team V? [Answer: Predictability and Variation.] 
  3. How does the degree of variability in output reflect internal team factors, external factors, and the nature of the team's work?

2. Back-of-the-envelope calculations

The next project has been estimated to require 80 stories for completion.
Scientists and Engineers often make calculations under simplifying assumptions on the back of an envelope. We can do that here.

Questions for the group / discussion

  1. How many sprints do you estimate that team P will take? Team V? [Just give a number.]
  2. What's the shortest and longest duration that you expect for each team?

Commentary

Under simple assumptions team P can be expected to keep delivering at 8 points per sprint and deliver in exactly 10 sprints. On average, team V will do the same, but could get lucky or unlucky. 

Most experienced practitioners will add some margin to an unbiased estimate such as these, say +/-30%. But this rather arbitrary, and does not reflect on the variability of the the team's delivery patterns.

Questions for the group / discussion

  1. If you ran this project for real, what factors could increase the actual project duration for either team?
  2. What  factors could decrease the duration?

Sample responses

Increase
  • External dependencies and delays
  • New stories emerge
  • Re-doing / re-working / iterating necessary
  • Change in technology
  • Change in team composition
  • Team members distracted / diverted / leave
  • Team members added late in the project
Decrease
  • Effective cutting of scope (some stories are cut)
  • Capable team members added early in the project
  • Shortcuts found

3. Play the game

Now we simulate a single sprint conducted by team V, first as a group, by rolling a single die.

E.g. rolling a 1 corresponds to a sprint in which 5 stories were delivered, a 2 implies 9, and so on.

Here's a complete simulation of an 80 point project:

Example: 8 and a bit sprints (call it 9) to deliver the project
Sprint 1: rolled 3, delivered 14, stories remaining 66
Sprint 2: rolled 2, delivered 9, stories remaining 57
Sprint 3: rolled 6, delivered 10, stories remaining 47
Sprint 4: rolled 4, delivered 5, stories remaining 42
Sprint 5: rolled 2, delivered 9, stories remaining 33
Sprint 6: rolled 5, delivered 5, stories remaining 28
Sprint 7: rolled 1, delivered 5, stories remaining 23
Sprint 8: rolled 3, delivered 14, stories remaining 9
Sprint 9: rolled 3, delivered 14, stories remaining -5

Distribute dice and get participants to roll a die, look up the result, and keep a running tally on the board to show how to simulate an entire project.

Now, have everyone play the game independently and note their own results. How many sprints does each simulation take?

Meanwhile draw up the beginnings of a chart for the class results on the board. [See next figure.] Have participants stack a dot for each of their games. About 30 simulations gives a nice visualisation of the distribution of sprint durations:



Here each dot corresponds to a single simulation's duration.

In the worst case, every sprint delivered 5 stories (which happened once). In the theoretically best case the team delivers 14 points every sprint (which didn't happen this time) and finishes in (just under) 6 sprints.

Empirically, we see that the likely delivery time is 10 to 13 sprints for team V, but it could be worse!

More formally: 
  • half the sprints take 12 or fewer sprints, making 12 the median result.
  • discarding the bottom 3 (10%) and top 3 (10%) results, we can forecast that the project should take between 10 and 13 sprints with 80% confidence. [But we really should run a lot more simulations to stabilise the distribution.]

To be really safe we need to assign 16 sprints, but a better strategy would be to make sure we're delivering stories in priority-order and start de-scoping if things are going slowly or especially if the last few stories just aren't that valuable.

4. The Cumulative Distribution

An alternative way to read off the median and a confidence range is via the cumulative distribution. We obtain this by adding dots as we move to the right (cumulating).



You can attempt to draw this on the whiteboard, but be sure to rescale. I usually just sketch the idea before moving on to the Skillfire app.

5. The forecasting app

Now, for a stable forecast we really need a lot more than than 30 simulations, so let's switch from dice to computer-based random-number generation and conduct 10,000 trials with the Skillfire project forecasting app: skillfire.co/project-forecaster



Just enter team V's historical data and the estimate for the new project size, press Simulate, and the app does the rest. The continuous graph is acheived by counting a partial sprint at the end, but otherwise this is simply what you would get if you sat around rolling a lot of dice.



Notice that the Median has shifted back down to 10 sprints (30 rolls was insufficient!) and the confidence interval is now set to 90% rather than 80%.

A great use of this tool is to help explain why further analysis or discovery won't eliminate uncertainty if team output is variable.

While we used six sprints of historical data for convenience (sides on a die), when using the app there's no need to restrict yourself to just six sprints of data.

6. Closing

Be sure to re-cap: we now have the technology to quantify uncertainty based on history, but this is only part of the conversation. Recall all the different ways project duration can get inflated (and the few by which it gets shortened!)

Remember: don't just use the app and present the results: play the game with colleagues and stakeholders first.

Let me know how you go!

More Q & A

Q: Where can I find out more about #NoEstimates?
A: Start by reading this interview with Neil Killick.

Q: Can I measure story points and use this as an #Estimates technique instead?
A: Absolutely. You can even try both!

Q: If scope typically increases by 20%, how do I factor that in?
A: Run simulations with inflated number of stories and report both.

Q: What else can I do with a lot of dice?
A: Play Tenzi! A standard Tenzi set comes with lots of dice.

Q: Can you show more simulations
A: Sure ...

Game 2: 7 sprints exactly (fast delivery)

Sprint 1: rolled 6, delivered 10, stories remaining 70
Sprint 2: rolled 6, delivered 10, stories remaining 60
Sprint 3: rolled 3, delivered 14, stories remaining 46
Sprint 4: rolled 3, delivered 14, stories remaining 32
Sprint 5: rolled 3, delivered 14, stories remaining 18
Sprint 6: rolled 2, delivered 9, stories remaining 9
Sprint 7: rolled 2, delivered 9, stories remaining 0

Game 3: 11 sprints (slower delivery)
Sprint 1: rolled 2, delivered 9, stories remaining 71
Sprint 2: rolled 6, delivered 10, stories remaining 61
Sprint 3: rolled 1, delivered 5, stories remaining 56
Sprint 4: rolled 6, delivered 10, stories remaining 46
Sprint 5: rolled 5, delivered 5, stories remaining 41
Sprint 6: rolled 6, delivered 10, stories remaining 31
Sprint 7: rolled 2, delivered 9, stories remaining 22
Sprint 8: rolled 2, delivered 9, stories remaining 13
Sprint 9: rolled 1, delivered 5, stories remaining 8
Sprint 10: rolled 1, delivered 5, stories remaining 3
Sprint 11: rolled 2, delivered 9, stories remaining -6


Sunday, September 6, 2015

Agile Lessons from the Martial Arts

My talk — Agile Lessons from the Martial Arts — delivered at LAST (Lean Agile Systems Thinking) Conference  2015.

You can read about my martial arts background here.

Slides


Notes

Here's the scoop, 10 lessons I've learned from the martial arts that apply to Agile practice and coaching in the business world.

In the session I largely present the martial arts side and ask the audience to draw parallels with Agile practice and coaching. Here I spell it out.

1. Trust is about relationship

"The martial arts begin with trust."

In jiu-jitsu (and other martial arts) we practice dangerous techniques: throws, arm-locks, strangles, kicks and punches. Without cooperation and trust the risk of injury would make it impossible to train safely and effectively. If we injure our training partners by breaking their limbs, dislocating joints, or dropping them on their heads we would have no-one to train with next week, to say nothing of the fallout of shattered relationships and the expense and indignity of law-suits!

In business settings I've observed that people operate principally with one of two definitions of trust.
  1. Trust as predictability: e.g. if you say you'll do something, you'll do it; if you say you can do something, that means you have that capability, etc.
  2. Trust as relationship: I can trust you not to "stab me in the back", because it is understood that by looking out for each other we we'll jointly benefit.
* * *

It's the second kind of trust that I value, and helps lay the foundation for high performance teams. The first kind (predictability), in my view, is really just professional courtesy and competence.

Without real trust you'll always being wasting energy watching your back.

2. Ritual and respect 

The Japanese martial arts are rife with ritual. One bows when one first greets one's instructor, when entering the dojo (training hall), when getting onto the training mat, when leaving the training mat, before working with a new partner, after finishing with a training partner. As part of the opening and closing ceremonies we get down on our knees and bow repeatedly: to a place of honour (symbolising the art, and previous generations of instructors and students), to the instructor, and to each other.

All this bowing and kneeling acts in part to put us in a humble mood. It reminds us of how much we owe our teachers and predecessors for sharing their knowledge, and each other for joining us in the act of training and study.

It is also reassuring. We sanctify the training hall by repeated ritual, and thereby turn it into a place of sanctuary, where we can leave our every-day stresses and troubles at the door and engage in cooperative and mutually beneficial activity.

* * *

In the very popular Agile practice of the daily stand-up (for example) we similarly engage in repeated, somewhat repetitive behaviour-with-a-purpose. As with the opening ceremony before starting a session of martial arts, we engage with each other in a place (often at a Scrum or Kanban wall) and ready ourselves to work together.

Having a team "space" also taps into something very human. It gives a sense of familiarity and safety amid the hubbub and unpredictability.

By practicing respect and courtesy you reinforce an excellent habit: paying attention and listening to other people. People who are overly arrogant tend to talk a lot and manipulate others. For me, listening and engaging is the superior approach.

3. The journey to mastery is like climbing a staircase 

When I started learning martial arts I apologised to my instructor about my lack of coordination: I regarded myself as a clumsy person. To his eternal credit he basically ignored my comment and told me to stick with the exercise.

This is the genius of Shu, the first phase of learning. Shu means, roughly, "learn the rules".

The way I learned the fundamentals of jiu-jitsu is so brilliantly structured and taught that as long as you "get with program", you will learn and develop a base level of competence. As a student all you need is an open mind, persistence, and self-belief.

Not everyone has these: if you think you already know better, how can you learn something new?

An open mind: Empty your cup
A master was trying to explain something to a student. Now this student was not a brand new student, but a senior student who had learned many things. He had knowledge and experience aplenty to draw upon. But each time the master tried to explain something new to the student, the student kept trying to hold it up against his own notions of the way the world is and how it ought be, and he was unable to see the lessons in what the master was trying to teach him.

Finally, the master poured a full serving of tea into his own cup, and into the cup of the student. Then he told the student he wanted to give to him some of the tea from his own cup. He began pouring tea from his cup into the student's cup, but the student's cup was already full, and all the tea from the master's cup spilled out over the cup onto the surface below.

The student said, "Master, you can't pour anything into my cup until I empty it to make room for what you are trying to give me.", and the master replied "Yes, I know. And I can't give you any new thoughts or ideas or perspectives on life's lessons until you clear out some thoughts that are already teeming in your mind to make room for what I have to teach you." 
Then the master paused for a brief moment, meeting the student's eyes with his own knowing look and calmly but sternly said: "If you truly seek understanding, then first, empty your cup!"

Patience 
When you attempt something new, either you experience immediate success, or not. For complex skills this initial experience is in no way the whole story.

Those who get off to a fast start, typically fall back to earth quickly, then has to work hard to recapture that success, and then hit a plateau before climbing to the next level:


The other possibility is that there is no immediate success: the initial experience starts with the plateau:


Interestingly, the slow-starters often do better in the long haul, because inner determination has been established from the outset. Shades of the tortoise and the hare.

Persistence
The long haul journey to mastery is a lot like climbing a very long stair-case. Periods of progress alternate with plateaus.


If you mistake that first plateau for "the point of diminishing returns", you'll never break through to higher levels of achievement.

All of this came as quite a contrast to how I had learned growing up. It seems to me that in western culture we over-subscribe to the myth of talent: one is encouraged to persist with whatever one shows initial promise in. We get excited when we find "prodigies", and too-often over-praise them for their precocious gifts rather than hard work.

Martial arts taught me that this is a shallow perspective. By dint of genetics or previous experience some things come easily. That's mainly luck. When you climb the stairway to mastery through long dedication, that's really earned.

The literal translation of kung fu is not specific to the martial arts. It means "Skill achieved through hard work".

* * *

I perceive an irony in the modern business world. On the one hand, because of the pace of change and competitiveness, Agile and Lean approaches are in demand to raise quality and crush delivery time-frames. But, the mastery of these ways of working is a long journey that cannot be rushed.

So my mental model of learning and teaching (above) helps me to gauge where an individual, team, or organisation is on the journey, and I vary my practice accordingly.

4. Weakness becomes strength

What does it feel like to experience one of these plateaus?

After trying a few martial arts at University I stuck with jiu-jitsu because I was able to progress. Not on a continuous upward climb mind you. The trick was that we studied many inter-related techniques, concurrently, and while I was not visibly improving on several fronts, we would nevertheless keep practicing, and there was some sense of accomplishment with some of the techniques, while there was the opportunity to keep coming back to the ones that posed difficulties.

The trick is to just do the work and not be self-judging, and to trust the system and the judgement of your instructor that you will progress. Worrying or judging yourself is a waste of energy. The trick is doing less, not more.

That said, when I started there was a particular technique, a hip throw with many moving parts and details, that was my "worst technique". Some techniques came intuitively, others I was able to get working with instruction and feedback, and then gradually smooth off, but this one felt really clunky. I felt like I was faking it.

But I kept coming back to this technique. The system of learning made it unavoidable. So for two years I practiced it, thought about it, took it to pieces, and it still felt clunky. But when it *finally* clicked I knew its details and nuances better than any other technique. 

Today it is one of my favourite techniques, and when I teach it my students tend to do it rather well. By spending so much time on the plateau I learned how to teach it better than the techniques that come easily!

* * *

Don't think that it is only the things that came to you easily that have value. Sure, your strengths may be valuable in execution, but when it comes to mentoring these may only serve as inspiration. If you've ever struggled to master something you probably have much more to offer aspirants when coaching that same skill. 

For things that came easily to me it's always harder to teach: I have to reverse-engineer and experiment to get results, and I never have the same empathy when it comes to teaching others because I walked a very different, easier path.

Faced with a difficult challenge, I look first for multiple approaches, look to take small steps, strive to be patient yet persistent, and chip away on more than one front.

5. How to teach

My experience at high-school was that the best teachers taught to the "middle" of the cohort and tried to do a bit to help the "gifted" and the laggards. Accordingly I stuck with the subjects that I excelled in, but didn't expect much from the teachers, and turned to my own ingenuity and the course material.

In the martial arts I have learned a far better system, incorporating both mastery learning, spiral learning, and learning by teaching.

I have touched on mastery learning and spiral learning already without saying so.

In mastery learning you do not progress to higher levels until you have mastered the existing material at a reasonably high level of proficiency. Instead of grading people on a curve and promoting by age cohort, students are tested for promotion to the next belt when they are ready. People learn at different rates, and some train more intensively than others at different times. We're not on a schedule.

Spiral learning is the circling back through material to achieve greater depth, not getting stuck, but not allowing avoidance of areas of difficulty either.

We also learn by teaching. In a typical jiu-jitsu class the instructor will demonstrate a technique by (for example) throwing a senior student and explaining key points. Then the class breaks into pairs and they practice together, taking turns to throw (or whatever) or be thrown. If there is a more experienced person in the pair, they can effectively give one-on-one instruction. This helps the senior clarify their understanding, while the junior gets personalised practice. Experienced peers will do more exploratory work around the technique.

Can you see how this approach allows people of varying standards and experience to learn without "streaming", teaches life-skills, and helps the students who are btoh above and below the middle standard to improve?

* * *

I believe that these approaches have great relevance in the workplace. One can coach intensively for mastery by focussing on particular practices or principles, and spiral back to things that didn't take root repeatedly.

When team members mentor each other is also remarkably powerful, and is one way to help fuse a "group of workers" into a team. Pair-programming is a par excellence example of this in an Agile setting.

Colleagues can learn together and from each other. We need more of this in the workplace.

6. There is more than one path to the top of the mountain

When I struggle to learn something (as a beginner) I need very structured teaching to break through that initial plateau. When I have something of a grasp I need to explore and improve.

One of my main instructors taught me to teach good technique first. "Some people will never get the underlying principles, but there is still value in learning the techniques." First concrete, then abstract.

Just learning the rules is the "Shu" stage. Next is "Ha", break the rules. This begins by asking "why?", and figuring out some of the principles.

No technique works in every situation: so we invent or stumble on variations that make different trade-offs. Understanding principle allows us to reconcile variety.

* * *

How do I apply this in an Agile setting? By learning lots of practices and the underlying principles I can craft bespoke approaches from my toolkit. It's not enough to have lots of tools, you need to be able to figure when to use them to best effect.

7. Order and Chaos

Fundamentally there are two kinds of practice in martial arts: choreographed drills (these tend to be cooperative) and competitive play: both the asymmetric self-defence and symmetric sparring.

Pre-arranged drills teach us technique and habit. Reflexive self-defence (where the attack is not known to the defender in advance) simulates the unpredictability of a real encounter.

It is also possible to explore the area in-between.

* * *

In the modern world we develop technical skills and processes, but it is a mistake that we can impose order on everything through planning. We need to develop our intuition and decision-making for when the unexpected occurs.

8. Under stress, you do what you have internalised

Learning techniques cooperatively gives us the building blocks to apply in a conflict situation. Much practice is required so that these techniques happen without "thinking": they must become second nature.

In martial arts, beginners will fall back on one or two favourite techniques. They may be able to do more if the pressure is off, but when the chips are down, they'll revert to what they know and what works for them.

*  *  *

Similarly, it's common to see people who are ostensibly Agile revert to command-and-control or cowboy development approaches or begin design up-front, when placed under stress, or when coaching support is taken away.

Transformation takes time, support, and reinforcement.

9. When in doubt, keep moving

When attacked, our ancestral response will be "fight, flight or freeze".

Freeze is typically the worst when fighting other humans. When you stop moving you become  a "sitting duck". Movement creates possibilities and openings.

* * *

Similarly, in a situation of organisational conflict or challenges, trying out different things, changing it up, can create new possibilities.

If you have done your time learning the rules (Shu) and broken the rules (Ha), you may be ready to forget the rules (Ri). This is the stage of "unthinking competence", whereby your internalised skill allows you to follow your intuition and see what emerges.

Ri is very advanced and should not be confused with being brave and clueless!

10. There are no limits

My journey in the martial arts has felt a bit like peeling an onion: with some effort you scrape off one layer only to discover there's something deeper and a bit juicier underneath. (Sometimes your eyes also water.)

They're called martial arts for a reason. There's craft and science to learning, but there's also opportunity for self-expression through refinement of one's learning and teaching, technical innovation, and execution.

Studying the martial arts has been a wonderful journey for me. Something not to rush, but to savour. Even though I feel I've figured out a thing or two, the more you know the more you become aware of your current limitations, but get ideas about what might help to improve.

* * *

When I first started consciously applying Agile approaches, I thought that they were only suitable to small-scale, already somewhat exceptional teams. Over a decade later my experience in the martial arts gives me hope (and tools) that they can be adapted and applied far more widely.

Saturday, August 29, 2015

Six degrees of separation (or less)

Science communicator Derek Muller explored the history of the six degrees of separation phenomenon in this piece on his video channel, Veratasium.



At the end of the video, he launched an experiment. Within just a few days, whoever can contact Derek via an email chain of people who know each other in real life, will receive a postcard from Derek.

I got contacted in one such chain and now know my "Muller number" is three. It was surprisingly quick, easy, and fun.

Interesting, too, were the people in the chain, and our connections. Here's how it went down.

  1. Jay McCarthy started the chain by emailing me. Jay is an American Computer Science professor and one of the developers of the Racket programming language. We met in person at St Louis at RacketCon 2014. I highly recommend Jay's fabulously entertaining talk. Jay emailed me because, like Derek, I am an Australian.
  2. I reside in Melbourne, Australia. I learned from his Wikipedia entry that Derek holds a PhD in physics education research from the University of Sydney. I decided to email Michelle Sowey, who is from Sydney and connected in academic and education circles. Michelle is the founder of The Philosophy Club, through which she and her husband, David Urbinder, perform amazing philosophical enquiries with school-aged children. My wife, PatchAndi, went to school with David.
  3. Michelle emailed her friend Kylie Sturgess, who hosts the award winning Token Skeptic podcast. Episode 103 is an interview with Derek Muller.
  4. Kylie has also met Derek in person at various science events, and was kind enough to complete the chain!
So we got from Jay to Derek in just four degrees of separation, in roughly 12 hours!

I look forward to seeing Derek's follow-up, and his postcard to Jay.

Sunday, May 17, 2015

Fabric mosaic technique - animation

Here's a time-lapse of an experimental fabric-mosaic technique that I'm working on with my wife, PatchAndi, as a possible addition to www.youpatch.com:

Pixelated Frida Kahlo

Those are actual photos of the construction, achieved by sticking pieces of fabric onto a backing piece, color by color.

Here's a simulation of the piece-by-piece construction:


The aggregation algorithm of similarly-colored rectangles is a little different from the original quilt design, available here. Size is roughly A2.


Sunday, January 25, 2015

Why is working harder such a popular management strategy?

The short answer: Working harder gives a short-term performance boost, but leads to a long-term decline in capability.


Organizations that commit to working smarter pay a short-term dip in output, but in the long-term enter into a virtuous cycle: improvement reduces effort for the same results, which is re-invested in further improvement.

Unfortunately, the opposite, vicious cycle is far more common. A decision to cut costs leads to working harder, which leads to a short-term performance boost with an unseen (delayed) trade-off. The decision-maker thinks "I got it right", feels vindicated, and applies linear thinking: more of the same should lead to further gains. This is wrong, but the bad effects -- losses in quality, capability, etc. --  are not be felt for some time. Meanwhile, committing to work "harder, not smarter" crowds out learning and process improvement.

Reference: Repenning and Sternan, Nobody ever gets credit for fixing problems that never happened [pdf]



Friday, January 23, 2015

Learn or Lose: Agile Coaching and Organizational Survival

by Daniel Prager
The illiterate of the 21st century will not be those who cannot read and write, but those who cannot learn, unlearn, and relearn. 
― Alvin Toffler 

Introduction

Overview: Change and evolve, or wither and die

For organizations to survive and thrive in the Information Age they must become truly adaptive. This will require a huge cultural transition away from the mechanistic, hierarchical, command-and-control approaches that were highly successful through the recent Industrial Age, but have now expired.

What is needed for survival is a transformation into true Learning Organizations. The essential cultural change is from a rigid focus on performance and risk-avoidance around a fixed (or infrequently changing) business model, to one that embraces continual learning, experiments, and small, typically reversible, changes.

Agile and Lean methods talk about learning and change, but I want to emphasize these facets as the central factor to cultural change. Without embracing ongoing learning and change, one may well upgrade one’s old approaches by cherry-picking elements of the new “Agile” way, but this is likely to only be a short-lived improvement. It is not enough.
 ...

Read the rest of my article on InfoQ