Wednesday, November 19, 2014

Otter's Law

For all the 'take back agile" and "agile smagile" and "apologizing for agile" and all the other post-agilists and so-called post-agilists in the world, I give you Otter's law:

Any methodology followed via obligation and knowledge-avoidance cannot produce positive change. 

I've ranted about change gone wrong and methodologies followed badly and horrible oversales and undersales and all, but it comes down to otter's law ultimately.

Too many companies have huge gains from using XP and Scrum and what-have-you. Too many others tried "the same thing" with entirely different results.

Some teams are whole-heartedly into the whole agile thing, and they seem to have pretty good. Others don't seem very excited and don't get much out of it. Is it excitement that they need?

I don't think so. I think that the lack of excitement and the lack of progress have the same root.

I think it's a lack of profluence in agile-as-practiced. Often mandated agile drives people up the old Christopher Avery "responsibility" chart all the way to "obligation." Not to responsibility.

The team doesn't have a sense that they're gaining ground at all. They may be afraid of misjudgment or criticism or failure at every step. They don't feel safe so they don't dig in for all they're worth. They tip their toes into the water, and do what they are required to do so that they don't seem to be resisting the management directive.

Worse, teams can reach a point of total anhedonia. They lose all ability to celebrate wins or feel a sense of accomplishment because nothing they do will ever be good enough. If they dig in really hard, they'll be told that it's nice to see them making some effort, but they're going to have to do a lot more. In such a circumstance, developers (testers included) can't see the sense in changing methodology and terminology. They're all in a state of learned helplessness depression. A wall of colored cards and sticky notes won't change that.

Are they digging into the material to find out how to do a great job? If not, then they are probably operating on "knowledge avoidance." They don't invest in researching because there's really no pay out for them. Instead, they'll do as they're told, as they understand it. There was no real shared mental model involved in the roll-out.

I think that people have reasons for not being gung-ho agilists to begin with. If you try using pressure to oblige (there's that root word again) them to act as if they're excited, the effort is not going to create the excitement you're looking for. It will instead establish a cargo cult.

But some teams do get a lot of excitement from the profluence, do see transformation in their code base and their companies, huge reduction in defects, huge increase in mastery and autonomy in team members, and huge bottom-line effects.

I'm not writing this to blame the people who are trying to push their company into a more ideal agile-like environment. I totally agree with that. Some managers see what is possible and want to adopt XP or Scrum for the way "good agile" can empower their organization. I don't think it's wrong for them to want it.

I'm also not blaming people who are in a system that makes them feel unsafe or guilty writing tests or refactoring existing code. They are citizens of a system. I don't expect them to stick their necks out and risk their position or reputation or standing among their peers.

I'm just thinking that every transformation runs the risk I describe with Otter's Law. If we only push people to obligation, we shouldn't expect to see responsibility automatically bloom.

The methodology doesn't come with its own culture, history, and mythology. It has to take root where you plant it.

Even the world's best teacher can't teach unit testing so well that everyone feels safe doing it. There is a control culture and a guilt culture to confront.

It has to be safe for people to try, even to fail, to adopt new practices or to take control of their code.

Monday, November 3, 2014

Wrong naming revisit

I was looking at the excellent work by appium, and saw this little snippet in an example:

var el = driver.FindElementById ("name_input");

el.Clear ();

el.SendKeys ("Appium User");

el.SendKeys (Keys.Return);

There's nothing horribly wrong with the name el, except that it's exactly what we ask people not to do with names. 

El is short for element (a bit of mental mapping, but the length of the name reflects the shortness of scope). But it is not informative.

The variable "el" (EE-EL) looks like "e1" (EE-ONE) to the casual reader. The name doesn't look right. 

Also, "element" is what it's made of, not what it's for.  Yes it's an element. But it means "the name input field" so "nameField" or"name_field" or "name" might be better. Or even "input" if it's the only input field in the block.

Appium is pretty cool. You should check it out. But remember that writing examples is both something rather important (people copy or imitate examples) and something hard to do. We write them thinking "the api is the important thing" or "the idea is what matters" but we forget how seriously people will take examples.

What is a Coach?

I'm a software org/team/individual coach. What the heck does that mean?

On a closed mailing list, Geepaw asked for our definitions of a software team "coach," and several offered opinions. 

This one is entirely mine. 

On reading my definition, John Kern interpreted it to mean "individuals" whereas I intended these to apply to teams and organizations as well. I thank him for bringing up the possible misinterpretation, and I leave it up to my readers to understand I mean it in a plural sense; organizations and teams have skills and habits and mindsets just as individuals do.

Without further ado:

A coach is someone whose:
1) work is with people
2) primary product is an improvement in their abilities 
3) secondary product is an improvement in the way they interact with teammates and peers.
4) teaching comes through interaction, not mere lecture or advice (that would be a counselor)
5) work is done at the request and permission of his client

I would also note that when we refer to "abilities" above, it is in the same sense that JB Rainsburger mentions, that of increasing capabilities and removing impediments. 

Thursday, October 23, 2014

Preplanning Poker: Is This Story Even Possible?

The story says "attach an ecommerce server."

Well, maybe it says "As a product manager I want my system to incorporate an ecommerce server so that I can connect money."

Can you get that done this iteration? It sounds like a three-story-point effort to me, right?

Hold On A Second

This story doesn't have a plot. It is a state of being. I don't think that  saying "once upon a time there was a little girl" would qualify me as a story teller. 

Right away I'm nervous. What the heck does it mean? What do we want to do here? 

Let's not throw this into the sprint backlog with (of all things) a story point number on it. Let's certainly not stick somebody's name on it. Let's think a little. 

We're not aligned on what this "story" means. 

The New Preplanning Poker

You already know about planning poker, and the benefits of silently estimating first, then comparing results. You know that it helps avoid anchoring and arguing and lets you see the degree of separation in estimates in the team. It's a nice consensus-seeking idea. 

I think we need to apply that concept forward to pre-planning (and to non-estimating teams). We will need five cards for this new preplanning poker, as follows:
  • Defer 
  • Accept 
  • Reject 
  • Explore
  • Split
The astute among you might notice that the acronym for this set of cards accidentally spells DARES. I guess that's okay if we're trying to determine whether we dare tackle this feature as given.

All you really need is five index cards per person and a marker, but if you really want something you can cut and print, try these (I left a blank because six cards looks better than 5):

So here is the story. Pick one of the cards, don't show its face. Ready? One... two.. three

Your answers

You picked defer. Why is this not a good time to add this feature? Why is later better? Is there something far more important to do? Too few developers and testers available this week? Is it scheduling? Availability of people? A cash flow problem? A production system down?

You picked accept. You think that this task is very well-defined and the criteria for success are obvious. You're ready to go, and you know how to do the work. I'm shocked, given the nature of the story but tell us what you know and what inspires your enthusiasm.

You picked reject. You don't think that the system needs an ecommerce system? Why? Do you have another way you'd rather we received money? Do you think this system should not receive money? Do you think that we should use this system in a way other than a money-gathering device? Why should we never do this? 

You picked explore. You think it's a good idea, and we should get involved, but you believe that there are technical issues involved that we don't understand. Is it platforms? Licensing? APIs? Languages? Authorization/Authentication issues? Architectural concerns? What do we need to know in order to move forward?  I think you are likely right - this may not be something you simply bolt on without some exploration of vendors and technologies and market segments. 

You picked split. That means that this story is not really scoped well. Maybe it needs to be rewritten as a series of stories, or a series of releases, so that each increment of this feature will be well-understood and can be tested and possibly documented. In this case, I agree with you. We might need to know who needs to pay, and for what, and what the flow is around each payment scenario. Each point of payment will likely need several stories to cover all the ways it can succeed or fail.

Will It Work?

I have played this game without cards at a few client sites. Sometimes I'm surprised at how vague a story can seem to me, but be perfectly clear to the local development team. Other times I'm surprised in the opposite way. 

We have had great story mapping and story splitting sessions result from these quick 5-way triage games (is that a quintage?) It takes only minutes and you can get a lot of focused discussion and backlog grooming done in a very short time. 

If you try it out, let me know how it worked for you.

Monday, October 20, 2014

Microtesting TDD: A Quick Checklist

Quick pointers:

  1. See each test fail at least once (so you can trust it).
  2. Make test fail messages helpful because they fail when you are working on something else.
  3. Prioritize!
  4. Use a list for tests you want to write. "Ignored" tests will do nicely.
  5. Run all the tests so you know when your last change has broken something.
  6. Keep your feedback loop as tight as you possibly can.

I get to see these all violated, so I thought I'd make a short list and save you some time. 

The first two go together. You want each test to fail so you can see the message. Some time in the future you'll make a change, and an older test will fail and you'll see the test class name, the test name, and the assert message. Those three should work together so that you know what kind of mistake you made. 

That goes with number 5, too. If you only run the one test you're working on, you may have dozens of breakages by the time you get around to running all the tests. If you work on a component team (not my favorite organization, but sometimes necessary) then you should run all the component's tests at least.  The more "distance" between the injection of an error and its detection as an error, the harder it is to isolate and reproduce and fix. That's why #6 is listed. 

That leaves 3 and 4, which are about planning your steps. You might find some power in the idea of a list. Start by thinking, and create a list of tests you want to write. Then pick which one you want to do next based on what is either simpler or most important. When you think of new tests, add them to the list. When you find some tests are no longer important or describe cases that are already covered, you drop them. I learned this by watching Kent Beck's TDD videos.

If I was to shorten the list, I would say: 
  1. Write short tests with great messages
  2. Track and the next small steps you intend to take.
  3. Keep safe by running all tests frequently. 
That's short, but not as actionable. In the long 6-step form, it's a little easier to take on.

Friday, October 17, 2014

Your Transition Isn't Very Agile

Agile's teaching of "thin, vertical slices" doesn't apply just to features.
Organizations move forward in thin vertical slices too.

Story mapping teaches us to do incremental, value-first programming and integrate the "threads of functions" all the time from end-to-end. 

CI teaches us that integrating thin slices frequently avoids pre-release integration nightmares (and post-release nightmares). 

Likewise, we leave room for learning and growing, because what we learn in iteration N may give us different options and opportunities in iteration N+1 and onward. We have an idea of where we want to go, but we are always seeking best value.

However, too few agile transitions are done in an agile way.
It's only reasonable that a pre-agile company would want a waterfall, Big-Design-Up-Front plan with staffing and milestones for an agile transition. But we, as post-transition coaches and consultants know better and are supposed to be giving advice and support.

And just when we're building software, building a transition in bottom-up, architecture-first, isolated, horizontal layers causes a lot of integration problems.

We shouldn't act surprised.

Monday, September 29, 2014

Programming Is Mostly Thinking

Pretend you have a really great programming day. 
You only have to attend a few meetings, have only a few off-topic conversations, don't get distracted or interrupted much, don't have to do a bunch of status or time reporting, and you put in a good six hours of serious programming [note: this RARELY happens in an 8-10 hour day]. 
I want to review your work in the morning, so I print out a diff of your day's work before going home. 
Sadly, overnight the version control system crashes and they have to recover from the previous day's backup. You have lost an entire day's work. 
If I give you the diff, how long will it take you to type the changes back into the code base and recover your six-hours' work?

Programming is 11/12ths Thinking

I've been touting this figure for some time now, and people keep asking me where the study is that produced such an odd number. 

Well, it's not pulled out of thin air and it's not the result of a thorough scientific study. 

I have done informal polls now for a few years, though I've not kept good records. My goal was not to become the scientist who cracks the statistical/mathematical code for programming activities. I was looking for a reasonable answer to a reasonable question.

However, this answer surprised me. In a long Quora post titled "How do programmers code so quickly?"  one responder offered that it was a combination of physical skills (muscle memory, skill with tools, debugging skills,  typing skill) and knowing where to search for info. His post was swamped and overwhelmed by posts explaining that typing and tools are not the most important aid to quick code production.

Software Factories

I have seen the stickers and slogans on stickers and social media for a long time that "typing is not the bottleneck" (though every once in a while the inability of some programmers to type is a bottleneck).

I am keenly aware that most management still subscribes to the idea that motion is work. They are fairly convinced that a lack of motion is a lack of work. That makes sense in a lawn care service, a factory assembly line, or a warehouse operation.

Nearly all of the visible work done in producing physical goods is motion. People roll steel, stamp, press, mill, pick and place, bolt/screw/rivet, and on. 

Modern factories produce goods with Computer Numerical Control machines, which produce perfect copies of an original model that may not even exist in real life. These machines work from abstract models -- just data, really -- and perform perfect motion. Humans tend the machines, rather than working the wood by hand.

I have some great guitars that were produced at affordable costs because of the degree of automation brought by such machines. 

Great boutique guitars are produced entirely by hand at higher cost and I don't put down that effort either. The world has room for both.

Software developers have perfected the factory. It runs flawlessly bit-perfect copies. You just click the "copy" or "download" button. It's so cheap that the purchasers happily cover the costs of the factory. Those who are cautious will double check the checksums that come with the download, but most people don't bother. The machines are reliable and efficient and quick and cheap. 

Once the initial model (really, just data) exists, then the marginal cost of all the bit-perfect copies is essentially zero. Yes, this is just copying and not creating, but that's what factories do. Custom shops might produce unique items (like guitars) but factories create copies of originals.

The software factory tends to give you a progress bar, so you can visualize the motion of bits, but in many ways you can say that the product doesn't really exist. It's a pattern of tiny charged v. uncharged areas of metal on a plate (well, probably) and you don't even pay for the plate or the magnet or the laser when you create the copy. It's already there.

Software is an intellectual good.

The Design Shop

In my years of working with Uncle Bob Martin, I heard him continually tell customers and students that software development is not a fabrication operation, but a design operation. Once the initial design is done, all the duplication is done by machines at nearly zero cost.

So what programmers and testers and POs and Scrum Masters and software management area all doing (if they're doing it right) is designing the data model that will later be used by the factory to create copies for use by customers, patrons, and other people in the community the software is intended to serve. 

Yet the mechanistic, Industrial-Age idea of software development as a factory persists, and developers dutifully try to make it look like they're doing physical labor at the detriment of the process. 

All intellectual activities are hard to observe and monitor. An idea that is 80% complete has no physical manifestation. It's an idea, and it's not done yet. Sometimes we have experiments or proof-of-concept code or notes, but they don't give an accurate "% complete" number as does physical work.

A chair being manufactured looks about 50% done at the 50% mark.  When it's done, it looks done.

A design for a chair may not exist on paper until it is more than 70% complete. And we don't know that it's really 70% done, because it's not finished being designed yet.

The Answer: Really?

I have asked this question at conventions, client companies, to my peers, to colleagues, and to strangers I have met for the first time when I find out they are programmers.

The answer I receive most often is "about a half hour."

I could use the 8-hour day, ignoring meetings and interruptions and status reports, but that feels like padding the answer. I stick to the six hours doing things that programmers identify as programming work.

There are twelve half-hours in six hours. One half-hour to retype all the changes made in six hours of hard programming work. 

What in the world can that mean? How can it be so little? 

The Meaning Behind the Answer

Right now I suspect a bunch of managers are about to go yell at their programmers for putting in a half-hour's work in an 8 hour day! That would be a horrible misunderstanding of what was actually happening.

What is really happening? 
  • Programmers were typing on-and-off all days. That 30 minutes is to recreate the net result of all the work they wrote, un-wrote, edited, and reworked through the day. It is not all the effort they put in, it is only the evidence of the effort.
  • Programmers are avoiding defects as best they can. In order to do that, they have to be continuously evaluating the code as they write it, hypothesizing the kinds of defects or security vulnerabilities they might be introducing. After all, they receive their harshest criticism for introducing defects into the shared code base. 
  • Programming is a kind of lossy compression. The code only says what the program must do when it is running. Why it chose one particular way over others, how it influences the rest of the system, what errors were introduce and removed, and what pitfalls it avoids are not (generally)present in the text of the program.
  • Must of the work is not in making the change, but in deciding how to make the change. Deciding requires us to understand the code that already exists. This is especially time-consuming where code is messy or the design is not very obvious in the source code. 
  • Programmers work in a social context, since all their results are integrated into a shared code base (and most use pair programming or other "many eyes" techniques). Programmers may be helping other programmers or testers or operations people get a handle on their work. Connecting and communicating with others has benefits and costs that don't appear in the code. 

Six hours of intellectual work (reading, researching, deciding, confirming, validating, verifying) translates to about 30 minutes worth of net lines-of-code change to a code base.

That's not additional lines of code. We often have weeks when we fix bugs and add features and have fewer lines of code at end of week than we had at the beginning of the week. I once got in trouble for having multiple weeks where we had negative lines of code -- we didn't know the 'grand boss' over our team was reporting SLOC as if it were progress. Sigh. 

Programmers will gladly explain that the work they did was reading, learning, understanding, sometimes guessing, researching, debugging, testing, compiling, running, hypothesizing and disproving their ideas of what the code should look like. In short, they were thinking and deciding.
Most of what goes on is intellectual work.

One of the quora responders wrote:

You see the fingers flying over the keyboard; you don't see the hours spent in talking to users, discussing the problems with coworkers, doing research and thinking the problems through.
Another suggested:

I achieve it firstly (to the extent that I do) by 'helping' the customer to eliminate the unnecessary notions from their idea, which they often mistakenly call 'requirements' and sometimes even say they are 'must have'. This is the biggest possible acceleration in the delivery of a solution because I can do an infinite amount of no work in no time at all.
And yet another:

Really good developers do 90% or more of the work before they ever touch the  keyboard; really understanding the requirements and devising an appropriate solution.
These are not unique unusual answers. I find that most of the time, "knowing what not to write", "doing less," "working in smaller steps", and "having first figured out what to do" are common answers. Programming is much more about thinking than about typing.

I have examined a lot of the change logs (diffs). It has consistently looked like 30+/-10 minutes of change on a good day (at least to me). 

I'm confident enough to tout this number as effectively true, though I should mention that no company I work with has so far been willing to delete a whole day's work to prove or disprove this experiment yet.  Remember, I have only estimates and examinations of daily diffs to work from. The result here is not scientific.

I should also let you know that people who do more typing or more cut/paste are often doing less thinking and understanding, which results in more errors and more burden on other programmers to understand and correct their code.

So What?

If programming is 1/12th motion and 11/12ths thinking, then we shouldn't push people to be typing 11/12ths of the time. We should instead provide the materials, environment, and processes necessary to ensure that the thinking we do is of high quality. 

Doing otherwise is optimizing the system for the wrong effect.

What if we changed our tactics, and intentionally built systems for thinking together about software and making decisions easier to make? I think that productivity lies in this direction.

So I invite you: how can you experiment with learning on-the-job to create systems where the thinking is optimized?