Full Test Access With Data Hiding for Production

I'm talking to users of statically-typed languages today. Functional and dynamic language people, cut them some slack and realize I'm not talking to you.

Breaking with Conventional Wisdom

There are certainly Object-Oriented design pundits who will argue that you should never, ever have public variables in a class. The reason is good enough, that the object should be responsible for its own state. You should tell it to do things, and it should do them. Likewise you should not manipulate its variables for it, since that requires a strong understanding of valid states for the objects, and this understanding is a strong coupling. Strong couplings between classes are bad things. All of that wisdom is correct and proven.

However, many programmers make bean-ish objects where every private variable has a getter and a setter. This helps to preserve uniformity of access (you can't tell if it's calculated or stored, method or variable), but is otherwise the same as exposing public variables. When uniform access is important, this is a good practice.

The Proposal

When writing tests, data-hiding and uniformity of access are unhelpful. While many say that a class should be tested through the interface it uses in production, we still recognize that testing is improved when there is unimpeded access to a class. How often do you hear of people foregoing tests or adding a "testable" work-around simply because a method is declared protected or private?

There is a way to harmonize both the desire for object self-management and the need for testing with full access.

Some (hopefully most) classes are not built to be used directly by other classes, but through interfaces. A method outside of the interface does not exist to the classes that use the interface. This invisibility is "data hiding" and is a good thing. Private methods approximate this invisibility too well, making details of representation invisible to tests as well as to users.

If we program through interfaces, then we find that adding public methods to classes (not their interfaces) is a non-issue. We are without fear of programmers pillaging our object's state. The non-interface methods do not pop up in the code hints or code completion windows. They do not exist, and an attempt to call our public non-interface methods are compiler errors. We have the data hiding we need.

Tests will usually be working against concrete implementations, and are not tied to the small set of methods presented by the interface. Tests automatically have "privileged access" to methods outside the interface of the class. Data hiding therefore can exist alongside full test access.

Some classes inherit implementation from others. It's a code sharing mechanism with a long and storied history. These derived classes will have access to the public methods of their base classes. So test them well.

Side Effects

There is an additional advantage to all classes implementing interfaces, in that interfaces make it easy to build test doubles. Some languages have facilities for partial-mocking of classes, but an interface only makes the work easier.

Moreover, most statically-typed languages have IDEs with refactoring support, so that "extract interface" is a relatively trivial thing to do. If it is easy to make an interface, easy to use an interface, easy to mock an interface, and the interface effectively hides the interface used by testing, it is hard to understand why people do not use them more.

UI components are a useful class of objects used through interfaces. They are called by some kind of a UI framework, through interfaces it provides. The UI framework doesn't know your class from any other class. So why do the UI classes keep private methods and private variables? The only callers of the class cannot call them even if they were made public. If some other classes in the UI need to use a UI component, let it be used through an interface which constrains the set of usable class features.

Another instance of classes not needing private variables are test harnesses and test fixtures. Against whom are they protecting their private state? Against the test runners? Against production programmers? Against other tests? Horse feathers! There is no real need for tests to preserve their internals.

Sound Bite

IFF we program to interfaces, THEN private implementation variables and methods are not only redundant but also impede testing. Let's consider a little shift in our thinking if it will make our work easier.

Even simplified, it can be daunting

SOLID design
FIRST-rate tests
Virtuous code

I understand the "just get it done mindset" because there is a lot of learning, thinking, and doing behind those three simple links. It is daunting. Any developer who can sling code could become considerably better at coding and more desirable as a teammate if he would learn and apply these concepts in his daily work, but it won't happen overnight.

Jeff and I built these 3x5 cards and simple explanations on the web site, but we don't pretend that you can live on these sound bites alone. The cards tell you what the words mean, and that's a good starting point (better than winging it), but the part that matters most is the part that won't fit on a card, the part where one practices.

I like the word "practice." Wherever you are writing code today, it is practice. You are expected to apply principled development and get better, but even if you are slap-dash hacking things together, you are building your mental habits and your reputation. You could have a reputation for overdesign, poor quality, quick hacks, and spaghetti coding. You could choose a reputation for steady progress, high quality, and code that makes next week's programming easier than this week's programming.

However daunting, it's doable. If we put a little of SOLID, FIRST, or the Virtues into our work today, we begin practicing principled development. This practice will deepen our understanding of principles and retrain our mental reflexes. Eventually our reputations will improve. We'll have some iffy moments along the way, but becoming a great programmer takes time.

The exciting part is that it can be done, we can do it, and a bunch of people have spent the time to outline it for us. This is what excites me about agile in a flash, where we can build references to the works of other software developers to further build our skills. Feel free to print some of the cards, post them in your workspace, and use them to retrain your programming habits. Drop us a line to let us know how it's going.

Even simplifed, it's daunting... but even daunting, it is the way forward.

When The Test Goes Green

My assertion today is: If we would read the code we're writing, we wouldn't write the code we're writing.

We know the red->green->refactor cycle, but I find repeatedly that programmers are "optimizing" out the last step. It tends to run more like red->green->next. Perhaps there is confusion over what is supposed to happen when the test turns green.

The first thing that is supposed to happen is that you read the code you've just written to see if it makes any sense at all. While this sounds perfectly sensible, it often does not happen. Programming is not just typing. It is easy for programmers to rush to complete their assignments and never really examine their work.

I saw an example recently where the old code said:

    DateTime date = DateTime.Parse(datestring);

As a minimal-perturbation change to push to green, the developer changed the code to say

   DateTime date = DateTime.Parse(DateTime.Now.ToString());

Had he or his partner actually read that code, they would have realized that it was a little silly, and replaced it with

  DateTime date = DateTime.Now;

However, I suspect that they never examined it past the green bar. It worked, and that is a virtue, but it was doing too much and the team would be totally embarrassed to see their blunder in print. I fixed it, and the programmers remain anonymous here. They're good programmers, but this time it seems they sailed past the green bar without a thought.

When the bar turns green, read your code. Read the test code and read the code it invokes. It is not at all unusual to find that you have done something short-sighted, complicated, or (God forbid) clever. That code needs to be cleaned up before your peers see it!

For a brilliant example, see Raymond Hettinger's video on pythonic code.

Once you read the code, you will know what is wrong with it, and then you can use the various refactorings provided by your development tool or outlined in the fine Refactoring book. You can take your code from one "green" (all tests passing)_ state to another until it is no longer apparent that you ever did anything silly to begin with.

Be particularly suspicious of any code you've written into your tests or test fixtures that ought to be in the classes you are testing. Tests are meant to be extremely lean and fast-running, whereas business objects are meant to be capable. If you start writing capable test fixtures that call upon lean business objects you know you've gone wrong. Refactor the code.

I wish that we could lengthen the sequence to red->green->read->refactor so people would understand the process and apply it. I suppose that we take too much of "refactor" for granted. Nevertheless, the secret to TDD is in that third step, and all refactoring begins with reading and evaluating the code we've written.

After all, a program unexamined is not worth writing.

Motivational Speech

My biggest problem with American culture is the concept of "the locker room speech that wins the game". I don't doubt that people get down and can use encouragement. A little validation can go a long way, and mistreatment breeds dysfunction. If you treat a winner like a loser, expect to see some losing. I get all of that (sparing you the painful personal history).

My problem is with the absurdist version of the story: the mythical team that is filled with execution issues, poor teamwork, low degree of individual skill, and an enemy with both superior skill and a history of success. In this story, it is one stirring talk from the tough-but-beloved coach that turns the team around. They charge the field and play like a commando team, each overcoming all personal skill issues and all of them meshing into a perfect winning machine.

In the real world, it is unlikely that the problems a technical team faces are motivational in nature. Maybe a team is struggling with technical problems. Maybe they haven't learned to support each other with their efforts and need some training. Maybe they lack the individual skills necessary for them to do a good job. Maybe they are skilled, but bound up in organizational boundaries, official roles, rules, and red tape. Maybe there is a very poor system of rewards and compensation, so that they are punished for doing what is best for the company. Commonly (for programming teams) the years of heroic last-ditch efforts have left the code base a non-performant, convoluted, fragile mess of hacks and band-aids. Maybe they aren't getting access to the Customer that will help them make good decisions.

It is ever-so-possible that most people on most teams in most companies are really dealing with actual problems. It could be that they're not just slacking or slumping, but that they've hit some walls. I have never yet seen a slow build, a performance bug, an intermittent failure, or a sloppy code base that yielded to emotional pressure instead of a breakthrough discovery. I've never seen a team burst through organizational barriers by happy think instead of hammering out working relationships and building a good history of collaboration together one feature at a time.

The problems our teams face are not all in their heads. The least sympathetic, least effective, least meaningful thing we can do when people are facing real technical or organizational problems is to try to prop them up with positive motivational speeches or "motivate" them with threats and insults.

If we want to be effective we must try giving our people the resources, aid, and freedom to solve problems. We have to free up some space for the work they have to do, and understand that there is no meaningful way to estimate the time to discovering a problem you do not yet understand. We have to allow them to experiment, make mistakes, learn, discover. I will not be helpful to a struggling team until I drop the motivation and apply some problem-solving or bring in some useful talent to help with the problems.

If we give in to the desire to give a locker room speech and our team succeeds, we have to resist the urge to claim credit. It is their hard work and skills that made it happen. I've been in the technical side of the software business for 30 years, and offer assurance that technical teams succeed despite these speeches, not because of them.
Contrary stories are welcome.

Code is not a Car

If a man was to borrow his his boss' car, drive it hard, fast and carelessly, and then return it with the interior filthy and ripped up, he would probably get fired (or be first in line for the next cut).

Doesn't it seem funny how many people do the same thing with their employers' source code?