Bash coolness

There is a clever feature in bash, in that you can do a lot of manipulations while accessing a variable. Note that I always use "clever" in a pejorative sense, but while this is clever (argh) it is also helpful to me. I find that I, like all users, will tolerate a little cleverness if it gets my work done.

In this case, it's the substitution features that get my attention.
For variables:

${VAR:-zzz} returns $VAR, or "zzz" if $VAR is empty
${VAR//x/y} returns $VAR after replacing all "x" with "y"
${VAR:=zzz} returns $VAR or "zzz" if VAR is unset, and assigns VAR (ick)

And of course the coolest version is the $() operator (AKA: command substitution), which used to be done with ugly backticks. This runs a command and returns the output of the command into a variable:

$(echo $fname| cut -d. -f1)
$(echo $(basename $fname) | cut -d -f1)

Which leads to this kind of "clever" coding:

dest=${DESTDIR:-/tmp}/$(echo $(basename ${source// /_})| cut -d. -f1|cut -d= -f1).avi

Which is a pure evil incantation that takes a filename ($source), prefixes it with the destination directory (which defaults to /tmp), strips the path off of it with basename, passes that to a cut filter to remove all dotted suffixes (leaving a raw filename) and then cuts off any suffix that has a "=" preceding it.

The result of passing a source like /aaa/bbb/ccc/ddd=12312.mov.tmp would be a string like /tmp/ddd.avi.

That's evil. It happens to be handy for a transcoding script of mine, but it's pure evil nonetheless. If I come across code like this in the future, I'll want to claw my eyes out.

But the substitution thing is handy.

It doesn't have to be in a single line. I think "one effect per line" and "one trick per line" are pretty good bash programmer rules. One shouldn't try to reach a maximum clever trick density. Really.

James Grenning boils it down

From the XP mailing list, James Grenning has this to offer:

> There are two values to software
> 1) Business value delivered
> 2) Ease at which the next important feature can be delivered

Fear of Changing Code

A common question on the TDD and XP mailing lists is how to get managers to approve refactoring and testing. The standard response is to ask in return why programmers need permission to write code that doesn't suck. The problem is that it is hard to know how to manage software development in a condition of rising Technical Debt.

I was a programmer a long time before doing any management. I had to learn to manage my unit of work, to do things people wanted to have done, etc. There is a certain amount of professional discipline necessary to be a good employee-programmer. I had a tendency to do X (which was required) and throw in some Y and Z. Where Y and Z were simplifications along the path to doing X, it's all good. This shouldn't need permission (one would hope) but what about when they were solving miscellaneous potential problems on the side? What if they scratched an itch? Under the reign of SAS70 interpretations and ISO 900x dreams and basic risk management, I had to learn to operate a little differently.

Avoiding fixes is an attempt at risk management. When you are reliant on manual testing and inspection (against Deming's advice), you are always concerned with the cost of testing and inspection. If technical debt is out of control there may be changes that take a week to code but which bounce around in QA (or QA test/fail/fix/retest cycles) for a month or even more. All estimation and planning is moot at that point.

"Change avoidance" is a knee-jerk response. A review board/person will choose which changes they can afford to test instead of choosing changes based on business value. The developer then takes a change ticket and performs the minimal work that clearly solves the problem as explicitly described in the ticket. The diff of the code change is reviewed, and the list of modules and screens modified will drive the laborious QA processes.

In this situation, collateral change is unwelcome. Even reformatting is fearful. Refactoring makes the diff harder to read. The unenlightened will place too much emphasis on a tidy diff rather than whether the change was an improvement overall. In this situation, even the smallest hygenic improvements will seem to be needlessly add risk.

A team finds itself making interest-only payments on technical debt.

Things won't get better, but the team will get better at working small in a big mess. This is akin to a teen boy clearing a narrow path from his bed to his door instead of cleaning his room. It's like washing only the dishes you need to cook and eat the current meal instead of doing them all after the meal. But not doing so feels like losing control.

"No passion so effectually robs the mind of all its powers of acting and reasoning as fear." -- Edmund Burke

"Fearing your own code is an indicator that you are headed for ruin. This fear is followed by self-loathing, project-loathing, career-loathing, divorce, infanticide, and finally chicken farming." -- Bob Martin

The problem is that technical debt is already out of control. Minimal, pretty diffs and long inspections won't create control. It will only foster the illusion of disciplined development.

Rewrites are doomed for the most part. The reasons are well documented and well understood. The risks of gaps and scope creep are legend.

Yet we see that code, left to a patch-n-release process, becomes messy and unmaintainable. It is the slow march of entropy and complication. Joel puts it this way:

Back to that two page function. Yes, I know, it's just a simple function to display a window, but it has grown little hairs and stuff on it and nobody knows why. Well, I'll tell you why: those are bug fixes. One of them fixes that bug that Nancy had when she tried to install the thing on a computer that didn't have Internet Explorer. Another one fixes that bug that occurs in low memory conditions. Another one fixes that bug that occurred when the file is on a floppy disk and the user yanks out the disk in the middle. That LoadLibrary call is ugly but it makes the code work on old versions of Windows 95.

This is why managers and indeed whole organizations want to tighten down on development, and demand minimal work though it brings minimal satisfaction. This dissatisfaction leads to cries for rewriting the application. Rewrite reduces the staff on hand for making changes, and can even lead to a total work-stop in which no value is delivered to the customer base.

The only way out is to pay down technical debt, which means making changes, which feels an awful lot like losing control. This is why the question comes up so often. Selling "unmanaged change" is hard to sell, and why software managers know and object if you reformat and refactor your code.

Shameful Confessions from a small Solo Project

I had a small task to build a subversion post-commit trigger. I didn't know my way around the subversion library, and so I started spiking... the old way. I fired up an interactive session and tried playing with it. I created a repo by hand and started running merges and copies and checkins, and had the trigger print stuff out. There is Nothing wrong with doing that, by the way.

The problem was that I didn't toss it out and TDD the new one into existence. I TDDed my way through any new routines, and extracted-n-tested any tricky bits until my main loop was much smaller, but I didn't toss it and TDD it back into existence. I kept the prototype.

As a result, I have this code that's pretty simple and nice, and there in the middle is this post-coding-tested wart. It works, it passed code review, but I'm not going to put this one up on my fridge. The routine has a triply-nested loop and is at least a dozen lines long. It's disgraceful. Worse yet, the tests are hard to read because of the deep structure.

First chance, I'm rewriting that thing. I have to recover some professional pride.

I'm becoming an Esther Derby fan, partly because of her statements about employee motivation and demotivators..

I became acquainted with her work through anagile toolkit podcast in which she was originally invited to discuss retrospectives and making them stick. The conversation turned toward motivation and demotivators, and now I have to go buy some books.

Better yet, I need to make some changes.

Opposite of Despair...

... is "pair".

Just had a good pairing session. Our result is small and simple, probably not enough to justify our time if you counted lines of code, but involved lots of learning and navigation and refactoring. It was a lesson in doing less.

I was in a bad mood earlier, but a good pairing session with working tests can change my outlook on the whole day. Wish I had one of these at 8:30 this morning!

Collaboration and production give me energy that other tasks can take away.

Quick Status

We've had some upheaval and change with RIF and new directions. There are essentially three projects running now. One is for data management and product support relief, one is for performance improvement, and one is a new feature that is expected to provide crucial value and income for the company. We also have a very small project going to lower costs and improve accuracy of some important financial data transfer with external parties.

What is going well:

• These really seem to be the right high-value things for us to work on.
  
• Customer involvement is up. Each team has direct contact with the people who will benefit from the feature, or a reasonable (set of) surrogate(s).
• Those who "get it" are beefing up their code-test efforts. I can't really say they're doing TDD properly, but there are ATs first and usually there are UTs written in the course of the work. We'll get there.
• Amongst certain team members, pairing is up.
• I'm able to spend more time on the testing practice
• We've begun doing code reviews
• Dan and I are working out the kinks in our branch management.

What is going badly:

• We're getting burnout from developers working really hard to get things done. Tempers are up, sarcasm is up, and I'm waiting to see how much quality is taking a hit.
  
• Tracking has lapsed despite my efforts. It's hard to say how close some projects are to completion, because there is no burn-up, no card wall, etc.
  
• Some projects are running on heroic efforts of individuals, and individuals who like being the unquestioned, process-free supermen. Compliance with process is the lowest I've seen it here.
• Pressure to be busier, look busier, stay longer is up. Sustainable pace is not really part of the game plan being pushed down. Mistakes being made, tests being skipped, branch management, etc. It never pays to rush software.
  
• Code review is a poor substitute for good pairing, though it beats poor pairing in some ways.
  

My greatest concerns at this time are:

• Without tracking, points-completed and points-remaining may not reach convergence in any predictable way. Features are likely to be late. I'm betting another three weeks in QA round-tripping. I hope not, but that's my expectation.
  
• In superman projects, I fear there is insufficient quality practice and "finished" work will get later and later, or will be pushed with glaring defects.
  
• Too much WIP. Things once started, even nearly-completed, are back-burnered.
  
• Burnout will increase, with lethargy and apathy and histrionics diminshing professionalism and discipline.

My hopes are:

• Customer involvement will continue.
• Existing tests will prove their value.
• We can resume sustainable pace and discipline soon.
  
• I am able to put enough time into testing to improve the experience and value
• Our branch management will be better.
• We'll be able to soon gauge whether these change will provide the value we anticipated.

XPairTise problem or am I just being dense?

The XPairTise Eclipse plugin is supposed to enable remote pairing, but the server is hard-coded to use "tcp://localhost:61616".

Localhost? How am I supposed to do remote pairing when the pairing server is bound to the loopback address? Are other people modifying the code to a fixed address? Does that make sense to anyone?

I guess I can download the project from version control, create a version with an option for the port to listen on, roll it into a new jar, and copy that into the directory where I start the app, but it would seem that nobody could use it as-is. No? Am I missing something?