Pick me! Pick me!

Posted by Stan Taylor on July 01, 2009 | | Comments (1)

I'm a member of several QA-related groups on LinkedIn, and I receive periodic update emails listing the discussions in the respective group. Often times, there is one or more posting in the group along the lines of, "If you want to expand your network then please invite me to [email address]. II will accept your Linked In invitation."

In some cases, these posters are recruiters, and I can understand their business reason for having a large network of people who they don't really know. In other cases, though, the poster seems to be some QA-related professional. I don't understand how these individuals think they can benefit from having a large network of strangers.

Any ideas?

One related observation: most of the these posters seem to be in India.

 

Agile testing link treasure trove

Posted by Stan Taylor on May 29, 2009 | | Comments (0)

Grig Gheorghiu has compiled a list of all agile testing-related links that he has put into his agile testing blog. Lots of good stuff there.

 

Types of automated testing

Posted by Stan Taylor on May 20, 2009 | | Comments (0)

Over at his I. M. Testy blog, BJ Rollison offers succinct definitions of five approaches to automated testing: record and playback automation, keyword or action-word driven automation, scripted automation, procedural automation, and model based automation. It's a good quick reference.

 

New gig

Posted by Stan Taylor on May 20, 2009 | | Comments (1)

I haven't posted much lately because I've been in transition between jobs. I'm in my first week in my new job as QA Lead at Hyperformix. The product dev teams at Hyperformix have just embraced agile in earnest and are about to wrap up their second sprints. Hyperformix's new dev director, Matt Roberts, has some good agile experience under his belt, and I was hired, in part, due to my experience with agile at Borland.

As I was going over the current sprint's team board with Matt this morning, it occurred to me that this was the first time I've gone into a new environment that's using scrum. Previously, when I started a new job, discussions about how the new company does things were completely one-sided. The person I was talking to would describe the process.

In this case, however, it was much more of a dialog since Matt and I share a common standard process and vocabulary. Matt would describe how they do something, and then I would ask, "But the book says to do it this other way." And Matt would respond by describing in more detail how they've implemented the scrum process in question or how they've chosen to deviate from the standard process and why. It was a very pleasant experience!

 

Agile as group therapy

Posted by Stan Taylor on April 28, 2009 |

Over at Borland's Agile Transformation group blog (which I contribute to), my colleague Jeff Brantley compares agile to group therapy. I think it's an apt analogy, and it is definitely in line with my thoughts regarding agile and AA.

 

Descriptive vs prescriptive testing

Posted by Stan Taylor on March 10, 2009 |

Over at his Collaborative Software Testing blog, Jonathan Kohl has an interesting post about Descriptive and Prescriptive Testing which he defines as follows:

A prescriptive style is a preference towards direction ("do this, do that") while a descriptive style is more reflective ("this is what we did"). Both involve desired outcomes or goals, but one attempts to plan the path to the outcome in more detail in advance, and the other relies on trying to reach the goals with the tools you have at hand, reflecting on what you did and identifying gaps, improving as you go, and moving towards that end goal.

Jonathan also explores the personality types that are drawn to each type of testing, and he relates how he recently incorporated descriptive testing into a prescriptive environment:

For example, I helped a friend out with a testing project a few weeks ago. They directed me to a test plan and classic scripted test cases. . . Within an hour or two of following test cases, I got worried about my mental state and energy levels. I stopped thinking and engaging actively with the application and I felt bored. I just wanted to hurry up and get through the scripted tests I'd signed on to execute and move on. I wanted to use the scripted test cases as lightweight guidance or test ideas to explore the application in far greater detail than what was described in the test cases. I got impatient and I had to work hard to keep my concentration levels up to do adequate testing. I finally wrapped up later that day, found a couple of problems, and emailed my friend my report.

And then he explains how he developed some descriptive testing:

The next day, mission fulfilled, I changed gears and used an exploratory testing approach. I created a coverage outline and used the test cases as a source of information to refer to if I got stuck. I also asked for the user manual and release notes. I did a small risk assessment and planned out different testing techniques that might be useful. I grabbed my favorite automated web testing tool and created some test fixtures with it so I could run through hundreds of tests using random data very quickly. That afternoon, I used my lightweight coverage to help guide my testing and found and recorded much more rich information, more bugs, and I had a lot of questions about vague requirements and inconsistencies in the application.

Like Jonathan, I definitely lean towards the descriptive testing approach. In fact, you might say that the experience that I recently described in my post Just enough process: the checklist was an attempt to balance out prescriptive and descriptive testing.

 

Team liturgies

Posted by Stan Taylor on March 03, 2009 |

Over at the Agile Software Development blog, Janusz Gorycki reminds us that scrum activities--daily stand-up, planning, retrospectives--play an important role beyond the purported goal of each activity:

The quality of the team and the caliber of its members is more important than the efficiency of its processes – nothing controversial about this statement really, this fact has been well documented in some pretty classic books on team management ("Good To Great” by Jim Collins comes to mind). And the old fashioned rituals are crucial in maintaining the team spirit and identity. This is because people happen to be wired in such a way that it makes them feel safe and comfortable if they can devote some time every day to repeatable old habits. And teams become better integrated if they have some common habits that they care to repeat every day – as a team. Liturgies and rituals do matter a lot – and there is no reason for these rituals to be overly efficient. That's not their purpose to be efficient.

I love it that he calls these scrum activities "liturgies." As a church-goer, I think it's an apt analogy. I recognize that there is indeed a certain value in just coming together with my fellow parishoners each week and at the very least, going through the motions together. Though with church, as with work, you need to try to keep the point of the ritual in mind.

 

Just enough process: the checklist

Posted by Stan Taylor on March 02, 2009 |

Following processes is important for ensuring quality, but often processes become an end in themselves. Therefore, one of the stock statements you'll hear from me is: we need just enough process and no more.

checklist.jpgIn one company where I worked, we had a couple of developers on our project who were inexperienced at doing UI work. Every time one of them delivered a UI screen, the QA engineer would immediately find numerous UI defects. It would then take quite some time for the QA engineer to document the defects, then more time for fixing and verifying them.

After we'd gone through this a few times, we realized we needed some process in place to try to prevent these defects in the first place. The problem was, the UI defects related to assumed requirements. Our requirements stated, for instance (not a real example), that the user needed to be able to specify the following information when adding a contact to the address book: first name, last name, street address, city, state, ZIP, etc. However, the UI defects that the QA team was finding lay in details at a lower level than our specifications: tab order, various types of form field validation, consistency of error messages, etc.

The obvious solution would have been to make our requirements more detailed. This approach, however, would have taken as much or more time than the defect process was taking and nobody wanted to deal with that level of requirements.

Instead, I noted that the types of data we were working with were common (actually, I think basic contact info was in fact one of our areas of functionality) or at least well understood by our team members for the domain of our application. The problem lay not so much in the requirements but in the fact that the inexperienced UI developers were not used to thinking about all the little 'gotchas' in UI work.

My solution was a UI checklist for each type of UI screen. For a data input form, for instance, it included things like:

I presented the checklists to the developers as follows: we would like to ask you to make a good faith effort to address all of the items on the appropriate checklists before delivering UI screens to QA. We all understand the domain and data well, so I figure that, for example, the QA engineer will agree with 80% of your choices on how you implemented these details, 10% of the time the QA engineer will disagree, and 10% of the time there will just be a bug.

After that point, the UIs that these developers delivered to QA were noticeably more mature first time around, and my 80/10/10 example turned out to be roughly correct.

I think that the last part--making it clear that we trusted the developers--was the key in the success of this program. Instead of focusing on the fact that these developers were inexperienced, we gave them some guidelines and showed them that we trusted them to do the right thing--both in just following through on their verbal commitment to make a good faith effort to address these details in the choices that they made in doing so.

 

Concerns about unit test quality

Posted by Stan Taylor on February 20, 2009 |

In a recent blog post, Nat Pryce expressed a concern that I've long had:

I have observed that few programmers are skeptical enough about the code they write. They focus on success cases, ignore edge cases and error cases, try to bang out as many features as possible, and don't notice that those features are incomplete or will not work in some situations.

And then Nat goes on to explain how test-driven development can help developers improve their thinking about the tests that they write:

As programmers get more experienced with writing automated tests, they get better at thinking about how their code might fail. That's because they get plenty of opportunity to observe where their tests are insufficient and think about how to avoid such problems in the future.

I don't disagree with Nat at all, but I am still concerned that it's a case of fitting a case of trying to fit a round peg into a square hole: the developer may get better at writing tests, but she's still a developer, not a testing specialist.

This is, of course, one of the reasons why we perform other types of testing in addition to unit tests. But if we want to make sure unit tests are robust, it seems like we need to involve the testing specialist in some way. And having the developer pair with the testing specialist to review unit tests is the most obvious solution to em. Even if the testing specialist's skill-reading codes are weak, the developer can at least describe the tests to the testing specialist (If the testing specialist has the skills to write unit tests, having him (help to) write the unit tests would be another alternative).

I've pursued this type of pairing a few times, but it has never worked out for more than a couple of weeks. The primary reason for this failure, in my opinion, is lack of buy-in from the developers. They don't see the value of having a testing specialist review their unit tests, or unit testing just somehow remains the property of the development club.

Any advice or experiences you can share would be helpful.

 

Agile or else!

Posted by Stan Taylor on February 20, 2009 |

(Via Agile in Action)