Awesome programming jargon
Posted by Stan Taylor on May 11, 2010 | Permalink
My favorites from here:
Bugfoot - A bug that isn’t reproducible and has been sighted by only one person.
Hindenbug - A catastrophic data-destroying bug. Oh, the humanity!
Shrug Report - A bug report with no error message or “how to reproduce” steps and only a vague description of the problem. Usually contains the phrase "doesn’t work."
Smug Report - A bug report submitted by a user who thinks he knows a lot more about the system's design than he really does. Filled with irrelevant technical details and one or more suggestions (always wrong) about what he thinks is causing the problem and how we should fix it.
SilkTest's replacement for the Exists() method
Posted by Stan Taylor on March 31, 2010 | Permalink | Comments (1)
I've recently started a new SilkTest project testing a web application, using SilkTest's new open agent and the 4Test scripting language. This post covers an aspect of what I've learned.
In the SilkTest 'classic' agent, you used the 'Exists()' method to test whether an object exists in the application under test, e.g.,:
if (Page.Object1.Object2.Object3.Exists())
// do something...
With the open agent's dynamic object recognition, the Find() method is what you need to use, but it took me some research to figure out how to do an if statement with the find() method. Here's a test:
if (Desktop.Find("//BrowserApplication//BrowserWindow//INPUT[@id='sysname']",{5, false}) != NULL)
// do something
You'll notice that I added an optional argument: {5,null}. These two values constitute a FINDOPTIONS data type record. The first one is the timeout. The second value is the important one for our purposes: it "determines whether the Find method throws an E_WINDOW_NOT_FOUND exception if no object is found or NULL is returned if no object is found. "
So, you set that value to FALSE and then test to see whether or not the Find() method returns NULL. If not null, the object exists.
Borland SilkTest's new open agent
Posted by Stan Taylor on March 31, 2010 | Permalink
I've been a user of Borland SilkTest off and on since the late 1990s. After not having used it for several years, I picked it up again when I worked at Borland 2006-2009. Since it was our company's own tool, we intended to use it to automate testing of the web UI application I was working on. However, we faced some significant challenges.
The first problem was the well-known problem with window declarations. Due to the depth of objects in our web-based UI, maintenance of the window declarations was an onerous task. The other problem was that the agent and recorder simply didn't interact very well with our AJAX-y UI (we were using ExtJS for the UI).
Since that time, Borland has released the new 'open agent' for SilkTest. I began using the open agent right before I left Borland and I'm creating a new project with it here at Polycom. I'm happy to say that the SilkTest dev team has done a really good job of overcoming the previous shortcomings of the 'classic' agent.
The biggest problem I'm facing now with my SilkTest open agent automation is access to information. The user documentation for the open agent is not as mature as the documentation for the open agent, and as a result, I've opened numerous support tickets with Micro Focus to figure out how to do things.
I figured I should use my blog to share some of the things I've learned about the open agent and its 4Test implementation. Stay tuned...
Testing and Toyota
Posted by Stan Taylor on March 21, 2010 | Permalink
Testing rock star James Bach has published several good blog posts about the Toyota braking problems: Advice to Lawyers Suing Toyota, Toyota Story Analysis, CNN Believes Whatever Computers Say.
The following line struck me from the 'Advice' post: “Extensive testing” has no fixed meaning. To management, and to anyone not versed in testing, ALL testing LOOKS extensive. This is because testing bores the hell out of most people, and even a little of it seems like a lot.
That's very true except when a bug slips through and gets caught by users.
That's when it's so much fun to remind management that they chose the amount of testing that let this bug slip past. Test lead to management: Remember way back when, I presented you with some options regarding testing: if we have X amount of time, we'll get Y amount of testing done, here's the way we prioritize the work and the associated risks, or if we have X+A amount of time, we can do Y+B amount of testing done, with the following risks.
Where's s/Waldo/Stan?
Posted by Stan Taylor on March 05, 2010 | Permalink | Comments (1)
As you may have noticed (those of you who still visit here), I haven't posted anything substantive here in several months. Lots of reasons both personal and professional.
But let's not dwell on the past. Let me tell you what I'm up to now.
At the first of this year, I started as the lead of a newly-forming test automation group at Polycom, working on the HDX video conferencing appliances. Although I'm not directly involved in the agile process here at Polycom, our new automation team is playing a crucial role in allowing the development organization to act in a more agile fashion.
Our product line is based on a codebase that has been around for a number of years, and since we develop entire appliances, it's a pretty complex development environment with traditional applications, OS development, drivers, video and audio codecs, interoperability with other devices (both ours and other manufacturers'), etc.
Each software change goes through three levels of 'promotions', as we call it around here. First, it's checked into the agile team's codebase. Then, the team's codebase is periodically promoted to the 'integration' code stream, and finally that codebase gets merged periodically into the global codebase.
Currently, promotions to the highest levels take place infrequently, these promotions are a significant event, and a lot of manual testing has to be done to ensure that the code to be promoted hasn't broken anything.
This process has several problems. First, as alluded to above, a tremendous amount of time is devoted to manual regression testing. Second, regression issues are frequently not identified at the team level, and there are frequent high severity bug crises during promotions testing or, if a bug escapes through that, after the promotion takes place. And finally, since promotions are not daily, teams do not pull down the latest shared code frequently. So, a team develops against stale shared code for up to weeks at a time, and when they are ready to promote their code, a lot has changed in the shared codebase, leading to significant code merge problems.
The solution to this problems is obvious to agilists: continuous integration and automated testing. And in fact, that's the initiative my group is involved in. Our goal is to have daily builds at all codebase levels and to run a suite of automated tests on each build. Considering the size and complexity of our environment, it's an ambitious project, but I'll keep you posted on the progress.
Documenting code changes with defect reports
Posted by Stan Taylor on January 20, 2010 | Permalink
Today, Rafe Colburn listed four reasons to file the bugs found in code reviews. A commenter points out that defect reports aren't the only way of communicating about changes to the code:
I guess it depends on the local culture, but in my experience, developers only look at a bug report if it’s assigned to them. The revision control system is a better way to see what’s happened recently.
In my ideal system, I would take things a little further: every commit must have one or more work items (requirements, defect reports) associated with it and an indication of whether each is in progress or completed. The argument for this is pretty simple: if you're not implementing a requirement or fixing a bug, then why the heck are you changing code?
Additionally, the build system should display the commits in the build, the work items associated with each commit as well as a list of the changed files and an easy way to view file diffs for changes in each commit.
As a QA engineer, my need to see completed work items is obvious. However, the list of changed files and diffs provide a different type of equally useful data. This information provides me an easy way to familiarize myself with the code, input in deciding how to test the change, and opportunities for starting discussions with the programmers about their code.
When I describe this system to programmers, their first thought is often that it requires a lot of red tape/documentation. I have only worked with a system like this once in my career, and in that situation, the programmers did not find it onerous. It's true that the programmers had to file defect reports for bugs that they fixed and found, but we relaxed our defect report standards in such cases; they didn't have to fill out severity, steps to reproduce, etc., which allowed the developers to spend a very short amount of time filing the reports. We decided that having a minimal 'placeholder' defect report was good enough in many such cases if it allowed buy-in from the developers. Besides, as mentioned above, the reporting in the build system was a backup source of information about code changes.
Works on my machine
Posted by Stan Taylor on December 21, 2009 | Permalink
My favorite MeFi comment about this video: In my mind, the development work is done in the United States, and the QA testing is done in India, and QA keeps filing bugs that the face tracking software doesn't work, and development engineers keep closing the bugs unfixed with the resolution "WORKS FOR ME".
Algorithms Gone Wild
Posted by Stan Taylor on December 15, 2009 | Permalink
As noted by Consumerist, the screen shot below is from a search on Overstock.com for the term 'Disney princess." Note the 'People who searched for..." box on the right:
Oops.
This is depressing
Posted by Stan Taylor on November 17, 2009 | Permalink
The Incompetence of American Airlines and the Fate of Mr. X
In short:
- Graphic designer has poor experience with American Airlines' web site
- Said designer mocks up a better page design for AA.com and blogs about it
- UX designer at American Airlines (who is clearly passionate about his work there) emails said designer, offering some insight into the challenges that they face with such a large team, a large web site/appliation, etc. Not all of the info is very flattering about AA, but, in my opinion at least, very typical of such companies and sites
- AA fires the UX designer.
From the graphic designer who started this whole thing:
When I first learned about [the firing], I was horrified. Mr. X is actually a good UX designer, and his email had me thinking there was hope for American Airlines. The guy clearly cared about his work and about the user experience at the company as a whole. But AA fired Mr. X because he cared. They fired him because he cared enough to reach out to a dissatisfied customer and help clear the company’s name in the best way he could.
UPDATE: I posted this to MetaFilter and got a lot of interesting comments.
Reinventing the wheel?
Posted by Stan Taylor on October 07, 2009 | Permalink
When I was job hunting earlier this year, I applied for an opening at S3 Technologies. At the time, I was puzzled by the emphasis on the military background of some of the company's founders.
Today, I happened to visit S3's web site and I was struck by something else. They seem to have invented a new methodology for software development:
Introducing Fight ClubFight Club is an important step in what we've termed SPS, or the S3 Production System.
SPS is a uniquely efficient method of software delivery that solves your specific problem. Fight Club refers to four experts in software development and business working together in a room, removed from all distractions, and having every resource they need to quickly build and deliver the perfect solution. We realize that every company that makes a product, whether it is an automobile, a refrigerator, or a software product, can be more functional and efficient in development and assembly. S3's scientists and analysts studied the best practices of global giants like Toyota and also looked at small but smart and growing technology businesses. In conducting our studies, we focused on best practices and time-tested methodologies and how they might be applied to the development of our solutions. We even improved a few of these processes along the way.
The result is SPS. We have refined the normal software model of design, build, test, and deploy in a manner that facilitates time to launch and quality of the product delivered. We believe in the "go and see" approach of Toyota, which means we sit down with your workers to see precisely how they're doing the job that we're going to automate. An intense verification and profile of your data beforehand will further tell us both what can and cannot be done.
And we've discovered there is often a difference between what a business thinks it "wants" and what it actually "needs." That's why we intensify the verification process and bring in quality assurance experts long before a solution is ready for development. We acquire a true understanding of what our customer needs and then we design and deliver the best product on the market faster than anyone else in the software business. And yes, we know that is a bold statement. But we're only saying it because it's true.
Wow. Why didn't anyone else ever think to study Toyota's manufacturing practices and apply it to software development? Furthermore, what an innovation: nobody has previously had the insight to put "experts in software development and business working together in a room, removed from all distractions, and having every resource they need" in order to "quickly build and deliver the perfect solution."
Very odd.
