Testing computer software

The best thing about Testing Computer Software is its practical point-by-point guide to everyday software testing, from creating a test plan, to writing effective bug reports, to working with programming staff and management to fix bugs.

That said, this book's early frame of reference shows how far we've come. (The book relies heavily on MS-DOS examples and features some truly embarrassing anachronisms, including the mention of testing dot-matrix printers and even EGA/VGA video modes.) The bibliography stops at sources from 1992 and features many references from the 1980s. Nowadays, automated software testing tools are the staple of any testing strategy. This book even advocates a wait-and-see approach to the "new" Microsoft Test.

These limitations aside, there is still a good deal to mine here. Much of the approach to testing is still very valid for any aspiring or working test engineer. Clearly, readers of the first edition will have little reason to upgrade to this second edition, but for anyone who appreciates a "classic" (and indeed a pioneering) text in the field of software testing, it's good to have Testing Computer Software in print again. --Richard Dragan

Topics covered: test case design, test planning, project lifecycle overview, software errors, boundary conditions, bug reports, regression testing, black box testing, software quality and reliability, managing test teams, printer testing, internationalization, and managing legal risk.

Booknews, Inc. , October 1, 1993
A realistic, pragmatic introductory guide to testing consumer and business software under normal business conditions, for testers and test managers, project managers, programmers, and students. The authors have all been test managers and software development managers at well-known Silicon Valley software companies. Annotation copyright Book News, Inc. Portland, Or. --This text refers to an out of print or unavailable edition of this title.

Booknews, Inc. , February 1, 1989
Covers some 400 types of software errors and how to cope with last minute changes; detect design errors in the user interface; set priority strategies; estimate, plan, and schedule tests; manage test groups--hiring, avoiding traps, dealing with outside testing companies. Annotation copyright Book News, Inc. Portland, Or. --This text refers to an out of print or unavailable edition of this title.

Booknews, Inc.
A realistic, pragmatic introduction to testing consumer and business software under normal business conditions. The authors have tested software and managed testers for Silicon Valley software publishers, and wrote the book as a training and survival guide for their staffs. The second edition is more geared to college courses.

The author, Cem Kaner, kaner@kaner.com , April 8, 1999
This is another printing of the Second Edition
This is the same book as the Van Nostrand Reinhold / Thomson Computer Press / Coriolis editions of Testing Computer Software.

If you already have the 2nd edition of Testing Computer Software, you already have this book. This is not a new edition.

The author, Cem Kaner, J.D., Ph.D. , June 29, 1997
Author's comments on Testing Computer Software
Testing Computer Software is about black box software testing. The approach is pragmatic, and intended to reflect the engineering style followed in Silicon Valley.

I started writing this book in 1983, while working at MicroPro (WordStar) as the Testing Technology Team Leader. The books, courses, and standards available to us didn't reflect our style of engineering, or the role of the Software Testing Group in our company. This was a widespread problem--our development style was typical of the best mass-market software developers in Silicon Valley.

Testing Computer Software reflects several realities:

First, the software is usually developed from very abbreviated specifications. You can't base a test plan on detailed internal and external specs because they don't exist, or they are impossibly out of date.

Second, the product is being developed for mass-market customers, not for a single customer who is paying for custom engineering. The single customer approves specs, signs off on them, and agrees to pay for a product that meets the spec--even if it's hard and disappointing to use. In contrast, in the mass market, customers don't look at the program until after you start selling it, and they don't buy it unless they want to.

Third, because the mass market's expectations change over time, the software's design, including feature set, will change significantly throughout the development of the product. Changes will reflect new market requirements and better understanding of the customer's needs for usable software. The solution isn't a bureaucratic change control system--you want to stand in front of a fast-moving train? OK, but you'll get run over. The NEXT test group manager will know that the real solution is to change your testing approach so that you can cope with continuing change. I'm not saying that we should embrace chaos. I'm saying that in the mass market software world, we have to expect and deal with a lot of change. Processes based on the fantasy that the "quality assurance" group can exercise extensive change control will not work.

Fourth, the typical testing group does BLACK BOX testing. The test group has little or no access to the source code. Test planning and test case design strategies that depend on knowledge gained from this access are unrealistic. By the way, this isn't a bad thing. Programmers do a lot of glass box testing, and they find a lot of bugs. By analyzing the product from the customer's viewpoint, black box testers often find several problems that glass box testers would miss.

Fifth, the book accepts the fact that software companies don't fix all the bugs that they find, before releasing the product to customers. I talk about advocating for bugs, reporting bugs in ways that will increase the probability that they're fixed. And I talk about setting up good bug tracking systems so that bugs don't go unfixed by mistake or by the whim of a single programmer. Testing Computer Software was apparently the first book on software testing/quality that accepted this reality and talked to testers about how to deal with it. Since then, that acceptance--and a responsible approach to risk management for analyzing the bugs and the areas of the program that will be undertested--has been called the "Good Enough Software" movement.

Testing Computer Software has been controversial. A very senior expert in the field tried to get the original book suppressed, writing to the publisher that that it was a "puerile, non-professional approach to software development," that it glorified the work of hackers "who are responsible for the disgusting lack of software quality that pervades this industry". He said the book would sell to a "market of amateur and subprofessional programmers and 'testers' for this kind of garbage."

Over the years, and through the next edition, Testing Computer Software has become more widely accepted. My publisher recently advised me that it's now the best selling book on software testing. I think that its success has rested on the belief that it reflects, that quality improvement isn't about preaching and moralizing about quality, and it isn't about mindlessly filling out forests worth of paperwork in a vain effort to regulate creative design. Quality improvement starts with an understanding of how we do things today, and of what works well in what we do today. From there, we can improve. Without that grounding in reality, we are lost. --This text refers to an out of print or unavailable edition of this title.

Table of contents

Fundamentals

An Example Test Series
The Objectives and Limits of Testing
Test Types and their Place in the Software Development Process
Software Errors
Reporting and Analyzing Bugs

Specific Testing Skills

The Problem Tracking System
Test Case Design
Testing Printers (and other devices)
Localization Testing
Testing User Manuals
Testing Tools
Test Planning and Test Documentation

Managing Testing Projects and Groups

Tying it Together
Legal Consequences of Defective Software
Managing a Testing Group

Appendix
References
Index
About the Authors