An interesting read from the book "Code Complete" by Steve McConnell suggests following:
1. Industry Average: "about 15 - 50 errors per 1000 lines of delivered
code."
This is usually representative of code that has some level of structured programming behind it, but probably includes a mix of coding techniques.
2. Microsoft Applications: "about 10 - 20 defects per 1000 lines of code during in-house testing, and 0.5 defect per KLOC (KLOC refers to as 1000 lines of code) in released product (Moore 1992)."
This is attributed to a combination of code-reading techniques and independent testing.
3. Harlan Mills 'cleanroom development' technique: It has been suggested to achieve rates as low as 3 defects per 1000 lines of code during in-house testing and 0.1 defect per 1000 lines of code in released product (Cobb and Mills 1990).
What is Bug/ Defect Density?
It sometimes becomes important to calculate Defect Density for an Application or a system. Defect Density is the compactness of defects in the application.
Applications are divided into functional areas or more technically KLOC (Thousand Lines of code). Thus, the average number of defects in a section or per KLOC of a software application is bug density.
How Is Bug Density Calculated?
Step #1: Collect the total no. of defects (for a release/build/cycle).
Step #2: Calculate the average no. of defects/Functional area or KLOC
Defect Density Industry Standard?
Well, this varies for every industry, application and every team. Manufacturing would have a specific threshold and it would be completely different for IT.
If it is high it shows poor quality. But it is, in turn, the seriousness of the individual defects that decide if the product is fit for use or not. Sometimes, many defects are just false indicators or are very low in severity and priority.
By the way, who would not like zero defect density. So, even though there is no specific standard, the lower this value, the better.
My experience:
Generally, a High severity (Sev-1) bug stops all use or significant use of the program. A severity 2 bug stops some users but not all. A severity 3 bug is inconvenient, has a workaround, or is an annoyance.
The most professional development organisation I ever worked in, which created software products for worldwide use, and had an enviable reputation for excellence, worked to an expectation of one severity 1 bug per 1000 lines of shipped code.
In my stints with working in Tata, Adobe Systems, Symantec and Expedia, I felt that bugs are unavoidable. They creep in - sometimes because of programmar's logical error and sometimes vulnerabilities in software/algorithm you are using. Also, sometimes you get bugs because of some novel use of hacking techniques.
In general, I observed during all these 12 years of working that there are following number of bugs, if you want to have strict answer on the basis of lines of code:
A few thoroughly tested application like a space-shuttle software - have achieved a level of 0 defects in 500,000 lines of code using a system of format development methods, peer reviews, and statistical testing. NASA, for instance, was able to achieve zero defects for the Space Shuttle Software, but at a cost of thousands of dollars per line of code. If people will die because there are bugs in the software then that kind of cost makes sense. Most projects simply cannot afford the same level of testing as NASA.
1. Industry Average: "about 15 - 50 errors per 1000 lines of delivered
code."
This is usually representative of code that has some level of structured programming behind it, but probably includes a mix of coding techniques.
2. Microsoft Applications: "about 10 - 20 defects per 1000 lines of code during in-house testing, and 0.5 defect per KLOC (KLOC refers to as 1000 lines of code) in released product (Moore 1992)."
This is attributed to a combination of code-reading techniques and independent testing.
3. Harlan Mills 'cleanroom development' technique: It has been suggested to achieve rates as low as 3 defects per 1000 lines of code during in-house testing and 0.1 defect per 1000 lines of code in released product (Cobb and Mills 1990).
What is Bug/ Defect Density?
It sometimes becomes important to calculate Defect Density for an Application or a system. Defect Density is the compactness of defects in the application.
Applications are divided into functional areas or more technically KLOC (Thousand Lines of code). Thus, the average number of defects in a section or per KLOC of a software application is bug density.
How Is Bug Density Calculated?
Step #1: Collect the total no. of defects (for a release/build/cycle).
Step #2: Calculate the average no. of defects/Functional area or KLOC
Defect Density Industry Standard?
Well, this varies for every industry, application and every team. Manufacturing would have a specific threshold and it would be completely different for IT.
If it is high it shows poor quality. But it is, in turn, the seriousness of the individual defects that decide if the product is fit for use or not. Sometimes, many defects are just false indicators or are very low in severity and priority.
By the way, who would not like zero defect density. So, even though there is no specific standard, the lower this value, the better.
My experience:
Generally, a High severity (Sev-1) bug stops all use or significant use of the program. A severity 2 bug stops some users but not all. A severity 3 bug is inconvenient, has a workaround, or is an annoyance.
The most professional development organisation I ever worked in, which created software products for worldwide use, and had an enviable reputation for excellence, worked to an expectation of one severity 1 bug per 1000 lines of shipped code.
In my stints with working in Tata, Adobe Systems, Symantec and Expedia, I felt that bugs are unavoidable. They creep in - sometimes because of programmar's logical error and sometimes vulnerabilities in software/algorithm you are using. Also, sometimes you get bugs because of some novel use of hacking techniques.
In general, I observed during all these 12 years of working that there are following number of bugs, if you want to have strict answer on the basis of lines of code:
- 1 Sev-1 bug per Kloc
- 3 Sev-2 bug per Kloc
- 9 Sev-3 bug per Kloc
- 4 Sev-4 bug per Kloc
A few thoroughly tested application like a space-shuttle software - have achieved a level of 0 defects in 500,000 lines of code using a system of format development methods, peer reviews, and statistical testing. NASA, for instance, was able to achieve zero defects for the Space Shuttle Software, but at a cost of thousands of dollars per line of code. If people will die because there are bugs in the software then that kind of cost makes sense. Most projects simply cannot afford the same level of testing as NASA.
