Showing posts with label quality. Show all posts
Showing posts with label quality. Show all posts
Wednesday, April 1, 2020
Friday, July 5, 2019
Number of Bugs per line of code
An interesting read from the book "Code Complete" by Steve McConnell suggests following:
1. Industry Average: "about 15 - 50 errors per 1000 lines of delivered
code."
This is usually representative of code that has some level of structured programming behind it, but probably includes a mix of coding techniques.
2. Microsoft Applications: "about 10 - 20 defects per 1000 lines of code during in-house testing, and 0.5 defect per KLOC (KLOC refers to as 1000 lines of code) in released product (Moore 1992)."
This is attributed to a combination of code-reading techniques and independent testing.
3. Harlan Mills 'cleanroom development' technique: It has been suggested to achieve rates as low as 3 defects per 1000 lines of code during in-house testing and 0.1 defect per 1000 lines of code in released product (Cobb and Mills 1990).
What is Bug/ Defect Density?
It sometimes becomes important to calculate Defect Density for an Application or a system. Defect Density is the compactness of defects in the application.
Applications are divided into functional areas or more technically KLOC (Thousand Lines of code). Thus, the average number of defects in a section or per KLOC of a software application is bug density.
How Is Bug Density Calculated?
Step #1: Collect the total no. of defects (for a release/build/cycle).
Step #2: Calculate the average no. of defects/Functional area or KLOC
Defect Density Industry Standard?
Well, this varies for every industry, application and every team. Manufacturing would have a specific threshold and it would be completely different for IT.
If it is high it shows poor quality. But it is, in turn, the seriousness of the individual defects that decide if the product is fit for use or not. Sometimes, many defects are just false indicators or are very low in severity and priority.
By the way, who would not like zero defect density. So, even though there is no specific standard, the lower this value, the better.
My experience:
Generally, a High severity (Sev-1) bug stops all use or significant use of the program. A severity 2 bug stops some users but not all. A severity 3 bug is inconvenient, has a workaround, or is an annoyance.
The most professional development organisation I ever worked in, which created software products for worldwide use, and had an enviable reputation for excellence, worked to an expectation of one severity 1 bug per 1000 lines of shipped code.
In my stints with working in Tata, Adobe Systems, Symantec and Expedia, I felt that bugs are unavoidable. They creep in - sometimes because of programmar's logical error and sometimes vulnerabilities in software/algorithm you are using. Also, sometimes you get bugs because of some novel use of hacking techniques.
In general, I observed during all these 12 years of working that there are following number of bugs, if you want to have strict answer on the basis of lines of code:
A few thoroughly tested application like a space-shuttle software - have achieved a level of 0 defects in 500,000 lines of code using a system of format development methods, peer reviews, and statistical testing. NASA, for instance, was able to achieve zero defects for the Space Shuttle Software, but at a cost of thousands of dollars per line of code. If people will die because there are bugs in the software then that kind of cost makes sense. Most projects simply cannot afford the same level of testing as NASA.
1. Industry Average: "about 15 - 50 errors per 1000 lines of delivered
code."
This is usually representative of code that has some level of structured programming behind it, but probably includes a mix of coding techniques.
2. Microsoft Applications: "about 10 - 20 defects per 1000 lines of code during in-house testing, and 0.5 defect per KLOC (KLOC refers to as 1000 lines of code) in released product (Moore 1992)."
This is attributed to a combination of code-reading techniques and independent testing.
3. Harlan Mills 'cleanroom development' technique: It has been suggested to achieve rates as low as 3 defects per 1000 lines of code during in-house testing and 0.1 defect per 1000 lines of code in released product (Cobb and Mills 1990).
What is Bug/ Defect Density?
It sometimes becomes important to calculate Defect Density for an Application or a system. Defect Density is the compactness of defects in the application.
Applications are divided into functional areas or more technically KLOC (Thousand Lines of code). Thus, the average number of defects in a section or per KLOC of a software application is bug density.
How Is Bug Density Calculated?
Step #1: Collect the total no. of defects (for a release/build/cycle).
Step #2: Calculate the average no. of defects/Functional area or KLOC
Defect Density Industry Standard?
Well, this varies for every industry, application and every team. Manufacturing would have a specific threshold and it would be completely different for IT.
If it is high it shows poor quality. But it is, in turn, the seriousness of the individual defects that decide if the product is fit for use or not. Sometimes, many defects are just false indicators or are very low in severity and priority.
By the way, who would not like zero defect density. So, even though there is no specific standard, the lower this value, the better.
My experience:
Generally, a High severity (Sev-1) bug stops all use or significant use of the program. A severity 2 bug stops some users but not all. A severity 3 bug is inconvenient, has a workaround, or is an annoyance.
The most professional development organisation I ever worked in, which created software products for worldwide use, and had an enviable reputation for excellence, worked to an expectation of one severity 1 bug per 1000 lines of shipped code.
In my stints with working in Tata, Adobe Systems, Symantec and Expedia, I felt that bugs are unavoidable. They creep in - sometimes because of programmar's logical error and sometimes vulnerabilities in software/algorithm you are using. Also, sometimes you get bugs because of some novel use of hacking techniques.
In general, I observed during all these 12 years of working that there are following number of bugs, if you want to have strict answer on the basis of lines of code:
- 1 Sev-1 bug per Kloc
- 3 Sev-2 bug per Kloc
- 9 Sev-3 bug per Kloc
- 4 Sev-4 bug per Kloc
A few thoroughly tested application like a space-shuttle software - have achieved a level of 0 defects in 500,000 lines of code using a system of format development methods, peer reviews, and statistical testing. NASA, for instance, was able to achieve zero defects for the Space Shuttle Software, but at a cost of thousands of dollars per line of code. If people will die because there are bugs in the software then that kind of cost makes sense. Most projects simply cannot afford the same level of testing as NASA.
Monday, October 16, 2017
Security is our duty and we shall deliver it! - A White Paper For Software Security Organizations
Recently, I wrote a White paper. It is titled as - 'Security is our duty and we shall deliver it!'
This paper could be best described in following words-
Quality Management, Information Security, Threat Hunting and Mitigation Plans for a Software Company or a Technology Start-up engaged in building, deploying or consulting in Software and Internet Applications.
The chief sections of the document would be:
You can read and download the article from:
https://www.slideshare.net/toughjamy/security-is-our-duty-and-we-shall-deliver-it-white-paper
Read on LinkedIn:
https://www.linkedin.com/pulse/security-our-duty-we-shall-deliver-mohd-anwar-jamal-faiz/
Following blog posts are must read for any Software Quality and Security Professional or an organization working in this field:
Cheers my readers.
You are my reason to be motivated.
- M. Anwar Jamal Faiz
This paper could be best described in following words-
Quality Management, Information Security, Threat Hunting and Mitigation Plans for a Software Company or a Technology Start-up engaged in building, deploying or consulting in Software and Internet Applications.
The chief sections of the document would be:
- Introduction to Enterprise Risk & Cyber Security
- The technologies we employ in
- Types of Software testing
- Some examples of Cyber Security Firms and what they do
- How we achieve a secure product
- InfoSec and Managed Security Service Provider
- Training and development
- Safeguarding against Phishing and Multi-Factor Authentication
- Threat hunting, mitigation and Vulnerability Management
- The denouement
You can read and download the article from:
https://www.slideshare.net/toughjamy/security-is-our-duty-and-we-shall-deliver-it-white-paper
Read on LinkedIn:
https://www.linkedin.com/pulse/security-our-duty-we-shall-deliver-mohd-anwar-jamal-faiz/
Following blog posts are must read for any Software Quality and Security Professional or an organization working in this field:
http://www.w3lc.com/2010/05/veracode-as-new-whitebox-testing-tool.html
http://www.w3lc.com/2012/02/analysis-of-valgrind-still-reachable.html
http://www.w3lc.com/2011/07/stress-testing-what-how-when.html
http://www.w3lc.com/2011/02/types-of-software-testing.html
http://www.w3lc.com/2010/10/dos-and-ddos-clarification-on-hacking.html
http://www.w3lc.com/2010/05/baseline-and-traceability-matrix.html
Cheers my readers.
You are my reason to be motivated.
- M. Anwar Jamal Faiz
Subscribe to:
Posts (Atom)