This post aims to throw light on questions such as, "what is stress testing", "how to do stress testing", "when to do stress testing". So basically, it gives you my visualization of stress testing. Specifically, how i see it and feel of it. In Stress testing you try to break the system under test. Rather you can also call it study of Emotional quotient of your system. How well it behaves when it is subjected to stress. Definitely this means of finding innovative ideas to give the system/process a really really tough time. This can be achieved by expanding its resources or by taking resources away from it. Often in the latter case (but sometimes in both) it is called negative testing. The main purpose behind this testing is to make sure that the system fails and recovers gracefully. So, the keyword here is RECOVERABILTY.
It is to be understood that the performance testing demands a very controlled environment. Since you are going to set baselines and compare against old baselines, you have to be very cautious about the environment, architecture, OS etc. But Stress testing enjoys the beauty of randomness. You may very well doubt on what beauty I am talking about. But seriously, only a Good tester can tell you how much he enjoys in inducing chaos, catches and unpredictability. I felt this pride and fulfillment many times in Adobe and Symantec.
It should be instilled deep that stress testing is not breaking the system purely for the pleasure of breaking it. Rather it allows observation of how the system reacts to failure. It tests recoverability, error messages, breach of security due to unpredictable behavior, hangs etc. For example the simplest you can do is to parallelly run processes that consume resources. This may be CPU, memory, disk, network, database consuming processes. Think about tools and utilities available. For instance read my post related to 'ulimit' tool on linux. [PS: There are thousands of tools. i am just giving an example, because that is also very helpful. And yes, for me you clicking on Advertisements is very helpful.] If no tool is working out you can think of writing you own test application. Simplest is an application with overriden new operator. Look my post on that. Find from references below. Think about doubling the baseline number for concurrent users. What happens if different protocols used to access same system at same time. What happens if the Files being processed are of very large size, or of very small size.
Remember my pictorial representation above. Stress testing means 'giving your application really really tough time. Think about innovative ways to test the Emotional quotient of your system.'
But do remember to come to this test only after your functional and integration testing has been done. If you are only responsible for some module, then you may stress test your module/API/DLL/lib/so (Hmm..anything) once your Acceptance and Unit Testing are finished. Obviously in Agile environment, due to time crunch, Functional testing and stress test can be started in parallel on different systems. Although the former is more important, but unfortunately time is the only thing you go out of often. So stress can be triggered on some machine. And you can analyze results of functional test. And if FT is pass, you can also analyze stress test result. For Antivirus products, you can have a stress file set that contains Zip compression bombs, large files, files having viruses. For Desktop publishing applications, your test set files may contain large application files, fuzzed files, or files not saved properly, or any set of files that are known to give tough times to your system.
References On my Blog:
Ulimit as a tool on Linux for Stress Test
Different types of Software Testing
Overriding new and delete operator for stress testing
Learning with learners
W3LC brings you Discussions For IT People. Stuffed with personal touch & humor by Anwar Jamal Faiz.
A great place to read and participate in IT discussions. With over 1.5 million hits from bright technical minds across the globe, W3LC urges to read, enjoy, and comment. Learning, ofcourse, would just be a side-effect. Also visit MeOnShow. And, Technology Job Puzzles.
As single word for this post-- Wow!!!!!!!!
ReplyDeletegood kt on the topic.
ReplyDeleteI like the way of explaining things. I would ask my team to go through the article once.
ReplyDeleteThanks to the anonymous reviewer.!!
ReplyDeleteYour review would be helpful on few others articles too.
Take care :)
Thanks Xuan !!
ReplyDeleteYour review would be helpful on few others articles too.
Take care :)
Thanks Rajiv Tomar !!
ReplyDeleteYour review would be helpful on few others articles too.
Take care :)
Additionally, it would be very helpful as a case study if you can post here or on my email id: toughjamy@yahoo.com telling me what reaction your team gave after reading the series of posts on stress testing. Thanks.