INTRODUCTION

This book will provide a brief introduction and overview of software testing, discuss about entire process carried out in the IT industry, and introduce the concept of blended testing. It aims at providing a practical know-how to the reader.

Software Testing, that define Software Quality, is of major importance to all the software companies in today’s highly competitive filed. Every software application is a jig-saw puzzle of many tiny programs, assembled together. Yes, a developer can develop products and check the functionalities of the product, but he/she can never certify the quality and stability of the product; especially, after the integration of his modules with those of his co-workers.

A product can be made better (or at its best) only after it goes through a proper process of development, testing and bug fixing. Not only should it go through this process, but there should also be a good and healthy relation between the developers and the testers to attain this objective.

Software testing is both a discipline and a process. Though software testing is part of the software development process, it should not be considered part of software development. It is a separate discipline from software development. Software development is the process of coding functionality to meet defined end-user needs. Software testing is a continuous process of both validating functionality, and, even more important, attempting to break the software.

Software companies face serious challenges in testing their products, and these challenges are growing bigger as software grows more complex. All developers know the frustration of having software bugs reported by users. The first and most important thing to be done is to recognize the complex nature of testing and take it seriously. This part of software testing gives you a basic idea of software testing, why it is required, what steps to follow, and how it is different etc.

Software testing is often used in association with the terms verification and validation. Verification is the checking or testing of items, including software, for conformance and consistency with an associated specification.

Software testing is just one kind of verification which also uses techniques such as reviews, analysis, inspections, and walkthroughs. Validation is the process of checking that what has been specified is what the user actually wanted. Validation ensures if we are doing the right job and verification ensures if we are doing the job right.

Debugging is different from software testing and should not be confused. Debugging is the process of analyzing and locating bugs when software fail to behave as expected. Debugging is therefore an activity which supports testing, but cannot replace testing.

Static analysis and dynamic analysis are other activities which are often associated with software testing. Static analysis investigates the source code and dynamic analysis looks at the behavior of the software while executing.

Industry Difference

There are some significant differences between software industry and other industries. If we take pharmaceutical industry there are many chemical manufacturers; there are many different production plants that mix these chemicals and make a specific drug. These drugs need to be packaged, transported to different cities and to retail pharmacies. Assuming an incident in which some drugs caused bad side effects in a particular city, the drug company need to ensure that it is addressed within hours, submit legal documents on quality, replace drugs and battle any lawsuits!

Assuming a manufacturing company that provides ball bearing to a car company and the dimension of the bearings are smaller by 0.1mm in dimension! Same issue to replace, transport, etc. Moreover we need more raw materials to rebuild some more parts; we need more time, transportation. What are we going to do with the facility drugs or bearings? It requires more expenses!

In software, there is no transportation cost, just redo and email/ftp the new software. There is no wastage of raw material. There is no waste disposal. Any other industry, the defect or mistakes are visible very obviously, quickly, if a food item is stale, we can feel immediately, if a car driver makes mistake while driving, it will be visible clearly, but in software, we cannot fine the defects easily, once the code is compiled and built all mistakes are buried in binaries.

Most of the times one portion of the software will work; another area will fail. We cannot see through the software fully, easily. No one is directly physically hurt if a software fails. If an artificial respirator fails, due to software defect the first doubt by instinct goes towards the equipment and not to the software that controls it. So, a mistake made in software is not easily detectible. If it is not easily detectible, it gives more room to make mistakes!

Testing requires a mindset to find issues. The first and foremost quality for that is “Attention to details”. Let us have a small exercise. Just mentally list one all the parts of a two wheeler or four wheeler vehicle. Count how many parts you are able to do so. Your time starts now.

Hope you are done. Le me ask a simple question. Did you list the fuel tank cover, fuel tank cover open knob, seat cover, engine temperature indicator, battery, foot rest…. Do you agree that you missed at least 2 parts out of the above list? So many years you see and use, yet you miss. That is what to be found out in testing. What a developer misses, that should be caught by a tester.

Did you observe one more thing where you listed out the parts? First minute you would have listed fast, next minute you slow down, 3rd minute onwards you feel like not getting any more parts. Human mind needs constant push. We need to push it saying “There is some more left, unnoticed”. So did it! A test must never give up.

Cost of Poor Quality

When Companies produced defective products and the products reached customers that are where the problem starts. If customer finds out the problem, the first thing that goes off is the trust of the customer on the product and the company. Rebuilding trust is a bigger problem than fixing the problem itself.

One leading mobile phone company called 2 million phones back from users, due to a faulty battery. A leading tire company called 60000 tires from car manufacturers since they found out that the tire catches fire at 60 mph on a curvy Road! A leading toy company went bankrupt, since a customer found out that paint used on the toys had poisonous lead content. A fortune 100 company incurred a cost of 0.75 billion dollars to replace its faulty gaming console component.

So prevention is better than cure.

I never fail attitude

I wrote my maths exam in 12th grade. I thought I would get 198 out of 200. But I got 192 out of 200. Am I wrong or my teacher who corrected the paper is wrong? We inherently make mistakes that are not visible to us. If another person sees it, the mistake is evident and obvious.

The same is triggered and amplified by an ego. If someone challenges a problem in our work our ego forces us to say “I am right”. This is instinct. To overcome this is a tough task. Every programmer or a software developer feels the same way. “My code never fails” on the same line, every tester feels “Your code always fails”. Also people feel hurt when the mistakes are pointed out. Some people are afraid to point the mistakes out on others. But we can learn better and faster from our mistakes.

Basic Questions on Testing

What is Software Testing

Software development is a process in which a software product or project is developed to fulfill the needs of a customer and deliver them with specified cost and time period, Who would want to make products of no standards or stability in today’s IT world? Well, if there are any, then they are the ones who do not consider Software Testing as a must. They also do not consider the quality of the product. It is a process of evaluating a system manual or automatic means and verifies that it satisfies specified requirements or identifies differences between expected and actual results.

Why to test

The very basic question that arises in everyone’s mind about testing is Why to test? .Talking in terms of real life situations, we Check any vegetables before we buy, we Check the television before we buy and we Check the car before we buy and so on. So, we check or inspect the article/ object, before us using that. This applies to software also and software is no exception to this.

To better understand the important of testing, let us take a typical banking application, which uses software. If the application is going to wrongly post the transaction, and the balance is not shown correctly, the bank will start losing its business. In a nuclear reactor, the control mechanism of nuclear explosion is taken care by a sophisticated software; and if that does not work properly, the rector may act as a super, it has the same ability to perform 10000 wrong operations in one second, if not tested properly.

So, testing becomes absolutely essential to make sure the software works properly and does the work that it is meant to perform. A product can be made better (or at its best) only after it goes through a proper process of development, testing and bug fixing. Not only should it go through this process, but there should also be a good and healthy relation between the developers and the testers to attain this objective.

What to test

Any working product which forms part of the software application has to be tested. This includes any programs and the associated data. The program may be an exe file or a servlet or an active server page etc. The data also must be tested for accuracy. Date includes the database information an any stored procedures or triggers that act on the data.

Among the programs and data, the programs need more attention and must be tested rigorously. If the programs are tested right, there is a higher probability that the data will also be correct, but not necessarily.

How often to test

Since the programs and date are tested, testing has to be performed, wherever they change. When a program (source code) is modified or newly developed, it has to be tested. The controlled conditions should include both normal and abnormal conditions. Testing should intentionally attempt to make things go wrong to determine if things happen when they shouldn’t or things don’t happen when they should. In short it can be termed as “Creatively Destroying the Product”.

Also, whenever the future and volume of data changes, the application has to be tested. As an analogy, when we buy a new television, we test it before taking it home. At the same time, in case of any repair work, we re-check or re-test the television unit, before we take it. This is applicable to software also.

Who tests

When a programmer initially codes a program, he/she himself/herself test it. This is similar to a student who writes an exam, checking the answers by self. This is to make sure that the price developed by the programmer meets the expectations of the programmer. This is first round of test, that the software piece undergoes. After the programmer makes sure that the software piece works fine, their is a separate team that ensures that the software product works as per the requirements. It is typically the testing team that does this round of testing. This way, the software is observed by different pairs of eyes. This is similar to a teacher evaluating the answer papers of the student, for correctness.

Finally the software is tested by the customer/client/end-sure. Because the software is going to be used ultimately by the client, in his/her place. This is similar to a person, buying a car, is actually doing a test drive of the car, before buying the car.

Product Company vs. Project Company

A Project company survives on the number of contacts that the company has and the number of Projects that the company gets from other forms. Whereas a Product based company’s existence depends entirely on how its product does in the market.

A Project Company will have the specifications made from the customer as to how the Application should be. Since a Project company will be doing the same kind of Project for some other Companies, they get to be better and know what the issues are and can handle them.

A Product company needs to develop it’s own specification and make sure that they are generic. Also it has to be made sure that the Application is compatible with other Applications. In a Product company, the application created will always be new in some way or the other, causing the application to be more vulnerable in terms of bugs. When upgrades are made for the different functionalities, care has to be taken that it will not cause any other module to not function. When any functionality is being implemented, have to make sure that they are introduced in such a way that it will not cause problems in the future for up gradation and for implementation of any other new functions.

Challenges in testing

To report a mistake on someone else’s work, is not that easy. People often feel hesitant to do that criticism. By not telling the mistakes, they do more damage to the work and the person who did it. It depends wholly on how you convey the issue rather than what you convey. There are people, by nature, who blame and who do not blame. When it comes to testing, we need to get rid of that subtle human mind block and call a spade a spade. Getting this mind set will be the first challenge.

The issues in software often surface as we start using the software. When a problem is identified when the customer uses, that puts more pressure on the people who tested the software and certified. This leads to project post-mortems late in the development cycle and it may even cause client dissatisfaction. So drawing the boundary of testing is another challenge.

Most of the fresh people who form the majority of the IT work force, do need action on the software. When a person codes, he/she sees the software running in front of the eyes; whereas in testing, you need to think a lot and document a lot. Many people are not used to this in their academia. This causes a discomfort in people and this takes time for people to gain the expertise of test planning.

The management teams always face time delays and pressure on testing. When sufficient time is not provided, the testing may not complete. The management will think of just completing the project schedule but may miss the project quality.

There is a great amount of difference between the service companies do testing and the product companies do testing. When people move from one type to the other, it takes time to adjust to the processes adhered by the companies. Process orientation is a human challenge every company faces.

In most of the countries, getting seasoned testers is a major challenge. Get the guy who cracks every software – very easy to say, but where are those guys? It takes a longer training period to create such a good code cracker. The reason is that the testing is more of a mind process and it involves a better understanding of the software than the creator of the software.

Tune your mind

Hope you had used the remote control to operate the television and the video player. Imagine that I sell a remote control unit to you and you want to ensure that it works perfect, before you pay me money. How will you test it?

Wait, do not start actually testing it. First write it down on a paper, a set of things that you will test on the remote control. How long do you plan to test the same? What else you need to test the remote control? Spend about 30 minutes and see how many tests you had written down.

You can observe a few things like this. First 10 minutes, you could have got a lot of ideas flowing in your mind and you would have written more tests. Then you could see some slow down in that process. During the last 5 minutes, you might have written only a few. Isn’t it?

You could see jump of thoughts when you read what you had written. Fifteenth point and sixteenth point, are they related to each other or different from each other totally? For example, 8^th point would be to test the battery life and 9^th point would be the forward option in video player panel. The tests will not be coherent when you read one after the other.

After 2 hours, you read what you had written. Do you understand what you had written? Or are you getting a doubt of why we had written this? Now, ask another person to read what you had written. Does that person understand what you had written or he/she has difficulties in understanding what you had written?

Now do this exercise, and spend 30 minutes for writing the tests for a lift (elevator), mobile phone, and microwave oven. You will see a change in you. What is that? Clarity in thought.