Automating Quality: Enhancing Software Testing Efficiency

Introduction

Nowadays software has become very important in an individual's life [1]. So, software should be safe and a reliable one.But there is a saying that 'Mistakes makes us human'. Therefore, there is a very high possibility that where humans are involved, chances of errors are very high [2]. Programming blunders can negatively influence the live activity and sometimes can even cause death [3].

So, it is necessary to treat such errors in the earlier stage itself, they get much more costlier in the further stages [4].

Let’s take an example to understand, a person named xyz did a survey [5], and was at a conclusion that around $500 billion is lost due to poor performance of softwares. National institute of Standard and Technology also gave a report that bugs and errors are costing a loss of approximately $59.6 billion to the USA economy, and also stated that 0.33% can be reduced by improving the Testing area [6] i.e the software testing phase that is done before the product to be delivered.

Software testing is defined as a process of evaluating a product with an intention of detecting any faults or errors or bugs and also to verify that the product or the software works as per demand or not [1]. Software Testing costs 50% of software development life cycle [1], [2], [7] i.e CPMCD communicating, planning, modeling, coding and testing and deployment. Aim of software testing is to find out errors but more than that by testing the software it gives the confidence to the team that software is working in a good condition and is ready for deployment [2].

There can be two ways for testing a software manual testing and automated testing [10].

Manual testing is a way in which the testers do not require any additional knowledge of testing tools but requires lots of 'experienced' people who are very good at testing and detecting bugs and errors. Now, let's consider second way of testing i.e automated testing, automated testing can be said as a programmer using a coding language and write a program to to automate a manual process. Automated testing requires the use of testing tools with some framework for executing tests. How can automated testing help over manual testing? So answer to this is automated software testing increases accuracy and also saves time and effort as well. Main important factors for successful and efficient testing are

• a.) selection of testing methodology,
• b.) selecting and using proper automation tools and framework. [12]

For software testing there are many kinds of approaches that can be used to make sure that the system behaves and works as it is expected to work.

Automated Software Testing

What is Automated Software testing? Automated software testing is a process in which we use some software to test the software that we want to be tested by comparing the actual outputs with the expected outputs [14]. So using automated tools we can automate some portions of manual testing but not all [11].

As mentioned earlier, using automated testing testers can save a large amount of time, as he can run many tests simultaneously in a very short duration of time. Automation testing not only saves time and effort but also saves money, enlarging the quality of software testing tasks [12]. As mentioned earlier automated software testing requires experienced people who are aware about software testing tools and framework so we can consider this as a drawback of automated software testing.

There are five different types of automation tools

1. Unit testing tools
2. Functional testing tools
3. Code coverage tools
4. Test Management tools
5. Performance testing tools

• Unit testing tools

Unit testing is a method of testing the most basic code of a project. And the unit testing tools are used for the process of unit testing process [12]. Widely used tools for automated tests are unit testing tools. The task of developers becomes easy i.e developers can write test cases in any language and the test cases can run or execute automatically. The unit testing tools are used to verify the working condition of an individual unit [14]. Examples of unit testing frameworks are JUnit, NUnit, jMockit, PHPUnit and many more.

• Functional testing tools

Functional testing is a method of testing to be sure that the system or software is working according to what user has asked for. Functional testing tools are used for the functional testing process. How functional testing works? The functional testing tools evaluate functions by giving them inputs and observing the outputs and comparing the observed output with the expected outputs [16]. Examples of functional testing tools are Selenium, HP QuickTest Professional, TestComplete, Ranorex,Test Studio and many more.

• Code Coverage tools

Code coverage is a method of testing which decides what part of the code is covered in automated software testing. It counts the number of lines or blocks of code using Code Coverage tools. Code coverage tools give the output as what part of the code requires more testing [18] and what part of the code is tested correctly. And Code Coverage tools always result in the output in percentage only, the % value closer to 100% says that there are very less chances of bugs and errors. Examples of Code Coverage tools are Cobertura, CodeCover, PITest, EMMA and many more [17].

• Test management tools

Test management tools is a method that is used to test the test cases, reports, results i.e the automated testing activities. Using this methodology, the work of the team becomes easy. Different test management tools provide different features of managing testing [20]. Examples of test management tools are Test Link, QA complete, Test Manager and many more.

• Performance testing tools

Execution testing instruments are utilized to help execution testing forms. Execution Testing will be trying perform to decide how the product will act as far as responsiveness and steadiness under different conditions what's more, remaining task at hand. It can likewise serve to examine, measure, approve or confirm other quality properties of programming,for example, versatility, unwavering quality, and asset use [21].

Execution testing apparatuses are utilized to assess the execution of programming or part in asset utilization, throughput and improvement reaction time with determined execution necessity. As indicated by Khan [22] Load and Stress testing are the two execution testing; [23] included Endurance and Spike testing, while [21] included: Soak, Breakpoint, Configuration, Isolation, and Internet testing among the other presentation testing types. Execution testing should be possible utilizing a wide assortment of devices. Examples of Performance testing tools are HP LoadRunner, JMeter, silk performer and many more.

Test Automation Framework

A test automation framework is a defined, extensible support structure within which the test automation suite is developed and implemented [24]. Automation Framework comprises tools and practices that are designed to help executing software testing more efficiently [25]. It includes both physical structures for test creation and implementation and logical interaction between components such as coding, standards, object repositories, process for storing test results or how to access external resources. A framework “Facilitates a standard way for adding, modifying and deleting the test scripts and functions”, and provides “scalability and reliability with less effort”[26].

Importance of Test Automation Framework

A tester can integrate scripts or record tests with an organized framework to get additional benefits. A well defined test automation framework helps in developing scripts that are easily maintainable , higher reusability of test components and obtaining high-quality test automation scripts. Using a framework for automation testing helps to increase test efficiency as well as speed, improves test accuracy, and reduces test maintenance cost as well as lowers risks. Test Automation provides support foundation to a variety of automated software test including: Unit testing, Performance Testing, Functional Testing and so on.[27]

Types of test Automation Framework

There are various test automation frameworks each having its own architecture and having their advantages and disadvantages. It’s important to select the right framework for automating software

• Linear Automation Framework

This is a first generation and one of the simplest testing frameworks that is used to test a web application’s user interface. Here, the tester records each step such as user input, or checkpoints, and then plays the script back to conduct the test in a sequential order without the requirement to create the function. It is also known as a record-and-playback framework.

• Modular Based Framework

This framework allows testing software by dividing the test into separate functions or modules, each will then be tested in isolation. For this a test script is created for each part and then it can be combined to create a larger test in a hierarchical manner. It is also known as functional decomposition framework[27]

• Library architecture Framework

This framework is an upgrade of a modular based framework. It identifies similar tasks within the software that are needed to be tested and groups them according to the functionalities instead of dividing the software under test into various independent modules that are to be tested in isolation. These functions are then stored into a library which is then called by test script whenever needed. Compared to the Modular based approach it results in high reusability as it groups functions and keeps it in a library that can be used by multiple test scripts.

• Data-Driven Framework

This is a famous practice in software testing in which test data is kept separate from script logic and data is stored in external data sources such as Excel Spreadsheet, CSV files, Text file, SQL tables. Then test scripts are connected with the external data source to read and populate the necessary data when needed. Unlike all the testing. Below are some most common test automation frameworks explained. data without modifying the test script

• Keyword-Driven Framework

This framework is an upgrade of a data driven framework, it not only separates test data and script logic but also takes this approach a step further. In this approach, keywords are also stored along with their object in an external data source, making it independent from the automation tool being used to execute these tests. These keywords can be labeled simply as “signup” or “login” or with a complex name such as “signup button” or “login button”. For this approach to work properly a shared object repository is needed to map objects to their associated actions[25].

• Hybrid Testing

Framework As the name suggests this hybrid framework is a combination of two or more frameworks set up to get the best practices from different frameworks suitable for automation needs. Frameworks are selected in such a manner that its weaknesses are avoided and the advantages of them are taken. As each software is different, the process to test the software is also different. As many teams are moving towards an agile model, setting up a flexible framework that can be used for automated testing is crucial. A hybrid framework can be easily adapted to get best test results[25].

Automation Tools

An Automation tool is a software with the help of which actual software is tested. Automation tool servers as a means of doing software testing. Creating high quality software standards requires proper use of testing techniques and tools for testing the software.

There are various testing automation tools available each has its advantages and disadvantages and has different purposes in software testing. A detailed analysis of these software should be frameworks discussed above, this framework separates test data from the test script, which allows tester to test the same function of software multiple times with different set of of most commonly used automation tools with its advantages and disadvantages.

JUnit

It is a unit testing framework for the Java programming language. It allows developers to write an oracle test case and to automatically execute test sets. JUnit specifically allows to consequently relapse test a few test sets. JUnit is a family of unit testing framework which is collectively known as xUnit that originated with SUnit[32].

Advantages

• It simplifies integration and facilitates changes.
• As developers can use it, bugs can be found and resolved earlier.
• It helps in making the coding process more agile.

Disadvantages

• It consumes developer time to write and check.
• It can only run tests on a single Java Virtual Machine and developers can’t use it where multiple JVMs are required .

Selenium

It is a framework for testing web applications that is compatible with various browsers and platforms. Testers can write test cases in various programming languages. It allows us to do tests like record and playback without the need to learn test scripting language. Selenium has become the most popular automation framework due to its many tools and various website plugins that makes it popular for web application testing.It also has a large and active users community.

Advantages

• Open Source tool.
• Large and active user community.
• Various web plugins are available.

Disadvantage

• Support may be slow from the done before selecting the automation tool which includes budget, skills required, testing requirement, application type.

This requires lots of time and effort. There is a explanation

Unified Functional Testing(UFT)

It is an automation tool for functional and regression testing. Unified Functional Testing was earlier known as QuickTest Professional. UFT is the most popular commercial tool for functional test automation[33]. UFT can be used to perform functional testing on desktop, web and mobile platforms. UFT uses Visual basic Scripting Edition to create test processes and control testing in the application.

Advantages

• For test creation and execution only basic programming skills are required.
• Dedicated user support and an established larger user community support.

Disadvantages

• UFT is not open source so we need to purchase the license of these tools.
• Only Supports VBScript
• High costs for upgrades and additional modules.

Katalon Studio

It provides a comprehensive set of features using which we can implement full automated testing for web, API and desktop and mobile applications. It is built on top of two open source frameworks Selenium and Appium. It allows teams to get started with test automation quickly and thus reduces the effort and expertise required to learn and integrate this framework[35].

Advantage

• Katalon Studio is open source.
• Integrating with frameworks and features for test case creation and execution is easy and faster.
• Doesn’t require advanced programming skills.

Disadvantage

• Only supports Java/ Groovy.
• Community support is poor.
• Provides less features.
• For integrating it with other tools requires good programming skills with experience.

TestComplete

TestComplete is a commercial platform for desktop, mobile and web application testing. It allows testers to build a robust testing framework that uses a wide spectrum of available software testing methodologies .It supports various languages for writing test scripts.

Advantages

• Supports various programming languages.
• Just fundamental programming skills are required.

Disadvantage

• Additional cost for extra modules and upgrades.
• It is not open source hence licensing and maintenance is required.

Conclusion

Test automation has become a basic piece of a fruitful programming testing. The most recent World Quality Report 2018–2019 features that test mechanization is the greatest bottleneck to convey 'Quality at Speed,' as it spares time, decreases cost, improves proficiency, and builds exactness [37]. In this manner, compelling and effective test mechanization can't be accomplished without the correct robotization instruments and system, this report presents a point by point clarification about different test robotization instruments and structures just as gives bits of knowledge into a portion of the significant components.