Report on Software Testing Tools TestNG

Contents

TOC o "1-3" h z u Synopsis2Background3Design4Analysis6Overview of Tools6Input Output8First Test Case8Output for first test case11Second Test Case Input13Output for second test case16Third test case input18Output for third test case20Fourth Test Case23Output for fourth test case26Fifth Test Case28Output for fifth test case31

6 Conclusion 33

7 Reference 34

Synopsis

TestNG is an open source automation testing in which NG stands for "Next Generation". TestNG is inspired from two most popular tools which are JUnit and NUnit.

Using TestNG tester can generate a proper report, and tester can easily come to know how many test cases are passed, failed and skipped.The tester can execute failed test case separately. For example, the tester has 5 test cases,one method is written for each test case (Assume that the program is written using the main method without using TestNG). When tester run this program first, three methods are executed successfully, and the fourth method is failed.

Then correct the errors present in the fourth method, now tester want to run only fourth method because first three methods are anyway executed successfully. This is not possible without using TestNG. To be exact,the TestNG provides an option, i.e., testing-failed.xml file in test-output folder. If the tester wants to run only failed test cases he only needs to run this XML file. It will execute only failed test cases.

TestNG tools provides better function than JUnit as there is no need to recompile any java class in order to to run a different set of test or suites, because java code can be separate from the way their tests is run.

Besides that, TestNG has the functionality to allow the user to generate the report in three formats which is HTML, XML and by combining ANT and TestNG, a primitive TestNG reports also could be generated. As this tool supports the new feature of testing frame- work, it enables the developer to execute the test case based on group. Due to the combination of two frameworks the design of this tools can also be called hybrid architecture there are several terminology in TestNG design which are suite, test, TestNG, test method, implementation steps, @ BeforeMethod contains prerequisites, @Test contains a business logic, and @AfterMethod contains saving.

An analytic discussions had been made to compare the strength and lim- itation between four software testing tools which are TestNG, selenium, Ishikawa tools and Quicktest tools and we also compare the feature for those four tools which covered about the test development tool, application under test, scripting language, programming skills, learning curves and ease of installation and use. Lastly, by using the TestNG tools, we had run-test five different codes to get the output which are the test cases for the codes and are presented in this report.

Background

TestNG is a new generation testing framework developed due to stagnation of JUnit at version 3.8.x TestNG is an open source automation testing in which NG stands for "Next Generation". TestNG is inspired from two most popular tools which are JUnit and NUnit. There are three major advantages of TestNG over JUnit:

Annotations in TestNG are easier to understand.

Test cases can be grouped easier in TestNG than JUnit.

Parallel testing is possible using TestNG but not in JUnit.

Generally, TestNG is a framework that is designed to overcome the limitations of JUnit framework. It allows tester to clearly separate Java code from the way tests is run. Any of the classes need not to be recompiled if the tester decides to run a different set of tests or suites. Most of the automation users are using this framework because of its advantages.By using this framework, its allow us to generate reports in both HTML and XML formats. Combining ANT and TestNG, we can generate primitive TestNG reports as well. TestNG are supported in new features of testing framework such as support for different annotations which are very helpful to give guideline to test cases execution.

To conclude, TestNG eliminates most of the limitations of the older frame- works and developer can enhance to write more powerful and flexible tests with the help of easy annotations, grouping, sequencing and para-metering.

Design

Framework of TestNG

This framework is inspired by JUnit and NUnit. It has many additional new func- tions such as explanations and can allow script writers to run their code in large thread pools. It can also be called hybrid architecture due to the combination of two frameworks based on Driver Data and framework, In addition, the implementation is operated by its internal suite file consisting of TestNG class.

Explanation of some terminology can be given as:

1199241195369

Figure 1: TestNG Framework as Diagram

Suite: Suite in TestNG is maintained and represented by XML file, Also set as

?suite?

Test:This may contain one or more classes to be implemented and defined as ?test? in one suite

TestNG class:Mandatory contains at least one of anotations TestNG and contains the method to be implemented, it is defined as ?class?

Test Method: @Test marks a method that has a real implementation code

Implementation steps:

1. Firstly @BeforeMethod is implemented, hereinafter all test methods will begin to be implemented and eventually @aftermethod will be implemented.

@ BeforeMethod contains prerequisites: like the opening browser, navigation to the desired URL and other navigation steps,

@Test contains a business logic or actual test case or arguably because it contains transaction steps

@AfterMethod contains saving or recording of revenue data and closing the browser.

Analysis

4.1 Overview of Tools

Selenium is probably the most popular automation framework that consists of many tools and plugins for web application testing. Selenium is known for its strong ability to support web application performance tests. Selenium is a popular choice in open source testing automation space, partly due to the large and active user development and community.

QuickTest Professional (QTP), probably the most popular commercial tool for test- ing automation works. UFT offers a comprehensive set of features which may include the most functional automated testing requirements on desktop, mobile, and Web platforms.

The Fishbone Diagram is designed by Ishikawa's Japanese quality control expert. The fishbone diagram is aimed at detecting the cause of the main problem through group thinking by asking why each potential cause occurs. Fishbone diagram tradi- tionally works from right to left with "bone" for every branch of the middle bone. The complexity of the fish bones lies in the smaller branchbones that support every potential cause and answer "why".

TestNG is a test framework that is struck by NUnit amd JUnit but presents sev- eral new functions that make it easier and powerful to use. It is an open-source automated test framework where NG means Next Generation. TestNG is far more influential than JUnit but is inspired by JUnit. It was developed to be better than JUnit, especially during an integrated class test.

Figure 2: Table Comparison Strength and Limitations

Input Output

First Test Case

Test executed alphabetically

We can use multiple @Test annotations in a single TestNG file. By default, methods annotated by @Test are executed alphabetically although the methods c-test, a-test, and b-test are not arranged alphabetically in the code.

Figure 3: Coding(1) for First Test

Figure 4: Coding(2) for First Test

Output for first test case

The output for the first case is as followed.

1292313145993

Figure 5: Output(1) for first test case

Figure 6: Output emailed report for TestABC

Second Test Case Input

Test executed based on the ascending value of priority

If methods used are to be executed in a different order, use the parameter "priority". Parameters are keywords that modify the annotation's function.

·

Parameters require you to assign a value to them. You do.this by placing a "=" next to them, and then followed by the value.

·

Parameters are enclosed in a pair of parentheses which are placed right after the annotation like the code snippet shown below.

Figure 7: Coding(1) for Second Test

Figure 8: Coding(2) for Second Test

Output for second test case

The output for the second case is as followed.

1080046146016

Figure 9: Output(1) for second test case

Figure 10: Output(2) for second test case

5.3Third test case input

TestNG code without Priority

·

The first method (openBrowser) states to initialize Firefox browser.

·

The second method (launchGoogle) states that launch Google.com is in the ini- tialized browser.

·

The third method (peformSeachAndClick1stLink)states that perform a search in the search box (with xpath (".//*[@title='Search']") with a search term as Facebook and;

·

The fourth and last method (FaceBookPageTitleVerification) states that click on search icon of Google and verify that browser title has been changed to Facebook - Google Search.

Figure 11: Coding(1) for Third Test

Figure 12: Coding(2) for Third Test

5.3.1Output for third test case

The output for the third case is as followed.

Figure 13: Output(1) for third test case

Figure 14: Output(2) for third test case

5.4Fourth Test Case

Methods with Same Priority

There may be a chance that methods may contain same priority. In those cases, TestNG considers the alphabetical order of the method names whose priority is same.

Figure 15: Coding(1) for Fourth Test

Figure 16: Coding(2) for Fourth Test

5.4.1Output for fourth test case

The output for the fourth case is as followed.

1079995145972

Figure 17: Output(1) for fourth test case

Figure 18: Output(2) for fourth test case

5.5Fifth Test Case

Dependent Test Cases in TestNG dependsOnMethods dependsOnGroups

For testing a user journey, you may need to use dependsOnMethods and depend- sOnGroups annotations of TestNG:

loginWebsite depends on openWebsite method

goToBasket depends on loginWebsite

logOutFromWebsite depends on goToBasket

10799951799974

29

Figure 20: Coding(2) for Fifth Test

5.5.1Output for fifth test case

The output for the fifth case is as followed.

1079995146005

Figure 21: Output(1) for fifth test case

Figure 22: Output(2) for fifth test case

Conclusion

Applications which are efficient, rich in quality, and free from bugs are en- sured by TestNG, which saves a lot of labor, time, and cost of the project. With all these promising growth numbers and trends, learning mobile app development and testing will be worth it and will have huge demand.

In conclusion, TestNG is an automation testing tool with a framework that is capable of making Selenium test easier to understand and can generate report that is easy to understand. There are a few main advantages of TestNG over JUnit. The annotations are easier, the test cases can be grouped more easily and TestNG also allows tester to create parallel tests. The console window in Eclipse generates a text-based result while the TestNG window is more useful because it gives graphical output of the test result plus other meaningful details such as run-time of each method and the chronological order by which methods are executed. TestNG is capable of generating HTML-based reports. Annotations can use parameters just like Java methods.

Reference

TestNG Tutorial: Annotations, Framework, Examples in Selenium. (n.d.). Retrieved from

Beust, C. (Dec 19, 2017). Next Generation Java Testing (TestNg and Ad- vanced Concept)

Ural, H., Yang, B.: A test sequence selection method for protocol testing. IEEE Trans. Commun. 39(4), 514-523 (1991)

Ruiz, A., Price, Y.W.: GUI Testing Made Easy. In: Proceedings of the Testing: Academic Industrial Conference Practice and Research Techniques, pp. 99-

103. IEEE Computer Society, Washington (2008)

Artzi S., Dolby J., Jensen S.H., M?ller A., Tip F. A Framework for Automated Testing of JavaScript Web Applications. In: Proceedings of the 33rd Interna- tional Conference on Software Engineering (ICSE '11), pp. 571-580. ACM, New York (2011)

Chang, T.H., Yeh, T., Miller, R.C.: GUI Testing Using Computer Vision. In: Conference on Human Factors in Computing Systems, pp. 1535-1544 (2010)

Cunha M., Paiva A., Ferreira H., Abreu R. PETTool: A Pattern-Based GUI Testing Tool. In: International Conference on Software Technology and Engi- neering, pp. 202-206 (2010)

TestNG Tutorial: Annotations, Framework, Examples in Selenium. Guru99.com Retrieved from: on Dec 26,2018