Parallel Testing : POC experience
Hello Readers,
Today we are going to discuss a topic that topic is Parallel Testing. In the last month, a new project has come in our company, after the project was developed to some extent, the client of this project mentioned to our manager a requirement regarding automation of manual tests to save our costings for future and therefore I made a POC on Parallel Testing with Selenium TestNG and shared its demo. I thought I should share that experience with you so I am trying to share it through this blog.
Parallel Testing
Test parallelization is the practice of executing multiple software tests simultaneously. It’s the opposite of serial testing, which requires one test to finish before starting another.
Parallel Testing is a process to leverage automation testing capabilities by allowing the execution of the same tests simultaneously in multiple environments, real device combinations, and browser configurations. The overarching goal of parallel testing is to reduce time and resource constraints. Unlike distributed testing, where the different test components running in a distributed manner interact with each other, the parallel test approach is free from any such interaction.
When to use Parallel Testing
Parallel Testing is best suited for scenarios where there is a large suite of test cases that need to be executed regularly or when there are strict deadlines that require faster test feedback. Some specific situations where Parallel Testing can be highly beneficial.
Regression Testing
When new code changes are made, you can perform Parallel Testing with the existing regression test suite in parallel, ensuring that no functionality is broken by the new code.
When conducting API regression testing, parallel execution of test cases against different API endpoints or versions can accelerate the process of verifying backward compatibility and identifying potential issues with API changes.
Browser Testing and Cross-Platform Testing
By executing automated test cases simultaneously on each platform, engineers can quickly identify any platform-specific bugs or inconsistencies.
Load and Performance Testing
When conducting load and performance testing, running tests in parallel can simulate a larger user base and generate more accurate results.
Mobile App Testing
With the vast array of mobile devices available, Parallel Testing can be employed to run test cases simultaneously on multiple devices, improving test coverage and minimizing the testing cycle.
Parallel Testing with Selenium TestNG
TestNG is a Java-based testing framework that helps to organize tests in a structured manner and enhances the maintainability and readability of test scripts. The large feature set of TestNG allows for the easy configuration of parallel tests. The auto-defined XML file in the TestNG framework allows testers to set the parallel attributes to methods/tests/classes by leveraging the allowance for multithreading in Java. This file can help determine the number of threads desired for parallel test execution.
import org.testng.annotations.AfterClass;
import org.testng.annotations.Test;
import org.testng.annotations.Test;
import java.util.concurrent.TimeUnit;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.edge.EdgeDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.ie.InternetExplorerDriver;
import org.testng.annotations.Test;
public class ParallelTestWithMultiThread {
WebDriver driver;
@Test()
public void testOnChromeWithBrowserStackUrl()
{
System.setProperty("webdriver.chrome.driver", ".\\Driver\\chromedriver.exe");
driver=new ChromeDriver();
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
driver.get("https://www.browserstack.com/");
driver.manage().window().maximize();
System.out.println("this is the test related to chrome browserstack homepage"+ " " +Thread.currentThread().getId());
}
@Test()
public void testOnChromeWithBrowserStackSignUp()
{
System.setProperty("webdriver.gecko.driver", ".\\Driver\\geckodriver.exe");
driver=new FirefoxDriver();
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
driver.get("https://www.browserstack.com/users/sign_up");
driver.manage().window().maximize();
driver.findElement(By.id("user_full_name")).sendKeys("<name>");
driver.findElement(By.id("user_email_login")).sendKeys("<login email id>");
driver.findElement(By.id("user_password")).sendKeys("<password>");
System.out.println("this is the test related to chrome browserstack login"+ " " +Thread.currentThread().getId());
}
@AfterClass
public void close()
{
driver.quit();
}
}The TestNG XML file below helps to set up the parallel attribute as well as the thread count for the task.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite" parallel="methods" thread-count="2">
<test name="Test">
<classes>
<class name="ParallelTestWithMultiThread"/>
</classes>
</test> <!-- Test -->
</suite> <!-- Suite -->With the above code you can run multiple tests at once and check the functionality of your module. There are many advantages of parallel testing that we need to consider.
Advantages of Parallel Testing
- Reduced Testing Time: By running multiple tests simultaneously, Parallel Testing significantly reduces the time required to complete the entire test suite.
- Faster Feedback Loop: Early detection of defects allows developers to address issues promptly, leading to faster bug fixes and a shorter development cycle.
- Increased Test Coverage: With the ability to test on multiple devices, browsers, or platforms concurrently, Parallel Testing ensures broader test coverage, identifying potential issues across various environments.
- Improved Resource Utilization: Efficiently using available resources and parallelizing test execution can optimize hardware usage and infrastructure costs.
- Scalability: Parallel Testing is highly scalable, enabling teams to execute tests across multiple machines or in cloud-based environments.
Challanges of Parallel Testing
- Inter-dependent tests: If you have a test that requires another test to be complete before the test can run, you won’t be able to run the tests in parallel. This limitation applies most often in situations where you don’t want to start one test if a previous test results in a failed state, in which case something is wrong and you want to avoid wasting time and resources running additional tests until you fix the issue that triggered the failure in the one test.
- Data dependencies: If multiple tests require access to the same data, such as an entry in a database, running the tests in parallel can be difficult or impossible because each test will be trying to fetch or alter the same data at the same time. Parallel tests should not have data dependencies.
- Infrastructure limitations: If your infrastructure is capable of supporting only a limited number of tests at one time, you’ll have to limit test parallelization. Attempting to run more simultaneous tests than your infrastructure can handle can result in tests that fail — probably inconsistently — due to a lack of available resources.
In my experience Parallel Testing will helpful when we have to create register multiple users on the platform or we have to make a group chat. It is basic level stress testing to verify our product is not failing due to parallel registration or group chat.
Parallel testing or Parallelism or multi-threading is the approach where the software or a program can execute several parts or components of other programs simultaneously. And, one of the benefits of the TestNG framework is that it allows testers to perform parallel testing. To run parallel tests or classes or methods in Selenium with the TestNG XML file, it is required to use the “parallel” attribute for the test suite in the testing.xml file.
For example parallel = “methods” for executing the methods that are annotated with @Test annotation in different threads parallel= “Tests” for running all test cases or methods in the same tag parallel= “classes” for running all the test cases in the same class or the same thread parallel= “instances” for running all the test cases in the same instance and the same thread.
Happy Testing,
