Sample POC Document for Automation Testing

In this article, we provide a detailed step-by-step guide on how to Implement Proof of Concept POC Document for Automation Testing.

Every organization has different testing processes and procedures. Manual Testing is important and irreplaceable. However, automation is now picking up speed.

Introducing automated testing to an organization is a challenge and the following points will determine if it is required at all.

Implement Proof of Concept POC Document for Automation Testing

#1. Duration of the project: Short-term or long-term – long-term projects are good candidates for automation

#2. How much regression testing is done in each testing cycle? – Projects that have repetitive and lengthy regression tests as automation reduces the overall testing time and ensures complete coverage.

#3. Stability of the application: Applications that are not susceptible to frequent changes should be considered for automation. The product which is not stable, where GUI/Functionality keeps changing, elements or its XPath on page keeps changing should not be automated until it is stable.

#4. Is the project data secure and does its testing require some complicated procedures? – In this case, it is best to go for manual testing.

#5. Does the organization have a budget for automation? – Automation will add to the additional expenditure for the organization like automation tool cost, resource cost, time required for framework development and writing/maintaining automation test scripts.

With automation, missing tests or taking test results for granted will never happen. This ensures 100% coverage of the given module each and every time the same is tested. Automation will also help perform the same test multiple times on multiple browsers and platforms.

The following figures will help us understand the process of automation testing

From a technical testing point of view, the QA team needs to understand the following aspects about their automation tool:

1. Platform and OS testing matrix
2. Data-driven capabilities
3. Reporting capabilities and report portability
4. Easy debugging and logging
5. Version control supported
6. Extensible & Customizable (able to integrate with other tools like Ant, TestNG)
7. Continuous Integration.
8. Email Notifications (Custom email message received if tests have passed successfully/ failed/or any network failure)
9. If cross-browser testing and multiple platform testing is required then the distributed testing environment is supported or not.

Select the correct automation tool

#1. The application under test is a web application or a desktop application.
#2. Choosing an open source tool vs. a paid one.
#3. Tools should fulfill the application’s testing requirements.
#4. Using the tool – the team’s expertise and comfort level in terms of using and learning the tool.
#5. Does it support reporting – If No what other options for reporting are available (open source or paid). If yes, then how good is it in terms of conveying correct data from presentations as well as content point of view?

Also read => The A to Z Guide on Selecting the Best Automation Tool

The tool evaluation includes the following:

While selecting an automation tool, it is very important to consider if it is supported on the application’s GUI implementation.

1. GUI is implemented using traditional HTML, AJAX, or other web development toolkits.
2. Does the GUI include videos, images or a lot of written content?
3. Is it interactive or only informative
4. Browsers required to be tested.

It is important to assess the tool on the above points to understand if the tool really meets the project’s testing requirements.

Developing a proof of concept on automation

Implementing an automated testing POC is a crucial and most often used method of introducing a tool to an organization. Once it is decided that automation is to be done and a tool has been chosen, it is time to create a prototype as a POC and present it to the management to showcase the real-time usage and benefits.

To do so:

1) Decide on the test cases that we will use for the POC.
2) It helps to pick the areas the clients will be most interested in.
3) Plan to show manual vs automation in a way that proves that there is no degradation in the quality by choosing automation.
4) Include a test case that fails and results in finding a defect – this helps reinforce that the tool can indeed find defects
5) Use assertions and validation points wherever necessary.
6) Show areas that clearly can and cannot be automated. Usually, the following aspects cannot be automated:

• Video steams
• Flash content (non-static content)
• Non-static images

7) Highlight if the tool satisfies the following requirements?

• Can it automate all the key features of the desired application
• Is automation possible on the same browser that is required by the project
• Will automation call for a change in application implementation? (For example, for automation, it is important that element identifiers are unique and do not change every time a page is invoked.)

POC results – they are usually one of the following:

1. Tools that meet the project requirements – Work out further details. Such as the cost of implementation – negotiate prices is necessary, finalize license fees, training & support costs, consultation, and implementation expenditures etc. In the case of open source, tools determine the maturity of the tool, learning resources available, learning curve, support available, etc. For both licensed and open source tools, maintenance costs have to be considered too. It has to be kept in mind that the benefits are substantial only over a long period of time.
2. If the tool does not meet the requirements and has limitations – the tool is no longer considered.
3. Tool partially meets the requirements – revisit and check if another satisfies the requirements better OR if automation is totally out of the picture OR if there is any other workaround with the same tool.

Once we present our proof of concept to the management and we get the go-ahead from them, the next step is implementing a pilot project using that tool.

Sample POC Document Template

There is no one perfect POC template. It generally includes:

1. Requirements for POC
2. Candidate for POC (All automation tools)
3. Project requirements
4. Pros and cons of each tool based on the project requirements
5. POC results

Download the sample POC document for automation testing:

=> Sample POC Document template 1
=> Sample POC Document template 2

Implementing a Pilot Project

We need to define our pilot project by:

• Quantifying business cases which will determine whether we should be using this tool or not.
• Define naming conventions and various guidelines for application tools.
• Benefits of a tool like financials and other things, what can be done and what cannot be done and also its possible workarounds.

Step #1. Choose test cases for the pilot

• Modules / Features important from a client perspective
• Functionality is easy to demonstrate (happy path end to end)
• Test cases that are difficult to test manually and once automated will simplify testing them
• Broken functionality to demonstrate how automation can help identify failed test cases

Step #2. Automation framework development

A test automation framework is a set of concepts, processes, procedures, practices, and environment. It is nothing but an integrated system that consists of rules to automate any given product. The system includes a set of functional libraries, APIs, test data, object repository and various other modules.  The framework and approach to scripting used for test automation has an effect on its costs.

The following scripting techniques can be used:

• Linear
• Hybrid
• Data-driven
• Keyword driven and
• Structured

Using any of the above techniques, a testing framework can be designed that will help in achieving a specific format to drive the test and simplify test execution and reporting.

Determine templates and naming conventions for objects, test cases, test suites, data repositories, etc.

Step #3. Script development and execution

Step #4. Reporting Does the tool have in-built reporting capabilities? Are the inbuilt reports capable of conveying all the required information precisely? Are we going to need another tool for reporting purposes like Crystal Reports, reportNG, etc.?

Step #5. Maintaining automation scripts

Presenting to the stakeholders

As much as the proof of concept and implementing a pilot is important, so is presenting it in the correct manner. The following points will help to present it in a positive way.

1. Begin with how much manual testing effort is put into every testing cycle, challenges faced during manual testing and how we can use automation to overcome them.
2. Explain how you selected the tool based on the proof of concept
3. Highlight the features of the automation tool and how it complements the testing requirements
4. While running through automation, explain how automation tools will not only help in faster test execution but also its capability of performing verification and bug identification.
5. Demonstrate how the report will show test case execution status
6. Highlight reporting features like colorful legends for different test case statuses, snapshots of failed test cases, and report portability
7. And finally show how much testing time will be reduced for every testing cycle.
8. Also explain how you are able to achieve the entire automation framework that you have developed and its benefits in terms of usage and maintenance.

Be prepared to answer questions related to how long it will take to automate a single simple or critical functionality. Also, if a minor change happens on the application front, how many script changes will be required and as to how much time will be taken to modify.

We hope this guide will be useful for our readers to start writing an automation testing POC document. Let us know if you have any doubts or questions. Feel free to provide your feedback. We would love to hear from you.