UAT Tips and Templates

What is UAT?

User Acceptance Test, or UAT or Acceptance Testing, all defines the single meaning.

According to The International Institute of Business Analysis – Body of Knowledge V2.0, User Acceptance Test or UAT is defined as "Test cases that users employ to judge whether the delivered system is acceptable. Each acceptance test describes a set of system inputs and expected results."

User acceptance test refers to the satisfactory test of solution by users before moving the solution to LIVE Environment. In UAT, users of the software validate the maximum possible scenarios that may come in LIVE environment, which are tested in the solution and found to be accurate.

If UAT is about testing the solution, then a question comes to mind: "What do Quality Assurance departments do at their end when they say they are testing the application?" For that I would simply say, there is a 360-degree difference in both types of testing, and the biggest difference is the goal/objective of both.

The goal of software testing is "to make sure the software meets the specification" or "to make sure that the developed software is bug free."

Whereas the goal of User Acceptance Testing is "to make sure that system completely supports the day-to-day business scenarios along with other known possible scenarios that may create a hurdle in business operations, and to make sure that software will not hurt the LIVE operation when it will be running in a LIVE environment."

UAT is considered a final stage of any software development initiative. Without successfully completing the UAT, the project cannot be considered as completed nor does any client accept.

The Myth:

While discussing the project issues with different colleagues, friends, community members and others, I have found that many Business Analysts try to implement the system directly in a LIVE environment, i.e., user starts entering LIVE entries and when all entries are entered and reports are matched, the system is considered as implemented and signed off.

Exceptional/abnormal scenarios are part of routine business, and they are also very hard to recall/identify. In the earlier-mentioned situation, when the user was focused on testing the system based on LIVE entries only, he definitely loses the focus on those abnormalities that arise usually in his business transactions. Further, there might be some special cases that were handled in some other way by the users; those cases will also be missed in testing based on LIVE data. All of these abnormalities, special cases and others issues will come someday in the LIVE environment when the user will be using the software, which will be the time the user will say, "I used to solve this case by pressing that button in legacy system" or "I did this case by doing this, this and this," and the vendor will ask for Change Request and two things will be charged (money and time), and due to time the business may suffer.

Consider the other scenario in which the user took his time with the business analyst and identified the maximum possible scenarios including normal/routine business transactions along with any abnormality or exceptional scenarios. And when the system is ready, users test all those scenarios in the system and after successful completion of testing, the system goes LIVE. This will minimize the chances of abnormality or exceptional cases in the LIVE environment.

Conducting UAT is equally important for both Vendor (Software Developer) and the Client (Software User). Thousands of reasons can be written on the importance and impact of not doing UAT; following, are some very important reasons for which UAT should be done in every project.

Reduce chances of error in LIVE Environment: Maximum possible scenarios are identified and tested before software moved to LIVE environment

Increase User Satisfaction: UAT provides full-fledge access of software to user, which gives him a lot of confidence as well as satisfaction to allow him to test the software that soon he will be using in a LIVE environment

Reduce risk of regulatory & other compliance: As in UAT, the system is tested on maximum business scenarios; the risk of regulatory and other compliances that may bring penalties in term of financial impact, opportunity loss or customer dissatisfaction can be minimized.

Reduce Time: In new/automated system, there is a chance that the system has automated some business processes along with some changes in existing processes, which might have increased some process steps found to be unnecessary or wasting time in the LIVE environment, UAT allows users to identify those unnecessary steps before going into the LIVE environment, it allows organizations to save time by reducing process steps that may take time in the LIVE environment and incur extra costs.

Business reputation: If due to software solution, organization is unable to provide the services to its customer or provide the services with delay or somehow impact customer by giving wrong figures or showing wrong transaction in customer’s account, this may blow the business reputation and definitely results in customer dissatisfaction, and with this, the company may lose a good amount of business that was successfully in hand having the legacy system in place.

Role of Business Analyst in UAT

Business Analyst as a neutral, non-technical, business side representative makes a good UAT conductor. Due to his focus of solving business problems, independent from developer and not having a technical mind, he can easily think in the shoes of the customer to identify the normal as well as complex, uncertain and abnormal scenarios along with real like data and help users in testing the same before going into the LIVE environment. And finally, the Business Analyst has a vested interest of high-quality software along with the solution of the business problem with value addition and so is motivated to perform rigorous testing of the system.

Skills Requirement of Business Analyst for UAT

As mentioned earlier, UAT is the last and final stage after which the system will go LIVE, and therefore, the crust of this activity is to make sure that maximum scenarios are tested in the system and if issues are found they are reported accordingly. Due to the criticality and importance of the UAT phase, the role of the UAT conductor requires multi-faceted skills. These qualities allow the person playing that role to perform this important activity; the business analyst must think in the shoes of the user to understand his problem. Absence of these skills may fail the overall UAT phase.

Further, following skills and competencies are required to be possessed by the Business Analyst to conduct effective/successful UAT:

People Handling: Business Analyst that holds good skills of people handling and can develop a good relationship with users in order to explain his point of view, and that skill also helps business analysts to understand the point of view of users. In UAT, users sometimes try to resist change or try to imply his point, but having a good relationship with the business analyst, the issue of ego doesn’t come between and things get concluded in a positive direction.

Domain Knowledge: As quoted in every business analysis-related article, "Domain Knowledge is mandatory for Business Analyst." Off-course[G1] , if a business analyst lacks the domain knowledge, he will not be able to conduct the successful UAT. Due to his limitation in business knowledge, he will not be able to identify the business scenarios, nor can he help the user in identification of the same, and also will not be able to question the wrong scenarios or wrong practices that the user requires to be added as scenario in the software.

Software Functional Knowledge: You must have heard a business analyst saying, "I need to talk to my technical team to get the idea how this screen will work?" Consider the confidence level of the user on the business analyst and software when a person who is facing him is telling him how to do UAT? Don’t know about his own solution.[G2]  Business analysts must understand the inside out of the whole solution; I would say, "He should be the person who has maximum knowledge of software working." With this skill set, he can conduct efficient UAT, as the issue of stuck due to software functionality.

Executor, Initiator: Business analysts should have the skills of execution; he should have the ability to drive the users according to the UAT Plan and in case of any issues related to user availability, system errors, other resource availability, any other showstoppers or issues of progress, he should escalate it to the right person immediately without wasting time. Business analysts should observe the situation and inform the relevant stakeholder in case he senses some risk or issue that is arising.

Positive Attitude: Business analysts should always maintain a positive attitude and consider the comments of users as areas of improvement and act accordingly rather than start being defensive or sometimes offensive about it. He should understand the user’s point of view and in case the user is of a different mindset, try to convince him positively with rationals[G3]  and arguments to support his opinion.

Common UAT Problems Faced by Business Analysts in UAT

1. User Availability: Issue # 1 of any UAT, even if users are marked as fulltime user to the project, still they will not be able to give you required time, due to their involvement in day-to-day operations. As most of the time organizations find it difficult to execute the full-time strategy, because the user assigned to automation project are usually more skillful than others in their department, and assigning them to the project for fulltime impacts the day-to-day operations of the organization, and if the organization is ready to do so, it will require their users’ interactions, which again impacts the user availability for UAT. Therefore, business analysts should maintain the record of user availability and escalate if user is not available as required for UAT.

2. Detail-Oriented Personality: There are some users who have a very detail-oriented personality or, in order words, they are perfectionists. These users are very hard to handle due to their expectations and requirements; they always want everything completed precisely and in detail. And their focus on detail drives them to the complex scenarios that a business has never faced before and might not face in the future as well, but they insist on testing those scenarios or handling of those scenarios in the software. That type of personality eats your UAT time like a grasshopper eats the grass. And as they are perfectionists, it is usually hard to explain your point of view to them and they also sometimes face problems in understanding the point-of-view of others, which moves the UAT phase into a never-ending cycle. But the important point that should be highlighted here is that this type of personality is problematic in UAT but can be good utilized in the requirement phase due to their detailed understanding of the business process.

3. Overlooking Personality: In UAT, you may face a personality that is easygoing and will not put required efforts on details of the system and its testing. This is the personality that will tell business analysts that "Everything is fine, all is good." That type of personality focuses on getting things done with simplicity; they sometimes do it because they don’t know about the pain they will be facing if UAT is not done effectively. That type of personality is very high risk for UAT as the chances of overlooking functionalities are too high, and Business Analysts should identify that personality and handle it by going into every detail and letting that user think that BAs want him to go in detail along with escalating the issue to the right level if required.

4. Issue Log Management & Prioritization: In UAT, many issues are identified, and if they are not logged and prioritized at right time, the whole UAT exercise will go wasted. While doing UAT sessions, the user identifies many issues related to application, and there could be a lot of issue types, some of which could be "GUI Related, Logical Observation, Application Bug, Business Not Mapped" etc. The bigger software has a greater type of issues along with their number as well. As a BA, you should follow a good mechanism of logging and managing the process of issues. Every issue reported should be logged-in in enough detail to be understandable by the user and technical TEAM both, as those issues will finally be reported to the technical TEAM for resolution. BAs should also consider scoping at this level, because there might be some issues that were not in initial scope due to "requirement not discussed" or some other reason. Those issues should be reported in log but BAs should identify them as "Out of Scope" and set the expectation of the user that this will not be handled in the current release of the software.

5. Understanding of Requirements: It has been observed that Business Analysts who are conducting UAT with user and were part of the initial requirement phase (Software Requirement Specifications Phase) were able to conduct the UAT more effectively than the business analysts who are assigned directly to UAT without their involvement in SRS Phase. This is due to the understanding of requirements, as if the BA is involved in the initial requirement phase he would have better and a detailed idea of what the specific requirement is all about, and if he is not, he might have his own point of view in place for specific requirements that create hassle for the user who is doing the UAT. Therefore, the recommendation is the BA who is doing UAT should be part of the initial requirement phase, and if he was not, he should go through each requirement in enough detail to understand the different aspects of requirement and its implications.

6. Complex/Demotivating/Offensive Personality: In projects, you face different types of personalities, and all of them impact the project in different ways at different stages. You might have seen some personality in your projects who used to say, "This project is not going to work," "This project is Pandora’s box," or my favorite, "We are playing GIGO (Garbage In Garbage Out)." Complex, demotivating or offensive personalities exist in projects and BAs cannot afford to avoid them. Good BAs should understand how to work with those personalities And how to get maximum out of them without going into never-ending arguments. These types of personalities are not very hard to handle and Business Analysts can handle them by maintaining his positive attitude, good relationships with individuals and good arguments to support his decision every time. And if things get uncontrollable, then BAs should know when and to whom the matters should be escalated.

Tasks performed by Business Analysts in UAT Phase

While doing UAT, business analysts perform different tasks based on the type of projects, duration and organization standards. The following tasks are generic to be followed in every UAT:

1.    Solution Validation: Validate that solution meets the Business Requirements

2.    Verify the Organization Readiness: BA should make sure that the end user is ready to use the software, by checking that the required resources along with relevant tools and trainings are delivered

3.    Identification & Validation of Scenarios: BA should identify scenarios that will be tested in UAT Phase and get those scenarios validated from end user

4.    Create Training Plan: BA should publish the training plan to engage the required resources

5.    Create UAT Plan: BA should publish the UAT plan so that required resources can be arranged

6.    Conduct the training of software: BA should allow user to do hands-on UAT by providing training of software, so that user satisfaction can be achieved

7.    Conduct UAT: UAT should be conducted keeping in mind the objective of UAT, which is to "make sure that system fulfills the day-to-day transaction of business along with any other known exceptions"

8.    Record the Results: UAT can only be effective if issues are logged religiously

9.    UAT Feedback: BA should from time to time confirm from user that solution fulfills the business needs as anticipated by user and update the feedback to related stakeholders

10.  Conduct UAT Signoff (Approval to GO LIVE)

Documentation created by Business Analyst in UAT

There could be different sets of documentation a Business Analyst does in UAT. The type and level of documentation is totally based on the methodology of the overall project, type of project and organization standards. E.g., By following the Water Fall, methodology in overall projects the level of formality in BA documentation becomes high and the number of documents increased, whereas in Agile there are low numbers of documents due to the low level of formality.

Following documents has been found to be useful for Business Analysts in the UAT phase; for better understanding, the list of documents is divided into sub-phases of UAT: 

UAT Planning

1.    for UAT (Must Have Document) & Business Scenarios Download Template 1 Template 2

2.    Business Process Flows to make sure that user is doing the right things (Must Have Document) Download Template

3.    Application Process Flows to map the business process on application to support user in identifying the relevant screen for each business process step (Must Have Document)Download Template

4.    Deployment Things To do to make sure setup/primary data is ready with user before initiating the UAT along with any other resources (users, trainings, machines, etc.) that are required for UAT Download Template

5.    Deployment Slip on successful deployment of application on client premises Download Template

6.    Training Plan to schedule the resources required to provide software training to users(Must Have Document) Download Template

7.    Training Script: This document is to prepare BA for the training and UAT session, in which BA identifies what screens he will be training and by entering what data and how?

8.    UAT Plan to schedule the resources required to conduct UAT (Must Have Document)Download Template

UAT Execution

9.    Training Signoff: User has accepted that training is done (high formality) Download Template

10.  UAT Issue Log: Should be maintained at any cost and shared with all stakeholders (Must Have Document) Download Template

11.  Daily UAT Summary to inform all stakeholders about the daily progress of UAT (Must Have Document) Download Template

UAT Closure

12. UAT Signoff is authorization from user to GO LIVE Download Template

13.  Customer Testimonial