The project phase in every UAT is crucial and should be handled with all the attention it requires. To better understand what this means, check what UAT testing means on the project. Most of the reasoning and steps involved are similar across situations.
The main goal here is to clarify UAT phase, to show to customer typical issues during UAT and give them tips how UAT can be handled. At Pricefx we have many presentations outlining the process, but this section will take you through the common features across the board. Learn all the kinks here.
In the coming lines we will share a sample we used with a client. This should serve you well as an illustration of what UAT testing entails on the project.
UAT overview
It is paramount to define and clarify before even beginning to test, what are the goals of testing While these may vary to a certain extent from customer to customer, in general, they rely on 3 major pillars:
Test E2E process in real environment - Pricefx software together with CRM, ERP and other software you use for pricing
Check that Pricefx software does what it should to support your business
Move the project to live, start using it and let it help to your business
Be advised that although it may be tempting to think so, UAT testing is in fact not a repetition of the tests from the Feature Sprints. This is also not the time to get familiarized and learn the application nor it is an opportunity to submit any change requests.
Also kepp in mind that before and during UAT there are certain responsibilities you must take on and dedicate time to. As such you should create UAT test cases and test accordingly on your QA partition. Any bugs you may (and very likely that you will) find must be reported in JIRA. Finally you should accept the solution so the we can formally conclude the development phase.
The time you should allocate on average for UAT is between 2 and 3 weeks depending on the size of the project and the user story library. Note that while 2-3 weeks are typically enough for testing, testers must be prepared and test every day for a while (at least 2 or 3 hours).
Good to know: Be aware and prepared that it is impossible to do the whole testing in one day.
Moreover, if you have a limited timeframe for testing you should be even more organized and have a structured approach ready.
Keep in mind that there is a set of expected documentation you should have completed before and during the UAT. It includes;
Test cases for UAT to help to testers to be clear about what must be tested during UAT and give them the possibility to set up the time for testing and discuss ambiguities before UAT
Reporting of test results
Bugs in JIRA
Tips for a smooth run:
Managers should have an overview of where testers are in the process of testing, what must still be tested
Testers know what they already tested so they can organize their testing
Correct bug reporting will speed-up fixing process
Typical issues during UAT
Like with any project, UAT makes no exception from encountering issues throughout. However, there are steps that you can take to avoid these for as much as possible. We have already covered some of them in the previous sections and we will surely tackle others as we move along. Before that though, let’s take a quick look at what are the most commonly encountered issues during UAT.
No plan for UAT testing
Testers are doing exploratory testing
UAT testing is not under control
Time expected for UAT testing is usually insifficient
It‘s not possible to monitor the progress
Problems with accepting the solution – UAT testers always want to test more
No priorities of test cases
Critical issues are reported at the end of UAT
No training for UAT testers
Reported bugs are not bugs but questions, change requests or misunderstandings
UAT testers are learning how to work with Pfx application instead of testing
UAT testers don‘t have time for testing because they are doing other work - there should be time dedicated for UAT
Tips for a smoother UAT
Before UAT
Select a group of UAT testers
Usually SMEs – check with them that tey will have time for testing
Make sure they have access to all systems they need for UAT
Create UAT test cases
Based on business cases, they should cover end to end scenarios
Should contain at least check lists of what should be verified
Choose UAT leader
Somebody from the project team who knows the solution
UAT testers are reporting results to him/her
Choose test management tool
It is possible to use X-Ray in Pricefx Jira (see below), note: Excel is not a test management tool!
Training of UAT tester – application and UAT process
Demo session with all UAT testers to show them how to use developed software and clarify expectations from UAT testing
Let testers check the functionality for themselves and help them with first issues and questions
It‘s useful when demo is done based on UAT test cases and the session is recorded
During UAT
Testing according to UAT test cases
UAT testers should have Test execution (if X-Ray is used)
Test cases should be prioritized so critical functionality is verified at the beginning of UAT
Keep in mind there is limited time for UAT phase, there is no time to retest everything from Feature Sprints
Report test results
UAT testers should log test results every day
Important part of test results are attached screens and description of what was tested
Report bugs in Jira
Issue Type = „Bug“
Reported bug should contain description of the issue, expected result and screens or other useful files
Retest all fixes
Close the bug and finish testing of the UAT test case
Check the progress of testing every day
UAT Leader should check the progress every day to see if somebody from UAT testers has good progress in testing
Acceptance
Acceptance after UAT testing means we can continue to Go-Live and using the solution in production. You may think that at this stage the solution should run flawlessly sand unless it does so you may be tempted to reject the test. Please note that there is no reason for not accepting because of medium and low bugs, as they can be fixed after Go-Live during Stabilization phase.
There is a time frame for UAT phase on the project and a clear decision should be made at the end of it.
Bug prioritization in JIRA
Priority | Explanation |
Critical | User is unable to use the Solution, resulting in a critical impact on business operations. This condition requires immediate resolution. Example: system is down. |
High | Any high priority defect which is incorrect price calculation. No workaround is available, and the defect is time sensitive. Example: defect is blocking pricing from being exported to ERP systems. |
Medium | The defect requires attention but does not block pricing. A workaround is available but may not be optimal. Example: a defect requires the user to perform manual calculations. |
Low | Minor loss of functionality. These may be monitoring issues, clarifications, documentation questions; or administrative tasks. Example: update the column label or any other cosmetic related defects. |
Test Manangement X-Ray Overview
For the past 3 years, Pricefx together with our customers have successfully used X-ray as one of our main testing tools. X-Ray presents many benefits and tow of the main ones are that it is free of charge for the project and it is a test management tool plug-in in JIRA.
Moreover, the test documentation and results of testing are visible for our project team and for the customer in the Testing Board.
Test cases are in separate issue types, not in Comments section and are linked to user stories, so it is clear why they were created and what they test.
X-Ray can be used for Feature Sprint and UAT testing and it is intuitive for everybody who is familiar with JIRA. As such it provides JIRA functionalities like exports, imports, workflows, and others.