During a ERP implementation, there will be a need to run End-To-End (E2E) tests to ensure the enterprise business flows are functional as per the business requirements and for the overall enterprise system architecture, design, and master data provisioning. The E2E business flow will crossover functional swim lanes, areas of specific business expertise, and touching finance at some point, typical of an ERP solution.
The E2E testing approach is documented in the project Test Plan, however, the tactical part is often where mistakes can be costly, impact the project deadlines, and the quality of the solution implemented. In this blog, I will discuss all tactical activities to successfully conduct E2E testing. These activities are typical for any ERP implementation: SAP, Oracle, etc.
High Level E2E Testing Tasks
- Planning and schedules
- E2E Business Process Design
- Identifying and Booking Resources
- E2E Testing Kickoff
- Scripts Development
- Test Environment Readiness
- Execution
- End of Test Reporting
- Debriefing Lessons Learned
- Sign-off
Planning and Schedules
Your planning and associated schedules are critical deliverables they will be tuned and confirmed during sessions with managers involved in your E2E testing. It should be finalized and vetted by all functional and technical managers, delivery managers, and project managers.
When looking at planning, you should also anticipate last-minute surprises and how to cope with them, a schedule compression as an example. You must also take into account the current project risks as they may impact your E2E testing phase.
You will have to revise schedules after the E2E Business Process Design and Resource Identification and Booking. During the E2E Business Process Design you may identify one or multiple batch jobs (internal or external), which can not be triggered manually and will break your execution in 2 or more days.
Being efficient is a key success factor you have to use your resource’s time efficiently, no point in booking a participant for 8 hours if the task is sequential and only last 1 hour, this is done via resources orchestration.
On global implementations or outsourcing, teams may be located in a different part of the globe and scattered across multiple time zones, you will have to account for it in your schedules.
E2E Business Process Design
This is your transactional and technical blueprint to construct your E2E testing flow. It could be representing Order-To-Cash (OTC), or Meter-To-Cash, or any of the enterprise lifeline E2E business flow. It will define the scope of your E2E testing.
The E2E business flow is built via sessions with the business teams for the transactional flow and also with the technical team to identify systems and interfaces to support the E2E testing.
Most information should be in the project business blueprint. There will be a need to validate and confirm the E2E flow and the state (As-IS and To-Be) with the business.
Identifying and Booking Resources
E2E testing implies crossing over functional areas, each with a specific business and technical expertise. You will need to precisely identify the resources required to conduct your E2E testing for each area crossed by your testing flow. The E2E Business Process Design will provide you with enough information to identify the functional and technical expertise required for your E2E testing.
Participants are often members of separate teams, you will need to clear the resources in advance with their manager. Most E2E testing involves third-party systems interfacing with your internal enterprise systems, you will need to have the vendor support your activities and have resources committed to the E2E testing.
E2E Testing Kickoff
A must-do to orient and align all E2E testing participants (including third-party vendors). In the kickoff, you will go over: objectives, goals, deliverables, success criteria, schedules, orchestration, known risks, and tools to use during the E2E testing.
Scripts Development
Once your E2E business flows are documented it is time to create step-by-step scripts and identify data to test it. This activity is based on the maturity of the organization in terms of past projects testing deliverables produced.
You may have to create E2E test scripts from scratch and identify data to support the execution. Or, you may use previously created test scripts for each functional area, assembling them and tuning them to construct a full E2E script.
Test data should be aligned with the E2E scripts. If specific variants needed do not exist in the test environment, manual data creation will be required.
You will need to estimate the time to execute each step and the overall E2E execution timing to adequately schedule and orchestrate the E2E tests.
Once completed you will have to run the E2E scripts in a test environment to validate the E2E scripts and data and revise them if need be.
Test Environment Readiness
Environment provisioning is a critical entry criteria, without it you won’t be able to execute your testing. You need to allocate enough time ahead of your testing to procure, build, and configure the environment.
Considerations:
- If you are using existing systems you will need to book them in advance for your execution period, as they may be in use by other ongoing projects. You may need a data refresh from production, which will need to be approved and will create a system outage for other projects using the systems
- You will have to create some test data manually if not available in your test systems
- Data migration may be necessary on one or multiple systems used during the tests
- All systems under test in your E2E testing must be connected to support your E2E testing
- Interfaces must be connected and connectivity tested
- When interfacing with vendor’s systems, you will need to make sure that you have a vendor test environment available for your testing
- All resources involved in the E2E testing must have access to the system and supporting tools
- All new transports (changes) must have met the exit criteria from the pervious phase, usually System Integration Test (SIT) and be deployed in the E2E environment
Execution
Before the formal E2E testing starts, you will need to do a dry run to validate that all systems are a go and have enough time to resolve the potential issues before the E2E tests formally starts. It would rather be embarrassing to have all the resource lined up the day of the execution and have to cancel the session.
Your execution should be broken down into 3 break and fix cycles, with enough time in between cycle for the technical team to resolve defects found during a cycle.
During the execution you will need to have defect meetings to go over the defect found, fixed, ready for retest, and closed. In the defect meetings you will need to have one third party vendor attending the meeting when defects are found on the vendor’s side.
My day running the E2E tests execution is fully dedicated to the E2E where I control the testing flow across multiple functional areas. I also open a phone bridge during the execution to ensure participants can promptly jump on the bridge for orchestration, issues, and do the hand-off from one team to the other. It also provide the team with the ability to quickly jump the bridge and resolve issues such as: roles and privilege's, data, or other technical issues that are not a functional defect.
End of Test Reporting
The metrics collected and reported should be stated in the Test Plan and your reporting should conform to the standards described in the Test Plan.
Debriefing and Lesson Learned
Post E2E execution setup a meeting with the participants to confirm the end of test reporting items and also collect lessons learn to be incorporated in future E2E test. Usually the project Manager will have a log of process improvement items part of the project Post Mortem lessons learned, your findings will be of value project process improvements within the organization.
Signoff
Post execution, setup a meeting to formally sign off on the E2E testing phase and formally give the go to move to the next phase: UAT. The presentation for that meeting will be high level synopsis of your end of test report in a few slides, a slide which shows clearly the deliverables produced to meet the exit criteria, a list of defects not resolved (at that point defects should all be low severity). The audience is usually management since you have reached the end of a critical milestones.
Next step is UAT, I will address the UAT best practice in an upcoming blog, stay tuned!
The Million Dollar Saving E2E Testing Follow-Up Activity
Now that you have your E2E test scripts and the data set to execute the scripts, what would be the next step to leverage from it, after all your E2E scripts are truly a way to test the enterprise business-critical path.
If you guess it right, it is automation! Automating E2E scripts will provide you with a valuable tool to automatically and rapidly test the full enterprise-critical business flows for any major release, and avoid costly incidents in production if the enterprise business flow has been impacted by a change.
By a change I don’t only mean a functional change, but also system patch deployments, system upgrades, infrastructure appliances changes, etc. Automation and reuse are key factors in lowering cost and reducing time.
Don’t expect a 100% testing coverage with automation, some areas will still require manual testing.
Take It To The Next Step And Kill 2 Birds With 1 Stone To Avoid Costly Downtime or System Outage in Production
On one of my projects, where production performance was a must, we used HP Loadrunner to automate the E2E testing. HP Loadrunner is usually used to do performance testing, however, in this case, we used HP Loadrunner for the E2E functional automation and also to measure system degradation against our baseline after a change and before production go-live.
Conclusion
Enterprise E2E testing is a critical activity for business continuity in production, best practice and discipline will provide you with the ability to enter and exit the E2E testing with success.
I hope the information in this blog will help you run successful E2E tests!
If you enjoyed and benefited from this blog, crypto donations are welcome to help support my future blogs:
BTC: bc1qm3f8ky4xcwg62nn9n9nx5dnmtkklehwkhz5753
ETH: 0x13c77781d7661420fB58b538a18A5B47eA50F462
XMR: 4AS1JSypyE84B7dYd4CRvDKQJHHwxKCY6KGC8NjXCyR2SD9qQLVVDknekE4eerDbkRRviUtG3Q1j8cE7Qjujcugw6W7deno
ADA: addr1q8vw4ct4zgsv4qan0nylcnk82gngsqtukyz43ms9htuthmwcatsh2y3qe2pmxlxfl38vw53x3qqhevg9trhqtwhch0ks8js0um
SOL: 7EvtSvhfZJRDEnnHmaWBKzqmkqi8yTv1FkJ3gP9S2YW6
LTC: LXhGaCqmWcr2VKuANqgVf81gsFZejVyfjZ
SHIB: 0x13c77781d7661420fB58b538a18A5B47eA50F462
