Introducing Test Engine: An open platform for automated testing of canvas apps
Testing is an important part of the software development life cycle. Adequate functional testing of applications helps to ensure that business processes stay unblocked, helps to reduce support costs, and helps to build trust in your applications. And as apps grow and become more complex, the ability to ensure that changes do not break or alter the app experience becomes more and more critical. Our mission is to empower all makers of Power Apps to easily create robust tests that can be seamlessly integrated into your organization’s application lifecycle management (ALM) practices.
Introducing Test Engine
We’re excited to announce the initial preview of Test Engine, an evolution of our Power Apps testing tools. Test Engine builds upon the key use cases of Test Studio, but takes it in a new, powerful direction through open source collaboration and use of the Playwright browser testing platform. The goals of Test Engine are to provide customers with a robust testing platform for all types of Power Apps, and to make it super easy to integrate automated testing into your app development processes.
Today’s announcement is the first step towards those goals. The project is provided as an Open Source project on GitHub that, when built, will create a local executable that you can use to execute tests.
This initial release supports the ability to author tests for Power Apps canvas applications. We plan to continue to iterate on this project, adding support for all types of Power Apps, as well as enhanced tooling to facilitate integration into your CI/CD systems like GitHub and Azure DevOps. For now, we welcome you to exercise this tool and give us feedback as we continue to build out the platform.
Power Fx test authoring
Test Engine allows you to define mock responses for connectors. This technique lets makers continue testing of Power Apps while isolating it from the remote APIs that the app connects to. This can be useful your apps hit endpoints that have side effects, like inserting rows into Dataverse tables.
Screenshot function and video recording
Test Engine introduces the new Power Fx function Screenshot() that allows you to take a screenshot of the app state at any point during the test run. For example, you may want to take a screenshot at the beginning and the end of your test suite to capture what the end user sees. In this example, a screenshot is captured at the beginning of each test case, and upon completion of the test suite:
Test Engine can also automatically record a video of the entire test run. Simply set the recordVideo setting to true. You can use the captured video to observe exactly what the end user is seeing during the test run. This can be very useful when diagnosing and investigating failed tests.
Only update tests when you update your app
We have heard from customers that using traditional browser automation tools to test Power Apps apps is difficult to get right and frequently breaks, requiring lots of updates to tests when the maker hasn’t made any changes to the app. This is because traditional testing methods require the test author to interact with the browser document object model (DOM) of the app as its UI is rendered in the web player. With Test Engine, all of this is handled by the platform. Your tests are written using Power Fx and referencing the names of the controls that you define at design-time. You will only need to update your test plans if you make changes to the app itself.
Coming Soon: Reuse tests recorded in Test Studio with Test Engine
If you are a user of Test Studio and already have tests recorded using that tool, we are providing a way that you can reuse them directly in Test Engine. The ability to download test suites from Test Studio will start rolling out to customer tenants towards the end of October 2022. The downloaded test suite can be used in Test Engine without any modification to the test plan file. Watch the Power Apps Studio release notes for information about when the rollout starts.
Visit Test Engine on GitHub and try it out. You can build the solution with a couple commands. Once you’ve done that, the Test Engine repo has a library of sample test plans and solutions you can use to get started. To use the samples, import the provided solutions into your tenant and run the corresponding test plan. Since it’s open source, we welcome contributions to both code and documentation. Let us know about any bugs, feedback, and feature requests using the Issues list within the GitHub project.