top of page

Analyzing and Visualizing Test Execution Data with Power BI





As testing becomes more complex and lengthy its very important every organisation must develop AI Enabled testing dashboards that determine health of business applications in terms of Stability, Performance, Ease of Use.


Power BI from Microsoft team one such solution which helps enterprises to add more value to the test data which gets generated for every test releases either in DEV, QA or PRODUCTION environments.


Power BI Solutions helps Enterprises achieve following tasks with respect to test data

  • Connect, import, shape, and transform test data for business intelligence (BI)

  • Visualize data, generate reports, and schedule automated refresh of test reports as more test cycles gets executed

  • Create and share test dashboards based on reports in Power BI desktop and Excel.

  • Use natural language queries

  • Create real-time dashboards based on test execution logs and test reports generation


Before we discuss how above mentioned goals can be achieve with Power BI Solution, Let's understand first features of Power BI platform.


What is Power BI?


Power BI is the collective name for an assortment of cloud-based apps and services that help organizations collate, manage, and analyze data from a variety of sources, through a user-friendly interface.


Power BI pulls data together and processes it, turning it into intelligible insights, often using visually compelling and easy-to-process charts and graphs. This allows users to generate and share clear and useful snapshots of what's happening in their business.


Power BI connects to a range of data sources, from basic Excel spreadsheets to databases, and both cloud-based and on-premise apps.


Power BI is something of an umbrella term and can refer to either a Windows desktop application called Power BI Desktop, an online SaaS (Software as a Service) service called Power BI Service, or mobile Power BI apps available on Windows phones and tablets, as well as for iOS and Android devices.


Power BI is built on the foundation of Microsoft Excel, and as such, the learning curve from Excel to Power BI is not that steep; anyone who can use Excel can use Power BI, but the latter is far more powerful than its spreadsheet counterpart.


What does Power BI actually do for testing?


Microsoft Power BI is used to generate test reports and surface insights based on a company's test data. Power BI can connect to a wide range of test data sets, and "tidies up" the info it's fed so that it can be better digested and understood.


The reports and visuals generated from this test data can then be shared with other users like Developers, Product Owners, Directors and CTOs to take informed business decisions.


Power BI helps users see not only what's happened in the past and what's happening in the present, but also what might happen in the future. And this predictions about future makes Power BI stands stronger compare to other products.


Power BI is infused with machine learning capabilities, meaning it can spot patterns in test data and use those patterns to make informed predictions and run "what if" scenarios. These estimates allow testers to generate testing forecasts, and prepare themselves to meet future testing demands and other key metrics.


Example: Machine Learning engine of Power BI takes test data from last 1000 test executions span across 50 Business Applications. Based on its powerful machine learning models it transform raw test data into intelligent data and create test visualizations that tells which areas of applications need more test efforts compare to other areas of application by analyzing defects data for every test release and for every business application.

Here are some of the key benefits of using Power BI in testing life cycle:

  • Businesses can input huge quantities of test data into Power BI that many other platforms would struggle to process

  • Built-in machine learning features can analyze test data and help users spot valuable trends and make educated test predictions

  • Test information can be visualized using powerful templates to allow businesses to better make sense of their test data

  • Power BI is cloud-based, so users get cutting edge intelligence capabilities and powerful algorithms that are updated regularly and helps in better predictions

  • Powerful personalization capabilities allow users to create test dashboards so they can access the test data they need quickly

  • Alerts can be set up on Test KPIs to keep users up to date important metrics and measurements.

  • Power BI has an intuitive interface that makes it far more user-friendly and easy to navigate then complex spreadsheets

  • The platform integrates with other popular business management tools like SharePoint, Office 365 as well as other non-Microsoft products like Spark, Hadoop, Google Analytics, SAP and Salesforce

  • With data security a massive talking point for modern businesses, Power BI ensures data is safe, offering granular controls on accessibility both internally and externally


What components make up Power BI?


The Power BI product is made up of a number of apps, each with their own features and uses. These include:


Power Query: a data connection tool that lets you transform, combine, and enhance test data from several sources like excels, word documents, pdfs, text logs, data bases like MS SQL, MYSQL, ORACLE.


Power Pivot: a data modeling tool for creating test data models


Power View: a data visualization tool that generates interactive charts, graphs, maps, and other visuals once test data models gets generated


Power Map: another visualization tool for creating immersive 3D visuals of test KPIs like Defect Density, Test Execution Time, Test Productivity, Test Coverage


Power Q&A: a question and answer engine that lets you ask questions about your test data in plain language


What versions of Power BI are available?


There are a handful of versions of Power BI to choose from, depending on your budget, your needs, and how you want to deploy it. These versions range from light use to comprehensive features, free to premium, so users can select the one that best meets their requirements.


You can opt for a Windows desktop application called Power BI Desktop, an online SaaS (Software as a Service) service called the Power BI service, a mobile Power BI app available on iOS and Android phones and tablets, or an on-premise version known as Power BI Report Server.


Here's a quick overview of the different versions, before we drill down further into each one:




Power BI Desktop — Free, intended for small to midsize businesses


Power BI Service

a) Power BI Pro — Paid per-user license, needed to get access to advanced features and the ability to share reports

b) Power BI Premium — Licenses by scale, intended for large businesses and enterprises


Power BI Mobile — Device-based app for phones and tables


Power BI Embedded — A white-label version of Power BI which Independent Software Vendors can embed in their own apps, rather than build their own analytical features


Power BI Report Server — An on-premise version of the Power BI Desktop app for businesses that need to keep their data and reports on their own servers


Lets understand now more about different products offered by Micro soft for BI space that help enterprises to build powerful Test Dashboards. Any product can be chosen as per business case and budget constraints.


Power BI Desktop:



Also known as Power BI Free, Power BI Desktop is a free desktop application you can install right on your own computer. Power BI Desktop works cohesively with the Power BI service by providing advanced data exploration, shaping, modeling, and report creation with highly interactive visualizations. You can save your work to a file, and publish your data and reports right to your Power BI site to share with others.


Power BI Free is included in all Office 365 Plans, and you can sign up for Power BI Free any time you like.


Desktop allows you to connect to data sources to Power BI, with no limits or restrictions other than the total amount of data you can feed in, and how much you can upload at a time. Desktop users get 10GB of total storage in the Power BI cloud, and can upload data 1GB at a time.


Desktop will cleanse and organize data, and you can create as many visualizations as you want; Desktop gives you full access to Power BI's library of visualization templates. You can also export data and reports to Excel. Using Power BI Desktop, you can refresh your reports up to eight times every day, either on the hour, or the half-hour.


The "catch" with Power BI Desktop is that you can't share anything with your peers. You can publish reports to the web, but these reports will then be made public, which isn't ideal when you're dealing with proprietary business data.


For beginners in data science space, Power BI Desktop is best application to start.


Power BI Service


Power BI Service (usually known simply as Power BI) is the full version of Power BI, and is hosted on Azure, Microsoft's cloud platform. There are two ways to license Power BI Service: Power BI Pro and Power BI Premium.


Power BI Pro


Power BI Pro isn't hugely different to Power BI Desktop. They have the same visualization options, the same limits on storage and file upload size, and the same report refresh allowance. The key difference is that with Pro, you can share your data, reports, and dashboards with others privately—provided they also have a Power BI Pro license.


There are a few other extras that come with the Pro license too. For example, with Pro you can create app work spaces, where you can put together collections of related dashboards and reports and create easily accessible content packs.


Power BI Premium


Power BI Premium is an alternative way to get access to Power BI. Rather than purchasing licenses for individual users like you would with Power BI Pro, with Premium, you pay for the amount of space and processing capacity you want to carve out for your business. There are six capacity models, each one with varying amounts of memory so users can choose the amount they need to run their Power BI platform.


Premium is intended for enterprise-level businesses that generate huge amounts of data, and require extensive access to the app. Shelling out for Premium means you get a dedicated portion of capacity to process your BI workloads, with all the necessary infrastructure supplied and supported by Microsoft.


Premium isn't a license as such; instead, you're paying for exclusive use of a predetermined amount of computing power. For those users who need to create and publish reports, share their dashboards, and collaborate with other users in app workspaces, a Power BI license is required in addition to Power BI Premium. For those who just need to access and consume Power BI content, though, no additional license is required; Power BI Premium will cover read-only use.


There are a few other features exclusive to Premium too, such as the ability to store BI assets on-premises using Power BI Report Server (more on Report Server later), up to 100 TB of data storage, and a 50GB cap on dataset size.


Power BI Report Server


Power BI Report Server is an on-premises server product that ships with Power BI Premium. Report Server allows users to take advantage of Power BI on-premise. For some businesses, uploading their data to the cloud isn't an option. This could be because there are data regulation laws applying to their particular industry, or maybe they don't have the necessary infrastructure or connectivity where they are to be able to reliably access SaaS platforms.


Power BI Report Server can be deployed behind a firewall, and is compatible with Power BI in the cloud so that businesses have the freedom to move a cloud-based version of Power BI when they're ready.


Report Server packs has similar functionality to Power BI Service, though it's largely focused on report generation, and there are a number of key features that are exclusive to its cloud-based sibling. Report Server users must use the Power BI Desktop app, and cannot work in a web browser. They also don't get access to dashboards, real-time streaming, content packs, app workspaces, natural language queries, the ability to analyze data in Excel, email subscriptions, or data alerts.


Power BI Embedded


Power BI Embedded allows Independent Software Vendors (ISVs) and developers to embed the functionality and capabilities of Power BI into their own apps on a pay-as-you-go, "white-label" basis. So, rather than building their own reporting features, they can simply add Power BI into their products, and Power BI will take care of all the reporting and data analysis needed. End users of the app won't even know that the app's analytics are powered by Power BI; it will just look like part of the app.


Power BI Mobile


Power BI Mobile is (unsurprisingly) Power BI's native mobile app and is available for Windows, iOS, and Android devices.


Through the app, users can get secure access to real-time dashboards and reports, whether that data is stored in the cloud, or on-premise in SQL Server. Mobile users can also create and share reports, submit natural language queries, and set up push notifications to get personal data alerts sent to their device.


Now lets look at Power BI's main competitors that can also help enterprises building intelligent dashboards:

  • Google Data Studio

  • Qlik Sense

  • Amazon QuickSight

  • IBM Cognos

  • Elasticsearch

  • SAS Visual Analytics

  • Jet Reports

  • TIBCO Spotfire

  • MicroStrategy Analytics


Building better QA for tomorrow

bottom of page