Skip to content

All.pytest allure.explanation

Pytest Allure - Explanation#

Overview#

  • Allure Report boosts collaboration and project quality by providing clear, detailed test reports that aid issue resolution for different team members
  • The goal is to set up all our tests to generate and publish this report
  • Here is the demo report from allure
  • The simplified flow for the Allure report is:
flowchart TD

subgraph A1[" "]
direction LR
        A[Run tests and generate\nthe Allure output] --> B[(Backup the Allure output\nto the AWS S3 bucket)]
        B --> C(Copy last history\nfrom the S3 bucket\nto the Allure output folder)
end
subgraph B1[" "]
direction LR
        D[Generate the HTML-report\nfrom the Allure output] --> E[(Publish the HTML-report\nto the S3 HTML bucket)]
end
A1 ==> B1 ==> C1[View the report in the browser]

Core Features#

Rich and Interactive Reports#

  • Allure generates visually appealing and interactive HTML reports, making it easy to analyze test results
  • Reports include detailed information about test cases, steps, attachments, and more

Annotations and Labels#

  • Allure uses annotations and labels to provide additional information about test methods, making it easier to understand and categorize test results
  • Annotations are used to mark and describe test methods, and labels help in categorizing and filtering tests
  • Allure maintains a history of test runs, allowing you to track changes in test results over time
  • Trends and statistics help identify patterns, improvements, or regressions in the application's behavior

Key Components#

Pytest Plugin for the Allure Output Generation#

In order to generate the Allure output, we need to install the allure-pytest plugin. For the time and efforts saving reasons, we will install it on-the-fly in the container where we will run the tests. This feature is introduced as the allure_dir parameter to the run_<type>_tests tasks. If specified the allure-pytest plugin will be installed and the results will be stored in the specified directory.

Allure Reporting Tool#

The CLI utility for generating Allure reports creates an HTML report from the Allure output. We'll install this utility using GitHub Actions workflow to generate the HTML report.

How It Works#

  • Allure Report is composed of a framework(pytest) adapter and the allure command-line utility
  • Run the tests with pytest options --alluredir=allure-results to store the results
  • The generated files include:
  • If required, add additional files to the allure-results to display additional info in the report
  • Additional files includes:
  • Use allure generate to generate the test report into the specified directory. For e.g.: allure generate allure-report

A test report generated by Allure can not only display data about the latest test launch, but also help you compare it with the data from previous reports. To do so, Allure can keep a history or previous reports.

In a tests report with the history included, you can:

  • See what statuses did a test have previously (see Details panel → History tab)
  • Find tests that changed status since last report (see Sorting and filtering → Filter tests by marks)
  • See how specific values change over time (see Graphs)