Feature Comparison

Test Management

Test Issue Type

Test Case in Jira as Issue type called Test

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

No

No

Test cases exist in New Zephyr Squad, but use a different data architecture.

In Legacy Squad, test cases are Jira issues.

In New Squad, test cases are internal objects. They are not Jira Issues, they belong to New Zephyr Squad database, managed by SmartBear.

For both Legacy Zephyr Squad and New Zephyr Squad, Test cases can have test steps with custom fields and attachments. Test cases can be added to a cycle and folder and executions can be created.

Test Cases

A test case is a test artifact developed for a particular testing need.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

Yes

Yes

Test cases exist in New Zephyr Squad, but use a different data architecture.

In Legacy Squad, test cases are Jira issues.

In New Squad, test cases are internal objects. They are not Jira Issues, they belong to New Zephyr Squad database, managed by Smartbear.

For both Legacy Zephyr Squad and New Zephyr Squad, Test cases can have test steps with custom fields and attachments.

Test cases can be added to a cycle and folder and executions can be created.

Test Cases custom fields

You can use custom fields to customize Test Cases according to your needs.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

Yes

Yes

The advanced type of custom fields are not migrated.

image-20221103-081927.png

Test Cases attachments

Use attachments for Test Cases to share insight with your team.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

Yes

Yes

N/A

Test Cases multiple components

Every Test Case accepts multiple components.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

No

Yes

If the component field in Legacy Squad contains multiple component values, one of the components needs to be assigned to the component field for the New Squad test case and all others need to be mapped to the New Squad test case custom field called “Squad components“ with the type Select List (Multi Choice).

Test Steps

Test Steps describe the execution steps and expected results that are documented against each one of those steps.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

Yes

Yes

N/A

Test Steps Wiki formatting

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

Yes

Yes

N/A

Test Step custom fields

Use custom fields to customize your Test Steps accordingly to your needs.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes (maximum 5)

Yes

Yes

N/A

Test Step attachments

Use Test Step attachments to improve collaboration with your team.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

Yes

Yes

N/A

Export Test Case with Test Steps

Export your test cases to XML, EXCEL or BDD-Gherkin.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

Yes

N/A

N/A

Import Test Cases

Using our import wizard, users can import test cases—including all test case data—into New Zephyr Squad Cloud in the following formats CSV, Excel(via CSV), XML).

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

Yes

N/A

N/A

Test Case repository

Use Test Repository to view and manage all Test cases within this Jira project.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

No

Yes

N/A

Does not exist in Legacy Squad

Test Case archiving

The archive button puts your Test Cases into the Archived Test Cases folder and makes them inactive throughout New Zephyr Squad but saves all the Test case data in case you want to restore them later.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

No

Yes

N/A

N/A

Test case history

The History tab allows tracking all the changes made on this test case.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes (via Jira)

Yes

Yes

N/A

Test Step history

Test step history allows you to view all changes of a particular test step within a test case.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

Yes

Yes

N/A

Test case search

The search bar filters title information, both Key and Name values. Results filter as you type.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes (via Jira)

Yes

N/A

N/A

Test case saved filter

Save filter option allows users to save custom queries that they often use.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes (via Jira)

Yes

No

N/A

Test Planning and Execution

Test Cycles

A test cycle is a focused set of test cases that are grouped to achieve specific testing goals.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

No

No

Test cases exist in New Zephyr Squad, but use a different data architecture.

In Legacy Squad, test cases are Jira issues.

In New Squad, test cases are internal objects. They are not Jira Issues, they belong to New Zephyr Squad database, managed by SmartBear.

For both Legacy Zephyr Squad and New Zephyr Squad, Test cases can have test steps with custom fields and attachments. Test cases can be added to a cycle and folder and executions can be created.

Test Cycle custom fields

Use custom fields to customize your Test Cycles accordingly to your needs.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

No

Yes

N/A

Does not exist in Legacy Squad

Test cycle attachments

Use attachments for Test Cycles to share insight with your team.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

No

Yes

N/A

Does not exist in Legacy Squad

Ad-hoc Test Cycles

Scale Cloud gives your team the flexibility to execute test cases individually, without having to add them to a test cycle. Just create and link test cases or acceptance tests to user stories/bugs, then initiate the test execution directly from the Jira issue.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

Yes (Ad-hoc test executions)

Yes

N/A

N-level folder structure

Group testing artifacts into folders and sub-folders to organize the work in the way that works for you.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

No

Yes

Yes

In New Squad, the multi-level folder structure for versions and test cycles can be created to organize the hierarchy. Then for each folder, a test cycle can be created which will be equivalent to a test cycle in Legacy Squad.

Export Test Cycles

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes (CSV, HTML, XML)

Yes through reports (Excel, pdf)

N/A

N/A

Test cycle search

The search bar filters title information, both Key and Name values. Results filter as you type.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

Yes

N/A

N/A

Test Execution

The Test Player is where actual testing occurs and is tracked within New Zephyr Squad. Regardless of how many test cases you want to execute, the Test Player is where the fun happens.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

Yes

TBD

N/A

Test Execution custom fields

Use custom fields to customize Test Execution accordingly to your needs.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes (maximum 5)

Yes

TBD

N/A

Test Execution attachments

Add attachments to the Test Execution to share test results with the team.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

Yes

Yes

N/A

Test Execution custom status

Use custom statuses to configure test execution according to the needs of your organization.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

Yes

Yes

N/A

Defect link on Test Execution

You can create a new defect or link an existing one to a test execution or to a test execution step.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

Yes

Yes

N/A

Export Test Execution

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes (CSV, HTML, XML)

Yes through reports (Excel, pdf)

N/A

N/A

Test Step Results

You can mark each step of your test case with different status accordingly to its execution results.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

Yes

Yes

N/A

Test Step Results attachments

Use attachments to Test Step to illustrate test step results to your team.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

Yes

Yes

N/A

Test Step results custom status

Use custom statuses to configure test execution according to the needs of your organization.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

Yes

Yes

N/A

Defect link on Test Step Results

Create or link a defect right from the test step to indicate the bug that contribute to the failed execution.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

Yes

Yes

N/A

Time tracking (during the execution)

Use the time tracking feature to measure the time it takes to perform a particular test case.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

Yes

Yes

N/A

Reporting on Test Execution

Make reports on test execution to increase the traceability of your testing.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

Yes

N/A

Similar capability of reporting exist in New Squad.

Reporting on Top Defects

Make reports on top defects to increase the traceability of your testing.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

No

No

Doesn't exist in New Squad

Traceability

Use the traceability section on the issue view to see the execution statuses of test cases/cycles/plans associated with this issue or manage your test process linked to a particular issue.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

Yes

N/A

Similar capability of reporting exist in New Squad.

Search Test Executions

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

No

N/A

Doesn’t exist in New Squad.

Test Execution filters

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

No

No

Doesn’t exist in New Squad.

Test Execution bulk operations

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes (Status/Move/Copy/Assignee/Associate Defects/Custom Fields/Delete)

Partially (Assign tester, environment, and iteration)

N/A

Doesn’t exist in New Squad.

Test Execution change history

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

No

Partially

Only at the test cycle level e.g. a test case has been removed or added or information about the test cycle has been edited.

For Execution changes, New Squad won’t record the history e.g. change of status of an execution.

User Management

Roles and groups

Jira has three default roles namely: Administrators, Developers, and Users.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes (via Jira)

Yes (via Jira)

N/A

N/A

Permission Control

Use permissions to manage user interaction with Scale/Squad.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Jira permissions (Cloud)

Jira and Zephyr permissions (Server)

Jira and New Zephyr Squad permissions

N/A

N/A

Control of Issue types supporting app

When you enable an issue type for New Zephyr Squad, you'll be able to see the Traceability Panel on the issue. If you disable an issue type, you won't be able to see this panel.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes for team-managed (next-gen) projects. No for company-managed (Classic) projects.

Yes

N/A

N/A

Reports and Advanced Analytics

Jira dashboard gadgets

Use gadgets to display summaries of Scale projects and test data on the dashboard.

Legacy Zephyr Squad

Legacy Zephyr Squad screenshot

New Zephyr Squad

New Zephyr Squad screenshot

Can be migrated

Additional information

9 gadgets

Legacy Zephyr Squad has following gadgets in the Jira dashboard:

  • Test Distribution

  • Test Execution

  • Test Execution Assignment

  • Test Execution Burndown

  • Test Execution Details

  • Test Execution Distribution

  • Test Execution Time Tracking

  • Test Progression

  • Top Defects Impacting Testing

image-20220419-114655.png

9 gadgets

New Zephyr Squad has the following 9 gadgets in the Jira dashboard on Cloud:

  • Test execution burn down

  • Test execution completion over time

  • Test execution completion over time by status

  • Test execution effort (overall)

  • Test execution results (overall)

  • Test execution results (progress)

  • Test execution results by component

  • Test execution results by test cycle

  • Test execution results by tester

image-20220420-192542.png

N/A

Main differences: The Test execution details and Top Defects Impacting Testing gadgets have no equivalent in New Squad.

Jira Reports

Reports in Jira help teams analyze progress on a project, track issues, manage their time, and predict future performance.

Legacy Zephyr Squad

Legacy Zephyr Squad screenshot

New Zephyr Squad

New Zephyr Squad screenshot

Can be migrated

Additional information

3

Legacy Zephyr Squad has three different charts in the Jira reports section:

  • Test Execution Burndown Chart

  • Test Execution Chart

  • Top Defects Report

image-20220419-123228.png

They are present under Reports section and as Jira dashboard gadgets.

  • Test execution burn down

  • Test execution results (overall)

  • Test execution results (progress)

N/A

New Squad doesn’t use Jira Reports module. The reports present in the Reports section in legacy Squad are present under Reports section and as Jira dashboard gadgets in New Squad.

The Top Defects Report does not have an exact equivalent in New Squad.

Zephyr Test metrics page

Use the Zephyr test metrics page to perform reporting on your Test process.

Legacy Zephyr Squad

Legacy Zephyr Squad Screenshot

New Zephyr Squad

New Zephyr Squad Screenshot

Can be migrated

Additional information

5 reports

In the Test Metrics page, Legacy Zephyr Squad provides following charts:

  • Daily Test Execution Progress

  • Test Executions By Test Cycle

  • Test Executions By Tester

  • Executions By Date

  • Executions By Date

Test Metrics is a dedicated page added in the Legacy Zephyr Squad menu to have the reporting.

image-20220419-124841.png

The filter criteria available are fixVersions and time interval.

The selected filters apply to all the charts and any selection on the charts like drill-drown affects other charts as well.

Yes

New Zephyr Squad has 13 reports available in the report tab.

image-20220420-201055.png

12 are related to test executions and 1 about traceability

  • Test execution results (summary)

  • Test execution results (detailed)

  • Test execution results (list)

  • Test execution results by test cycle

  • Test execution results (progress)

  • Test execution results (overall)

  • Test execution results by tester

  • Test execution burn down

  • Test execution completion over time by status

  • Test execution completion over time

  • Test execution effort (overall)

  • Test execution results by component

  • Traceability Report

Yes

Main differences Test Metric page:

  • The Executions By Date page has no equivalent in New Squad.

  • The information are available in several reports in New Squad and in a single page in Legacy Squad.

Jira Agile Board

Link Test Cycles/Folders to a sprint

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

No

N/A

Doesn’t exist in New Squad as test cases are not Jira issues.

Test Execution view in Issue Backlog

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

No

N/A

Doesn’t exist in New Squad as test cases are not Jira issues.

Behaviour Driven Development

Integration with Cucumber/Gherkin

Support of Gherkin keywords to write down test script.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

Partially

New Zephyr Squad doesn’t support BDD feature file creation, so following keywords are not supported:

  • Feature

  • Scenario

  • Background

N/A

Advanced features (Gherkin validations, tag management)

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

Gherkin validations: Yes

Tag management: No

N/A

N/A

Automation

Supported automation frameworks

Support of Gherkin keywords to write down test script.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

  • Eggplant

    • via Zip file for Upload

    • via Results folder and .CSV file path for Zbot

  • JUnit (via JUnit xml)

  • Selenium (via Selenium XML)

  • TestNG (via TEstNG XML)

  • Tricentis Tosca (via Tosca XML)

  • UFT (via UFT XML)

  • Cucumber (via Cucumber JSON)

  • Robot (via JUnit XML)

  • PyTest (via JUnit XML)

  • JUnit (via JUnit)

  • Selenium (via JUnit)

  • TestNG (via JUnit)

  • Tricentis Tosca (via JUnit)

  • UFT (via JUnit)

  • Cucumber (via Cucumber JSON)

  • Robot (via JUnit)

  • PyTest (via JUnit)

N/A

New Squad relies on REST API for test automation.

Test automation integration UI

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

No

N/A

New Squad relies on REST API for test automation.

ZBot

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

No

N/A

N/A

Support on CI/CD tools

New Zephyr Squad's Bamboo integration enables you to automate tests outside Jira, run them in Bamboo during the CI/CD pipeline, and have results sent back to New Zephyr Squad for reporting and tracking.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes (via post build actions)

Yes (via post build actions)

N/A

N/A

REST API support

Expose data via a REST API which allows you to access the data programmatically and build your own integrations.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

Yes

N/A

The apis for Legacy Zephyr Squad and New Zephyr Squad are not similar although they cover the same functionalities. Customers must review them before using it.

SmartBear tools integration

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Integrates with following tools of SmartBear:

  • TestComplete

  • ReadyAPI

  • LoadNinja

  • CrossBrowserTesting

No

N/A

N/A

Configurations/Admin Operations

Reindexing

Test Case in Jira as Issue type called Test

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes, (Project, Versions, Components, Test Executions)

No need

No

Not needed in New Zephyr Squad.

Custom Status

Add custom status to customize the app according to the needs of your test process.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes (Test Steps and Test Executions)

Yes (Test cases, plans, cycles, executions)

Yes

N/A

Custom environment

Add custom environments to customize the app according to the needs of your test process.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Test case environment is Jira environment field

Yes

Yes

N/A

Custom labels

Add custom labels to customize the app according to the needs of your test process.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Test case label is Jira label field

Yes

Yes

N/A

Custom priorities

Add custom priorities to customize the app according to the needs of your test process.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Test case priority is Jira priority field.

Yes

Yes

N/A

Others

Team-managed (Next-Gen) project support

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

Yes

N/A

Issue Type configuration is needed in Legacy Zephyr Squad to use the team-managed project. However, it's not needed in the Zephyr Squad New Experience.

Internationalisation support (i18n)

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes:

  • English

  • German

  • French

  • Spanish

  • Russian

  • Korean

  • Japanese

  • Chinese

  • Turkish

Yes:

  • English

  • Portuguese

  • Russian

  • German

  • Italian

  • Spanish

N/A

N/A

GDPR complaint

Personal information transferred outside of the European Economic Area will be subject to the appropriate safeguards pursuant to GDPR and other applicable law, including without limitation, standard contractual clauses approved by the European Commission.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

Yes

N/A

N/A

Cloud Security participant

The Cloud Security Participant badge identifies cloud apps enrolled in the Marketplace Bug Bounty Program, and will evolve to include more rigorous testing and security assessments.

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Yes

Yes

N/A

N/A

Data storage

Legacy Zephyr Squad

New Zephyr Squad

Can be migrated

Additional information

Jira Servers - Test Cases, Test Cases custom fields, Test Cases attachments

SmartBear Servers - Test Steps, Test Executions, Test Step Results, Test Cycles/Folders, Test Execution History, Test Step History, Test Execution custom statuses, Test Steps custom statuses, Test Execution custom fields, Test Steps custom fields, Test Execution attachments, Test Steps attachments, Test Automation jobs

Smartbear servers

N/A

New Squad doesn’t affect Jira instance performance by not storing data on Jira servers.

Publication date: