ArchivesSpace E2E Test Suite Documentation

Introduction

The E2E (End-to-End) Test Suite for ArchivesSpace is designed to automate and streamline the testing of key user workflows within the application. By using Cucumber in combination with Gherkin syntax, this test suite allows both technical and non-technical users to create, understand, and maintain test scenarios easily. Cucumber uses Gherkin, a human-readable language, to write tests in a way that is easy to follow, making it an ideal choice for teams that want to integrate testing with a collaborative, community-driven approach.

Purpose of the E2E Test Suite

The primary purpose of the E2E test suite is to ensure that critical functionality in the ArchivesSpace application works as expected across different versions and updates. These automated tests simulate real-world user interactions to verify that the application performs as intended.

Why Use Cucumber and Gherkin?

The Cucumber framework, with its Gherkin syntax, is well-suited for collaborative, behavior-driven development (BDD). The Gherkin language allows writing test scenarios in a structured, plain-text format that focuses on behavior and expected outcomes, making it easy to write tests that are understandable by everyone involved in the development process, including:

QA Engineers – Writing test scenarios and understanding results.
Developers – Implement the corresponding step definitions and ensure that functionality aligns with the business requirements.
Product Managers/Stakeholders – Reading and validating test scenarios to ensure the application works as intended from a user's perspective.

By using this approach, even contributors with minimal technical experience can understand and contribute to the testing effort, which fosters community involvement and helps maintain robust testing coverage across the project.

Key Features of the E2E Test Suite:

Human-Readable Scenarios: Test scenarios are written in Gherkin, which allows contributors to write and understand tests with minimal technical knowledge. The format focuses on what the system does in specific situations, making it accessible to non-developers.
Automated Testing: The suite is designed to run tests automatically, simulating user behavior and ensuring that new changes don't break existing functionality.
Seamless Integration into Development Workflow: The E2E tests integrate with CI/CD pipelines, ensuring that tests are run whenever code is committed or merged, and providing immediate feedback on the health of the application.
Comprehensive Test Coverage: The test suite covers critical user workflows, such as logging in, searching, creating, editing, and deleting records, ensuring that the application remains stable and functional after every update.

How the Test Suite Works

Gherkin Syntax: Contributors write Gherkin feature files to define test scenarios using a simple, natural language format with the following key components:
- Feature: A high-level description of a feature being tested.
- Scenario: A specific use case or behavior being tested.
- Given: A description of the starting state or precondition.
- When: The action or event that triggers the behavior being tested.
- Then: The expected outcome or result after the action has been performed.
Cucumber Step Definitions: Each Gherkin step (Given, When, Then) corresponds to a step definition written in Ruby. These step definitions contain the code that performs the actions described in the Gherkin scenarios, interacting with the application to carry out the specified behavior.
Automated Test Execution: Once the tests are written, they run automatically using the Cucumber framework. The tests simulate a user’s journey through the application, ensuring that the expected behaviors occur.
Reporting and Results: After running the tests, Cucumber provides detailed output on test results, including pass/fail statuses, logs, and error messages that help developers and testers identify and resolve issues.

Prerequisites

Before contributing to the E2E test suite, you’ll need:

Basic understanding of Git and how to create a Pull Request (PR)
A working knowledge of Gherkin syntax for writing feature files
Access to the ArchivesSpace repository and testing environment

Optional:

A basic understanding of Cucumber framework to interpret the tests

Setting Up the E2E Test Suite

Install Ruby

The currently required Ruby version is in the root folder inside the .ruby-version file.

It is strongly recommended to use a Ruby version manager to be able to switch to any version. The most popular version manager available for Linux and macOS, is rbenv.

You can find the installation guide here https://github.com/rbenv/rbenv#readme .

If you are on Windows, a separate rbenv installer exists here: https://github.com/RubyMetric/rbenv-for-windows#readme .

If you wish to use a different manager or installation method, you can choose one of the following: https://www.ruby-lang.org/en/documentation/installation/

Download and install the E2E test suite

Clone the Repository: If you haven’t already, clone the main archivesspace repository containing the E2E tests:

git clone https://github.com/archivesspace/archivesspace.git

Navigate to the e2e-tests folder, and run:

rbenv install
gem install bundler
bundle install

Run the tests

Run on the remote host

To run the tests on the https://e2e.archivesspace.org, run:

bundle exec cucumber

Run on localhost

Set up the application with:

docker compose -f docker-compose.yml up

Wait until everything is up and running. You can check if the staff interface is running on http://localhost:8080.

Then, to run the tests, open another terminal and run:

bundle exec cucumber HOST=localhost staff_features/

Linters

To ensure compliance with our syntax guidelines, run:

bundle exec rubocop

Viewing Test Results

After running the tests, the results will be displayed in the terminal. You can also configure Cucumber to generate more detailed reports (e.g., HTML or JSON) using additional flags.

Understanding Cucumber and Gherkin

The tests are written using Cucumber, which uses Gherkin syntax to describe the test scenarios. Gherkin is a business-readable language that allows writing tests in a structured format with the following keywords:

Feature: A feature is a high-level description of the functionality you're testing.
Scenario: A scenario describes a particular use case or behavior of the application that you're testing.
Given: Sets up the initial state of the system.
When: Describes the action or event that triggers the behavior.
Then: Defines the expected outcome after the action is performed.
And: These are used to add additional conditions or expectations.

Example Gherkin scenario:

Feature: Accession Create
Scenario: Accession is created

Given an administrator user is logged in
And the user is on the New Accession page
When the user fills in 'Identifier'
And the user clicks on 'Save'
Then the 'Accession' created message is displayed
And the Accession is created

In this example:

The Feature is Accession create.
The Scenario describes a successful creation of an accession.
Given, When, and Then describe the actions and expectations.

Contributing to the Test Suite

To contribute to the E2E test suite, follow these steps:

Fork the Repository: Fork the archivesspace/e2e-tests repository to your GitHub account.
Create a New Branch: Create a new branch from master for your changes:
git checkout -b accession_create
Write Your Gherkin Feature File: Gherkin feature files are located in the ./staff_features directory. Create a new .feature file or update an existing one.
Create Step Definitions: Each Gherkin step (Given, When, Then) must have a corresponding step definition in Ruby. These definitions are located in the step_definitions directory.
Example of Step Definition for a Given step:
# frozen_string_literal: true Given 'the user is on the New Accession page' do visit "#{STAFF_URL}/accessions/new" end
Run the Tests Locally: Before pushing your changes, make sure the test suite passes locally:
bundle exec cucumber HOST=localhost staff_features/accessions/accession_create.rb
Commit Your Changes: Once you’ve added the new feature or step definition, commit your changes with a descriptive message:
git add . git commit -m "Accession create"
Push Changes and Create a Pull Request: Push your changes to your forked repository and create a pull request (PR) to the main ArchivesSpace repository.
git push -u origin accession_create
In the PR description, explain the changes made, such as the new feature or scenario added.

Maintenance Plan

To ensure the E2E test suite stays relevant and effective, we have the following maintenance plan:

Test Updates:
- Tests will be updated whenever there is a new feature or bug fix in the ArchivesSpace application.
- A designated Testing sub-team will review and update test scenarios regularly to align with new features or changes to the application.
Review Process:
- All PRs will be reviewed by a member of the Testing sub-team or a senior developer to ensure they follow the Gherkin syntax and that the step definitions are correctly implemented.
Monitoring and Troubleshooting:
- Regular monitoring will ensure the tests are running smoothly and any flaky tests are addressed.
- If issues arise with specific tests, contributors can troubleshoot by reviewing the step definitions or checking for recent changes in the application that might affect test scenarios.

Troubleshooting

If you encounter issues while running the tests, here are some common troubleshooting steps:

Ensure Dependencies Are Installed Correctly: Run the following command to ensure all dependencies are up-to-date:
bundle install
Review Cucumber Output: Cucumber provides detailed error messages in the console when tests fail. Look for any issues related to step definitions or incorrect Gherkin syntax.
Check Step Definitions: Ensure that every Given, When, and Then step in your Gherkin scenario has a corresponding step definition in the step_definitions directory.
Clear Cache and Reset Test Data: In some cases, clearing the test environment or resetting test data may resolve issues with tests.
Check Application Changes: Review recent updates to the ArchivesSpace application to ensure that the test steps are still relevant and correct.

Presentation

Tutorial videos

Part 1 - Overview

Part 2 - BDD Gherkin and Git

Part 3 - Workshop

How to contribute End to End test scenarios