General process

• The Standards and Testing Archivist (ArchivesSpace program team member) shares a list of Jira tickets ready for testing with the Testing Sub-Team Leads in the Testing Team Google Drive.

• Over the course of 2 weeks, testers follow the Testing Protocol below.

• The Testing Sub-Team Leads and Standards and Testing Archivist should hold at least 2 “drop-in” working sessions per round of testing so that there is dedicated time to work on testing and an easy place to ask questions.

Testing Website

You can use the hosted website at: test.archivesspace.org to test or you can use the Docker files to run your own instance on your computer. New builds are installed regularly, which may result in the test site being down from time to time; if issues persist, reach out to the Standards and Testing Archivist. The API should be accessible on the testing website; access at test.archivesspace.org/staff/api.

To use Docker files, see Running with Docker and note that you will need to point it at the latest version of the main branch, not the latest release version that comes by default with the Docker configuration packages. You can do this by adding a docker-compose.override file to the top level directory of your Docker directory (inside the archivesspace directory) containing the following:

For Mac users, OrbStack is recommended (it is much faster than Docker Desktop). To run Docker commands using Orbstack after downloading it, use the command docker context use orbstack in your terminal.

Testing Protocol

Due to the variety of issues to test, it is difficult to develop a testing protocol that covers everything.  However, you should be able to follow the guidelines below for most testing assignments:

• When you get your testing assignment, assign yourself as a watcher by clicking on “Start watching this issue” in the right corner(ish) of the JIRA page.

• Test the feature or bug

  • If you don’t know how to test the assigned issue, add a question/comment in JIRA to ask for assistance (you can also try the Testing subteam Slack channel for small questions)

NOTE:  Do NOT change the status to Tested and Accepted -> Ready for Inclusion in Release Candidate.  The Testing Team will not be the ones who make the final determination as to whether an issue is accepted or rejected.  Instead add your opinion to the comments in JIRA. See this video for more guidance.

• If the feature or bug fix worked as expected, add a comment to the issue in JIRA indicating that it worked. If you’re not sure it worked, add a comment describing what happened when you performed the test.

• If multiple people are assigned the same issue to test, all assignees should comment in JIRA about their experience, even if it’s just to say: Ditto.

• If the feature or bug fix did not work as expected, add a comment to the issue in JIRA indicating that it didn’t work. It’s also a good idea to include the steps you took, the results you received and the result you expected.  Feel free to attach screenshots and/or additional documentation if needed to clarify your results.

Regression testing process

Overview

• The Standards and Testing Archivist (ArchivesSpace program team member) shares a Google Sheet with the full testing protocol with the Testing Sub-Team Leads, in the Testing Team Google Drive.

• The Leads fill out the testing assignment tab, assigning at least two testers to each tab of the spreadsheet/area of the application (see Specifics below), and also create a “Screenshots” folder and “Files” folder in the relevant folder on the Testing Team Google Drive for documenting issues during regression testing.

• Over the course of 3 weeks, testers follow the Testing Protocol detailed in the Testing Protocol spreadsheet for the specific version being tested.

• The Testing Leads and Standards and Testing Archivist should hold at least 2 “drop-in” working sessions per round of testing so that there is dedicated time to work on testing and an easy place to ask questions.

Specifics

• Each tab of the testing protocol represents a functional area of ArchivesSpace, such as Agents, Built-in Reports, and Spreadsheet importers. Of the two testers assigned to each, one should have high familiarity with that functional area and one should have low familiarity (if possible given the current team’s makeup).

• Spreadsheets will have Conditional Statements (WHEN I do x, THEN y happens) to assist testers who are unfamiliar with their assigned functional area(s).

• If issues unrelated to the functional area a team member is testing arise:

  • The team member should first Slack message or email the rest of the testing team to see if others are experiencing the same issues/have already recorded it in their own testing

  • If not, they should submit a new Jira ticket for the issue encountered

• Screenshots are extremely helpful when documenting problems. If a screenshot it taken to illustrate an issue, the team member will:

  • Name the screenshot with their last name, spreadsheet tab, and a numerical indicator:

    • ex. Paquette_Subjects_1.png

  • Place the screenshot in the Screenshots folder for the round of regression testing

  • In the “Notes” section of the team member’s Testing Protocol spreadsheet, write an explanation of the issue and have the text link to the screenshot in the Screenshots folder. Be sure to note “See screenshot” at the end of the note.