General process

• Christine Di Bella (ArchivesSpace Senior Program Manager) shares a list of Jira tickets ready for testing to the Testing Sub-Team Lead.

• The Lead makes a copy of the Jira ticket list, puts it in the appropriate folder in the Testing Team Google Drive assigns at least 2 testers from the sub-team to each ticket.

• Over the course of 2 weeks, testers follow the Testing Protocol below.

• The Testing Sub-Team Lead and Sub-Lead should hold at least 2 “drop-in” working sessions per round of testing so that there is dedicated time to work on testing and an easy place to ask questions.

Testing Website

You can use the hosted website at: http://test.archivesspace.org/ to test or you can download and install your own copy.  New builds will be installed nightly for quicker testing.  (In the past, if an issue was rejected there usually wasn’t time to fix it in the sprint so it had to be moved to the next sprint).

Testing Protocol

Due to the variety of issues to test, it is difficult to develop a testing protocol that covers everything.  However, you should be able to follow the guidelines below for most testing assignments:

• When you get your testing assignment, assign yourself as a watcher by clicking on “Start watching this issue” in the right corner(ish) of the JIRA page.

• Test the feature or bug

  • If you don’t know how to test the assigned issue, add a question/comment in JIRA to ask for assistance.

NOTE:  Do NOT click on the Accept or Reject buttons.  The Testing Team will not be the ones who make the final determination as to whether an issue is accepted or rejected.  Instead add your opinion to the comments in JIRA.

• If the feature or bug fix worked as expected, add a comment to the issue in JIRA indicating that it worked.

  • If you’re not sure it worked, add a comment describing what happened when you performed the test.  A follow up group (Q/A group) will decide it the story is accepted or rejected.

• If multiple people are assigned the same issue to test, all assignees should comment in JIRA about their experience, even if it’s just to say: Ditto.

• If the feature or bug fix did not work as expected, add a comment to the issue in JIRA indicating that it didn’t work. It’s also a good idea to include the steps you took, the results you received and the result you expected.  Feel free to attach screenshots and/or additional documentation if needed to clarify your results.

• New builds of the test site will be loaded nightly so you should be able to re-test problem issues the next day or so

Regression testing process

Overview

• Christine Di Bella (ArchivesSpace Senior Program Manager) shares a Google Sheet with the full testing protocol with the Testing Sub-Team Lead.

• The Lead creates a separate spreadsheet assigning at least two testers to each tab of the spreadsheet, and also created a “Screenshots” folder for documenting issues during the regression testing.

• Over the course of 3 weeks, testers follow the Testing Protocol detailed in the Testing Protocol spreadsheet for the specific version being tested.

• The Testing Sub-Team Lead and Sub-Lead should hold at least 2 “drop-in” working sessions per round of testing so that there is dedicated time to work on testing and an easy place to ask questions.

Specifics

• Each tab of the testing protocol represents a functional area of ArchivesSpace, such as Agents, Built-in Reports, and Spreadsheet importers. Of the two testers assigned to each, one should have high familiarity with that functional area and one should have low familiarity (if possible given the current team’s makeup).

• Spreadsheets will have Conditional Statements (WHEN I do x, THEN y happens) to assist testers who are unfamiliar with their assigned functional area(s).

• If issues unrelated to the functional area a team member is testing arise:

  • The team member should first Slack message or email the rest of the testing team to see if others are experiencing the same issues/have already recorded it in their own testing

  • If not, they should submit a new Jira ticket for the issue encountered

• Screenshots are extremely helpful when documenting problems. If a screenshot it taken to illustrate an issue, the team member will:

  • Name the screenshot with their last name, spreadsheet tab, and a numerical indicator:

    • ex. Paquette_Subjects_1.png

  • Place the screenshot in the Screenshots folder for the round of regression testing

  • In the “Notes” section of the team member’s Testing Protocol spreadsheet, write an explanation of the issue and have the text link to the screenshot in the Screenshots folder. Be sure to note “See screenshot” at the end of the note.