2015-12-11 Meeting notes

Discuss responsibility of migration sub-team for AT/Archon migration tools

1. On the list questions have arisen about Archon migrations, but with Nathan Stevens no longer on the project no developers are officially tasked with updating the migration tools--this responsibility has passed to this group.
   
   
2. What does this mean?

Brad had placed statement "In 2015/16, the migration sub-team will also become responsible for supporting migration of Archon and Archivists' Toolkit data into ArchivesSpace.  This work will involve assisting organizations that are migrating their AT or Archon data and indicating to ArchivesSpace developers how migration scripts might need to be revised" in our charge.

Chris Fitzpatrick had also stated on the listserv that the maintenance of the tools was now being passed to the community. This had also been stated at the Hack-a-Thon recently. Noah stated that he believed this to be the case.

For this group, this might mean:

• Respond to people on the user list when questions arise about the migration tools. Phil and Terry have already done a great deal about this with individuals, doing test migrations for other repositories, etc. Need to be clear with people that this is volunteer service, that sub-team members are not employees of the project and are just helping out.
• If there are actual bugs in the scripts themselves, members of team could submit pull requests for fixes. Brian reported that no one but Nathan had worked on those reports. Terry Reese had reported that he had been working on tweaking the migration tool, but this is still a work in progress and not published. Meant to clean up the text at the end of the biographical/historical note. Looking at the AT Migration tool, there are currently 0 contributors. This version was uploaded a month ago and is made for 1.4. The new migration path for institutions that have not migrated yet is to make the migration into 1.4.2 and then update from there. If this is the new migration path, it was suggested that this should be noted in the documentation--Noah will look into this and update as needed.

Most listserv traffic has been about the Archon migration tool. Brian and others suggested that most of the problems in this regard are not caused by the tool itself, but by modeling differences between AT and Archon--may not require bug fixing as much as massaging/reinterpreting data. Matt G. suggested that some work could still be done to improve migration process (e.g., fields added by the tool that weren't in the original data), though. It may be that this group could be used to mediate feature requests, which could then be passed along to developers, Terry or others for bug fixes.

How would this differ from "official" migration guidelines at: http://www.archivesspace.org/migrations

a. Need to review documentation for users on preparing data for migration, migration process, correcting errors found in process
i. Currently documents of this type are at archivesspace.org/migrations, but having information available in Github would make them more easily maintainable
ii. Should at a minimum be linked to from the Github README
b. Phil had been talking with Brad about developing FAQ to address common problems encountered during migration
i. Having these linked from where the tools are downloaded would be useful
ii. Common problems might include specifying where the JAR file should be located, creating the MySQL database, etc.
iii. Noah suggested that within the Github README page for each migration tool we include an FAQ section--would need to get permissions to edit the page
iv. Phil will review the guidelines for Archon and try to boil them down to a list of FAQs. Documentation might need to be reviewed anyhow to make sure that it is current.
v. May need to consult with the Documentation Group to see if they have a larger plan for providing this information.

Work Plan - 2015-2016

1. No updates from Terry for pre-processing EAD tasks. Has been working on setting up the Github repositories for this work. Two areas of work include mapping and establishing rules for processing. Once these rules are articulated they can be posted and then referred to from Schematron and other conversion stylesheets. The main problem is determining what the requirements are for ArchivesSpace and how it interprets EAD. A lot of this is implicit or overly detailed in the documentation--needs to be looked at again and fleshed out. No one currently knows all the conditions for mapping between EAD and ArchivesSpace. Terry suggested developing a document in the repository defining what the effective subset of EAD in ArchivesSpace is. Noah asked what tasks would be helpful to collaborate on, included gathering data on import problems and error messages and then aggregating and interpreting these messages. This should then suggest the rules behind the messages. Aiming to have this work started by the next call.
2. Noah reported that JIRA cleanup may be more complicated than expected, and will take more time. Also noted that he has been looking through a list of migration problems identified at the recent Hack-a-Thon, and has put at least one of these into the system.
3. Michael reported on his work with the import/export mappings, starting with a spreadsheet of problems developed at Harvard last year. He will continue to gather comments, and to review other documentation. Noah mentioned that he and Phil have been talking about testing the MARCXML import/export, and will be building a spreadsheet similar to that created at Harvard.
4. Cory reported that he has continued working with Brad Westbrook on the EAC-CPF modeling--this hasn't really focused specifically on EAD3, only where the two standards intersect in areas such as date entries. Noah suggested that Brad and Cory might present on this work at a future meeting.

Draft of migration / data cleanup guidelines for container management (Maureen and Patrick G.): https://docs.google.com/document/d/1aIKCpExEaIvc_miEX8Eick1Ix2dsS9ayDQz9YLsSnoI/edit?usp=sharing

Work on the container management plugin continues, but Maureen and Patrick G. have been working on documentation for cleanup needed for implementation. They are hoping to have a new draft ready for next week. This document will be very useful for planning for migration, but no one on the call had implemented yet and had experience with cleanup. It was asked if there was documentation available for how data is moved within the database during the migration process, but Brian stated that he was not aware of anything in Github.

Noah was interrupted by a fire drill at 11:48 AM MST, which brought the call to a premature end.