ArchivesSpace re-org and committer groups proposal

Owned by Mark Cooper
Last updated: Nov 07, 2016 by Sally Vermaaten (Unlicensed)
14 min read

Sally = purple

Max = orange

Dave = Blue

Green = group call notes

Brown = Esmé

Re-org:

https://docs.google.com/spreadsheets/d/1S5C217M3JUZfHSlLarQjAUqCofJ9jMi_C2VIGvyT8KA/edit?usp=sharing

Github structure: 

Propose creating / maintaining these organizations: 

Most documentation should be in confluence wiki & links in Github readmes.

https://github.com/Islandora/islandora/wiki/Islandora-Committers


___________________________________________________________________________________________________________________________________________________

https://github.com/archivesspace/ - discussed 10/24/2016 & 11/7/2016

Purpose

Contains the core ArchivesSpace repositories, including the main project and migration tools. All code here is officially supported and maintained by at least the ArchivesSpace Program Team.

Core Committers Group - text adapted from Islandora https://github.com/Islandora/islandora/wiki/Islandora-Committers

ArchivesSpace is open source and released under ECL, v 2.0. The software and associated documentation is developed collectively by a community of contributors and committers. All interested community members are encouraged to contribute to the project. Contributors who demonstrate sustained engagement with the project through quality participation in meetings, mailing lists, documentation and code updates can be nominated by existing committers to also become a committers. It should be emphasized that committers need not be limited to software developers. Community members with skills in documentation and testing, for example, can also be committers. 

Rights

Committers share the following rights:

  • Write access to the codebase
  • Privileges for nominating and voting on new committers 
  • Release management privileges

Responsibilities

Committers share the following responsibilities:

  • Monitor and respond to project mailing lists
  • Attend committer's group meetings
  • Monitor and vet pull request and bug fixes on ASpace JIRA
  • Review and commit code contributions
  • Ensure code contributions are properly licensed
  • Welcome new contributors and help people with the process of contributing 
  • Stay active in the community

Committers are encouraged to speak with their organizations to clarify how contributions to the ArchivesSpace community fits into their role and what amount of time can be dedicated to contributing to the ArchivesSpace community.

Committers 

The following is an alphabetized list of the current ArchivesSpace committers:

NameOrganization

















A full list of all known community and emeritus contributors can be found here.

Committers Calls

ArchivesSpace Committers Calls are monthly meetings of the developers of ArchivesSpace, who discuss modifications and innovations in the code, and review new and open tickets. Calls are open to interested

Process for Adding a New Committer Process

This section describes the process for handling the voting of a new committer. Voting is handled by emailing the Committer List.

Summary: 

    1. Any existing committer can call a vote (insert link to template here - similar to templates/committerVote.txt)  – sounds good
    2. Close a vote (insert link to template here - similar to templates/closeCommitterVote.txt)  <---- need half of people voting +1 and no -1 ones
    3. If the vote passes with no vetoes, the new committer is invited to join (insert link to template here - similar to templates/committerInvite.txt)

If they accept, then do:

    1. Add to Committer team of GitHub archivesspace organization
    2. Add to Committer team of GitHub archivesspace-labs organization
    3. Add to archivesspace-committers google-group, group used nearly exclusively for this voting process
    4. Add to committers wiki page: ArchivesSpace Committers
    5. Announce the new committer (insert link to template here - similar to template/committerAnnounce.txt) - Users group & Google Group


Guidelines for assessing new candidates for committership

When a contributor is nominated to become a committer, the following guidelines should be used by existing committers to evaluate the nominee's suitability:

  • Ability to work cooperatively with peers - How do we evaluate? By the interactions they have through mail. By how they respond to criticism. By how they participate in decision-making process.
  • Ability to be a mentor - How do we evaluate? By the interactions they have through mail. By how clear they are and how willing they are to point at appropriate background materials (or even create them).
  • Community - How do we evaluate? By the interactions they have through mail. Do they help to answer questions raised on the mailing list; do they show a helpful attitude and respect for other's ideas.
  • Commitment - How do we evaluate? By time, by sticking through tough issues, by helping on not-so-fun tasks as well. 
    • Do committers have an ongoing committment of time? ← may not want to impose barriers but if they're not active they're not contributing
  • Personal skill/ability - How do we evaluate? A solid general understanding of the project. Quality of discussion in mail. Patches (where applicable) easy to apply with only a cursory review. All contributors should have submitted at least one pull request to the project.

-------------

Code Contribution Guidelines - from Contributing to ArchivesSpace 

Note - have excerpted text here but just may want to keep this as a separate page as is - have included here so we can review & make final decisions - e.g. the travis / jenkins issue?

  1. Make sure that you have completed an Individual Contributor License Agreement, or that your institution has completed an Corporate Contributor License Agreement.

  2. Find or create a bug report or feature request ticket in JIRA GitHub (to make sure you’re not duplicating work and document the intent of your contribution). It helps to explain both the existing behavior and the desired behavior that your change will implement.

  3. Set up a GitHub account if you have not done so before.

  4. Fork the ArchivesSpace repository on GitHub.

  5. Create a feature branch.

  6. Make changes in your fork. We advise contributors to follow these guidelines to expedite the contribution review process.

    1. Follow established style guidelines

      1. Rails: https://github.com/bbatsov/rails-style-guide

      2. JRuby: https://github.com/jruby/jruby/wiki/JRubyStyleGuide

      3. RSpec: http://betterspecs.org/

    2. Include unit tests sufficient to cover the feature(s) you add or bug(s) you fix, and make sure the test suite passes. (Can we rely upon Travis or a Jenkins instance for this, or should we suggest that contributors run tests locally?) → would be good to run travis locally

    3. If you’re adding a feature or otherwise changing documented behavior, modify the documentation to reflect your changes.

  7. Once your work is done, squash the commits in your branch — see One Commit per Pull Request for some guidelines — and rebase it on the latest in the upstream master branch. We appreciate succinct but explanatory commit messages.

  8. Push your updated branch to your fork.

  9. Create a Pull Request on GitHub.

  10. Respond to feedback as the community reviews your contribution.




additional pointers from Dspace code contribution guidelines we could incorporate → group thought these looked like good additions


  • The code is stable and has no stability or security concerns
  • The code is properly using existing APIs, etc.
  • The code is not too specific to one institution's local policies or workflows. (I.e. we will review the code to ensure it looks to be generally useful to most institutions, or configurable enough such that others can change it to match their own local policies/workflows)
  • Any third-party tools/libraries used by your code have compatible open source licenses. See Licensing of Contributions

There is some work on these kinds of contributing guidelines going on in the Hydra community right now, here are some updated docs that might have some good stuff to include here:

---------------

Code approval process - how to get your code into ArchivesSpace - for non-committers  (adapted from https://wiki.duraspace.org/display/DSPACE/Code+Contribution+Guidelines)

Encourage smaller contributions and smaller pull requests in sequence when possible. 

Before you start, make sure you're talking to people – use email or Github issues? 

  • if you have a bigger, open-ended question then list
  • more focused / smaller question, then Github issue

may need to use multiple channels anyways

ACTION - merge this kind of doc w/ the Core Contribution Guidlines


0. Share Early, Share Often!

The overriding mantra is share early, share often. Here are a few things to consider before you begin working on your code:

  • Please be sure to share your plans with the ArchivesSpace community on the (ArchivesSpace users?) list (or via one of the monthly Developer Meetings) before embarking on any sizable development effort. This will ensure you achieve your goals in a way that is consistent with the ArchivesSpace architecture and plans of the rest of the community. It will minimize the chances of a scenario where you have invested a large amount of time and effort into a body of code that does not fit in with the ArchivesSpace architecture or the consensus of the community. (Note - do we need some kind of distinction between large & and small contributions? Larger contributions should be evaluated by the User Community - perhaps via a process similar to this: http://islandora.ca/developers/lsap)
  • Develop incrementally; try and implement and contribute a basic form of your feature as soon as possible, rather than aiming to implement a complete and 'polished' solution. This will help ensure you're on the right track with regards to the rest of the ArchivesSpace community and platform. The sooner your code is part of the core code base, the less time you will have to spend 'chasing' the main code base, i.e. keeping your changes up-to-date with that core code base.
  • Obtain the ArchivesSpace code using GitHub (see also Development with Git). This will make code management much easier. It's very simple to do; see Developer Guidelines and Tools.
  • Read Code Contribution Guidelines (this page) to ensure you are following ArchivesSpace conventions. This will ensure your code is more likely to be immediately accepted as part of out-of-the-box ArchivesSpace.
  • Ensure that any third-party tools/libraries that you plan to utilize are released under compatible open source licenses. See the Licensing of Contributions section below.
  • For Larger Initiatives/Codebases: If you are building out a much larger project, we highly recommend notifying the community of the work early on via an email to both Google Group and Member List. This can help find collaborators or get early feedback. We also recommend you develop your project in GitHub, as it provides easier ways to review/collaborate with other developers.

1. Make your code available (preferrably in GitHub) and create a ticket in our Issue Tracker 

Once your code is ready, you must make your code available to the ArchivesSpace Committers Group for review. The easiest way for us to review your code is by putting your code into GitHub (just create your own account – it's free!). Then, submit a "Pull Request" to our GitHub repository (see Development with Git). Alternatively, if you are not yet comfortable with GitHub, you may create a patch (and upload it to our DSpace Issue Tracker). However, please be aware that submitting a patch may delay the review process (see below note)

In either case, you must also create a new ticket in our ArchivesSpace JIRA Issue Tracker. This ensures that the ArchivesSpace Developers are notified of your contribution, and acts as a place for us to comment on the work or make suggestions for improvements. (is this the right process for us?)

2. Code Review Process

Once the code is made available, the Committers Group will take time to review the work and provide feedback/comments. Code must be reviewed by at least one other committer from another organization before it can be merged. 

Usually, one (or more) committers who are interested in this work will contact you and discuss any feedback we have, and whether or not there would need to be some general changes before we could accept it. Some patches/features are readily accepted (because they are stable and look good), others may require more work (if there are concerns or issues that Committers notice).

The timeframe of a code review will vary, based on how much time the Committers have. Smaller changes may be reviewed within days, while larger changes/features may take many weeks to do a full review. All Committers are volunteers and only have a small amount of time to provide to the project in a given week. We will make every effort to get back to you with feedback within a few weeks. But, if you haven't heard anything, feel free to ask! (is this the right process for us?)

We'll also want to define 'rights and responsibilities' of core committers that specify good practice such as not merging your own code, etc. <--- Fedora has a good  'rights and responsibilities' doc that we can probably use as a model. (this one: https://fedoraproject.org/wiki/Package_maintainer_responsibilities??)

3. Reworking Code (if necessary) & Next Steps

After the code review & feedback, interested Committers may help you to rework the code (if needed). They'll also provide you with next steps on getting the code into ArchivesSpace. If it can be accepted immediately, it will be. If not, we'll try to help figure out the best route forward.

How you can help speed up the process

As our Committers are all volunteers, they don't always have the time to rework code changes for you. If you want your code change accepted in a timely manner, please offer to make the changes yourself (otherwise your patch suggestion may wait in a "holding queue" until someone has enough time to work on any necessary fixes).

If you are unsure of next steps, please let us know by adding a comment to your issue in the Issue Tracker. Communication is absolutely necessary to ensure that we can help you rework anything that needs reworking. If we don't hear from you, we'll assume you are hard at work. So, if you've run into issues, please let us know! If, locally, you don't have the time or expertise to do the rework that is necessary, also let us know. We can try to locate a community developer to help out, and/or ask both the Committers Team and the ArchivesSpace Community Advisory Team if they know of any interested developers with time to spare.

4. Acceptance!

Once your code is accepted, it will be released in the next version of DSpace software! It is time to celebrate, as your name will be added to the prestigious list of DSpace Contributors!


----------------------

General discussion notes:

Oversight for the core committers comes from the Program Team developer(s) and other established core committers ... Islandora - has text about this; but doesn't start from scratch 

  • For starters - get a starter group 5 -6 people
  • Page with these are the committers
  • Github - links to email addresses, etc.
  • Rights 
  • What we need: 
    • Describe core committer nominations process 
    • Define rights & responsibilities
      • include guideline that you get your employer's blessing?
    • In Github page describe who people are w/ links to who core committers are & what their communication process is - e.g. development meetings (e.g. biweekly or monthly) - Hydra & Fedora have weekly - for ?s, work underway - e.g. release; i want to make this big change, should I wait or is that cool?; also a good place to just get updates on technical progress of project (e.g. this is a useful-looking page: http://islandora.org/developers)


Sally's thoughts on structuring this documentation from looking at other OSS community examples:


___________________________________________________________________________________________________________________________________________________

https://github.com/archivesspace­plugins/ 

Purpose: contains plugins officially supported by at least the ArchivesSpace Program Team.
These plugins are tested and updated as necessary to be compatible with new releases.
Provides a focal point to discover plugins and contribute to their development. 

Committer Policy: 

  • Linking to same guideline but noting plugins can be difficult to test so that requirement could be lowered
  • ACTION - Approval process to be considered 'official plugin' 
  • fedora doesn't have a formal process - instead there must be 2 people willing to support - if one committer leaves have to find 2nd or it's deprecated; ASpace is different in that it's a program responsibilitity. 
    • could instead use principles of: generic applicability; plugin quality (against coding guidelines)
    • approvers - core committer group + program team members; to assess applicability - putting out message on list (community voting- how many people would use)


Maintained by: Program Team, Core Committers. (plugin committers?) <--- believe we decided on 8/24 not to have a separate plugin committers group




___________________________________________________________________________________________________________________________________________________

https://github.com/archivesspace­labs/ - discussed 10/24/2016

I wrote something up here: https://docs.google.com/document/d/1a_4AEgXHHZnbKWwjRcEVw8FWdme1cldKWvII9cLRYxI/edit?usp=sharing

Purpose: contains proof of concepts, examples and experimental projects. There is no official
support from the Program Team for anything in labs. The projects are not guaranteed to be up
to date with the latest release and are used on a at­your­own­risk basis. Program Team and
other “official” committers can contribute to labs but that does not confer official support to any
project. Note - Labs branding should make it clear that none of the projects are officially supported. 


Committer Policy: 

  • This area is open to any contributions and there are no minimum requirements for adding scripts, etc. 
  • Adherence with y coding guidelines and inclusion of some documentation is encouraged but not required. 

(Core committers? Committers Oversight?) will conduct a early review of archivesspacelabs and will move clearly abandoned or obsolete code to deprecated.

Maintained by: Community Committers.

  • group within larger Community Committers group that could provide some kind of oversight / determination re. what's deprecated - Patrick & Max happy to help with this (not sure if it's intergrations sub-team long term)

As far as I know, there's currently no automated way to say "this fork is a reference, keep this fork up to date with its parent." Thus, anything forked in to this org will need more or less constant maintenance, or some sort of scripted update mechanism, in order to maintain currency.  Given that this seems primarily intended as a linking mechanism to externally provided code, would it maybe make sense to have the primary entry point into this be a Github Pages page, with a link to each internal/external tool and other explanatory content.  I think the barrier to entry won't be any higher than with a collection of forked repos, and the maintenance burden would be lower.

As per the notes from last time, since we're trying to encourage "as much collaboration, sharing and committing as possible," it might be nice to lower barriers if we had some resources readily available from this space. I'm not suggesting we create any, just point to existing guidelines/tutorials on GitHub lingo, getting things in there, best practices for READMEs (even if we aren't requiring any documentation), etc., who to ask for help, etc.

A good example to follow here is the IIIF Awesome repository.  It contains only a README with links to IIIF examples and resources, and contributing guidelines.  So instead of having a whole GitHub organization, this would be a repo (in the main archivesspace org?).

Group liked the IIIF Awesome repository and what Max has developed as a model.

  • How do we handle 'internal' program staff contributions e.g. container checker?
  • Not really helping the making 'it easier to share' / fork projects - does help with discovery
    • Labs repo still good idea 
    • Approval may be the distinguishing factor? Wiki page v. GitHub 
  • IIIF-awesome copyright statement - applies only to wiki page - let's leave it off and not worry about it 
    • contributions to labs will require some kind of license - if there's a repository for people to add scripts, maybe there could be a copyright statement

DECISIONS: 

  • Hybrid approach - Some contribution of projects into Labs + IIIF-awesome type page
    • updating page - anyone with commit privileges can be on the list to review pull requests on that page
    • anyone can be added to org 
  • Cool ArchivesSpace projects 
  • those wanting to contribute scripts could just add to list - email contact if someone's totally lost

Github Organization Blurb: Experimental, example, and proof-of-concept projects.  Feel free to start new collaborative projects here, or add a link to external projects to the Cool ArchiveSpace Projects README.  Projects are not guaranteed to be compatible with the latest release, or officially supported by the ArchivesSpace Program Team (though Program Team members may contribute).

ACTIONS

  •  Esmé Cowles (Unlicensed) - Text describing what the organization is for - e.g. on text box at top - couple of sentences 



___________________________________________________________________________________________________________________________________________________

https://github.com/archivesspace­deprecated/


Purpose: contains repositories that should no longer be used with ArchivesSpace or that have
become generally obsolete.

Maintained by: Program Team, Core Committers.



Community Committers: Any developer wanting to contribute a project to labs or work within it.
Access to labs (write access).There are no minimum requirements for adding resources to labs. Adherence with y coding guidelines and inclusion of some documentation is encouraged but not required. 

Need to decide how labs will work. Will it be forking external projects or encouraging developers 
to work within it directly and transfer existing projects to it? Former is more hands on and will 
require keeping the forks up to date, latter may be harder to get buy­in (devs can also do both 
quite easily by adding labs as an additional remote ­­ should probably encourage that for devs 
who want to maintain control, but can also push to labs). 


About the forks: would we have to keep forks up-to-date? Could that be part of the yearly review to see if things get deprecated (would that be too much)? I like the idea of encouraging devs to make it a remote.

Process / criteria for adding new members: 

Oversight for the community committers comes from TAC (a new subgroup or the existing
comitters oversight group?). One member of the subgroup will be responsible for adding new
members (should be a committer or promoted to one?). Criteria for new members? Anyone who
requests access?



Deprecation process:

To deprecate a repository it must first be announced to the mailing list for input. If there are no
objections then the the Program Team Manager should confirm the status.With approval the 

repository can be transferred into deprecated two weeks after the initial message.


If there are regular tech team meetings it can be raised. 



___________________________________________________________________________________________________________________________________________________


Actions from September 14th, 2016 call:

Actions from previous calls:

  • ACTION - Code in this organization must adhere to x guidelines (test coverage and/or documentation) - cleaning up text - Sally - with group to review afterwards; 
    • Codifying existing practices
    • Test coverage; passing test


Actions from November 7th, 2016 call: 

Possible to stage implementation? - Yes

  • re-organization could be done first - would basically need Mark's original spreadsheet & descriptions of groups - ready to 
  • core committer group – would probably want to wait for Tech Lead to be put into place

Overall Plan: