main.tex

% -*- mode: latex; mode: visual-line; fill-column: 9999; coding: utf-8 -*-
%%% MDAKits whitepaper
%%%
%%%
\documentclass[9pt,whitepaper,pubversion]{livecomsMDA}
% use livecoms+MDA hack template (under CC-BY 4.0)
%
% Use the 'pubversion' option for adding the citation and publication
% information to the document footer, when the DOI is assigned.


\usepackage[utf8]{inputenc}
\usepackage[english]{babel}
\babelhyphenation[english]{MDAnalysis MDAKits MDAKit}

\usepackage{graphicx}
\usepackage{hyperref}
\usepackage{authblk}
\usepackage{minted}
\usepackage{csquotes}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% update the release before publishing a new version
%
\newcommand{\versionnumber}{0.1.0}  % you should update the minor version number in preprints and major version number of submissions.

%%% point to the MDAKits repository because we want community feedback there
\newcommand{\githubrepository}{\url{https://github.com/MDAnalysis/MDAKits}}
% this should be the main github repository for this article

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\title{MDAKits: Supporting and promoting the development of community
  packages leveraging the MDAnalysis library [v\versionnumber]}
\author[1]{Irfan Alibay}
\author[2]{Jonathan Barnoud}
\author[3]{Oliver Beckstein}
\author[4]{Richard J Gowers}
\author[5]{Fiona Naughton}
\author[4]{Lily Wang}

% see
% https://github.com/livecomsjournal/article_templates/blob/master/templates/livecoms-template-reviews.tex
% for \corr, \orcid, \contrib, \presentadd usage

\affil[1]{Department of Biochemistry, The University of Oxford, United Kingdom}
\affil[2]{Centro Singular de Investigación en Tecnoloxías Intelixentes, Santiago de Compostela, Spain}
\affil[3]{Department of Physics, Arizona State University, Tempe, AZ, USA}
\affil[4]{Open Molecular Software Foundation, Irvine, CA, USA}
\affil[5]{Cardiovascular Research Institute, University of California, San Francisco, San Francisco, CA, USA}

% put in author order
\orcid{Irfan Alibay}{0000-0001-5787-9130}
\orcid{Jonathan Barnoud}{0000-0003-0343-7796}
\orcid{Oliver Beckstein}{0000-0003-1340-0831}
\orcid{Richard J Gowers}{0000-0002-3241-1846}
\orcid{Fiona Naughton}{0000-0003-0162-1346}
\orcid{Lily Wang}{0000-0002-6095-6704}


\corr{IAlibay@mdanalysis.org}{Irfan Alibay}
\corr{mdanalysis@numfocus.org}{The MDAnalysis Development Team}


\blurb{This document is maintained online on GitHub at \githubrepository; to provide feedback, suggestions, or help improve it, please visit the GitHub repository and participate via the issue tracker.}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%% PUBLICATION INFORMATION
%%% Fill out these parameters when available
%%% These are used when the "pubversion" option is invoked
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% NOTE: The hacked livecoms cls file ignores the entries
%       that are labeled IGNORED
\pubDOI{10.6084/m9.figshare.20520726}
\pubvolume{IGNORED}     % IGNORED
\pubissue{IGNORED}      % IGNORED
\pubyear{2022}
\articlenum{IGNORED}    % IGNORED
\datereceived{IGNORED}  % IGNORED
\dateaccepted{IGNORED}  % IGNORED

\begin{document}
\begin{frontmatter}
  \maketitle
  \begin{abstract}
    The open sharing of code that abides by the basic principles of FAIR (findability, accessibility, interoperability, and reusability) is essential to robust, reproducible, and transparent science. However, scientists typically are not supported in making the substantial effort required to make software FAIR-compliant, or incentivized with academic recognition or reward. Here we propose a framework to support a broad ecosystem of MDAnalysis toolkits, or “MDAKits”, with the goal to lower the barrier for researchers to produce FAIR software. We envision that MDAKits will be independent add-on packages building on MDAnalysis that meet a set of software package standards to be listed on a centralized MDAKit registry. We will continually assess packages based on criteria such as the presence of tests and documentation with the aim of encouraging continuous improvement. To lower the barrier of entry for new developers, we will provide tools such as cookiecutter templates and assist with technical support towards fulfilling these criteria. We will also work with journals such as the Journal of Open Source Software to streamline the pathway to publication, creating academic incentive for researchers to publish code. Through the MDAKits framework, we aim to foster the creation of a diverse ecosystem of sustainable community-driven downstream tools.
  \end{abstract}
\end{frontmatter}

\clearpage

\tableofcontents

\section{Introduction}
\label{sec:introduction}

\subsection{Scientific code frequently fails to meet FAIR tenets, impeding scientific progress}
\label{sec:FAIR}

Software has become increasingly essential to research. In many areas, it underlies fundamental tasks such as generating, processing, analyzing, storing, visualizing, and communicating the key results and insights ultimately published. Despite this, software is typically not central to the publication peer review process in many scientific fields. Consequently, scientific code frequently fails to meet the basic tenets of FAIR: findability, accessibility, interoperability, and reusability \cite{chue_hong_fair_2021, wilkinson_fair_2016}, 

With the publication of “The FAIR Guiding Principles for scientific data management and stewardship” in 2016 and the follow-up FAIR Principles for Research Software in 2022, it has become increasingly acknowledged that abiding by the principles of FAIR is crucial to promoting robust, reproducible, and efficient scientific discovery and innovation  \cite{chue_hong_fair_2021, wilkinson_fair_2016}. We believe that extending FAIR principles to include open-source software not only significantly advances that goal, but furthermore is necessary for transparent research. Open sharing of code brings a number of substantial benefits to the scientific community. For example, scientists can accurately replicate a given methodology or re-use previous code, reducing duplication of effort and reducing the risk of implementation errors. Indeed, the molecular simulation community in particular has made a concerted effort over recent years to encourage the open sharing of scientific codes \cite{walters_code_2020}. For example, as of July 2022, over 4700 GitHub repositories containing Python code that makes use of MDAnalysis \cite{michaud-agrawal_mdanalysis_2011, gowers_mdanalysis_2016} have been made publicly available.

However, simply sharing code is not sufficient to fulfill FAIR guidelines. In fact, making software FAIR compliant requires significant investment and often expert knowledge on the part of the developers, especially if the code was written specifically for a particular research project. For example, the Python ecosystem is so dynamic that it is common for research code to rapidly become obsolete or unusable if a new version of a key library is released. To fulfill the Reusability tenet of FAIR alone, code should include documentation, version control, and dependency management. Ideally, it would also include unit tests, examples, and packaging. Even when code is released in reference to a publication, it often falls short of ideal FAIR standards. A short survey of publications in Scopus \cite{noauthor_scopus_nodate} and the Journal of Open Source Software \cite{noauthor_journal_nodate} over 2017–-2021 identified that out of a total 720 papers citing MDAnalysis \cite{michaud-agrawal_mdanalysis_2011, gowers_mdanalysis_2016}, only 43 linked to code available on a version control platform such as GitHub, GitLab, or Bitbucket. Of these, only 18 met the requirements of best practices: they implemented unit tests, comprehensive documentation, and some means of installation.

Two major factors contribute to the lack of open-source FAIR compliant code. Firstly, code is typically written by scientists with no formal training or support in programming, for whom implementing FAIR principles can pose an intimidating and tedious barrier. Secondly, despite the substantial investment of effort and time required to implement best practices, publishing FAIR software is not typically appreciated with academic recognition or reward. Fostering a culture of open-source FAIR software requires addressing both.

\subsection{Centralized open-source packages such as MDAnalysis offer a limited solution}
\label{sec:centralization}

One solution is to consolidate scientific code around a small number of large, central packages. MDAnalysis \cite{michaud-agrawal_mdanalysis_2011, gowers_mdanalysis_2016} is a widely-used open-source Python library for molecular simulation data. With over 16 years of development by more than 160 developers, MDAnalysis has refined its code base to offer a mature, robust, flexible API that offers a range of high-performance tools to extract, manipulate, and analyze data from the majority of common simulation formats. MDAnalysis tools have been used for a variety of scientific applications ranging from exploring protein-ligand interactions \cite{alibay_ialibaymdrestraintsgenerator_2021, kokh_workflow_2020, bouysset_prolif_2021}, to understanding lipid behavior \cite{wilson_investigating_2021, smith_lipyphilic_2021}, to assessing the behavior of novel materials \cite{gowers_kugupukugupu_2021, loche_maicos_2022}. 

Until recently, MDAnalysis encouraged users to contribute their code back into the library to make it available to others. Notable examples of this include the waterdynamics \cite{araya-secchi_characterization_2014} and ENCORE \cite{tiberti_encore_2015} analysis modules. This approach, also successfully taken by packages such as cpptraj \cite{roe_ptraj_2013} and the GROMACS tools \cite{abraham_gromacs_2015}, has a number of key advantages for users and the original developers:
\begin{itemize}
    \item MDAnalysis can ensure that the code follows best practices (including documentation and tests).
    \item Code is promoted and made freely accessible to all MDAnalysis users.
    \item Maintenance, support, and potential updates are performed by the experienced MDAnalysis developer team, ensuring that the contributed code remains functional even while the other parts of the library change. The original developers can thus focus on other work.
\end{itemize}
However, the many costs of this approach can, under some conditions, result in unsustainable, untenable disadvantages:
\begin{itemize}
    \item Ensuring that the code follows best practices often requires long review periods and strict code-style adherence, thus slowing down the availability of the new code in a released version of the package.
    \item The necessity of keeping the API stable between major releases precludes quick releases of breaking changes. In general, a mature package such as MDAnalysis has a slow release cycle, so new features and bug fixes can take months to become available in new releases.
    \item As MDAnalysis implicitly agrees to maintain any code that has been added, a certain level of understanding and expertise is required from the maintainers. If the core developer team lacks expertise in a specific discipline or subdiscipline, adding new code in these areas introduces a substantial maintenance burden should the original code contributors not be available to help with maintenance. Consequently, it is impractical to include recently released or cutting-edge techniques in the core library.
    \item Introducing new package dependencies incurs software stack maintenance costs for many users who may not require this additional code.
    \item Code contributors lose complete ownership of their code.
\end{itemize}
The many disadvantages listed above can severely limit the usefulness of centralizing code around one monolithic package. Indeed, encountering these issues when attempting to expand the core MDAnalysis library attests that this approach is not the most suited for the MDAnalysis community.

\subsection{An ecosystem of downstream packages may yield more sustainable progress}
\label{sec:ecosystemadvantages}

We believe that a sustainable alternative solution is for communities such as MDAnalysis to encourage, educate, and foster researchers in their efforts towards developing individual software. We propose to overcome the difficulties of implementing FAIR best practices through the provision of structured technical assistance. Specifically, we envision that suitable tooling and documentation could be provided to ease the development of new packages, as well as a platform (which we refer to as a “registry”) where packages that meet certain standards can be advertised to the community. This idea is not novel; it is reminiscent of other successful ecosystems such as PLUMED’s PLUMED-NEST \cite{bonomi_promoting_2019}, AiiDA’s plugin registry \cite{noauthor_aiida_nodate}, or the napari-hub \cite{chan_zuckerberg_initiative_napari_nodate} of plugins for the napari image viewer \cite{sofroniew_napari_2022}, all of which list available tools that are known to work in their respective user communities.

With the help of tooling such as cookiecutter templates and example repositories, we can model best practices, promote the use of helpful tools, e.g., for checking code coverage, and reduce the work required to set up processes such as continuous integration, versioned documentation, packaging and deployment. Developers can also reach out to the MDAnalysis community for feedback, technical assistance, or even make connections with new co-developers and potential users. Decoupled from MDAnalysis’s release cycle, developers would be able to introduce new changes as required, keeping complete control over their code-base. Joining an MDAnalysis registry would allow for frequent and streamlined communication between MDAnalysis and downstream developers, allowing developers to be efficiently forewarned about potential breaking changes.

Although establishing such an ecosystem of MDAnalysis-supported packages would likely require substantial investment from MDAnalysis developers, this approach is nonetheless likely to be far more sustainable than centralizing around a super-package. Offering technical assistance to individual developers in implementing best practices will constitute a large part of the effort; however, we believe that this would remain lower than the effort associated with adding additional functionality to the core MDAnalysis library. Furthermore, once the ecosystem has been established, we hope that a growing portion of the community will participate in taking care of the registry and developers; and that the culture of following best practices and publishing code will gain momentum in itself. 

In part, we hope that this momentum will be driven by users and user expectations. Users of the MDAnalysis ecosystem would gain huge benefit from the provision of a package registry. They would be able to see new software as it gets added, rather than having to comb through literature or rely on developers advertising the code themselves. They would also be able to easily verify the current development status of a package – e.g., the registry could contain information about the health of a given codebase, such as whether it contains unit tests, sufficient documentation, and which versions of MDAnalysis it is compatible with. Packages on the registry would also come with easy-to-find instructions on how to easily install and run a given package, significantly lowering the technical barrier to use and experimentation. As the maintenance remains the burden of the package owners, unfortunately the risk remains that packages on the registry may eventually become out-of-date, which is indeed one of the major disadvantages of this approach. However, the registry significantly increases the likelihood that packages will reach users who will become sufficiently motivated to contribute or take over their maintenance and development.

In the rest of this document we detail how MDAnalysis proposes to implement such an ecosystem of toolkits, which we will henceforth call “MDAKits” (MDAnalysis Toolkits). We detail our expectations for MDAKits in terms of best practices and how their registration and continuous validation is anticipated to work.

\section{The MDAKit framework}
\label{sec:mdakitframework}

\begin{figure}[hbt!]
    \centering
    \includegraphics[width=\linewidth]{figures/MDAKitFramework.png}
    \caption{Workflow diagram of the MDAKit framework. Starting from the creation of an MDAKit package, with the help of documentation and the MDAKit cookiecutter, the package then goes through the process of being added to the MDAKit registry, undergoing continuous validation and review and eventually reaching the stage of publication.}
    \label{fig:workflow}
\end{figure}

The MDAKit framework (Fig.~\ref{fig:workflow}) is designed to be a complete workflow to help and incentivize developers to go from the initial stages of package development all the way through to the long term maintenance of a mature codebase, while adhering to best practices.


\subsection{Main goals}
\label{sec:maingoals}

As such, the main goals of the proposed MDAKit framework are:
\begin{enumerate}
    \item To help as many packages as possible implement best practices and develop user communities.
    \item To ensure that members of the MDAnalysis community can easily identify new packages of interest and know to what extent they are suitable for production use.
    \item To improve contacts between MDAnalysis core library developers and those developing packages using MDAnalysis.
    \item To encourage participation from the community at all steps of the process.
\end{enumerate}
We wish to state three main points that the framework is \emph{not} designed for early on:
\begin{enumerate}
    \item The MDAKit framework is not intended to restrict the packages which can participate. It is our view that all packages at any stage of their development are of value to the community. As such, we aim for framework components to be as non-blocking as possible.
    \item It is not the intention of any parts of this framework to take ownership of the packages which participate within it. The original code developers retain full ownership and responsibility for their packages and may optionally participate in any part of this framework.
    \item We also do not want to block future contributions to the core library. If new code in MDAKits prove particularly popular, and the MDAKit developers are amenable to contributing these back into the core library, the MDAnalysis team will work with them to integrate additional functionality into MDAnalysis itself
\end{enumerate}

\subsection{Overview of the framework}
\label{sec:overviewframework}

The MDAKit framework (Fig.~\ref{fig:workflow}) is a multi-step process. In the first step of the MDAKit framework, developers create an initial package which is intended to achieve a set purpose of their choice. To help with this process, MDAnalysis provides a cookiecutter template specifically for MDAKits \cite{wang_cookiecutter_nodate}, alongside documentation on best practices and how to optimally use the MDAnalysis API. An overview of what we consider to be best practices for the contents of MDAKit packages is included in Section~\ref{sec:definitions}. We note that at this point MDAKits are not expected to fully adhere to best practices, but should at least meet the minimum requirements defined in Section~\ref{sec:definitions} before moving to the next step along this process.

Once a package is suitably developed, code owners are encouraged to add the details of their code to the “MDAKit registry” which will advertise their package to the MDAnalysis community and offer continual validation and review tools to help with package maintenance. Section~\ref{sec:registry} contains more information about the MDAKit registry, including the registration process (Section~\ref{sec:registration}). Briefly, the registration process involves submitting a metadata file to the registry that contains essential information about the MDAKit, such as where the source code is provided, who the code authors are, and how to install the MDAKit. The contents of this metadata file will be reviewed both by automatic code checks and the MDAnalysis developer team before being  added to the registry. We want to highlight  that this process does not include checks on scientific validity or code health. In fact, none of the processes in this framework account for the scientific validity of the MDAKits. While members of the community are free to offer help, scientific or technical validity is beyond the scope of what is feasible with the MDAnalysis registry.

Upon registration, the MDAKit will be automatically advertised to the MDAnalysis community (see Section~\ref{sec:advertising}). In the first instance this will amount to a set of auto-generated pages which will expose the details in the metadata file provided in the registration step. Additional tags and badges will also be included which reflect the current status and health of the package. Examples include:
\begin{itemize}
    \item whether or not it is compatible with the latest versions of MDAnalysis
    \item what percentage of the codebase is covered by unit tests
    \item what type or extent of documentation is provided
    \item what Python versions are currently supported.
\end{itemize}

This status information will be provided as part of checks done during the continual validation and review steps (see Sections~\ref{sec:continualvalidation} and \ref{sec:continualreview}) of the framework. These steps will involve a mix of regularly scheduled automatic (e.g., linters and unit test execution) checks and more infrequent manual (e.g., code reviews) processes. It is our intention that code health analysis will help developers maintain and improve their codes, as well as suitably warn potential users about issues they may encounter when using a given codebase.

Where possible, the framework will encourage a code review process to be carried out by members of the MDAnalysis community. The aim here is to work with developers in identifying potential areas of improvements for both MDAKits and the core MDAnalysis library (see Sections~\ref{sec:continualreview} and \ref{sec:feedingback}). We aim to tie this process closely to the review processes of journals such as the Journal of Open Source Software \cite{noauthor_journal_nodate}, which would help lower the barrier towards and encourage an eventual publication (Section~\ref{sec:publication}).

\section{Defining MDAKits: best practice package features}
\label{sec:definitions}

Here we list requirements that we believe MDAKits should strive to fulfill in order to meet best practices in Python package usability and maintenance. To help with implementing these, a cookiecutter is provided which offers a template for potential MDAKits to follow \cite{wang_cookiecutter_nodate}. We want to emphasize again that the aim of the MDAKit project is to encourage best practices whilst also minimizing barriers to sharing code where possible. Therefore, only a minimal set of requirements listed here as \emph{required} are necessary for MDAKits to be included in the MDAKit registry. Similarly, we do not mean to enforce the label of MDAKit on any package; the process is fully optional and the code owners may choose to associate themselves with it.

\begin{Checklists}
\begin{checklist}{MDAKit: required features}
\textbf{All MDAKits will have to implement the features on this list in order to become registered.}
\begin{itemize}
\item Code in the package \emph{uses MDAnalysis} (\ref{sec:usesmdanalysis}).
\item Open source code is published under an \emph{OSI approved license} (\ref{sec:opensource}).
\item Code is \emph{versioned} and provided in an \emph{accessible version-controlled repository} (\ref{sec:versioning}).
\item Code \emph{authors and maintainers are clearly designated} (\ref{sec:authors}).
\item \emph{Documentation} is provided (\ref{sec:documentation}).
\item \emph{Tests and continuous integration} are present (\ref{sec:tests}).
\end{itemize}
\end{checklist}

\begin{checklist}{MDAKit: optional features}
\textbf{Features that are highly recommended to be implemented.}
\begin{itemize}
\item Code is \emph{installable as a standard package} (\ref{sec:packaging}).
\item Information on \emph{bug reporting, user discussions, and community guidelines} is made available (\ref{sec:community}).
\end{itemize}
\end{checklist}
\end{Checklists}

 
\subsection{Code using MDAnalysis (required)}
\label{sec:usesmdanalysis}

This is the base requirement of all MDAKits. The intent of the MDAKit framework is to support packages existing downstream from the MDAnalysis core library. MDAKits should therefore contain code using MDAnalysis components which are intended by the package authors to address the MDAKit’s given purpose.
 
\subsection{Open source code under an OSI approved license (required)}
\label{sec:opensource}

The core aim of MDAKits is to encourage the open sharing of codes to potential users within the MDAnalysis community and beyond. To achieve this, we require that codes under this framework be released as open source. Here we define open source as being under an Open Source Initiative (OSI) approved license \cite{open_source_initiative_licenses_nodate}.

As of writing, the MDAnalysis library is currently licensed under GPLv2+ \cite{noauthor_gnu_nodate}. Due to limitations with this license type, we cannot currently recommend other licenses than GPLv2+ for codes importing MDAnalysis. However, we hope to relicense to a less restrictive license. In this event, MDAKits will be able to adopt a wider range of OSI approved licenses.
 
\subsection{Versioning and provision under an accessible version-controlled repository (required)}
\label{sec:versioning}

The ability to clearly identify changes in a codebase is crucial to enabling reproducible science. By referencing a specific release version, it is possible to trace back any bug fixes or major changes which could lead to a difference in results obtained with a later version of the same codebase. Whilst we encourage the use of Semantic Versioning ("semver") \cite{preston-werner_semantic_nodate}, any PEP440 \cite{noauthor_pep_nodate-1} compliant versioning specification, would be suitable for MDAKits.

Beyond versioning releases, it is also crucial to be able to develop code in a sustainable and collaborative manner. The most popular way of achieving this is through the use of version control through Git \cite{noauthor_git_nodate}. We require all MDAKits to be held in a publicly facing version controlled repository such as GitHub \cite{github_inc_github_2022}, GitLab \cite{gitlab_inc_gitlab_2022}, or Bitbucket \cite{atlassian_bitbucket_2022}.

\subsection{Designated code authors and maintainers (required)}
\label{sec:authors}

In order for users to be able to contact the code owners and maintainers, all MDAKits should clearly list their authors and a means of contacting the persons responsible for maintaining the codebase. To incentivize and recognize contributors throughout the life of a project, we recommend the use of a version controlled “authors” file which lists the authors to a codebase over time.
 
\subsection{Documentation (required)}
\label{sec:documentation}

Describing what a given code does and how to use it is a key component of open sharing. Ideally a package would include a complete description of the entire codebase, including both API documentation and some kind of user guide with worked examples on how the code could be used in certain scenarios. Whilst this is recommended as best practices for an MDAKit, we recognize that this is not  always feasible, especially in the early stages of development. Therefore, the minimum requirement for MDAKits is to have a readme file which details the key aspects of the MDAKit, such as what it is intended to do, how to install it, and a basic usage example.

For best practices, we strongly recommend using docstrings (see PEP 257 \cite{noauthor_pep_nodate}) to document code components and using a tool such as ReadTheDocs \cite{read_the_docs_inc_read_2022} to build, version and host documentation in a user-friendly manner. We also recommend using duecredit \cite{halchenko_duecreditduecredit_2021} to provide the correct attributions to a given method if it has been published previously.

\subsection{Tests and continuous integration (required)}
\label{sec:tests}

Testing is a critical component to ensure that code behaves as intended. Not only does it prevent erroneous coding, but it also assures users that the code they rely on is working as intended. We require at least a single regression test for major functionality to qualify for the registry (i.e. if a toolkit implements a new analysis method, at least one test that checks to see if the analysis code yields the expected value on provided data;  regression tests can often double as example documentation).

Ideally one should do full unit testing of the contents of a code, ensuring that not only a specific outcome is reached, but also that each smaller component works. As part of best practices, we highly recommend implementing tests using a framework such as pytest \cite{krekel_pytest-devpytest_2004} for executing tests and codecov \cite{codecov_llc_codecov_2022} to capture which lines are covered by the tests. We strongly encourage that a minimum of at least 80\% of the code lines be covered by tests. 

To ensure that tests are run regularly, the recommended best practice is to implement a continuous integration pipeline that performs the tests every time new code is introduced. We encourage the use of free pipelines such as GitHub Actions \cite{github_inc_github_2022-2} to implement continuous integration.

\subsection{Packaging}
\label{sec:packaging}

Providing a standard means of installing code as a package is important to ensure that other code can correctly link to (i.e., \mintinline{python}{import} in the case of Python) and use its contents. Whilst it can be easy to expect users to simply read a Python script, look at its required dependencies, and install them manually, this can quickly become unreasonable should the code grow beyond a single file. Additionally, the lack of clearly defined versions, including the intended Python versions, can lead to inoperable code.

As best practices we heavily encourage the use of setuptools \cite{noauthor_pypasetuptools_2022} or an alternative such as poetry \cite{noauthor_poetry_nodate} for package installation. We also encourage that packages be available on common package repositories such as PyPi \cite{noauthor_pypi_nodate} and conda-forge \cite{conda-forge_community_conda-forge_2015}. The use of such repositories and their respective package managers can significantly lower the barrier to installing a package, enabling new users to rapidly get started using it.

\subsection{Bug reporting, user discussions, and community guidelines}
\label{sec:community}

To help maintain and grow the project, it is important to specify where users can raise any issues they might have about the project or simply ask questions about its operation. To achieve this, we recommend at the very least adding documentation that points users to an issue tracker.

Key to successfully building a user community is ensuring that there are proper guidelines in place for how users will interact with a project \cite{grossfield_how_2021}. As best practices we recommend making a code of conduct available that defines how users should interact with developers and each other within a project. It is also advised to provide information on how users can contribute to the project as part of its documentation.


\section{The MDAKit registry}
\label{sec:registry}

As defined in Section~\ref{sec:mdakitframework}, once MDAKits are created, we encourage that they be added to the MDAKit registry. The registry not only provides a platform to advertise MDAKits to the MDAnalysis user community, but also offers tools and workflows to help packages improve and continue to be maintained. Here we describe the various processes which will occur within the registry. We note that we expect the exact details of how these processes will be implemented to evolve over time based on feedback from MDAKit developers and other members of the MDAnalysis community.
 
\subsection{MDAKit registry contents}
\label{sec:registrycontents}

The main aim of the registry is to hold information about MDAKits. The contents of the registry will therefore center around a list of packages and the metadata associated with each MDAKit. This metadata will take the form of two files: one containing user-provided information on the package contents (see Section~\ref{sec:registration}), and the other a set of mostly auto-generated details indicating the code health of the package (see Section~\ref{sec:advertising}). 

This metadata will be used for two purposes: continuous integration testing and documentation. Continuous testing, helper methods and workflows will be used to regularly install MDAKits and run their test suite (if available) to check if they still work as intended. Should the tests fail, package maintainers will be automatically contacted and failure information will be recorded in the code health metadata to inform users. For the registry documentation, the metadata will be used to provide user-facing information about the various MDAKits in the registry, their contents, how to install them, and their current status as highlighted by continuous integration tests. The registry will also include further information and user guides on the MDAKit framework, helping developers implement the contents of this whitepaper.

\subsection{Registering MDAKits}
\label{sec:registration}

\begin{listing*}[bht!]
    \centering
    \begin{minted}{yaml}
## Required entries
project_name: propktraj
authors: https://github.com/Becksteinlab/propkatraj/blob/master/AUTHORS
maintainers:
  - orbeckst
  - IAlibay
description: <
    Calculate pKa estimates over the length of a trajectory using
    PROPKA 3. Currently only handles protein pka.
license: GPL-3.0
project_home: https://github.com/Becksteinlab/propkatraj
documentation_home: https://github.com/Becksteinlab/propkatraj/blob/master/README.md
documentation_type: README

## Optional entries
install: pip install propkatraj
python_requires: >=3.8
mdanalysis_requires: >2.0.0
test_run: 
  - pip install pytest
  - pytest --pyargs propkatraj.tests
codecov: https://codecov.io/gh/Becksteinlab/propkatraj/branch/master
development_status: Mature
changelog: 
publications:
  - https://doi.org/10.1021/ct200133y
  - https://doi.org/10.1085/jgp.201411219
  - https://doi.org/10.5281/zenodo.3942720
    \end{minted}
    \caption{YAML metadata file for an MDAKit entry of the propkatraj package, stored as \texttt{mdakits/propkatraj/metadata.yaml} in the registry repository.}
    \label{fig:metadatapropkatraj}
\end{listing*}

A key feature of the MDAKit framework is the process of adding MDAKits to the registry. As previously defined, our intent is to offer a low barrier to entry and have packages be registered early in their development cycles. This allows developers to benefit from the MDAKit registry validation and review processes early on, hopefully lowering the barrier to further improvements and encouraging early user interactions and feedback.
 
From an MDAKit developer standpoint, the registration process involves opening a pull request against the MDAKit registry adding a new YAML file with metadata about the project. The metadata, as detailed in Listing~\ref{fig:metadatapropkatraj}, contains information such as the MDAKit description, source code location, install instructions, how to run tests, and where to find usage documentation. Complete details about the metadata file specification will be provided in the MDAKit registry documentation.

After a pull request is opened, the MDAnalysis developers will review the contents of the submission based on the following criteria:
\begin{enumerate}
    \item If the required features for MDAKits are met (Section~\ref{sec:definitions}), that is:
    \begin{enumerate}
        \item Does the MDAKit contain code using MDAnalysis?
        \item Is the MDAKit license appropriate?
        \item Is the MDAKit code offered through a suitable version-controlled platform?
        \item Are the MDAKit authors and maintainers clearly designated in the metadata file?
        \item Is there at least minimal documentation in place detailing the MDAKit and its functionality?
        \item Are there at least minimal regression tests available within the MDAKit code?
    \end{enumerate}
    \item If the metadata file passes linting and integration checks
    \item That there are no potential breaches of community guidelines
\end{enumerate}
Once the criteria are fulfilled the metadata will be merged and the MDAKit will be considered registered. Updates to the MDAKit metadata can be carried out at any time after registration by opening pull requests to change the metadata file contents.

\subsection{Advertising MDAKits}
\label{sec:advertising}

Registered MDAKits will be automatically added to the registry’s public facing documentation. This involves an indexable list of entries for all registered MDAKits. Each entry will display available information from the provided metadata, e.g., what the MDAKit does, any relevant keywords, how to obtain the source code, how to install the package, and where to find relevant documentation. Alongside this information will also be a set of badges which describe the current health of the codebase, allowing users to rapidly identify which packages are currently active, and their level of code maturity. This will include information such as: which MDAnalysis library versions the package is compatible with, how much test coverage does the package have, what type of MDAnalysis API extensions are provided (e.g., using base classes such as AnalysisBase or ReaderBase), and whether integration tests are currently failing.

Information about MDAKits will be continually updated, either through automatic checks or manual additions provided by package owners updating the metadata files. As we aim for the MDAKit registry to be immutable (aside from special cases covered by Section~\ref{sec:removal}), should an MDAKit stop being maintained, it will not be removed from the index but instead labeled as abandoned.

\subsection{Continual validation}
\label{sec:continualvalidation}

The MDAKit registry will implement workflows to validate the code health of registered packages. This will mostly center around a test matrix that will regularly run to check if the latest MDAKit release can be installed and if unit tests pass with both the latest release of MDAnalysis and the development version. Should tests fail regularly, an issue will be automatically raised on the MDAKit registry issue tracker contacting the package maintainers and letting them know of the failure. The auto-generated code health metadata for the MDAKit will also be updated to reflect whether or not the tests are currently failing or passing.

In the future we will hope to expand these tests to include more historical releases of the MDAKits and the MDAnalysis library, checks for different architectures (non-x86), and operating systems. We may also expand the checks to consider the cross-compatibility of MDAKits with each other, offering insights on which packages can be safely used together.

\subsection{Continual review}
\label{sec:continualreview}

To help package growth and improvements, it is our goal for the registry to become a platform that allows members of the MDAnalysis community to offer feedback on MDAKits over the lifetime of their inclusion on the registry. Unfortunately, as MDAnalysis developers can only devote limited time towards the registry, offering regularly scheduled comprehensive reviews of packages is too large an undertaking to be practical.

Instead, we aim to use a system of badges and achievements to push packages towards gradual improvements. For example, we may offer an achievement that encourages MDAKits to use high performance PBC-aware distance routines defined in `MDAnalysis.lib.distances` instead of relying on NumPy’s `linalg` method to find the distance between two points. Once MDAKit owners believe that they have suitably updated their code to match this, they can open a pull request highlighting these changes and have developers review these smaller, more focused updates.

MDAKit users will also be encouraged to provide feedback, request improvements, and report bug fixes. However, this should happen outside the scope of the registry; instead, we will ask for users to use the MDAKit’s own issue tracker for these.

\subsection{Feeding back into the MDAnalysis library}
\label{sec:feedingback}

The existence of the MDAKits framework does not preclude the addition of new codes and methods to the core MDAnalysis library. The MDAKit registry, and especially the ongoing review process, will provide a platform for MDAnalysis and MDAKit developers to interact and work together to identify common goals and areas of improvements for both upstream and downstream packages. In particular, MDAnalysis developers will work with MDAKit developers to see if any popular MDAKit methods, components or other means to improve core method performance and lower the barrier to downstream package development can and should be implemented back into the core MDAnalysis library.

\subsection{Towards publication}
\label{sec:publication}

We have laid out a number of best practices here that we encourage MDAKits to fulfill. These essentially amount to the majority of the contribution criteria for submissions to software-focused journals such as the Journal Open Source Software (JOSS) \cite{noauthor_journal_nodate}. In order to incentivize developers, we will heavily encourage MDAKits to consider submission to a journal such as JOSS \cite{noauthor_journal_nodate} once they meet the required levels of best practices. To aid in this process, the MDAnalysis developers will in the first instance work with journal editors at JOSS to create a streamlined process to submit MDAKits as JOSS entries \cite{noauthor_submitting_2018}. The details of this process are still under development.

\subsection{Raising issues, concerns, and paths to registry removal}
\label{sec:removal}

If community members (users, developers or otherwise) have concerns about an MDAKit, we primarily encourage them to raise issues on the MDAKit’s issue tracker. However, in situations where the MDAKit maintainers cannot respond, or if the concern relates to code of conduct breaches, MDAnalysis developers may step in. If an MDAKit has systemic issues with its correctness, the MDAKit may be given special annotations warning users about the issues before using the code. We generally view the MDAKit registry as a permanent record, and will avoid removing packages after registration even if they become fully obsolete. However, we reserve the right to remove packages at our discretion in specific cases, notably code of conduct breaches and violation of the GitHub terms of service \cite{github_inc_github_2022-1}.

\subsection{Long term registry maintenance and support}
\label{sec:maintenance}

As with most MDAnalysis projects, long-term support for the MDAKit framework and especially the registry is expected to be carried out by contributors from the MDAnalysis community. Members of the MDAnalysis core development team will lead the maintenance of the registry and also be responsible for passing judgment on serious events such as code of conduct breaches. In the long term, we hope that any gains in popularity of the MDAKits framework will be accompanied by an increase in community involvement in reviews and other maintenance tasks.


\section{Conclusions}
\label{sec:conclusions}

In this document we outline our plans to implement an MDAnalysis framework, termed MDAKits, to assist and incentivize the creation of FAIR-compliant packages that use and extend MDAnalysis. We describe the current state of scientific code, which is typically published either in independent repositories of varying quality, or as additions to a large, monolithic package. We summarize the limitations of each approach that result in code that falls short of FAIR principles, or may end up impractical to sustain as a long-term strategy. We propose the MDAKits framework as an alternative solution to support developers in creating new packages, guiding them through the process of achieving best practices and FAIR compliance.

In Section~\ref{sec:mdakitframework} we lay out the aims and structure of an MDAKit, summarizing the minimal and optimal requirements that we think necessary to build sustainable, reusable software. These include publishing code under a suitable open-source license, the use of version control, comprehensive documentation, thorough unit tests, and packaging the software following modern best practices. In Section~\ref{sec:definitions} we outline our vision for the MDAKit registry, a public facing repository that promotes MDAKits to the MDAnalysis community. The MDAKit registry will offer regular checks and reviews in order to help improve and maintain the listed MDAKits. We describe a prospective workflow that begins from the initial registration of MDAKits and reaches as far as eventual publication in software-focused journals such as JOSS.

This document is just the first step and broad guide to our vision of developing a rich, diverse software ecosystem, and we are still in the early stages of implementing MDAKits. While we expect that we may need to revisit and refine our strategy to best serve the needs of the community, we believe that the fundamental framework outlined here will bring great benefit to the software written and used by scientists, and thereby empower transparent and reproducible research. 


\section*{Acknowledgements}

We gratefully acknowledge the 163 developers and countless community members who have contributed to the MDAnalysis project over the last 16 years and NumFOCUS for its support as our fiscal sponsor.


%\section*{Author Contributions}
%%%%%%%%%%%%%%%%
% This section mustt describe the actual contributions of
% author. Since this is an electronic-only journal, there is
% no length limit when you describe the authors' contributions,
% so we recommend describing what they actually did rather than
% simply categorizing them in a small number of
% predefined roles as might be done in other journals.
%
% See the policies ``Policies on Authorship'' section of https://livecoms.github.io
% for more information on deciding on authorship and author order.
%%%%%%%%%%%%%%%%
%
%(Explain the contributions of the different authors here)

% We suggest you preserve this comment:
%For a more detailed description of author contributions,
%see the GitHub issue tracking and changelog at \githubrepository.

%\section*{Other Contributions}
%%%%%%%%%%%%%%%
% You should include all people who have filed issues that were
% accepted into the paper, or that upon discussion altered what was in the paper.
% Multiple significant contributions might mean that the contributor
% should be moved to authorship at the discretion of the a
%
% See the policies ``Policies on Authorship'' section of https://livecoms.github.io for
% more information on deciding on authorship and author order.
%%%%%%%%%%%%%%%

%(Explain the contributions of any non-author contributors here)
% We suggest you preserve this comment:
%For a more detailed description of contributions from the community and others, see the GitHub issue tracking and changelog at \githubrepository.


\section*{Potentially Conflicting Interests}

The authors declare no potential conflicts of interest.

\section*{Funding Information}

This work is made possible thanks to a grant from the Chan-Zuckerberg Initiative (grant number 2021-237663), supporting MDAnalysis and the MDAKit project under an EOSS4 award.

Jonathan Barnoud has received financial support from the Agencia Estatal de Investigación (Spain) (REFERENCIA DEL PROYECTO / AEI / CÓDIGO AXUDA), the Xunta de Galicia - Consellería de Cultura, Educación e Universidade (Centro de investigación de Galicia accreditation 2019-2022 ED431G-2019/04 and Reference Competitive Group accreditation 2021-2024, CÓDIGO AXUDA) and the European Union (European Regional Development Fund - ERDF)


\section*{Author Information}
\makeorcid


% The bib file is read-only. It is linked to the Zotero group
% https://www.zotero.org/groups/4747225/mdanalysis
% Add references there and then sync: go to the bib file and hit the "Refresh" button.

\bibliography{zotero-mdanalysis}


\end{document}