SPEC Purpose and Process

Description#

The SPEC process is designed to identify areas of shared concern between projects in the scientific Python ecosystem and to produce collaboratively written, community adopted guidelines for addressing these. Such guidelines are known as SPECs: Scientific Python Ecosystem Coordination documents.

Specifically, the purpose of the SPEC process is

1. to help unify the ecosystem for users and developers;
2. to provide guidance to projects on technical issues or processes;
3. to document standard APIs, development tools, and community practices; and
4. to foster ecosystem-wide discussions of common problems and to develop shared solutions.

Projects in the ecosystem have an existing, diverse set of proposal processes and development constraints. SPECs complement these: they are a mechanism to encourage shared practices and improve uniformity of experience across projects. SPECs may, for example, capture established practices so that new projects can learn from them; or they may propose a new practice that the authors believe will benefit the ecosystem as a whole.

Projects decide for themselves whether to adopt any given SPEC—often, this would be through team consensus. A SPEC may not be a good fit for every single project, and thus there is no expectation that all SPECs must be adopted by all projects. That said, SPECs serve their purpose through being adopted by several projects—and their authority stems from the extent to which they are.

Participants in the SPEC process must adhere to our Code of Conduct.

Key Terms#

Scientific Python Ecosystem

The ecosystem is a loose federation of community developed Python projects widely used in scientific research that interact well with one another and that follow similar norms of development, documentation, testing, and so forth.

SPEC Core Projects

The Core Projects are a small subset of the ecosystem consisting of mature, community developed projects that are (a) depended upon by most of the other projects and (b) responsible for reviewing, discussing, implementing, and endorsing SPEC documents.

SPEC Steering Committee

The Steering Committee leads the SPEC project and manages the SPEC process including moderating the SPECs discussion forum, accepting SPEC documents, and maintaining the SPEC process documents.

SPEC Process

The SPEC process is outlined in this document and the associated SPEC Steering Committee and SPEC Core Projects documents. This process is managed and overseen by the Steering Committee, and functions in collaboration with the Core Projects, community members, and projects across the ecosystem.

SPEC Document

A SPEC document provides operational guidelines for projects and helps coordinate the ecosystem to provide a more unified experience for users and developers. These documents are developed collaboratively with the Core Projects and other interested ecosystem projects and community members.

Format#

SPECs are UTF-8 encoded text files using Markdown format and stored in the SPEC repository. The SPEC documents are converted to HTML by code in the scientific-python.org repository using Hugo and deployed to https://scientific-python.org/specs/. Each SPEC has a corresponding discussion with the same title, where anyone can comment, ask questions, or vote on existing comments.

Implementation#

The Steering Committee manages the SPEC process and will provide guidance to contributors throughout the process. In this section, we provide an overview of the main decision points in the SPEC process and provide guidance for how to get started with a new SPEC proposal.

Decision Points#

A SPEC passes through four decision points over the course of its development and implementation: Accept, Publish, Endorse, and Adopt.

graph LR

START[ ]--> |Propose| A[Accept]
A--> |Develop| B[Publish]
B--> |Canvass| C[Endorse]
C--> |Recommend| D[Adopt]

click A callback "Steering Committee Action"
click B callback "Author Action"
click C callback "Core Project Action"
click D callback "Ecosystem Action"

style START fill:#FFFFFF, stroke:#FFFFFF;

The authors starts by proposing a SPEC, as outlined in New SPEC Proposals. The decision to accept a SPEC is made by the Steering Committee, at which point it is added to the main branch of the SPEC repository, clearly labeled as a draft. Proposed SPECs are accepted once (a) the draft is written to clearly explain the area of common concern and a general approach to a shared solution and (b) there are contributors (from at least two Core Projects) interested in working on the new SPEC and in championing it to their projects as well as the larger community. Additional details may be found in Steering Committee documentation.

In the accepted phase, the authors develop their SPEC, in consultation with Core Projects and interested community members. This is done in a collabortive and iterative process, focused on ensuring that the SPEC is broadly applicable and likely to be widely adopted. The intent is that most SPECs will have authors from several projects, including Core Projects. Once authors consider their SPEC complete, they publish it, removing its draft status.

The author now canvasses their project with the Core Projects, so they may endorse it. Once a SPEC is endorsed, substantive changes require the approval of all endorsing Core Projects.

A SPEC is recommended for wide-spread adoption once it is endorsed by two (or more) Core Projects. Additional details may be found in Core Project documentation. Individual projects choose to adopt the SPEC according to their own decision-making processes. Each SPEC describes what adopting it means in its Ecosystem Adoption section. Ecosystem projects are welcome to adopt a SPEC at any point in the process, however, it may make sense to wait until a SPEC is endorsed by several Core Projects to ensure that the SPEC has been vetted and is deemed stable enough for widespread adoption. Once a SPEC is endorsed, it may still evolve, but the barrier for modifying the SPEC will increase substantially (since all endorsing projects would need to agree to changes). Projects that adopt a SPEC early should engage in the collaborative process leading to the SPEC being endorsed by the Core Projects, to ensure that their use cases are incorporated.

Badges#

Projects can highlight their adoption of specific SPECs with a SPEC badge. For example, for SPEC 0, we recommend using

Alternatively, you can use one badge to indicate adoption of multiple SPECs. For example, to indicate adoption of SPECs 0, 1, and 4, we recommend the following

New SPEC Proposals#

A good SPEC proposal focuses on a single key recommendation or idea for coordinating projects in the scientific Python ecosystem. Before proposing a SPEC, we highly recommended that you first vet the idea by doing one or more of the following:

1. discuss the idea with at least one project in the ecosystem,
2. discuss the idea with at least one other member of the ecosystem, or
3. create a minimal, proof of concept prototype.

Before a proposed SPEC can be accepted, the idea must be discussed on the discussion forum under the SPECS/Ideas topic. Thereafter a new SPEC document must be submitted as a pull request to the SPEC repository.

Use the quickstart.py script to create the new SPEC document. Located at the top-level of the SPEC repository, the script will ask you a few questions¹ and then create a new file appropriately named with a basic template for you to complete (e.g., spec-0000/index.md). Once the SPEC is in reasonable shape, file a pull request against the SPEC repository. The Steering Committee then considers the SPEC as presented in the pull request and will provide additional guidance.

Notes#