Reorganize qldpc.codes.distance into subpackages by method

docs/source/contributing/standards/code.rst

@@ -38,10 +38,10 @@ Code style and organization
 * Numba code uses ``always_compiled`` or ``compiled_on_request`` from ``qldpc/_compilation.py``, never ``njit(cache=True)``; that module's docstring says which to use, and ``_compilation_test.py`` enforces its rules.
 * ``src/qldpc/_code_style_test.py`` checks license headers, section headers, and the order of plain and underscore names.
 * The top-level modules and subpackages of ``qldpc.circuits`` and ``qldpc.decoders`` import each other in layers: each imports only from itself and from lower layers, and never from the root of its package.
-  ``_LAYERS`` in the package's ``layering_test.py`` lists the layers and is the only place to update when adding a top-level module or subpackage; changes inside a subpackage need no edits.
+  ``_LAYERS`` in the package's ``_layering_test.py`` lists the layers and is the only place to update when adding a top-level module or subpackage; changes inside a subpackage need no edits.
   Imports in functions and under ``TYPE_CHECKING`` count; tests and ``conftest.py`` files may import from any layer.
   To keep an import that breaks the layers, end its line (for an import that spans several lines, its first line) with ``# layering-exception: <reason>``; the check fails if such a comment is no longer needed.
-  To layer another package, add a ``layering_test.py`` to its directory that calls ``check_import_layering`` from ``src/qldpc/conftest.py``.
+  To layer another package, add a ``_layering_test.py`` to its directory that calls ``check_import_layering`` from ``src/qldpc/conftest.py``.
 * Do not weaken a failure with ``noqa``, ``type: ignore``, or coverage exclusions unless the exceptional condition is real and documented.
 
 .. _simplicity-and-clarity:

docs/source/contributing/standards/decoders.rst

@@ -64,7 +64,7 @@ These requirements apply when changing how qLDPC builds, coerces, or resolves de
 
 * Coerce user-supplied error decoders with ``_as_error_decoder`` from ``decoders/protocols.py`` inside ``qldpc.decoders``, or ``build_error_decoder`` from ``decoders/_resolution.py`` outside it, and call ``decode_errors``.
 * Reject prebuilt decoders with ``reject_prebuilt_decoder`` from ``decoders/_resolution.py`` when a method constructs the matrix that it decodes.
-* Keep the import layers of ``qldpc.decoders`` that ``_LAYERS`` in ``decoders/layering_test.py`` lists, and add a new top-level module or subpackage to a layer there; the :doc:`code standard <code>` describes the rule and its exceptions.
+* Keep the import layers of ``qldpc.decoders`` that ``_LAYERS`` in ``decoders/_layering_test.py`` lists, and add a new top-level module or subpackage to a layer there; the :doc:`code standard <code>` describes the rule and its exceptions.
 * Keep ``build_error_decoder`` specific to error decoders.
   Dispatch on explicit capabilities such as ``compile_decoder_for_dem``, ``decode_observables``, and ``ErrorDecoder``, never on output length.
 * In code-capacity callers under ``qldpc.codes``, resolve decoder inputs with ``get_code_capacity_decoder`` from ``codes/_code_capacity.py``.

src/qldpc/_compilation.py

@@ -40,9 +40,8 @@
 their module-level names, which are looked up when it is compiled, so a compiled function calls
 compiled functions.  This is why every function that a compiled-on-request function calls must be
 always-compiled or compiled on request too.  Other modules must call a compiled-on-request function
-through the module that defines it, as in ``information_sets.get_overlapping_family(...)``, and
-never through a name imported by ``from``, which would keep the version that was current at import
-time.
+through the module that defines it, as in ``basis_families.get_overlapping_family(...)``, and never
+through a name imported by ``from``, which would keep the version that was current at import time.
 
 To choose between them, use :func:`always_compiled` for the per-word loops and for the helper
 functions that other compiled code calls: every search needs them, and the disk cache makes

src/qldpc/circuits/__init__.py

@@ -4,7 +4,7 @@
 
 The top-level modules and subpackages of this package import each other in layers, from shared
 foundations to complete workflows: each imports only from itself and from lower layers, which rules
-out import cycles between them.  ``layering_test.py`` lists the layers and checks these imports,
+out import cycles between them.  ``_layering_test.py`` lists the layers and checks these imports,
 apart from any that are marked as exceptions.
 """
 

src/qldpc/circuits/certification/_fault_search.py

@@ -24,7 +24,7 @@
 
 from qldpc._binary import get_rank_packed, popcount_word, trailing_zeros_word
 from qldpc._compilation import always_compiled
-from qldpc.codes.distance._search.coset_leader_search import (
+from qldpc.codes.distance._coset_leader_search import (
     CosetLeaderSearch,
     CosetSearchMethod,
     MinWeightBound,

src/qldpc/circuits/certification/fault_propagation.py

@@ -14,7 +14,7 @@
 from qldpc import math
 from qldpc._binary import get_rank_packed, pack_bits
 from qldpc.codes.distance import DistanceMethod
-from qldpc.codes.distance._search.coset_leader_search import CosetSearchMethod, MinWeightBound
+from qldpc.codes.distance._coset_leader_search import CosetSearchMethod, MinWeightBound
 
 from .._post_selection import get_flag_detectors, get_selected_detectors, is_integer
 from ..bookkeeping import DetectorRecord

src/qldpc/circuits/certification/fault_propagation_test.py

@@ -18,7 +18,7 @@
 
 from qldpc import circuits
 from qldpc.codes.distance import DistanceMethod
-from qldpc.codes.distance._search.coset_leader_search import MinWeightBound
+from qldpc.codes.distance._coset_leader_search import MinWeightBound
 
 from . import fault_propagation
 

src/qldpc/codes/conftest.py

@@ -15,7 +15,7 @@
 
 from qldpc import codes
 from qldpc.codes.distance import _tuning
-from qldpc.codes.distance._search import automorphism_families
+from qldpc.codes.distance._information_sets import automorphism_families
 from qldpc.objects import Pauli
 
 # Changes fields of the active search tuning until the end of a test (see the tune fixture).

src/qldpc/codes/distance/README.md

@@ -27,54 +27,71 @@ Users reach it through the `get_distance_exact` method of binary codes in `src/q
   It is another exact method, selected by `method="connected_support"`, and it also finds light vectors with a given syndrome.
 - **Scan:** the enumeration of the combinations of rows of each information set, layer by layer.
   Counting scans in `_counting/scan.py`.
-  The Brouwer–Zimmermann distance search in `_search/brouwer_zimmermann.py` and the Brouwer–Zimmermann coset search in `_search/coset_leader_search.py` scan with the compiled scans of `_search/affine_search.py`.
-  All of them step through a layer with `advance_support` in `_search/information_sets.py`.
+  The Brouwer–Zimmermann distance search in `_brouwer_zimmermann/distance.py` and the Brouwer–Zimmermann coset searches in `_brouwer_zimmermann/coset.py` scan with the compiled scans of `_brouwer_zimmermann/affine_search.py`.
+  All of them step through a layer with `advance_support` in `_information_sets/support_stepping.py`.
+
+## Layout
+
+- `core.py` and `logical_counts.py` are the public interfaces of the distances and the counts.
+- `_coset_leader_search.py` is the entry module of the coset-leader search, which modules elsewhere in qLDPC import.
+- `_brouwer_zimmermann/` holds the Brouwer–Zimmermann searches, which scan information sets, and `_connected_support/` the searches over connected supports.
+  Each holds its distance search, and the coset searches with which the coset-leader search answers its queries by that method.
+  Exhaustive enumeration fits in one module, `_brute_force.py`.
+- `_counting/` holds the word scan of the counts, the local Clifford gates of deformed counters, and the MacWilliams method.
+- `_information_sets/` holds the families of information sets and their certificates, which the Brouwer–Zimmermann searches and the counts share.
+  The method name `"information_sets"` is an alias of `"brouwer_zimmermann"`, for distances and counts alike.
+- `_cosets.py` holds what the coset searches of both methods share.
+- `_tuning.py` holds the settings of the searches, and `_binary.py` the binary input checks, nested bases, and the symplectic-to-Hamming map.
 
 ## Where to look
 
 | To change… | Look in (this directory) |
 | --- | --- |
 | A distance method, or the validation of method and backend names | `core.py`, and where `logical_counts.py`, `_counting/scan.py`, `get_distance_exact`, `get_distance`, and `CSSCode._get_distance_of_sectors` in `src/qldpc/codes/common.py` dispatch on the method name |
-| The Brouwer–Zimmermann (BZ) distance search | `_search/brouwer_zimmermann.py` |
-| The compiled scans of layers that the BZ distance search and the BZ coset-leader search share, one body specialized by constant flags | `_search/affine_search.py` |
-| The connected-support search, of distances and of vectors with a given syndrome | `_search/connected_support.py` |
-| The packed information sets of a BZ distance search or a BZ coset-leader search, and the certificate of their steps | `_search/affine_instance.py` |
-| How basis families and their certificates are built, and how searches plan their steps: where they stop, and the work of the steps up to there | `_search/information_sets.py` |
-| Automorphism families: the images of one information set under automorphisms of a code, and the check of those automorphisms, which the connected-support search uses too | `_search/automorphism_families.py` |
-| Completion tables, which bound the words that complete the first rows of a BZ support with two more rows: a distance search skips the supports whose words cannot be lighter than its best word, a count those whose words all weigh more than its target weight, and a BZ coset query on a route by tails those whose words all weigh more than its maximum weight. A BZ search for the minimum weight of a coset skips those too, and those whose words cannot be lighter than its lightest word so far. Also the choice of whether to build them | `_search/completion_bounds.py` |
-| Information sets with bounded column coverage, by matroid partitioning | `_search/matroid_partition.py` |
-| Exhaustive enumeration, for small codes, fallbacks, and the `brute_force` method | `_search/brute_force.py` |
-| Minimum weights of syndrome classes of Pauli errors (coset-leader weights), by BZ or by connected-support searches, with two classical searches for the CSS weight on CSS states, used by `src/qldpc/circuits/certification/` and `src/qldpc/experimental/circuits/benchmarking/residual_weights.py` | `_search/coset_leader_search.py` |
-| The families of information sets with which a BZ coset-leader search answers costly queries, the gate that chooses them for each maximum weight, and the routes whose queries weigh words by their tails and skip completions by tables | `_search/coset_routes.py` |
 | The counter, its deformed counters, or `compile_search_setup` | `logical_counts.py` |
+| The queries of the minimum weights of syndrome classes of Pauli errors (coset-leader weights), and how a query splits into coset searches, used by `src/qldpc/circuits/certification/` and `src/qldpc/experimental/circuits/benchmarking/residual_weights.py` | `_coset_leader_search.py` |
+| The Brouwer–Zimmermann (BZ) distance search | `_brouwer_zimmermann/distance.py` |
+| The compiled scans of layers that the BZ distance search and the BZ coset-leader search share, one body specialized by constant flags | `_brouwer_zimmermann/affine_search.py` |
+| The packed information sets of a BZ distance search or a BZ coset-leader search, and the certificate of their steps | `_brouwer_zimmermann/affine_instance.py` |
+| The BZ coset searches of the coset-leader search | `_brouwer_zimmermann/coset.py` |
+| The tables of the steps and tails of BZ coset searches | `_brouwer_zimmermann/coset_tables.py` |
+| The families of information sets with which a BZ coset-leader search answers costly queries, the gate that chooses them for each maximum weight, and the routes whose queries weigh words by their tails and skip completions by tables | `_brouwer_zimmermann/coset_routes.py` |
+| The connected-support search, of distances and of vectors with a given syndrome | `_connected_support/search.py` |
+| The connected-support coset searches of the coset-leader search | `_connected_support/coset.py` |
+| Exhaustive enumeration, for small codes, fallbacks, and the `brute_force` method, with its packed symplectic weights | `_brute_force.py` |
 | The scan of a count, or how its basis families follow local gates | `_counting/scan.py` |
 | The local Clifford gates of deformed counters | `_counting/local_cliffords.py` |
 | The MacWilliams counting method | `_counting/macwilliams.py` |
+| The parts that the coset searches of both methods share: their answers (`MinWeightBound`), words with unit syndromes, packed rows, sums of the rows that a support selects, and tables of the errors of weight zero or one | `_cosets.py` |
+| How basis families are built | `_information_sets/basis_families.py` |
+| Certificates, and how searches plan their steps: where they stop, and the work of the steps up to there | `_information_sets/certificates.py` |
+| The step from one support of a layer to the next, which every scan of layers shares | `_information_sets/support_stepping.py` |
+| Automorphism families: the images of one information set under automorphisms of a code, and the check of those automorphisms, which the connected-support search uses too | `_information_sets/automorphism_families.py` |
+| Completion tables, which bound the words that complete the first rows of a BZ support with two more rows: a distance search skips the supports whose words cannot be lighter than its best word, a count those whose words all weigh more than its target weight, and a BZ coset query on a route by tails those whose words all weigh more than its maximum weight. A BZ search for the minimum weight of a coset skips those too, and those whose words cannot be lighter than its lightest word so far. Also the choice of whether to build them | `_information_sets/completion_bounds.py` |
+| Information sets with bounded column coverage, by matroid partitioning | `_information_sets/matroid_partition.py` |
 | The settings that a search reads when it starts, such as the thresholds that choose between exhaustive enumeration, the kinds of information-set families, and completion tables, the threshold of states before which a connected-support search asks for automorphisms, the coefficients of cost models, memory limits, and batch sizes (`SearchTuning`, `use_tuning`), and the constants that size caches when qLDPC is imported | `_tuning.py` |
-| Binary input checks, nested bases, or packed weights | `_binary.py` |
+| Binary input checks, nested bases, or the map of symplectic vectors to binary vectors of twice their weight | `_binary.py` |
 | Bit packing, bit counts, or row reduction over GF(2), which other subpackages share | `src/qldpc/_binary.py` |
 | How numba code is compiled or cached | `src/qldpc/_compilation.py` |
 
 ## How the modules depend on each other
 
 From the top:
 
-1. `logical_counts.py`, the public interface of the counts, and `core.py`, the public interface of the distances.
-2. The `_counting` package, the counting methods.
-3. `_search/brouwer_zimmermann.py`, `_search/brute_force.py`, `_search/connected_support.py`, and `_search/coset_leader_search.py`, the searches.
-4. The modules that the searches build on.
-   `_search/affine_instance.py` packs the information sets of a BZ distance or BZ coset-leader search, with the certificate of their steps.
-   `_search/coset_routes.py` chooses the families of information sets with which a BZ coset-leader search answers each maximum weight.
-   `_search/affine_search.py` holds the compiled scans of layers that the BZ distance and BZ coset-leader searches share.
-   `_search/information_sets.py` builds the basis families of those two searches and of the counting scans.
-   `_search/automorphism_families.py` builds automorphism families from them.
-   `_search/completion_bounds.py` bounds the words that complete the first rows of a support with two more rows.
-5. `_search/matroid_partition.py`, which `_search/information_sets.py` uses to build some of those families.
-6. `_tuning.py`, the settings of the searches and counts, and the sizes of their caches.
-7. `_binary.py`, the foundation.
-
-A module imports only modules of its own level or below.
-`layering_test.py` checks this for `logical_counts.py`, `core.py`, and the `_counting`, `_search`, `_tuning`, and `_binary` units; the order within `_search/` is a convention.
+1. `logical_counts.py`, which takes `NoDistance` from `core.py`.
+2. `core.py` and `_coset_leader_search.py`.
+3. The methods `_brouwer_zimmermann/`, `_connected_support/`, and `_counting/`.
+4. `_information_sets/`, `_cosets.py`, and `_brute_force.py`, which the methods build on.
+   The connected-support search uses `infer_weight_divisor`, `verify_automorphisms`, and `get_orbits` of `_information_sets/`, and the Brouwer–Zimmermann distance search uses `_brute_force.py` for its exhaustive routes.
+5. `_tuning.py`.
+6. `_binary.py`.
+
+A top-level module or subpackage imports only from lower levels, and a module also from its own subpackage.
+`_layering_test.py` checks this.
+Within a subpackage, the order is a convention.
+In `_information_sets/`, `basis_families` builds on `certificates` and `matroid_partition`, `automorphism_families` on `basis_families` and `certificates`, and `completion_bounds` on `certificates`.
+In `_brouwer_zimmermann/`, `distance` and `coset` build on `affine_instance` and `affine_search`, `coset` also on `coset_routes` and `coset_tables`, and those two on `affine_instance`.
+In `_connected_support/`, `coset` builds on `search`, and in `_counting/`, `scan` on `local_cliffords`.
 Modules of every level may also import `src/qldpc/_binary.py`, the bit-packed linear algebra over GF(2) that other subpackages share.
 `src/qldpc/codes/common.py` imports this package, so `logical_counts.py` imports the code classes of `codes.common` and `codes.quantum` inside the functions that use them.
 Modules elsewhere in qLDPC, such as `src/qldpc/codes/common.py`, modules in `src/qldpc/circuits/certification/`, and the deprecation registry in `src/qldpc/_deprecations/core.py`, use private names of this package, so grep `src/qldpc` for a private name before you move or rename it.
@@ -88,9 +105,9 @@ Section 4.4 of `brouwer_zimmermann.md` says where each heuristic of a Brouwer–
 - A search reads the active tuning once, when it starts, and passes the values that it needs down to the stages and loops that use them.
   The distance functions of `core.py` read it for a distance search, `CSSLogicalCounter.count_logical_ops` for a count, and `CosetLeaderSearch` when it is built; `get_distance_brouwer_zimmermann_many` and `get_distance_connected_support` read it only if their callers pass none.
   Pass these values as arguments, including to compiled code, and do not read the tuning anywhere else, so that one search never mixes two tunings.
-  A value that the code keeps in a cache that outlives a search also keys that cache, as the fields of `_CosetTuning` key the cached searches of `CosetLeaderSearch`.
+  A value that the code keeps in a cache that outlives a search also keys that cache, as the fields of `CosetTuning` key the cached searches of `CosetLeaderSearch`.
 - Only a value that qLDPC fixes before any search starts, such as the size of a cache, which `functools.lru_cache` fixes at import, is a constant in `_tuning.py`, read as `_tuning.NAME`.
-  Constants fixed by the algorithm or by native integer widths, such as `_WORK_CAP`, and the fixed seeds of random generators, which make results reproducible rather than tune anything, stay in the modules that use them.
+  Constants fixed by the algorithm or by native integer widths, such as `WORK_CAP`, and the fixed seeds of random generators, which make results reproducible rather than tune anything, stay in the modules that use them.
 - Document each field and constant in its own `#:` comment directly above it, so that Sphinx renders it.
   Say, in this order, what it decides and in what unit; how a smaller or larger value changes a search, including the effects of its smallest and largest values; how its default was chosen and on what data, or that this is not recorded, in one clause; and what to measure to retune it.
   Keep each comment readable without knowing the method that uses the value, and within a hundred words.