Optional xhatter feasibility cuts, V1 (binary first-stage) — addresses #601

.github/workflows/test_pr_and_main.yml

@@ -116,6 +116,10 @@ jobs:
         run: |
           coverage run $COV_ARGS -m pytest mpisppy/tests/test_xhat_from_file.py -v
 
+      - name: Test xhat feasibility cuts
+        run: |
+          coverage run $COV_ARGS -m pytest mpisppy/tests/test_xhat_feasibility_cuts.py -v
+
       - name: Test feasible_xhat
         run: |
           coverage run $COV_ARGS -m pytest mpisppy/tests/test_feasible_xhat.py -v

doc/src/extensions.rst

@@ -28,6 +28,9 @@ command-line flags:
 - ``--grad-rho`` -- activates gradient-based rho (see :ref:`rho_setting`)
 - ``--use-norm-rho-updater`` -- activates the norm rho updater
 - ``--use-primal-dual-rho-updater`` -- activates the primal-dual rho updater
+- ``--xhat-feasibility-cuts-count <N>`` -- feasibility cuts from the
+  xhatter when its candidate is infeasible; binary first-stage only
+  (see :ref:`xhat_feasibility_cuts`)
 
 The rest of this help file describes extensions released with mpisppy along
 with some hints for including them in your own cylinders driver program.

doc/src/index.rst

@@ -57,6 +57,7 @@ MPI is used.
    jensens.rst
    feasible_xhat.rst
    xhat_from_file.rst
+   xhat_feasibility_cuts.rst
    smps.rst
    agnostic.rst
    generic_admm.rst

doc/src/spokes.rst

@@ -148,7 +148,15 @@ An example is shown in ``examples.hydro.hydro_cylinders.py`` (this particular ex
 is intended to show the coding, not normal behavior. It is sort of an edge case:
 including this option causes the upper bound to immediately be Z*)
 
-
+Feasibility cuts from xhatters
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+For problems without complete recourse, any xhatter can optionally
+emit a no-good feasibility cut when its candidate :math:`\hat{x}` is
+infeasible in some scenario; the hub then installs the cut into every
+scenario so the same :math:`\hat{x}` is not revisited. See
+:ref:`xhat_feasibility_cuts`.
+
 slam_heuristic
 ^^^^^^^^^^^^^^
 

doc/src/xhat_feasibility_cuts.rst

@@ -0,0 +1,163 @@
+.. _xhat_feasibility_cuts:
+
+Xhat Feasibility Cuts
+=====================
+
+For two-stage problems **without complete recourse**, a
+candidate first-stage solution ``xhat`` proposed by an xhatter spoke
+(``xhatlooper``, ``xhatshufflelooper``, ``xhatspecific``,
+``xhatxbar``) can turn out to be infeasible in one or more scenarios.
+Today those candidates are simply discarded and nothing prevents the
+same ``xhat`` from being proposed again a few iterations later.
+
+When this feature is enabled, an xhatter that detects such an
+infeasibility emits a **no-good feasibility cut** on the first-stage
+variables, and the hub installs that cut into every scenario's
+subproblem. The result is that the same ``xhat`` (and any other
+assignment with the same pattern on binaries) is excluded from future
+consideration.
+
+This is a first-milestone implementation: it is **two-stage only**
+and **valid only when every first-stage (nonant) variable is binary**.
+See :ref:`xhat_feas_cuts_boundaries` below.
+
+Enabling the Feature
+--------------------
+
+``generic_cylinders`` exposes a single integer flag:
+
+.. code-block:: bash
+
+   --xhat-feasibility-cuts-count N
+
+where ``N`` is the maximum number of feasibility cuts the xhatter may
+emit per iteration. The default is ``0``, which disables the feature
+entirely. Any positive integer both turns the feature on and sizes the
+shared-memory buffer the xhatter uses to send cuts.
+
+Nothing else needs to be specified. The hub-side installer
+(``mpisppy.extensions.xhat_feasibility_cut_extension.XhatFeasibilityCutExtension``)
+is attached automatically through :ref:`cfg_vanilla <drivers>` when the
+flag is positive.
+
+Example
+-------
+
+.. code-block:: bash
+
+   python -m mpisppy.generic_cylinders \
+       --module-name my_binary_first_stage_model \
+       --num-scens 10 \
+       --solver-name gurobi \
+       --max-iterations 50 \
+       --default-rho 1.0 \
+       --lagrangian \
+       --xhatshuffle \
+       --xhat-feasibility-cuts-count 3
+
+The cap of 3 says "emit at most 3 cuts per xhatter iteration". Cuts
+accumulate across iterations inside each scenario's
+``_mpisppy_model.xhat_feasibility_cuts`` constraint container.
+
+.. _xhat_feas_cuts_boundaries:
+
+Scope and Limitations
+---------------------
+
+The first-milestone cut is the textbook no-good inequality
+
+.. math::
+
+   \sum_{i:\, \hat{x}_i = 1} (1 - x_i) + \sum_{i:\, \hat{x}_i = 0} x_i \;\geq\; 1
+
+which is valid only when every :math:`x_i` is binary. If any
+first-stage nonant is integer (not bounded to :math:`\{0, 1\}`) or
+continuous, the cut cannot exclude the infeasible ``xhat`` correctly,
+so the feature **refuses to run** rather than silently generate
+invalid relaxations:
+
+.. code-block:: text
+
+   RuntimeError: --xhat-feasibility-cuts-count > 0 requires every
+   first-stage (nonant) variable to be binary; found non-binary nonant
+   '<var name>' (key (<node>, <i>)) on scenario '<sname>' with
+   domain <domain>. The first-milestone feasibility-cut generator is
+   no-good-only. Support for integer and continuous first-stage
+   variables is planned as a follow-up milestone (pyomo Benders /
+   Farkas extension).
+
+The error is raised at hub setup time (before any PH work begins),
+so a misconfiguration is caught immediately.
+
+**Integer first-stage variables with bounds** :math:`[0, 1]` **are
+accepted** — semantically those are binary. Declaring a var as
+``pyo.Integers`` with ``bounds=(0, 1)`` works just as well as
+``pyo.Binary``.
+
+Multi-stage
+-----------
+
+V1 is two-stage only. Enabling
+``--xhat-feasibility-cuts-count`` on a multi-stage model raises at
+hub setup time:
+
+.. code-block:: text
+
+   RuntimeError: --xhat-feasibility-cuts-count > 0 is two-stage only
+   in V1. Multi-stage support is planned as a follow-up milestone
+   (the install side needs to group cuts by scenario branch). See
+   doc/designs/xhat_feasibility_cuts_design.md.
+
+The cut row encodes coefficients positionally against each scenario's
+``nonant_indices``. In two-stage every scenario shares the same ROOT
+nonants under nonanticipativity, so applying the same row to every
+scenario is consistent. In multi-stage, scenarios on different
+branches have different per-stage-2+ variables at the deeper indices,
+so the same row applied positionally lands coefficients on unrelated
+variables. A multi-stage-correct installer needs to group cuts by
+branch; that work is deferred to a follow-up milestone.
+
+Interaction with Proper Bundles
+-------------------------------
+
+The hub installer appends cuts to each scenario's
+``xhat_feasibility_cuts`` constraint container. When a scenario
+object is actually a proper bundle, the cut is installed against the
+bundle's canonical nonant set (``s._mpisppy_data.nonant_indices``)
+exactly as the cross-scenario cut machinery does. Nonanticipativity
+inside the bundle ensures the cut takes effect on every per-scenario
+block.
+
+Follow-up Milestones
+--------------------
+
+- **Multi-stage support.** A multi-stage-correct installer needs to
+  group each cut by the branch of the source scenario and install it
+  only on scenarios sharing that branch through the cut's deepest
+  node. The current installer applies one cut row uniformly to every
+  scenario, which is only valid in two-stage; that's why V1 hard-fails
+  at setup on a multi-stage model. Tracking as a follow-up alongside
+  V2.
+- Lifting the binary-only restriction requires generating **Farkas
+  feasibility cuts** from the dual ray of an infeasible second-stage
+  LP. The upstream
+  ``pyomo.contrib.benders.benders_cuts.BendersCutGeneratorData``
+  currently only produces optimality cuts; supporting the
+  infeasibility case is a Pyomo PR. Once that lands, the xhatter will
+  be able to emit valid cuts for integer and continuous first-stage
+  variables as well (LP recourse only).
+- **Cut-pool management** is deferred to
+  `issue #670 <https://github.com/Pyomo/mpi-sppy/issues/670>`_. Today
+  cuts accumulate indefinitely in the per-scenario constraint
+  container, the same way cross-scenario cuts do in
+  ``CrossScenarioExtension``. For long runs, use the
+  ``--xhat-feasibility-cuts-count`` cap to bound the per-iteration
+  growth.
+
+See Also
+--------
+
+- :ref:`Spokes` — overview of the xhat spokes.
+- :ref:`Extensions` — the broader extension mechanism.
+- ``doc/designs/xhat_feasibility_cuts_design.md`` — the design document with
+  the full milestone plan and the ``V1/V2/V3`` scope table.

examples/run_all.py

@@ -217,6 +217,18 @@ def do_one_mmw(dirname, modname, runefstring, npyfile, mmwargstring):
            f"--num-scens=3 --solver-name={solver_name} "
            f"--max-iterations=3 --default-rho=1 --lagrangian --xhatshuffle "
            f"--output-dir=solutions_ws {usar_problem_args}")
+    # usar has a binary first-stage (is_active_depot), so it exercises
+    # --xhat-feasibility-cuts-count end-to-end: buffer registration on
+    # the xhatter spoke, the hub extension's binary-only startup check,
+    # and the sync loop. The xhatter may or may not actually hit an
+    # infeasibility on this small instance — the cut-emission path is
+    # unit-tested separately in test_xhat_feasibility_cuts.py — but this
+    # smoke entry guards the full-run plumbing from regressions.
+    do_one("usar", "wheel_spinner.py", 3,
+           f"--num-scens=3 --solver-name={solver_name} "
+           f"--max-iterations=3 --default-rho=1 --lagrangian --xhatshuffle "
+           f"--xhat-feasibility-cuts-count=3 "
+           f"--output-dir=solutions_ws_feas {usar_problem_args}")
 
     # netdes
     # NOTE: Pyomo OBBT does not support persistent solvers as of Aug 2025

examples/usar/wheel_spinner.py

@@ -70,6 +70,9 @@ def wheel_spinner(
     vanilla_fn = vanilla.aph_hub if cnfg["run_async"] else vanilla.ph_hub
     hub_dict = vanilla_fn(*vanilla_args, **vanilla_kwargs)
 
+    # Optional feasibility cuts from xhatters (requires binary first-stage).
+    hub_dict = vanilla.add_xhat_feasibility_cuts(hub_dict, cnfg)
+
     spoke_dicts = []
     for spoke_name in SUPPORTED_SPOKES:
         if getattr(cnfg, spoke_name):
@@ -90,6 +93,7 @@ def main() -> None:
     cnfg.aph_args()
     for spoke_name in SUPPORTED_SPOKES:
         getattr(cnfg, spoke_name + "_args")()
+    cnfg.xhat_feasibility_cut_args()
     add_common_args(cnfg)
     cnfg.parse_command_line()
 

mpisppy/cylinders/spwindow.py

@@ -36,6 +36,7 @@ class Field(enum.IntEnum):
     BEST_XHAT=600 # buffer having the best xhat and its total cost per scenario
     RECENT_XHATS=601 # buffer having some recent xhats and their total cost per scenario
     XFEAS=602 # buffer having a feasible x and its total cost per scenario
+    XHAT_FEASIBILITY_CUT=700  # feasibility cuts emitted by xhat spokes
     WHOLE=1_000_000
 
 
@@ -47,6 +48,9 @@ class Field(enum.IntEnum):
 
 # these could be modified by the user...
 field_length_components.total_number_recent_xhats = pyo.Param(mutable=True, initialize=10, within=pyo.NonNegativeIntegers)
+# max cuts an xhat spoke may emit per iteration (set by the xhatter
+# spoke from cfg.xhat_feasibility_cuts_count before field registration)
+field_length_components.xhat_feasibility_cuts_per_iter = pyo.Param(mutable=True, initialize=0, within=pyo.NonNegativeIntegers)
 
 _field_lengths = {
         Field.SHUTDOWN : 1,
@@ -65,6 +69,9 @@ class Field(enum.IntEnum):
         Field.BEST_XHAT : field_length_components._local_nonant_length + field_length_components._local_scenario_length,
         Field.RECENT_XHATS : field_length_components.total_number_recent_xhats * (field_length_components._local_nonant_length + field_length_components._local_scenario_length),
         Field.XFEAS: field_length_components._local_nonant_length + field_length_components._local_scenario_length,
+        # rows: [constant, nonant_coef_1, ..., nonant_coef_N]; trailing slot holds the
+        # actual number of cuts written this batch (0..per_iter).
+        Field.XHAT_FEASIBILITY_CUT : field_length_components.xhat_feasibility_cuts_per_iter * (field_length_components._total_number_nonants + 1) + 1,
         }
 
 
@@ -81,6 +88,10 @@ def __init__(self, opt):
         field_length_components._local_scenario_length.value = len(opt.local_scenarios)
         field_length_components._total_number_nonants.value = opt.nonant_length
         field_length_components._total_number_scenarios.value = len(opt.local_scenarios)
+        # user-tunable cap on feasibility cuts per iteration (0 = off)
+        field_length_components.xhat_feasibility_cuts_per_iter.value = int(
+            opt.options.get("xhat_feasibility_cuts_count", 0)
+        )
 
         self._field_lengths = {k : pyo.value(v) for k, v in _field_lengths.items()}
 

mpisppy/cylinders/xhatbase.py

@@ -12,10 +12,18 @@
 import math
 
 import mpisppy.cylinders.spoke as spoke
+from mpisppy.cylinders.spwindow import Field
 from mpisppy.utils.xhat_eval import Xhat_Eval
 
 class XhatInnerBoundBase(spoke.InnerBoundNonantSpoke):
 
+    # Advertise the optional feasibility-cut send buffer. When
+    # cfg.xhat_feasibility_cuts_count == 0 the buffer is size 1 (just the
+    # trailing count slot) and never written; see spwindow.FieldLengths
+    # and XhatBase._try_one.
+    send_fields = (*spoke.InnerBoundNonantSpoke.send_fields,
+                   Field.XHAT_FEASIBILITY_CUT,)
+
     @abc.abstractmethod
     def xhat_extension(self):
         raise NotImplementedError
@@ -98,7 +106,32 @@ def _try_file_xhat(self):
         if self.cylinder_rank == 0:
             print(f"[xhat-from-file] evaluating {path!r} "
                   f"({expected} nonants)")
-        Eobj = self.opt.evaluate(nonant_cache)
+        try:
+            Eobj = self.opt.evaluate(nonant_cache)
+        except Exception:
+            Eobj = None
+        # Same predicate XhatBase._try_one uses to detect infeasibility
+        # in some scenario. When True and feasibility cuts are enabled,
+        # pack a no-good cut so the hub extension can install it on
+        # every scenario — same path the regular xhatter takes.
+        infeasP = 0.0
+        try:
+            infeasP = self.opt.no_incumbent_prob()
+        except Exception:
+            pass
+        if infeasP != 0.:
+            from mpisppy.extensions.xhatbase import pack_no_good_feasibility_cut
+            try:
+                emitted = pack_no_good_feasibility_cut(self.opt)
+            except Exception as e:
+                emitted = False
+                if self.cylinder_rank == 0:
+                    print(f"[xhat-from-file] feasibility-cut emit failed: {e!r}")
+            if self.cylinder_rank == 0:
+                tag = "emitted" if emitted else "skipped"
+                print(f"[xhat-from-file] candidate infeasible "
+                      f"(infeasP={infeasP}); feasibility cut {tag}")
+            Eobj = None
         # Restore nonants so the main loop starts from clean state.
         self.opt._restore_nonants()
         if Eobj is not None and math.isfinite(Eobj):