Skip to content

[medium] Fix KeyError when the configured decaying model is not found - #60

Open
elhoim wants to merge 1 commit into
MISP:mainfrom
elhoim:fix/decaying-model-not-found-keyerror
Open

elhoim wants to merge 1 commit into
MISP:mainfrom
elhoim:fix/decaying-model-not-found-keyerror

Conversation

@elhoim

@elhoim elhoim commented Sep 17, 2026

Copy link
Copy Markdown
Member

BLUF

  • Priority: medium.
  • pb_curate_disable_decayed_indicators crashes with KeyError: 'attributes' whenever curation_decaying_model names a model that is missing or disabled.
  • Root cause: playbook_results["decaying_model_meta"] is initialised as an empty dict but indexed ["attributes"] unconditionally.
  • Effect: the two friendly "decaying model not found" messages are dead code in exactly the case they were written for.
  • Fix: one line per notebook — initialise the entry as {"attributes": []}.
  • Both notebooks of the pair are changed so the clean and executed variants stay in sync.
  • No README.md or playbook.json change: this is a bug fix, not a new playbook.

What was wrong

The playbook-configuration cell sets up the results container with an empty entry:

playbook_results = {"events": {},
                    "attributes": {},
                    "decaying_model_meta": {},
                    "timestamps": {}
                    }

The decaying-model cell only fills that entry inside a conditional — when the
model index contains a model whose name matches curation_decaying_model and
whose enabled flag is True:

if model["DecayingModel"]["name"] == playbook_config["curation_decaying_model"] and model["DecayingModel"]["enabled"] == True:
    playbook_results["decaying_model_meta"] = {"description": ..., "lifetime": ..., "attributes": []}

Immediately afterwards, and again in the "Apply the model" block, the key is read
unconditionally:

if len(playbook_results["decaying_model_meta"]["attributes"]) > 0:

Concrete impact

Point curation_decaying_model at a model that does not exist on the MISP server,
or at one that exists but is disabled, and the cell dies on the first of those two
lines with KeyError: 'attributes' instead of taking its else branch. The user
gets a traceback rather than the intended message naming the model that could not
be found, and both else branches — the very ones written to handle this case —
are unreachable.

What this changes

The results container now declares the shape it is actually used with:

                    "decaying_model_meta": {"attributes": []},

That is the whole change, applied identically to
pb_curate_disable_decayed_indicators.ipynb and
pb_curate_disable_decayed_indicators-with_output.ipynb so the pair stays in
sync. The successful path is untouched: the matching-model branch still replaces
the entire dict, attributes list included. The stored outputs in the executed
notebook cover the found path and needed no update.

How this was verified

The playbook-configuration cell and the decaying-model cell were extracted from
both notebooks and executed against a stub whose decayingModel/index.json
returns an empty list, i.e. the "model not found" case. The misp.search() call
sits inside the guarded branch and never fires, so nothing contacts a server.

  • Before: KeyError: 'attributes', in both notebooks.
  • After: MISP decaying modle curation not found. followed by
    MISP decaying model curation not found. and
    Finished searching. Processed 0 attributes, decayed 0 attributes., in both
    notebooks.

Notebook and JSON structural validation across the repository still passes, so the
nbconvert workflow is unaffected.


Local verification

cd /home/elhoim/tmp/misp-review/mwork/misp-playbooks-1 && python3 - <<'PYEOF'
import json, sys
class R:
    def json(self): return []
class M:
    def _prepare_request(self, *a, **k): return R()
ok = True
for path in ("misp-playbooks/pb_curate_disable_decayed_indicators.ipynb",
             "misp-playbooks/pb_curate_disable_decayed_indicators-with_output.ipynb"):
    nb = json.load(open(path)); ns = {"misp": M(), "org_list": []}
    print("---", path)
    try:
        for c in nb["cells"]:
            if c["cell_type"] == "code" and "decaying_model_meta" in "".join(c["source"]):
                exec(compile("".join(c["source"]), path, "exec"), ns)
        print("   OK: model-not-found path completed without KeyError")
    except KeyError as e:
        ok = False; print("   KeyError:", e)
sys.exit(0 if ok else 1)
PYEOF

Output:

--- misp-playbooks/pb_curate_disable_decayed_indicators.ipynb
MISP decaying modle curation not found.

Searching for decayed attributes based on model curation ...
MISP decaying model curation not found.
Finished searching. Processed 0 attributes, decayed 0 attributes.
   OK: model-not-found path completed without KeyError
--- misp-playbooks/pb_curate_disable_decayed_indicators-with_output.ipynb
MISP decaying modle curation not found.

Searching for decayed attributes based on model curation ...
MISP decaying model curation not found.
Finished searching. Processed 0 attributes, decayed 0 attributes.
   OK: model-not-found path completed without KeyError
EXIT=0
(ANSI colour escapes omitted above.)
  • Verified on a scratch work tree with the patch applied.
  • The same check was re-run with the change stashed and fails on unpatched code, so it discriminates rather than merely passing.

In pb_curate_disable_decayed_indicators, playbook_results holds
"decaying_model_meta" as an empty dict and only fills it when the
model named by curation_decaying_model is found and enabled. The
two statements that follow index ["attributes"] unconditionally,
so pointing the playbook at a missing or disabled model raises
KeyError: 'attributes' and both "model not found" else-branches
become unreachable.

Initialise the entry as {"attributes": []} so the not-found case
reaches its message. The matching-model branch reassigns the whole
dict, so the successful path is unchanged.

Applied to both notebooks of the pair to keep them in sync.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant