netfilter 1.64 watch() silently fails to process F2B_CHANNEL messages on Docker 29 / nft host

[LouisMylle]

Contribution guidelines

• I've read the contribution guidelines and wholeheartedly agree

Checklist prior issue creation

• I understand that failure to follow below instructions may cause this issue to be closed.
• I understand that vague, incomplete or inaccurate information may cause this issue to be closed.
• I understand that this form is intended solely for reporting software bugs and not for support-related inquiries.
• I understand that all responses are voluntary and community-driven, and do not constitute commercial support.
• I confirm that I have reviewed previous issues to ensure this matter has not already been addressed.
• I confirm that my environment meets all prerequisite requirements as specified in the official documentation.

Description

netfilter 1.64 watch() thread silently fails to process F2B_CHANNEL messages

The mailcow netfilter container starts cleanly and reports Watching Redis channel F2B_CHANNEL, but the watch() thread silently swallows every message. No regex match logging, no F2B_QUEUE update, no ban applied — even with 11+ valid attempts from a public IP that matches an F2B_REGEX rule perfectly.

Verification that pubsub itself works

A second Python subscriber instantiated inside the same netfilter container, against the same Redis instance (172.22.1.249:6379, same password, same db), receives F2B_CHANNEL messages immediately:

docker compose exec -T netfilter-mailcow python3 <<'EOF'
import redis, os, time
r = redis.StrictRedis(host='172.22.1.249', port=6379, password=os.environ['REDISPASS'], decode_responses=True)
ps = r.pubsub(); ps.subscribe('F2B_CHANNEL')
print(r.execute_command('PUBSUB','NUMSUB','F2B_CHANNEL'))  # ['F2B_CHANNEL', 2]
end = time.time()+20
while time.time() < end:
    msg = ps.get_message(timeout=1.0)
    if msg: print(msg)
EOF

Output (with 11 publishes sent during the listen window):
['F2B_CHANNEL', 2] # netfilter + this subscriber
GOT [message]: 'mailcow UI: Invalid password for testuser by 198.18.0.66'
GOT [message]: 'mailcow UI: Invalid password for testuser by 198.18.0.66'
... etc

Verification that the regex matches the test message

Same Python interpreter, against same Redis-stored regex:

patterns = json.loads(r.get('F2B_REGEX'))
test = 'mailcow UI: Invalid password for testuser by 198.18.0.66'
# Result: Rule 5: MATCH 198.18.0.66

So the listener thread is alive but not processing

• Threads: 7 confirmed in /proc/1/status
• ls /proc/1/task/ returns 7 thread IDs
• INFO: Watching Redis channel F2B_CHANNEL logged immediately on startup
• Yet no '%s matched rule id %s' log line ever appears, even with LOG_LEVEL=debug
• F2B_QUEUE remains empty
• F2B_ACTIVE_BANS remains empty
• MAILCOW chain only contains the isolation rule, never any ban entries

The pubsub.listen() loop in main.py line 262 either never iterates, or iterates and silently catches every message without taking action.

Environment caveats

• Host runs native nftables alongside iptables-nft shim (Docker default on Ubuntu 24.04). iptables -L returns "table filter is incompatible, use nft tool" — output below is via nft list table ip filter.
• /etc/nftables.conf has custom ip raw prerouting rules (port whitelisting and drops for a secondary host IP). These do not touch the ip filter table that netfilter manages.
• Docker 29.3.1 — newer than mailcow's typical test matrix. Cannot rule out a Docker version interaction.

Despite the non-standard firewall setup:

• pubsub publishes confirmed reaching subscribers (verified with parallel Python subscriber inside the same netfilter container)
• regex matching confirmed working (verified standalone with the exact F2B_REGEX from Redis)
• subscription confirmed active (PUBSUB NUMSUB F2B_CHANNEL returns 1 with only netfilter running)
• only the watch() loop's processing is silently broken

Steps to reproduce:

1. Fresh docker compose restart netfilter-mailcow
2. Confirm INFO: Watching Redis channel F2B_CHANNEL appears in logs
3. Verify subscriber is registered: redis-cli PUBSUB NUMSUB F2B_CHANNEL → returns 1
4. Send 11 publishes that match F2B_REGEX rule 5:

   for i in {1..11}; do
     docker compose exec redis-mailcow redis-cli -a "$REDISPASS" \
       PUBLISH F2B_CHANNEL "mailcow UI: Invalid password for testuser by 198.18.0.66"
     sleep 0.5
   done

5. Observe: netfilter logs nothing about matching/banning. F2B_QUEUE, F2B_ACTIVE_BANS, and MAILCOW chain remain unchanged.
6. Run a second Python subscriber inside the same container — it receives every message just fine, proving the issue is specific to netfilter's watch() loop.

Logs:

Full netfilter startup logs (clean, every restart looks like this):
2026-05-02 13:33:09 INFO: Clearing all bans
2026-05-02 13:33:09 INFO: Clear completed: ip
2026-05-02 13:33:10 INFO: Using NFTables backend
2026-05-02 13:33:10 INFO: Clearing all bans
2026-05-02 13:33:10 INFO: Initializing mailcow netfilter chain
2026-05-02 13:33:10 INFO: MAILCOW ip chain created successfully.
2026-05-02 13:33:10 INFO: Setting MAILCOW isolation
2026-05-02 13:33:10 INFO: Watching Redis channel F2B_CHANNEL

After 11 publishes that should trigger rule 5: no further log lines from netfilter at all. Complete silence.

Which branch are you using?

master (stable)

Which architecture are you using?

x86_64

Operating System:

Ubuntu 24.04.4 LTS

Server/VM specifications:

8 GB / 4 vCPU

Is Apparmor, SELinux or similar active?

yes (apparmor, 120 profiles loaded)

Virtualization technology:

KVM

Docker version:

29.3.1

docker-compose version or docker compose version:

v2.39.4

mailcow version:

2026-03b

Reverse proxy:

none

Logs of git diff:

No netfilter-related modifications. Only modification is in
data/assets/ssl-example/* (replaced with a self-generated example cert,
unrelated to this issue).

Logs of iptables -L -vn:

Note: iptables -L errors with "table filter is incompatible, use nft tool"
on this host. Equivalent output via nft list table ip filter,
with public IPs and admin allowlist IPs redacted:
Warning: table ip filter is managed by iptables-nft, do not touch!
table ip filter {
chain INPUT {
type filter hook input priority filter; policy accept;
counter packets 2590 bytes 947677 jump MAILCOW comment "mailcow"
ip saddr <ADMIN_IP_1> ip daddr <PRIMARY_IP> tcp dport 8081 counter packets 0 bytes 0 accept
ip saddr <ADMIN_IP_2> ip daddr <PRIMARY_IP> tcp dport 8081 counter packets 0 bytes 0 accept
ip saddr <ADMIN_IP_3> ip daddr <PRIMARY_IP> tcp dport 8081 counter packets 3 bytes 180 accept
iifname "lo" counter packets 1349853 bytes 3117913422 accept
ct state related,established counter packets 17817977 bytes 6512983952 accept
tcp dport 22 counter packets 211428 bytes 13973863 accept
ip daddr <PRIMARY_IP> tcp dport 25 counter packets 141 bytes 7512 accept
ip daddr <PRIMARY_IP> tcp dport 53 counter packets 25 bytes 1002 accept
ip daddr <PRIMARY_IP> tcp dport 80 counter packets 486426 bytes 25318151 accept
ip daddr <PRIMARY_IP> tcp dport 110 counter packets 3 bytes 140 accept
ip daddr <PRIMARY_IP> tcp dport 143 counter packets 130 bytes 8430 accept
ip daddr <PRIMARY_IP> tcp dport 443 counter packets 605011 bytes 32655143 accept
ip daddr <PRIMARY_IP> tcp dport 465 counter packets 37 bytes 2055 accept
ip daddr <PRIMARY_IP> tcp dport 587 counter packets 75 bytes 3736 accept
ip daddr <PRIMARY_IP> tcp dport 993 counter packets 311 bytes 21975 accept
ip daddr <PRIMARY_IP> tcp dport 995 counter packets 17 bytes 816 accept
ip daddr <PRIMARY_IP> tcp dport 3000 counter packets 1382 bytes 63540 accept
ip daddr <PRIMARY_IP> tcp dport 4190 counter packets 4 bytes 160 accept
ip daddr <PRIMARY_IP> udp dport 53 counter packets 456 bytes 29494 accept
ip daddr <PRIMARY_IP> udp dport 51820 counter packets 126 bytes 21108 accept
ip saddr <ADMIN_HOME> ip daddr <PRIMARY_IP> tcp dport 8000 counter packets 1 bytes 64 accept
ip saddr <ADMIN_OFFICE> ip daddr <PRIMARY_IP> tcp dport 8000 counter packets 0 bytes 0 accept
ip daddr <PRIMARY_IP> tcp dport 8000 counter packets 22 bytes 1564 drop
ip saddr <ADMIN_HOME> ip daddr <PRIMARY_IP> tcp dport 8081 counter packets 0 bytes 0 accept
ip saddr <ADMIN_OFFICE> ip daddr <PRIMARY_IP> tcp dport 8081 counter packets 0 bytes 0 accept
ip daddr <PRIMARY_IP> tcp dport 8081 counter packets 32 bytes 1447 drop
ip daddr <PRIMARY_IP> counter packets 1036491 bytes 265979447 drop
ip daddr <SECONDARY_IP> counter packets 92649 bytes 5782053 drop
}
chain FORWARD {
type filter hook forward priority filter; policy accept;
counter packets 8932 bytes 4595022 jump MAILCOW comment "mailcow"
counter packets 110501 bytes 56176272 jump DOCKER-FORWARD
counter packets 68642371 bytes 42216120309 jump DOCKER-USER
}

chain OUTPUT {
type filter hook output priority filter; policy accept;
}

chain DOCKER-FORWARD {
counter packets 110501 bytes 56176272 jump DOCKER-CT
counter packets 60266 bytes 15166117 jump DOCKER-INTERNAL
counter packets 60266 bytes 15166117 jump DOCKER-BRIDGE
iifname "br-39b45ca31e3c" counter packets 25639 bytes 5000163 accept
iifname "br-mailcow" counter packets 18394 bytes 5128728 accept
iifname "br-8d47601a8250" counter packets 9409 bytes 4520779 accept
iifname "br-013464e7c4ca" counter packets 0 bytes 0 accept
iifname "docker0" counter packets 0 bytes 0 accept
}

chain DOCKER-USER {
ip saddr <ADMIN_IP_1> ip daddr <PRIMARY_IP> tcp dport 8081 counter packets 0 bytes 0 return
ip saddr <ADMIN_IP_2> ip daddr <PRIMARY_IP> tcp dport 8081 counter packets 0 bytes 0 return
ct original ip daddr <SECONDARY_IP> counter packets 0 bytes 0 drop
ip saddr <ADMIN_IP_3> ip daddr <PRIMARY_IP> tcp dport 8081 counter packets 0 bytes 0 return
ip saddr <ADMIN_HOME> ip daddr <PRIMARY_IP> tcp dport 8000 counter packets 0 bytes 0 return
ip saddr <ADMIN_OFFICE> ip daddr <PRIMARY_IP> tcp dport 8000 counter packets 0 bytes 0 return
ip daddr <PRIMARY_IP> tcp dport 8000 counter packets 0 bytes 0 drop
ip saddr <ADMIN_HOME> ip daddr <PRIMARY_IP> tcp dport 8081 counter packets 0 bytes 0 return
ip saddr <ADMIN_OFFICE> ip daddr <PRIMARY_IP> tcp dport 8081 counter packets 0 bytes 0 return
ip daddr <PRIMARY_IP> tcp dport 8081 counter packets 0 bytes 0 drop
counter packets 68649270 bytes 42217488834 return
}

chain DOCKER {
ip daddr 172.22.1.250 iifname != "br-mailcow" oifname "br-mailcow" tcp dport 12345 counter packets 0 bytes 0 accept
ip daddr 172.22.1.250 iifname != "br-mailcow" oifname "br-mailcow" tcp dport 4190 counter packets 1 bytes 60 accept
ip daddr 172.22.1.250 iifname != "br-mailcow" oifname "br-mailcow" tcp dport 995 counter packets 7 bytes 420 accept
ip daddr 172.22.1.250 iifname != "br-mailcow" oifname "br-mailcow" tcp dport 993 counter packets 141 bytes 9358 accept
ip daddr 172.22.1.250 iifname != "br-mailcow" oifname "br-mailcow" tcp dport 143 counter packets 26 bytes 1628 accept
ip daddr 172.22.1.250 iifname != "br-mailcow" oifname "br-mailcow" tcp dport 110 counter packets 1 bytes 40 accept
ip daddr 172.22.1.249 iifname != "br-mailcow" oifname "br-mailcow" tcp dport 6379 counter packets 0 bytes 0 accept
ip daddr 172.22.1.253 iifname != "br-mailcow" oifname "br-mailcow" tcp dport 587 counter packets 2 bytes 112 accept
ip daddr 172.22.1.253 iifname != "br-mailcow" oifname "br-mailcow" tcp dport 465 counter packets 10 bytes 616 accept
ip daddr 172.22.1.253 iifname != "br-mailcow" oifname "br-mailcow" tcp dport 25 counter packets 11 bytes 636 accept
ip daddr 172.22.1.6 iifname != "br-mailcow" oifname "br-mailcow" tcp dport 8443 counter packets 0 bytes 0 accept
ip daddr 172.22.1.6 iifname != "br-mailcow" oifname "br-mailcow" tcp dport 8080 counter packets 0 bytes 0 accept
ip daddr 172.22.1.2 iifname != "br-mailcow" oifname "br-mailcow" tcp dport 3306 counter packets 0 bytes 0 accept
iifname != "br-mailcow" oifname "br-mailcow" counter packets 0 bytes 0 drop
iifname != "docker0" oifname "docker0" counter packets 0 bytes 0 drop
(unrelated docker bridges trimmed for brevity)
}

chain DOCKER-BRIDGE {
oifname "br-mailcow" counter packets 199 bytes 12870 jump DOCKER
(unrelated docker bridges trimmed for brevity)
}

chain DOCKER-CT {
oifname "br-mailcow" ct state related,established counter packets 16740 bytes 10620221 accept
(unrelated docker bridges trimmed for brevity)
}

chain DOCKER-INTERNAL {
}

chain MAILCOW {
iifname != "br-mailcow" oifname "br-mailcow" tcp dport { 3306, 6379, 8983, 12345 } counter packets 0 bytes 0 drop comment "mailcow isolation"
}
}

Logs of ip6tables -L -vn:

IPv6 disabled at Docker level (ip6tables: false in /etc/docker/daemon.json)
and at kernel level (net.ipv6.conf.all.disable_ipv6=1).
No ip6 nft tables exist:
nft list table ip6 filter
Error: No such file or directory

Logs of iptables -L -vn -t nat:

Same compatibility issue. Equivalent via nft list table ip nat,
with public IP redacted:
Warning: table ip nat is managed by iptables-nft, do not touch!
table ip nat {
chain PREROUTING {
type nat hook prerouting priority dstnat; policy accept;
fib daddr type local counter packets 10382 bytes 1157434 jump DOCKER
}
chain OUTPUT {
type nat hook output priority dstnat; policy accept;
ip daddr != 127.0.0.0/8 fib daddr type local counter packets 0 bytes 0 jump DOCKER
}

chain POSTROUTING {
type nat hook postrouting priority srcnat; policy accept;
ip saddr 172.22.1.0/24 ip daddr != 172.22.1.0/24 counter packets 6985 bytes 541835 snat to <PRIMARY_IP> comment "mailcow"
ip saddr 172.17.0.0/16 oifname != "docker0" counter packets 0 bytes 0 masquerade
ip saddr 172.22.1.0/24 oifname != "br-mailcow" counter packets 7 bytes 454 masquerade
oifname "eth0" snat to <PRIMARY_IP>
(unrelated docker bridge masquerade rules trimmed for brevity)
}

chain DOCKER {
ip daddr 127.0.0.1 iifname != "br-mailcow" tcp dport 13306 counter packets 0 bytes 0 dnat to 172.22.1.2:3306
ip daddr 127.0.0.1 iifname != "br-mailcow" tcp dport 8080 counter packets 0 bytes 0 dnat to 172.22.1.6:8080
ip daddr 127.0.0.1 iifname != "br-mailcow" tcp dport 8443 counter packets 0 bytes 0 dnat to 172.22.1.6:8443
iifname != "br-mailcow" tcp dport 25 counter packets 11 bytes 636 dnat to 172.22.1.253:25
iifname != "br-mailcow" tcp dport 465 counter packets 10 bytes 616 dnat to 172.22.1.253:465
iifname != "br-mailcow" tcp dport 587 counter packets 2 bytes 112 dnat to 172.22.1.253:587
ip daddr 127.0.0.1 iifname != "br-mailcow" tcp dport 7654 counter packets 0 bytes 0 dnat to 172.22.1.249:6379
iifname != "br-mailcow" tcp dport 110 counter packets 1 bytes 40 dnat to 172.22.1.250:110
iifname != "br-mailcow" tcp dport 143 counter packets 26 bytes 1628 dnat to 172.22.1.250:143
iifname != "br-mailcow" tcp dport 993 counter packets 141 bytes 9358 dnat to 172.22.1.250:993
iifname != "br-mailcow" tcp dport 995 counter packets 7 bytes 420 dnat to 172.22.1.250:995
iifname != "br-mailcow" tcp dport 4190 counter packets 1 bytes 60 dnat to 172.22.1.250:4190
ip daddr 127.0.0.1 iifname != "br-mailcow" tcp dport 19991 counter packets 0 bytes 0 dnat to 172.22.1.250:12345
(unrelated dnat rules for other docker bridges trimmed for brevity)
}
}

Logs of ip6tables -L -vn -t nat:

IPv6 disabled at Docker level (ip6tables: false in /etc/docker/daemon.json)
and at kernel level (net.ipv6.conf.all.disable_ipv6=1).
No ip6 nft tables exist.

DNS check:

docker exec -it $(docker ps -qf name=acme-mailcow) dig +short stackoverflow.com @172.22.1.254
198.252.206.1

[milkmaker]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.