Skip to content

Use maps for wildcard matching#45

Merged
wranders merged 1 commit into
wranders:masterfrom
bluekeyes:wildcard-maps
Jun 8, 2026
Merged

Use maps for wildcard matching#45
wranders merged 1 commit into
wranders:masterfrom
bluekeyes:wildcard-maps

Conversation

@bluekeyes

Copy link
Copy Markdown
Contributor

Changes

Improve performance by using maps to implement wildcard matching. Wildcards behave the same as before, but matching is at least two orders of magnitude faster with large wildcard lists.

Justification

The previous implementation of wildcard matching iterated through the full list of wildcards looking for one that matched the query name. This made query performance depend on the number of list entries, which is a problem for large lists. By switching to map lookups, query performance now depends only on the number of subdomains in the query.

The old implementation also performed string concatenation as part of each wildcard entry check, requiring allocations that hurt performance. The following benchmark results show the difference between the original implementation (Linear), the original implementation without the allocations (LinearNoAlloc), and the new version (Map):

goos: linux
goarch: amd64
pkg: github.com/wranders/coredns-filter
cpu: 12th Gen Intel(R) Core(TM) i7-12700K
BenchmarkWildcardListExternal_Linear-20            624  1937816 ns/op  77500 B/op  1566 allocs/op
BenchmarkWildcardListExternal_LinearNoAlloc-20    3612   327248 ns/op   2237 B/op    46 allocs/op
BenchmarkWildcardListExternal_Map-20            458190     2600 ns/op   2236 B/op    46 allocs/op

Removing the allocations improves performance by >5x while switching to the map is a >700x improvement. These tests used the committed "small" OISD list with ~47k entries. The "big" list currently contains 487k entries, so I expect the improvement is larger as the list size grows.

Note that the performance of the old version hasn't caused any practical problems for me yet. Rather, while evaluating blocklist/blackhole/sinkhole plugins for CoreDNS, I spotted this optimization. Your plugin otherwise seems to be one of the most flexible and feature-complete versions, so thanks for releasing and maintaining it!

The previous implementation of wildcard matching iterated through the
full list of wildcards looking for one that matched the query name. This
made query performance depend on the number of list entries, which is a
problem for large lists. By switching to map lookups, query performance
now depends only on the number of subdomains in the query.

The old implementation also performed string concatenation as part of
each wildcard entry check, requiring allocations that hurt performance.
The following benchmark results show the difference between the original
implementation (Linear), the original implementation without the
allocations (LinearNoAlloc), and the new version (Map):

  goos: linux
  goarch: amd64
  pkg: github.com/wranders/coredns-filter
  cpu: 12th Gen Intel(R) Core(TM) i7-12700K
  BenchmarkWildcardListExternal_Linear-20            624  1937816 ns/op  77500 B/op  1566 allocs/op
  BenchmarkWildcardListExternal_LinearNoAlloc-20    3612   327248 ns/op   2237 B/op    46 allocs/op
  BenchmarkWildcardListExternal_Map-20            458190     2600 ns/op   2236 B/op    46 allocs/op

Removing the allocations improves performance by >5x while switching to
the map is a >700x improvement. These tests used the committed "small"
OISD list with ~47k entries. The "big" list currently contains 487k
entries, so I expect the improvement is larger as the list size grows.
@wranders

wranders commented Jun 8, 2026

Copy link
Copy Markdown
Owner

Looks good, thanks for putting this together.

  1. Not sure why I thought moving wildcards to a slice instead of using the map they were already in would be better. I think it was probably a panic-based decision since the old regex-based implementation was just a network DoS disguised as a DNS server.

  2. Evaluating wildcards before regex is such an obvious improvement, I can't believe I didn't see that before. This is why I love open source collabs.

Merging. A new tag and release should be put out soon-ish.

@wranders wranders merged commit f7d35e9 into wranders:master Jun 8, 2026
1 check failed
@bluekeyes bluekeyes deleted the wildcard-maps branch June 8, 2026 12:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants