Implement HalFilters via RI injection#132
Conversation
| ret = "/".join(ib[name] + [dest]) if name in ib else dest | ||
|
|
||
| ob = fl.get("outbound", {}) | ||
| ret = "/".join(ob[self.name] + [ret]) if self.name in ob else ret |
There was a problem hiding this comment.
I'm growing increasingly displeased with verbatim string matching for RIs. Starting to think it might be worth adding an abstraction to handle RI comparison. Something where a check for irc will pass for both irc and irc/##halibot, with the former referring only to the agent (general case), the latter referring to a specific resource to/from irc.
As it stands, it occurred to me that this doesn't support per-channel filters, which might be useful for limiting output to a specific channel/user, rather than for all resources behind a particular agent. I can hack in support, but there's a deeper issue here I feel.
There was a problem hiding this comment.
I agree, we really shouldn't be spliting/joining on "/" everywhere. It definitely should be abstracted somewhere.
There was a problem hiding this comment.
Should we open a new issue to track this? I think this is fine for now, but we probably want some way to allow per-channel filtering as well.
| ret = "/".join(ib[name] + [dest]) if name in ib else dest | ||
|
|
||
| ob = fl.get("outbound", {}) | ||
| ret = "/".join(ob[self.name] + [ret]) if self.name in ob else ret |
There was a problem hiding this comment.
I agree, we really shouldn't be spliting/joining on "/" everywhere. It definitely should be abstracted somewhere.
| name = dest.split("/")[0] | ||
|
|
||
| ib = fl.get("inbound", {}) | ||
| ret = "/".join(ib[name] + [dest]) if name in ib else dest |
There was a problem hiding this comment.
ret = "/".join(ib.get(name, []) + [dest]) is shorter but maybe more opaque
There was a problem hiding this comment.
Ahhhh this was the one liner I knew was in there somewhere. Opague, but cleaner.
7b19468 to
867c0c5
Compare
|
Update: cleaned up the |
867c0c5 to
efcad66
Compare
efcad66 to
ce7d959
Compare
Functions like .send_to() really should be a helper function that sets the target of a message for you (and handles multiple targets), but it shouldn't be the bottom sending function itself. This commit moves the actual send logic (including copy) out of the .send_to function, and into its own function called .raw_send(). A `Message` object parameter MUST be populated with a valid `.target` and `.origin` parameter.
Problem: How can we introduce rate limiting, self-censoring, or other kind of
logic into Halibot instances without adding this to each module or agent?
This patch introduces a new Halibot object type "HalFilter". These filters can
be inserted in between a normal message send path, to silently drop or modify
message content. These filters are NOT meant for large behaviors.
Filters are instantiated similarly to other HalObjects, using the
"filter-instances" config key. To "attach" a filter to an agent or module, a
new filter table needs to be defined in config.json.
Here is an example:
"filters": {
"inbound": {
"irc0": [ "rlfilter0" ]
},
"outbound": {
"quote0": [ "rlfilter1" ]
}
}
In this example, "rlfilter0" and "rlfilter1" are instances of a rate-limiting
filter. The above declaration states that all messages from any source being
sent to irc0 must first pass through rlfilter0. Also, any messages flowing out
from module quote0 must first pass through rlfilter1.
As rlfilter0 and rlfilter1 are different instances, they may have different
tolerances for their rate limiter. So, for example, this could restrict the
amount of spam coming from one particular module more strictly than all
messages flowing to an agent.
ce7d959 to
c48d22a
Compare
3 participants
In the never ending quest to make Halibot aware of its own spam, here is an implementation of filters that inject themselves into a send path based on a
config.jsonfilter table. Filters are implemented as a newHalObjecttype, and now also have their ownfilter-instanceskey inconfig.json. See below for an example config.Overview
When a message is sent via
.send_to()(and thus also.reply(),.dispatch()), the baseHalObjectclass first checks the filter table inconfig.jsonfor any filters that may need to be applied. The filter table is a top-level config keyfilter, that contains two sub keys:inboundandoutbound. The inbound key maps a target object to a list of filters that should be applied first. The outbound key maps the sending object to a list of filters. The sending logic first prepends any valid inbound filters, then prepends any outbound filters. Finally, the message is sent to the first item in the RI list.When a filter receives a message object, it first strips off the first item in the RI list. This preserves the chain, and prevents a loop. When the message exits the filter chain, the target should be the original intended target prior to the filter injection. See example below for how this chain resolves.
Why...
...another
HalObjectI think it makes sense that it maintains its own state, including message queue. A filter may want to simply delay sending a message, instead of outright denying it. A filter may want to modify a message and completely change the target. Also, it's much much much easier to steal all the same module/agent loading logic.
...not make this a module?
In this implementation,
HalFilterstake a special role that makes them quite different from other modules or agents. They aren't intended to process the data themselves, only to slightly tweak some magic in between. They are like a detour when traveling, if received at the same destination, it doesn't particularly matter which road was taken to get there.They also require a bit of extra special magic that hasn't quite made it into Halibot yet: special routing. This allows us to play with a baby version of a routing scheme we discussed long ago, without completely overhauling the current system.
...do you need #131
Because without it, it would send messages in a loop forever.
There needs to be a way to inject the filter's RIs into the target path somehow, so
.send_toseemed to be a good place to put it. Only then did I realize, it is super easy to get caught in a loop that way. So, I neededHalFilterto send a message without any possible accidental filter injection. Thus,.raw_sendwas born: a way to send a message to a target without any interference.Example
config.json{ "filter-instances": { "rlfilter0": { "of": "RateLimitFilter:Default", "rate": 10 }, "rlfilter1": { "of": "RateLimitFilter:Default", "rate": 6000 } }, "filters": { "inbound": { "irc0": [ "rlfilter0" ] }, "outbound": { "quote0": [ "rlfilter1" ] } } }In this example, all messages flowing to agent
irc0would first pass throughrlfilter0. Similarly, all messages passing from modulequote0would first pass throughrlfilter1.The admin decided to do this so that the Halibot instance can only send to any irc endpoints once every ten seconds. Furthermore, since
quote0can be a somewhat spammy module, the admin specifically puts that module behind an outbound filter, soquote0can only reply once every ten minutes.When a
!quotecommand is received, both filters apply. That is, the reply message for the irc endpoint fromquote0must successfully pass throughrlfilter1andrlfilter0. The target RI generated is thusrlfilter1/rlfilter0/irc0/##somechannel.Getting into more detail, the chain resolves thusly:
irc0/##halibotrlfilter1andrlfilter0, so the new RI isrlfilter1/rlfilter0/irc0/##halibotrlfilter1receives the message, accepts it, and sends to the new RIrlfilter0/irc0/##halibotrlfilter0receives the message, accepts it, and sends to the new RIirc0/##halibotirc0receives the message, and does the normal agenty thingNOTE: in all cases,
msg.originremains the same:quote0.Draft Notes
As this is a draft, I am very aware at some lacking features, implementations etc. As of now, here are the tasks still needed to be completed: