Commit graph

19 commits

Author SHA1 Message Date
David Yip
2bb5842541 Thread scopes through #matches?. #454.
Also add an apply_to_mentions attribute on Glitch::KeywordMute, which is
used to calculate scope.  Next up: additions to the test suite to
demonstrate how scoping works.
2018-06-03 23:00:50 -05:00
David Yip
d55d5ccc9d keyword mute: Store keywords as a list
This has a couple of advantages over the regex approach:

- Keywords are individually addressable, which makes it easier to gather
  statistics (#363)
- Keywords can be individually applied to different feeds, e.g. skipping
  mentions (#454)

It *does* end up creating many more Regexp objects.  I'm not yet sure if
the difference is significant.
2018-06-03 18:12:55 -05:00
Daggertooth
b60e38c985 Escape metachars in keywords 2018-05-04 14:58:11 -05:00
David Yip
e484bd61e0 Also treat non-whole-word mutes as case-insensitive. #450. 2018-04-30 20:08:22 -05:00
David Yip
062f72c593 Use Html2Text to generate plaintext for keyword mutes. #236.
This allows us to match URLs inside link hrefs.
2018-02-10 10:36:16 -06:00
David Yip
00cb763893 Glitch::FilterHelper -> Glitch::KeywordMuteHelper. #234.
The class helps out with keyword mutes, not just some general concept of
"filtering".
2018-02-10 10:36:15 -06:00
David Yip
a5f1c1f469 Fill out some examples for Glitch::FilterHelper. #234.
Also add HTML entity decoding to Glitch::FilterHelper, which is needed
to e.g. match "<" to the tag-stripped version of "<p><3</p>" or
"<p>&lt;3</p>".
2018-02-10 02:40:27 -06:00
David Yip
e365675e0a Strip HTML from keyword mute input. #234. 2018-02-10 02:40:27 -06:00
David Yip
b4725cad17 Replace =~ with #matches?. #208.
=~ made sense when we were passing it through to a regex, but we're no
longer doing that: TagMatcher looks at individual tags and returns a
value that *looks* like what you get out of #=~ but really isn't that
meaningful.  Probably a good idea to not subvert convention like this
and instead use a name with guessable intent.
2017-11-15 18:27:25 -06:00
David Yip
21e138379d Prefix cache keys with the matcher type. #208.
We already know about one regex limitation, which is that they cannot
segment words in e.g. Japanese, Chinese, or Thai.  It may also end up
that regex matching is too slow compared to other methods.

However, the regex is an implementation detail.  We still want the
ability to switch between "occurs anywhere" and "match whole word", and
caching the matcher result is likely to still be important (since the
matcher itself won't change nearly as often as status ingress rate).
Therefore, we ought to be able to change the cache keys to reflect a
change of data structure.

(Old cache keys expire within minutes, so they shouldn't be too big of
an issue.  Old cache keys could also be explicitly removed by an
instance administrator.)
2017-11-15 18:12:31 -06:00
David Yip
2c6a2d588c Match keyword mute filter on hashtags. #208.
It is reasonable to expect someone to enter #foo to mute hashtag #foo.
However, tags are recorded on statuses without the preceding #.
To adjust for this, we build a separate tag matcher and use
Tag::HASHTAG_RE to extract a hashtag from the hashtag syntax.
2017-11-15 18:12:29 -06:00
David Yip
13ec4a9a20 Maintain case-insensitivity when merging multiple matchers (#213)
When given two regexps, Regexp.union preserves the options set (or not
set) on each regex; this meant that none of the multiline (m),
case-insensitivity (i), or extended syntax (x) options were set.  Our
regexps are written expecting the m, i, and x options were set on all of
them, so we need to make sure that we preserve that behavior.
2017-11-13 11:06:02 -06:00
David Yip
b8c462c42b Remove nil check in Glitch::KeywordMute#=~.
@regex can no longer be nil, so we don't need to check it.
2017-10-24 19:03:59 -05:00
David Yip
05fecda52d Switch to Regexp.union for building the mute expression.
Also make the keyword-building methods private: they always probably
should have been private, but now I have encoded enough fun and games
into them that it now seems wrong for them to *not* be private.
2017-10-24 18:31:34 -05:00
David Yip
763fd4b275 Only cache the regex text, not the regex itself.
It is possible to cache a Regexp object, but I'm not sure what happens
if e.g. that object remains in cache across two different Ruby versions.
Caching a string seems to raise fewer questions.
2017-10-23 19:31:59 -05:00
David Yip
e8314ee2a6 KeywordMute matcher: more closely mimic Regexp#=~ behavior.
Regexp#=~ returns nil if it does not match.  An empty mute set does not
match any status, so KeywordMute::Matcher#=~ ought to return nil also.
2017-10-22 01:12:21 -05:00
David Yip
3b2bf30644 Don't add \b to whole-word keywords that don't start with word characters.
Ditto for ending with \b.

Consider muting the phrase "(hot take)".  I stipulate it is reasonable
to enter this with the default "match whole word" behavior.  Under the
old behavior, this would be encoded as

    \b\(hot\ take\)\b

However, if \b is before the first character in the string and the first
character in the string is not a word character, then the match will
fail.  Ditto for after.  In our example, "(" is not a word character, so
this will not match statuses containing "(hot take)", and that's a very
surprising behavior.

To address this, we only add leading and trailing \b to keywords that
start or end with word characters.
2017-10-22 00:38:54 -05:00
David Yip
b2ad79547c Apply keyword mutes to reblogs. 2017-10-21 15:44:47 -05:00
David Yip
74141adca1 Move KeywordMute into Glitch namespace.
There are two motivations for this:

1. It looks like we're going to add other features that require
   server-side storage (e.g. user notes).

2. Namespacing glitchsoc modifications is a good idea anyway: even if we
   do not end up doing (1), if upstream introduces a keyword-mute feature
   that also uses a "KeywordMute" model, we can avoid some merge
   conflicts this way and work on the more interesting task of
   choosing which implementation to use.
2017-10-21 14:54:36 -05:00