Commit graph

1213 commits

Author SHA1 Message Date
Claire
522fbf9d13 Add ability to filter individual posts (#18945)
* Add database table for status-specific filters

* Add REST endpoints, entities and attributes

* Show status filters in /filters interface

* Perform server-side filtering for individual posts filters

* Fix filtering on context mismatch

* Refactor `toServerSideType` by moving it to its own module

* Move loupe and delete icons to their own module

* Add ability to filter individual posts from WebUI

* Replace keyword list by warnings (expired, context mismatch)

* Refactor server-side filtering code

* Add tests
2022-08-25 04:27:47 +02:00
Eugen Rochko
e642516264 Change e-mail domain blocks to match subdomains of blocked domains (#18979) 2022-08-24 19:00:55 +02:00
Eugen Rochko
b2e1224baa Add ability to block sign-ups from IP (#19037) 2022-08-24 19:00:37 +02:00
Eugen Rochko
abb4dca69b Fix unicode regression in #18809 (#18863) 2022-07-22 03:17:56 +02:00
Eugen Rochko
265c09fef1 Add ability to follow hashtags (#18809) 2022-07-17 13:49:29 +02:00
Claire
0e504e67f3 Add support for editing labelling of one's own role (#18812)
Still disallow edition of rank or permissions
2022-07-17 13:37:30 +02:00
Eugen Rochko
57cf1c5a98 Fix nil error when rendering featured hashtags on profile (#18808)
Regression from #18795
2022-07-14 01:23:10 +02:00
Eugen Rochko
38d04135bf Change how hashtags are normalized (#18795)
* Change how hashtags are normalized

* Fix tests
2022-07-13 15:03:28 +02:00
Eugen Rochko
89ef936126 Change custom emoji file size limit from 50 KB to 256 KB (#18788) 2022-07-09 22:07:17 +02:00
Eugen Rochko
e164d6a687 Add customizable user roles (#18641)
* Add customizable user roles

* Various fixes and improvements

* Add migration for old settings and fix tootctl role management
2022-07-05 02:41:40 +02:00
Jeong Arm
386980c196 Support audio/vnd.wave (#18737)
See: https://datatracker.ietf.org/doc/html/rfc2361
And Misskey uses this mime type for wav file.
2022-06-28 19:49:35 +02:00
Claire
90e5a9bd98 Revamp post filtering system (#18058)
* Add model for custom filter keywords

* Use CustomFilterKeyword internally

Does not change the API

* Fix /filters/edit and /filters/new

* Add migration tests

* Remove whole_word column from custom_filters (covered by custom_filter_keywords)

* Redesign /filters

Instead of a list, present a card that displays more information and handles
multiple keywords per filter.

* Redesign /filters/new and /filters/edit to add and remove keywords

This adds a new gem dependency: cocoon, as well as a npm dependency:
cocoon-js-vanilla. Those are used to easily populate and remove form fields
from the user interface when manipulating multiple keyword filters at once.

* Add /api/v2/filters to edit filter with multiple keywords

Entities:
- `Filter`: `id`, `title`, `filter_action` (either `hide` or `warn`), `context`
  `keywords`
- `FilterKeyword`: `id`, `keyword`, `whole_word`

API endpoits:
- `GET /api/v2/filters` to list filters (including keywords)
- `POST /api/v2/filters` to create a new filter
  `keywords_attributes` can also be passed to create keywords in one request
- `GET /api/v2/filters/:id` to read a particular filter
- `PUT /api/v2/filters/:id` to update a new filter
  `keywords_attributes` can also be passed to edit, delete or add keywords in
   one request
- `DELETE /api/v2/filters/:id` to delete a particular filter
- `GET /api/v2/filters/:id/keywords` to list keywords for a filter
- `POST /api/v2/filters/:filter_id/keywords/:id` to add a new keyword to a
   filter
- `GET /api/v2/filter_keywords/:id` to read a particular keyword
- `PUT /api/v2/filter_keywords/:id` to edit a particular keyword
- `DELETE /api/v2/filter_keywords/:id` to delete a particular keyword

* Change from `irreversible` boolean to `action` enum

* Remove irrelevent `irreversible_must_be_within_context` check

* Fix /filters/new and /filters/edit with update for filter_action

* Fix Rubocop/Codeclimate complaining about task names

* Refactor FeedManager#phrase_filtered?

This moves regexp building and filter caching to the `CustomFilter` class.

This does not change the functional behavior yet, but this changes how the
cache is built, doing per-custom_filter regexps so that filters can be matched
independently, while still offering caching.

* Perform server-side filtering and output result in REST API

* Fix numerous filters_changed events being sent when editing multiple keywords at once

* Add some tests

* Use the new API in the WebUI

- use client-side logic for filters we have fetched rules for.
  This is so that filter changes can be retroactively applied without
  reloading the UI.
- use server-side logic for filters we haven't fetched rules for yet
  (e.g. network error, or initial timeline loading)

* Minor optimizations and refactoring

* Perform server-side filtering on the streaming server

* Change the wording of filter action labels

* Fix issues pointed out by linter

* Change design of “Show anyway” link in accordence to review comments

* Drop “irreversible” filtering behavior

* Move /api/v2/filter_keywords to /api/v1/filters/keywords

* Rename `filter_results` attribute to `filtered`

* Rename REST::LegacyFilterSerializer to REST::V1::FilterSerializer

* Fix systemChannelId value in streaming server

* Simplify code by removing client-side filtering code

The simplifcation comes at a cost though: filters aren't retroactively
applied anymore.
2022-06-28 09:42:13 +02:00
Eugen Rochko
e6eb0a29b9 Add notifications for new reports (#18697) 2022-06-27 09:30:15 +02:00
Claire
d0cc795d0f Add /api/v1/admin/domain_allows (#18668)
- `GET /api/v1/admin/domain_allows` lists allowed domains
- `GET /api/v1/admin/domain_allows/:id` shows one by ID
- `DELETE /api/v1/admin/domain_allows/:id` deletes a given domain from the list
  of allowed domains
- `POST /api/v1/admin/domain_allows` to allow a new domain:
  if that domain is already allowed, the existing DomainAllow will be returned
2022-06-23 23:12:01 +02:00
Eugen Rochko
0eb2db6b52 Add administrative webhooks (#18510)
* Add administrative webhooks

* Fix error when webhook is deleted before delivery worker runs
2022-06-09 21:57:36 +02:00
Claire
a0556ea3f1 Remove dependency on running Redis server for db:setup (#18560) 2022-06-01 19:23:31 +02:00
Claire
9f3bf956ad Add /api/v1/admin/domain_blocks (#18247)
* Add /api/v1/admin/domain_blocks

Fixes #18140

- `GET /api/v1/admin/domain_blocks` lists domain blocks
- `GET /api/v1/admin/domain_blocks/:id` shows one by ID
- `DELETE /api/v1/admin/domain_blocks/:id` deletes a given domain block
- `POST /api/v1/admin/domain_blocks` to create a new domain block:
  if it conflicts with an existing one, returns an error with
  an attribute `existing_domain_block` with the rendered domain block

* Simplify conflict handling as suggested in review
2022-06-01 17:31:36 +02:00
Alexander Ivanov
a59e8245f5 Add support for webp uploads (#18506) 2022-05-27 20:06:40 +02:00
Claire
73c8032178 Remove unused filtered_languages column (#18533)
* Remove unused `filtered_languages` column

Fixes #18522

* Fix tests
2022-05-27 20:05:22 +02:00
Eugen Rochko
4bb50e32e4 Fix being able to report otherwise inaccessible statuses (#18528) 2022-05-26 22:08:02 +02:00
Eugen Rochko
35ebb5571e Fix follower and other counters being able to go negative (#18517) 2022-05-26 20:32:48 +02:00
Eugen Rochko
5be3d071f2 Fix regression in tootctl search deploy caused by unloaded attribute (#18514) 2022-05-26 18:05:47 +02:00
Claire
f5e6c776c1 Change unapproved and unconfirmed account to not be accessible in the REST API (#17530)
* Change unapproved and unconfirmed account to not be accessible in the REST API

* Change Account#searchable? to reject unconfirmed and unapproved users

* Disable search for unapproved and unconfirmed users in Account.search_for

* Disable search for unapproved and unconfirmed users in Account.advanced_search_for

* Remove unconfirmed and unapproved accounts from Account.searchable scope

* Prevent mentions to unapproved/unconfirmed accounts

* Fix some old tests for Account.advanced_search_for

* Add some Account.advanced_search_for tests for existing behaviors

* Add some tests for Account.search_for

* Add Account.advanced_search_for tests unconfirmed and unapproved accounts

* Add Account.searchable tests

* Fix Account.without_unapproved scope potentially messing with previously-applied scopes

* Allow lookup of unconfirmed/unapproved accounts through /api/v1/accounts/lookup

This is so that the API can still be used to check whether an username is free
to use.
2022-05-26 15:50:33 +02:00
Claire
e220fcf1d4 Fix unnecessary query on status creation (#17901) 2022-05-26 00:20:30 +02:00
Claire
2baba513d9 Fix warning an account outside of a report closing all reports for that account (#18387)
* Fix warning an account outside of a report closing all reports for that account

* Make it clear what actions solve other reports

* Revert "Make it clear what actions solve other reports"

This reverts commit ad006de821f72e75480701298d13f0945b509059.
2022-05-23 20:38:29 +02:00
Eugen Rochko
eda9c41ed8 Change algorithm of tootctl search deploy to improve performance (#18463) 2022-05-22 22:16:43 +02:00
Eugen Rochko
9d160dae82 Fix preferred posting language returning unusable value in REST API (#18428) 2022-05-16 19:13:36 +02:00
Eugen Rochko
f1707c6d50 Refactor how Redis locks are created (#18400)
* Refactor how Redis locks are created

* Fix autorelease duration on account deletion lock
2022-05-13 00:02:35 +02:00
Claire
0e0b96b8e7 Fix account warnings not being recorded in audit log (#18338)
* Fix account warnings not being recorded in audit log

Fixes #18334

* Only record warnings if they are not associated to another action
2022-05-06 21:40:20 +02:00
Claire
a31975d369 Fix possible crash when a post references an invalid media attachment (#18211) 2022-05-01 00:55:26 +02:00
Eugen Rochko
1cd4518c29 Remove IP matching from e-mail domain blocks (#18190)
Clear out e-mail domain blocks created from automatically resolved DNS records
2022-04-29 23:27:03 +02:00
Eugen Rochko
a8e694233c Fix opening and closing Redis connections instead of using a pool (#18171)
* Fix opening and closing Redis connections instead of using a pool

* Fix Redis connections not being returned to the pool in CLI commands
2022-04-29 22:43:07 +02:00
Eugen Rochko
2fb1c02eb7 Change trending statuses to only show one status from each account (#18181)
Calculate trends in temporary sets to avoid having to manage items
that go below the decay threshold while not having any moments
where a half-processed set is accessible to end-users
2022-04-29 22:42:42 +02:00
Eugen Rochko
4543f42811 Change half-life of trending status scores from 6 hours to 2 hours (#18182) 2022-04-29 11:46:59 +02:00
Eugen Rochko
553889bc7c Fix single Redis connection being used across all threads (#18135)
* Fix single Redis connection being used across all Sidekiq threads

* Fix tests
2022-04-28 17:47:34 +02:00
Claire
ef902a72cf Fix instance actor being incorrectly created when running migrations (#18109)
* Add migration test about instance actor key

* Fix old migration

* Work around incorrect database state
2022-04-26 21:22:09 +02:00
Jeong Arm
4af30ac16b Let votes statuses are also searchable (#18070) 2022-04-23 21:47:27 +02:00
Claire
5e13634c6b Fix crash in alias settings page (#18004) 2022-04-09 20:11:06 +02:00
Eugen Rochko
d2265cab26 Fix dangling language-specific trends (#17997)
- Change score half-life for trending statuses from 2 to 6 hours
- Change score threshold for trimming old items from 1 to 0.3
2022-04-08 19:35:31 +02:00
Eugen Rochko
4ba66d256a Change e-mail notifications to only be sent when recipient is offline (#17984)
* Change e-mail notifications to only be sent when recipient is offline

Change the default for follow and mention notifications back on

* Add preference to always send e-mail notifications

* Change wording
2022-04-08 18:03:31 +02:00
Eugen Rochko
e3be46c743 Fix trends returning less results per page when filtered in REST API (#17996)
- Change filtering and pagination to occur in SQL instead of Redis
- Change rank/score displayed on trends in admin UI to be locale-specific
2022-04-08 17:10:53 +02:00
Eugen Rochko
aa6bc541d3 Fix pagination header on empty trends responses in REST API (#17986) 2022-04-07 18:06:15 +02:00
Eugen Rochko
f982d56b4e Remove sign-in token authentication, instead send e-mail about new sign-in (#17970) 2022-04-06 20:58:12 +02:00
Claire
812b2048f6 Fix error MethodError in Chewy::Strategy::Sidekiq::Worker (#17912)
Also refactor a bit to reduce code duplication.
2022-03-31 00:49:24 +02:00
Eugen Rochko
bbde6bcf6e Fix being able to bypass e-mail restrictions (#17909) 2022-03-30 14:45:52 +02:00
Claire
0fde990a01 Fix /api/v1/admin/accounts (#17887)
* Fix /api/v1/admin/accounts

Compatibility was broken since #17009 which changed the underlying filter class
without changing the controller.

This commits restore support for the old parameters.

* Add /api/v2/admin/accounts with the new parameters

* Add tests

* Add missing filter for `silenced` status

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
2022-03-28 23:57:38 +02:00
Eugen Rochko
89e3309153 Change how unconfirmed accounts are displayed in admin UI (#17874)
Fix #17815
2022-03-26 02:53:13 +01:00
Eugen Rochko
88f93d03fe Fix edits with no actual changes being allowed (#17843)
* Fix edits with no actual changes being allowed locally

* Fix edits with no actual changes being allowed through ActivityPub

* Fix false positive changes caused by description processing in model

* Fix not recording poll expiration update

* Fix test

* Revert changes to ProcessStatusUpdateService

* Various fixes and improvements

* Fix code style issues

* Various changes and improvements

* Add guard clause
2022-03-26 00:38:44 +01:00
Eugen Rochko
c4a97fc0c8 Add offset pagination to trends in REST API (#17872) 2022-03-26 00:26:50 +01:00
Eugen Rochko
85b7406ad8 Fix individually approved/rejected statuses/links showing as pending review (#17787) 2022-03-15 07:51:55 +01:00