Commit graph

851 commits

Author SHA1 Message Date
Eugen Rochko
6894f5c114 Add forwarding of reported replies to servers being replied to (#25341) 2023-07-08 20:00:02 +02:00
Claire
9a0cbecd67 Fix crash in admin interface when viewing a remote user with verified links (#25796) 2023-07-07 18:10:17 +02:00
Claire
dab0be5aed Merge pull request from GHSA-55j9-c3mp-6fcq 2023-07-06 15:06:49 +02:00
Claire
d7ed682130 Merge pull request from GHSA-9pxv-6qvf-pjwc
* Fix timeout handling of outbound HTTP requests

* Use CLOCK_MONOTONIC instead of Time.now
2023-07-06 15:06:23 +02:00
Claire
cac4f7fdd7 Fix OAuth apps page crashing when listing apps with certain admin API scopes (#25713) 2023-07-04 18:58:23 +02:00
Eugen Rochko
edb6aab974 Revert "Rails 7 update" (#25667) 2023-07-02 11:14:22 +02:00
Matt Jankowski
5cadbaa296 Rails 7 update (#24241) 2023-07-02 10:38:53 +02:00
Claire
1569b36e28 Fix batch attachment deletion leaving empty directories (#25587) 2023-06-27 09:36:11 +02:00
S.H
2c6f88ad71 Remove media attachment only when file was exist (#25586) 2023-06-26 16:38:19 +02:00
Eugen Rochko
bcee25e36c Change files to be deleted in batches instead of one-by-one (#23302) 2023-06-26 14:17:41 +02:00
Claire
4e861795a4 Add translate="no" to outgoing mentions and links (#25524) 2023-06-20 18:10:19 +02:00
Matt Jankowski
d05a68a12e Reduce sleep time in request pool spec (#25470) 2023-06-19 13:48:25 +02:00
Nick Schonning
ef344388c5 Autofix Rubocop Regex Style rules (#23690)
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2023-06-06 14:50:51 +02:00
Eugen Rochko
2252e4d8bb Add webhook templating (#23289)
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2023-06-06 10:42:47 +02:00
Matt Jankowski
14dc8a6a5f Update Admin::Metrics::Measure classes for Rails 7 (#25236) 2023-06-05 20:46:04 +02:00
Matt Jankowski
c74040abfa Rails 7 compatibility fix for Admin::Metrics::Dimension classes (#25277) 2023-06-05 16:52:33 +02:00
Darius Kazemi
6debddcf89 Add exclusive lists (#22048)
Co-authored-by: Liam Cooke <liam@liamcooke.com>
Co-authored-by: John Holdun <john@johnholdun.com>
Co-authored-by: Effy Elden <effy@effy.space>
Co-authored-by: Lina Reyne <git@lina.pizza>
Co-authored-by: Lina <20880695+necropolina@users.noreply.github.com>
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2023-06-05 09:37:02 +02:00
Matt Jankowski
a6f0456756 Remove unused #hash_to_object method in SettingsHelper (#25213) 2023-06-01 09:47:05 +02:00
Claire
8ee98cfd2b Add a confirmation screen when suspending a domain (#25144) 2023-06-01 09:37:38 +02:00
Christian Schmidt
971eafc709 Translate CW, poll options and media descriptions (#24175)
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2023-06-01 00:10:21 +02:00
Matt Jankowski
68296cfb32 Remove unused Settings::Extend module (#25214) 2023-05-31 23:34:48 +02:00
Matt Jankowski
9e8323fdb9 Fix Performance/MapCompact cop (#24797)
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2023-05-23 10:49:12 +02:00
Matt Jankowski
d64a2c7aa8 Fix Performance/StartWith cop (#24818) 2023-05-23 10:16:50 +02:00
Claire
da9237adb5 Fix “Authorized applications” inefficiently and incorrectly getting last use date (#25060) 2023-05-22 14:03:38 +02:00
Emelia Smith
8b4d7b22ed Allow reports with long comments from remote instances, but truncate (#25028) 2023-05-22 13:15:21 +02:00
Claire
0635b1c228 Change OpenGraph-based embeds to allow fullscreen (#25058) 2023-05-22 12:25:56 +02:00
Claire
c8998a2678 Change profile updates to be sent to recently-mentioned servers (#24852) 2023-05-12 13:13:04 +02:00
Claire
c752e939d5 Change AccessTokensVacuum to also delete expired tokens (#24868) 2023-05-11 04:40:03 +02:00
Nick Schonning
f6fd5e4ad0 Autofix Rubocop Style/ConcatArrayLiterals (#23755) 2023-05-04 05:57:40 +02:00
Matt Jankowski
53682a7b4f Fix Performance/DeletePrefix cop (#24796) 2023-05-02 21:07:45 +02:00
Matt Jankowski
4d1f9d4278 Rename with_lock to with_redis_lock to avoid confusion with ActiveRecord's method (#24741) 2023-05-02 18:16:07 +02:00
Matt Jankowski
276a0f7ff9 Fix Performance/Count cop (#24793) 2023-05-02 18:06:43 +02:00
Matt Jankowski
621e53f8fc Fix Performance/Sum cop (#24788) 2023-05-02 16:10:40 +02:00
Matt Jankowski
182bb4595f Fix Lint/DuplicateBranch cop (#24766) 2023-05-02 12:57:11 +02:00
Claire
abce7cf2b9 Rewrite import feature (#21054) 2023-05-02 12:08:48 +02:00
dependabot[bot]
c943082b1b Bump connection_pool from 2.3.0 to 2.4.0 (#24290)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2023-04-30 15:45:02 +09:00
Nick Schonning
519ae8c0c6 Drop EOL Ruby 2.7 (#24237) 2023-04-27 01:46:18 +02:00
Matt Jankowski
f01a643897 Remove instance variables from helper usage (#24203) 2023-04-23 22:35:54 +02:00
Eugen Rochko
c75fccf033 Change user settings to be stored in a more optimal way (#23630)
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2023-03-30 14:44:00 +02:00
Matt Jankowski
4e6cf24ce9 Update retention model sql query (#24249) 2023-03-26 00:37:57 +01:00
Claire
3b17f2f718 Add warning for object storage misconfiguration (#24137) 2023-03-16 22:47:01 +01:00
Christian Schmidt
7583523bf0 Replace Status#translatable? with language matrix in separate endpoint (#24037) 2023-03-16 11:07:24 +01:00
Eugen Rochko
f0734513f4 Refactor monkey-patching of PrivateAddressCheck (#24122) 2023-03-16 04:33:38 +01:00
Claire
bde1c7aa96 Fix dashboard crash on ElasticSearch server error (#23751) 2023-03-15 04:01:49 +01:00
Claire
23bce96f0c Change sidekiq-bulk's batch size from 10,000 to 1,000 jobs in one Redis call (#24034) 2023-03-15 03:45:15 +01:00
Christian Schmidt
170d3a0bc8 Unescape HTML entities (#24019) 2023-03-08 19:56:41 +01:00
Matt Jankowski
8ea6707b2b Remove unused TOCGenerator (#23986) 2023-03-06 17:33:58 +01:00
Jean byroot Boussier
08c2938b4a Upgrade to latest redis-rb 4.x and fix deprecations (#23616)
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2023-03-04 16:38:28 +01:00
Christian Schmidt
cc0c37cd69 Only offer translation for supported languages (#23879) 2023-03-03 21:06:31 +01:00
Nick Schonning
247bf6e812 Enable Rubocop Style/StringConcatenation defaults (#23792) 2023-02-22 09:54:36 +09:00
Nick Schonning
dd63a919f0 Autofix Rubocop Style/RescueStandardError (#23745) 2023-02-20 11:01:20 +01:00
Nick Schonning
e6312104c7 Autofix Rubocop remaining Layout rules (#23679) 2023-02-20 06:58:28 +01:00
Nick Schonning
b8f17330ea Autofix Rubocop Rails/Pluck (#23730) 2023-02-20 02:28:40 +01:00
Nick Schonning
be1ec56c6d Autofix Rubocop Style/StringLiterals (#23695) 2023-02-19 07:38:14 +09:00
Nick Schonning
49f1168050 Autofix Rubocop Style/RedundantBegin (#23703) 2023-02-19 07:09:40 +09:00
Nick Schonning
793f8c7dd5 Autofix Rubocop Style/IfUnlessModifier (#23697) 2023-02-18 12:37:47 +01:00
Nick Schonning
4553bc78ba Autofix Rubocop Style/PerlBackrefs (#23649) 2023-02-18 14:20:20 +09:00
Nick Schonning
b7d4391c4d Autofix Rubocop Lint/AmbiguousOperatorPrecedence (#23681) 2023-02-18 04:30:23 +01:00
Nick Schonning
a71a0802d6 Autofix Rubocop Lint/SymbolConversion (#23683) 2023-02-18 03:23:49 +01:00
Nick Schonning
31ca76fe2c Run rubocop formatting except line length (#23632) 2023-02-18 06:56:20 +09:00
Claire
ea4cffac37 Fix unbounded recursion in post discovery (#23506)
* Add a limit to how many posts can get fetched as a result of a single request

* Add tests

* Always pass `request_id` when processing `Announce` activities

---------

Co-authored-by: nametoolong <nametoolong@users.noreply.github.com>
2023-02-10 22:16:37 +01:00
Nick Schonning
ae45c9ccd4 Apply Rubocop Rails/WhereNot (#23448)
* Apply Rubocop Rails/WhereNot

* Update spec for where.not
2023-02-08 10:39:57 +01:00
Nick Schonning
4a49c50e64 Apply Rubocop Performance/BlockGivenWithExplicitBlock (#23441)
* Apply Rubocop Performance/BlockGivenWithExplicitBlock

* Unprefix used block parameter
2023-02-08 10:36:23 +01:00
Nick Schonning
67a6f688ca Apply Rubocop Performance/StringIdentifierArgument (#23444) 2023-02-08 02:36:20 +01:00
Nick Schonning
6af127a98e Apply Rubocop Performance/DoubleStartEndWith (#23442) 2023-02-08 10:06:48 +09:00
Nick Schonning
38ec6d27d8 Autofix Rails/EagerEvaluationLogMessage (#23429)
* Autofix Rails/EagerEvaluationLogMessage

* Update spec for debug block syntax
2023-02-07 03:44:36 +01:00
Eugen Rochko
502c919bfd Change number of stored items in home feed from 400 to 800 (#23349) 2023-02-01 11:24:22 +01:00
Claire
ae94782edb Fix spurious admin dashboard warning when using ElasticSearch 7.x (#23064)
Some 7.x ElasticSearch versions support some 6.x nodes, thus the version check
is inadequate. I am not sure there is a good way to check if a server
implements all the 7.x APIs, so check server version and minimum wire version
instead.
2023-01-18 16:21:48 +01:00
Markus Unterwaditzer
b7f37e68ad fix: allow verification when page size exceeds 1MB (using HTML5 parser) (#22879)
* fix: allow verification when page size exceeds 1MB
Truncates the page after 1MB instead

Closes #15316

* switch to HTML5 parser, fix rubocop errors

* undo rubocop fixes

Co-authored-by: Chris Zubak-Skees <chriszs@gmail.com>
2023-01-11 21:59:13 +01:00
Kaspar V
1cd04e5d77 Fix/remove calling private method with send in model (#22951)
* fix(status): remove send usage for private unlink_from_conversations

- make unlink_from_conversations public method
- rename unlink_from_conversations to unlink_from_conversations!
- fix send call on private method in statuses_vacuum and batched_remove_status_service

* fix(feeds_vacuum): replace find_in_batches with in_batches

because active record query results should be a little more efficient than
itterating with map and each. Postgres can grasp such lists of ids much quicker
than ruby can.
Will probably make allmost no difference, but cannot hurt either.
2023-01-11 21:57:24 +01:00
Vyr Cossont
b2c7ca296c Check OpenSearch compatibility version instead of regular version (#22422)
Fixes #18535.
2023-01-05 13:44:31 +01:00
Jeong Arm
5180ae5bf6 Fix home TL could contain post from who blocked me (#22849)
* Fix home tl contains post from who blocked me

* Add test

* Fix feed_manager's build_crutches

blocked_by was not includes status' owner

* Add test for status from I blocked

* Fix typo
2023-01-05 13:30:38 +01:00
Claire
e1d8048bc4 Fix status cache hydration discrepancy (#19879) 2022-12-15 18:55:55 +01:00
Claire
c83083eef5 Fix inability to use local LibreTranslate without setting ALLOWED_PRIVATE_ADDRESSES (#21926)
Fixes #20029
2022-12-15 17:04:38 +01:00
s0
289097866b Fix crash in elasticsearch_check.rb (#21006)
Nil unwrap causes the admin dashboard to crash/500 when the Chewy client info version number value is nil.
This occurs when running another ES-compatible backend such as MeiliSearch.
Obviously it would be good for chewy to recognise upstream but at least avoiding the crash would be fine.
2022-12-15 15:56:48 +01:00
Jeong Arm
53b6623136 Don't delivery a reply to domains which are blocked by author (#22117)
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-12-15 15:52:06 +01:00
Claire
aec7de494f Fix unbounded recursion in account discovery (#22025)
* Fix trying to fetch posts from other users when fetching featured posts

* Rate-limit discovery of new subdomains

* Put a limit on recursively discovering new accounts
2022-12-07 00:15:24 +01:00
Claire
1bef772764 Fix some performance issues with /admin/instances (#21907)
/admin/instances?availability=failing remains wholly unefficient
2022-12-01 10:32:10 +01:00
Kaspar V
f2ec356cb6 refactor(vacuum statuses): reduce amount of db queries and load for each query - improve performance (#21487)
* refactor(statuses_vacuum): remove dead code - unused

Method is not called inside class and private.
Clean up dead code.

* refactor(statuses_vacuum): make retention_period present test explicit

This private method only hides functionality.
It is best practice to be as explicit as possible.

* refactor(statuses_vacuum): improve query performance

- fix statuses_scope having sub-select for Account.remote scope by
  `joins(:account).merge(Account.remote)`
- fix statuses_scope unnecessary use of `Status.arel_table[:id].lt`
  because it is inexplicit, bad practice and even slower than normal
  `.where('statuses.id < ?'`
- fix statuses_scope remove select(:id, :visibility) for having reusable
  active record query batches (no re queries)
- fix vacuum_statuses! to use in_batches instead of find_in_batches,
  because in_batches delivers a full blown active record query result,
  in stead of an array - no requeries necessary
- send(:unlink_from_conversations) not to perform another db query, but
  reuse the in_batches result instead.
- remove now obsolete remove_from_account_conversations method
- remove_from_search_index uses array of ids, instead of mapping
  the ids from an array - this should be more efficient
- use the in_batches scope to call delete_all, instead of running
  another db query for this - because it is again more efficient
- add TODO comment for calling models private method with send

* refactor(status): simplify unlink_from_conversations

- add `has_many through:` relation mentioned_accounts
- use model scope local instead of method call `Status#local?`
- more readable add account to inbox_owners when account.local?

* refactor(status): searchable_by way less sub selects

These queries all included a sub-select. Doing the same with a joins
should be more efficient.
Since this method does 5 such queries, this should be significant,
since it technically halves the query count.

This is how it was:

```ruby
[3] pry(main)> Status.first.mentions.where(account: Account.local, silent: false).explain
  Status Load (1.6ms)  SELECT "statuses".* FROM "statuses" WHERE "statuses"."deleted_at" IS NULL ORDER BY "statuses"."id" DESC LIMIT $1  [["LIMIT", 1]]
  Mention Load (1.5ms)  SELECT "mentions".* FROM "mentions" WHERE "mentions"."status_id" = $1 AND "mentions"."account_id" IN (SELECT "accounts"."id" FROM "accounts" WHERE "accounts"."domain" IS NULL) AND "mentions"."silent" = $2  [["status_id", 109382923142288414], ["silent", false]]
=> EXPLAIN for: SELECT "mentions".* FROM "mentions" WHERE "mentions"."status_id" = $1 AND "mentions"."account_id" IN (SELECT "accounts"."id" FROM "accounts" WHERE "accounts"."domain" IS NULL) AND "mentions"."silent" = $2 [["status_id", 109382923142288414], ["silent", false]]
                                                    QUERY PLAN
------------------------------------------------------------------------------------------------------------------
 Nested Loop  (cost=0.15..23.08 rows=1 width=41)
   ->  Seq Scan on accounts  (cost=0.00..10.90 rows=1 width=8)
         Filter: (domain IS NULL)
   ->  Index Scan using index_mentions_on_account_id_and_status_id on mentions  (cost=0.15..8.17 rows=1 width=41)
         Index Cond: ((account_id = accounts.id) AND (status_id = '109382923142288414'::bigint))
         Filter: (NOT silent)
(6 rows)
```

This is how it is with this change:

```ruby
[4] pry(main)> Status.first.mentions.joins(:account).merge(Account.local).active.explain
  Status Load (1.7ms)  SELECT "statuses".* FROM "statuses" WHERE "statuses"."deleted_at" IS NULL ORDER BY "statuses"."id" DESC LIMIT $1  [["LIMIT", 1]]
  Mention Load (0.7ms)  SELECT "mentions".* FROM "mentions" INNER JOIN "accounts" ON "accounts"."id" = "mentions"."account_id" WHERE "mentions"."status_id" = $1 AND "accounts"."domain" IS NULL AND "mentions"."silent" = $2  [["status_id", 109382923142288414], ["silent", false]]
=> EXPLAIN for: SELECT "mentions".* FROM "mentions" INNER JOIN "accounts" ON "accounts"."id" = "mentions"."account_id" WHERE "mentions"."status_id" = $1 AND "accounts"."domain" IS NULL AND "mentions"."silent" = $2 [["status_id", 109382923142288414], ["silent", false]]
                                                    QUERY PLAN
------------------------------------------------------------------------------------------------------------------
 Nested Loop  (cost=0.15..23.08 rows=1 width=41)
   ->  Seq Scan on accounts  (cost=0.00..10.90 rows=1 width=8)
         Filter: (domain IS NULL)
   ->  Index Scan using index_mentions_on_account_id_and_status_id on mentions  (cost=0.15..8.17 rows=1 width=41)
         Index Cond: ((account_id = accounts.id) AND (status_id = '109382923142288414'::bigint))
         Filter: (NOT silent)
(6 rows)
```
2022-11-27 20:41:18 +01:00
Claire
c9b53436d7 Fix emoji substitution not applying only to text nodes in backend code (#20641)
Signed-off-by: Claire <claire.github-309c@sitedethib.com>

Signed-off-by: Claire <claire.github-309c@sitedethib.com>
2022-11-14 20:26:21 +01:00
Bearice Ren
192fe87f51 fixes ArgumentError when proxy is used (#20420)
* fixes ArgumentError when proxy is used

* Update app/lib/request.rb

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-11-11 21:31:03 +01:00
Eugen Rochko
e6e139ac2b Change incoming activity processing to happen in ingress queue (#20264) 2022-11-10 14:21:51 +01:00
James Tucker
c4ff2640c1 Improve performance by avoiding regex construction (#20215)
```ruby
10.times { p /#{FOO}/.object_id }
10.times { p FOO_RE.object_id }
```
2022-11-10 05:49:30 +01:00
Claire
18a11da3b5 Change Request connection logic to try both IPv6 and IPv4 when available (#20108)
Fixes #19751
2022-11-08 16:36:26 +01:00
James Tucker
f48dcd025b Improve performance by avoiding method cache busts (#19957)
Switch to monkey-patching http.rb rather than a runtime extend of each
response, so as to avoid busting the global method cache. A guard is
included that will provide developer feedback in development and test
environments should the monkey patch ever collide.
2022-11-08 04:00:27 +01:00
Claire
2ed7a4124b Fix filter handling in status cache hydration (#19963) 2022-11-07 19:47:48 +01:00
Claire
0735081cc2 Fix additional issues with status cache hydration (#19747)
* Spare one SQL query when hydrating polls

* Improve tests

* Fix more discrepancies

* Fix possible crash when the status has no application set
2022-11-04 20:01:33 +01:00
Claire
e97b0e0700 Fix various issues with store hydration (#19746)
- Improve tests
- Fix possible crash when application of a reblogged post isn't set
- Fix discrepancies around favourited and reblogged attributes
- Fix discrepancies around pinned attribute
- Fix polls not being hydrated
2022-11-04 19:33:16 +01:00
Claire
4b0e72b2b7 Fix reblogs being discarded after the reblogged status (#19731) 2022-11-04 16:31:44 +01:00
Eugen Rochko
5b639f9e15 Add caching for payload serialization during fan-out (#19642) 2022-11-04 13:21:06 +01:00
Eugen Rochko
9788e248fb Fix improperly checking for blocked domain on followed hashtags (#19472)
Fix #19469
2022-10-26 20:40:56 +02:00
Eugen Rochko
d53e1fc6d9 Fix vacuum scheduler missing lock, locks never expiring (#19458)
Remove vacuuming of orphaned preview cards
2022-10-26 12:10:48 +02:00
Eugen Rochko
317383f397 Revert "Remove preference to aggregate reblogs in home/list feeds (#18112)" (#19463)
This reverts commit dd5b90e4f2.
2022-10-26 09:10:18 +02:00
Claire
d2d2ec29f4 Add mention of the translation provider when translating a post (#19433) 2022-10-24 18:37:57 +02:00
Claire
119b5adbe0 Change “Translate” button to only show up when a translation backend is configured (#19434)
* Change “Translate” button to only show up when a translation backend is configured

Fixes #19346

* Add `translation` attribute to /api/v2/instance to expose whether the translation feature is enabled

Fixes #19328
2022-10-24 18:30:58 +02:00
Takeshi Umeda
46755e25a2 Add featured tag add/remove activity handler (#19408) 2022-10-22 11:49:41 +02:00
Eugen Rochko
0fdfbe555e Change public accounts pages to mount the web UI (#19319)
* Change public accounts pages to mount the web UI

* Fix handling of remote usernames in routes

- When logged in, serve web app
- When logged out, redirect to permalink
- Fix `app-body` class not being set sometimes due to name conflict

* Fix missing `multiColumn` prop

* Fix failing test

* Use `discoverable` attribute to control indexing directives

* Fix `<ColumnLoading />` not using `multiColumn`

* Add `noindex` to accounts in REST API

* Change noindex directive to not be rendered by default before a route is mounted

* Add loading indicator for detailed status in web UI

* Fix missing indicator appearing while account is loading in web UI
2022-10-20 14:35:29 +02:00
Eugen Rochko
babebc4e8a Add logged-out access to the web UI (#18961) 2022-09-29 04:39:33 +02:00
Yamagishi Kazutoshi
18159affbf Fix auto detect language for translate service (#19244) 2022-09-27 23:33:56 +02:00
Eugen Rochko
7a482567f8 Add retention policy for cached content and media (#19232) 2022-09-27 03:08:19 +02:00