Commit graph

584 commits

Author SHA1 Message Date
ThibG
5a89fb4017 Fix old migration script not being able to run if it fails midway (#15361)
* Fix old migration script not being able to run if it fails midway

Improve the robustness of a migration script likely to fail because of database
corruption so it can run again once database corruptions are fixed.

* Display a specific error message in case of index corruption

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-18 17:51:24 +01:00
Eugen Rochko
816df80b86 Add option to obfuscate domain name in public list of domain blocks (#15355)
- Replace the middle of the domain with * characters (except for periods)
- Add SHA-256 digest of the domain name in tooltip
2020-12-18 08:30:41 +01:00
Claire
e590177b29 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/models/form/admin_settings.rb`:
  New setting added upstream. Ported it.
- `app/views/statuses/_simple_status.html.haml`:
  Upstream removed RTL classes. Did the same.
- `config/settings.yml`:
  New setting added upstream. Ported it.
2020-12-15 14:27:06 +01:00
ThibG
3debd888a6 Add indication to admin UI of whether a report has been forwarded (#13237)
* Add indication to admin UI of whether a report has been forwarded

* Rework how forwarded status is displayed

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-15 04:30:15 +01:00
Eugen Rochko
df8874b24e Fix performance on instances list in admin UI (#15282)
- Reduce duplicate queries
- Remove n+1 queries
- Add accounts count to detailed view
- Add separate action log entry for updating existing domain blocks
2020-12-14 09:06:34 +01:00
Thibaut Girka
64ef99a31e Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/follower_accounts_controller.rb`:
  Conflict due to upstream changing suspension logic while
  glitch-soc has an extra option to hide followers count.
  Ported upstream changes.
2020-11-08 14:20:35 +01:00
Eugen Rochko
38be9af436 Add support for reversible suspensions through ActivityPub (#14989) 2020-11-08 00:28:39 +01:00
Thibaut Girka
db1f81553e Merge branch 'master' into glitch-soc/merge-upstream 2020-11-07 18:23:02 +01:00
Takeshi Umeda
385c2eac6f Add account sensitized (#14361)
* Add account sensitized

* Fix i18n normalize

* Fix description and spec

* Fix spec

* Fix wording
2020-11-04 20:45:01 +01:00
Thibaut Girka
6fdc4e8d47 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `.github/dependabot.yml`:
  Updated upstream, we deleted it to not be flooded by Depandabot.
  Kept deleted.
- `Gemfile.lock`:
  Puma updated on both sides, went for the most recent version.
- `app/controllers/api/v1/mutes_controller.rb`:
  Upstream updated the serializer to support timed mutes, while
  glitch-soc added a custom API ages ago to get information that
  is already available elsewhere.
  Dropped the glitch-soc-specific API, went with upstream changes.
- `app/javascript/core/admin.js`:
  Conflict due to changing how assets are loaded. Went with upstream.
- `app/javascript/packs/public.js`:
  Conflict due to changing how assets are loaded. Went with upstream.
- `app/models/mute.rb`:
  🤷
- `app/models/user.rb`:
  New user setting added upstream while we have glitch-soc-specific
  user settings. Added upstream's user setting.
- `config/settings.yml`:
  Upstream added a new user setting close to a user setting we had
  changed the defaults for. Added the new upstream setting.
- `package.json`:
  Upstream dependency updated “too close” to a glitch-soc-specific
  dependency. No real conflict. Updated the dependency.
2020-10-21 19:10:50 +02:00
OSAMU SATO
ba9252b32d Add duration parameter to muting. (#13831)
* Adding duration to muting.

* Remove useless checks
2020-10-13 01:01:14 +02:00
Eugen Rochko
da4c730c47 Add IP-based rules (#14963) 2020-10-12 16:33:49 +02:00
Thibaut Girka
7a84a84fc1 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile.lock`:
  Not a real conflict, upstream updated dependencies that were too close to
  glitch-soc-only ones in the file.
- `app/controllers/oauth/authorized_applications_controller.rb`:
  Upstream changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's theming system.
  Ported upstream changes.
- `app/controllers/settings/base_controller.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's theming system.
  Ported upstream changes.
- `app/controllers/settings/sessions_controller.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's theming system.
  Ported upstream changes.
- `app/models/user.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc not preventing moved accounts from logging
  in.
  Ported upstream changes while keeping the ability for moved accounts to log
  in.
- `app/policies/status_policy.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's local-only toots.
  Ported upstream changes.
- `app/serializers/rest/account_serializer.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's ability  to hide followers count.
  Ported upstream changes.
- `app/services/process_mentions_service.rb`:
  Upstream refactored and changed the logic surrounding suspended accounts.
  Minor conflict due to glitch-soc's local-only toots.
  Ported upstream changes.
- `package.json`:
  Not a real conflict, upstream updated dependencies that were too close to
  glitch-soc-only ones in the file.
2020-09-28 14:13:30 +02:00
Eugen Rochko
02c4c91faa Add option to be notified when a followed user posts (#13546)
* Add bell button

Fix #4890

* Remove duplicate type from post-deployment migration

* Fix legacy class type mappings

* Improve query performance with better index

* Fix validation

* Remove redundant index from notifications
2020-09-18 17:26:45 +02:00
Eugen Rochko
e514304a76 Change account suspensions to be reversible by default (#14726) 2020-09-15 14:37:58 +02:00
ThibG
50d0c1e95f Add configuration option to filter replies in lists (#9205)
* Add database support for list show-reply preferences

* Add backend support to read and update list-specific show_replies settings

* Add basic UI to set list replies setting

* Add specs for list replies policy

* Switch "cycling" reply policy link to a set of radio inputs

* Capitalize replies_policy strings

* Change radio button design to be consistent with that of the directory explorer
2020-09-01 13:31:28 +02:00
Thibaut Girka
37e746eab6 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/accounts_controller.rb`:
  Upstream change too close to a glitch-soc change related to
  instance-local toots. Merged upstream changes.
- `app/services/fan_out_on_write_service.rb`:
  Minor conflict due to glitch-soc's handling of Direct Messages,
  merged upstream changes.
- `yarn.lock`:
  Not really a conflict, caused by glitch-soc-only dependencies
  being textually too close to updated upstream dependencies.
  Merged upstream changes.
2020-08-30 16:13:08 +02:00
santiagorodriguez96
f142983484 Add WebAuthn as an alternative 2FA method (#14466)
* feat: add possibility of adding WebAuthn security keys to use as 2FA

This adds a basic UI for enabling WebAuthn 2FA. We did a little refactor
to the Settings page for editing the 2FA methods – now it will list the
methods that are available to the user (TOTP and WebAuthn) and from
there they'll be able to add or remove any of them.
Also, it's worth mentioning that for enabling WebAuthn it's required to
have TOTP enabled, so the first time that you go to the 2FA Settings
page, you'll be asked to set it up.
This work was inspired by the one donde by Github in their platform, and
despite it could be approached in different ways, we decided to go with
this one given that we feel that this gives a great UX.

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>

* feat: add request for WebAuthn as second factor at login if enabled

This commits adds the feature for using WebAuthn as a second factor for
login when enabled.
If users have WebAuthn enabled, now a page requesting for the use of a
WebAuthn credential for log in will appear, although a link redirecting
to the old page for logging in using a two-factor code will also be
present.

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>

* feat: add possibility of deleting WebAuthn Credentials

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>

* feat: disable WebAuthn when an Admin disables 2FA for a user

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>

* feat: remove ability to disable TOTP leaving only WebAuthn as 2FA

Following examples form other platforms like Github, we decided to make
Webauthn 2FA secondary to 2FA with TOTP, so that we removed the
possibility of removing TOTP authentication only, leaving users with
just WEbAuthn as 2FA. Instead, users will have to click on 'Disable 2FA'
in order to remove second factor auth.
The reason for WebAuthn being secondary to TOPT is that in that way,
users will still be able to log in using their code from their phone's
application if they don't have their security keys with them – or maybe
even lost them.

* We had to change a little the flow for setting up TOTP, given that now
  it's possible to setting up again if you already had TOTP, in order to
  let users modify their authenticator app – given that now it's not
  possible for them to disable TOTP and set it up again with another
  authenticator app.
  So, basically, now instead of storing the new `otp_secret` in the
  user, we store it in the session until the process of set up is
  finished.
  This was because, as it was before, when users clicked on 'Edit' in
  the new two-factor methods lists page, but then went back without
  finishing the flow, their `otp_secret` had been changed therefore
  invalidating their previous authenticator app, making them unable to
  log in again using TOTP.

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>

* refactor: fix eslint errors

The PR build was failing given that linting returning some errors.
This commit attempts to fix them.

* refactor: normalize i18n translations

The build was failing given that i18n translations files were not
normalized.
This commits fixes that.

* refactor: avoid having the webauthn gem locked to a specific version

* refactor: use symbols for routes without '/'

* refactor: avoid sending webauthn disabled email when 2FA is disabled

When an admins disable 2FA for users, we were sending two mails
to them, one notifying that 2FA was disabled and the other to notify
that WebAuthn was disabled.
As the second one is redundant since the first email includes it, we can
remove it and send just one email to users.

* refactor: avoid creating new env variable for webauthn_origin config

* refactor: improve flash error messages for webauthn pages

Co-authored-by: Facundo Padula <facundo.padula@cedarcode.com>
2020-08-24 16:46:27 +02:00
Thibaut Girka
67564cbb48 Merge branch 'master' into glitch-soc/merge-upstream 2020-07-08 15:23:41 +02:00
Takeshi Umeda
b0f98e64f1 Fix RemoveDuplicatedIndexesPghero to check for the existence of the index (#14259) 2020-07-08 09:23:23 +02:00
Thibaut Girka
c2347f6cf6 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `package.json`:
  Not really a conflict, just some glitch-soc-specific dependency
  too close to an upstream-updated one.
2020-07-07 15:58:45 +02:00
Eugen Rochko
0edde9d6c8 Fix media attachments enumeration (#14254)
* Fix media attachment enumeration

* Switch media_attachments id to snowflake ids

Co-authored-by: Thibaut Girka <thib@sitedethib.com>
2020-07-07 15:26:51 +02:00
Thibaut Girka
2a2af880b8 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `.env.production.sample`:
  Upstream deleted it, I decided to keep it.
- `package.json`:
  Upstream updated a dependency textually too close to wavesurfer.js
  which has been deleted from upstream but is kept in glitch-soc for now.
2020-06-30 23:00:20 +02:00
ThibG
3afc7977b1 Add user notes on accounts (#14148)
* Add UserNote model

* Add UI for user notes

* Put comment in relationships entity

* Add API to create user notes

* Copy user notes to new account when receiving a Move activity

* Address some of the review remarks

* Replace modal by inline edition

* Please CodeClimate

* Button design changes

* Change design again

* Cancel note edition when pressing Escape

* Fixes

* Tweak design again

* Move “Add note” item, and allow users to add notes to themselves

* Rename UserNote into AccountNote, rename “comment” Relationship attribute to “note”
2020-06-30 19:19:50 +02:00
Eugen Rochko
a3ce01a102 Add customizable thumbnails for audio and video attachments (#14145)
- Change audio files to not be stripped of metadata
- Automatically extract cover art from audio if it exists
- Add `thumbnail` parameter to `POST /api/v1/media`, `POST /api/v2/media` and `PUT /api/v1/media/:id`
- Add `icon` to represent it in attachments in ActivityPub
- Fix `preview_url` containing URL of missing missing image when there is no thumbnail instead of null
- Fix duration of audio not being displayed on public pages until the file is loaded
2020-06-29 13:56:55 +02:00
Thibaut Girka
0e33582bff Merge branch 'master' into glitch-soc/merge-upstream 2020-06-26 13:02:14 +02:00
Lerk
f0f2147105 remove duplicated indexes according to pghero (#13695) 2020-06-25 12:15:34 +02:00
Thibaut Girka
c1eb60642b Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/concerns/sign_in_token_authentication_concern.rb`:
  Conflict due to glitch-soc's theming system.
  Ported upstream changes.
- `app/controllers/concerns/two_factor_authentication_concern.rb`:
  Conflict due to glitch-soc's theming system.
  Ported upstream changes.
2020-06-24 15:33:41 +02:00
ThibG
6d4a71856b Fix unique username constraint for local users not being enforced in database (#14099)
This should not be an issue in practice because of the Rails-level uniqueness
check, but local accounts having a NULL domain means the uniqueness constraint
did not apply to them (since no two NULL values are considered equal).
2020-06-21 12:41:38 +02:00
Thibaut Girka
83dc54c487 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/activitypub/collections_controller.rb`:
  Conflict due to glitch-soc having to take care of local-only
  pinned toots in that controller.
  Took upstream's changes and restored the local-only special
  handling.
- `app/controllers/auth/sessions_controller.rb`:
  Minor conflicts due to the theming system, applied upstream
  changes, adapted the following two files for glitch-soc's
  theming system:
  - `app/controllers/concerns/sign_in_token_authentication_concern.rb`
  - `app/controllers/concerns/two_factor_authentication_concern.rb`
- `app/services/backup_service.rb`:
  Minor conflict due to glitch-soc having to handle local-only
  toots specially. Applied upstream changes and restored
  the local-only special handling.
- `app/views/admin/custom_emojis/index.html.haml`:
  Minor conflict due to the theming system.
- `package.json`:
  Upstream dependency updated, too close to a glitch-soc-only
  dependency in the file.
- `yarn.lock`:
  Upstream dependency updated, too close to a glitch-soc-only
  dependency in the file.
2020-06-09 10:39:20 +02:00
Eugen Rochko
2dbf6bc5ad Add e-mail-based sign in challenge for users with disabled 2FA (#14013) 2020-06-09 10:23:06 +02:00
ThibG
36f31b5cad Add blurhash to preview cards (#13984)
Fixes #13001
2020-06-05 23:10:41 +02:00
Eugen Rochko
b864e78db8 Add E2EE API (#13820) 2020-06-02 19:24:53 +02:00
Thibaut Girka
4853a87df8 Merge branch 'master' into glitch-soc/merge-upstream 2020-05-13 22:11:49 +02:00
Eugen Rochko
b5124e6d04 Reset secret of web app that could have been exposed by Doorkeeper (#13688)
There are no obvious ways it could be misused, as the secret is not
really used for anything, but it is best to secure it for the future

Follow-up to #13613
2020-05-10 18:18:12 +02:00
Thibaut Girka
4e4e5316c1 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile.lock`:
  Not a real conflict, just a glitch-soc-only dependency too close to a
  dependency that got updated upstream. Updated as well.
- `app/models/status.rb`:
  Not a real conflict, just a change too close to glitch-soc-changed code
  for optionally showing boosts in public timelines.
  Applied upstream changes.
- `app/views/layouts/application.html.haml`:
  Upstream a new, static CSS file, conflict due to glitch-soc's theming
  system, include the file regardless of the theme.
- `config/initializers/content_security_policy.rb`:
  Upstream dropped 'unsafe-inline' from the 'style-src' directive, but
  both files are very different. Removed 'unsafe-inline' as well.
2020-05-10 16:19:56 +02:00
Eugen Rochko
96921d909c Fix old unique jobs digests not having been cleaned up (#13683) 2020-05-09 18:37:39 +02:00
Eugen Rochko
7a43d20550 Fix migrations failing due to strong-migrations update (#13680) 2020-05-08 20:23:16 +02:00
Thibaut Girka
dfa13deaa1 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/statuses_controller.rb`:
  Upstream disabled the embed controller for reblogs.
  Not a real conflict, but glitch-soc has an extra line to deal
  with its theming system.
  Ported upstream changes.
- `app/javascript/packs/public.js`:
  Upstream made changes to get rid of most inline CSS, this changes
  javascript for public pages, which in glitch are split between
  different files. Ported those changes.
- `app/models/status.rb`:
  Upstream changed the block check in `Status#permitted_for` to
  include domain-block checks. Not a real conflict with glitch-soc,
  but our scope is slightly different, as our scope for
  unauthenticated access do not include instance-local toots.
  Ported upstream changes.
- `app/serializers/rest/instance_serializer.rb`:
  Not a real conflict, upstream added a new field to the instance
  serializer, the conflict is one line above since we added more of
  that.
  Ported upstream changes.
- `app/views/settings/profiles/show.html.haml`:
  Upstream got rid of most inline CSS and moved hidden elements
  to data attributes in the process, in fields were we have
  different values.
  Ported upstream changes while keeping our glitch-specific
  values.
- `app/views/statuses/_simple_status.html.haml`:
  Upstream got rid of inline CSS on an HAML line we treat
  differently, stripping empty text nodes.
  Ported upstream changes to the style attribute, keeping
  the empty text node stripping behavior.
2020-05-03 21:23:49 +02:00
Eugen Rochko
adfb22a9dd Add separate cache directory for non-local uploads (#12821) 2020-04-26 23:29:08 +02:00
Thibaut Girka
43a9ed3de8 Merge branch 'master' into glitch-soc/merge-upstream 2020-04-17 20:16:24 +02:00
Takeshi Umeda
1eb59ace5c Fix migration 20200407202420_migrate_unavailable_inboxes (#13481) 2020-04-16 08:03:24 +02:00
Eugen Rochko
2d3219549b Change delivery failure tracking to work with hostnames instead of URLs (#13437) 2020-04-15 20:33:24 +02:00
Thibaut Girka
c13682bc0d Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/javascript/packs/public.js`:
  Upstream modified code that we split between multiple files due
  to glitch-soc's theming system.
  Ported those changes.
2020-04-05 18:32:06 +02:00
ThibG
ef48ffc127 Fix PostgreSQL load when linking in announcements (#13250)
* Fix PostgreSQL load when linking in announcements

Fixes #13245 by caching status lookups

Since statuses are supposed to be known already and we only
need their URLs and a few other things, caching them should
be fine.

Since it's only used by announcements so far, there won't
be much statuses to cache.

* Perform status lookup when saving announcements, not when rendering them

* Change EntityCache#status to fetch URLs instead of looking into the database

* Move announcement link lookup to publishing worker

* Address issues pointed out during review
2020-04-05 12:51:22 +02:00
Thibaut Girka
99fc46d023 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `README.md`:
  Our README.md files are completely different. Discarded upstream changes.
- `app/javascript/core/admin.js`:
  Updating rails-ujs, no real conflict, but a comment to close to changed
  code. Various glitch-soc-only files have been updated to match those changes,
  though.
- `package.json`:
  No real conflict, just an additional dependency in glitch-soc that was too
  close to something updated upstream. Took upstream's changes.
2020-03-22 16:10:44 +01:00
Eugen Rochko
9ced8b6dd5 Add option to include resolved DNS records when blacklisting e-mail domains in admin UI (#13254)
* Add shortcuts to blacklist a user's e-mail domain in admin UI

* Add option to blacklist resolved MX and IP records for e-mail domains
2020-03-12 22:35:20 +01:00
Eugen Rochko
da9d81c4ac Add titles to warning presets in admin UI (#13252) 2020-03-12 17:57:59 +01:00
ThibG
cd7c22da32 Add federation support for the "hide network" preference (#11673)
* Change ActivityPub follower/following collections to not link first page

* Add support for hiding followers and following of remote users

* Switch to using a single `hide_collections` column

* Address code style remarks
2020-03-09 00:10:29 +01:00
Eugen Rochko
f459919552 Change local media attachments to perform heavy processing asynchronously (#13210)
Fix #9106
2020-03-08 23:56:18 +01:00
Thibaut Girka
9604766a18 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/serializers/rest/account_serializer.rb`:
  Upstream added code too close to glitch-soc-specific followers-hiding code.
  Ported upstream changes.
2020-01-27 15:46:50 +01:00
Eugen Rochko
0cc5eb50dd Fix interactive delays in database migrations with no TTY (#12969)
Fix #12587
2020-01-27 11:04:42 +01:00
Eugen Rochko
9d3bba9576 Fix expired announcements being re-published (#12964) 2020-01-26 22:43:18 +01:00
Thibaut Girka
a8e43875a5 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/statuses_controller.rb`:
  Minor conflict due to theming system
2020-01-24 14:37:06 +01:00
Eugen Rochko
f95fd62bac Add announcements (#12662)
* Add announcements

Fix #11006

* Add reactions to announcements

* Add admin UI for announcements

* Add unit tests

* Fix issues

- Add `with_dismissed` param to announcements API
- Fix end date not being formatted when time range is given
- Fix announcement delete causing reactions to send streaming updates
- Fix announcements container growing too wide and mascot too small
- Fix `all_day` being settable when no time range is given
- Change text "Update" to "Announcement"

* Fix scheduler unpublishing announcements before they are due

* Fix filter params not being passed to announcements filter
2020-01-23 22:00:13 +01:00
notozeki
7d03ceb9ef Fix slow query of federated timeline (#12886) 2020-01-21 18:56:34 +01:00
Thibaut Girka
82e197787f Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `Gemfile.lock`:
  No real conflict, glitch-soc-only dependency (redcarpet) too close to an
  upstream one (rdf-normalize)
- `README.md`:
  we have different READMEs, discarded upstream's changes
- `app/views/admin/custom_emojis/index.html.haml`:
  No real conflict, different context because of glitch-soc theming
- `lib/mastodon/statuses_cli.rb`:
  Upstream added code to keep bookmarked statuses, we were already doing so
  with slightly different code. Discarded upstream's changes.
- `package.json`:
  No real conflict, glitch-soc-only dependency (favico.js) too close to
  an upstream one
2020-01-12 15:57:34 +01:00
ThibG
3325ad0cc3 Fix old migrations failing because of strong_migrations update (#12787)
Fixes #12768

Some migrations were overlooked in #12692
2020-01-11 05:52:24 +09:00
Thibaut Girka
2591cedcad Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `app/controllers/application_controller.rb`:
  Conflict due to theming system.
- `app/controllers/oauth/authorizations_controller.rb`:
  Conflict due to theming system.
2020-01-04 23:04:42 +01:00
ThibG
b194caade4 Fix old migrations failing because of strong_migrations update (#12692)
Fixes #12690

The `strong_migrations` update from f673e3d120
introduced a check for `change_column_null` specific to Postgres. This rejects
old migrations.

This commit just wraps old migrations with `safety_assured` to bypass this
check. Alternatives would have been to:
- Disable that check entirely (a possibility added in that same
  `strong_migrations` version) for Mastodon, but it makes sense to write new
  migrations without such a strong lock.
- Rewrite the old migrations to do it in a way that do not require an exclusive
  lock. I thought fixing those old migrations for performance wasn't worth the
  pain. Also, if I understand correctly, the next version of
  `strong_migrations` is going to include a helper to do that. We could update
  those migrations at that point.
2019-12-29 05:39:08 +01:00
Thibaut Girka
fa4a92e340 Merge branch 'master' into glitch-soc/master
Conflicts:
- `config/locales/en.yml`
  No real conflict, upstream added a translatable string “too close” to
  one specific to glitch-soc
- `lib/mastodon/statuses_cli.rb`
  Fixes made upstream, while changed in glitch-soc to keep bookmarked statuses
- `package.json`
  No real conflict, additional dependency in glitch-soc
2019-12-19 13:52:54 +01:00
Eugen Rochko
62ea1bff7d Fix unsafe column type change in migration (#12653) 2019-12-18 19:04:43 +01:00
scd31
27f1d5ee7d Increase max backup size (#12602)
* Increased max backup size

* partially reverted schema.rb
2019-12-17 13:30:58 +01:00
Thibaut Girka
2cc60e8b09 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- package.json
  Not really a conflict, caused by an additional dependency in glitch-soc.
- yarn.lock
  Not really a conflict, caused by an additional dependency in glitch-soc.
2019-12-07 12:28:59 +01:00
ThibG
3dd5448883 Fix old migration failing with new status default scope (#12493) 2019-12-04 04:34:31 +01:00
Thibaut Girka
13f738e229 Add documentation about the migration hack 2019-11-20 17:18:00 +01:00
Thibaut Girka
a96375d2ef Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- README.md
  discarded upstream changes
- app/controllers/api/v1/bookmarks_controller.rb
  finally merged upstream, some code style fixes
  and slightly changed pagination code
- app/controllers/application_controller.rb
  changed upstream to always return HTML error pages
  slight conflict caused by theming code
- app/models/bookmark.rb
  finally merged upstream, no real conflict
- spec/controllers/api/v1/bookmarks_controller_spec.rb
  finally merged upstream, slightly changed pagination code
2019-11-20 15:36:09 +01:00
ThibG
517c1cd062 Add bookmarks (#7107)
* Add backend support for bookmarks

Bookmarks behave like favourites, except they aren't shared with other
users and do not have an associated counter.

* Add spec for bookmark endpoints

* Add front-end support for bookmarks

* Introduce OAuth scopes for bookmarks

* Add bookmarks to archive takeout

* Fix migration

* Coding style fixes

* Fix rebase issue

* Update bookmarked_statuses to latest UI changes

* Update bookmark actions to properly reflect status changes in state

* Add bookmarks item to single-column layout

* Make active bookmarks red
2019-11-13 23:02:10 +01:00
Thibaut Girka
e7655951d7 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- `package.json`
2019-11-06 14:38:28 +01:00
ThibG
6b5aebf0ca Add abilityto add oneself to lists (#12271)
* Add ability to add oneself to lists

* Change search results to include oneself when searching through followers

* Mark follow relation as optional in ListAccount
2019-11-04 13:02:01 +01:00
Thibaut Girka
9506d9cafc Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- README.md
- app/javascript/styles/mastodon/components.scss
  conflicts caused by image URLs being different
- app/models/status.rb
  as_home_timeline removed, kept glitch-soc-only as_direct_timeline
- app/views/statuses/_simple_status.html.haml
- config/locales/en.yml
  some strings were changed upstream
- spec/models/status_spec.rb
  as_home_timeline removed, kept glitch-soc-only as_direct_timeline
2019-10-10 17:26:08 +02:00
Eugen Rochko
1da1a9878f Fix old migration trying to use new column due to default status scope (#12095)
Fix #12087
2019-10-07 06:05:14 +02:00
Eugen Rochko
6262407066 Fix existing user records with now-renamed pt locale (#12092)
Fix #12082
2019-10-07 04:14:36 +02:00
Thibaut Girka
c2d6ef1b64 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/media_attachment.rb
  Upstream raised max image size from 8MB to 10MB while our limit is
  configurable. Raised the default to 10MB.
2019-10-03 11:10:12 +02:00
Eugen Rochko
ced431f02f Fix account counters being overwritten by parallel writes (#12045) 2019-10-02 04:59:37 +02:00
Thibaut Girka
cc94c90bf1 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- Gemfile
- Gemfile.lock
- app/controllers/about_controller.rb
- app/controllers/auth/sessions_controller.rb
2019-09-30 12:23:57 +02:00
ThibG
f088e090e4 Add voters count support (#11917)
* Add voters count to polls

* Add ActivityPub serialization and parsing of voters count

* Add support for voters count in WebUI

* Move incrementation of voters count out of redis lock

* Reword “voters” to “people”
2019-09-29 22:58:01 +02:00
Eugen Rochko
d73822dc01 Fix older migrations not working due to new default scope (#11983)
Fix #11952, regression from #11623
2019-09-28 01:05:26 +02:00
Eugen Rochko
a2d3728c36 Add account migration UI (#11846)
Fix #10736

- Change data export to be available for non-functional accounts
- Change non-functional accounts to include redirecting accounts
2019-09-19 20:58:19 +02:00
Thibaut Girka
4837d6275a Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/auth/sessions_controller.rb
  Minor conflict due to glitch-soc's theming code
2019-09-18 17:25:56 +02:00
abcang
68a85ef015 Add users remember_token index (#11881) 2019-09-18 10:58:08 +02:00
Thibaut Girka
2e38d889a3 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- Gemfile
- app/controllers/api/v1/search_controller.rb
  Conflict because we changed the number of default results to be
  configurable
- app/lib/settings/scoped_settings.rb
  Addition of a new “noindex” site-wide setting,
  conflict due to our change of the two other site-wide settings
  (default flavour and skin instead of theme)
- spec/controllers/application_controller_spec.rb
  Addition of a new “noindex” site-wide setting,
  conflict due to our change of the two other site-wide settings
  (default flavour and skin instead of theme)
2019-09-13 18:13:43 +02:00
Eugen Rochko
89989b6255 Add timeline read markers API (#11762)
Fix #4093
2019-09-06 13:55:51 +02:00
Thibaut Girka
eac7c66d2a Merge branch 'master' into glitch-soc/merge-upstream 2019-09-05 11:36:41 +02:00
Eugen Rochko
436266ca27 Change trending hashtags to not disappear instantly after midnight (#11712) 2019-09-02 18:11:13 +02:00
Thibaut Girka
449a0727a7 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/status.rb
- app/services/remove_status_service.rb
- db/schema.rb

All conflicts were due to the addition of a `deleted_at` attribute
to Statuses and reworked database indexes.
2019-08-29 12:07:50 +02:00
Eugen Rochko
9c65b122fc Fix slow local timeline query (#11648)
Fix #11643
2019-08-24 04:12:27 +02:00
Eugen Rochko
32c781ec6b Add soft delete for statuses for instant deletes through API (#11623)
* Add soft delete for statuses to allow them to appear instant

* Allow reporting soft-deleted statuses and show them in the admin UI

* Change index for getting an account's statuses
2019-08-22 21:55:56 +02:00
Thibaut Girka
c060523dc2 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/home_controller.rb
- app/controllers/shares_controller.rb
- app/javascript/packs/public.js
- app/models/status.rb
- app/serializers/initial_state_serializer.rb
- app/views/home/index.html.haml
- app/views/layouts/public.html.haml
- app/views/public_timelines/show.html.haml
- app/views/shares/show.html.haml
- app/views/tags/show.html.haml
- config/initializers/content_security_policy.rb
- config/locales/en.yml
- config/webpack/shared.js
- package.json
2019-08-19 21:49:35 +02:00
ThibG
f17b0ff712 Add invite comments (#10465) 2019-08-19 11:40:42 +02:00
Eugen Rochko
848cb2a697 Add more accurate hashtag search (#11579)
* Add more accurate hashtag search

Using ElasticSearch to index hashtags with edge n-grams and score
them by usage within the last 7 days since last activity. Only
hashtags that have been reviewed and are listable can appear in
searches, unless they match the query exactly

* Fix search analyzer dropping non-ascii characters
2019-08-18 03:45:51 +02:00
Thibaut Girka
81c6f2fd0e Merge branch 'master' into glitch-soc/merge-upstream 2019-08-08 08:44:33 +02:00
ThibG
072e696aa1 Add domain block notes (#11515)
* Add database columns for adding notes to domain blocks/restrctions

* Add admin UI to set private and public comments when blocking a domain

* Add text for private and public comments on domain blocks

* Show domain block comments in admin UI

* Add comments to the domain block undo page

* Make UnblockDomainService more robust regarding upgraded domain blocks

* Allow editing domain blocks

* Rename button from “undo domain block” to “view domain block” in account admin UI

* Change test to unsilence silenced users from upgraded blocks
2019-08-07 20:20:23 +02:00
Thibaut Girka
7a55d93244 Merge branch 'master' into glitch-soc/merge-upstream 2019-08-06 13:16:53 +02:00
Eugen Rochko
05e9cd13eb Change admin UI for hashtags and add back whitelisted trends (#11490)
Fix #271

Add back the `GET /api/v1/trends` API with the caveat that it does
not return tags that have not been allowed to trend by the staff.

When a hashtag begins to trend (internally) and that hashtag has
not been previously reviewed by the staff, the staff is notified.

The new admin UI for hashtags allows filtering hashtags by where
they are used (e.g. in the profile directory), whether they have
been reviewed or are pending reviewal, they show by how many people
the hashtag is used in the directory, how many people used it
today, how many statuses with it have been created today, and it
allows fixing the name of the hashtag to make it more readable.

The disallowed hashtags feature has been reworked. It is now
controlled from the admin UI for hashtags instead of from
the file `config/settings.yml`
2019-08-05 19:54:29 +02:00
Thibaut Girka
77416866a7 Merge branch 'master' into glitch-soc/merge-upstream 2019-08-05 13:13:28 +02:00
Eugen Rochko
c90eba765d Add hashtag score for better sorting of autosuggestions (#11427)
* Add hashtag score for better sorting of autosuggestions

* Do not use `~<~` operator with no text_pattern_ops index
2019-07-30 13:10:40 +02:00
Thibaut Girka
4973ba2d1f Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/directories_controller.rb
- package.json
- yarn.lock
2019-07-30 12:22:33 +02:00
Eugen Rochko
825dc3ca22 Add whitelist mode (#11291) 2019-07-30 11:10:46 +02:00
Eugen Rochko
c111bd01a4 Fix tag normalization and migration not removing duplicate tags (#11441)
Fix #11428
2019-07-29 20:40:21 +02:00
Thibaut Girka
6c7a3268c9 Merge branch 'master' into glitch-soc/merge-upstream 2019-07-28 16:28:05 +02:00
Eugen Rochko
b8386076c5 Change hashtags to preserve first-used casing (#11416) 2019-07-28 05:59:51 +02:00
Thibaut Girka
124a1d3880 Merge branch 'master' into glitch-soc/merge-upstream 2019-07-24 16:23:53 +02:00
Daigo 3 Dango
ac219b6f88 List columns within the method (#11377)
To avoid the exception:

NoMethodError: undefined method `perform' for nil:NilClass
.../vendor/bundle/ruby/2.6.0/gems/strong_migrations-0.4.1/lib/strong_migrations/migration.rb:14:in `method_missing'
.../vendor/bundle/ruby/2.6.0/gems/activerecord-5.2.3/lib/active_record/migration.rb:604:in `method_missing'
.../db/migrate/20170918125918_ids_to_bigints.rb:69:in `<class:IdsToBigints>'
.../db/migrate/20170918125918_ids_to_bigints.rb:3:in `<top (required)>'
2019-07-23 11:08:11 +02:00
Thibaut Girka
eecce7e59b Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- Gemfile.lock
- app/controllers/accounts_controller.rb
- app/controllers/admin/dashboard_controller.rb
- app/controllers/follower_accounts_controller.rb
- app/controllers/following_accounts_controller.rb
- app/controllers/remote_follow_controller.rb
- app/controllers/stream_entries_controller.rb
- app/controllers/tags_controller.rb
- app/javascript/packs/public.js
- app/lib/sanitize_config.rb
- app/models/account.rb
- app/models/form/admin_settings.rb
- app/models/media_attachment.rb
- app/models/stream_entry.rb
- app/models/user.rb
- app/serializers/initial_state_serializer.rb
- app/services/batched_remove_status_service.rb
- app/services/post_status_service.rb
- app/services/process_mentions_service.rb
- app/services/reblog_service.rb
- app/services/remove_status_service.rb
- app/views/admin/settings/edit.html.haml
- config/locales/simple_form.pl.yml
- config/settings.yml
- docker-compose.yml
2019-07-19 18:26:49 +02:00
ThibG
c2126e3f98 Add ActivityPub actor representing the entire server (#11321)
* Add support for an instance actor

* Skip username validation for local Application accounts

* Add migration script to create instance actor

* Make Codeclimate happy

* Switch to id -99 for instance actor

* Remove unused `icon` and `image` attributes from instance actor

* Use if/elsif/else instead of return + ternary operator

* Add instance actor to fresh installs

* Use instance actor as instance representative

Use instance actor for forwarding reports, relay operations, and spam
auto-reporting.

* Seed database in test environment

* Fix single-user mode

* Fix tests

* Fix specs to accomodate for an extra `Account`

* Auto-reject follows on instance actor

Following an instance actor might make sense, but we are not handling that
right now, so auto-reject.

* Fix webfinger lookup and serialization for instance actor

* Rename instance actor

* Make it clear in the HTML view that the instance actor should not be blocked

* Raise cache time for instance actor as there's no dynamic content

* Re-use /about/more with a flash message for instance actor profile
2019-07-19 01:44:42 +02:00
Eugen Rochko
f70b20a01c Add a spam check (#11217)
* Add a spam check

* Use Nilsimsa to generate locality-sensitive hashes and compare using Levenshtein distance

* Add more tests

* Add exemption when the message is a reply to something that mentions the sender

* Use Nilsimsa Compare Value instead of Levenshtein distance

* Use MD5 for messages shorter than 10 characters

* Add message to automated report, do not add non-public statuses to
automated report, add trust level to accounts and make unsilencing
raise the trust level to prevent repeated spam checks on that account

* Expire spam check data after 3 months

* Add support for local statuses, reduce expiration to 1 week, always create a report

* Add content warnings to the spam check and exempt empty statuses

* Change Nilsimsa threshold to 95 and make sure removed statuses are removed from the spam check

* Add all matched statuses into automatic report
2019-07-13 16:45:50 +02:00
ThibG
80b5a5d735 Fix old migration script depending on the StreamEntry model (#11278) 2019-07-10 17:09:10 +02:00
Thibaut Girka
6739c423ff Merge branch 'master' into glitch-soc/merge-upstream 2019-06-28 21:36:50 +02:00
Eugen Rochko
1c612f24e4 Add categories for custom emojis (#11196)
Fix #7940
2019-06-28 15:54:10 +02:00
ThibG
585aa16ec0 Fix old migrations failing because of new version of strong_migrations (#11018) 2019-06-09 22:55:28 +02:00
Thibaut Girka
c23e0e761e Fix old migrations failing because of new version of strong_migrations
Fixes #1099
2019-06-09 22:32:12 +02:00
Thibaut Girka
a4e0071538 Merge remote-tracking branch 'upstream/master' into glitch-soc/merge-upstream
Conflicts:
- Gemfile.lock
- config/webpack/rules/css.js
- package.json
- yarn.lock
2019-06-03 14:54:30 +02:00
ThibG
e083961f32 Fix old migrations failing because of new version of strong_migrations (#10904) 2019-05-30 22:35:29 +03:00
Thibaut Girka
5481469322 Fix old migrations failing because of new version of strong_migrations 2019-05-30 21:20:42 +02:00
Eugen Rochko
865c72cde3 Change default to single column UI (#10847)
Keep current setting for users who were active within the last month
2019-05-30 14:30:38 +03:00
Thibaut Girka
105f03a988 Add database row to hold status content type 2019-05-17 23:51:14 +02:00
Thibaut Girka
422dd06806 Merge branch 'master' into glitch-soc/merge-upstream 2019-05-15 17:11:40 +02:00
ThibG
f1a3135809 Record account suspend/silence time and keep track of domain blocks (#10660)
* Record account suspend/silence time and keep track of domain blocks

* Also unblock users who were suspended/silenced before dates were recorded

* Add tests

* Keep track of suspending date for users suspended through the CLI

* Show accurate number of accounts that would be affected by unsuspending an instance

* Change migration to set silenced_at and suspended_at

* Revert "Also unblock users who were suspended/silenced before dates were recorded"

This reverts commit a015c65d2d1e28c7b7cfab8b3f8cd5fb48b8b71c.

* Switch from using suspended and silenced to suspended_at and silenced_at

* Add post-deployment migration script to remove `suspended` and `silenced` columns

* Use Account#silence! and Account#suspend! instead of updating the underlying property

* Add silenced_at and suspended_at migration to post-migration

* Change account fabricator to translate suspended and silenced attributes

* Minor fixes

* Make unblocking domains always retroactive
2019-05-14 19:05:02 +02:00
Thibaut Girka
1bf48b01e3 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/statuses_controller.rb
  minor conflict because of glitch-soc's theming system
- app/controllers/stream_entries_controller.rb
  minor conflict because of glitch-soc's theming system
2019-05-10 17:09:12 +02:00
Jeong Arm
5ad2a7b5dc Record deleted(by mod) status to prevent re-appear (#10732)
* Record deleted(by mod) status to prevent re-appear

* Move to Tombstone

* Add missing migration script
2019-05-09 22:03:02 +02:00
Thibaut Girka
73bc58c656 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/media_attachment.rb
2019-05-04 16:37:26 +02:00
Eugen Rochko
a6d2fe7165 Add blurhash (#10630)
* Add blurhash

* Use fallback color for spoiler when blurhash missing

* Federate the blurhash and accept it as long as it's at most 5x5

* Display unknown media attachments as blurhash placeholders

* Improve style of embed actions and spoiler button

* Change blurhash resolution from 3x3 to 4x4

* Improve dependency definitions

* Fix code style issues
2019-04-27 03:24:09 +02:00
Thibaut Girka
cde0ac1aa7 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/views/admin/pending_accounts/index.html.haml
  No real conflict, upstream changes too close to glitch-specific
  theming system changes.
- config/navigation.rb
  Upstream redesigned the settings navigation, took those changes
  and re-inserted the flavours menu.
2019-04-10 09:16:08 +02:00
Eugen Rochko
7eb05b8724 Add "why do you want to join" field to invite requests (#10524)
* Add "why do you want to join" field to invite requests

Fix #10512

* Remove unused translations

* Fix broken registrations when no invite request text is submitted
2019-04-09 23:06:30 +09:00
Thibaut Girka
c37bc0b268 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- config/locales/en.yml
  Conflict caused by the glitch-soc-specific “flavour” string being too close
  to the newly introduced “identity_proofs” string. Just included both.
2019-03-20 13:54:00 +01:00
Eugen Rochko
a3c41f3532 Add Keybase integration (#10297)
* create account_identity_proofs table

* add endpoint for keybase to check local proofs

* add async task to update validity and liveness of proofs from keybase

* first pass keybase proof CRUD

* second pass keybase proof creation

* clean up proof list and add badges

* add avatar url to keybase api

* Always highlight the “Identity Proofs” navigation item when interacting with proofs.

* Update translations.

* Add profile URL.

* Reorder proofs.

* Add proofs to bio.

* Update settings/identity_proofs front-end.

* Use `link_to`.

* Only encode query params if they exist.

URLs without params had a trailing `?`.

* Only show live proofs.

* change valid to active in proof list and update liveness before displaying

* minor fixes

* add keybase config at well-known path

* extremely naive feature flagging off the identity proof UI

* fixes for rubocop

* make identity proofs page resilient to potential keybase issues

* normalize i18n

* tweaks for brakeman

* remove two unused translations

* cleanup and add more localizations

* make keybase_contacts an admin setting

* fix ExternalProofService my_domain

* use Addressable::URI in identity proofs

* use active model serializer for keybase proof config

* more cleanup of keybase proof config

* rename proof is_valid and is_live to proof_valid and proof_live

* cleanup

* assorted tweaks for more robust communication with keybase

* Clean up

* Small fixes

* Display verified identity identically to verified links

* Clean up unused CSS

* Add caching for Keybase avatar URLs

* Remove keybase_contacts setting
2019-03-18 21:00:55 +01:00
Thibaut Girka
365de06816 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/accounts_controller.rb
- app/controllers/follower_accounts_controller.rb
- app/controllers/statuses_controller.rb

All conflicts caused by the additional `use_pack` used for glitch-soc's theming
system.
2019-03-18 18:03:27 +01:00
ThibG
9efb9d7f72 Set and store report URIs (#10303)
Fixes #10271
2019-03-17 15:34:56 +01:00
Thibaut Girka
c40e812a66 Merge branch 'master' into glitch-soc/merge-upstream 2019-03-14 20:11:17 +01:00
ThibG
bd3ad7fd16 Migrate existing open_registrations setting to the new registrations_mode (#10269)
* Migrate existing `open_registrations` setting to the new `registrations_mode`

Fixes #10263

* Remove unrelated db changes that have creeped in
2019-03-14 20:10:43 +01:00
Thibaut Girka
741766a01d Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/views/admin/settings/edit.html.haml:
  Conflict due to how the theming settings change.
2019-03-14 17:17:20 +01:00
Eugen Rochko
73fb7bfa0f Admission-based registrations mode (#10250)
Fix #6856
Fix #6951
2019-03-14 05:28:30 +01:00
ThibG
401a3cfdf0 Fix AddLockVersionToPolls migration (#10200) 2019-03-07 01:50:06 +01:00
Thibaut Girka
8124b60a99 Fix AddLockVersionToPolls migration 2019-03-06 23:56:43 +01:00
Thibaut Girka
f123b809a7 Merge branch 'master' into glitch-soc/merge-upstream 2019-03-06 22:39:22 +01:00
ThibG
4f2eb43d8a Add optimistic lock to avoid race conditions when handling votes (#10196)
* Add optimistic lock to avoid race conditions when handling votes

* Force-reload polls when getting `ActiveRecord::StaleObjectError`
2019-03-06 19:53:57 +01:00
Thibaut Girka
c0b876f726 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/status.rb
- db/schema.rb

Both conflicts are caused by us having extra database columns.
2019-03-05 19:23:16 +01:00
ThibG
7da087dabe Store remote votes URI (#10158)
* Store remote votes URI

* Add spec for accepting remote votes

* Make poll vote id generation work the same way as follows
2019-03-04 22:51:23 +01:00
Eugen Rochko
4a3acdc916 Add polls (#10111)
* Add polls

Fix #1629

* Add tests

* Fixes

* Change API for creating polls

* Use name instead of content for votes

* Remove poll validation for remote polls

* Add polls to public pages

* When updating the poll, update options just in case they were changed

* Fix public pages showing both poll and other media
2019-03-03 22:18:23 +01:00
Thibaut Girka
1c37ad108c Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/oauth/authorized_applications_controller.rb
  Two changes too close to each other
- app/controllers/settings/sessions_controller.rb
- app/lib/user_settings_decorator.rb
  Two changes too close to each other
- app/models/media_attachment.rb
  New changes too close to glitch-soc only changes.
- app/models/user.rb
  Two changes too close to each other.
- app/services/remove_status_service.rb
  Kept direct timeline code which had been removed upstream.
- app/views/settings/preferences/show.html.haml
  Two changes too close to each other.
- config/locales/en.yml
  Introduction of a new string too close to glitch-soc-only's “flavour”
- config/locales/ja.yml
  Introduction of a new string too close to glitch-soc-only's “flavour”
- config/locales/pl.yml
  Introduction of a new string too close to glitch-soc-only's “flavour”
- config/locales/simple_form.en.yml
  Introduction of a new string too close to glitch-soc-only's “skin”
- config/locales/simple_form.pl.yml
  Introduction of a new string too close to glitch-soc-only's “skin”
- config/settings.yml
  Reverted upstream's decision of enabling posting application by default.
2019-02-10 21:10:09 +01:00
Eugen Rochko
d2d4d38f22 Add featured hashtags to profiles (#9755)
* Add hashtag filter to profiles

GET /@:username/tagged/:hashtag
GET /api/v1/accounts/:id/statuses?tagged=:hashtag

* Display featured hashtags on public profile

* Use separate model for featured tags

* Update featured hashtag counters on-write

* Limit featured tags to 10
2019-02-04 04:25:59 +01:00
Eugen Rochko
1f303e5591 Add option to overwrite imported data (#9962)
* Add option to overwrite imported data

Fix #7465

* Add import for domain blocks
2019-02-03 03:59:51 +01:00
Thibaut Girka
3be4189998 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- README.md: kept our version
2019-01-28 21:14:47 +01:00
Eugen Rochko
49aaaa77a6 Fix slow fallback of CopyAccountStats migration setting stats to 0 (#9930) 2019-01-28 04:18:35 +01:00
Thibaut Girka
b49d415f37 Merge branch 'master' into glitch-soc/merge-upstream
No conflicts.
2019-01-19 18:28:37 +01:00
ThibG
d0008b5051 Add tombstones for remote statuses (#9830)
* Add Tombstone model to remember object deletion

* Do not recreate a status if it has been deleted

* Record Tombstone for remote deleted items

Also, only record deleted items from same-host actors

* Clear an user's tombstones when their key change
2019-01-18 15:56:55 +01:00
Thibaut Girka
a78447a6a4 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- .eslintrc.yml
  Removed, as upstream removed it.
- app/controllers/admin/statuses_controller.rb
  Minor code cleanup when porting one of our features.
- app/models/account.rb
  Note length validation has changed upstream.
  We now use upstream's validation (dropped legacy glitch-soc
  account metadata stuff) but with configurable limit.
- app/services/post_status_service.rb
  Upstream has added support for scheduled toots, refactoring
  the code a bit. Adapted our changes to this refactoring.
- app/views/stream_entries/_detailed_status.html.haml
  Not a real conflict, changes too close.
- app/views/stream_entries/_simple_status.html.haml
  Not a real conflict, changes too close.
2019-01-10 21:00:30 +01:00
Thibaut Girka
8d47495251 Revert "Revert "Add handler for Move activity (#9629)""
This reverts commit 6ab727bf6a.
2019-01-10 18:46:17 +01:00
Eugen Rochko
fdbd6466f2 Fix regression in custom emoji migration (#9742)
Fix #9741
2019-01-07 09:47:00 +01:00
Eugen Rochko
bed1c8ca9a Fix unique violation in downcase custom emoji domain migration (#9733)
Fix #9727
2019-01-06 23:53:46 +01:00
Eugen Rochko
ae1aaa3b8a Add scheduled statuses (#9706)
Fix #340
2019-01-05 12:43:28 +01:00
Thibaut Girka
6ab727bf6a Revert "Add handler for Move activity (#9629)"
This reverts commit 4e3131f02f.
2019-01-02 15:36:59 +01:00
Thibaut Girka
44f2224606 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts manually resolved:
- app/services/post_status_service.rb
- config/locales/simple_form.pl.yml
- config/routes.rb
- config/webpack/loaders/sass.js
- config/webpack/shared.js
- package.json
- yarn.lock
2019-01-02 15:36:53 +01:00
Eugen Rochko
4e3131f02f Add handler for Move activity (#9629) 2018-12-29 02:24:36 +01:00
Eugen Rochko
bbf9f4f93b Add REST API for creating an account (#9572)
* Add REST API for creating an account

The method is available to apps with a token obtained via the client
credentials grant. It creates a user and account records, as well as
an access token for the app that initiated the request. The user is
unconfirmed, and an e-mail is sent as usual.

The method returns the access token, which the app should save for
later. The REST API is not available to users with unconfirmed
accounts, so the app must be smart to wait for the user to click a
link in their e-mail inbox.

The method is rate-limited by IP to 5 requests per 30 minutes.

* Redirect users back to app from confirmation if they were created with an app

* Add tests

* Return 403 on the method if registrations are not open

* Require agreement param to be true in the API when creating an account
2018-12-24 19:12:38 +01:00
Thibaut Girka
48e6b2360d Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- config/routes.rb
  Upstream changed some admin routes, conflict was because of an added :show
  action for statuses on our side. Kept it.
2018-12-23 11:28:28 +01:00
Eugen Rochko
2e0d617b8b Add moderation warnings (#9519)
* Add moderation warnings

Replace individual routes for disabling, silencing, and suspending
a user, as well as the report update route, with a unified account
action controller that allows you to select an action (none,
disable, silence, suspend) as well as whether it should generate an
e-mail notification with optional custom text. That notification,
with the optional custom text, is saved as a warning.

Additionally, there are warning presets you can configure to save
time when performing the above.

* Use Account#local_username_and_domain
2018-12-22 20:02:09 +01:00
Thibaut Girka
04204427d0 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/directories_controller.rb
- app/controllers/settings/applications_controller.rb
- app/controllers/settings/base_controller.rb
- app/controllers/settings/deletes_controller.rb
- app/controllers/settings/exports_controller.rb
- app/controllers/settings/follower_domains_controller.rb
- app/controllers/settings/imports_controller.rb
- app/controllers/settings/migrations_controller.rb
- app/controllers/settings/notifications_controller.rb
- app/controllers/settings/preferences_controller.rb
- app/controllers/settings/sessions_controller.rb
- app/controllers/settings/two_factor_authentication/confirmations_controller.rb
- app/controllers/settings/two_factor_authentication/recovery_codes_controller.rb
- app/controllers/settings/two_factor_authentications_controller.rb

Conflicts were due to some refactoring already made in glitch-soc
when introducing flavours.
2018-12-15 10:45:53 +01:00
Adam Copp
55ebf8a6e8 Make custom emoji domains case insensitive #9351 (#9474)
* Make custom emoji domains case sensitive #9351

* Fixup style in downcase_domain to comply with codeclimate.

* switch if! to unless

* Don't use transactions, operate in batches.

Also revert spurious schema change.
2018-12-11 05:30:57 +01:00
Thibaut Girka
b62953b3d2 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/javascript/packs/public.js
- app/models/user.rb
- config/settings.yml
- db/schema.rb

Moved public.js changes to settings.js.
2018-12-09 16:08:04 +01:00
Eugen Rochko
fd8ca11f19 Add profile directory (#9427)
Fix #5578
2018-12-06 17:36:11 +01:00
Thibaut Girka
b2e9a23ac9 Add database support for list show-reply preferences 2018-11-28 14:46:07 +01:00
Thibaut Girka
35b6fb4e62 Merge branch 'master' into glitch-soc/merge-upstream 2018-11-27 15:10:58 +01:00
Eugen Rochko
9606e4dabf Change identities id column to a bigint (#9371)
* fix: change Identity's id column to a bigint

This appears to be the last model created using a 5.0 migration,
where column types defaulted to `integer` rather than `bigint`.

This migration changes the column type to match that of all of the
other ID columns.

* Change user_id column in identities to bigint and fix down-migration
2018-11-27 13:56:25 +01:00
Thibaut Girka
478abe3a58 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/models/status.rb

Resolved by taking both changes (not a real conflict, just changes too close
to each other).
2018-11-27 13:23:02 +01:00
Eugen Rochko
62c0e112ea Extract counters from accounts table to account_stats table (#9295) 2018-11-19 00:43:52 +01:00
Thibaut Girka
33ad16faa7 Merge branch 'master' into glitch-soc/merge-upstream 2018-10-31 10:08:36 +01:00
Eugen Rochko
1fad3301a1 Remove progress estimate from MigrateAccountConversations (#9168) 2018-10-30 21:09:33 +01:00
Thibaut Girka
c954f89bdd Merge branch 'master' into glitch-soc/merge-upstream 2018-10-28 08:37:49 +01:00
Eugen Rochko
44e64ee534 Add progress indicators to MigrateAccountConversations (#9101)
* Add progress indicators to MigrateAccountConversations

* Avoid running expensive query for explain

* Use exec_query instead of execute
2018-10-28 06:40:45 +01:00
Thibaut Girka
65cc5faf80 Merge branch 'master' into glitch-soc/merge-upstream 2018-10-27 18:54:26 +02:00
Eugen Rochko
7c01fbf4fd Add locality check to ActivityPub::FetchRemoteAccountService (#9109)
* Add locality check to ActivityPub::FetchRemoteAccountService

Fix #8643

Because there are a few places where it is called, it is difficult
to confirm if they all previously checked it for locality. It's better
to make sure within the service.

* Remove faux-remote duplicates of local accounts
2018-10-26 23:08:34 +02:00
Thibaut Girka
c32a5f86b6 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- app/controllers/admin/base_controller.rb
- app/controllers/filters_controller.rb
- app/controllers/invites_controller.rb
- app/controllers/settings/deletes_controller.rb
- app/controllers/settings/exports_controller.rb
- app/controllers/settings/follower_domains_controller.rb
- app/controllers/settings/migrations_controller.rb
- app/controllers/settings/notifications_controller.rb
- app/controllers/settings/preferences_controller.rb
- app/controllers/settings/two_factor_authentication/recovery_codes_controller.rb
- app/javascript/packs/public.js
- app/views/settings/profiles/show.html.haml

Conflicts were mostly due to the addition of body classes to the settings page,
this was caused by rejecting upstream changes for most of those files and
modifying Settings::BaseController instead.

Another cause of conflicts was the deletion of client-side checking of
display name / bio length, this was modified in app/javascript/core/settings.js
instead.
2018-10-26 20:41:43 +02:00
Eugen Rochko
db85c8f0f6 Migrate all old direct messages to new conversations schema (#9085) 2018-10-25 01:44:02 +02:00
Thibaut Girka
46259a36d0 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
- .github/ISSUE_TEMPLATE/bug_report.md
  Took our version.
- CONTRIBUTING.md
  Updated the embedded copy of upstream's version.
- README.md
  Took our version.
- app/policies/status_policy.rb
  Not a real conflict, took code from both.
- app/views/layouts/embedded.html.haml
  Added upstream's changes (dns-prefetch) and fixed
  `%body.embed`
- app/views/settings/preferences/show.html.haml
  Reverted some of upstream changes, as we have a
  page dedicated for flavours and skins.
- config/initializers/content_security_policy.rb
  Kept our version of the CSP.
- config/initializers/doorkeeper.rb
  Not a real conflict, took code from both.
2018-10-22 17:51:38 +02:00
Eugen Rochko
40d23fc4d1 Add option to block reports from domain (#8830) 2018-10-20 08:02:44 +02:00
Eugen Rochko
e3b2234382 Add unread indicator to conversations (#9009) 2018-10-19 01:47:29 +02:00
Eugen Rochko
654520ec8c Improve support for aspects/circles (#8950)
* Add silent column to mentions

* Save silent mentions in ActivityPub Create handler and optimize it

Move networking calls out of the database transaction

* Add "limited" visibility level masked as "private" in the API

Unlike DMs, limited statuses are pushed into home feeds. The access
control rules between direct and limited statuses is almost the same,
except for counter and conversation logic

* Ensure silent column is non-null, add spec

* Ensure filters don't check silent mentions for blocks/mutes

As those are "this person is also allowed to see" rather than "this
person is involved", therefore does not warrant filtering

* Clean up code

* Use Status#active_mentions to limit returned mentions

* Fix code style issues

* Use Status#active_mentions in Notification

And remove stream_entry eager-loading from Notification
2018-10-17 17:13:04 +02:00
Thibaut Girka
8cb843039c Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	app/javascript/packs/public.js

Changes made to app/javascript/packs/publics.js were applied to
app/javascript/core/settings.js
2018-10-09 21:08:26 +02:00
Eugen Rochko
ad0f37e924 Track historical space stats in PgHero to determine PostgreSQL growth (#8906) 2018-10-09 19:35:14 +02:00
Thibaut Girka
74e411f4e8 Merge branch 'master' into glitch-soc/merge-upstream 2018-10-08 13:51:33 +02:00
Eugen Rochko
c9b5168ebd Add conversations API (#8832)
* Add conversations API

* Add web UI for conversations

* Add test for conversations API

* Add tests for ConversationAccount

* Improve web UI

* Rename ConversationAccount to AccountConversation

* Remove conversations on block and mute

* Change last_status_id to be a denormalization of status_ids

* Add optimistic locking
2018-10-07 23:44:58 +02:00
Thibaut Girka
bdbbbb1bdb Merge branch 'master' into glitch-soc/merge-upstream 2018-10-07 19:47:56 +02:00
Eugen Rochko
97ca9b60a4 Add fallback for PostgreSQL without upsert in CopyStatusStats (#8903)
Fix #8590
2018-10-07 04:40:37 +02:00
Thibaut Girka
f627ea99e4 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	db/migrate/20170716191202_add_hide_notifications_to_mute.rb
	spec/controllers/application_controller_spec.rb

Took our version, upstream changes were only minor style linting.
2018-10-05 15:23:57 +02:00
aus-social
c883b1ffc9 lint pass 2 (#8878)
* Code quality pass

* Typofix

* Update applications_controller_spec.rb

* Update applications_controller_spec.rb
2018-10-04 17:38:04 +02:00
aus-social
a53bcb6213 Lint pass (#8876) 2018-10-04 12:36:53 +02:00
Thibaut Girka
86d2a9d480 Merge branch 'master' into glitch-soc/merge-upstream 2018-08-22 20:56:32 +02:00
Eugen Rochko
e3e11e107a Add missing indices for ON DELETE CASCADE constraints (#8332)
Fix #8327
2018-08-21 20:11:34 +02:00
Thibaut Girka
c043697caa Merge branch 'master' into glitch-soc/merge-upstream 2018-08-19 09:27:18 +02:00
Eugen Rochko
c28ed5d760 Do not use WHERE NOT IN in CopyStatusStats migration (#8281)
Fix #8275

As the batch operation progresses, the statuses_stats table grows,
and the WHERE NOT IN subquery becomes more expensive
2018-08-19 02:01:49 +02:00
Thibaut Girka
767cc06fe2 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	.circleci/config.yml
	app/controllers/authorize_follows_controller.rb
	app/javascript/packs/public.js

Moved new stuff from packs/public.js to core/public.js.
Added appropriate use_pack in new controllers.
2018-08-18 18:04:49 +02:00
Eugen Rochko
9e5285bfb1 Make CopyStatusStats migration use batches of 1000 to avoid locks (#8256) 2018-08-18 03:44:50 +02:00
Thibaut Girka
334f478db1 Merge branch 'master' into glitch-soc/merge-upstream
Conflicts:
	app/models/status.rb
	db/migrate/20180528141303_fix_accounts_unique_index.rb
	db/schema.rb

Resolved by taking upstream changes (no real conflicts, just glitch-soc
specific code too close to actual changes).
2018-08-17 17:43:54 +02:00
Thibaut Girka
ad7e23cc32 Split custom filter migration script using the post-deployment migration system
There were some concerns with the custom filter migration script dropping a table,
thus making it unsafe to run in a zero-downtime setting. Upstream introduced
a way to run migrations after deployment, so revisit the old migration script to
make use of this.
2018-08-16 20:30:09 +02:00
ThibG
d329c3ab5e Fix FixAccountsUniqueIndex migration (#8212) 2018-08-15 20:23:12 +02:00
Eugen Rochko
659556ab1f Upgrade Doorkeeper to 4.4.1 (#8197) 2018-08-14 20:24:47 +02:00
Eugen Rochko
15c77f178f Fix null constraint violation in copy status stats migration (#8198) 2018-08-14 20:24:36 +02:00
Eugen Rochko
bf5acc3eae Move status counters to separate table, count replies (#8104)
* Move status counters to separate table, count replies

* Migration to remove old counter columns from statuses table

* Fix schema file
2018-08-14 19:19:32 +02:00