Commit graph

951 commits

Author SHA1 Message Date
David Yip
3c3e6d0e9b Merge remote-tracking branch 'origin/master' into merge-upstream
Conflicts:
 	app/controllers/follower_accounts_controller.rb
 	app/controllers/following_accounts_controller.rb
 	app/controllers/settings/preferences_controller.rb
 	app/lib/user_settings_decorator.rb
 	app/models/user.rb
 	config/locales/simple_form.en.yml
2018-05-18 08:47:10 -05:00
Eugen Rochko
aeceb63275 Add preference to hide following/followers lists (#7532)
* Add preference to hide following/followers lists

- Public pages
- ActivityPub collections (does not return pages but does give total)
- REST API (unless it's your own) (does not federate)

Fix #6901

* Add preference

* Add delegation

* Fix issue

* Fix issue
2018-05-18 02:26:51 +02:00
Thibaut Girka
efd2441d1f Merge branch 'master' into glitch-soc/merge 2018-05-14 20:51:50 +02:00
Eugen Rochko
52a2498f60 Fix nil error in StatusFilter (#7470)
Fix #7462
2018-05-13 20:26:30 +02:00
Thibaut Girka
d0b753db6b Merge branch 'master' into glitch-soc/merge
Conflicts:
	app/controllers/invites_controller.rb
	app/serializers/initial_state_serializer.rb
	config/locales/ko.yml
2018-05-11 18:12:42 +02:00
ThibG
b59416500e Catch Paperclip processing failures (fixes #6378) (#7439) 2018-05-11 13:14:50 +02:00
Eugen Rochko
e86a4fe36b Add REST API for Web Push Notifications subscriptions (#7445)
- POST /api/v1/push/subscription
- PUT /api/v1/push/subscription
- DELETE /api/v1/push/subscription
- New OAuth scope: "push" (required for the above methods)
2018-05-11 11:49:12 +02:00
Thibaut Girka
af504e62ff Merge branch 'master' into glitch-soc/master
Conflicts:
	app/models/account.rb
	app/views/accounts/_header.html.haml
2018-05-10 00:03:28 +02:00
David Yip
106a0d5c70 Merge remote-tracking branch 'glitchsoc/master' into gs-master 2018-05-08 09:43:33 -05:00
Eugen Rochko
7ff3c849d1 Bot nameplates (#7391)
* Store actor type in database

* Add bot nameplate to web UI, add setting to preferences, API, AP
Fix #7365

* Fix code style issues
2018-05-07 09:31:07 +02:00
Eugen Rochko
4055ce490c Enable custom emojis in profiles (notes, field values, display names) (#7374)
Follow-up to #6124
2018-05-06 11:48:51 +02:00
Eugen Rochko
0de45574e0 4 profile fields max, store only 255 characters per name/value (#7348)
Fix #7303
2018-05-05 21:11:19 +02:00
Eugen Rochko
8913ccb434 Store home feeds for 7 days instead of 14 (#7354)
* Store home feeds for 7 days instead of 14

Reduces workload for status fan-out to active followers

* Fix test for user model
2018-05-05 00:54:24 +02:00
Daggertooth
b60e38c985 Escape metachars in keywords 2018-05-04 14:58:11 -05:00
David Yip
087c79669a Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	app/controllers/follower_accounts_controller.rb
 	app/controllers/following_accounts_controller.rb
    	db/schema.rb
2018-05-04 14:16:30 -05:00
Eugen Rochko
154076e8e7 Store URIs of follows, follow requests and blocks for ActivityPub (#7160)
Same URI passed between follow request and follow, since they are
the same thing in ActivityPub. Local URIs are generated during
creation using UUIDs and are passed to serializers.
2018-05-04 21:14:34 +02:00
David Yip
976347fa78 Merge remote-tracking branch 'origin/master' into merge-upstream 2018-05-04 03:23:57 -05:00
David Yip
dde7fa8bbe Remove unnecessary code from MediaAttachment#set_type_and_extension.
Well, it seems unnecessary, anyway.
2018-05-04 02:38:04 -05:00
David Yip
c87f1d99e4 Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	.travis.yml
 	Gemfile.lock
 	README.md
 	app/controllers/settings/follower_domains_controller.rb
 	app/controllers/statuses_controller.rb
 	app/javascript/mastodon/locales/ja.json
 	app/lib/feed_manager.rb
 	app/models/media_attachment.rb
 	app/models/mute.rb
 	app/models/status.rb
 	app/services/mute_service.rb
 	app/views/home/index.html.haml
 	app/views/stream_entries/_simple_status.html.haml
 	config/locales/ca.yml
 	config/locales/en.yml
 	config/locales/es.yml
 	config/locales/fr.yml
 	config/locales/nl.yml
 	config/locales/pl.yml
 	config/locales/pt-BR.yml
 	config/themes.yml
2018-05-03 17:23:44 -05:00
ThibG
f2ecedaab5 Fixes/do not override timestamps (#7336)
* Revert "Fixes/do not override timestamps (#7331)"

This reverts commit 9c1e1005f4.

* Document Snowflake ID corner-case a bit more

Snowflake IDs are used for two purposes: making object identifiers harder to
guess and ensuring they are in chronological order. For this reason, they
are based on the `created_at` attribute of the object.

Unfortunately, inserting items with older snowflakes IDs will break the
assumption of consumers of the paging APIs that new items will always have
a greater identifier than the last seen one.

* Add `override_timestamps` virtual attribute to not correlate snowflake ID with created_at
2018-05-03 23:02:46 +02:00
Eugen Rochko
92c5474861 Fix n+1 queries in StatusThreadingConcern (#7321) 2018-05-03 10:41:58 +02:00
Eugen Rochko
5dd68ba919 Fix cache_associated no longer working (#7320) 2018-05-03 10:41:41 +02:00
abcang
fcfc44de66 Keep notification when muting_notifications is true (#7311)
* Keep notification when muting_notifications is true

* Retrun mute object

* Fix test
2018-05-02 16:14:51 +02:00
Eugen Rochko
4258e304c5 Allow updating bio fields via PUT /api/v1/accounts/update_credentials (#7288)
Add raw bio fields to the source attribute on GET /api/v1/accounts/verify_credentials
2018-05-02 15:57:37 +02:00
Eugen Rochko
d6a2530d73 Speed up test suite by not generating RSA keys in test environment (#7296)
One RSA keypair for all fabricated test accounts is enough
2018-05-02 15:45:24 +02:00
David Yip
e484bd61e0 Also treat non-whole-word mutes as case-insensitive. #450. 2018-04-30 20:08:22 -05:00
Eugen Rochko
1a653fff95 Add entity cache (#7271)
* Add entity cache

Use a caching layer for mentions and custom emojis that are
dynamically extracted from text.

Reduce duplicate text extractions

* Fix code style issue
2018-04-27 01:38:10 +02:00
Emelia Smith
49d9ea6998 Implement the ability for instances to define a list of disallowed hashtags (#7176)
The goal here isn't to prevent these hashtags from existing, but just to strongly curtail their usage; The hashtags may still exist in the database via federated status, or from being created prior to this feature.
2018-04-23 23:52:58 +02:00
Emelia Smith
1071ecfebc Prevent suspended accounts from appearing in AccountSearchService (#7246) 2018-04-23 21:27:18 +02:00
Akihiko Odaki
e22f682df0 Paginate descendant statuses in public page (#7148) 2018-04-23 19:27:35 +02:00
Yamagishi Kazutoshi
e83e74f36d Update dependencies for Ruby (2018-04-23) (#7237)
* Update annotate to version 2.7.3

* Update aws-sdk-s3 to version 1.9.2

* Update browser to version 2.5.3

* Update capistrano to version 3.10.2

* Update domain_name to version 0.5.20180417

* Update http to version 3.2.0

* Update lograge to version 0.10.0

* Update oj to version 3.5.1

* Update parallel_tests to version 2.21.3

* Update puma to version 3.11.4

* Update rubocop to version 0.55.0

* Update scss_lint to version 0.57.0

* Update simplecov to version 0.16.1

* Update tty-command to version 0.8.0

* Update tty-prompt to version 0.16.0

* Update pkg-config to version 1.3.0

* Update fog-local to version 0.5.0

* Update fog-openstack to version 0.1.25

* Update devise-two-factor to version 3.0.3

* bundle update
2018-04-23 11:29:17 +02:00
Eugen Rochko
39f36c0901 Detect and prevent image bombs, max. processable dimension 4096^2 (#7229) 2018-04-23 09:16:38 +02:00
Eugen Rochko
0ad5e05e4a Rescue Mastodon::LengthValidationError in Remoteable (#7228)
Fix #7198 by allowing records with optional attachments to save
2018-04-22 15:42:00 +02:00
Eugen Rochko
85c3f7eb1b Force convert to JPG for preview card thumbnails to avoid animations (#7109)
* Force convert to JPG for preview card thumbnails to avoid animations

Fix #7093

* Conditionally convert to JPG only if original is GIF
Coalesce and strip on all formats to ensure no animated APNGs
2018-04-21 21:34:36 +02:00
David Yip
48f5f736a0 db: make schema.rb version subsume bookmarks migration. 2018-04-18 19:08:35 -05:00
David Yip
bae5435010 Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	app/controllers/home_controller.rb
 	app/controllers/stream_entries_controller.rb
 	app/javascript/mastodon/locales/ja.json
 	app/javascript/mastodon/locales/pl.json
2018-04-18 18:48:12 -05:00
Kaito Sinclaire
447741e456 Direct messages column (#4514)
* Added a timeline for Direct statuses
* Lists all Direct statuses you've sent and received
* Displayed in Getting Started
* Streaming server support for direct TL

* Changes to match other timelines in 2.0
2018-04-18 13:09:06 +02:00
Eugen Rochko
4f460eba18 Fix warning about using SQL in order for Account#partitioned (#7159) 2018-04-17 13:50:48 +02:00
David Yip
9697c6d9f0 Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	db/schema.rb
2018-04-14 09:12:50 -05:00
Eugen Rochko
466442b764 Add bio fields (#6645)
* Add bio fields

- Fix #3211
- Fix #232
- Fix #121

* Display bio fields in web UI

* Fix output of links and missing fields

* Federate bio fields over ActivityPub as PropertyValue

* Improve how the fields are stored, add to Edit profile form

* Add rel=me to links in fields

Fix #121
2018-04-14 12:41:08 +02:00
David Yip
26f770cf72 Merge remote-tracking branch 'glitchsoc/master' into gs-master
Conflicts:
 	db/schema.rb
2018-04-14 03:50:31 -05:00
David Yip
605016368d Merge pull request #419 from ThibG/glitch-soc/features/bookmarks
Bookmarks
2018-04-13 16:38:15 -05:00
David Yip
fd98bfd108 Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	Gemfile.lock
 	config/application.rb
2018-04-13 16:36:46 -05:00
Yamagishi Kazutoshi
9761b940ac Upgrade Rails to version 5.2.0 (#5898) 2018-04-12 14:45:17 +02:00
David Yip
ac1ac20ee9 Merge remote-tracking branch 'tootsuite/master'
Conflicts:
 	app/controllers/statuses_controller.rb
2018-04-12 03:30:57 -05:00
Thibaut Girka
12f8473561 Add backend support for bookmarks
Bookmarks behave like favourites, except they aren't shared with other
users and do not have an associated counter.
2018-04-11 20:40:29 +02:00
Akihiko Odaki
6c2344d4dd Paginate ancestor statuses in public page (#7102)
This also limits the statuses returned by API, but pagination is not
implemented in Web API yet. I still expect it brings user experience
better than making a user wait to fetch all ancestor statuses and flooding
the column with them.
2018-04-11 12:35:09 +02:00
Jenkins
e4e0aa5d21 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-04-11 00:50:09 +00:00
Emelia Smith
5a166a19a7 Feature: Improve reports ui (#7032)
* Further improvements to Reports UI

- Clean up notes display
- Clean up add new note form
- Simplify controller
- Allow reopening a report with a note
- Show created at date for reports
- Fix report details table formatting

* Show history of report using Admin::ActionLog beneath the report

* Fix incorrect log message when reopening a report

* Implement fetching of all ActionLog items that could be related to the report

* Ensure adding a report_note updates the report's updated_at

* Limit Report History to actions that happened between the report being created and the report being resolved

* Fix linting issues

* Improve report history builder

Thanks @gargron for the improvements
2018-04-10 20:27:59 +02:00
Paul Woolcock
ed21e48ad6 Change custom emoji search to ILIKE instead of = (#7099) 2018-04-10 15:46:27 +02:00
Emelia Smith
765bfd0cf5 Feature: Allow staff to change user emails (#7074)
* Admin: Show unconfirmed email address on account page

* Admin: Allow staff to change user email addresses

* ActionLog: On change_email, log current email address and new unconfirmed email address
2018-04-10 09:16:06 +02:00
MIYAGI Hikaru
62c6b71866 Fix GIFV encoding params (#7098)
- Explicitly specify video codec.
  When ffmpeg isn't compiled with libx264 but openh264, mpeg4 is selected as video codec.
- Swap avarage bitrate and max bitrate.
2018-04-10 09:11:55 +02:00
Akihiko Odaki
31e1667933 Exclude status itself from context query (#7083)
ancestor_statuses and descendant_statuses used to include the root status
itself, but the behavior is confusing because the root status is not
an ancestor nor descendant.
2018-04-09 09:58:53 +02:00
Jenkins
62ce70f28c Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-04-03 19:17:20 +00:00
Emelia Smith
5373b138e8 Feature: Report improvements (#6967) (#7000)
* Implement Assignment of Reports (#6967)

* Change translation of admin.report.comment.label to "Report Comment" for clarity

As we'll soon add the ability for reports to have comments on them, this clarification makes sense.

* Implement notes for Reports

This enables moderators to leave comments about a report whilst they work on it

* Fix display of report moderation notes

* Allow reports to be reopened / marked as unresolved

* Redirect to reports listing upon resolution of report

* Implement "resolve with note" functionality

* Add inverse relationship for report notes

* Remove additional database querying when loading report notes

* Fix tests for reports

* Fix localisations for report notes / reports
2018-04-02 22:04:14 +02:00
David Yip
21394c5462 Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	app/javascript/mastodon/locales/en.json
 	app/javascript/mastodon/locales/ja.json
 	app/javascript/mastodon/locales/pl.json
 	app/views/accounts/_header.html.haml
2018-04-01 20:24:50 -05:00
David Underwood
44274257de [WIP] Enable custom emoji on account pages and in the sidebar (#6124)
Federate custom emojis with accounts
2018-04-01 23:55:42 +02:00
David Yip
2567d0a704 Merge remote-tracking branch 'origin/master' into gs-master 2018-03-27 10:44:12 -05:00
David Yip
7cdf2b05b6 Merge remote-tracking branch 'origin/master' into gs-master
Conflicts:
 	app/javascript/styles/mastodon/components.scss
 	app/models/media_attachment.rb
2018-03-27 10:26:47 -05:00
Eugen Rochko
ebba6fb7f5 Fix UniqueUsernameValidator comparison (#6926)
Comparison was downcasing only one side, therefore if previously
existing account had a non-lowercase spelling, it would be ignored
when checking for duplicates.

New rake task `mastodon:maintenance:find_duplicate_usernames` will
help find constraint violations that might have occured from the
presence of this bug.

Bump version to 2.3.3
2018-03-27 04:33:57 +02:00
Akihiko Odaki
acece7a2e6 Validate HTTP response length while receiving (#6891)
to_s method of HTTP::Response keeps blocking while it receives the whole
content, no matter how it is big. This means it may waste time to receive
unacceptably large files. It may also consume memory and disk in the
process. This solves the inefficency by checking response length while
receiving.
2018-03-26 14:02:10 +02:00
Jenkins
ae55717f50 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-03-25 15:17:21 +00:00
Akihiko Odaki
472fba6771 Change columns in notifications nonnullable (#6764) 2018-03-24 12:51:28 +01:00
Akihiko Odaki
11c19f6cc9 Close http connection in perform method of Request class (#6889)
HTTP connections must be explicitly closed in many cases, and letting
perform method close connections makes its callers less redundant and
prevent them from forgetting to close connections.
2018-03-24 12:49:54 +01:00
Jenkins
60eafb173f Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-03-23 10:17:18 +00:00
Eugen Rochko
f4e8e39c99 Restore username validation to disallow dots, for now (#6863)
Usernames with dots in them do not work with routes, because the dot usually separates the desired page format (e.g. json). I don't want to mess with changing route constraints for this patch release.
2018-03-22 11:30:22 +01:00
ThibG
7d7a725e82 Fix MENTION_RE to not match nil usernames (#6862) 2018-03-22 10:45:48 +01:00
Jenkins
a9bcd383b8 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-03-21 17:17:20 +00:00
Eugen Rochko
53b1100144 Permit dots in usernames with conditions (#6844)
* Permit dots in usernames with conditions

- Dot cannot be the start or end of username
- a.lice and al.ice are considered the same during sign-up

* Fix regex mixin flags
2018-03-21 10:26:53 +01:00
Jenkins
eba1b109db Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-03-19 20:17:18 +00:00
Alexander
4fc9827b69 bugfixes and gem update (#6831)
* update to new version of devise_pam_authenticatable2

* fix behaviour if suffix is nil, fix environment loading, fix user email creation

* code cleanup/fix linter warning
2018-03-19 20:08:56 +01:00
Jenkins
3d16993fbe Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-03-17 17:17:21 +00:00
Daniel Hunsaker
8f374100ed Handle Mastodon::HostValidationError when pulling remoteable assets (#6782)
This will prevent, for example, `rake mastodon:redownload_avatars` from crashing when an instance is no longer responding to connection attempts, instead silently continuing as expected.
2018-03-17 13:27:50 +01:00
David Yip
9ca0336c15 Merge remote-tracking branch 'origin/master' into merge-upstream
Conflicts:
	README.md
2018-03-12 17:52:19 -05:00
Eugen Rochko
51b32f06fa Fix #6715: Make catalan words with the L geminate letter work in hashtags (#6741) 2018-03-11 14:55:49 +01:00
Jenkins
54608d0486 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-03-09 00:17:17 +00:00
Renato "Lond" Cerqueira
34b3d46480 Rescue when there's no extension in the remotable (#6358)
* Rescue when there's no extension in the remotable
Sometimes the remotable is pointing to a directory with no file
extension. Maybe it should not be expecting to identify based on
extensions to begin with, but since it's the case, it should be ready
for it.

* Fix codeclimate issue

* Check if filename is nil instead of rescueing exception
Suggestion made in the PR

* Avoid concatenation issue if filename is nil
If filename is nil, extname was undefined

* Invert condition
Address PR comments
2018-03-08 02:25:10 +01:00
Eugen Rochko
0c2a08adeb Remove text requirement when media attached from statuses (#6672) 2018-03-07 08:28:52 +01:00
Effy Elden
4a8046df66 Add additional first_name and last_name SAML attribute statement options, and modify Omniauthable concern to use full_name or first_name + last_name if not available (#6669) 2018-03-07 06:19:10 +01:00
Eugen Rochko
ba865c44d8 Fix #3807: Increase avatars to 400x400 max (#6651)
But do not upscale when they are smaller
2018-03-06 06:29:01 +01:00
Akihiko Odaki
649565e005 Provide default OTP_SECRET value for development environment (#6617) 2018-03-04 20:28:24 +01:00
Eugen Rochko
b1f8dfd3cf Federate pinned statuses over ActivityPub (#6610)
* Federate pinned statuses over ActivityPub

* Display pinned toots in web UI

Fix #6117

* Fix migration

* Fix tests

* Update outbox_serializer.rb

* Update remove_serializer.rb

* Update add_serializer.rb

* Update fetch_featured_collection_service.rb
2018-03-04 09:19:11 +01:00
David Yip
c08c971dd3 Merge remote-tracking branch 'origin/master' into merge-upstream
Conflicts:
 	README.md
 	app/controllers/follower_accounts_controller.rb
 	app/controllers/following_accounts_controller.rb
 	app/serializers/rest/instance_serializer.rb
 	app/views/stream_entries/_simple_status.html.haml
 	config/locales/simple_form.ja.yml
2018-03-02 21:46:44 -06:00
Eugen Rochko
9721b7746a Fix #942: Seamless LDAP login (#6556) 2018-02-28 19:04:53 +01:00
Eugen Rochko
0ebd2a1718 Federated reports (#6570)
* Fix #2176: Federated reports

* UI for federated reports

* Add spec for ActivityPub Flag handler

* Add spec for ReportService
2018-02-28 06:54:55 +01:00
Ian McCowan
88461c1abe Fix prev/next links on public profile page (#6497)
* Fix prev/next links on public profile page

* Don't make pagination urls if no available statuses

* Fix empty check method

* Put left chevron before prev page link

* Add scope for pagination "starting at" a given id

* Status pagination try 2:

s/prev/older and s/next/newer
"older" on left, "newer" on right
Use new scope for "newer" link
Extract magic 20 page size to constant
Remove max_id from feed pagination as it's not respected

* Reinstate max_id for accounts atom stream

* normalize
2018-02-26 03:31:28 +01:00
Eugen Rochko
f35356609b Ensure the app does not even start if OTP_SECRET is not set (#6557)
* Ensure the app does not even start if OTP_SECRET is not set

* Remove PAPERCLIP_SECRET (it's not used by anything, actually)

Imports are for internal consumption and the url option isn't even
used correctly, so we can remove the hash stuff from them
2018-02-26 01:31:44 +01:00
imncls
c0aabbec0f Merge branch 'master' of https://github.com/tootsuite/mastodon
# Conflicts:
#	app/controllers/settings/exports_controller.rb
#	app/models/media_attachment.rb
#	app/models/status.rb
#	app/views/about/show.html.haml
#	docker_entrypoint.sh
#	spec/views/about/show.html.haml_spec.rb
2018-02-23 23:28:31 +09:00
Ghislain Loaec
d1806f5dc4 New variable OAUTH_REDIRECT_AT_SIGN_IN + Ref #6538 (not only SAML strategies) (#6540) 2018-02-23 01:16:17 +01:00
Ghislain Loaec
deea524052 New env variable: SAML_SECURITY_ASSUME_EMAIL_IS_VERIFIED + fixes #6533 (#6538) 2018-02-22 23:31:25 +01:00
Eugen Rochko
865c7e7178 Focal points (#6520)
* Add focus param to media API, center thumbnails on focus point

* Add UI for setting a focal point

* Improve focal point icon on upload item

* Use focal point in upload preview

* Add focalPoint property to ActivityPub

* Don't show focal point button for non-image attachments
2018-02-22 00:35:46 +01:00
Eugen Rochko
c9ed272a4a Fix #6525: Make sure file is opened in LazyThumbnail processor (#6529) 2018-02-22 00:28:19 +01:00
Eugen Rochko
f6884555d7 Account archive download (#6460)
* Fix #201: Account archive download

* Export actor and private key in the archive

* Optimize BackupService

- Add conversation to cached associations of status, because
  somehow it was forgotten and is source of N+1 queries
- Explicitly call GC between batches of records being fetched
  (Model class allocations are the worst offender)
- Stream media files into the tar in 1MB chunks
  (Do not allocate media file (up to 8MB) as string into memory)
- Use #bytesize instead of #size to calculate file size for JSON
  (Fix FileOverflow error)
- Segment media into subfolders by status ID because apparently
  GIF-to-MP4 media are all named "media.mp4" for some reason

* Keep uniquely generated filename in Paperclip::GifTranscoder

* Ensure dumped files do not overwrite each other by maintaing directory partitions

* Give tar archives a good name

* Add scheduler to remove week-old backups

* Fix code style issue
2018-02-21 23:21:32 +01:00
Eugen Rochko
c34ff9fb82 Fix avatar and header issues by using custom geometry detector (#6515)
* Fix avatar and header issues by using custom geometry detector

Revert a part of #6508. The file passed to dynamic styles method
was not actually a file, but an instance of Paperclip::Attachment,
which broke all styles by always returning {} from the method.

One problem with GIF avatars was that Paperclip::GeometryDetector
reported wrong dimensions for them, e.g. 120x120 GIF avatar would
for some reason be detected as 120x53. By writing our own geometry
parser, we can use FastImage, which also happens to be faster than
ImageMagick, to detect image dimensions, which are also correct.

Unfortunately, this PR does not implement skipping a `convert`
entirely if the dimensions are already correct, as I found no easy
way to write that behaviour into Paperclip without rewriting the
Paperclip::Thumbnail class.

* Only invoke convert if dimension or format needs to be changed
2018-02-21 03:40:12 +01:00
Jenkins
e768ef35b1 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-02-19 15:17:14 +00:00
Eugen Rochko
f759ec9d8d Don't resize avatars/headers if their dimensions are already right (#6508)
Also don't apply "-quality 80" option which is probably the reason
for slight color differences between original and remote image
(because it would apply it twice, once on original instance, and
again on the receiving instance)
2018-02-19 16:06:12 +01:00
Eugen Rochko
750f3e64c9 Cache relationships in API (#6482)
* Cache relationships in API

* Fetch relationships for search results in UI

* Only save one account's maps in each cache item
2018-02-18 03:14:46 +01:00
Eugen Rochko
3f2a113d32 Push discovered status through streaming API within a time window (#6484)
Time window of 6 hours
2018-02-17 14:28:48 +01:00
David Yip
596deb050b Merge remote-tracking branch 'tootsuite/master' into merge-upstream
Conflicts:
	Gemfile
	config/locales/simple_form.pl.yml
2018-02-17 00:02:37 -06:00
Eugen Rochko
9b7490cede Save video metadata and improve video OpenGraph tags (#6481)
* Save metadata from video attachments, put correct dimensions into OG tags

* Add twitter:player for videos

* Fix code style and test
2018-02-16 07:22:20 +01:00
Jenkins
3a1f58e9eb Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-02-11 04:10:16 +00:00
David Yip
d9cc3b60fb Merge remote-tracking branch 'origin/merge-upstream' 2018-02-10 22:08:45 -06:00
David Yip
062f72c593 Use Html2Text to generate plaintext for keyword mutes. #236.
This allows us to match URLs inside link hrefs.
2018-02-10 10:36:16 -06:00
David Yip
00cb763893 Glitch::FilterHelper -> Glitch::KeywordMuteHelper. #234.
The class helps out with keyword mutes, not just some general concept of
"filtering".
2018-02-10 10:36:15 -06:00
David Yip
a5f1c1f469 Fill out some examples for Glitch::FilterHelper. #234.
Also add HTML entity decoding to Glitch::FilterHelper, which is needed
to e.g. match "<" to the tag-stripped version of "<p><3</p>" or
"<p>&lt;3</p>".
2018-02-10 02:40:27 -06:00
David Yip
e365675e0a Strip HTML from keyword mute input. #234. 2018-02-10 02:40:27 -06:00
Eugen Rochko
7ca5a06505 Full-text search for authorized statuses (#6423)
* Add full-text search for authorized statuses

- Search API will return statuses that match the query
- Only for logged in users
- Only if you are author of the status,
- Or you were mentioned in it
- Or you favourited or reblogged it
- Configuration over `ES_ENABLED`, `ES_HOST`, `ES_PORT`, `ES_PREFIX`
- Run `rails chewy:deploy` to create & populate index

Fix #5880
Fix #4293
Fix #1152

* Add commented out docker-compose configuration for ES container

* Optimize index import, filter search results

* Add basic normalization to the index

* Add better stemming and normalization to the index

* Skip webfinger request if search query includes both @ and a space

* Fix code style

* Visually separate search result sections

* Fix code style issues
2018-02-09 23:04:47 +01:00
David Yip
799b205766 Merge remote-tracking branch 'origin/master' into merge-upstream
Conflicts:
	.travis.yml
	app/lib/user_settings_decorator.rb
	app/models/user.rb
	app/serializers/initial_state_serializer.rb
	app/views/stream_entries/_detailed_status.html.haml
	app/views/stream_entries/_simple_status.html.haml
	config/locales/simple_form.en.yml
2018-02-09 09:25:53 -06:00
Eugen Rochko
219b28e172 Add preference to always display sensitive media (#6448) 2018-02-09 00:26:57 +01:00
Jenkins
777559fa5f Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-02-08 23:17:14 +00:00
abcang
c1263f5db5 Clear account cache of notification target_status (#6442) 2018-02-08 15:33:23 +01:00
Akihiko Odaki
72e284028f Change user_id column non-nullable (#6435) 2018-02-07 16:35:44 +01:00
David Yip
4f8122a98c Merge remote-tracking branch 'origin/master' into merge-upstream
Conflicts:
	.env.production.sample
	app/controllers/auth/confirmations_controller.rb
	db/schema.rb
2018-02-04 16:36:19 -06:00
Eugen Rochko
555e7205da Make PAM gem optional, allow configuration over environment (#6415) 2018-02-04 15:05:53 +01:00
Renato "Lond" Cerqueira
6528c0c101 Add option to show only local toots in timeline preview (#6292)
* Add option to show only local toots in timeline preview
Right know, toots from all the known fediverse are shown in the main
page of an instance. That however doesn't reflect the instance itself.
With this option the admin may choose to display only local toots so
that users checking the instance get a better idea of internal
conversations.

* Fix issues pointed by codeclimate and eslint

* Add default message for community timeline

* Update pl.yml
2018-02-04 06:00:10 +01:00
Eugen Rochko
5322013f25 CAS + SAML authentication feature (#6425)
* Cas authentication feature

* Config

* Remove class_eval + Omniauth initializer

* Codeclimate review

* Codeclimate review 2

* Codeclimate review 3

* Remove uid/email reconciliation

* SAML authentication

* Clean up code

* Improve login form

* Fix code style issues

* Add locales
2018-02-04 05:42:13 +01:00
David Yip
6d1023b2e9 Merge remote-tracking branch 'tootsuite/master' into merge-upstream
Conflicts:
      app/javascript/styles/mastodon/components.scss
2018-02-02 08:39:52 -06:00
Alexander
23ce0c86da pam authentication (#5303)
* add pam support, without extra column

* bugfixes for pam login

* document options

* fix code style

* fix codestyle

* fix tests

* don't call remember_me without password

* fix codestyle

* improve checks for pam usage (should fix tests)

* fix remember_me part 1

* add remember_token column because :rememberable requires either a password or this column.

* migrate db for remember_token

* move pam_authentication to the right place, fix logic bug in edit.html.haml

* fix tests

* fix pam authentication, improve username lookup, add comment

* valid? is sometimes not honored, return nil instead trying to authenticate with pam

* update devise_pam_authenticatable2 and adjust code. Fixes sideeffects observed in tests

* update devise_pam_authenticatable gem, fixes for codeconventions, fix finding user

* codeconvention fixes

* code convention fixes

* fix idention

* update dependency, explicit conflict check

* fix disabled password updates if in pam mode

* fix check password if password is present, fix templates

* block registration if account is maintained by pam

* Revert "block registration if account is maintained by pam"

This reverts commit 8e7a083d650240b6fac414926744b4b90b435f20.

* fix identation error introduced by rebase

* block usernames maintained by pam

* document pam settings better

* fix code style
2018-02-02 10:18:55 +01:00
Jenkins
3cd4a0804b Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-01-24 19:17:11 +00:00
Eugen Rochko
f378280e49 Fix #6331 (#6341)
UserTrackingConcern is circumvented by SessionsController#create
because it calls warden, which calls the User#update_tracked_fields!
method directly. Move returning user logic to that method.
2018-01-23 20:52:30 +01:00
Akihiko Odaki
b5162e2aff Rename ResolveRemoteAccountService to ResolveAccountService (#6327)
The service used to be named ResolveRemoteAccountService resolves local
accounts as well.
2018-01-22 14:25:09 +01:00
David Yip
d45ebe26dc Merge remote-tracking branch 'origin/master' into merge-upstream
Conflicts:
	app/javascript/styles/mastodon/components.scss
	app/javascript/styles/mastodon/modal.scss
2018-01-19 15:22:10 -06:00
Yamagishi Kazutoshi
49b76b3025 Change belongs_to_required_by_default to true (#5888) 2018-01-19 20:56:47 +01:00
Eugen Rochko
b0f2bff481 Replace tutorial modal with welcome e-mail (#6273)
* Remove onboarding modal

* Welcome e-mail

* Send welcome e-mail after confirmation

* Remove obsolete translations
2018-01-18 19:17:25 +01:00
Jenkins
6e821c4273 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-01-15 06:17:15 +00:00
Eugen Rochko
67ee638c28 Move e-mail digest task to sidekiq, reduce workload, improve hint (#6252) 2018-01-15 04:34:28 +01:00
Jenkins
1f469edac1 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-01-08 10:17:15 +00:00
Eugen Rochko
c8f0be1d6d Revert #5772 (#6221) 2018-01-08 10:57:52 +01:00
Jenkins
796a39a283 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-01-04 23:17:11 +00:00
Patrick Figel
bc5487a1c2 Fix email confirmation link not updating email (#6187)
A change introduced in #6125 prevents
`Devise::Models::Confirmable#confirm` from being called for existing
users, which in turn leads to `email` not being set to
`unconfirmed_email`, breaking email updates. This also adds a test
that would've caught this issue.
2018-01-05 00:15:35 +01:00
Jenkins
a7b8cb9493 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-01-03 20:17:13 +00:00
Akihiko Odaki
7a0479c830 Allow to dereference Follow object for ActivityPub (#5772)
* Allow to dereference Follow object for ActivityPub

* Accept IRI as object representation for Accept activity
2018-01-03 18:08:57 +01:00
Jenkins
b42e6973a1 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2018-01-03 04:17:11 +00:00
Patrick Figel
3c20cfd734 Add confirmation step for email changes (#6071)
* Add confirmation step for email changes

This adds a confirmation step for email changes of existing users.
Like the initial account confirmation, a confirmation link is sent
to the new address.

Additionally, a notification is sent to the existing address when
the change is initiated. This message includes instruction to reset
the password immediately or to contact the instance admin if the
change was not initiated by the account owner.

Fixes #3871

* Add review fixes
2018-01-02 16:55:00 +01:00
David Yip
01f91695f5 Merge remote-tracking branch 'origin/master' into merge-upstream
Conflicts:
	app/controllers/auth/confirmations_controller.rb
2017-12-30 17:20:07 -06:00
Eugen Rochko
9427823d5c Add more instance stats APIs (#6125)
* Add GET /api/v1/instance/peers API to reveal known domains

* Add GET /api/v1/instance/activity API

* Make new APIs disableable, exclude private statuses from activity stats

* Fix code style issue

* Fix week timestamps
2017-12-29 19:52:04 +01:00
David Yip
d53f727653 Merge branch 'gs-master' into prevent-local-only-federation
Conflicts:
	db/schema.rb
2017-12-15 12:20:56 -06:00
David Yip
b8e6074c53 Examples for Status#set_locality and .as_tag_timeline.
This commit also:

- exposes the local-only emoji so that it can be used in examples
- allows local_only to be set explicitly, i.e. for timeline filtering
  specs
2017-12-14 02:27:42 -06:00
Jenkins
1366e96a02 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2017-12-13 18:17:16 +00:00
Eugen Rochko
d6088c5308 Add filters to admin UI for custom emojis (#6003) 2017-12-13 13:28:31 +01:00
Eugen Rochko
aea3d0814b Clean up admin UI for accounts (#6004)
* Add staff filter to admin UI for accounts, remove obsolete columns

* Only display OStatus section in admin UI for accounts when OStatus data
2017-12-13 12:15:10 +01:00
David Yip
a56c9ac5dc Merge remote-tracking branch 'tootsuite/master' into merge-upstream 2017-12-12 02:54:13 -06:00
Akihiko Odaki
a90fa13235 Change account_id non-nullable in lists (#5979) 2017-12-12 04:11:17 +01:00
Eugen Rochko
3b69e9b99c Exclude moved accounts from search results (#5984) 2017-12-12 02:14:33 +01:00
Erin
eb0660f6ef move outbox filtering to Status#permitted_for (as per @ekiru) 2017-12-11 15:28:04 -06:00
Erin
6f3321fe4f set local_only in a before_create callback instead of status service 2017-12-10 21:39:27 -06:00
Erin
7fc6f40a88 remove vestigial Status#local_only? definition 2017-12-10 21:25:36 -06:00
Erin
278d34a0ec Status#not_local_only scope should match nils too 2017-12-10 20:35:42 -06:00
Erin
b463321c6f filter local-only statuses from public pages 2017-12-10 17:23:01 -06:00
Erin
993449ee59 status: stub local_only?, add scope, add marked_local_only? 2017-12-10 17:04:28 -06:00
Erin
5eadff9883 add a local_only column to the statuses table 2017-12-10 16:41:25 -06:00
Andrea Scarpino
e8198cd298 Fix account and tag searches with leading/trailing spaces (#5965)
* Strip leading & trailing spaces from account query

* Strip leading & trailing spaces from tag search
2017-12-10 19:35:46 +01:00
cwm
41bbfeec7c add initial components based off of tootsuite pr #1507 2017-12-09 10:26:22 -06:00
Eugen Rochko
f16141ebac Limit users to 50 lists, remove pagination from lists API (#5933) 2017-12-09 01:32:29 +01:00
Eugen Rochko
46cce9ee4b Ensure link thumbnails are not stretched to super low quality (#5932) 2017-12-09 00:56:16 +01:00
Jenkins
099159741b Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2017-12-07 04:17:14 +00:00
Yamagishi Kazutoshi
a7f20ed185 Remove duplicate annotate (#5910)
* Remove duplicate annotate

* Remove blank line
2017-12-07 04:53:42 +01:00
Akihiko Odaki
ce3989fc6a Add embed_url to preview cards (#5775) 2017-12-07 03:37:43 +01:00
beatrix
f0b37f92a9 Merge pull request #229 from glitch-soc/glitch-theme
Advanced Next-Level Flavours And Skins For Mastodon™
2017-12-06 17:44:07 -05:00
Jenkins
a3a7b6a848 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2017-12-06 20:17:13 +00:00
Yamagishi Kazutoshi
fba46b6072 Using double splat operator (#5859) 2017-12-06 11:41:57 +01:00
Eugen Rochko
b037fbf9f4 Remove rabl dependency (#5894)
* Remove rabl dependency

* Replicate old Oj configuration
2017-12-06 15:04:49 +09:00
Eugen Rochko
ff4c28a305 Lists redis clean-up (#5886)
* When list is deleted, remove feed from redis

* Clean up list feeds of inactive users
2017-12-05 23:20:27 +01:00
Eugen Rochko
1e7f022fa2 Add list of lists component to web UI (#5811)
* Add list of lists component to web UI

* Add list adding

* Add list removing

* List editor modal

* Add API account search limited by following=true relation

* Rework list editor modal

* Remove mandatory pagination of GET /api/v1/lists/:id/accounts

* Adjust search input placeholder

* Fix rspec (#5890)

* i18n: (zh-CN) Add missing translations for #5811 (#5891)

* i18n: (zh-CN) yarn manage:translations -- zh-CN

* i18n: (zh-CN) Add missing translations for #5811

* Fix some issues

- Display loading/missing state for list timelines
- Order lists alphabetically in overview
- Fix async list editor reset
- Redirect to /lists after deleting unpinned list
- Redirect to / after pinning a list

* Remove dead list columns when a list is deleted or fetch returns 404
2017-12-05 23:02:27 +01:00
David Yip
1221e3075d Merge branch 'gs-master' into glitch-theme 2017-12-04 11:07:01 -06:00
kibigo!
882055afd0 Rename themes -> flavours ? ? 2017-12-03 23:26:40 -08:00
Jenkins
822dea26c3 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2017-12-02 14:17:11 +00:00
Yamagishi Kazutoshi
1447ef1d44 Add invite filter (#5862) 2017-12-01 16:40:02 +01:00
kibigo!
36b9f4df56 Skins support 2017-11-30 19:29:47 -08:00
Jenkins
a7a59f927d Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2017-11-30 03:17:12 +00:00
Eugen Rochko
2421fd745a Fix too many forwards (#5854)
* Avoid sending explicit Undo->Announce when original deleted

* Do not forward a reply back to the server that sent it

* Deduplicate inboxes of rebloggers' followers for delete forwarding

* Adjust test

* Fix wrong class, bad SQL, wrong variable, outdated comment
2017-11-30 03:50:05 +01:00
David Yip
a1d9c68198 Merge remote-tracking branch 'origin/master' into gs-master 2017-11-28 11:45:13 -06:00
Yamagishi Kazutoshi
d6846dce20 Show "expired" in invite index (#5838)
* Show "expired" in invite index

* "Force expire" -> "Deactivate"
2017-11-28 15:41:02 +01:00
aschmitz
ce341c7cf6 Allow hiding of reblogs from followed users (#5762)
* Allow hiding of reblogs from followed users

This adds a new entry to the account menu to allow users to hide
future reblogs from a user (and then if they've done that, to show
future reblogs instead).

This does not remove or add historical reblogs from/to the user's
timeline; it only affects new statuses.

The API for this operates by sending a "reblogs" key to the follow
endpoint. If this is sent when starting a new follow, it will be
respected from the beginning of the follow relationship (even if
the follow request must be approved by the followee). If this is
sent when a follow relationship already exists, it will simply
update the existing follow relationship. As with the notification
muting, this will now return an object ({reblogs: [true|false]}) or
false for each follow relationship when requesting relationship
information for an account. This should cause few issues due to an
object being truthy in many languages, but some modifications may
need to be made in pickier languages.

Database changes: adds a show_reblogs column (default true,
non-nullable) to the follows and follow_requests tables. Because
these are non-nullable, we use the existing MigrationHelpers to
perform this change without locking those tables, although the
tables are likely to be small anyway.

Tests included.

See also <https://github.com/glitch-soc/mastodon/pull/212>.

* Rubocop fixes

* Code review changes

* Test fixes

This patchset closes #648 and resolves #3271.

* Rubocop fix

* Revert reblogs defaulting in argument, fix tests

It turns out we needed this for the same reason we needed it in muting:
if nil gets passed in somehow (most usually by an API client not passing
any value), we need to detect and handle it.

We could specify a default in the parameter and then also catch nil, but
there's no great reason to duplicate the default value.
2017-11-28 15:00:35 +01:00
nullkal
35e3791ba4 Fix account migration feature (#5837)
* Make removable account migration

* Fix error during update of account migration setting

* Add notice when update account migration setting
2017-11-28 14:31:23 +01:00
David Yip
bc10a0dced Merge remote-tracking branch 'tootsuite/master' 2017-11-27 22:33:26 -06:00
Eugen Rochko
55d9abcceb Add UI for setting up account migration (#5832) 2017-11-27 22:47:06 +01:00
David Yip
c9dd67a733 Merge remote-tracking branch 'origin/master' into gs-master 2017-11-27 11:29:04 -06:00
Eugen Rochko
bc8c951083 Add consumable invites (#5814)
* Add consumable invites

* Add UI for generating invite codes

* Add tests

* Display max uses and expiration in invites table, delete invite

* Remove unused column and redundant validator

- Default follows not used, probably bad idea
- InviteCodeValidator is redundant because RegistrationsController
  checks invite code validity

* Add admin setting to disable invites

* Add admin UI for invites, configurable role for invite creation

- Admin UI that lists everyone's invites, always available
- Admin setting min_invite_role to control who can invite people
- Non-admin invite UI only visible if users are allowed to

* Do not remove invites from database, expire them instantly
2017-11-27 16:07:59 +01:00
Jenkins
7fcbe5faee Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2017-11-25 05:17:15 +00:00
William Pitcock
a4bb95d8b3 status: preserve visibility attribute when reblogging (infoleak fix) (#5789)
this should fix *all* remaining visibility-related mastodon ostatus infoleaks.
thanks to @csaurus@gnusocial.de for pointing out the infoleak.
2017-11-25 01:36:08 +01:00
abcang
3db3ca1258 Fix unnecessary order (#5807) 2017-11-24 18:41:07 +01:00
Jenkins
359ba16721 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2017-11-24 01:17:20 +00:00
Eugen Rochko
efc97d404c Add logging of admin actions (#5757)
* Add logging of admin actions

* Update brakeman whitelist

* Log creates, updates and destroys with history of changes

* i18n: Update Polish translation (#5782)

Signed-off-by: Marcin Mikołajczak <me@m4sk.in>

* Split admin navigation into moderation and administration

* Redesign audit log page

* 🇵🇱 (#5795)

* Add color coding to audit log

* Change dismiss->resolve, log all outcomes of report as resolve

* Update terminology (e-mail blacklist) (#5796)

* Update terminology (e-mail blacklist)

imho looks better

* Update en.yml

* Fix code style issues

* i18n-tasks normalize
2017-11-24 02:05:53 +01:00
Jenkins
9c9b105d43 Merge remote-tracking branch 'tootsuite/master' into glitchsoc/master 2017-11-19 15:17:12 +00:00
abcang
defb2fd0b0 Fix N+1 at notification (#5752) 2017-11-19 15:32:48 +01:00
David Yip
1dc24ece41 Merge remote-tracking branch 'tootsuite/master' 2017-11-19 01:30:29 -06:00
Yamagishi Kazutoshi
bffc1b880b Add “staff” badge to admin user pages (#5735)
* Allow staff badge to change from setting to hidden

* i18n: Update Polish translation
2017-11-19 04:13:57 +01:00
Eugen Rochko
b89f614bd7 Profile redirect notes (#5746)
* Serialize moved accounts into REST and ActivityPub APIs

* Parse federated moved accounts from ActivityPub

* Add note about moved accounts to public profiles

* Add moved account message to web UI

* Fix code style issues
2017-11-18 19:39:02 +01:00
beatrix
27ca589183 Merge pull request #208 from yipdw/keyword-mute
Run keyword mutes on hashtags
2017-11-18 12:06:40 -05:00
David Yip
08751360a0 Update annotations on Follow, FollowRequest, and Mute.
Follow and FollowRequest had conflicts in their schema annotations, so I
ran latest migrations and let annotate_models fix them up.
2017-11-17 17:41:15 -06:00
David Yip
f1279d8ff3 Merge remote-tracking branch 'origin/master' into merge-upstream 2017-11-17 17:40:00 -06:00
Eugen Rochko
e149067561 Lists (#5703)
* Add structure for lists

* Add list timeline streaming API

* Add list APIs, bind list-account relation to follow relation

* Add API for adding/removing accounts from lists

* Add pagination to lists API

* Add pagination to list accounts API

* Adjust scopes for new APIs

- Creating and modifying lists merely requires "write" scope
- Fetching information about lists merely requires "read" scope

* Add test for wrong user context on list timeline

* Clean up tests
2017-11-18 00:16:48 +01:00
David Yip
5f689cdf78 Merge remote-tracking branch 'origin/master' into gs-master 2017-11-16 22:52:37 -06:00
Eugen Rochko
3df46ce5a8 Fix some rubocop style issues (#5730) 2017-11-17 10:06:26 +09:00
Eugen Rochko
21d01d5809 Display moderator badge (green), admin badge changed to red (#5728) 2017-11-17 01:22:38 +01:00
ysksn
bf1d4a2cb9 Fix AccountFinderConcern::AccountFinder#with_usernames (#5722) 2017-11-16 12:13:03 +01:00
Surinna Curtis
02ae609d9a Merge tootsuite/master at ab48f2b239 2017-11-16 01:21:16 -06:00
David Yip
b4725cad17 Replace =~ with #matches?. #208.
=~ made sense when we were passing it through to a regex, but we're no
longer doing that: TagMatcher looks at individual tags and returns a
value that *looks* like what you get out of #=~ but really isn't that
meaningful.  Probably a good idea to not subvert convention like this
and instead use a name with guessable intent.
2017-11-15 18:27:25 -06:00
David Yip
21e138379d Prefix cache keys with the matcher type. #208.
We already know about one regex limitation, which is that they cannot
segment words in e.g. Japanese, Chinese, or Thai.  It may also end up
that regex matching is too slow compared to other methods.

However, the regex is an implementation detail.  We still want the
ability to switch between "occurs anywhere" and "match whole word", and
caching the matcher result is likely to still be important (since the
matcher itself won't change nearly as often as status ingress rate).
Therefore, we ought to be able to change the cache keys to reflect a
change of data structure.

(Old cache keys expire within minutes, so they shouldn't be too big of
an issue.  Old cache keys could also be explicitly removed by an
instance administrator.)
2017-11-15 18:12:31 -06:00
David Yip
2c6a2d588c Match keyword mute filter on hashtags. #208.
It is reasonable to expect someone to enter #foo to mute hashtag #foo.
However, tags are recorded on statuses without the preceding #.
To adjust for this, we build a separate tag matcher and use
Tag::HASHTAG_RE to extract a hashtag from the hashtag syntax.
2017-11-15 18:12:29 -06:00
beatrix
ed1cf698a2 Merge pull request #212 from aschmitz/feat/mute-reblogs
Allow hiding reblogs on a per-follow basis
2017-11-15 12:01:17 -05:00
ysksn
475b7cecb5 Delegate some methods of User to @settings (#5706)
* Move some tests of User into Settings::ScopedSettings

* Add a test for User@settings
2017-11-15 16:05:20 +01:00
Surinna Curtis
582dd1b7c7 Optional notification muting (#5087)
* Add a hide_notifications column to mutes

* Add muting_notifications? and a notifications argument to mute!

* block notifications in notify_service from hard muted accounts

* Add specs for how mute! interacts with muting_notifications?

* specs testing that hide_notifications in mutes actually hides notifications

* Add support for muting notifications in MuteService

* API support for muting notifications (and specs)

* Less gross passing of notifications flag

* Break out a separate mute modal with a hide-notifications checkbox.

* Convert profile header mute to use mute modal

* Satisfy eslint.

* specs for MuteService notifications params

* add trailing newlines to files for Pork :)

* Put the label for the hide notifications checkbox in a label element.

* Add a /api/v1/mutes/details route that just returns the array of mutes.

* Define a serializer for /api/v1/mutes/details

* Add more specs for the /api/v1/mutes/details endpoint

* Expose whether a mute hides notifications in the api/v1/relationships endpoint

* Show whether muted users' notifications are muted in account lists

* Allow modifying the hide_notifications of a mute with the /api/v1/accounts/:id/mute endpoint

* make the hide/unhide notifications buttons work

* satisfy eslint

* In probably dead code, replace a dispatch of muteAccount that was skipping the modal with launching the mute modal.

* fix a missing import

* add an explanatory comment to AccountInteractions

* Refactor handling of default params for muting to make code cleaner

* minor code style fixes oops

* Fixed a typo that was breaking the account mute API endpoint

* Apply white-space: nowrap to account relationships icons

* Fix code style issues

* Remove superfluous blank line

* Rename /api/v1/mutes/details -> /api/v2/mutes

* Don't serialize "account" in MuteSerializer

Doing so is somewhat unnecessary since it's always the current user's account.

* Fix wrong variable name in api/v2/mutes

* Use Toggle in place of checkbox in the mute modal.

* Make the Toggle in the mute modal look better

* Code style changes in specs and removed an extra space

* Code review suggestions from akihikodaki

Also fixed a syntax error in tests for AccountInteractions.

* Make AddHideNotificationsToMute Concurrent

It's not clear how much this will benefit instances in practice, as the
number of mutes tends to be pretty small, but this should prevent any
blocking migrations nonetheless.

* Fix up migration things

* Remove /api/v2/mutes
2017-11-15 03:56:41 +01:00
Yamagishi Kazutoshi
3a51544e9a Add uniqueness to block email domains (#5692) 2017-11-14 20:37:17 +01:00
David Yip
13ec4a9a20 Maintain case-insensitivity when merging multiple matchers (#213)
When given two regexps, Regexp.union preserves the options set (or not
set) on each regex; this meant that none of the multiline (m),
case-insensitivity (i), or extended syntax (x) options were set.  Our
regexps are written expecting the m, i, and x options were set on all of
them, so we need to make sure that we preserve that behavior.
2017-11-13 11:06:02 -06:00
Daniel Hunsaker
96504fd1ec Update model annotations to use BIGINT for IDs (#5461)
All the migrations have been updated to use BIGINTs for ID fields in the DB, but ActiveRecord needs to be told to treat those values as BIGINT as well. This PR does that.
2017-11-12 16:18:50 +01:00
aschmitz
76008c3e8c Updates per code review
Thanks, @valerauko!
2017-11-11 14:37:23 -06:00
Eugen Rochko
97178b6d86 Add moderator role and add pundit policies for admin actions (#5635)
* Add moderator role and add pundit policies for admin actions

* Add rake task for turning user into mod and revoking it again

* Fix handling of unauthorized exception

* Deliver new report e-mails to staff, not just admins

* Add promote/demote to admin UI, hide some actions conditionally

* Fix unused i18n
2017-11-11 20:23:33 +01:00
aschmitz
2f1fdd37d3 Per-user reblog hiding implementation/fixes/tests
Note that this will only hide/show *future* reblogs by a user, and does
nothing to remove/add reblogs that are already in the timeline. I don't
think that's a particularly confusing behavior, and it's a lot easier
to implement (similar to mutes, I believe).
2017-11-10 22:04:54 -06:00
aschmitz
2aedd8eaad "Show reblogs" per-follower UI/database changes
TODO:

* Tests (particularly for FollowRequests).
* Anything to respect the setting when putting reblogs in timelines.
2017-11-09 08:41:10 -06:00
ysksn
26f52a2551 Add and Remove tests for FollowRequest (#5622)
* Add a test for FollowRequest#authorize!

* Remove tests

There is no need to test
ActiveModel::Validations::ClassMethods#validates.

* Make an alias of destroy! as reject!

Instead of defining the method,
make an alias of destroy! as reject! because of reducing test.
2017-11-09 14:36:52 +01:00
ThibG
4912f539c3 Twidere mention workaround (#5552)
* Work around Twidere and Tootdon bug

Tootdon and Twidere construct @user@domain handles from mentions in toots based
solely on the mention text and account URI's domain without performing any
webfinger call or retrieving account info from the Mastodon server.

As a result, when a remote user has WEB_DOMAIN ≠ LOCAL_DOMAIN, Twidere and
Tootdon will construct the mention as @user@WEB_DOMAIN. Now, this will usually
resolve to the correct account (since the recommended configuration is to have
WEB_DOMAIN perform webfinger redirections to LOCAL_DOMAIN) when processing
mentions, but won't do so when displaying them (as it does not go through the
whole account resolution at that time).

This change rewrites mentions to the resolved account, so that displaying the
mentions will work.

* Use lookbehind instead of non-capturing group in MENTION_RE

Indeed, substitutions with the previous regexp would erroneously eat any
preceding whitespace, which would lead to concatenated mentions in the
previous commit.

Note that users will “lose” up to one character space per mention for their
toots, as that regexp is also used to remove the domain-part of mentioned
users for character counting purposes, and it also erroneously removed the
preceding character if it was a space.
2017-11-07 19:08:14 +01:00
Eugen Rochko
7f1a0241c1 Add ability to disable login and mark accounts as memorial (#5615)
Fix #5597
2017-11-07 19:06:44 +01:00
nullkal
19f82492ab Show the local couterpart of emoji when it exists in /admin/custom_emojis (#5467)
* Show the local couterpart of emoji when it exists in admin/custom_emojis

* Fix indentation

* Fix error

* Add class table-action-link to Overwrite link

* Make it enable to overwrite emojis

* Make Code Climate happy
2017-11-07 14:49:32 +01:00
David Yip
2aec3c073b Merge branch 'master' into gs-master 2017-10-27 09:45:25 -05:00
nullkal
0a9cc96cdc Feature: Unlisted custom emojis (#5485) 2017-10-27 16:11:30 +02:00
beatrix
ce738197b9 Merge pull request #179 from glitch-soc/keyword-mute
Keyword muting
2017-10-25 17:37:48 -04:00
David Yip
eef50289c6 Merge remote-tracking branch 'STJrInuyasha/feature/direct-timeline' into gs-direct-timeline 2017-10-25 16:01:20 -05:00
David Yip
b8c462c42b Remove nil check in Glitch::KeywordMute#=~.
@regex can no longer be nil, so we don't need to check it.
2017-10-24 19:03:59 -05:00
David Yip
05fecda52d Switch to Regexp.union for building the mute expression.
Also make the keyword-building methods private: they always probably
should have been private, but now I have encoded enough fun and games
into them that it now seems wrong for them to *not* be private.
2017-10-24 18:31:34 -05:00
David Yip
763fd4b275 Only cache the regex text, not the regex itself.
It is possible to cache a Regexp object, but I'm not sure what happens
if e.g. that object remains in cache across two different Ruby versions.
Caching a string seems to raise fewer questions.
2017-10-23 19:31:59 -05:00
Matthew Walsh
737ab88c86 Added a timeline for Direct statuses
* Lists all Direct statuses you've sent and received
* Displayed in Getting Started
* Streaming server support for direct TL
2017-10-22 18:35:14 -07:00
David Yip
e8314ee2a6 KeywordMute matcher: more closely mimic Regexp#=~ behavior.
Regexp#=~ returns nil if it does not match.  An empty mute set does not
match any status, so KeywordMute::Matcher#=~ ought to return nil also.
2017-10-22 01:12:21 -05:00
David Yip
3b2bf30644 Don't add \b to whole-word keywords that don't start with word characters.
Ditto for ending with \b.

Consider muting the phrase "(hot take)".  I stipulate it is reasonable
to enter this with the default "match whole word" behavior.  Under the
old behavior, this would be encoded as

    \b\(hot\ take\)\b

However, if \b is before the first character in the string and the first
character in the string is not a word character, then the match will
fail.  Ditto for after.  In our example, "(" is not a word character, so
this will not match statuses containing "(hot take)", and that's a very
surprising behavior.

To address this, we only add leading and trailing \b to keywords that
start or end with word characters.
2017-10-22 00:38:54 -05:00
David Yip
b2ad79547c Apply keyword mutes to reblogs. 2017-10-21 15:44:47 -05:00
David Yip
74141adca1 Move KeywordMute into Glitch namespace.
There are two motivations for this:

1. It looks like we're going to add other features that require
   server-side storage (e.g. user notes).

2. Namespacing glitchsoc modifications is a good idea anyway: even if we
   do not end up doing (1), if upstream introduces a keyword-mute feature
   that also uses a "KeywordMute" model, we can avoid some merge
   conflicts this way and work on the more interesting task of
   choosing which implementation to use.
2017-10-21 14:54:36 -05:00
David Yip
05ee0aeb8b Allow keywords to match either substrings or whole words.
Word-boundary matching only works as intended in English and languages
that use similar word-breaking characters; it doesn't work so well in
(say) Japanese, Chinese, or Thai.  It's unacceptable to have a feature
that doesn't work as intended for some languages.  (Moreso especially
considering that it's likely that the largest contingent on the Mastodon
bit of the fediverse speaks Japanese.)

There are rules specified in Unicode TR29[1] for word-breaking across
all languages supported by Unicode, but the rules deliberately do not
cover all cases.  In fact, TR29 states

    For example, reliable detection of word boundaries in languages such
    as Thai, Lao, Chinese, or Japanese requires the use of dictionary
    lookup, analogous to English hyphenation.

So we aren't going to be able to make word detection work with regexes
within Mastodon (or glitchsoc).  However, for a first pass (even if it's
kind of punting) we can allow the user to choose whether they want word
or substring detection and warn about the limitations of this
implementation in, say, docs.

[1]: https://unicode.org/reports/tr29/
     https://web.archive.org/web/20171001005125/https://unicode.org/reports/tr29/
2017-10-21 14:54:36 -05:00
David Yip
be472f276d Invalidate cached matcher objects on KeywordMute commit. #164. 2017-10-21 14:54:36 -05:00
David Yip
aa57929688 Use more idiomatic string concatentation. #164.
The intent of the previous concatenation was to minimize object
allocations, which can end up being a slow killer.  However, it turns
out that under MRI 2.4.x, the shove-strings-in-an-array-and-join method
is not only arguably more common but (in this particular case) actually
allocates *fewer* objects than the string concatenation.

Or, at least, that's what I gather by running this:

    words = %w(palmettoes nudged hibernation bullish stockade's tightened Hades
    Dixie's formalize superego's commissaries Zappa's viceroy's apothecaries
    tablespoonful's barons Chennai tollgate ticked expands)

    a = Account.first

    KeywordMute.transaction do
      words.each { |w| KeywordMute.create!(keyword: w, account: a) }

      GC.start

      s1 = GC.stat

      re = String.new.tap do |str|
        scoped = KeywordMute.where(account: a)
        keywords = scoped.select(:id, :keyword)
        count = scoped.count

        keywords.find_each.with_index do |kw, index|
          str << Regexp.escape(kw.keyword.strip)
          str << '|' if index < count - 1
        end
      end

      s2 = GC.stat

      puts s1.inspect, s2.inspect

      raise ActiveRecord::Rollback
    end

vs this:

    words = %w( palmettoes nudged hibernation bullish stockade's tightened Hades Dixie's
    formalize superego's commissaries Zappa's viceroy's apothecaries tablespoonful's
    barons Chennai tollgate ticked expands
    )

    a = Account.first

    KeywordMute.transaction do
      words.each { |w| KeywordMute.create!(keyword: w, account: a) }

      GC.start

      s1 = GC.stat

      re = [].tap do |arr|
        KeywordMute.where(account: a).select(:keyword, :id).find_each do |m|
          arr << Regexp.escape(m.keyword.strip)
        end
      end.join('|')

      s2 = GC.stat

      puts s1.inspect, s2.inspect

      raise ActiveRecord::Rollback
    end

Using rails r, here is a comparison of the total_allocated_objects and
malloc_increase_bytes GC stat data:

                 total_allocated_objects        malloc_increase_bytes
string concat    3200241 -> 3201428 (+1187)     1176 -> 45216 (44040)
array join       3200380 -> 3201299 (+919)      1176 -> 36448 (35272)
2017-10-21 14:54:36 -05:00
David Yip
f3a57932c0 Make use of the regex attr_reader. #164.
It would also have been valid to get rid of the attr_reader, but I like
being able to reach inside KeywordMute::Matcher without resorting to
instance_variable_get tomfoolery.
2017-10-21 14:54:36 -05:00
David Yip
a3ee8592a8 Rework KeywordMute interface to use a matcher object; spec out matcher. #164.
A matcher object that builds a match from KeywordMute data and runs it
over text is, in my view, one of the easier ways to write examples for
this sort of thing.
2017-10-21 14:54:36 -05:00
David Yip
ddcb129101 Spec out KeywordMute interface. #164. 2017-10-21 14:54:21 -05:00
David Yip
c123b710ad Add KeywordMute model.
Gist of the proposed keyword mute implementation:

Keyword mutes are represented server-side as one keyword per record.
For each account, there exists a keyword regex that is generated as one
big alternation of all keywords.  This regex is cached (in Redis, I
guess) so we can quickly get it when filtering in FeedManager.
2017-10-21 14:53:41 -05:00
David Yip
e19bc7835a Merge remote-tracking branch 'upstream/master' into gs-master 2017-10-16 09:23:59 -05:00
Nolan Lawson
a54c272a1f Add option to reduce motion (#5393)
* Add option to reduce motion

* Use HOC to wrap all Motion calls

* fix case-sensitive issue

* Avoid updating too frequently

* Get rid of unnecessary change to _simple_status.html.haml
2017-10-16 09:36:15 +02:00
David Yip
ec18363c3b Merge remote-tracking branch 'origin/master' into gs-master 2017-10-16 01:29:02 -05:00
unarist
490f3146f2 Optimize Status#permitted_for 500x (account timeline) (#5373)
The main change of this PR is removing `order by visibility` hack.

This was introduced to force using of `index_statuses_on_account_id` instead of PK index, but it seems no longer needed probably due to `index_statuses_on_account_id_id`. Removing this avoids reading all rows, so really improves first fetching of the user who has lot of statuses.

I have also changed JOIN to IN + subquery, which slightly faster in most cases.
2017-10-13 16:53:43 +02:00
Eugen Rochko
7ccc1c5748 Fix UserTrackingConcern firing on every request, optimize some queries (#5368)
- For some reason, :if option on before_action did not work. It got
  executed every time, returned false, and the action run anyway,
  which led to the current_sign_in_at and sign_in_count being
  updated on every request
- Return "do not filter" early in FeedManager#filter_from_home? if
  the status is authored by receiver. Usually this method is not
  called for own statuses at all, but it is called when Feed#get
  uses the database
- Return early if #reload_stale_associations! has nothing to load
  to save a database query with WHERE 1=0
2017-10-13 16:44:29 +02:00
Lex Alexander
526b66e871 Retoot count increases without reason (#5363)
* Retoot count increases without reason

-The store_uri method for Statuses was being called on after_create and causing reblogs to be incremented twice.
-This calls it when the transaction is finished by using after_create_commit.
-Fixes #4916.

* Added test case for after_create_commit callback for checking reblog count.

* Rewrote test to keep original, but added one for only the after_create_commit callback.
2017-10-13 02:52:09 +02:00
kibigo!
02827345ae Merge upstream 2.0ish #165 2017-10-11 10:43:10 -07:00
nullkal
4a45cc7566 foreign_key, non-nullable, dependent: destroy in account_moderation_notes (#5294)
* Add foreign key constraint to column `account` in `account_moderation_notes`

* Change account_id and target_account_id to non-nullable in account_moderation_notes

* Add dependent: :destroy to account and target_account in account_moderation_notes
2017-10-10 13:12:17 +02:00
beatrix
6f578c25c6 try to tighten up local only toot stuff, like... properly (#163)
* try to tighten up local only toot stuff, like... properly

* try to un-break tests
2017-10-09 09:56:17 -04:00
Eugen Rochko
007fce8c10 Set snowflake IDs for backdated statuses (#5260)
- Rename Mastodon::TimestampIds into Mastodon::Snowflake for clarity
- Skip for statuses coming from inbox, aka delivered in real-time
- Skip for statuses that claim to be from the future
2017-10-08 17:34:34 +02:00
DJ Sundog
e50cd23407 update indentation 2017-10-07 13:53:46 -07:00
DJ Sundog
c7cae1bf0f add faststart to audio transcoding 2017-10-07 19:59:22 +00:00
DJ Sundog
c3e5510289 adding support for audio uploads, transcoded to mp4 videos 2017-10-07 19:54:10 +00:00
nullkal
81489970fb Add moderation note (#5240)
* Add moderation note

* Add frozen_string_literal

* Make rspec pass
2017-10-07 20:26:43 +02:00
Eugen Rochko
60925ce0ae Encode custom emojis as resolveable objects in ActivityPub (#5243)
* Encode custom emojis as resolveable objects in ActivityPub

* Improve code style
2017-10-07 17:43:42 +02:00
Eugen Rochko
b4af50c521 Improve admin UI for custom emojis, add copy/disable/enable (#5231) 2017-10-05 23:42:05 +02:00