Commit graph

1163 commits

Author SHA1 Message Date
Eugen Rochko
6baf5099a6 Refactor fetching of remote resources (#11251) 2019-07-10 18:59:28 +02:00
Eugen Rochko
d0b0b63b1a Refactor domain block checks (#11268) 2019-07-09 03:27:35 +02:00
Eugen Rochko
e17c937f65 Remove unused remote unfollow controller (#11250) 2019-07-08 12:04:06 +02:00
Eugen Rochko
56f0203c66 Refactor controllers for statuses, accounts, and more (#11249) 2019-07-08 12:03:45 +02:00
Eugen Rochko
fbbcbd940d Remove Atom feeds and old URLs in the form of GET /:username/updates/:id (#11247) 2019-07-07 16:16:51 +02:00
Eugen Rochko
4931208dd8 Remove Salmon and PubSubHubbub (#11205)
* Remove Salmon and PubSubHubbub endpoints

* Add error when trying to follow OStatus accounts

* Fix new accounts not being created in ResolveAccountService
2019-07-06 23:26:16 +02:00
Eugen Rochko
d9f42ba34b Remove deprecated REST API GET /api/v1/statuses/:id/card (#11213) 2019-07-05 02:15:24 +02:00
Eugen Rochko
6cae1b40bf Remove deprecated REST API GET /api/v1/timelines/direct (#11212) 2019-07-05 02:14:56 +02:00
Eugen Rochko
e782fcef4f Add request pool to improve delivery performance (#10353)
* Add request pool to improve delivery performance

Fix #7909

* Ensure connection is closed when exception interrupts execution

* Remove Timeout#timeout from socket connection

* Fix infinite retrial loop on HTTP::ConnectionError

* Close sockets on failure, reduce idle time to 90 seconds

* Add MAX_REQUEST_POOL_SIZE option to limit concurrent connections to the same server

* Use a shared pool size, 512 by default, to stay below open file limit

* Add some tests

* Add more tests

* Reduce MAX_IDLE_TIME from 90 to 30 seconds, reap every 30 seconds

* Use a shared pool that returns preferred connection but re-purposes other ones when needed

* Fix wrong connection being returned on subsequent calls within the same thread

* Reduce mutex calls on flushes from 2 to 1 and add test for reaping
2019-07-02 00:34:38 +02:00
Eugen Rochko
1c612f24e4 Add categories for custom emojis (#11196)
Fix #7940
2019-06-28 15:54:10 +02:00
ThibG
8adac384ed Fix account URI in UpdatePollSerializer (#11194)
* Fix account URI in UpdatePollSerializer

Fixes #11185

* Add specs
2019-06-27 19:41:55 +02:00
ThibG
28806d5d6d Add support for Audio activities (#11189)
Fixes #11127
2019-06-26 19:32:36 +02:00
ThibG
42c803fe61 Apply filters to poll options (#11174)
* Apply filters to poll options in WebUI

Fixes #11128

* Apply filters to poll options server-side

* Add poll options to searchable text
2019-06-25 14:45:14 +02:00
Eugen Rochko
eac19f2300 Change domain blocks to automatically support subdomains (#11138)
* Change domain blocks to automatically support subdomains

If a more authoritative domain is blocked (example.com), then the
same block will be applied to a subdomain (foo.example.com)

* Match subdomains of existing accounts when blocking/unblocking domains

* Improve code style
2019-06-22 00:13:10 +02:00
Eugen Rochko
433036ab8c Add moderation API (#9387)
Fix #8580
Fix #7143
2019-06-20 02:52:34 +02:00
Eugen Rochko
ffbca47356 Fix sanitizer making block level elements unreadable (#10836)
Fix #10834
2019-06-16 21:46:36 +02:00
Eugen Rochko
b788e58ea4 Change /settings/preferences to redirect to appearance, add /settings/preferences/other (#10988) 2019-06-07 16:51:08 +02:00
Eugen Rochko
8bf8d6d9d9 Change preferences page into appearance, notifications, and other (#10977) 2019-06-07 03:39:24 +02:00
ThibG
be6e54a70c Do not misattribute inlined boosts if attributedTo isn't present (#10967)
* Do not misattribute inlined boosts if `attributedTo` isn't present

Fixes #10950

* Fix tests
2019-06-04 23:24:31 +02:00
Eugen Rochko
f3a02e70a8 Fix poll API not requiring authentication on non-public polls (#10960)
* Fix poll API not requiring authentication on non-public polls

That API does not reveal the content of the status, i.e. the question
itself, nor who the author is, nor which status it belongs to, but it
does reveal the poll options and how many answers they got

Fix #10959

* Add test
2019-06-04 20:10:26 +02:00
ThibG
04469eeecd Fix NotifyService test with regards to reblogs (#10928)
Fixes #10890
2019-06-02 18:08:26 +02:00
trwnh
140a741979 Update remote bio test from 160 to 500 (#10799) 2019-05-21 13:29:06 +02:00
Paul Woolcock
626844dfb1 Add account_id param to GET /api/v1/notifications (#10796)
* Add `from_account` to notifications API

this adds the ability to filter notifications by the account they
originated from

* passing a non-existent user should cause none to be returned

* Fix codeclimate warnings

* fix more codeclimate warnings

* make requested changes:

* use account id instead of user@domain
* name the param `account_id` instead of `from_account`

* Don't use `return` in a lambda
2019-05-21 13:28:49 +02:00
trwnh
d8e5268939 Change bio limit from 160 to 500 (#10790)
* Change note_length validator from 160 to 500

* Change input maxlength from 160 to 500

* update bio test from 160 to 500

* Multiply a string 30 times instead of 10
2019-05-19 22:51:44 +02:00
ThibG
4ad004d2ef Fix “invited by” not showing up for invited accounts in admin interface (#10791) 2019-05-19 21:40:36 +02:00
ThibG
1fec2ff78f Prevent from publicly boosting one's own private toots (#10775) 2019-05-18 00:28:51 +02:00
ThibG
f1a3135809 Record account suspend/silence time and keep track of domain blocks (#10660)
* Record account suspend/silence time and keep track of domain blocks

* Also unblock users who were suspended/silenced before dates were recorded

* Add tests

* Keep track of suspending date for users suspended through the CLI

* Show accurate number of accounts that would be affected by unsuspending an instance

* Change migration to set silenced_at and suspended_at

* Revert "Also unblock users who were suspended/silenced before dates were recorded"

This reverts commit a015c65d2d1e28c7b7cfab8b3f8cd5fb48b8b71c.

* Switch from using suspended and silenced to suspended_at and silenced_at

* Add post-deployment migration script to remove `suspended` and `silenced` columns

* Use Account#silence! and Account#suspend! instead of updating the underlying property

* Add silenced_at and suspended_at migration to post-migration

* Change account fabricator to translate suspended and silenced attributes

* Minor fixes

* Make unblocking domains always retroactive
2019-05-14 19:05:02 +02:00
ThibG
a571b07557 Prevent silenced local users from notifying remote users not following them (#10575)
* Prevent silenced local users from notifying remote users not following them

This is an attempt to extend the local restrictions of silenced users to the
federation.

* Add tests

* Add tests for making sure private status don't get sent over OStatus
2019-05-09 22:05:43 +02:00
Eugen Rochko
c3b1f6963b Change e-mail whitelist/blacklist to not be checked when invited (#10683)
* Change e-mail whitelist/blacklist to not be checked when invited

And only when creating an account, not when updating it later

Fix #10648

* Fix test
2019-05-03 23:44:44 +02:00
ThibG
8c716f917f Provide a link to existing domain block when trying to block an already-blocked domain (#10663)
* When trying to block an already-blocked domain, provide a link to the block

* Fix styling for links in flash messages

* Allow blocks to be upgraded but not downgraded
2019-05-03 20:36:36 +02:00
ThibG
af3971a0e6 Check that an invite link is valid before bypassing approval mode (#10657)
* Check that an invite link is valid before bypassing approval mode

Fixes #10656

* Add tests

* Only consider valid invite links in registration controller

* fixup
2019-05-02 04:30:12 +02:00
Eugen Rochko
6302bed0b5 Fix Keybase verification using wrong domain for remote accounts (#10547) 2019-04-10 20:28:43 +02:00
Alex Gessner
da3d516fcf compare usernames case-insensitively on new proof creation flow (#10544)
* compare usernames case-insensitively on new proof creation flow

* Fix code style issue
2019-04-10 18:05:11 +02:00
Eugen Rochko
6d5b05e3d5 Add invite request to pending account notification e-mail (#10528)
Fix sorting of the pending accounts page
2019-04-10 00:36:01 +02:00
Hinaloe
da343470c4 Allow set the voting period to just 5 minutes (#10525)
* Add spec of PollValidator for #10190

* Raise fraction less than 1 second

* format

* simplify time initialize
2019-04-09 17:02:12 +02:00
Eugen Rochko
7eb05b8724 Add "why do you want to join" field to invite requests (#10524)
* Add "why do you want to join" field to invite requests

Fix #10512

* Remove unused translations

* Fix broken registrations when no invite request text is submitted
2019-04-09 23:06:30 +09:00
ThibG
110dbb5286 Export and import show_reblogs together with following list (#10495)
* Refactor imports

* Export show_reblogs when exporting list of followed users

* Add support for importing show_reblogs with following collection

* Fix tests
2019-04-08 07:28:27 +02:00
Eugen Rochko
c84c30c542 Improve blocked view of profiles (#10491)
* Revert "Fix filtering of favourited_by, reblogged_by, followers and following (#10447)"

This reverts commit 0317f37c6f.

* Revert "Hide blocking accounts from blocked users (#10442)"

This reverts commit 4cd944d364.

* Improve blocked view of profiles

- Change "You are blocked" to "Profile unavailable"
- Hide following/followers in API when blocked
- Disable follow button and show "Profile unavailable" on public profile as well
2019-04-07 04:59:13 +02:00
ThibG
e87214826b Export and import hide_notifications alongside user mutes (#10335)
* Export hide_notifications along with user mutes

* Import hide_notifications along with muted users list

* Add headers for CSV exports
2019-04-03 18:17:43 +02:00
ThibG
4cd944d364 Hide blocking accounts from blocked users (#10442)
* Revert "Add indication that you have been blocked in web UI (#10420)"

This reverts commit d31affe69b.

* Revert "Add `blocked_by` relationship to the REST API (#10373)"

This reverts commit a8bb10633d.

* Hide blocking accounts from search results

* Filter blocking accouts from account followers

* Filter blocking accouts from account's following accounts

* Filter blocking accounts from “reblogged by” and “favourited by” lists

* Remove blocking account from URL search

* Return 410 on trying to fetch user data from a user who blocked us

* Return 410 in /api/v1/account/statuses for suspended or blocking accounts

* Fix status filtering when performing URL search

* Restore some React improvements

Restore some cleanup from d31affe69b

* Refactor by adding `without_blocking` scope
2019-04-01 20:06:13 +02:00
ThibG
ec3dab2b7f Fix more keybase-related test failures (#10443) 2019-04-01 19:10:44 +02:00
slice
67cd7abbb1 Improvements to image upload validation and creation (#10431)
* Check if image value is nil? before creating an image

Check if uploaded images aren't nil before creating SiteUpload models
for them.

* Validate presence of file in SiteUpload

* Fix file presence validation

* Fabricate SiteUpload#file

* Add link to Creative Commons license
2019-04-01 07:30:46 +02:00
ThibG
3ce98a995e Fix failing keybase-related test (#10428) 2019-03-30 18:15:23 +01:00
Eugen Rochko
a82bc7f5ae Add ActivityPub representation for identity proofs (#10414)
* Add ActivityPub representation for identity proofs

* Add tests
2019-03-30 02:12:06 +01:00
Alex Gessner
be7c92061c squashed identity proof updates (#10375) 2019-03-28 18:01:09 +01:00
Eugen Rochko
364e49ae1f Rename :poll to :preloadable_poll and :owned_poll to :poll on Status (#10401)
Also, fix some n+1 queries

Resolve #10365
2019-03-28 04:44:59 +01:00
Eugen Rochko
367ab6764f Remove unused ActivityPub @context values depending on response (#10378)
Fix #8078
2019-03-27 15:55:23 +01:00
Eugen Rochko
c21bdc81d3 Add validations to admin settings (#10348)
* Add validations to admin settings

- Validate correct HTML markup
- Validate presence of contact username & e-mail
- Validate that all usernames are valid
- Validate that enums have expected values

* Fix code style issue

* Fix tests
2019-03-23 14:07:04 +01:00
ThibG
11c3ceb56e Do not try fetching keys of unknown accounts on a Delete from them (#10326) 2019-03-20 17:20:16 +01:00
Eugen Rochko
a3c41f3532 Add Keybase integration (#10297)
* create account_identity_proofs table

* add endpoint for keybase to check local proofs

* add async task to update validity and liveness of proofs from keybase

* first pass keybase proof CRUD

* second pass keybase proof creation

* clean up proof list and add badges

* add avatar url to keybase api

* Always highlight the “Identity Proofs” navigation item when interacting with proofs.

* Update translations.

* Add profile URL.

* Reorder proofs.

* Add proofs to bio.

* Update settings/identity_proofs front-end.

* Use `link_to`.

* Only encode query params if they exist.

URLs without params had a trailing `?`.

* Only show live proofs.

* change valid to active in proof list and update liveness before displaying

* minor fixes

* add keybase config at well-known path

* extremely naive feature flagging off the identity proof UI

* fixes for rubocop

* make identity proofs page resilient to potential keybase issues

* normalize i18n

* tweaks for brakeman

* remove two unused translations

* cleanup and add more localizations

* make keybase_contacts an admin setting

* fix ExternalProofService my_domain

* use Addressable::URI in identity proofs

* use active model serializer for keybase proof config

* more cleanup of keybase proof config

* rename proof is_valid and is_live to proof_valid and proof_live

* cleanup

* assorted tweaks for more robust communication with keybase

* Clean up

* Small fixes

* Display verified identity identically to verified links

* Clean up unused CSS

* Add caching for Keybase avatar URLs

* Remove keybase_contacts setting
2019-03-18 21:00:55 +01:00
ThibG
9efb9d7f72 Set and store report URIs (#10303)
Fixes #10271
2019-03-17 15:34:56 +01:00
ThibG
3daa23c783 Fix reblogs privacy (#10302)
* Fix reblogs privacy

* Fix Announce processing specs
2019-03-17 14:54:09 +01:00
Eugen Rochko
4fa2d55d4e Add relationship manager UI (#10268) 2019-03-16 11:23:22 +01:00
ysksn
7e16669c44 Add specs for action log helper (#9605)
* Add specs for ActionLogHelper

* Make some methods private

methods below never referenced from outside of their module:

- #linkable_log_target
- #log_target_from_history
2019-03-16 00:57:23 +09:00
Eugen Rochko
793587060b Fix language detection of non-latin alphabets even at few characters (#10276) 2019-03-15 05:07:09 +01:00
Eugen Rochko
73fb7bfa0f Admission-based registrations mode (#10250)
Fix #6856
Fix #6951
2019-03-14 05:28:30 +01:00
Eugen Rochko
56822606f4 Redesign landing page (#10232) 2019-03-12 17:34:00 +01:00
Aurélien Reeves
ae33129975 Squish username before validation (#10239)
* Squish username before validation (#10101)

Fix #10101

* Move before_validation hook to a private method

Also add Unicode wite-spaces to the spec to support the use of squish
over strip.
2019-03-11 20:48:24 +01:00
ThibG
05aa8b9303 Reject existing Follows when suspending a remote account (#10230)
* Reject existing Follows when suspending a remote account

Partial fix to #10229

* Add tests
2019-03-10 16:18:58 +01:00
ThibG
dcbe6c16ba Do not allow adding votes to expired polls (#10214)
* Do not allow adding votes to expired polls

* Only validate expires_at on create
2019-03-08 00:54:50 +01:00
Eugen Rochko
53336aa646 Add test ensuring that unknown object types are rejected (#10166) 2019-03-05 11:46:36 +09:00
ThibG
7da087dabe Store remote votes URI (#10158)
* Store remote votes URI

* Add spec for accepting remote votes

* Make poll vote id generation work the same way as follows
2019-03-04 22:51:23 +01:00
Eugen Rochko
03b5da7b6f Add tests for ActivityPub poll processing (#10143) 2019-03-04 01:13:42 +01:00
Eugen Rochko
4a3acdc916 Add polls (#10111)
* Add polls

Fix #1629

* Add tests

* Fixes

* Change API for creating polls

* Use name instead of content for votes

* Remove poll validation for remote polls

* Add polls to public pages

* When updating the poll, update options just in case they were changed

* Fix public pages showing both poll and other media
2019-03-03 22:18:23 +01:00
ThibG
9f3f47e91f Improved remote thread fetching (#10106)
* Fetch up to 5 replies when discovering a new remote status

This is used for resolving threads downwards. The originating
server must add a “replies” attributes with such replies for it to
be useful.

* Add some tests for ActivityPub::FetchRepliesWorker

* Add specs for ActivityPub::FetchRepliesService

* Serialize up to 5 public self-replies for ActivityPub notes

* Add specs for ActivityPub::NoteSerializer

* Move exponential backoff logic to a worker concern

* Fetch first page of paginated collections when fetching thread replies

* Add specs for paginated collections in replies

* Move Note replies serialization to a first CollectionPage

The collection isn't actually paginable yet as it has no id nor
a `next` field. This may come in another PR.

* Use pluck(:uri) instead of map(&:uri) to improve performances

* Fix fetching replies when they are in a CollectionPage
2019-02-28 15:22:21 +01:00
Eugen Rochko
693f2353bc Add type, limit, offset, min_id, max_id, account_id to search API (#10091)
* Add type, limit, offset, min_id, max_id, account_id to search API

Fix #8939

* Make the offset work on accounts and hashtags search as well

* Assure brakeman we are not doing mass assignment here

* Do not allow paginating unless a type is chosen

* Fix search query and index id field on statuses instead of created_at
2019-02-26 15:21:36 +01:00
Eugen Rochko
6068b479f6 Fix Announce activities of unknown statuses not fetching those statuses (#10065)
Regression from #9998
2019-02-17 15:16:36 +01:00
Eugen Rochko
397f180493 Add logging for rejected ActivityPub payloads and add tests (#10062) 2019-02-17 03:38:25 +01:00
Eugen Rochko
0230b3c41d Filter incoming Announce activities by relation to local activity (#10041)
* Filter incoming Announce activities by relation to local activity

Reject if announcer is not followed by local accounts, and is not
from an enabled relay, and the object is not a local status

Follow-up to #10005

* Fix tests
2019-02-15 18:19:45 +01:00
ThibG
99dcb6d910 Alternative handling of private self-boosts (#9998)
* When self-boosting, embed original toot into Announce serialization

* Process unknown self-boosts from Announce object if it is more than an URI

* Add some self-boost specs

* Only serialize private toots in self-Announces
2019-02-13 18:36:23 +01:00
Franck Zoccolo
ffe97d5dda Add support for IPv6 only MXes in Email validation (#10009)
* Add support for IPv6 only MXes

* Fixed email validator tests
2019-02-12 14:48:04 +01:00
Eugen Rochko
5adc3f5676 Fix URL linkifier grabbing full-width spaces and quotations (#9997)
Fix #9993
Fix #5654
2019-02-09 20:13:11 +01:00
Hinaloe
cce3319bc2 Only URLs extract with pre-escaped text (#9991)
* [test] add japanese hashtag testcase

* Only URLs extract with pre-escaped text

( https://github.com/tootsuite/mastodon/issues/9989 )
2019-02-09 03:39:38 +01:00
Eugen Rochko
d2d4d38f22 Add featured hashtags to profiles (#9755)
* Add hashtag filter to profiles

GET /@:username/tagged/:hashtag
GET /api/v1/accounts/:id/statuses?tagged=:hashtag

* Display featured hashtags on public profile

* Use separate model for featured tags

* Update featured hashtag counters on-write

* Limit featured tags to 10
2019-02-04 04:25:59 +01:00
Eugen Rochko
1f303e5591 Add option to overwrite imported data (#9962)
* Add option to overwrite imported data

Fix #7465

* Add import for domain blocks
2019-02-03 03:59:51 +01:00
Jakub Mendyk
234e8105da Allow most kinds of characters in URL query (fixes #8408) (#8447)
* Allow unicode characters in URL query strings

Fixes #8408

* Alternative approach to unicode support in urls

Adds PoC/idea to approch this problem.
2019-02-02 19:01:18 +01:00
ThibG
b1c76ef610 Prevent posting toots with media attachments from someone else (#9921) 2019-01-26 23:59:39 +01:00
ThibG
a238beb870 Fix scheduled toot with media immediately creating a toot (#9894)
* Add test for not persisting status when attaching media to scheduled toot

* Prevent status used for validation from being persisted to the database

Fixes #9893

Thanks to tateisu for the help investigating this.
2019-01-21 20:03:04 +01:00
ThibG
8de5d28891 Reject existing Follow in addition to sending a Block (#9811)
Mastodon expects remote servers to remove follow relationships upon receiving
a Block. However, the spec only evokes Block activities in a C2S context, never
in a S2S context.

This PR, in addition to federating the Block, explicitly sends a Reject for any
affected follow relationship, which makes a bit more sense with regards to the
spec.
2019-01-18 15:57:19 +01:00
Eugen Rochko
380b246728 Redesign public hashtag page to use a masonry layout (#9822) 2019-01-16 19:47:46 +01:00
Moritz Heiber
f2cefd9d73 Disable Same-Site cookie implementation to fix SSO issues on WebKit browsers (#9819) 2019-01-15 23:11:46 +01:00
Renato "Lond" Cerqueira
ed55b4f8fb Fix undefined method error in sidekiq (#9807)
* Fix undefined method error in sidekiq

Body can be not nil but still be empty, which causes a
`NoMethodError: undefined method `[]' for nil:NilClass` further in the
code. This checks for an empty body to avoid the issue.

* Fix codeclimate issue
2019-01-14 17:28:41 +01:00
ysksn
081f27a3d1 Add a spec for Admin::ActionLog (#9775) 2019-01-11 07:28:09 +00:00
ysksn
743fdf3e8e Add specs for Admin::AccountAction (#9767) 2019-01-11 07:26:03 +00:00
ysksn
ff774bc6e3 Not to skip executable specs (#9753)
* Not to skip executable specs

* Combine specs

Combine specs to one to reduce multiple slow http post.
2019-01-10 15:12:31 +01:00
Eugen Rochko
f2ec1803cb Redesign admin instances area (#9645) 2019-01-08 13:39:49 +01:00
ysksn
c3725cc78f Remove pending (#9752)
Some specs have already been added.
2019-01-08 09:42:56 +01:00
ysksn
4d1e6abef3 Remove spec files (#9751)
Nothing to test.
2019-01-08 12:18:46 +09:00
ysksn
a59aac6351 Add pending specs for jsonld helper (#9750)
* Add specs for JsonLdHelper#first_of_value

* Add specs for JsonLdHelper#supported_context?
2019-01-08 12:18:27 +09:00
Eugen Rochko
ae1aaa3b8a Add scheduled statuses (#9706)
Fix #340
2019-01-05 12:43:28 +01:00
ysksn
cd51dfdd76 Add specs for UrlValidator (#9699) 2019-01-03 13:10:20 +09:00
ysksn
3cab7b9a17 Add specs for UnreservedUsernameValidator (#9698)
* Add specs for UnreservedUsernameValidator

* Use instance variable
2019-01-03 13:10:02 +09:00
Eugen Rochko
6936271536 Improve e-mail digest (#9689)
- Reduce time-to-digest from 20 to 7 days
- Fetch mentions starting from +1 day since last login
- Fix case when last login is more recent than last e-mail
- Do not render all mentions, only 40, but show number in subject
- Do not send digest to moved accounts
- Do send digest to silenced accounts
2019-01-02 10:47:32 +01:00
ThibG
ffcaa4719a Ensure blocked user unfollows blocker if Block/Undo Block are processed out of order (#9687)
* Ensure blocked user unfollows blocker if Block/Undo Block are processed out of order

* Add specs for Block causing unfollow and for out-of-order Block + Undo
2019-01-02 01:12:02 +01:00
ThibG
bdc44c3558 Reduce usage of LD signatures (#9659)
* Do not LDS-sign Follow, Accept, Reject, Undo, Block

* Do not use LDS for Create activities of private toots

* Minor cleanup

* Ignore unsigned activities instead of misattributing them

* Use status.distributable? instead of querying visibility directly
2018-12-30 09:48:59 +01:00
ysksn
41afd670c9 Add specs for FollowLimitValidator (#9655) 2018-12-29 08:24:52 +01:00
ysksn
5eb6d7e3e0 Add specs for BlackListedEmailValidator (#9651)
* Add specs for BlackListedEmailValidator

* Use instance variable
2018-12-29 07:23:44 +01:00
ysksn
a3b4684435 Add specs for DisallowedHashtagsValidator (#9653)
In order to implement tests easier, `#select_tags` created.
2018-12-29 07:22:51 +01:00
Eugen Rochko
4e3131f02f Add handler for Move activity (#9629) 2018-12-29 02:24:36 +01:00
ysksn
44c1647ea1 Add specs for StatusPinValidator (#9648) 2018-12-28 18:09:32 +09:00
ysksn
c4559969b4 Add pending specs for StatusLengthValidator (#9647)
* Add pending specs of StatusLengthValidator

* Use instance variable
2018-12-28 08:18:47 +01:00
Eugen Rochko
bbf9f4f93b Add REST API for creating an account (#9572)
* Add REST API for creating an account

The method is available to apps with a token obtained via the client
credentials grant. It creates a user and account records, as well as
an access token for the app that initiated the request. The user is
unconfirmed, and an e-mail is sent as usual.

The method returns the access token, which the app should save for
later. The REST API is not available to users with unconfirmed
accounts, so the app must be smart to wait for the user to click a
link in their e-mail inbox.

The method is rate-limited by IP to 5 requests per 30 minutes.

* Redirect users back to app from confirmation if they were created with an app

* Add tests

* Return 403 on the method if registrations are not open

* Require agreement param to be true in the API when creating an account
2018-12-24 19:12:38 +01:00
ThibG
a2492a89fb Limit maximum visibility of local silenced users to unlisted (#9583)
Fixes #9580
2018-12-24 19:06:14 +01:00
Eugen Rochko
2e0d617b8b Add moderation warnings (#9519)
* Add moderation warnings

Replace individual routes for disabling, silencing, and suspending
a user, as well as the report update route, with a unified account
action controller that allows you to select an action (none,
disable, silence, suspend) as well as whether it should generate an
e-mail notification with optional custom text. That notification,
with the optional custom text, is saved as a warning.

Additionally, there are warning presets you can configure to save
time when performing the above.

* Use Account#local_username_and_domain
2018-12-22 20:02:09 +01:00
ysksn
3beb22f65f Add specs for CustomEmojiFilter (#9599) 2018-12-21 18:52:57 +01:00
ysksn
0ef0de1099 Add specs for AdminMailer (#9597) 2018-12-21 09:34:34 +01:00
ysksn
25fffd5fcb Add specs for InstancePresenter (#9596) 2018-12-21 08:59:56 +01:00
ysksn
76cad0240a Add specs for ReportNotePolicy (#9592) 2018-12-20 17:52:18 +01:00
ysksn
6aeb34dc4c Add specs for policies (#9591)
* Add spec for RelayPolicy

* Add specs for SubscriptionPolicy

* Add specs for SettingsPolicy

* Add specs for TagPolicy

* Add specs for ReportPolicy
2018-12-20 17:52:07 +01:00
ysksn
8cf484e5a7 Add specs for UserPolicy (#9593) 2018-12-20 17:51:55 +01:00
ysksn
e756191722 Add specs for InvitePolicy (#9589) 2018-12-20 04:23:09 +01:00
ysksn
d4b4e87153 Add specs for EmailDomainBlockPolicy (#9586) 2018-12-20 03:51:41 +01:00
ysksn
fcb045060e Add specs for InstancePolicy (#9587) 2018-12-20 03:51:31 +01:00
ysksn
ce51a08674 Add specs for DomainBlockPolicy (#9585) 2018-12-20 03:47:51 +01:00
ysksn
31f7af3cad Add specs for CustomEmojiPolicy (#9584) 2018-12-20 03:24:28 +01:00
ysksn
dcab109a1c Add specs for BackupPolicy (#9576) 2018-12-19 18:24:15 +01:00
ysksn
d338a2099e Add specs for AccountPolicy (#9575) 2018-12-19 08:56:59 +01:00
ysksn
48a70180e2 Add specs for AccountModerationNotePolicy (#9571) 2018-12-19 07:24:03 +01:00
ysksn
bae098d118 Add specs for StatusPolicy (#9569) 2018-12-19 05:19:20 +01:00
ysksn
66b994aa7a Add spec for AccountableConcern#log_action (#9559) 2018-12-18 16:43:03 +01:00
ThibG
f76665a276 Ignore low-confidence CharlockHolmes guesses when parsing link cards (#9510)
* Add failing test for windows-1251 link cards

* Ignore low-confidence CharlockHolmes guesses

Fixes #9466

* Fix no method error when charlock holmes cannot detect charset
2018-12-17 19:19:45 +01:00
ysksn
977e743d7a Add spec for Api::V1::Timelines::DirectController (#9547) 2018-12-17 11:36:20 +01:00
ysksn
9414dea099 Add specs for Api::V1::Instances::PeersController (#9546) 2018-12-17 11:35:55 +01:00
ysksn
07c0f6dcab Add spec for Api::V1::EndorsementsController (#9543) 2018-12-17 11:32:44 +01:00
ysksn
4b972f332e Add spec for Api::V1::Instances::ActivityController (#9545) 2018-12-17 11:32:24 +01:00
ysksn
18469c65e8 Add specs for Accounts::PinsController (#9542) 2018-12-17 06:03:51 +01:00
ysksn
f96bf30b4a Add spec for Admin::ActionLogsController#index (#9522) 2018-12-14 20:37:01 +01:00
ysksn
d5c95283ef Add specs for RemoteInteractionController (#9524) 2018-12-14 20:36:40 +01:00
ysksn
bd7183fab9 Add spec for Admin::DashboardController#index (#9523) 2018-12-14 20:36:18 +01:00
Sumit Khanna
b1b6e5911d Error message for avatar image that's too large. #9204 (#9518)
* Error message for avatar image that's too large. #9204

* Code climate/formatting

* Removed avatar error message

* Moved valid image dimentions check to update service

* removed unnescessary begin block

* code climate formatting

* code climate indent fix
2018-12-14 05:07:21 +01:00
ysksn
5309e80ecb Add spec for Settings::ExportsController#create (#9512) 2018-12-13 02:53:52 +01:00
Adam Copp
55ebf8a6e8 Make custom emoji domains case insensitive #9351 (#9474)
* Make custom emoji domains case sensitive #9351

* Fixup style in downcase_domain to comply with codeclimate.

* switch if! to unless

* Don't use transactions, operate in batches.

Also revert spurious schema change.
2018-12-11 05:30:57 +01:00
Eugen Rochko
8d3910afdd Improve e-mail MX validator and add tests (#9489) 2018-12-10 22:53:25 +01:00
ysksn
dfd6445334 Add specs for activitypub collections controller (#9484)
* Add specs for ActivityPub::CollectionsController#show

* Raise ActiveRecord::RecordNotFound

Raising ActiveRecord::NotFound raises NameError: uninitialized constant
ActiveRecord::NotFound.
2018-12-10 21:39:25 +01:00
ysksn
5507c086e5 Add spec for Admin::SuspentionsController#new (#9483) 2018-12-10 21:38:21 +01:00
ysksn
a6641d0541 Fix Admin::TagsController#unhide (#9481) 2018-12-10 21:37:38 +01:00
ysksn
582ff3684c Add specs for Admin::InvitesController (#9471) 2018-12-10 01:19:28 +09:00
ysksn
246dfe9d47 Remove pending spec (#9453) 2018-12-07 16:53:55 +01:00
ysksn
d61410c306 Add specs for ActivityPub::InboxesController (#9456) 2018-12-07 16:40:01 +01:00
ysksn
f607161253 Add specs for Admin::AccountModerationNotesHelper (#9455) 2018-12-07 16:39:20 +01:00
ysksn
994c20699c Remove pending spec (#9454)
Since dots are not allowed in username,
this spec is no longer needed.
2018-12-07 16:38:50 +01:00
ysksn
26345b7cf2 Add specs for AccountTagStat model (#9452) 2018-12-07 16:37:56 +01:00
Eugen Rochko
fd8ca11f19 Add profile directory (#9427)
Fix #5578
2018-12-06 17:36:11 +01:00
ysksn
6d73528424 Remove pending spec (#9442)
`#from_account` isn't defined.
2018-12-06 17:39:15 +09:00
ysksn
23607d43da Add spec for Identity.find_for_oauth (#9441) 2018-12-06 17:38:49 +09:00
ThibG
5e2cd7f672 Fix thread depth computation in statuses_controller (#9426)
* Add test that should currently fail

* Fix depth computation (will still fail if statuses have been filtered out)

* Fix handling of broken threads
2018-12-05 02:12:29 +01:00
ThibG
80ef517747 Allow hyphens in the middle of remote user names (#9345)
Fixes #9309

This only allows hyphens in the middle of a username, much like dots,
although I don't have a compelling reason to do so other than keeping
the changes minimal.
2018-11-27 12:28:01 +01:00
Eugen Rochko
305556274a Redesign admin accounts index (#9340)
* Improve overview of accounts in admin UI

- Display suspended status, role, last activity and IP prominently
- Default to showing local accounts
- Default to not showing suspended accounts

* Remove unused strings

* Fix tests

* Allow filtering accounts by IP mask
2018-11-26 15:53:27 +01:00
valerauko
2178c50fed Ensure replied-to is a status not a boost (#9129)
* Ensure replied-to is a status not a boost

* Consider case of not a reply

* Add test case for replying to boost

* Move reblog-reply resolution to model

* Remove unnecessary comment
2018-11-25 16:35:21 +01:00
Eugen Rochko
1d16575d71 Sort self-replies to the top of descendants (#9320)
Fix #6463
2018-11-24 20:48:50 +01:00
Eugen Rochko
f3fee3cd27 Fix connect timeout not being enforced (#9329)
* Fix connect timeout not being enforced

The loop was catching the timeout exception that should stop execution, so the next IP would no longer be within a timed block, which led to requests taking much longer than 10 seconds.

* Use timeout on each IP attempt, but limit to 2 attempts

* Fix code style issue

* Do not break Request#perform if no block given

* Update method stub in spec for Request

* Move timeout inside the begin/rescue block

* Use Resolv::DNS with timeout of 1 to get IP addresses

* Update Request spec to stub Resolv::DNS instead of Addrinfo

* Fix Resolve::DNS stubs in Request spec
2018-11-22 20:12:04 +01:00
valerauko
a6ebcda59b Ignore JSON-LD profile in mime type comparison (#9179)
Ignore JSON-LD profile in mime type comparison
2018-11-22 12:49:07 +01:00
Eugen Rochko
62c0e112ea Extract counters from accounts table to account_stats table (#9295) 2018-11-19 00:43:52 +01:00
Eugen Rochko
7a939f7cfc Remove intermediary arrays when creating hash maps from results (#9291) 2018-11-16 15:02:18 +01:00
Eugen Rochko
5f613d55e6 Optimize the process of following someone (#9220)
* Eliminate extra accounts select query from FollowService

* Optimistically update follow state in web UI and hide loading bar

Fix #6205

* Asynchronize NotifyService in FollowService

And fix failing test

* Skip Webfinger resolve routine when called from FollowService if possible

If an account is ActivityPub, then webfinger re-resolving is not necessary
when called from FollowService. Improve options of ResolveAccountService
2018-11-08 21:05:42 +01:00
James Kiesel
d2a9ea58da Allow joining several hashtags in a single column (#8904)
* Nascent tag menu on frontend

* Hook up frontend to search

* Tag intersection backend first pass

* Update yarnlock

* WIP

* Fix for tags not searching correctly

* Make radio buttons function

* Simplify radio buttons with modeOption

* Better naming

* Rearrange options

* Add all/any/none functionality on backend

* Small PR cleanup

* Move to service from scope

* Small cleanup, add proper service tests

* Don't use send with user input :D

* Set appropriate column header

* Handle auto updating timeline

* Fix up toggle function

* Use tag value correctly

* A bit more correct to use 'self' rather than 'all' in status scope

* Fix some style issues

* Fix more code style issues

* Style select dropdown more better

* Only use to_id'ed value to ensure no SQL injection

* Revamp frontend to allow for multiple selects

* Update backend / col header to account for more flexible tagging

* Update brakeman ignore

* Codeclimate suggestions

* Fix presenter tag_url

* Implement initial PR feedback

* Handle additional tag streaming

* CodeClimate tweak
2018-11-05 18:53:25 +01:00
Eugen Rochko
3981da6f90 Revert "Fix FetchAtomService content type handling (#9132)" (#9171)
This reverts commit c20d55f7dd.
2018-10-31 00:43:34 +01:00
valerauko
c20d55f7dd Fix FetchAtomService content type handling (#9132)
* Add profile to json+ld in Accept

It's required by the ActivityPub spec

* Use headers['Content-type'] instead of mime_type

mime_type strips the profile from the content type, but it's still available raw in the headers hash

* Add test for ld+json with profile
2018-10-30 15:07:57 +01:00
ThibG
1cbcb09d67 Do not hide boost notifications from followed people with hidden boosts (#9147)
* Do not hide boost notifications from followed people with hidden boosts

Not displaying boosts from a followed user in the Home timeline and not
having notifications when they reblog your own content are two very
separate concerns, tying them together seem counter-intuitive and unwanted.

* Update specs accordingly
2018-10-30 00:47:31 +01:00
Eugen Rochko
eaf49c5d94 Bump sanitize from 4.6.6 to 5.0.0 (#9140) 2018-10-29 14:05:25 +01:00
takayamaki
17229858c8 fix: Execute PAM authentication tests on CircleCI (#9029)
and use 'if' option of context block
2018-10-20 17:28:04 +02:00
Eugen Rochko
7e8733a518 Do not test PAM authentication by default (#9027)
* Do not test PAM authentication by default

* Disable PAM tests if PAM is not enabled
2018-10-20 07:32:26 +02:00
Eugen Rochko
654520ec8c Improve support for aspects/circles (#8950)
* Add silent column to mentions

* Save silent mentions in ActivityPub Create handler and optimize it

Move networking calls out of the database transaction

* Add "limited" visibility level masked as "private" in the API

Unlike DMs, limited statuses are pushed into home feeds. The access
control rules between direct and limited statuses is almost the same,
except for counter and conversation logic

* Ensure silent column is non-null, add spec

* Ensure filters don't check silent mentions for blocks/mutes

As those are "this person is also allowed to see" rather than "this
person is involved", therefore does not warrant filtering

* Clean up code

* Use Status#active_mentions to limit returned mentions

* Fix code style issues

* Use Status#active_mentions in Notification

And remove stream_entry eager-loading from Notification
2018-10-17 17:13:04 +02:00
Eugen Rochko
0a5b65533d Improve signature verification safeguards (#8959)
* Downcase signed_headers string before building the signed string

The HTTP Signatures draft does not mandate the “headers” field to be downcased,
but mandates the header field names to be downcased in the signed string, which
means that prior to this patch, Mastodon could fail to process signatures from
some compliant clients. It also means that it would not actually check the
Digest of non-compliant clients that wouldn't use a lowercased Digest field
name.

Thankfully, I don't know of any such client.

* Revert "Remove dead code (#8919)"

This reverts commit 65d1a2d10a.

* Restore time window checking, change it to 12 hours

By checking the Date header, we can prevent replaying old vulnerable
signatures. The focus is to prevent replaying old vulnerable requests
from software that has been fixed in the meantime, so a somewhat long
window should be fine and accounts for timezone misconfiguration.

* Escape users' URLs when formatting them

Fixes possible HTML injection

* Escape all string interpolations in Formatter class

Slightly improve performance by reducing class allocations
from repeated Formatter#encode calls

* Fix code style issues
2018-10-12 00:15:55 +02:00
ashleyhull-versent
00d31a292c rubocop issues - Cleaning up (#8912)
* cleanup pass

* undo mistakes

* fixed.

* revert
2018-10-08 04:50:11 +02:00
ashleyhull-versent
ea57aca5e1 Replace SVG asset with Custom mascot (#8766) 2018-10-08 00:20:45 +02:00
Eugen Rochko
c9b5168ebd Add conversations API (#8832)
* Add conversations API

* Add web UI for conversations

* Add test for conversations API

* Add tests for ConversationAccount

* Improve web UI

* Rename ConversationAccount to AccountConversation

* Remove conversations on block and mute

* Change last_status_id to be a denormalization of status_ids

* Add optimistic locking
2018-10-07 23:44:58 +02:00
Jeong Arm
708c12af1e Leave unknown language as nil if account is remote (#8861)
* Force use language detector if account is remote

* Set unknown remote toot's language as nil
2018-10-05 19:17:46 +02:00
aus-social
c883b1ffc9 lint pass 2 (#8878)
* Code quality pass

* Typofix

* Update applications_controller_spec.rb

* Update applications_controller_spec.rb
2018-10-04 17:38:04 +02:00
Eugen Rochko
48c2ab8cf5 Limit the number of people that can be followed from one account (#8807)
Configurable soft limit of 7,500, and above that, configurable
ratio of 1.1 * followers, controlled by:

- MAX_FOLLOWS_THRESHOLD
- MAX_FOLLOWS_RATIO

Fix #2311
2018-10-04 17:36:11 +02:00
Eugen Rochko
1ae62b87ed Change admin accounts default sort to most recent (#8813) 2018-10-04 16:05:38 +02:00
Eugen Rochko
24f168e58e Fix link verification for remote accounts (#8868) 2018-10-04 15:47:03 +02:00
aus-social
a53bcb6213 Lint pass (#8876) 2018-10-04 12:36:53 +02:00
Eugen Rochko
5ddcdf3753 Support min_id-based pagination in REST API (#8736)
* Allow min_id pagination in Feed#get

* Add min_id pagination to home and list timeline APIs

* Add min_id pagination to account statuses, public and tag APIs

* Remove unused stub in reports API

* Use min_id pagination in notifications, favourites, and fix order

* Fix HomeFeed#from_database not using paginate_by_id
2018-09-28 02:23:45 +02:00
Eugen Rochko
ad0670fe87 Revert Font Awesome 5 upgrade (#8810)
* Revert "Fix some icon names changed by the Font Awesome 5. (#8796)"

This reverts commit c8a1e945d9.

* Revert "Migrate to font-awesome 5.0. (#8799)"

This reverts commit ae20afbc80.

* Revert "Fix some icons names, unavailable in fontawesome5 (free license). (#8792)"

This reverts commit 30b1bb0704.

* Revert "Update the icon name changed by the Font Awesome 5. (#8776)"

This reverts commit 84bcf89764.

* Revert "Add bot icon to bot avatars and migrate to newer version of Font Awesome (#8484)"

This reverts commit cc784f3c16.
2018-09-28 02:11:14 +02:00
Naoki Kosaka
ae20afbc80 Migrate to font-awesome 5.0. (#8799) 2018-09-27 17:08:56 +02:00
ThibG
5a049410f2 Refactor active_nav_class for use with multiple paths (#8757) 2018-09-23 20:42:13 +02:00
Eugen Rochko
52168073a3 Support link verification with redirects (#8735)
(e.g. URL shortener)
2018-09-20 00:10:35 +02:00
Yamagishi Kazutoshi
6d970d6e71 Fix failed profile verification when rel attribute including values other than me (#8733) 2018-09-19 16:47:31 +02:00
Eugen Rochko
d3105031f8 Redesign forms, verify link ownership with rel="me" (#8703)
* Verify link ownership with rel="me"

* Add explanation about verification to UI

* Perform link verifications

* Add click-to-copy widget for verification HTML

* Redesign edit profile page

* Redesign forms

* Improve responsive design of settings pages

* Restore landing page sign-up form

* Fix typo

* Support <link> tags, add spec

* Fix links not being verified on first discovery and passive updates
2018-09-18 16:45:58 +02:00
luzpaz
1bce70d3c7 Misc. typos (#8694)
Found via `codespell -q 3 --skip="./app/javascript/mastodon/locales,./config/locales"`
2018-09-14 00:53:09 +02:00
Eugen Rochko
72a8ca84e0 Add force_login option to OAuth authorize page (#8655)
* Add force_login option to OAuth authorize page

For when a user needs to sign into an app from multiple accounts
on the same server

* When logging out from modal header, redirect back after re-login
2018-09-09 04:10:44 +02:00
Sorin Davidoi
0b1c90bf88 feat(cookies): Use the same-site attribute to lax (#8626)
CSFR-prevention is already implemented but adding this doesn't hurt.

A brief introduction to Same-Site cookies (and the difference between strict and
lax) can be found at
https://blog.mozilla.org/security/2018/04/24/same-site-cookies-in-firefox-60/

TLDR: We use lax since we want the cookies to be sent when the user navigates
safely from an external site.
2018-09-08 23:54:28 +02:00
Eugen Rochko
601a0dc6cc Add preference for report notification e-mails, skip for duplicates (#8559)
If an unresolved report for the same target account already exists,
no new notification is generated
2018-09-02 00:11:58 +02:00
Renato "Lond" Cerqueira
d516e7fa62 Fix autoplay issue with spoiler tag (#8540)
Add tests to avoid similar issues in the future
2018-08-31 15:16:59 +02:00
Renato "Lond" Cerqueira
c7423078ce Add animate custom emoji param to embed pages (#8507)
* Add animate custom emoji param to embed pages

* Rename param, use it for avatars and gifs

* Fix issues pointed by codeclimate and breaking test

* Ignore brakeman warning
2018-08-30 23:14:01 +02:00
Renato "Lond" Cerqueira
900d3e214b Fix nil host in remotable (#8508)
Host can be nil in urls like
'https:https://example.com/path/file.png'
2018-08-29 21:13:49 +02:00
sundevour
45af08148d formatter spec fixes & clarification (#8481)
updates some "context" and "it" lines to have clearer explanations
updates "context" lines to properly describe function input, and "it" lines to describe results
2018-08-29 01:20:56 +02:00
Eugen Rochko
f507c9713f Display pending message on admin relays UI (#8494)
* Add missing specs for relay accept/reject

* Display pending message on admin relays UI
2018-08-28 05:39:43 +02:00
Jakub Mendyk
9c11d260e1 Make Api::V1::MutesController paginate properly (#8472)
Fixes #8463
2018-08-26 21:30:17 +02:00
Eugen Rochko
660475198b Add theme identifier to body classes for easier custom CSS styling (#8439)
Add forgotten custom CSS admin setting strings
2018-08-25 22:55:25 +02:00
Eugen Rochko
413a28499d Allow mods to disable login, improve message when login disabled (#8329)
* Allow moderators to disable/enable login

* Instead of rejecting login, show forbidden error when login disabled

Avoid confusion because when login is rejected, the message is that
the account is not activated, which is wrong.

* Fix tests
2018-08-23 23:26:29 +02:00
Jakub Mendyk
289b4f2838 Add ability to change an instance default theme from the administration panel (#7092) (#8381)
* Add default_settings class method to ScopedSettings

ScopedSettings was extended to use value of unscoped setting instead of
only using defaults set in config/settings.yml for selected settings.
This adds possibility for admins to set default values of users' settings,
for example default theme (as requested in #7092).

* Add ability to change an instance default theme

Closes #7092
2018-08-23 14:17:35 +02:00
Eugen Rochko
bb7dce98a0 Improve federated ID validation (#8372)
* Fix URI not being sufficiently validated with prefetched JSON

* Add additional id validation to OStatus documents, when possible
2018-08-22 20:55:14 +02:00
masarakki
86a912d6f0 exclude-other-silenced-accounts (#7528) 2018-08-22 13:20:50 +02:00
Eugen Rochko
801b0f583f Add confirmation step to account suspensions (#8353)
* Add confirmation page for suspensions

* Suspension confirmation closes reports, linked from report UI

* Fix tests
2018-08-22 11:53:41 +02:00
Eugen Rochko
35c99eeb0f Increase reach of Delete->Actor activities (#8305)
Fix #7316
2018-08-20 13:28:05 +02:00
Eugen Rochko
6eca34ffb8 Fix error when trying to update counters for statuses that are gone (#8251) 2018-08-18 03:03:23 +02:00
Eugen Rochko
273e8f0474 Add remote interaction dialog for toots (#8202)
* Add remote interaction dialog for toots

* Change AuthorizeFollow into AuthorizeInteraction, support statuses

* Update brakeman.ignore

* Adjust how interaction buttons are display on public pages

* Fix tests
2018-08-18 03:03:12 +02:00
ThibG
20ced948f2 Implement Undo { Accept { Follow } } (fixes #8234) (#8245)
* Add Follow#revoke_request!

* Implement Undo { Accept { Follow } } (fixes #8234)
2018-08-17 16:24:56 +02:00