Commit graph

598 commits

Author SHA1 Message Date
aschmitz
97c02c3389 Make IdsToBigints (mostly!) non-blocking (#5088)
* Make IdsToBigints (mostly!) non-blocking

This pulls in GitLab's MigrationHelpers, which include code to make
column changes in ways that Postgres can do without locking. In general,
this involves creating a new column, adding an index and any foreign
keys as appropriate, adding a trigger to keep it populated alongside
the old column, and then progressively copying data over to the new
column, before removing the old column and replacing it with the new
one.

A few changes to GitLab's MigrationHelpers were necessary:

* Some changes were made to remove dependencies on other GitLab code.
* We explicitly wait for index creation before forging ahead on column
  replacements.
* We use different temporary column names, to avoid running into index
  name length limits.
* We rename the generated indices back to what they "should" be after
  replacing columns.
* We rename the generated foreign keys to use the new column names when
  we had to create them. (This allows the migration to be rolled back
  without incident.)

# Big Scary Warning

There are two things here that may trip up large instances:

1. The change for tables' "id" columns is not concurrent. In
   particular, the stream_entries table may be big, and does not
   concurrently migrate its id column. (On the other hand, x_id type
   columns are all concurrent.)
2. This migration will take a long time to run, *but it should not
   lock tables during that time* (with the exception of the "id"
   columns as described above). That means this should probably be run
   in `screen` or some other session that can be run for a long time.
   Notably, the migration will take *longer* than it would without
   these changes, but the website will still be responsive during that
   time.

These changes were tested on a relatively large statuses table (256k
entries), and the service remained responsive during the migration.
Migrations both forward and backward were tested.

* Rubocop fixes

* MigrationHelpers: Support ID columns in some cases

This doesn't work in cases where the ID column is referred to as a
foreign key by another table.

* MigrationHelpers: support foreign keys for ID cols

Note that this does not yet support foreign keys on non-primary-key
columns, but Mastodon also doesn't yet have any that we've needed to
migrate.

This means we can perform fully "concurrent" migrations to change ID
column types, and the IdsToBigints migration can happen with effectively
no downtime. (A few operations require a transaction, such as renaming
columns or deleting them, but these transactions should not block for
noticeable amounts of time.)

The algorithm for generating foreign key names has changed with this,
and therefore all of those changed in schema.rb.

* Provide status, allow for interruptions

The MigrationHelpers now allow restarting the rename of a column if it
was interrupted, by removing the old "new column" and re-starting the
process.

Along with this, they now provide status updates on the changes which
are happening, as well as indications about when the changes can be
safely interrupted (when there are at least 10 seconds estimated to be
left before copying data is complete).

The IdsToBigints migration now also sorts the columns it migrates by
size, starting with the largest tables. This should provide
administrators a worst-case scenario estimate for the length of
migrations: each successive change will get faster, giving admins a
chance to abort early on if they need to run the migration later. The
idea is that this does not force them to try to time interruptions
between smaller migrations.

* Fix column sorting in IdsToBigints

Not a significant change, but it impacts the order of columns in the
database and db/schema.rb.

* Actually pause before IdsToBigints
2017-10-02 21:28:59 +02:00
Eugen Rochko
f4ca116ea8 After 7 days of repeated delivery failures, give up on inbox (#5131)
- A successful delivery cancels it out
- An incoming delivery from account of the inbox cancels it out
2017-09-29 03:16:20 +02:00
Eugen Rochko
887cd94e96 Increase attachment descriptions to 420 characters (#5139)
Blaze it
2017-09-29 02:30:00 +02:00
Eugen Rochko
76f360c625 If HTTP signature is wrong and webfinger cache is stale, retry with resolve (#5129)
If the signature could not be verified and the webfinger of the account
was last retrieved longer than the cache period, try re-resolving the
account and then attempting to verify the signature again
2017-09-28 17:50:14 +02:00
Eugen Rochko
4ec1771165 Add ability to specify alternative text for media attachments (#5123)
* Fix #117 - Add ability to specify alternative text for media attachments

- POST /api/v1/media accepts `description` straight away
- PUT /api/v1/media/:id to update `description` (only for unattached ones)
- Serialized as `name` of Document object in ActivityPub
- Uploads form adjusted for better performance and description input

* Add tests

* Change undo button blend mode to difference
2017-09-28 15:31:31 +02:00
Ondřej Hruška
daad07b1d5
Merge remote-tracking branch 'origin/better-themeing' into merging-upstream 2017-09-28 09:20:59 +02:00
Ondřej Hruška
83bda6c1a8
Merge commit 'b9f59ebcc68e9da0a7158741a1a2ef3564e1321e' into merging-upstream 2017-09-28 09:18:35 +02:00
Ondřej Hruška
9330ea1f4d
Merge commit '4aea3f88a6d30f102a79c2da7fcfac96465ba1a8' into merging-upstream 2017-09-28 09:12:17 +02:00
Eugen Rochko
3caf0ba923 Fix empty query sent to postgres for custom emojis (#5121) 2017-09-27 04:14:03 +02:00
kibigo!
86e617a839 Better themeing support!! 2017-09-25 19:24:32 -07:00
unarist
5c82c2b75f Increase max height of preview card image (#5092)
We added horizontal layout to preview card for wide image. However, max height of the thumbnail is still limited to 120px and it makes nearly square images to too small for that layout.

This PR increases max height as well as max width.
2017-09-25 17:10:49 +02:00
Eugen Rochko
1e02ba111a Add emoji autosuggest (#5053)
* Add emoji autosuggest

Some credit goes to glitch-soc/mastodon#149

* Remove server-side shortcode->unicode conversion

* Insert shortcode when suggestion is custom emoji

* Remove remnant of server-side emojis

* Update style of autosuggestions

* Fix wrong emoji filenames generated in autosuggest item

* Do not lazy load emoji picker, as that no longer works

* Fix custom emoji autosuggest

* Fix multiple "Custom" categories getting added to emoji index, only add once
2017-09-23 14:47:32 +02:00
Eugen Rochko
293972f716 New API: GET /api/v1/custom_emojis to get a server's custom emojis (#5051) 2017-09-23 01:57:23 +02:00
Eugen Rochko
9c8e602163 Fix custom emojis not detected when used in content warning (#5049) 2017-09-23 01:50:17 +02:00
Akihiko Odaki
0de82dd316 Do not filter statuses with unknown languages (#5045) 2017-09-22 19:33:17 +02:00
unarist
c8580eb806 Use file extensions in addition to MIME types for file picker (#5029)
Currently we're using a list of MIME types for `accept` attribute on `input[type="file"]` for filter options of file picker, and actual file extensions will be infered by browsers. However, infered extensions may not include our expected items. For example, "image/jpeg" seems to be infered to
only ".jfif" extension in Firefox.

To ensure common file extensions are in the list, this PR adds file extensions in addition to MIME types. Also having items in both format is encouraged by HTML5 spec.

https://www.w3.org/TR/html5/forms.html#file-upload-state-(type=file)
2017-09-20 19:07:23 +02:00
Akihiko Odaki
bb4d005a83 Introduce OStatus::TagManager (#5008) 2017-09-19 18:08:08 +02:00
Andrew
0401a24558 Add support for multiple themes (#4959)
* Add support for selecting a theme

* Fix codeclimate issues

* Look up site default style if current user is not available due to e.g. not being logged in

* Remove outdated comment in common.js

* Address requested changes in themes PR

* Fix codeclimate issues

* Explicitly check current_account in application controller and only check theme availability if non-nil

* codeclimate

* explicit precedence with &&

* Fix code style in application_controller according to @nightpool's suggestion, use default style in embedded.html.haml

* codeclimate: indentation + return
2017-09-19 16:36:23 +02:00
Eugen Rochko
81cec35dbf Custom emoji (#4988)
* Custom emoji

- In OStatus: `<link rel="emoji" name="coolcat" href="http://..." />`
- In ActivityPub: `{ type: "Emoji", name: ":coolcat:", href: "http://..." }`
- In REST API: Status object includes `emojis` array (`shortcode`, `url`)
- Domain blocks with reject media stop emojis
- Emoji file up to 50KB
- Web UI handles custom emojis
- Static pages render custom emojis as `<img />` tags

Side effects:

- Undo #4500 optimization, as I needed to modify it to restore
  shortcode handling in emojify()
- Formatter#plaintext should now make sure stripped out line-breaks
  and paragraphs are replaced with newlines

* Fix emoji at the start not being converted
2017-09-19 02:42:40 +02:00
unarist
dd6f9a1b82 Validate uri presence for remote status (#4985) 2017-09-17 15:21:57 +02:00
Akihiko Odaki
198a9a4fa4 Remove local_only scope in Status (#4977) 2017-09-17 05:29:43 +02:00
Eugen Rochko
54edb4b853 When accessing uncached media attachment, redownload it (#4955)
* When accessing uncached media attachment, redownload it

* Prevent re-download of rejected media
2017-09-16 03:01:45 +02:00
Eugen Rochko
04046a4983 Fix #4908 - Do not keep remote file names, generate random (#4934) 2017-09-14 16:41:59 +02:00
Surinna Curtis
fd9a171129 fix typos in the migration 2017-09-13 22:35:48 -05:00
Surinna Curtis
f9d7b8a94f Refactor handling of default params for muting to make code cleaner 2017-09-13 21:47:30 -05:00
Surinna Curtis
211f0a9513 add an explanatory comment to AccountInteractions 2017-09-13 21:47:30 -05:00
Surinna Curtis
0284fd723b Allow modifying the hide_notifications of a mute with the /api/v1/accounts/:id/mute endpoint 2017-09-13 21:47:30 -05:00
Surinna Curtis
7dbcc7ed3d Expose whether a mute hides notifications in the api/v1/relationships endpoint 2017-09-13 21:47:30 -05:00
Surinna Curtis
44207b6af6 Add muting_notifications? and a notifications argument to mute! 2017-09-13 21:47:30 -05:00
Eugen Rochko
9239e4ce4d Uploads for admin site settings (#4913)
* Improve OpenGraph tags for about pages

* Add thumbnail admin setting

* Fix error

* Fix up
2017-09-14 00:04:30 +02:00
nullkal
da77f65c46 Add instance search feature (#4925) 2017-09-13 12:30:07 +02:00
Eugen Rochko
0a6b098668 Fix count numbers from ActivityPub not being saved (#4899)
They are marked as read-only by Rails, but we know what we are doing,
so we are un-marking them as such.

The mastodon:maintenance:update_counter_caches task is not really
supposed to be run anymore (it was a one-time thing during an upgrade)
however, just in case, I have modified it to not touch ActivityPub
accounts.

Also, no point writing to logger from these rake tasks, since they
are not to be run from cron. Better to give stdout feedback.
2017-09-12 00:16:03 +02:00
Surinna Curtis
3cac5bc2c3 Fix a spuriously failing spec that assumed we required short bios like upstream 2017-09-10 19:43:52 -05:00
David Yip
932571fa22 Merge tag 'v1.6.0' into sync/upstream 2017-09-10 13:04:27 -05:00
Eugen Rochko
7d7844a47f Default follows for new users (#4871)
When a new user confirms their e-mail, bootstrap their home timeline
by automatically following a set of accounts. By default, all local
admin accounts (that are unlocked). Can be customized by new admin
setting (comma-separated usernames, local and unlocked only)
2017-09-10 09:58:38 +02:00
David Yip
67ad453373 Merge tag 'v1.6.0rc4' into sync/upstream-1.6.0rc4
Conflicts:
      app/javascript/mastodon/features/getting_started/index.js
      app/javascript/packs/public.js
      app/javascript/styles/components.scss
2017-09-09 23:56:21 -05:00
David Yip
514fc908a3 Merge tag 'v1.6.0rc3' into sync/upstream 2017-09-09 14:28:08 -05:00
David Yip
b9f7bc149b Merge branch 'origin/master' into sync/upstream
Conflicts:
	app/javascript/mastodon/components/status_list.js
	app/javascript/mastodon/features/notifications/index.js
	app/javascript/mastodon/features/ui/components/modal_root.js
	app/javascript/mastodon/features/ui/components/onboarding_modal.js
	app/javascript/mastodon/features/ui/index.js
	app/javascript/styles/about.scss
	app/javascript/styles/accounts.scss
	app/javascript/styles/components.scss
	app/presenters/instance_presenter.rb
	app/services/post_status_service.rb
	app/services/reblog_service.rb
	app/views/about/more.html.haml
	app/views/about/show.html.haml
	app/views/accounts/_header.html.haml
	config/webpack/loaders/babel.js
	spec/controllers/api/v1/accounts/credentials_controller_spec.rb
2017-09-09 14:27:47 -05:00
Eugen Rochko
dd6ede554f Fix #4834 - Adjust Status#local and Status#remote scopes (#4839) 2017-09-07 20:18:34 +02:00
voidSatisfaction
8185f98872 Feat add validation for report comment: characters under 1000 valid (#4833) 2017-09-07 09:55:42 +02:00
Eugen Rochko
11bddd31ce Fix locking migration on statuses table. Nullable column and NO default value (#4825) 2017-09-06 20:57:52 +02:00
Eugen Rochko
e7adbf572a Switch to static URIs, new URI format in both protocols for new statuses (#4815)
* Decouple Status#local? from uri being nil

* Replace on-the-fly URI generation with stored URIs

- Generate URI in after_save hook for local statuses
- Use static value in TagManager when available, fallback to tag format
- Make TagManager use ActivityPub::TagManager to understand new format
- Adjust tests

* Use other heuristic for locality of old statuses, do not perform long query

* Exclude tombstone stream entries from Atom feed

* Prevent nil statuses from landing in Pubsubhubbub::DistributionWorker

* Fix URI not being saved (#4818)

* Add more specs for Status

* Save generated uri immediately

and also fix method order to minimize diff.

* Fix alternate HTML URL in Atom

* Fix tests

* Remove not-null constraint from statuses migration to speed it up
2017-09-06 19:01:28 +02:00
abcang
6ec1aa372d Validate data of Imports (#4782) 2017-09-02 20:45:42 +02:00
unarist
bfa7f9ebf2 Fix WebPush (regression from #4524) (#4778)
* Fix NoMethodError in Web::PushSubscription

```
undefined method `site_contact_email' for #<Class:0x00005976d13c40>

/usr/local/bundle/gems/activerecord-5.1.3/lib/active_record/dynamic_matchers.rb:22:in `method_missing'
/usr/local/bundle/gems/attr_encrypted-3.0.3/lib/attr_encrypted.rb:295:in `method_missing'
/usr/local/bundle/gems/attr_encrypted-3.0.3/lib/attr_encrypted/adapters/active_record.rb:129:in `method_missing_with_attr_encrypted'
/mastodon/app/models/web/push_subscription.rb:53:in `push_payload'
```

* Specify serializer in Web::NotificationSerializer
2017-09-02 16:50:42 +02:00
Eugen Rochko
7dc5035031 Make PreviewCard records reuseable between statuses (#4642)
* Make PreviewCard records reuseable between statuses

**Warning!** Migration truncates preview_cards tablec

* Allow a wider thumbnail for link preview, display it in horizontal layout (#4648)

* Delete preview cards files before truncating

* Rename old table instead of truncating it

* Add mastodon:maintenance:remove_deprecated_preview_cards

* Ignore deprecated_preview_cards in schema definition

* Fix null behaviour
2017-09-01 16:20:16 +02:00
Eugen Rochko
74437c6bff Refactor Web::PushSubscription, remove welcome message (#4524)
* Refactor Web::PushSubscription, remove welcome message

* Add missing helper

* Use locale of the receiver on push notifications (#4519)

* Remove unused translations

* Fix dir on notifications
2017-09-01 13:35:23 +02:00
nullkal
39e7a763ff Use request.remote_ip instead of request.ip (#4744) 2017-08-30 10:24:30 +02:00
Eugen Rochko
4c76402ba1 Serialize ActivityPub alternate link into OStatus deletes, handle it (#4730)
Requires moving Atom rendering from DistributionWorker (where
`stream_entry.status` is already nil) to inline (where
`stream_entry.status.destroyed?` is true) and distributing that.

Unfortunately, such XML renderings can no longer be easily chained
together into one payload of n items.
2017-08-29 16:11:05 +02:00
abcang
ce9a5f358e rescue HTTP::ConnectionError in RemoteFollowController#create (#4726) 2017-08-28 19:12:09 +02:00
nullkal
c2af138113 Allow multiple pinned statuses to be shown and make them be ordered b… (#4690)
* Allow multiple pinned statuses to be shown and make them be ordered by pinned date

* Set timestamps NOT NULL

* Make single-line pinned_statuses

* Spec for pinned_statuses

* Remove redundant empty line
2017-08-25 18:50:52 +02:00
Eugen Rochko
9caa90025f Pinned statuses (#4675)
* Pinned statuses

* yarn manage:translations
2017-08-25 01:41:18 +02:00
Eugen Rochko
c66fe2aeba Minor performance improvement for test suite (#4678) 2017-08-24 13:31:55 +02:00
Colin Mitchell
871c0d251a Application prefs section (#2758)
* Add code for creating/managing apps to settings section

* Add specs for app changes

* Fix controller spec

* Fix view file I pasted over by mistake

* Add locale strings. Add 'my apps' to nav

* Add Client ID/Secret to App page. Add some visual separation

* Fix rubocop warnings

* Fix embarrassing typo

I lost an `end` statement while fixing a merge conflict.

* Add code for creating/managing apps to settings section

- Add specs for app changes
- Add locale strings. Add 'my apps' to nav
- Add Client ID/Secret to App page. Add some visual separation
- Fix some bugs/warnings

* Update to match code standards

* Trigger notification

* Add warning about not sharing API secrets

* Tweak spec a bit

* Cleanup fixture creation by using let!

* Remove unused key

* Add foreign key for application<->user
2017-08-22 18:33:57 +02:00
Eugen Rochko
f391a4673a Periodically remove expired PuSH subscribers (#4654) 2017-08-21 22:56:33 +02:00
Eugen Rochko
3534e115e5 Do not try to re-subscribe to unsubscribed accounts (#4653) 2017-08-21 17:32:41 +02:00
Eugen Rochko
ca7ea1aba9 Redesign public profiles (#4608)
* Redesign public profiles

* Responsive design

* Change public profile status filtering defaults and add options

- No longer displays private/direct toots even if you are permitted access
- By default omits replies
- "With replies" option
- "Media only" option

* Redesign account grid cards

* Fix style issues
2017-08-16 17:12:58 +02:00
Eugen Rochko
b7370ac8ba ActivityPub delivery (#4566)
* Deliver ActivityPub Like

* Deliver ActivityPub Undo-Like

* Deliver ActivityPub Create/Announce activities

* Deliver ActivityPub creates from mentions

* Deliver ActivityPub Block/Undo-Block

* Deliver ActivityPub Accept/Reject-Follow

* Deliver ActivityPub Undo-Follow

* Deliver ActivityPub Follow

* Deliver ActivityPub Delete activities

Incidentally fix #889

* Adjust BatchedRemoveStatusService for ActivityPub

* Add tests for ActivityPub workers

* Add tests for FollowService

* Add tests for FavouriteService, UnfollowService and PostStatusService

* Add tests for ReblogService, BlockService, UnblockService, ProcessMentionsService

* Add tests for AuthorizeFollowService, RejectFollowService, RemoveStatusService

* Add tests for BatchedRemoveStatusService

* Deliver updates to a local account to ActivityPub followers

* Minor adjustments
2017-08-13 00:44:41 +02:00
Eugen Rochko
dd7ef0dc41 Add ActivityPub inbox (#4216)
* Add ActivityPub inbox

* Handle ActivityPub deletes

* Handle ActivityPub creates

* Handle ActivityPub announces

* Stubs for handling all activities that need to be handled

* Add ActivityPub actor resolving

* Handle conversation URI passing in ActivityPub

* Handle content language in ActivityPub

* Send accept header when fetching actor, handle JSON parse errors

* Test for ActivityPub::FetchRemoteAccountService

* Handle public key and icon/image when embedded/as array/as resolvable URI

* Implement ActivityPub::FetchRemoteStatusService

* Add stubs for more interactions

* Undo activities implemented

* Handle out of order activities

* Hook up ActivityPub to ResolveRemoteAccountService, handle
Update Account activities

* Add fragment IDs to all transient activity serializers

* Add tests and fixes

* Add stubs for missing tests

* Add more tests

* Add more tests
2017-08-08 21:52:15 +02:00
Yamagishi Kazutoshi
81c41d8681 Add coalesce option to avatar and header convert processor (#4552)
Resolve #3199
2017-08-08 15:49:14 +02:00
Yamagishi Kazutoshi
1b5d26735e Revert "Set false to animated options for thumbnail processor" (#4550)
* Revert "Adjust tags and accounts page (#4534)"

This reverts commit a3e53bd442.

* Revert "feat: Cache status height to avoid expensive renders (#4439)"

This reverts commit 8eb6d171e6.

* Revert "Refactor Avatar and AvatarOverlay to have 'account' as prop instead of src and staticSrc (#4526)"

This reverts commit 5942347407.

* Revert "Update dependencies for Ruby (#4543)"

This reverts commit 22db947225.

* Revert "[Docker] Add multicore support to "make" and "bundler" (#4544)"

This reverts commit 5d408fd9aa.

* Revert "It makes no sense to try using invalid or expired link again (#4521)"

This reverts commit 47579ec58c.

* Revert "i18n: Update Polish translation (#4545)"

This reverts commit 3363a05539.

* Revert "Set false to animated options for thumbnail processor (#4547)"

This reverts commit 87f10d476c.
2017-08-08 01:49:56 +02:00
Ondřej Hruška
7a1ca8b0df Merge remote-tracking branch 'upstream/master' 2017-08-07 22:43:52 +02:00
Yamagishi Kazutoshi
87f10d476c Set false to animated options for thumbnail processor (#4547)
Resolve #3199

Fix the aspect ratio of animated GIF whose background is transparent.
2017-08-07 14:41:21 +02:00
kibigo!
8150689b48 Merge upstream (#111) 2017-08-01 13:20:29 -07:00
Sorin Davidoi
d4c4820c03 fix(web_push_notifications): Send message with welcome notifications (#4471)
* fix(web_push_notifications): Send grouping title in welcome notification

* fix(web_push_notifications): Read message from last notification
2017-08-01 06:00:13 +02:00
Eugen Rochko
53b2b1b238 Count all URLs in text as 23 characters flat, do not count domain part of usernames (#4427)
* Count all URLs in text as 23 characters flat, do not count domain part of usernames

* Add new status text counting logic to web UI
2017-07-29 00:06:29 +02:00
Eugen Rochko
f0bb2c6d1e Fix web push notifications "boost" icon not being loaded (regression from #4426) (#4431) 2017-07-28 20:26:54 +02:00
Eugen Rochko
13bb1ddc7f Use consistent icons for web push notifications, same as web UI (#4426) 2017-07-28 18:13:42 +02:00
Sorin Davidoi
e67fc997dc feat(web_push_notifications): Group notifications (#4388) 2017-07-28 05:06:22 +02:00
Sorin Davidoi
e1798d0eb0 fix(push_subscription): Use star icon for favourite action (#4396) 2017-07-27 15:46:33 +02:00
Ondřej Hruška
7e5691804d Merge git://github.com/tootsuite/mastodon into tootsuite-master 2017-07-25 21:36:22 +02:00
sdukhovni
06d905f415 Allow domain blocks that only reject media without silencing or suspending (#4325)
* Allow domain blocks to reject media without silencing or suspending

* Fix typo

* Hide 'Reject media' button when superfluous, instead of disabling it

* Properly hide 'reject media' checkbox on page load if needed

This may happen when resubmitting the domain block form after an error.

* Don't ask whether undoing a media-only block should be retroactive

* Rename :media_only block to :noop

* Display :noop block as None in frontend, allow blocks that do nothing

* Remove 'coding' line auto-added by emacs
2017-07-24 14:26:55 +02:00
Eugen Rochko
7964bfccdb Add rake task mastodon:feeds:build to regenerate all active users' feeds (#4303) 2017-07-23 01:15:04 +02:00
beatrix
e7edb4d1ee Merge pull request #87 from tootsuite/master
merge upstream
2017-07-20 11:24:32 -04:00
Sorin Davidoi
dcb9497148 fix(push_subscription): Guard against malformed favourite notification (#4271) 2017-07-19 22:37:52 +02:00
Eugen Rochko
bbdcfd6baf Add ActivityPub attributes to accounts (#4273) 2017-07-19 17:06:46 +02:00
Eugen Rochko
c2c93f8cd6 Fix deprecation warning for non-pipeline assets (#4267) 2017-07-19 12:00:05 +02:00
Ondřej Hruška
d69fa9e1f4 Merge changes from upstream with the CSS reload fix 2017-07-18 18:58:47 +02:00
Yamagishi Kazutoshi
3267e4a785 Add unfollow modal (optional) (#4246)
* Add unfollow modal

* unfollowing someone

* remove unnecessary prop
2017-07-18 17:14:43 +02:00
abcang
4d42a38954 Improve admin page (#4121)
* Improve admin page

* Fix test

* Add spec

* Improve select style
2017-07-18 16:38:22 +02:00
Sorin Davidoi
8387b3928e fix(push-subscriptions): Refactor how Sidekiq jobs are handled (#4226) 2017-07-18 16:25:40 +02:00
kibigo!
09cfc079b0 Merge upstream (#81) 2017-07-15 14:33:15 -07:00
Eugen Rochko
cd9b2ab2f7 Fix #2672 - Connect signed PuSH subscription requests to instance domain (#4205)
* Fix #2672 - Connect signed PuSH subscription requests to instance domain

Resolves #2739

* Fix return of locate_subscription

* Fix tests
2017-07-14 23:01:20 +02:00
Eugen Rochko
1618b68bfa HTTP signatures (#4146)
* Add Request class with HTTP signature generator

Spec: https://tools.ietf.org/html/draft-cavage-http-signatures-06

* Add HTTP signature verification concern

* Add test for SignatureVerification concern

* Add basic test for Request class

* Make PuSH subscribe/unsubscribe requests use new Request class

Accidentally fix lease_seconds not being set and sent properly, and
change the new minimum subscription duration to 1 day

* Make all PuSH workers use new Request class

* Make Salmon sender use new Request class

* Make FetchLinkService use new Request class

* Make FetchAtomService use the new Request class

* Make Remotable use the new Request class

* Make ResolveRemoteAccountService use the new Request class

* Add more tests

* Allow +-30 seconds window for signed request to remain valid

* Disable time window validation for signed requests, restore 7 days
as PuSH subscription duration (which was previous default due to a bug)
2017-07-14 20:41:49 +02:00
Eugen Rochko
e2685ccc81 Fix #4149, fix #1199 - Store emojis as unicode (#4189)
- Use unicode when selecting emoji through picker
- Convert shortcodes to unicode when storing text input server-side
- Do not convert shortcodes in JS anymore
2017-07-14 19:47:53 +02:00
Eugen Rochko
c42092ba7a Add option to opt out of search engines on public profile/status pages (#4199) 2017-07-14 16:41:02 +02:00
masarakki
a49be27145 add validation to tag name (#4194) 2017-07-14 11:02:49 +02:00
Sorin Davidoi
0c7c188c45 Web Push Notifications (#3243)
* feat: Register push subscription

* feat: Notify when mentioned

* feat: Boost, favourite, reply, follow, follow request

* feat: Notification interaction

* feat: Handle change of public key

* feat: Unsubscribe if things go wrong

* feat: Do not send normal notifications if push is enabled

* feat: Focus client if open

* refactor: Move push logic to WebPushSubscription

* feat: Better title and body

* feat: Localize messages

* chore: Fix lint errors

* feat: Settings

* refactor: Lazy load

* fix: Check if push settings exist

* feat: Device-based preferences

* refactor: Simplify logic

* refactor: Pull request feedback

* refactor: Pull request feedback

* refactor: Create /api/web/push_subscriptions endpoint

* feat: Spec PushSubscriptionController

* refactor: WebPushSubscription => Web::PushSubscription

* feat: Spec Web::PushSubscription

* feat: Display first media attachment

* feat: Support direction

* fix: Stuff broken while rebasing

* refactor: Integration with session activations

* refactor: Cleanup

* refactor: Simplify implementation

* feat: Set VAPID keys via environment

* chore: Comments

* fix: Crash when no alerts

* fix: Set VAPID keys in testing environment

* fix: Follow link

* feat: Notification actions

* fix: Delete previous subscription

* chore: Temporary logs

* refactor: Move migration to a later date

* fix: Fetch the correct session activation and misc bugs

* refactor: Move migration to a later date

* fix: Remove follow request (no notifications)

* feat: Send administrator contact to push service

* feat: Set time-to-live

* fix: Do not show sensitive images

* fix: Reducer crash in error handling

* feat: Add badge

* chore: Fix lint error

* fix: Checkbox label overlap

* fix: Check for payload support

* fix: Rename action "type" (crash in latest Chrome)

* feat: Action to expand notification

* fix: Lint errors

* fix: Unescape notification body

* fix: Do not allow boosting if the status is hidden

* feat: Add VAPID keys to the production sample environment

* fix: Strip HTML tags from status

* refactor: Better error messages

* refactor: Handle browser not implementing the VAPID protocol (Samsung Internet)

* fix: Error when target_status is nil

* fix: Handle lack of image

* fix: Delete reference to invalid subscriptions

* feat: Better error handling

* fix: Unescape HTML characters after tags are striped

* refactor: Simpify code

* fix: Modify to work with #4091

* Sort strings alphabetically

* i18n: Updated Polish translation

it annoys me that it's not fully localized :P

* refactor: Use current_session in PushSubscriptionController

* fix: Rebase mistake

* fix: Set cacheName to mastodon

* refactor: Pull request feedback

* refactor: Remove logging statements

* chore(yarn): Fix conflicts with master

* chore(yarn): Copy latest from master

* chore(yarn): Readd offline-plugin

* refactor: Use save! and update!

* refactor: Send notifications async

* fix: Allow retry when push fails

* fix: Save track for failed pushes

* fix: Minify sw.js

* fix: Remove account_id from fabricator
2017-07-13 22:15:32 +02:00
unarist
c2753fdfb4 Make tag search case insensitive again (#4184) 2017-07-13 19:31:33 +02:00
Eugen Rochko
880a5eb25c Fix boolean columns sometimes having a null value (#4162)
* Fix boolean columns sometimes having a null value

* Fix wrong value being set instead of null
2017-07-13 03:12:25 +02:00
Eugen Rochko
e48d3bfd01 Fix #1010 - When spoiler text is set, enforce sensitivity too (#4176) 2017-07-13 01:47:23 +02:00
kibigo!
79d898ae0a Merge upstream!! #64 <3 <3 2017-07-12 02:03:17 -07:00
Eugen Rochko
056b5ed72f Improve UI of admin site settings (#4163) 2017-07-12 03:24:04 +02:00
Eugen Rochko
e6c81a635b Fix #2848 - Rake task to redownload avatars/headers (#4156)
Can be filtered by a specific domain
Resolves #2292
2017-07-11 17:25:49 +02:00
Yamagishi Kazutoshi
2b9721d1b3 Add setting a always mark media as sensitive (#4136) 2017-07-10 14:00:32 +02:00
Eugen Rochko
8b2cad5637 Refactor JSON templates to be generated with ActiveModelSerializers instead of Rabl (#4090) 2017-07-07 04:02:06 +02:00
Damien Erambert
18d3fa953b Add a setting allowing the use of system's default font in Web UI (#4033)
* add a system_font_ui setting on the server

* Plug the system_font_ui on the front-end

* add EN/FR locales for the new setting

* put Roboto after all other fonts

* remove trailing whitespace so CodeClimate is happy

* fix user_spec.rb

* correctly write user_spect this time

* slightly better way of adding the classes

* add comments to the system-font stack for clarification

* use .system-font for the class instead

* don't use multiple lines for comments

* remove trailing whitespace

* use the classnames module for consistency

* use `mastodon-font-sans-serif` instead of Roboto directly
2017-07-06 22:39:56 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
a37cf9548c Explicitly require MIME::Types (#4083) 2017-07-05 23:58:03 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
f85dbe83c8 Remove sort in Feed (#4050)
In from_redis method, statuses retrieved from the database was mapped to
the IDs retrieved from Redis. It was equivalent to order from high to low
because those IDs are sorted in the same order.
Statuses are ordered with the ID by default, so we do not have to reorder.
Sorting statuses in the database is even faster since the IDs are indexed
with B-tree.
2017-07-03 13:17:27 +02:00
abcang
a6d02cff36 Rescue exceptions caused by FetchLinkCardService (#4045) 2017-07-03 11:03:34 +02:00
abcang
1273fbf86e Rescue Addressable::URI::InvalidURIError at Remotable (#4017) 2017-06-30 13:38:36 +02:00
Eugen Rochko
71bc75e6ac Do not fail to create access token if superapp was never created (#3986) 2017-06-28 17:43:48 +02:00
beatrix-bitrot
ddafde942c Merge remote-tracking branch 'upstream/master' 2017-06-27 20:46:13 +00:00
Surinna Curtis
4f36aad6e8 don't count bio metadata against bio length on server 2017-06-26 09:34:31 -05:00
beatrix-bitrot
c0a665865e update bio length to 500 2017-06-26 02:18:52 +00:00
Eugen Rochko
ed7dc1704d Bind web UI access tokens to sessions (#3940)
* Add overview of active sessions

* Better display of browser/platform name

* Improve how browser information is stored and displayed for sessions overview

* Fix test

* Fix #2347 - Bind web UI access token to session

When you logout, session also destroys the access token, so it's no longer
valid. If access token is destroyed some other way, the session is also
destroyed, requiring a re-login.

Fix #1681 - Add scheduler to remove revoked access tokens and grants

* Fix test
2017-06-25 23:51:32 +02:00
Eugen Rochko
f7301bd5b9 Add overview of active sessions (#3929)
* Add overview of active sessions

* Better display of browser/platform name

* Improve how browser information is stored and displayed for sessions overview

* Fix test
2017-06-25 16:54:30 +02:00
Sorin Davidoi
2211e8d1cd Revocable sessions (#3616)
* feat: Revocable sessions

* fix: Tests using sign_in

* feat: Configuration entry for the maximum number of session activations
2017-06-23 18:50:53 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
e27f792c24 Some minor change and spec for Account (#3813)
* Introduce domains method to Account relation

Account had followers_domains method, which was excessively specific.
Let relation of Account have domains method instead.

* Move follow_mapping in Account to AccountInteractions

* Introduce shared examples for AccountAvatar inclusion

* Cover Account more
2017-06-22 18:34:27 +02:00
Eugen Rochko
0190aac240 Fix regression from #3842 (#3892)
* Fix regression from #3842

Simplify the query by omitting all direct statuses. Private statuses
are allowed because they are from accounts we are following (so
by definition)

Resolves #3887 (alternative)

* Adjust test
2017-06-22 02:38:50 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
bab5a18232 Filter direct statuses in Status.as_home_timeline (#3842)
The classes using Status.as_home_timeline, namely Feed and
PrecomputeFeedService are expected to filter direct statuses as
FanOutWriteService does, but their filtering were incomplete or missing.

This commit solves the problem by filtering direct statuses in
as_home_timeline as the other similar methods such as as_public_timeline
does.
2017-06-20 20:41:23 +02:00
unarist
a20cf3b64e Fix RemoteFollow behavior (#3868)
* Invalid acct is an error. not "2 errors".
* Empty input should be different error from invalid acct
2017-06-20 20:40:56 +02:00
Eugen Rochko
4a618908e8 Account deletion (#3728)
* Add form for account deletion

* If avatar or header are gone from source, remove them

* Add option to have SuspendAccountService remove user record, add tests

* Exclude suspended accounts from search
2017-06-14 18:01:27 +02:00
Eugen Rochko
dc8a6244fc Fix #2619 - When redis feed is empty, fall back to database (#3721)
* Fix #2619 - When redis feed is empty, fall back to database

* Use redis value to return feed from database only while RegenerationWorker
hasn't finished running

* Fix specs

* Replace usage of reject!
2017-06-14 13:37:03 +02:00
Eugen Rochko
85d5518b6b Fix #3675 - Adjust quality settings of converted GIFs to reduce filesize (#3723) 2017-06-13 00:51:48 +02:00
René Klačan
85d405c810 Fix Account model deprecation warnings (#3689)
```
DEPRECATION WARNING: The behavior of `attribute_changed?` inside of after callbacks will be changing in the next version of Rails. The new return value will reflect the behavior of calling the method after `save` returned (e.g. the opposite of what it returns now). To maintain the current behavior, use `saved_change_to_attribute?` instead. (called from block in <class:Account> at /Users/rene/Workspace/personal/ruby/mastodon/app/models/account.rb:60)
DEPRECATION WARNING: The behavior of `attribute_changed?` inside of after callbacks will be changing in the next version of Rails. The new return value will reflect the behavior of calling the method after `save` returned (e.g. the opposite of what it returns now). To maintain the current behavior, use `saved_change_to_attribute?` instead. (called from block in <class:Account> at /Users/rene/Workspace/personal/ruby/mastodon/app/models/account.rb:60)
DEPRECATION WARNING: The behavior of `attribute_changed?` inside of after callbacks will be changing in the next version of Rails. The new return value will reflect the behavior of calling the method after `save` returned (e.g. the opposite of what it returns now). To maintain the current behavior, use `saved_change_to_attribute?` instead. (called from block in <class:Account> at /Users/rene/Workspace/personal/ruby/mastodon/app/models/account.rb:60)
DEPRECATION WARNING: The behavior of `attribute_changed?` inside of after callbacks will be changing in the next version of Rails. The new return value will reflect the behavior of calling the method after `save` returned (e.g. the opposite of what it returns now). To maintain the current behavior, use `saved_change_to_attribute?` instead. (called from block in <class:Account> at /Users/rene/Workspace/personal/ruby/mastodon/app/models/account.rb:61)
DEPRECATION WARNING: The behavior of `attribute_changed?` inside of after callbacks will be changing in the next version of Rails. The new return value will reflect the behavior of calling the method after `save` returned (e.g. the opposite of what it returns now). To maintain the current behavior, use `saved_change_to_attribute?` instead. (called from block in <class:Account> at /Users/rene/Workspace/personal/ruby/mastodon/app/models/account.rb:62)
DEPRECATION WARNING: The behavior of `attribute_changed?` inside of after callbacks will be changing in the next version of Rails. The new return value will reflect the behavior of calling the method after `save` returned (e.g. the opposite of what it returns now). To maintain the current behavior, use `saved_change_to_attribute?` instead. (called from block in <class:Account> at /Users/rene/Workspace/personal/ruby/mastodon/app/models/account.rb:63)
```

Here's PR describing changes to Dirty API https://github.com/rails/rails/pull/25337
2017-06-11 17:01:32 +02:00
Yamagishi Kazutoshi
f3e8bc9f8f Refactor UpdateRemoteProfileService (#3690) 2017-06-11 10:41:59 +02:00
Eugen Rochko
cdff1da901 Correct validators so that existing error messages would look correct (#3668) 2017-06-09 19:46:01 +02:00
Matt Jankowski
022008a2a6 Language detection defaults to nil (#3666)
* Default to nil for statuses.language

* Language detection defaults to nil instead of instance UI default
2017-06-09 18:09:37 +02:00
Yamagishi Kazutoshi
6384041d17 Add includes to Report#statuses (#3655) 2017-06-09 15:07:02 +02:00
Eugen Rochko
177dd8bb53 Fix regression from #3592 - validation condition nesting (#3644) 2017-06-08 19:10:48 +02:00
Eugen Rochko
c207b4bb33 Fix db:seed - only run some validations when the field was changed (#3592)
* Fix db:seed - only run some validations when the field was changed

* Add tests
2017-06-08 09:22:01 -04:00
Yamagishi Kazutoshi
0a0b9a271a Improve RuboCop rules (compatibility to Code Climate) (#3636)
08f8de84eb/Gemfile.lock (L38)
Code Climate is using RuboCop v0.46.0.

Change several rules to maintain compatibility.
2017-06-08 13:24:28 +02:00
unarist
004672aa6c Fix tag search order and not to use tsvector (#3611)
* Sort results by the name
* Switch search method to simple `LIKE` matching instead of tsvector/tsquery

Previously we used scores from ts_rank_cd() to sort results, but it didn't work
because the function returns same score for all results. It's not for calculate
similarity of single words. Sometimes this bug even push out exact matching tag
from results.

Additionally, PostgreSQL supports prefix searching with standard btree index.
Using it offers simpler code, but also less index size and some speed.
2017-06-06 16:07:06 +02:00
Matt Jankowski
6201f96b8a Introduce StatusThreadingConcern (#3490)
* Add a StatusFilter class to identify visibility of statuses by accounts

* Extract StatusThreadingConcern from Status

* Clarify purpose of checking for nil account
2017-06-05 16:07:44 +02:00
Eugen Rochko
f7a30e2fae Added support for configurable reserved usernames (fix of #1382) (#3566)
* Added support for configurable reserved usernames

* Added reserved usernames from mastodon issue 1355

* Fix reserved usernames
2017-06-05 01:03:45 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
19084d3c6c Remove Targetable (#3548) 2017-06-04 14:56:10 +02:00
Matt Jankowski
2212dc4aaa Update Rails to version 5.1.1 (#3121)
* Update rails to version 5.1.1

* Run `rails app:update`

* Remove the override of polymorphic activity relationship

* Silence warning about otp_secret attribute being unknown to rails

* We will only introduce form_with where we want to use remote data
2017-06-01 20:53:37 +02:00
Matt Jankowski
d1e08bd38c Handle nil and blank cases in Account finders (#3500) 2017-06-01 08:20:36 -04:00
Matt Jankowski
a132332b86 Add Instance class to list admin records (#3443) 2017-05-31 20:38:44 +02:00
Matt Jankowski
b25e42a77f Misc tidying and clean ups (#3445)
* Remove trailing whitespace in i18n mailers

* Use query methods instead of #present? on AR attributes

* Delegate Status#account_domain method

* Delegate Mention #account_username and #account_acct methods
2017-05-31 20:38:17 +02:00
Matt Jankowski
2cc3111a77 Expand spec coverage and refactor the Account.find_ methods (#3485)
* Move specs for account finder methods to concern spec

* Move account finder methods to concern

* Improve spec wording

* Use more explicit comparison to ensure correct return value

* Add coverage for .find_local! and .find_remote!

* Add some methods to the finder

* Use arel on matching_username method

* Avoid ternary in matching domain method

* Simplify finder methods

* Use an AccountFinder class to simplify lookup
2017-05-31 20:28:45 +02:00
alpaca-tc
3652a39de0 Account.find_remote should be raise ActiveRecord::RecordNotFound (#3476) 2017-05-31 15:10:26 +02:00
Akihiko Odaki
7db98aa70e Refactor User and spec (#3431)
* Protect send_devise_notification of User

* Improve spec for User
2017-05-30 15:28:56 +02:00
Jack Jennings
3a2003ba86 Extract authorization policy for viewing statuses (#3150) 2017-05-29 18:22:22 +02:00
Atsushi Yamamoto
402c19a924 Add preference setting for delete toot modal (#3368)
* Set delete_modal preference to true by default
* Does not show confirmation modal if delete_modal is false
* Add ja translation for preference setting page
2017-05-29 17:56:13 +02:00
Yamagishi Kazutoshi
bca334cd28 Remove arguments in association reader (#3305) 2017-05-25 16:28:14 +02:00
alpaca-tc
08dce5e607 Refactor Setting (#3302)
* @object is not needed

* Remove unneeded dependencies

* Do not call private method

* Prefer #respond_to_missing? over #respond_to?

`#respond_to?` doesn't support `User.settings.method(:method_name)`

* Use find_or_initialize_by instead of
2017-05-25 14:14:36 +02:00
masarakki
3b59f9c6c2 fix-cache-returns-nil (#3213) 2017-05-25 05:18:08 +02:00
Akihiko Odaki
bf575a1f5e Introduce recent to Follow (#3247)
Introduce recent to Follow, as Account and other models have.
This change also adds specs for the scope and the dependents.
2017-05-23 13:12:19 +02:00
Akihiko Odaki
8fd174298d Cover AccountsController more in spec (#3229)
* Introduce recent scope to Status and StreamEntry

Introduce recent scope to Status and StreamEntry as Account has.

* Cover AccountsController more in AccountsController
2017-05-23 02:53:01 +02:00
Matt Jankowski
e1b42e9aa0 Add coverage for ReportFilter and AccountFilter (#3236) 2017-05-22 15:50:58 -04:00
Matt Jankowski
ec34ec63b1 Specs for cleanup workers (#3235)
* Add spec files for feed and media cleanup workers

* Add coverage for feed and media cleanup schedulers

* Clean up feed and media cleanup workers
2017-05-22 19:36:21 +02:00
Akihiko Odaki
2af5cd96fe Fix mutes_controller error and incorrect statuses_controller report (#3202)
This commit fixes a regression in commit
f554807563.
2017-05-21 13:32:13 +02:00
Matt Jankowski
8f4b7c1820 Filter languages with opt out (#3175)
* Remove allowed_languages and add filtered_languages

* Use filtered_languages instead of allowed_languages
2017-05-20 17:32:44 +02:00
beatrix
724fc3cbdf guard against empty domain block list in status scope (#3161) 2017-05-20 13:38:13 +02:00
Eugen Rochko
f8ee136c29 Fix federated timeline excluding local toots when any domain blocks are set (#3151) 2017-05-19 21:19:41 +02:00
Eugen Rochko
f1ab70649b Add buttons to block and unblock domain (#3127)
* Add buttons to block and unblock domain

* Relationship API now returns "domain_blocking" status for accounts,
rename "block entire domain" to "hide entire domain", fix unblocking domain,
do not block notifications from domain-blocked-but-followed people, do
not send Salmons to domain blocked users

* Add test

* Personal domain blocks shouldn't affect Salmon after all, since in this
direction of communication the control is very thin when it comes to
public stuff. Best stay consistent and not affect federation in this way

* Ignore followers and follow request from domain blocked folks,
ensure account domain blocks are not created for empty domain,
and avoid duplicates in validation

* Purge followers when blocking domain (without soft-blocks, since they
are useless here)

* Add tests, fix local timeline being empty when having any domain blocks
2017-05-19 21:05:32 +02:00
alpaca-tc
198ae3e366 Define instance method outside #included (#3128) 2017-05-19 11:41:45 +02:00
Matt Jankowski
6e4c7d6211 Conditional validations no longer accept strings for if/unless (#3124) 2017-05-19 03:11:23 +02:00
Eugen Rochko
620d0d8029 Account domain blocks (#2381)
* Add <ostatus:conversation /> tag to Atom input/output

Only uses ref attribute (not href) because href would be
the alternate link that's always included also.

Creates new conversation for every non-reply status. Carries
over conversation for every reply. Keeps remote URIs verbatim,
generates local URIs on the fly like the rest of them.

* Conversation muting - prevents notifications that reference a conversation
(including replies, favourites, reblogs) from being created. API endpoints
/api/v1/statuses/:id/mute and /api/v1/statuses/:id/unmute

Currently no way to tell when a status/conversation is muted, so the web UI
only has a "disable notifications" button, doesn't work as a toggle

* Display "Dismiss notifications" on all statuses in notifications column, not just own

* Add "muted" as a boolean attribute on statuses JSON

For now always false on contained reblogs, since it's only relevant for
statuses returned from the notifications endpoint, which are not nested

Remove "Disable notifications" from detailed status view, since it's
only relevant in the notifications column

* Up max class length

* Remove pending test for conversation mute

* Add tests, clean up

* Rename to "mute conversation" and "unmute conversation"

* Raise validation error when trying to mute/unmute status without conversation

* Adding account domain blocks that filter notifications and public timelines

* Add tests for domain blocks in notifications, public timelines
Filter reblogs of blocked domains from home

* Add API for listing and creating account domain blocks

* API for creating/deleting domain blocks, tests for Status#ancestors
and Status#descendants, filter domain blocks from them

* Filter domains in streaming API

* Update account_domain_block_spec.rb
2017-05-19 01:14:30 +02:00