Commit graph

1714 commits

Author SHA1 Message Date
Eugen Rochko
8913ccb434 Store home feeds for 7 days instead of 14 (#7354)
* Store home feeds for 7 days instead of 14

Reduces workload for status fan-out to active followers

* Fix test for user model
2018-05-05 00:54:24 +02:00
Eugen Rochko
154076e8e7 Store URIs of follows, follow requests and blocks for ActivityPub (#7160)
Same URI passed between follow request and follow, since they are
the same thing in ActivityPub. Local URIs are generated during
creation using UUIDs and are passed to serializers.
2018-05-04 21:14:34 +02:00
Shuhei Kitagawa
995d9e8485 Add missing tests for report.rb (#7324) 2018-05-03 10:42:36 +02:00
Eugen Rochko
68ba4f23e0 Serialize webfinger XML with Ox instead of Nokogiri (#7319)
25ms -> 0.5ms
2018-05-02 22:28:46 +02:00
Eugen Rochko
ca1c696dbd Slightly reduce RAM usage (#7301)
* No need to re-require sidekiq plugins, they are required via Gemfile

* Add derailed_benchmarks tool, no need to require TTY gems in Gemfile

* Replace ruby-oembed with FetchOEmbedService

Reduce startup by 45382 allocated objects

* Remove preloaded JSON-LD in favour of caching HTTP responses

Reduce boot RAM by about 6 MiB

* Fix tests

* Fix test suite by stubbing out JSON-LD contexts
2018-05-02 18:58:48 +02:00
abcang
fcfc44de66 Keep notification when muting_notifications is true (#7311)
* Keep notification when muting_notifications is true

* Retrun mute object

* Fix test
2018-05-02 16:14:51 +02:00
Eugen Rochko
d6a2530d73 Speed up test suite by not generating RSA keys in test environment (#7296)
One RSA keypair for all fabricated test accounts is enough
2018-05-02 15:45:24 +02:00
Shuhei Kitagawa
a110cce92b Add missing tests for user.rb (#7306) 2018-05-02 14:13:52 +02:00
Surinna Curtis
72793b3fc7 Support Actors/Statuses with multiple types (#7305)
* Add equals_or_includes_any? helper in JsonLdHelper

* Support arrays in JSON-LD type fields for actors/tags/objects.

* Spec for resolving accounts with extension types

* Style tweaks for codeclimate
2018-05-02 12:40:24 +02:00
MIYAGI Hikaru
50e673409f Append '.test' to hostname in stub data (#7260) 2018-04-25 14:12:28 +02:00
Emelia Smith
1071ecfebc Prevent suspended accounts from appearing in AccountSearchService (#7246) 2018-04-23 21:27:18 +02:00
Akihiko Odaki
e22f682df0 Paginate descendant statuses in public page (#7148) 2018-04-23 19:27:35 +02:00
Eugen Rochko
599aa39aca Remove "nsfw" category for sensitive statuses in OStatus serializer (#7048)
Fix #7011
2018-04-22 22:09:03 +02:00
Yamagishi Kazutoshi
32b71c8a25 Fix randomly fail (similar #7219) (#7225) 2018-04-22 11:41:39 +02:00
Yamagishi Kazutoshi
3637c4983b Reset locale on registration tests (#7219) 2018-04-21 23:37:07 +02:00
Yamagishi Kazutoshi
7da649efb4 Use raw status code on have_http_status (#7214) 2018-04-21 21:35:07 +02:00
Yamagishi Kazutoshi
0d4342e146 Introduce rspec-retry (#7206) 2018-04-20 19:31:30 +02:00
Eugen Rochko
354a6e4026 Improve report layout (#7188)
* Use table for statuses in report

* Display reported account and reporter in the same table

* Split accounts and general report info into two tables again

* Redesign report statuses table, notes, merge notes and action log

* Remove unused translations

* Fix code style issue

* Fix code style issue

* Fix code style issue
2018-04-20 02:28:48 +02:00
Kaito Sinclaire
447741e456 Direct messages column (#4514)
* Added a timeline for Direct statuses
* Lists all Direct statuses you've sent and received
* Displayed in Getting Started
* Streaming server support for direct TL

* Changes to match other timelines in 2.0
2018-04-18 13:09:06 +02:00
Akihiko Odaki
540d2ddfe4 Redirect to account status page for page of status stream entry (#7104)
Commit 6c2344d4dd missed a change for
stream entry page. Instead of duplicating the change, redirect to account
status page. It would also help crawlers (of search engines, for example)
to understand a stream entry URL and its corresponding status URL points
to the same page.
2018-04-16 10:04:31 +02:00
Eugen Rochko
466442b764 Add bio fields (#6645)
* Add bio fields

- Fix #3211
- Fix #232
- Fix #121

* Display bio fields in web UI

* Fix output of links and missing fields

* Federate bio fields over ActivityPub as PropertyValue

* Improve how the fields are stored, add to Edit profile form

* Add rel=me to links in fields

Fix #121
2018-04-14 12:41:08 +02:00
Renato "Lond" Cerqueira
a4732040f0 Allow more than the max pins if account is not local (#7105)
Sidekiq sometimes throws errors for users that have more pinned items
than the allowed by the local instance. It should only validate the
number of pins for local accounts.
2018-04-12 20:36:02 +02:00
Alexander
8a9da4c414 update gem, test pam authentication (#7028)
* update gem, test pam authentication

* add description for test parameters

* fix inclusion of optional group
2018-04-11 21:40:38 +02:00
Akihiko Odaki
6c2344d4dd Paginate ancestor statuses in public page (#7102)
This also limits the statuses returned by API, but pagination is not
implemented in Web API yet. I still expect it brings user experience
better than making a user wait to fetch all ancestor statuses and flooding
the column with them.
2018-04-11 12:35:09 +02:00
Paul Woolcock
ed21e48ad6 Change custom emoji search to ILIKE instead of = (#7099) 2018-04-10 15:46:27 +02:00
Emelia Smith
765bfd0cf5 Feature: Allow staff to change user emails (#7074)
* Admin: Show unconfirmed email address on account page

* Admin: Allow staff to change user email addresses

* ActionLog: On change_email, log current email address and new unconfirmed email address
2018-04-10 09:16:06 +02:00
Levi Bard
6828abc1d7 Enable updating additional account information from user preferences via rest api (#6789)
* Enable updating additional account information from user preferences via rest api
Resolves #6553

* Pacify rubocop

* Decoerce incoming settings in UserSettingsDecorator

* Create user preferences hash directly from incoming credentials instead of going through ActionController::Parameters

* Clean up user preferences update

* Use ActiveModel::Type::Boolean instead of manually checking stringified number equivalence
2018-04-08 13:43:10 +02:00
Eugen Rochko
7f1e924fa9 Add contact account to landing page ("Administered by") (#6984) 2018-04-04 01:11:26 +02:00
Emelia Smith
078aa8b5ae Improve require_admin! and require_staff! filters (#7018)
Previously these returns 302 redirects instead of 403s, which meant posting links to admin pages in slack caused them to unfurl, rather than stay as a link. Additionally, require_admin! doesn't appear to be actively used, on require_staff!
2018-04-03 13:07:32 +02:00
Emelia Smith
5373b138e8 Feature: Report improvements (#6967) (#7000)
* Implement Assignment of Reports (#6967)

* Change translation of admin.report.comment.label to "Report Comment" for clarity

As we'll soon add the ability for reports to have comments on them, this clarification makes sense.

* Implement notes for Reports

This enables moderators to leave comments about a report whilst they work on it

* Fix display of report moderation notes

* Allow reports to be reopened / marked as unresolved

* Redirect to reports listing upon resolution of report

* Implement "resolve with note" functionality

* Add inverse relationship for report notes

* Remove additional database querying when loading report notes

* Fix tests for reports

* Fix localisations for report notes / reports
2018-04-02 22:04:14 +02:00
David Underwood
44274257de [WIP] Enable custom emoji on account pages and in the sidebar (#6124)
Federate custom emojis with accounts
2018-04-01 23:55:42 +02:00
unarist
0fb7252ac4 Add a spec for UniqueUsernameValidator (#6927)
Note that this spec has a pending test about dots in the username,
because allowing it has been reverted for now.
2018-03-27 05:22:58 +02:00
Akihiko Odaki
acece7a2e6 Validate HTTP response length while receiving (#6891)
to_s method of HTTP::Response keeps blocking while it receives the whole
content, no matter how it is big. This means it may waste time to receive
unacceptably large files. It may also consume memory and disk in the
process. This solves the inefficency by checking response length while
receiving.
2018-03-26 14:02:10 +02:00
Akihiko Odaki
472fba6771 Change columns in notifications nonnullable (#6764) 2018-03-24 12:51:28 +01:00
Akihiko Odaki
11c19f6cc9 Close http connection in perform method of Request class (#6889)
HTTP connections must be explicitly closed in many cases, and letting
perform method close connections makes its callers less redundant and
prevent them from forgetting to close connections.
2018-03-24 12:49:54 +01:00
Rey Tucker
4c91be94d3 request: in the event of failure, try other IPs (#6761) (#6813)
* request: in the event of failure, try other IPs (#6761)

In the case where a name has multiple A/AAAA records, we should
try subsequent records instead of immediately failing when we have a
failure on the first IP address.

This significantly improves delivery success when there are network
connectivity problems affecting only IPv4 or IPv6.

* fix method call style

* request_spec: adjust test case to use Addrinfo

* request: Request/open: move private addr check to within begin/rescue

* request_spec: add case to test failover, fix exception check

* Double Addrinfo.foreach so that it correctly yields instances
2018-03-20 09:06:08 +01:00
Eugen Rochko
29f412da2c Fix #6657 - Use target instead of origin in Remove activity (#6664) 2018-03-07 03:54:46 +01:00
Akihiko Odaki
e68459fc5b Redirect from Web tag timeline to public tag timeline if not signed in (#6633)
This is also implemented in Pawoo:
ceafdbd1bb
2018-03-05 19:29:36 +01:00
Akihiko Odaki
a497940b0f Do not default site_title with site_hostname in InstanceHelper (#6624)
site_title is "Mastodon" by default configuration, and there is no need to
default site_title with site_hostname in InstanceHelper.
2018-03-04 20:29:49 +01:00
Akihiko Odaki
008bc9f47d Set the default locale in config (#6580)
Previously the default locale was set by Localized concern for controllers,
but it was not enforced for mailers.

config is enforced throughout the application and an appropriate place to
set the default locale.
2018-03-04 09:21:35 +01:00
Eugen Rochko
b1f8dfd3cf Federate pinned statuses over ActivityPub (#6610)
* Federate pinned statuses over ActivityPub

* Display pinned toots in web UI

Fix #6117

* Fix migration

* Fix tests

* Update outbox_serializer.rb

* Update remove_serializer.rb

* Update add_serializer.rb

* Update fetch_featured_collection_service.rb
2018-03-04 09:19:11 +01:00
Eugen Rochko
359e441fb2 Fix missing focalPoint in ActivityPub JSON (#6609) 2018-03-04 07:21:41 +01:00
Eugen Rochko
0ebd2a1718 Federated reports (#6570)
* Fix #2176: Federated reports

* UI for federated reports

* Add spec for ActivityPub Flag handler

* Add spec for ReportService
2018-02-28 06:54:55 +01:00
Akihiko Odaki
a5a434a8f6 Raise Mastodon::HostValidationError when host for HTTP request is private (#6410) 2018-02-24 19:16:11 +01:00
Akihiko Odaki
7ac6e09dde Do not push status to feed if its reblog is already inserted (#6488)
A complemental change for precompute_feed_service_spec.rb also fixes its
random failure which is caused by the Snowlake randomization of the order
of an original status and its reblog.
2018-02-24 05:40:18 +01:00
Eugen Rochko
ad1f541bd7 Redesign landing page (again) (#6486)
* Redesign landing page (again)

* Move login form in small version to the right column

* Display closed registrations message

* Add site setting for the hero image

* Fix test

* Increase spacing, maximum width, change call to action section
2018-02-22 01:03:48 +01:00
Eugen Rochko
cdb12bd39b Fix bug in relationships API introduced by #6482 (#6527)
It was merge when it needed to be deep_merge. And added some tests
2018-02-21 23:22:12 +01:00
Eugen Rochko
f6884555d7 Account archive download (#6460)
* Fix #201: Account archive download

* Export actor and private key in the archive

* Optimize BackupService

- Add conversation to cached associations of status, because
  somehow it was forgotten and is source of N+1 queries
- Explicitly call GC between batches of records being fetched
  (Model class allocations are the worst offender)
- Stream media files into the tar in 1MB chunks
  (Do not allocate media file (up to 8MB) as string into memory)
- Use #bytesize instead of #size to calculate file size for JSON
  (Fix FileOverflow error)
- Segment media into subfolders by status ID because apparently
  GIF-to-MP4 media are all named "media.mp4" for some reason

* Keep uniquely generated filename in Paperclip::GifTranscoder

* Ensure dumped files do not overwrite each other by maintaing directory partitions

* Give tar archives a good name

* Add scheduler to remove week-old backups

* Fix code style issue
2018-02-21 23:21:32 +01:00
abcang
0923d23789 Improve performance of feed_manager_spec (#6517) 2018-02-20 16:50:12 +01:00
Kazushige Tominaga
77f0e250a4 Added fetch_remote_status_service call spec case actibitypub (#6500)
* Added #link_header spec

* Added #call spec

* Delete spec of private methods

* Added call test case activitypub
2018-02-18 16:34:03 +01:00
Akihiko Odaki
ca6c8b92d9 Isolate each specs for cache store (#6450)
The cache store is explicitly used by some specs, but they were not
isolated and therefore not reliable. This fixes the issue by clearing
the cache after each specs.
2018-02-17 22:35:05 +01:00
Akihiko Odaki
553f6efcb0 Limit the languages used for notification mailer test (#6487)
Some available languages lack translations for notification mails. Now it
tests for two languages which is certain to have required translations:
German and English.

German is the language the current project owner, Eugen Rochko speaks, and
providing English translations for new messages is de facto mandatory.
2018-02-17 14:27:51 +01:00
Eugen Rochko
9b7490cede Save video metadata and improve video OpenGraph tags (#6481)
* Save metadata from video attachments, put correct dimensions into OG tags

* Add twitter:player for videos

* Fix code style and test
2018-02-16 07:22:20 +01:00
Daniel King
845ea13622 Fix URLs incorrectly having trailing hyphen removed (#6465)
In cases where a URL has a trailing hyphen the FetchLinkCardService incorrectly removes the hyphen when it is parsed

The hyphen is not a reserved character in the URI spec https://tools.ietf.org/html/rfc3986#section-2.2
2018-02-11 23:49:18 +01:00
Kazushige Tominaga
74c0abdcf7 Added FetchRemoteAccountService spec (#6456)
* Added #link_header spec

* Added #call spec

* Delete spec of private methods

* Added #call spec
2018-02-10 17:10:57 +01:00
Kazushige Tominaga
ab35601c79 Added #call spec (#6455)
* Added #link_header spec

* Added #call spec

* Delete spec of private methods
2018-02-10 03:31:38 +01:00
Eugen Rochko
7ca5a06505 Full-text search for authorized statuses (#6423)
* Add full-text search for authorized statuses

- Search API will return statuses that match the query
- Only for logged in users
- Only if you are author of the status,
- Or you were mentioned in it
- Or you favourited or reblogged it
- Configuration over `ES_ENABLED`, `ES_HOST`, `ES_PORT`, `ES_PREFIX`
- Run `rails chewy:deploy` to create & populate index

Fix #5880
Fix #4293
Fix #1152

* Add commented out docker-compose configuration for ES container

* Optimize index import, filter search results

* Add basic normalization to the index

* Add better stemming and normalization to the index

* Skip webfinger request if search query includes both @ and a space

* Fix code style

* Visually separate search result sections

* Fix code style issues
2018-02-09 23:04:47 +01:00
Kazushige Tominaga
c208dc4ce7 Perform request spec (#6446)
* Added #link_header spec

* Added #perform_request spec
2018-02-09 08:12:35 +09:00
abcang
1f2aa77758 Fix response of signature_verification_failure_reason (#6441) 2018-02-08 05:00:45 +01:00
Kazushige Tominaga
39eb9eec05 Added #link_header spec (#6439) 2018-02-08 08:17:53 +09:00
Eugen Rochko
5322013f25 CAS + SAML authentication feature (#6425)
* Cas authentication feature

* Config

* Remove class_eval + Omniauth initializer

* Codeclimate review

* Codeclimate review 2

* Codeclimate review 3

* Remove uid/email reconciliation

* SAML authentication

* Clean up code

* Improve login form

* Fix code style issues

* Add locales
2018-02-04 05:42:13 +01:00
abcang
d3760fd25c Fix mistake in cache deletion (#6408) 2018-02-02 10:10:18 +01:00
Akihiko Odaki
b5162e2aff Rename ResolveRemoteAccountService to ResolveAccountService (#6327)
The service used to be named ResolveRemoteAccountService resolves local
accounts as well.
2018-01-22 14:25:09 +01:00
Akihiko Odaki
fabc4af7ff Rename FetchRemoteResourceService to ResolveURLService (#6328)
The service used to be named FetchRemoteResourceService resolves local
URL as well.
2018-01-22 14:24:22 +01:00
Aboobacker MK
a1979b50c9 Redirect to 2FA creation page when otp_secret is not available (#6314) 2018-01-21 13:21:28 +01:00
Eugen Rochko
5f9c017c01 Make text e-mails consistent with HTML ones in UserMailer (#6291)
* Make text e-mails consistent with HTML ones in UserMailer

* Fix UserMailer specs
2018-01-20 01:32:05 +01:00
Eugen Rochko
92df0bba59 Fix regeneration marker not expiring (#6290)
* Fix regeneration key not getting expired

* Add rake task to remove old regeneration markers
2018-01-18 20:29:56 +01:00
Eugen Rochko
b0f2bff481 Replace tutorial modal with welcome e-mail (#6273)
* Remove onboarding modal

* Welcome e-mail

* Send welcome e-mail after confirmation

* Remove obsolete translations
2018-01-18 19:17:25 +01:00
Renato "Lond" Cerqueira
b2632dfad7 When must_be_following_dm is on, only notify if recipient dm'ed user (#6283)
* When must_be_following_dm is on, only notify if recipient dm'ed user
Currently, when must_be_following_dm is on, if a user sends a direct
message replying to any status from the recipient, the recipient gets a
notification. This should not be the case, as if the recipient posted
something publicly this can be used to spam their notifications.

* Refactor replied_to_status_is_direct_message?
Following suggestion in PR
2018-01-18 16:12:10 +01:00
Eugen Rochko
9cfbbbce1f Fix home regeneration (#6251)
* Fix regeneration marker not being removed after completion

* Return HTTP 206 from /api/v1/timelines/home if regeneration in progress
Prioritize RegenerationWorker by putting it into default queue

* Display loading indicator and poll home timeline while it regenerates

* Add graphic to regeneration message

* Make "not found" indicator consistent with home regeneration
2018-01-17 23:56:03 +01:00
Mike Burns
0b8a96e767 Use be_within instead of eq for a to_f test match (#6275)
Floating point values are notoriously hard to pin down, so use the
`be_within` matcher to verify the approximate value.
2018-01-17 12:45:09 +01:00
Eugen Rochko
000b0baac2 HTML e-mails for NotificationMailer (#6263)
* HTML e-mails for NotificationMailer (except digest)

* Add HTML template for digest

* Fix build
2018-01-16 20:20:15 +01:00
Eugen Rochko
9613c3238c HTML e-mails for UserMailer (#6256)
- premailer gem to turn CSS into inline styles automatically
- rework UserMailer templates
- reword UserMailer templates
2018-01-16 03:29:11 +01:00
Eugen Rochko
c8f0be1d6d Revert #5772 (#6221) 2018-01-08 10:57:52 +01:00
Eugen Rochko
d4e0d2563c Fix bad URL schemes being accepted (#6219)
* Fix actors accepting invalid URI schemes or different host between URI and URL

* Fix statuses accepting invalid URI scheme or different host to actor

* Adjust tests to new requirements

* Improve readability of mismatching_origin?/invalid_origin? methods
2018-01-08 05:00:23 +01:00
Yamagishi Kazutoshi
c89bb8ae40 Fix force_ssl conditional (#6201) 2018-01-07 15:19:23 +01:00
Naoki Kosaka
3bc13de62f Fix enforce HTTPS in production. (#6180) 2018-01-05 20:04:22 +01:00
Patrick Figel
bc5487a1c2 Fix email confirmation link not updating email (#6187)
A change introduced in #6125 prevents
`Devise::Models::Confirmable#confirm` from being called for existing
users, which in turn leads to `email` not being set to
`unconfirmed_email`, breaking email updates. This also adds a test
that would've caught this issue.
2018-01-05 00:15:35 +01:00
ThibG
5ee5272b69 Don't normalize URLs in toots (#6134)
* Don't normalize URLs in toots

URL normalization is ill-defined and may cause certain links to break.

* Change specs since we are not normalizing user-provided URLs
2018-01-03 20:51:33 +01:00
Akihiko Odaki
7a0479c830 Allow to dereference Follow object for ActivityPub (#5772)
* Allow to dereference Follow object for ActivityPub

* Accept IRI as object representation for Accept activity
2018-01-03 18:08:57 +01:00
puckipedia
95adc48b92 [!] Sanitize incoming classlist properly (#6162)
* Sanitize classlist properly

* Actually properly sanitize every class after the first

* Improve Formatter spec to check for multiple classes and non-space whitespace
2018-01-03 03:54:08 +01:00
Patrick Figel
3c20cfd734 Add confirmation step for email changes (#6071)
* Add confirmation step for email changes

This adds a confirmation step for email changes of existing users.
Like the initial account confirmation, a confirmation link is sent
to the new address.

Additionally, a notification is sent to the existing address when
the change is initiated. This message includes instruction to reset
the password immediately or to contact the instance admin if the
change was not initiated by the account owner.

Fixes #3871

* Add review fixes
2018-01-02 16:55:00 +01:00
Akihiko Odaki
d8a0c19166 Fix XML oEmbed support discovery (#6104) 2017-12-27 03:29:49 +01:00
Eugen Rochko
7f286d1af0 Fix #5953 - Add GET /api/v1/accounts/:id/lists (#5983) 2017-12-12 03:55:39 +01:00
Akihiko Odaki
109886507e Remove preview_card fabricator (#5975)
preview_card fabricator has a removed attribute, status, and is no longer
functional.
2017-12-11 22:22:08 +09:00
Eugen Rochko
87af0bf6cf Rate limit by user instead of IP when API user is authenticated (#5923)
* Fix #668 - Rate limit by user instead of IP when API user is authenticated

* Fix code style issue

* Use request decorator provided by Doorkeeper
2017-12-09 14:20:02 +01:00
abcang
7fc5cf7a12 Keep WebPush settings (#5879) 2017-12-09 02:31:37 +01:00
aschmitz
cbaddc6447 Break out nested relationship API keys (#5887)
* Break out nested relationship API keys

This closes #5856 by restoring the existing behavior of the `muting`
and `following` keys (returning booleans rather than truthy or false).
It adds `showing_reblogs` and `muting_notifications` keys:

* `showing_reblogs` returns true if:
  1. You've requested to follow the user, with reblogs shown, or
  2. You are following the user, with reblogs shown.
* `muting_notifications` returns true if you have muted the user and
  their notifications as well.

* Rubocop fix

* Fix pulling reblog/mute status from relationships

I could swear this had passed tests before, but apparently not.
Works now.

* More test fixes

Really, you'd expect this to be more straightforward.
2017-12-06 16:10:54 +01:00
Yamagishi Kazutoshi
fba46b6072 Using double splat operator (#5859) 2017-12-06 11:41:57 +01:00
Eugen Rochko
1e7f022fa2 Add list of lists component to web UI (#5811)
* Add list of lists component to web UI

* Add list adding

* Add list removing

* List editor modal

* Add API account search limited by following=true relation

* Rework list editor modal

* Remove mandatory pagination of GET /api/v1/lists/:id/accounts

* Adjust search input placeholder

* Fix rspec (#5890)

* i18n: (zh-CN) Add missing translations for #5811 (#5891)

* i18n: (zh-CN) yarn manage:translations -- zh-CN

* i18n: (zh-CN) Add missing translations for #5811

* Fix some issues

- Display loading/missing state for list timelines
- Order lists alphabetically in overview
- Fix async list editor reset
- Redirect to /lists after deleting unpinned list
- Redirect to / after pinning a list

* Remove dead list columns when a list is deleted or fetch returns 404
2017-12-05 23:02:27 +01:00
Eugen Rochko
cfe67ba832 Add semi-support for Video/Image objects in ActivityPub (#5848)
* Add semi-support for Video/Image objects in ActivityPub

Video and Image objects will create corresponding status records
with manually crafted text contents (title + URL)

* Extract html-url-finding logic into JsonLdHelper

* Fallback to id when url missing, extract supported object types
2017-11-30 04:06:20 +01:00
Eugen Rochko
2421fd745a Fix too many forwards (#5854)
* Avoid sending explicit Undo->Announce when original deleted

* Do not forward a reply back to the server that sent it

* Deduplicate inboxes of rebloggers' followers for delete forwarding

* Adjust test

* Fix wrong class, bad SQL, wrong variable, outdated comment
2017-11-30 03:50:05 +01:00
aschmitz
ce341c7cf6 Allow hiding of reblogs from followed users (#5762)
* Allow hiding of reblogs from followed users

This adds a new entry to the account menu to allow users to hide
future reblogs from a user (and then if they've done that, to show
future reblogs instead).

This does not remove or add historical reblogs from/to the user's
timeline; it only affects new statuses.

The API for this operates by sending a "reblogs" key to the follow
endpoint. If this is sent when starting a new follow, it will be
respected from the beginning of the follow relationship (even if
the follow request must be approved by the followee). If this is
sent when a follow relationship already exists, it will simply
update the existing follow relationship. As with the notification
muting, this will now return an object ({reblogs: [true|false]}) or
false for each follow relationship when requesting relationship
information for an account. This should cause few issues due to an
object being truthy in many languages, but some modifications may
need to be made in pickier languages.

Database changes: adds a show_reblogs column (default true,
non-nullable) to the follows and follow_requests tables. Because
these are non-nullable, we use the existing MigrationHelpers to
perform this change without locking those tables, although the
tables are likely to be small anyway.

Tests included.

See also <https://github.com/glitch-soc/mastodon/pull/212>.

* Rubocop fixes

* Code review changes

* Test fixes

This patchset closes #648 and resolves #3271.

* Rubocop fix

* Revert reblogs defaulting in argument, fix tests

It turns out we needed this for the same reason we needed it in muting:
if nil gets passed in somehow (most usually by an API client not passing
any value), we need to detect and handle it.

We could specify a default in the parameter and then also catch nil, but
there's no great reason to duplicate the default value.
2017-11-28 15:00:35 +01:00
ThibG
f33fd11c87 Fix handling of temporary failures in ProcessMentionsService (#5842)
* Add test for temporary account resolving failures in ProcessMentionsService

* Fix processing of mentions to already-known remote accounts on temporary failures
2017-11-28 15:00:22 +01:00
Eugen Rochko
bc8c951083 Add consumable invites (#5814)
* Add consumable invites

* Add UI for generating invite codes

* Add tests

* Display max uses and expiration in invites table, delete invite

* Remove unused column and redundant validator

- Default follows not used, probably bad idea
- InviteCodeValidator is redundant because RegistrationsController
  checks invite code validity

* Add admin setting to disable invites

* Add admin UI for invites, configurable role for invite creation

- Admin UI that lists everyone's invites, always available
- Admin setting min_invite_role to control who can invite people
- Non-admin invite UI only visible if users are allowed to

* Do not remove invites from database, expire them instantly
2017-11-27 16:07:59 +01:00
ysksn
4dd95c79a5 Add tests for Streamable (#5771) 2017-11-24 14:42:57 +01:00
ysksn
5dd8a36c09 Add tests for AccountRelationshipsPresenter (#5805) 2017-11-24 14:41:04 +01:00
Eugen Rochko
efc97d404c Add logging of admin actions (#5757)
* Add logging of admin actions

* Update brakeman whitelist

* Log creates, updates and destroys with history of changes

* i18n: Update Polish translation (#5782)

Signed-off-by: Marcin Mikołajczak <me@m4sk.in>

* Split admin navigation into moderation and administration

* Redesign audit log page

* 🇵🇱 (#5795)

* Add color coding to audit log

* Change dismiss->resolve, log all outcomes of report as resolve

* Update terminology (e-mail blacklist) (#5796)

* Update terminology (e-mail blacklist)

imho looks better

* Update en.yml

* Fix code style issues

* i18n-tasks normalize
2017-11-24 02:05:53 +01:00
ysksn
4086db0929 Add tests for Remotable (#5768) 2017-11-21 13:59:36 +09:00
abcang
defb2fd0b0 Fix N+1 at notification (#5752) 2017-11-19 15:32:48 +01:00
ysksn
42cb1425c6 Add tests for AccountInteractions (#5751)
* Add tests for class methods of AccountInteractions

* Add tests for instance methods of AccountInteractions
2017-11-19 04:16:14 +01:00
ysksn
a7d85563bd Add a test for Settings::Extend#settings (#5720) 2017-11-19 04:15:32 +01:00
ysksn
c09c37af98 Add tests for Status#title (#5718) 2017-11-19 04:15:17 +01:00
Eugen Rochko
e149067561 Lists (#5703)
* Add structure for lists

* Add list timeline streaming API

* Add list APIs, bind list-account relation to follow relation

* Add API for adding/removing accounts from lists

* Add pagination to lists API

* Add pagination to list accounts API

* Adjust scopes for new APIs

- Creating and modifying lists merely requires "write" scope
- Fetching information about lists merely requires "read" scope

* Add test for wrong user context on list timeline

* Clean up tests
2017-11-18 00:16:48 +01:00
ysksn
08cb63c360 Remove empty strings (#5732) 2017-11-17 10:52:30 +09:00
ysksn
a766ece166 Add tests for Status#verb (#5717) 2017-11-16 15:46:29 +09:00
ysksn
ab48f2b239 Add tests for Status#hidden? (#5719) 2017-11-16 11:07:27 +09:00
ysksn
475b7cecb5 Delegate some methods of User to @settings (#5706)
* Move some tests of User into Settings::ScopedSettings

* Add a test for User@settings
2017-11-15 16:05:20 +01:00
ysksn
2e5aec4479 Add a test for Tag#to_param (#5705) 2017-11-15 16:04:41 +01:00
Surinna Curtis
582dd1b7c7 Optional notification muting (#5087)
* Add a hide_notifications column to mutes

* Add muting_notifications? and a notifications argument to mute!

* block notifications in notify_service from hard muted accounts

* Add specs for how mute! interacts with muting_notifications?

* specs testing that hide_notifications in mutes actually hides notifications

* Add support for muting notifications in MuteService

* API support for muting notifications (and specs)

* Less gross passing of notifications flag

* Break out a separate mute modal with a hide-notifications checkbox.

* Convert profile header mute to use mute modal

* Satisfy eslint.

* specs for MuteService notifications params

* add trailing newlines to files for Pork :)

* Put the label for the hide notifications checkbox in a label element.

* Add a /api/v1/mutes/details route that just returns the array of mutes.

* Define a serializer for /api/v1/mutes/details

* Add more specs for the /api/v1/mutes/details endpoint

* Expose whether a mute hides notifications in the api/v1/relationships endpoint

* Show whether muted users' notifications are muted in account lists

* Allow modifying the hide_notifications of a mute with the /api/v1/accounts/:id/mute endpoint

* make the hide/unhide notifications buttons work

* satisfy eslint

* In probably dead code, replace a dispatch of muteAccount that was skipping the modal with launching the mute modal.

* fix a missing import

* add an explanatory comment to AccountInteractions

* Refactor handling of default params for muting to make code cleaner

* minor code style fixes oops

* Fixed a typo that was breaking the account mute API endpoint

* Apply white-space: nowrap to account relationships icons

* Fix code style issues

* Remove superfluous blank line

* Rename /api/v1/mutes/details -> /api/v2/mutes

* Don't serialize "account" in MuteSerializer

Doing so is somewhat unnecessary since it's always the current user's account.

* Fix wrong variable name in api/v2/mutes

* Use Toggle in place of checkbox in the mute modal.

* Make the Toggle in the mute modal look better

* Code style changes in specs and removed an extra space

* Code review suggestions from akihikodaki

Also fixed a syntax error in tests for AccountInteractions.

* Make AddHideNotificationsToMute Concurrent

It's not clear how much this will benefit instances in practice, as the
number of mutes tends to be pretty small, but this should prevent any
blocking migrations nonetheless.

* Fix up migration things

* Remove /api/v2/mutes
2017-11-15 03:56:41 +01:00
ysksn
bde85899a5 Add tests for StreamEntry (#5687)
* Add tests for StreamEntry

- `#object_type`
- `#verb`
- `#mentions`

* Fix to test results instead of implementations
2017-11-15 02:00:58 +01:00
Eugen Rochko
80873c1134 Add option to block direct messages from people you don't follow (#5669)
* Add option to block direct messages from people you don't follow

Fix #5326

* If the DM responds to a toot by recipient, allow it through

* i18n: Update Polish translation (for #5669) (#5673)
2017-11-14 21:12:57 +01:00
ysksn
63714ad94a Add a test for SiteUpload#cache_key (#5685) 2017-11-14 20:44:11 +01:00
Yamagishi Kazutoshi
1c83ce9b81 Refactor remote_follow_spec.rb (#5690) 2017-11-14 20:41:17 +01:00
Yamagishi Kazutoshi
3a51544e9a Add uniqueness to block email domains (#5692) 2017-11-14 20:37:17 +01:00
ysksn
9b7f9f3527 Add tests for Setting (#5683) 2017-11-14 11:08:04 +09:00
ysksn
b390208bfa Add tests for SessionActivation (#5668)
* Fabricate SessionActivation

not only user_id but user association.

* Add tests for SessionActivation
2017-11-13 09:54:48 +09:00
ysksn
56d569aa21 Add tests for RemoteProfile (#5665) 2017-11-12 16:23:31 +09:00
ysksn
9f78fb2132 Add tests for RemoteFollow (#5651)
* Add tests for RemoteFollow.initialize

* Add tests for RemoteFollow#valid?

* Add tests for RemoteFollow#subscribe_address_for
2017-11-10 16:56:02 +01:00
ysksn
471c7f61d9 Add tests for Notification (#5640)
* Add tests for Notification#target_status

* Add tests for Notification#browserable?

* Add tests for Notification.reload_stale_associations!
2017-11-09 14:37:10 +01:00
ysksn
26f52a2551 Add and Remove tests for FollowRequest (#5622)
* Add a test for FollowRequest#authorize!

* Remove tests

There is no need to test
ActiveModel::Validations::ClassMethods#validates.

* Make an alias of destroy! as reject!

Instead of defining the method,
make an alias of destroy! as reject! because of reducing test.
2017-11-09 14:36:52 +01:00
Nanamachi
e1ee50976c Rewrite account_controller_spec (#5633)
* make accounts_controller_spec DRY

* Add blocked user spec
2017-11-09 14:36:17 +01:00
ysksn
321a4d35c6 Add tests for MediaAttachment (#5620)
- `#local?`
- `#needs_redownload?`
- `#to_param`
2017-11-08 15:29:07 +09:00
ysksn
a69f061c30 Add tests for CustomEmoji#local? and #object_type (#5621) 2017-11-08 15:28:17 +09:00
ysksn
0e22b993e6 Implement tests for Account#refresh! (#5601) 2017-11-06 13:54:41 +09:00
ysksn
ffab17400b Add test for Account#save_with_optional_media! (#5603)
There was a test when some of the properties are invalid, but none when all
of them are valid.
2017-11-06 13:54:12 +09:00
ysksn
515afedcd0 Implement tests for Account#possibly_stale? (#5591) 2017-11-05 17:20:05 +09:00
unarist
4472c5723a Fix Cocaine::ExitStatusError when upload small non-animated GIF (#5489)
Looks like copied tempfile need to be flushed before further processing. This issue won't happen if the uploaded file has enough file size.
2017-10-26 22:48:35 +09:00
aschmitz
4dbdda6b89 Clean up reblog tracking keys, related improvements (#5428)
* Clean up reblog-tracking sets from FeedManager

Builds on #5419, with a few minor optimizations and cleanup of sets
after they are no longer needed.

* Update tests, fix multiply-reblogged case

Previously, we would have lost the fact that a given status was
reblogged if the displayed reblog of it was removed, now we don't.

Also added tests to make sure FeedManager#trim cleans up our reblog
tracking keys, fixed up FeedCleanupScheduler to use the right loop,
and fixed the test for it.
2017-10-17 11:45:06 +02:00
Eugen Rochko
7571d79c8d Keep references to all reblogs of a status on home feed (#5419)
* Keep references to all reblogs of a status on home feed

When inserting reblog: Add to set of reblogs of this status on
the feed, if original status was present in the feed, add it to
that set as well.

When removing a reblog: Remove it from that set. Take random
remaining item from the set. If one exists, re-insert it into feed,
otherwise do not re-insert anything.

Fix #4210

* When original is removed, toss out reblog references
2017-10-16 20:44:31 +02:00
unarist
1fa6d20981 Fix un-reblogged status being at wrong position in the home timeline (#5418)
We've changed un-reblogging behavior when we implement Snowflake, to insert un-reblogged status at the position reblogging status existed.

However, our API expects home timeline is ordered by status ids, and max_id/since_id filters by zset score. Due to this, un-reblogged status appears as a last item of result set, and timeline expansion may skips many statuses.

So this reverts that change...reblogged status inserted at corresponding position to its id.
2017-10-16 15:58:23 +02:00
Yamagishi Kazutoshi
fb5040885c Replace JavaScript Testing Framework from Mocha to Jest (#5412) 2017-10-16 09:33:08 +02:00
unarist
69b74d26cd Use atomUri in Undo activity of Announce (#5376)
This allows deletion of reblogs which delivered before with OStatus URI.
2017-10-14 14:42:09 +02:00
Eugen Rochko
3378a3af96 Improve spec of Feed and UserTrackingConcern (#5367) 2017-10-13 11:00:11 +02:00
Lex Alexander
526b66e871 Retoot count increases without reason (#5363)
* Retoot count increases without reason

-The store_uri method for Statuses was being called on after_create and causing reblogs to be incremented twice.
-This calls it when the transaction is finished by using after_create_commit.
-Fixes #4916.

* Added test case for after_create_commit callback for checking reblog count.

* Rewrote test to keep original, but added one for only the after_create_commit callback.
2017-10-13 02:52:09 +02:00
Akihiko Odaki
783422f8f5 Fix pagination in Api::V1::BlocksController (#5285) 2017-10-09 17:30:31 +02:00
Eugen Rochko
007fce8c10 Set snowflake IDs for backdated statuses (#5260)
- Rename Mastodon::TimestampIds into Mastodon::Snowflake for clarity
- Skip for statuses coming from inbox, aka delivered in real-time
- Skip for statuses that claim to be from the future
2017-10-08 17:34:34 +02:00
nullkal
81489970fb Add moderation note (#5240)
* Add moderation note

* Add frozen_string_literal

* Make rspec pass
2017-10-07 20:26:43 +02:00
Eugen Rochko
b98cd0041b Redesign public hashtag pages (#5237) 2017-10-07 20:00:35 +02:00
Eugen Rochko
60925ce0ae Encode custom emojis as resolveable objects in ActivityPub (#5243)
* Encode custom emojis as resolveable objects in ActivityPub

* Improve code style
2017-10-07 17:43:42 +02:00
Nolan Lawson
14db4829dc Update emoji-mart to v2.1.1 (#5256) 2017-10-07 12:17:02 +02:00
Nolan Lawson
651aba0e12 Fix thinking_face emoji autocomplete (#5238) 2017-10-06 12:03:13 +02:00
Nolan Lawson
2b34471d9d Compress and combine emoji data (#5229) 2017-10-06 03:42:34 +02:00
ThibG
a89d69a269 Fix regression in FetchRemoteResourceService (#5217)
* Fix regression in FetchRemoteResourceService

* Update specs to match interface changes made in #5114
2017-10-05 00:21:44 +02:00
utam0k
51f8cbfbeb Implement EmailBlackList (#5109)
* Implement BlacklistedEmailDomain

* Use Faker::Internet.domain_name

* Remove note column

* Add frozen_string_literal comment

* Delete unnecessary codes

* Sort alphabetically

* Change of wording

* Rename BlacklistedEmailDomain to EmailDomainBlock
2017-10-04 15:16:10 +02:00
Yamagishi Kazutoshi
6bd4e3bf97 Separate notifications preferences from general preferences (#4447)
* Separate notifications preferences from general preferences

* Refine settings/notifications/show

* remove preferences.notifications
2017-10-04 10:22:52 +02:00
aschmitz
5b2d855d86 Non-Serial ("Snowflake") IDs (#4801)
* Use non-serial IDs

This change makes a number of nontrivial tweaks to the data model in
Mastodon:

* All IDs are now 8 byte integers (rather than mixed 4- and 8-byte)
* IDs are now assigned as:
  * Top 6 bytes: millisecond-resolution time from epoch
  * Bottom 2 bytes: serial (within the millisecond) sequence number
  * See /lib/tasks/db.rake's `define_timestamp_id` for details, but
    note that the purpose of these changes is to make it difficult to
    determine the number of objects in a table from the ID of any
    object.
* The Redis sorted set used for the feed will have values used to look
  up toots, rather than scores. This is almost always the same as the
  existing behavior, except in the case of boosted toots. This change
  was made because Redis stores scores as double-precision floats,
  which cannot store the new ID format exactly. Note that this doesn't
  cause problems with sorting/pagination, because ZREVRANGEBYSCORE
  sorts lexicographically when scores are tied. (This will still cause
  sorting issues when the ID gains a new significant digit, but that's
  extraordinarily uncommon.)

Note a couple of tradeoffs have been made in this commit:

* lib/tasks/db.rake is used to enforce many/most column constraints,
  because this commit seems likely to take a while to bring upstream.
  Enforcing a post-migrate hook is an easier way to maintain the code
  in the interim.
* Boosted toots will appear in the timeline as many times as they have
  been boosted. This is a tradeoff due to the way the feed is saved in
  Redis at the moment, but will be handled by a future commit.

This would effectively close Mastodon's #1059, as it is a
snowflake-like system of generating IDs. However, given how involved
the changes were simply within Mastodon, it may have unexpected
interactions with some clients, if they store IDs as doubles
(or as 4-byte integers). This was a problem that Twitter ran into with
their "snowflake" transition, particularly in JavaScript clients that
treated IDs as JS integers, rather than strings. It therefore would be
useful to test these changes at least in the web interface and popular
clients before pushing them to all users.

* Fix JavaScript interface with long IDs

Somewhat predictably, the JS interface handled IDs as numbers, which in
JS are IEEE double-precision floats. This loses some precision when
working with numbers as large as those generated by the new ID scheme,
so we instead handle them here as strings. This is relatively simple,
and doesn't appear to have caused any problems, but should definitely
be tested more thoroughly than the built-in tests. Several days of use
appear to support this working properly.

BREAKING CHANGE:

The major(!) change here is that IDs are now returned as strings by the
REST endpoints, rather than as integers. In practice, relatively few
changes were required to make the existing JS UI work with this change,
but it will likely hit API clients pretty hard: it's an entirely
different type to consume. (The one API client I tested, Tusky, handles
this with no problems, however.)

Twitter ran into this issue when introducing Snowflake IDs, and decided
to instead introduce an `id_str` field in JSON responses. I have opted
to *not* do that, and instead force all IDs to 64-bit integers
represented by strings in one go. (I believe Twitter exacerbated their
problem by rolling out the changes three times: once for statuses, once
for DMs, and once for user IDs, as well as by leaving an integer ID
value in JSON. As they said, "If you’re using the `id` field with JSON
in a Javascript-related language, there is a very high likelihood that
the integers will be silently munged by Javascript interpreters. In most
cases, this will result in behavior such as being unable to load or
delete a specific direct message, because the ID you're sending to the
API is different than the actual identifier associated with the
message." [1]) However, given that this is a significant change for API
users, alternatives or a transition time may be appropriate.

1: https://blog.twitter.com/developer/en_us/a/2011/direct-messages-going-snowflake-on-sep-30-2011.html

* Restructure feed pushes/unpushes

This was necessary because the previous behavior used Redis zset scores
to identify statuses, but those are IEEE double-precision floats, so we
can't actually use them to identify all 64-bit IDs. However, it leaves
the code in a much better state for refactoring reblog handling /
coalescing.

Feed-management code has been consolidated in FeedManager, including:

* BatchedRemoveStatusService no longer directly manipulates feed zsets
* RemoveStatusService no longer directly manipulates feed zsets
* PrecomputeFeedService has moved its logic to FeedManager#populate_feed

(PrecomputeFeedService largely made lots of calls to FeedManager, but
didn't follow the normal adding-to-feed process.)

This has the effect of unifying all of the feed push/unpush logic in
FeedManager, making it much more tractable to update it in the future.

Due to some additional checks that must be made during, for example,
batch status removals, some Redis pipelining has been removed. It does
not appear that this should cause significantly increased load, but if
necessary, some optimizations are possible in batch cases. These were
omitted in the pursuit of simplicity, but a batch_push and batch_unpush
would be possible in the future.

Tests were added to verify that pushes happen under expected conditions,
and to verify reblog behavior (both on pushing and unpushing). In the
case of unpushing, this includes testing behavior that currently leads
to confusion such as Mastodon's #2817, but this codifies that the
behavior is currently expected.

* Rubocop fixes

I could swear I made these changes already, but I must have lost them
somewhere along the line.

* Address review comments

This addresses the first two comments from review of this feature:

https://github.com/tootsuite/mastodon/pull/4801#discussion_r139336735
https://github.com/tootsuite/mastodon/pull/4801#discussion_r139336931

This adds an optional argument to FeedManager#key, the subtype of feed
key to generate. It also tests to ensure that FeedManager's settings are
such that reblogs won't be tracked forever.

* Hardcode IdToBigints migration columns

This addresses a comment during review:
https://github.com/tootsuite/mastodon/pull/4801#discussion_r139337452

This means we'll need to make sure that all _id columns going forward
are bigints, but that should happen automatically in most cases.

* Additional fixes for stringified IDs in JSON

These should be the last two. These were identified using eslint to try
to identify any plain casts to JavaScript numbers. (Some such casts are
legitimate, but these were not.)

Adding the following to .eslintrc.yml will identify casts to numbers:

~~~
  no-restricted-syntax:
  - warn
  - selector: UnaryExpression[operator='+'] > :not(Literal)
    message: Avoid the use of unary +
  - selector: CallExpression[callee.name='Number']
    message: Casting with Number() may coerce string IDs to numbers
~~~

The remaining three casts appear legitimate: two casts to array indices,
one in a server to turn an environment variable into a number.

* Only implement timestamp IDs for Status IDs

Per discussion in #4801, this is only being merged in for Status IDs at
this point. We do this in a migration, as there is no longer use for
a post-migration hook. We keep the initialization of the timestamp_id
function as a Rake task, as it is also needed after db:schema:load (as
db/schema.rb doesn't store Postgres functions).

* Change internal streaming payloads to stringified IDs as well

This is equivalent to 591a9af356faf2d5c7e66e3ec715502796c875cd from
#5019, with an extra change for the addition to FeedManager#unpush.

* Ensure we have a status_id_seq sequence

Apparently this is not a given when specifying a custom ID function,
so now we ensure it gets created. This uses the generic version of this
function to more easily support adding additional tables with timestamp
IDs in the future, although it would be possible to cut this down to a
less generic version if necessary. It is only run during db:schema:load
or the relevant migration, so the overhead is extraordinarily minimal.

* Transition reblogs to new Redis format

This provides a one-way migration to transition old Redis reblog entries
into the new format, with a separate tracking entry for reblogs.

It is not invertible because doing so could (if timestamp IDs are used)
require a database query for each status in each users' feed, which is
likely to be a significant toll on major instances.

* Address review comments from @akihikodaki

No functional changes.

* Additional review changes

* Heredoc cleanup

* Run db:schema:load hooks for test in development

This matches the behavior in Rails'
ActiveRecord::Tasks::DatabaseTasks.each_current_configuration, which
would otherwise break `rake db:setup` in development.

It also moves some functionality out to a library, which will be a good
place to put additional related functionality in the near future.
2017-10-04 09:56:37 +02:00
Akihiko Odaki
2e6db37776 Validate id of ActivityPub representations (#5114)
Additionally, ActivityPub::FetchRemoteStatusService no longer parses
activities.
OStatus::Activity::Creation no longer delegates to ActivityPub because
the provided ActivityPub representations are not signed while OStatus
representations are.
2017-10-04 01:13:48 +02:00
ThibG
707cd936e8 Fix HTTP responses for salmon and ActivityPub inbox processing (#5200)
* Return sensible HTTP status for ActivityPub inbox processing

* Return sensible HTTP status for salmon slap processing

* Return additional information to debug signature verification failures
2017-10-03 23:21:19 +02:00
Nolan Lawson
bfea083fbf Add spec for emoji_index_light.js (#5199) 2017-10-03 20:54:38 +02:00
Nolan Lawson
05842e15db Fix emoji sequence bug in substring-trie (#5191)
Fixes #5188
2017-10-03 13:11:22 +02:00
Eugen Rochko
0783f18274 Make Chrome splash screen same color as web UI's background color (#5169) 2017-10-02 01:23:32 +02:00
Akihiko Odaki
51f48c8960 Fix order of paginated accounts in FollowerDomainsController and spec (#3357)
* Fix order of paginated accounts in FollowerDomainsController

Unordered pagination could result in unexpected behavior.

* Cover Settings::FollowerDomainsController more
2017-09-30 23:06:09 +02:00
Eugen Rochko
5983fa43f5 Fix #5104 - GET /api/v1/apps/verify_credentials to confirm app works (#5112) 2017-09-30 22:05:42 +02:00
Eugen Rochko
2fac5e21c2 Upgrade to React 16 (#5119)
* Upgrade to React 16.0.0

* Disable some uncritical tests while chai-enzyme remains incompatible
2017-09-30 04:29:56 +02:00
Eugen Rochko
cb8ecac1ce After 7 days of repeated delivery failures, give up on inbox (#5131)
- A successful delivery cancels it out
- An incoming delivery from account of the inbox cancels it out
2017-09-29 03:16:20 +02:00
Eugen Rochko
5f137e7ec3 Increase attachment descriptions to 420 characters (#5139)
Blaze it
2017-09-29 02:30:00 +02:00
Eugen Rochko
795e624d5d Add ability to specify alternative text for media attachments (#5123)
* Fix #117 - Add ability to specify alternative text for media attachments

- POST /api/v1/media accepts `description` straight away
- PUT /api/v1/media/:id to update `description` (only for unattached ones)
- Serialized as `name` of Document object in ActivityPub
- Uploads form adjusted for better performance and description input

* Add tests

* Change undo button blend mode to difference
2017-09-28 15:31:31 +02:00
Eugen Rochko
10a2b3dd4c Follow-up to #4582 and #5027, removing dead code (#5101) 2017-09-26 01:06:27 +02:00
Eugen Rochko
1efe904a28 Fix #5059 - Stop processing payload if it's from local account (#5100) 2017-09-26 01:06:13 +02:00
Eugen Rochko
8b7c639bbe Fix emojify() by generating a mapping to existing Twemoji files (#5080)
A new rake task emojis:generate downloads a full list of valid
unicode sequences from unicode.org and checks it against existing
Twemoji files, finally generating a map from each sequence to the
existing file (e.g. when there's multiple ways an emoji can be
expressed). The map is dumped into app/javascript/mastodon/emoji_map.json

That file is loaded by emojione_light.js (now a misnomer) which
decorates it further with shortcodes taken from emoji-mart's index.
2017-09-25 18:36:33 +02:00
Akihiko Odaki
77424c156d Add missing validations in ActivityPub::Activity::Create (#5096) 2017-09-25 18:33:11 +02:00
Eugen Rochko
da257754b5 Add emoji autosuggest (#5053)
* Add emoji autosuggest

Some credit goes to glitch-soc/mastodon#149

* Remove server-side shortcode->unicode conversion

* Insert shortcode when suggestion is custom emoji

* Remove remnant of server-side emojis

* Update style of autosuggestions

* Fix wrong emoji filenames generated in autosuggest item

* Do not lazy load emoji picker, as that no longer works

* Fix custom emoji autosuggest

* Fix multiple "Custom" categories getting added to emoji index, only add once
2017-09-23 14:47:32 +02:00
Eugen Rochko
f621e05356 New API: GET /api/v1/custom_emojis to get a server's custom emojis (#5051) 2017-09-23 01:57:23 +02:00
Eugen Rochko
16a5d1450c Switch from EmojiOne to Twemoji, different emoji picker (#5046)
* Switch from EmojiOne to Twemoji, different emoji picker

* Make emoji-mart use a local spritesheet

* Fix emojify test

* yarn manage:translations
2017-09-23 01:41:00 +02:00
Akihiko Odaki
f9dd41621f Do not filter statuses with unknown languages (#5045) 2017-09-22 19:33:17 +02:00
Eugen Rochko
f7810f56a1 Make dropdowns render into portal, expand animation (#5018)
* Make dropdowns render into portal, expand animation

* Improve actions modal style
2017-09-22 04:59:17 +02:00
aschmitz
618df7a5ee Change IDs to strings rather than numbers in API JSON output (#5019)
* Fix JavaScript interface with long IDs

Somewhat predictably, the JS interface handled IDs as numbers, which in
JS are IEEE double-precision floats. This loses some precision when
working with numbers as large as those generated by the new ID scheme,
so we instead handle them here as strings. This is relatively simple,
and doesn't appear to have caused any problems, but should definitely
be tested more thoroughly than the built-in tests. Several days of use
appear to support this working properly.

BREAKING CHANGE:

The major(!) change here is that IDs are now returned as strings by the
REST endpoints, rather than as integers. In practice, relatively few
changes were required to make the existing JS UI work with this change,
but it will likely hit API clients pretty hard: it's an entirely
different type to consume. (The one API client I tested, Tusky, handles
this with no problems, however.)

Twitter ran into this issue when introducing Snowflake IDs, and decided
to instead introduce an `id_str` field in JSON responses. I have opted
to *not* do that, and instead force all IDs to 64-bit integers
represented by strings in one go. (I believe Twitter exacerbated their
problem by rolling out the changes three times: once for statuses, once
for DMs, and once for user IDs, as well as by leaving an integer ID
value in JSON. As they said, "If you’re using the `id` field with JSON
in a Javascript-related language, there is a very high likelihood that
the integers will be silently munged by Javascript interpreters. In most
cases, this will result in behavior such as being unable to load or
delete a specific direct message, because the ID you're sending to the
API is different than the actual identifier associated with the
message." [1]) However, given that this is a significant change for API
users, alternatives or a transition time may be appropriate.

1: https://blog.twitter.com/developer/en_us/a/2011/direct-messages-going-snowflake-on-sep-30-2011.html

* Additional fixes for stringified IDs in JSON

These should be the last two. These were identified using eslint to try
to identify any plain casts to JavaScript numbers. (Some such casts are
legitimate, but these were not.)

Adding the following to .eslintrc.yml will identify casts to numbers:

~~~
  no-restricted-syntax:
  - warn
  - selector: UnaryExpression[operator='+'] > :not(Literal)
    message: Avoid the use of unary +
  - selector: CallExpression[callee.name='Number']
    message: Casting with Number() may coerce string IDs to numbers
~~~

The remaining three casts appear legitimate: two casts to array indices,
one in a server to turn an environment variable into a number.

* Back out RelationshipsController Change

This was made to make a test a bit less flakey, but has nothing to
do with this branch.

* Change internal streaming payloads to stringified IDs as well

Per
https://github.com/tootsuite/mastodon/pull/5019#issuecomment-330736452
we need these changes to send deleted status IDs as strings, not
integers.
2017-09-20 14:53:48 +02:00
Akihiko Odaki
b1ba673029 Introduce OStatus::TagManager (#5008) 2017-09-19 18:08:08 +02:00
Eugen Rochko
eb345f8e8b Custom emoji (#4988)
* Custom emoji

- In OStatus: `<link rel="emoji" name="coolcat" href="http://..." />`
- In ActivityPub: `{ type: "Emoji", name: ":coolcat:", href: "http://..." }`
- In REST API: Status object includes `emojis` array (`shortcode`, `url`)
- Domain blocks with reject media stop emojis
- Emoji file up to 50KB
- Web UI handles custom emojis
- Static pages render custom emojis as `<img />` tags

Side effects:

- Undo #4500 optimization, as I needed to modify it to restore
  shortcode handling in emojify()
- Formatter#plaintext should now make sure stripped out line-breaks
  and paragraphs are replaced with newlines

* Fix emoji at the start not being converted
2017-09-19 02:42:40 +02:00
unarist
3978894d05 Validate uri presence for remote status (#4985) 2017-09-17 15:21:57 +02:00
unarist
4a4e555fd4 Fix an error when actor json couldn't be fetched in ResolveRemoteAccountService (#4979)
* Fix an error when actor json couldn't be fetched in ResolveRemoteAccountService

* Add specs
2017-09-17 11:54:23 +02:00
Akihiko Odaki
8c96e0d496 Remove local_only scope in Status (#4977) 2017-09-17 05:29:43 +02:00
unarist
134feb2ca8 Escape URL parts on formatting local status (#4975) 2017-09-16 21:33:52 +02:00
Akihiko Odaki
d4a1ddd46a Fix filterable_languages method of SettingsHelper (#4966) 2017-09-16 14:59:41 +02:00
Eugen Rochko
2d1048d041 When web UI URL used while logged out, redirect to static page (#4954) 2017-09-15 00:57:08 +02:00
ふぁぼ原
c71727ca55 Enable to recognize most kinds of characters as URL paths (#4941) 2017-09-14 18:03:20 +02:00
Eugen Rochko
9ff429aa6d Uploads for admin site settings (#4913)
* Improve OpenGraph tags for about pages

* Add thumbnail admin setting

* Fix error

* Fix up
2017-09-14 00:04:30 +02:00
ThibG
53a5e16c5f Fix ActivityPub handling of replies with WEB_DOMAIN (#4895) (#4904)
* Fix ActivityPub handling of replies when LOCAL_DOMAIN ≠ WEB_DOMAIN (#4895)

For all intents and purposes, `local_url?` is used to check if an URL refers
to the Web UI or the various API endpoints of the local instances. Those things
reside on `WEB_DOMAIN` and not `LOCAL_DOMAIN`.

* Change local_url? spec, as all URLs handled by Mastodon are based on WEB_DOMAIN
2017-09-13 14:22:16 +02:00
Eugen Rochko
d4fd2a39e6 Clean up and improve generated OpenGraph tags (#4901)
- Return all images as og:image
- Return videos as og:image (preview) and og:video
- Return profile:username on profiles
2017-09-12 05:39:38 +02:00
Eugen Rochko
e054961051 Fix error when following locked accounts (#4896) 2017-09-11 23:50:37 +02:00
Eugen Rochko
078c84fbb5 Fix POST /api/v1/follows error when already following (#4878) 2017-09-10 15:09:06 +02:00
Eugen Rochko
6628ea4a82 Default follows for new users (#4871)
When a new user confirms their e-mail, bootstrap their home timeline
by automatically following a set of accounts. By default, all local
admin accounts (that are unlocked). Can be customized by new admin
setting (comma-separated usernames, local and unlocked only)
2017-09-10 09:58:38 +02:00
Eugen Rochko
86cf6905cc Fix errors preventing UnsubscribeService from working (#4866) 2017-09-09 17:36:27 +02:00
unarist
05ef0ead4a Handle stream_entry URL correctly in ActivityPub (#4854)
In before, the method uses stream_entry id as status id, so replied status was wrongly selected.

This PR uses StatusFinder which was introduced with `Api::Web::EmbedsController`.
2017-09-08 18:20:03 +02:00
Eugen Rochko
a27821f725 Fix language filter codes (#4841)
* Fix language filter codes

CLD3 returns BCP-47 language identifier, filter settings expect
identifiers in the ISO 639-1 format. Convert between formats,
and exclude duplicate languages from filter choices (zh-CN->zh)

* Fix zh name
2017-09-08 12:32:22 +02:00
Eugen Rochko
6fb8be482b Fetch statuses/following/followers numbers from ActivityPub collections (#4840) 2017-09-08 12:00:17 +02:00
voidSatisfaction
ac63da7732 Feat add validation for report comment: characters under 1000 valid (#4833) 2017-09-07 09:55:42 +02:00
Eugen Rochko
73c613dcfc Switch to static URIs, new URI format in both protocols for new statuses (#4815)
* Decouple Status#local? from uri being nil

* Replace on-the-fly URI generation with stored URIs

- Generate URI in after_save hook for local statuses
- Use static value in TagManager when available, fallback to tag format
- Make TagManager use ActivityPub::TagManager to understand new format
- Adjust tests

* Use other heuristic for locality of old statuses, do not perform long query

* Exclude tombstone stream entries from Atom feed

* Prevent nil statuses from landing in Pubsubhubbub::DistributionWorker

* Fix URI not being saved (#4818)

* Add more specs for Status

* Save generated uri immediately

and also fix method order to minimize diff.

* Fix alternate HTML URL in Atom

* Fix tests

* Remove not-null constraint from statuses migration to speed it up
2017-09-06 19:01:28 +02:00
Eugen Rochko
181cbbcaf8 Fix #4794 - Fake instant follow in API response when account is believed unlocked (#4799) 2017-09-05 17:48:13 +02:00
nullkal
36872cd22c Show pinned statuses only in the top of the profile page (#4803)
* Show pinned statuses only in the top of the profile page

* Refactor AccountsController#show_pinned_statuses?
2017-09-04 12:53:18 +02:00
abcang
d73174b629 Validate data of Imports (#4782) 2017-09-02 20:45:42 +02:00
unarist
34603e0154 Raise an error for remote url in StatusFinder (#4776)
* Raise an error for remote url in StatusFinder

Previous implementation had allowed remote url with status id which also exists on local.

Then that bug leads /api/web/embed to return wrong embed url.

* Fix oembed_controller_spec
2017-09-02 17:42:47 +02:00
Eugen Rochko
e9e271878e Make PreviewCard records reuseable between statuses (#4642)
* Make PreviewCard records reuseable between statuses

**Warning!** Migration truncates preview_cards tablec

* Allow a wider thumbnail for link preview, display it in horizontal layout (#4648)

* Delete preview cards files before truncating

* Rename old table instead of truncating it

* Add mastodon:maintenance:remove_deprecated_preview_cards

* Ignore deprecated_preview_cards in schema definition

* Fix null behaviour
2017-09-01 16:20:16 +02:00
Eugen Rochko
02bd661d76 Remove identity context from output of LinkedDataSignature (#4753) 2017-08-31 21:32:09 +02:00
unarist
5a6b15f014 Don't process ActivityPub payload if signature is invalid (#4752)
* Don't process ActivityPub payload if signature is invalid

* Fix style issue
2017-08-31 17:18:49 +02:00
Eugen Rochko
ca76d11434 Forward ActivityPub creates that reply to local statuses (#4709)
* Forward ActivityPub creates that reply to local statuses

* Fix test

* Fix wrong signers
2017-08-30 15:37:02 +02:00
Eugen Rochko
651c659fd6 Update status embeds (#4742)
- Use statuses controller for embeds instead of stream entries controller
- Prefer /@:username/:id/embed URL for embeds
- Use /@:username as author_url in OEmbed
- Add follow link to embeds which opens web intent in new window
- Use redis cache in development
- Cache entire embed
2017-08-30 10:23:43 +02:00
Eugen Rochko
3135d20283 Serialize ActivityPub alternate link into OStatus deletes, handle it (#4730)
Requires moving Atom rendering from DistributionWorker (where
`stream_entry.status` is already nil) to inline (where
`stream_entry.status.destroyed?` is true) and distributing that.

Unfortunately, such XML renderings can no longer be easily chained
together into one payload of n items.
2017-08-29 16:11:05 +02:00
unarist
9edbeec86c Fix Delete activity handling when the status has been reblogged (#4729) 2017-08-28 22:08:11 +02:00
unarist
e17945907a Fix deletion of status which has been reblogged (#4728) 2017-08-28 21:38:59 +02:00
abcang
4fb007764b rescue HTTP::ConnectionError in RemoteFollowController#create (#4726) 2017-08-28 19:12:09 +02:00
Eugen Rochko
5147147da9 Add handling of Linked Data Signatures in payloads (#4687)
* Add handling of Linked Data Signatures in payloads

* Add a way to sign JSON, fix canonicalization of signature options

* Fix signatureValue encoding, send out signed JSON when distributing

* Add missing security context
2017-08-26 13:47:38 +02:00
nullkal
f9ebff4b13 Allow multiple pinned statuses to be shown and make them be ordered b… (#4690)
* Allow multiple pinned statuses to be shown and make them be ordered by pinned date

* Set timestamps NOT NULL

* Make single-line pinned_statuses

* Spec for pinned_statuses

* Remove redundant empty line
2017-08-25 18:50:52 +02:00
Eugen Rochko
3e20cb7025 Pinned statuses (#4675)
* Pinned statuses

* yarn manage:translations
2017-08-25 01:41:18 +02:00
Eugen Rochko
cab7fa158a Add configuration to disable private status federation over PuSH (#4582) 2017-08-24 17:51:32 +02:00
unarist
bab7127ac9 Fetch reblogs as Announce activity instead of Note object (#4672)
* Process Create / Announce activity in FetchRemoteStatusService

* Use activity URL in ActivityPub for reblogs

* Redirect to the original status on StatusesController#show
2017-08-24 16:21:42 +02:00
Eugen Rochko
d44b84bebd Minor performance improvement for test suite (#4678) 2017-08-24 13:31:55 +02:00
nullkal
da85aff49c Use checkboxes for application scope setting (#4671) 2017-08-23 15:16:20 +02:00
Eugen Rochko
b13501268f Fix up the applications area (#4664)
- Section it into "Development" area
- Improve UI of application form, index, and details
2017-08-23 00:59:35 +02:00
Daigo 3 Dango
fd9d66dfd4 Add Mastodon::Source.url (#4643)
* Add Mastodon::Source.url

* Update spec

* Refactor

Move things frmo Mastodon::Source to Mastodon::Version
2017-08-22 22:54:19 +02:00
Colin Mitchell
63b98318c5 Application prefs section (#2758)
* Add code for creating/managing apps to settings section

* Add specs for app changes

* Fix controller spec

* Fix view file I pasted over by mistake

* Add locale strings. Add 'my apps' to nav

* Add Client ID/Secret to App page. Add some visual separation

* Fix rubocop warnings

* Fix embarrassing typo

I lost an `end` statement while fixing a merge conflict.

* Add code for creating/managing apps to settings section

- Add specs for app changes
- Add locale strings. Add 'my apps' to nav
- Add Client ID/Secret to App page. Add some visual separation
- Fix some bugs/warnings

* Update to match code standards

* Trigger notification

* Add warning about not sharing API secrets

* Tweak spec a bit

* Cleanup fixture creation by using let!

* Remove unused key

* Add foreign key for application<->user
2017-08-22 18:33:57 +02:00
unarist
1cb7c1a273 Fix bugs which OStatus accounts may detected as ActivityPub ready (#4662)
* Fallback to OStatus in FetchAtomService

* Skip activity+json link if that activity is Person without inbox
* If unsupported activity was detected and all other URLs failed, retry with ActivityPub-less Accept header

* Allow mention to OStatus account in ActivityPub

* Don't update profile with inbox-less Person object
2017-08-22 18:30:15 +02:00
Eugen Rochko
d76bd63226 Do not try to re-subscribe to unsubscribed accounts (#4653) 2017-08-21 17:32:41 +02:00
Yamagishi Kazutoshi
7503ada9b5 Use URI.join even when S3 enabled (#4652) 2017-08-21 12:42:16 +02:00
Eugen Rochko
39b76d47a5 Fix #4637 - Re-add missing doorkeeper_authorize for /api/v1/verify_credentials (#4650) 2017-08-21 00:41:08 +02:00
Eugen Rochko
aee1db8fa1 Improve ActivityPub/OStatus compatibility (#4632)
*Note: OStatus URIs are invalid for ActivityPub. But we have them for
as long as we want to keep old OStatus-sourced content and as long as
we remain OStatus-compatible.*

- In Announce handling, if object URI is not a URL, fallback to object URL
- Do not use specialized ThreadResolveWorker, rely on generalized handling
- When serializing notes, if parent's URI is not a URL, use parent's URL
2017-08-19 18:44:48 +02:00
Yamagishi Kazutoshi
c0642a0042 Don't send Link header when don't know prev and next links (#4633) 2017-08-18 10:42:59 +02:00
Eugen Rochko
ce7e12bd72 Put ActivityPub alternate link into Atom, prefer it when processing Atom (#4623) 2017-08-18 01:03:18 +02:00
nightpool
6cce0bb080 use existing inflections instead of custom helper (#4624)
* use existing inflections instead of custom helper

* use ActiveSupport versions
2017-08-17 23:20:50 +02:00
nullkal
a90b5796a1 Show what protocol is used for accounts in admin/accounts#index (#4622)
* Show what protocol used for in admin/accounts#index

* Add frozen_string_literal
2017-08-17 17:52:40 +02:00
unarist
83b48812e4 Fix ActivityPub follow interaction and add more specs (#4601) 2017-08-14 16:57:46 +02:00
Yamagishi Kazutoshi
ac7fb7c820 Add support for searching AP users (#4599)
* Add support for searching AP users

* use JsonLdHelper
2017-08-14 14:08:34 +02:00
Eugen Rochko
1c7cbbcb8c Set correct content-type for ActivityPub JSON (#4592) 2017-08-14 04:16:43 +02:00
Eugen Rochko
3473aac8d8 Hook up URL-based resource look-up to ActivityPub (#4589) 2017-08-14 02:29:36 +02:00
Eugen Rochko
34962d9f9d Add alternate links to ActivityPub resources from HTML/HEAD variants (#4586) 2017-08-13 00:45:04 +02:00
Eugen Rochko
5516767c75 ActivityPub delivery (#4566)
* Deliver ActivityPub Like

* Deliver ActivityPub Undo-Like

* Deliver ActivityPub Create/Announce activities

* Deliver ActivityPub creates from mentions

* Deliver ActivityPub Block/Undo-Block

* Deliver ActivityPub Accept/Reject-Follow

* Deliver ActivityPub Undo-Follow

* Deliver ActivityPub Follow

* Deliver ActivityPub Delete activities

Incidentally fix #889

* Adjust BatchedRemoveStatusService for ActivityPub

* Add tests for ActivityPub workers

* Add tests for FollowService

* Add tests for FavouriteService, UnfollowService and PostStatusService

* Add tests for ReblogService, BlockService, UnblockService, ProcessMentionsService

* Add tests for AuthorizeFollowService, RejectFollowService, RemoveStatusService

* Add tests for BatchedRemoveStatusService

* Deliver updates to a local account to ActivityPub followers

* Minor adjustments
2017-08-13 00:44:41 +02:00
Eugen Rochko
1b6ced5887 Add Digest header to requests with body, handle acct and URI keyId (#4565) 2017-08-09 23:54:14 +02:00
Eugen Rochko
f18739fd60 Add ActivityPub inbox (#4216)
* Add ActivityPub inbox

* Handle ActivityPub deletes

* Handle ActivityPub creates

* Handle ActivityPub announces

* Stubs for handling all activities that need to be handled

* Add ActivityPub actor resolving

* Handle conversation URI passing in ActivityPub

* Handle content language in ActivityPub

* Send accept header when fetching actor, handle JSON parse errors

* Test for ActivityPub::FetchRemoteAccountService

* Handle public key and icon/image when embedded/as array/as resolvable URI

* Implement ActivityPub::FetchRemoteStatusService

* Add stubs for more interactions

* Undo activities implemented

* Handle out of order activities

* Hook up ActivityPub to ResolveRemoteAccountService, handle
Update Account activities

* Add fragment IDs to all transient activity serializers

* Add tests and fixes

* Add stubs for missing tests

* Add more tests

* Add more tests
2017-08-08 21:52:15 +02:00
Sorin Davidoi
ed7abec8f5 feat: Cache status height to avoid expensive renders (#4439)
* feat: Cache status height to avoid expensive renders

* feat: Escape content and emojify in reducers

* fix(css): Remove backface-visibility: hidden from .scrollable

* fix(statuses): Avoid creating DOMParses inside a loop
2017-08-07 20:32:03 +02:00
Ondřej Hruška
199affb141 Refactor Avatar and AvatarOverlay to have 'account' as prop instead of src and staticSrc (#4526)
* Refactored Avatar and AvatarOverlay (DRY) to have 'account' as prop.
Also removed animate attribute from compose navigation bar, which should
have never been there. Added test for avatar overlay.

* fix broken tests

* god dammit another bug in tests! travis please let this pass

* formatting in avatar overlay
2017-08-07 19:44:55 +02:00
Eugen Rochko
fd69694749 Add "signed in as" header to some pages (#4523) 2017-08-05 04:24:58 +02:00
nullkal
62b92a4c0a Redirect to PasswordController#new when reset_password_token is invalid (#4506) 2017-08-03 17:45:45 +02:00
Eugen Rochko
39b0588256 Count all URLs in text as 23 characters flat, do not count domain part of usernames (#4427)
* Count all URLs in text as 23 characters flat, do not count domain part of usernames

* Add new status text counting logic to web UI
2017-07-29 00:06:29 +02:00
Akihiko Odaki
ebb58bc36e Cover Api::V1:FavouritesController more (#4409) 2017-07-28 04:37:44 +02:00
Sorin Davidoi
9cb72c6600 fix(dropdown_menu): Open as modal on mobile (#4295)
* fix(dropdown_menu): Open as modal on mobile

* fix(dropdown_menu): Open modal on touch

* fix(dropdown_menu): Show status

* fix(dropdown_menu): Max dimensions and reduce padding

* chore(dropdown_menu): Test new functionality

* refactor: Use DropdownMenuContainer instead of DropdownMenu

* feat(privacy_dropdown): Open as modal on touch devices

* feat(modal_root): Do not load actions-modal async
2017-07-27 22:31:59 +02:00
Akihiko Odaki
7843855f7f Introduce access token fabricators (#4401) 2017-07-27 15:16:07 +02:00
Yamagishi Kazutoshi
ba1f0b8426 Fallback to site_hostname when site_title is empty (#4394) 2017-07-27 15:13:49 +02:00
Akihiko Odaki
e39c922727 Remove redundant fabrication in the spec for Api::V1::FavouritesController (#4391) 2017-07-27 15:13:10 +02:00
Ondřej Hruška
5eb1a1aaca Fix multipoint shortcode bug (#4387)
* Fix multipoint shortcode bug

* Added testcase for composite emoji shortcode
2017-07-27 01:02:15 +02:00
Eugen Rochko
fae1d9052b Redesign /terms page (#4338)
* Redesign /terms page

* Fix tests

* Remove unused keys
2017-07-24 17:15:15 +02:00
Eugen Rochko
6bfadd0390 Bump to 1.5.0rc1 (#4318) 2017-07-24 16:21:08 +02:00
Sorin Davidoi
78ff8d3421 feat: Enable push subscription for mobile devices by default (#4321) 2017-07-23 23:27:23 +02:00
Eugen Rochko
6dce974c85 Fix intermittent test failures due to accidentally reused class instance between threads (#4287) 2017-07-21 12:45:13 +02:00
Eugen Rochko
65a1c48653 Use the same emoji data on the frontend and backend (#4284)
* Use the same emoji data on the frontend and backend

* Move emoji.json to repository, add tests

This way you don't need to install node dependencies if you only
want to run Ruby code
2017-07-21 04:27:40 +02:00
Eugen Rochko
75ab767791 Fix webfinger retries (#4275)
* Do not raise unretryable exceptions in ResolveRemoteAccountService

* Removed fatal exceptions from ResolveRemoteAccountService

Exceptions that cannot be retried should not be raised. New exception
class for those that can be retried (Mastodon::UnexpectedResponseError)
2017-07-20 01:59:07 +02:00
Eugen Rochko
df59dc6639 Refactor ResolveRemoteAccountService (#4258)
* Refactor ResolveRemoteAccountService

* Remove trailing whitespace

* Use redis locks around critical ResolveRemoteAccountService code

* Add test for race condition of lock
2017-07-19 14:44:04 +02:00
Eugen Rochko
3852032953 Correct OStatus inflection (Ostatus -> OStatus) (#4255) 2017-07-19 01:37:26 +02:00
Sorin Davidoi
fa9c117921 feat(emoji): Add back title attribute (#4253) 2017-07-18 22:49:24 +02:00
Yamagishi Kazutoshi
87bb16a236 Add unfollow modal (optional) (#4246)
* Add unfollow modal

* unfollowing someone

* remove unnecessary prop
2017-07-18 17:14:43 +02:00
Akihiko Odaki
b0f97d9a87 Introduce Ostatus name space (#4164)
* Wrap methods of ProcessFeedService::ProcessEntry in classes

This is a change same with 002ed7dc62, except
that it has the following changes:

* Revert irrelevant change in find_or_create_conversation
* Fix error handling for RemoteActivity

* Introduce Ostatus name space
2017-07-18 16:39:47 +02:00
abcang
e3251e74c1 Improve admin page (#4121)
* Improve admin page

* Fix test

* Add spec

* Improve select style
2017-07-18 16:38:22 +02:00
Eugen Rochko
c99f41dc3c Improve ActivityPub representations (#3844)
* Improve webfinger templates and make tests more flexible

* Clean up AS2 representation of actor

* Refactor outbox

* Create activities representation

* Add representations of followers/following collections, do not redirect /users/:username route if format is empty

* Remove unused translations

* ActivityPub endpoint for single statuses, add ActivityPub::TagManager for better
URL/URI generation

* Add ActivityPub::TagManager#to

* Represent all attachments as Document instead of Image/Video specifically
(Because for remote ones we may not know for sure)

Add mentions and hashtags representation to AP notes

* Add AP-resolvable hashtag URIs

* Use ActiveModelSerializers for ActivityPub

* Clean up unused translations

* Separate route for object and activity

* Adjust cc/to matrices

* Add to/cc to activities, ensure announce activity embeds target status and
not the wrapper status, add "id" to all collections
2017-07-15 03:01:39 +02:00
Eugen Rochko
c6e46f7421 Fix #2672 - Connect signed PuSH subscription requests to instance domain (#4205)
* Fix #2672 - Connect signed PuSH subscription requests to instance domain

Resolves #2739

* Fix return of locate_subscription

* Fix tests
2017-07-14 23:01:20 +02:00
Eugen Rochko
c38739d81e Fix subsequent replies to unresolved status not being filtered from home (#4190)
Resolves #4177 - smaller changeset
2017-07-14 22:31:38 +02:00
Eugen Rochko
c5fa4aba91 HTTP signatures (#4146)
* Add Request class with HTTP signature generator

Spec: https://tools.ietf.org/html/draft-cavage-http-signatures-06

* Add HTTP signature verification concern

* Add test for SignatureVerification concern

* Add basic test for Request class

* Make PuSH subscribe/unsubscribe requests use new Request class

Accidentally fix lease_seconds not being set and sent properly, and
change the new minimum subscription duration to 1 day

* Make all PuSH workers use new Request class

* Make Salmon sender use new Request class

* Make FetchLinkService use new Request class

* Make FetchAtomService use the new Request class

* Make Remotable use the new Request class

* Make ResolveRemoteAccountService use the new Request class

* Add more tests

* Allow +-30 seconds window for signed request to remain valid

* Disable time window validation for signed requests, restore 7 days
as PuSH subscription duration (which was previous default due to a bug)
2017-07-14 20:41:49 +02:00
Sorin Davidoi
8207918890 🎄🔨 Force tree shake emojione (#4202)
* chore(yarn): Install babel-plugin-preval as development dependency

* feat(babel): Add preval as a plugin

* feat(emojione_light): Prevaled module what tree-shaked emojione

* refactor(emoji): Use emojione_light

* feat: Preload emojione_picker bundle

* fix(emojione_light): Do not use Object.entries

* fix(emojify): Update tests

* chore(emojione_light): Remove silly ascii art
2017-07-14 20:30:12 +02:00
unarist
1065b0ac32 Follow renaming of microformats2 gem (#4203) 2017-07-14 19:57:49 +02:00
Eugen Rochko
5138dde794 Fix #4149, fix #1199 - Store emojis as unicode (#4189)
- Use unicode when selecting emoji through picker
- Convert shortcodes to unicode when storing text input server-side
- Do not convert shortcodes in JS anymore
2017-07-14 19:47:53 +02:00
masarakki
d0a037ae79 add validation to tag name (#4194) 2017-07-14 11:02:49 +02:00
Sorin Davidoi
ecab38fd66 Web Push Notifications (#3243)
* feat: Register push subscription

* feat: Notify when mentioned

* feat: Boost, favourite, reply, follow, follow request

* feat: Notification interaction

* feat: Handle change of public key

* feat: Unsubscribe if things go wrong

* feat: Do not send normal notifications if push is enabled

* feat: Focus client if open

* refactor: Move push logic to WebPushSubscription

* feat: Better title and body

* feat: Localize messages

* chore: Fix lint errors

* feat: Settings

* refactor: Lazy load

* fix: Check if push settings exist

* feat: Device-based preferences

* refactor: Simplify logic

* refactor: Pull request feedback

* refactor: Pull request feedback

* refactor: Create /api/web/push_subscriptions endpoint

* feat: Spec PushSubscriptionController

* refactor: WebPushSubscription => Web::PushSubscription

* feat: Spec Web::PushSubscription

* feat: Display first media attachment

* feat: Support direction

* fix: Stuff broken while rebasing

* refactor: Integration with session activations

* refactor: Cleanup

* refactor: Simplify implementation

* feat: Set VAPID keys via environment

* chore: Comments

* fix: Crash when no alerts

* fix: Set VAPID keys in testing environment

* fix: Follow link

* feat: Notification actions

* fix: Delete previous subscription

* chore: Temporary logs

* refactor: Move migration to a later date

* fix: Fetch the correct session activation and misc bugs

* refactor: Move migration to a later date

* fix: Remove follow request (no notifications)

* feat: Send administrator contact to push service

* feat: Set time-to-live

* fix: Do not show sensitive images

* fix: Reducer crash in error handling

* feat: Add badge

* chore: Fix lint error

* fix: Checkbox label overlap

* fix: Check for payload support

* fix: Rename action "type" (crash in latest Chrome)

* feat: Action to expand notification

* fix: Lint errors

* fix: Unescape notification body

* fix: Do not allow boosting if the status is hidden

* feat: Add VAPID keys to the production sample environment

* fix: Strip HTML tags from status

* refactor: Better error messages

* refactor: Handle browser not implementing the VAPID protocol (Samsung Internet)

* fix: Error when target_status is nil

* fix: Handle lack of image

* fix: Delete reference to invalid subscriptions

* feat: Better error handling

* fix: Unescape HTML characters after tags are striped

* refactor: Simpify code

* fix: Modify to work with #4091

* Sort strings alphabetically

* i18n: Updated Polish translation

it annoys me that it's not fully localized :P

* refactor: Use current_session in PushSubscriptionController

* fix: Rebase mistake

* fix: Set cacheName to mastodon

* refactor: Pull request feedback

* refactor: Remove logging statements

* chore(yarn): Fix conflicts with master

* chore(yarn): Copy latest from master

* chore(yarn): Readd offline-plugin

* refactor: Use save! and update!

* refactor: Send notifications async

* fix: Allow retry when push fails

* fix: Save track for failed pushes

* fix: Minify sw.js

* fix: Remove account_id from fabricator
2017-07-13 22:15:32 +02:00
unarist
ae26d7b557 Make tag search case insensitive again (#4184) 2017-07-13 19:31:33 +02:00
Eugen Rochko
8abeec1f4f Improve UI of admin site settings (#4163) 2017-07-12 03:24:04 +02:00
Eugen Rochko
49026e27e8 Redesign the landing page, mount public timeline on it (#4122)
* Redesign the landing page, mount public timeline on it

* Adjust the standalone mounted component to the lacking of router

* Adjust auth layout pages to new design

* Fix tests

* Standalone public timeline polling every 5 seconds

* Remove now obsolete translations

* Add responsive design for new landing page

* Address reviews

* Add floating clouds behind frontpage form

* Use access token from public page when available

* Fix mentions and hashtags links, cursor on status content in standalone mode

* Add footer link to source code

* Fix errors on pages that don't embed the component, use classnames

* Fix tests

* Change anonymous autoPlayGif default to false

* When gif autoplay is disabled, hover to play

* Add option to hide the timeline preview

* Slightly improve alt layout

* Add elephant friend to new frontpage

* Display "back to mastodon" in place of "login" when logged in on frontpage

* Change polling time to 3s
2017-07-11 15:27:59 +02:00
Sorin Davidoi
1a5ae3019d refactor: Rewrite immutablejs import statements using destructuring (#4147) 2017-07-11 01:00:14 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
3db69012fd Fix PrecomputeFeedService for filtered statuses (#4148) 2017-07-11 01:00:01 +02:00
STJrInuyasha
098eef3211 Remote following success page (#4129)
* Added a success page to remote following
Includes follow-through links to web (the old redirect target) and back to the remote user's profile

* Use Account.new in spec instead of a fake with only id
(fixes spec)

* Fabricate(:account) over Account.new

* Remove self from the success text
(and all HTML with it)
2017-07-10 18:05:06 +02:00
Yamagishi Kazutoshi
865f5170a8 Add setting a always mark media as sensitive (#4136) 2017-07-10 14:00:32 +02:00
nullkal
07024f56df Use charlock_holmes instead of nkf at FetchLinkCardService (#4080)
* Specs for language detection

* Use CharlockHolmes instead of NKF

* Correct mistakes

* Correct style

* Set hint_enc instead of falling back and strip_tags

* Improve specs

* Add dependencies
2017-07-08 22:44:31 +02:00
Eugen Rochko
0d54e620d9 Replace OEmbed and initial state Rabl templates with serializers (#4110)
* Replace OEmbed Rabl template with serializer

* Replace initial state rabl with serializer
2017-07-08 14:51:05 +02:00
Eugen Rochko
0217e15dd3 Fix #4058 - Use a long-lived cookie to keep track of user-level sessions (#4091)
* Fix #4058 - Use a long-lived cookie to keep track of user-level sessions

* Fix tests, smooth migrate from previous session-based identifier
2017-07-07 23:25:15 +02:00
Eugen Rochko
20e15ecfb3 Refactor JSON templates to be generated with ActiveModelSerializers instead of Rabl (#4090) 2017-07-07 04:02:06 +02:00
Damien Erambert
387de88e2f Add a setting allowing the use of system's default font in Web UI (#4033)
* add a system_font_ui setting on the server

* Plug the system_font_ui on the front-end

* add EN/FR locales for the new setting

* put Roboto after all other fonts

* remove trailing whitespace so CodeClimate is happy

* fix user_spec.rb

* correctly write user_spect this time

* slightly better way of adding the classes

* add comments to the system-font stack for clarification

* use .system-font for the class instead

* don't use multiple lines for comments

* remove trailing whitespace

* use the classnames module for consistency

* use `mastodon-font-sans-serif` instead of Roboto directly
2017-07-06 22:39:56 +02:00
abcang
8041c97d52 Fix Nokogiri::HTML at FetchLinkCardService (#4072) 2017-07-05 14:54:21 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
d5586563c7 Remove sort in Feed (#4050)
In from_redis method, statuses retrieved from the database was mapped to
the IDs retrieved from Redis. It was equivalent to order from high to low
because those IDs are sorted in the same order.
Statuses are ordered with the ID by default, so we do not have to reorder.
Sorting statuses in the database is even faster since the IDs are indexed
with B-tree.
2017-07-03 13:17:27 +02:00
Nolan Lawson
28da1d54d5 Faster emojify() by avoiding str.replace() entirely (#4049) 2017-07-03 11:02:36 +02:00
Nolan Lawson
ef21f8bc7e Faster emojify() algorithm, avoid regex replace (#4019)
* Faster emojify() algorithm, avoid regex replace

* add semicolon
2017-06-30 17:29:22 +02:00
Matt Jankowski
eb39fb7211 Add controller spec for manifests controller (#4003) 2017-06-30 13:43:34 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
aea653f05d Do not raise an error if PrecomputeFeed could not find any status (#4015) 2017-06-30 13:39:42 +02:00
Yamagishi Kazutoshi
022878b743 Upgrade jsdom to version 11.0.0 (#3994) 2017-06-29 13:03:03 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
4be3cf318d Cover Admin::AccountsController more (#3327) 2017-06-29 01:43:10 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
05f73fbdab Reduce number of commands in FeedManager#trim (#3989) 2017-06-29 01:17:26 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
fa7649409b Overwrite old statuses with reblogs in PrecomputeFeedService (#3984) 2017-06-28 14:50:23 +02:00
Eugen Rochko
d0221b05bc Fix #1624 - Send e-mail notifications to admins about new reports (#3949) 2017-06-27 00:04:00 +02:00
Eugen Rochko
b90a44b6d0 Fix #3910 - Require OTP authentication to disable 2FA (#3935)
* Fix #3910 - Require OTP authentication to disable 2FA. Also, remove ability
to generate new OTP backup codes *after* initial backup codes were handed
out during activation

* Restore recovery code re-generation

* Improve display of some 2FA elements
2017-06-25 23:51:46 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
4ca14209d1 Cover Auth::RegistrationsController more (#3353) 2017-06-25 21:42:55 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
071c6f392e Cover Admin::DomainBlocksController more (#3329)
Also domain_block fabricator now sets unique domains
2017-06-25 21:42:36 +02:00
Eugen Rochko
c465c5b3a8 Add overview of active sessions (#3929)
* Add overview of active sessions

* Better display of browser/platform name

* Improve how browser information is stored and displayed for sessions overview

* Fix test
2017-06-25 16:54:30 +02:00
Sorin Davidoi
1280559503 Revocable sessions (#3616)
* feat: Revocable sessions

* fix: Tests using sign_in

* feat: Configuration entry for the maximum number of session activations
2017-06-23 18:50:53 +02:00
Eugen Rochko
8fb81e1372 Add important test for full-width hashtags (#3911) 2017-06-23 17:01:53 +02:00
Yamagishi Kazutoshi
539dbaa049 Remove unused variables (#3906) 2017-06-23 16:05:04 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
561ca10d86 Some minor change and spec for Account (#3813)
* Introduce domains method to Account relation

Account had followers_domains method, which was excessively specific.
Let relation of Account have domains method instead.

* Move follow_mapping in Account to AccountInteractions

* Introduce shared examples for AccountAvatar inclusion

* Cover Account more
2017-06-22 18:34:27 +02:00
Eugen Rochko
e7810e4031 Fix regression from #3842 (#3892)
* Fix regression from #3842

Simplify the query by omitting all direct statuses. Private statuses
are allowed because they are from accounts we are following (so
by definition)

Resolves #3887 (alternative)

* Adjust test
2017-06-22 02:38:50 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
77dcf442e7 Filter direct statuses in Status.as_home_timeline (#3842)
The classes using Status.as_home_timeline, namely Feed and
PrecomputeFeedService are expected to filter direct statuses as
FanOutWriteService does, but their filtering were incomplete or missing.

This commit solves the problem by filtering direct statuses in
as_home_timeline as the other similar methods such as as_public_timeline
does.
2017-06-20 20:41:23 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
baa248a801 Update fabricator for MediaAttachment to attach a file according to type (#3862)
This fixes a random spec failures since commit
c8020b2105.
2017-06-20 20:40:45 +02:00
unarist
55376105f5 Fix RTL detection on Ruby side (#3867)
This fixes below bugs:

* pipe characters being counted as RTL character
* only first word being checked
2017-06-20 18:45:09 +02:00
masarakki
d947978192 setting-for-account-deletable (#3852) 2017-06-19 15:12:31 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
481f277e58 Cover Block more (#3837) 2017-06-19 11:31:37 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
ac1f0e49f0 Cover DomainBlock more (#3838) 2017-06-19 11:31:27 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
c8020b2105 Cover Export more (#3840) 2017-06-19 11:30:27 +02:00
Eugen Rochko
8bed91d94c Rename FollowRemoteAccountService to ResolveRemoteAccountService (#3847)
Rename Activitypub to ActivityPub
2017-06-19 01:51:04 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
5fcbc71c35 Cover Favourite more (#3841) 2017-06-19 01:38:50 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
e990a4a566 Spec AccountDomainBlock (#3816) 2017-06-18 02:59:49 +02:00
nightpool
0048f7ac72 Whitelist allowed classes for federated statuses (#3810)
* Whitelist allowed classes for federated statuses

Allowed classes are currently:

 - Any microformats class (h/p/u/dt/e-*)
 - the classes mention, hashtag, ellipses and invisible.

this last one is somewhat suspect, but Mastodon currently uses it to render hidden link text.

resolved #3790

* Fix code style
2017-06-17 20:26:05 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
f391f81f3d Spec UserMailer (#3757) 2017-06-17 01:15:00 +02:00
Eugen Rochko
7181178007 Fix regression from #3490 - filter out hidden statuses from ancestors/descendants even if the viewer is anonymous (#3752) 2017-06-15 02:00:08 +02:00
Eugen Rochko
c883e96d95 Fix account delete form not accepting password, update suspended (#3745)
account before removing content for quicker feedback to end-users
2017-06-14 20:30:12 +02:00
Eugen Rochko
6f8f401ea1 Batched remove status service (#3735)
* Make Pubsubhubbub::DistributionWorker handle both single stream entry
arguments, as well as arrays of stream entries

* Add BatchedRemoveStatusService, make SuspendAccountService use it

* Improve method names

* Add test

* Add more tests

* Use PuSH payloads of 100 to have a clear mapping of
1000 input statuses -> 10 PuSH payloads

It was nice while it lasted
2017-06-14 18:01:35 +02:00
Eugen Rochko
1c7e2ddd65 Account deletion (#3728)
* Add form for account deletion

* If avatar or header are gone from source, remove them

* Add option to have SuspendAccountService remove user record, add tests

* Exclude suspended accounts from search
2017-06-14 18:01:27 +02:00
Eugen Rochko
973d6ef93a Fix #2619 - When redis feed is empty, fall back to database (#3721)
* Fix #2619 - When redis feed is empty, fall back to database

* Use redis value to return feed from database only while RegenerationWorker
hasn't finished running

* Fix specs

* Replace usage of reject!
2017-06-14 13:37:03 +02:00
unarist
1ce6a20bde Fix locale related specs (#3707)
* Use I18n.locale instead of ":en"
* Reset I18n.locale value after locale changing tests
2017-06-12 10:58:03 +02:00
Eugen Rochko
8b893afde7 Fix removal of status sending the original status to mentioned users instead of delete Salmon (#3672)
* Fix removal of status sending the original status to mentioned users instead
of delete Salmon, add test

* Create remove_status_service_spec.rb
2017-06-11 17:13:43 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
5c54ae381a Localize date in digest and cover NotificationMailer more (#3694)
* Localize date in digest

* Cover NotificationMailer more
2017-06-11 12:04:35 +02:00
René Klačan
ecdf17a2d7 Make sure email is case insensitive on all places (#3688)
When case insensitivity is enabled via devise's `config.case_insensitive_keys` then `.find_for_authentication` method needs to be used instead of `.find_by` because second mentioned returns `nil` when valid email with different cases is passed.

More info https://github.com/plataformatec/devise/wiki/How-To:-Use-case-insensitive-emails
2017-06-11 02:29:08 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
a5729feb11 Improve default language decision and spec (#3351)
* Improve default language decision

This change allows to takes account of accepted language determined by
the user agent even if the custom default locale of the instance is
configured.

* Cover Localized more

* Fix code style
2017-06-10 09:44:02 +02:00
Matt Jankowski
e31cd944c2 Move create/destroy actions for api/v1/statuses to namespace (#3678)
Each of mute, favourite, reblog has been updated to:

- Have a separate controller with just a create and destroy action
- Preserve historical route names to not break the API
- Mild refactoring to break up long methods
2017-06-10 09:39:26 +02:00
Matt Jankowski
dc717cf425 Move reblogged_by and favourited_by actions out of api/v1/statuses and into unique controllers (#3646)
* Add specs for api statuses routes

* Update favourited_by and reblogged_by api routes

* Move methods into new controllers

* Use load_accounts methods to simplify index actions

* Clean up load_accounts methods

* Clean up link header generation

* Check for link headers in specs

* Remove unused actions from api/v1/statuses controller

* Remove specs for moved actions
2017-06-09 14:12:40 -04:00
Matt Jankowski
b439018342 Language detection defaults to nil (#3666)
* Default to nil for statuses.language

* Language detection defaults to nil instead of instance UI default
2017-06-09 18:09:37 +02:00
Eugen Rochko
006594e181 Fix db:seed - only run some validations when the field was changed (#3592)
* Fix db:seed - only run some validations when the field was changed

* Add tests
2017-06-08 09:22:01 -04:00
Yamagishi Kazutoshi
5da41e0cae Improve RuboCop rules (compatibility to Code Climate) (#3636)
08f8de84eb/Gemfile.lock (L38)
Code Climate is using RuboCop v0.46.0.

Change several rules to maintain compatibility.
2017-06-08 13:24:28 +02:00
Matt Jankowski
76f986d07b Clean up for api/base controller (#3629)
* Move ApiController to Api/BaseController

* API controllers inherit from Api::BaseController

* Add coverage for various error cases in api/base controller
2017-06-07 20:09:25 +02:00
unarist
c40e5aea44 Use "match_array" only for order independent assertions (#3626) 2017-06-07 12:59:28 -04:00
Matt Jankowski
9a43a1051a Coverage improvement and concern extraction for rate limit headers in API controller (#3625)
* Coverage for rate limit headers

* Move rate limit headers methods to concern

* Move throttle check to condition on before_action

* Move match_data variable into method

* Move utc timestamp to separate method

* Move header setting into smaller methods

* specs cleanup
2017-06-07 17:23:26 +02:00
unarist
5ef958b99f Fix 500 errors on searching invalid URLs (#3613) 2017-06-06 16:44:48 +02:00
unarist
117d333a84 Fix tag search order and not to use tsvector (#3611)
* Sort results by the name
* Switch search method to simple `LIKE` matching instead of tsvector/tsquery

Previously we used scores from ts_rank_cd() to sort results, but it didn't work
because the function returns same score for all results. It's not for calculate
similarity of single words. Sometimes this bug even push out exact matching tag
from results.

Additionally, PostgreSQL supports prefix searching with standard btree index.
Using it offers simpler code, but also less index size and some speed.
2017-06-06 16:07:06 +02:00
Yamagishi Kazutoshi
615c580665 Add quotes ESLint rules (#3602)
* Add quotes ESLint rule

* Add jsx-quotes ESlint rule

* Sort ESLint rules
2017-06-06 03:56:36 +02:00
Matt Jankowski
d339ac2c76 Introduce StatusThreadingConcern (#3490)
* Add a StatusFilter class to identify visibility of statuses by accounts

* Extract StatusThreadingConcern from Status

* Clarify purpose of checking for nil account
2017-06-05 16:07:44 +02:00
Daigo 3 Dango
e1142e190b Redirect to streaming_api_base_url (#3579)
* Redirect to streaming_api_base_url

When Rails receives a request to streaming API, it most likely
means that there is another host which is configured to respond
to it. This is to redirect clients to that host if
`STREAMING_API_BASE_URL` is set as another host.

* Use the new Ruby 1.9 hash syntax
2017-06-05 12:09:29 +02:00
Yamagishi Kazutoshi
d4f2db0902 Fix randomly fail (regression from #3560) (#3580) 2017-06-05 10:03:36 +02:00
Yamagishi Kazutoshi
49dee153d3 Skip image length check (regression from #3528) (#3581) 2017-06-05 10:03:01 +02:00
René Klačan
b64a43d38f Don't follow account if it's already followed (#3575)
Closes https://github.com/tootsuite/mastodon/issues/3102
2017-06-05 03:24:18 +02:00
Eugen Rochko
4a6ceb24c9 Added support for configurable reserved usernames (fix of #1382) (#3566)
* Added support for configurable reserved usernames

* Added reserved usernames from mastodon issue 1355

* Fix reserved usernames
2017-06-05 01:03:45 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
ec5f146bca Cover WebfingerResource more (#3560) 2017-06-04 17:13:37 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
1653be63ac Spec ScopedSettings (#3559) 2017-06-04 17:07:39 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
10da1db4e6 Remove some code in TagManager and spec (#3547)
* Do not fall back to StreamEntry if object_type is unavailable in TagManager

Since 24c77e57b2, when Status, the only model
with stream_entry, and StreamEntry got its own logic in uri_for and
url_for, the purpose of the fallbacks to activity_type of StreamEntry
became unclear.

This commit removes the fallbacks. When adding another model with
stream_entry in future, consider to update uri_for and url_for.

* Cover TagManager more
2017-06-04 16:14:25 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
734a309e93 Spec InlineRablScope (#3542) 2017-06-04 15:00:08 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
5c1868df66 Spec HashObject (#3544) 2017-06-04 14:59:52 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
3dd0d62583 A minor change for ProviderDiscovery and spec (#3543)
* Do not default the format in ProviderDiscovery

The format should be determined when discovering, as it is in the current
implementation, and it is a flaw if it is not determined.

* Spec ProviderDiscovery
2017-06-04 14:59:40 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
267468e232 Remove some arguments of Formatter.instance.format and spec (#3541)
* Remove some arguments of Formatter.instance.format

* Improve spec for Formatter
2017-06-04 14:58:57 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
502d0a1f04 Spec Extractor (#3540) 2017-06-04 14:57:02 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
39ddcf59a9 Spec action verification of StreamEntryFinder (#3549) 2017-06-04 14:56:31 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
e285f6086c Remove redundant code in AtomSerializer and spec (#3528)
* Remove redundant status.reply? in AtomSerializer#object

* Cover AtomSerializer more
2017-06-03 20:51:08 +02:00
Yamagishi Kazutoshi
9df077e370 Fix spec for #2388 (#3526) 2017-06-03 14:40:55 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
2c511bbad4 Spec response for forgery (#3248)
Remove protect_from_forgery in ApiController, which is disabled by the
following skip_before_action, as well.
2017-06-01 20:56:55 +02:00
Yamagishi Kazutoshi
ae7f6beef1 Improve tests for JavaScript (#3496)
- Upgrade dependencies
    - chai (3.5.0 -> 4.0.1)
    - chai-enzyme (0.6.1 -> 0.7.1)
    - sinon (2.2.0 -> 2.3.2)
- Change extensions from .jsx to .js
- Don't assign `React` to `global`
- Check code format using ESLint
2017-06-01 17:27:15 +02:00
Yamagishi Kazutoshi
339d73541b Add scenarios for log in (#3497) 2017-06-01 17:25:59 +02:00
Matt Jankowski
d327e1d105 Remove usernames and hashtags from language detection (#3503)
* Add failing specs for hashtag and username extraction in language detector

* Remove usernames and hashtags from text before language detection

* Handle multiple instances of special case, and reduce whitespace
2017-06-01 09:29:14 -04:00
Matt Jankowski
25990c77c0 Handle nil and blank cases in Account finders (#3500) 2017-06-01 08:20:36 -04:00
Matt Jankowski
1066fd1ab5 Spec coverage and refactor for the api/v1/accounts controllers (#3451) 2017-05-31 21:36:24 +02:00
Matt Jankowski
81ce4e4318 Move admin/pubsubhubbub controller to admin/subscriptions (#3442) 2017-05-31 20:39:35 +02:00
Matt Jankowski
0985a9ff7d Improve spec coverage and clean up api/v1/blocks controller (#3464) 2017-05-31 20:34:51 +02:00
Matt Jankowski
ae721f376a Improve spec coverage and clean up api/v1/follow_requests controller (#3465) 2017-05-31 20:32:11 +02:00
Matt Jankowski
e32b9ecbf6 Improve spec coverage and clean up api/v1/domain_blocks controller (#3466) 2017-05-31 20:31:14 +02:00
Matt Jankowski
6237abaaa3 Improve spec coverage and clean up api/v1/favourites controller (#3472) 2017-05-31 20:30:39 +02:00
Matt Jankowski
ff4d3f11b5 Expand spec coverage and refactor the Account.find_ methods (#3485)
* Move specs for account finder methods to concern spec

* Move account finder methods to concern

* Improve spec wording

* Use more explicit comparison to ensure correct return value

* Add coverage for .find_local! and .find_remote!

* Add some methods to the finder

* Use arel on matching_username method

* Avoid ternary in matching domain method

* Simplify finder methods

* Use an AccountFinder class to simplify lookup
2017-05-31 20:28:45 +02:00
Matt Jankowski
3e95a6c9b7 Improve spec coverage and clean up api/v1/mutes controller (#3481) 2017-05-31 20:27:34 +02:00
Matt Jankowski
b89dbac199 Refactor and spec coverage for api/v1/timelines actions (#3482) 2017-05-31 20:27:17 +02:00
Yamagishi Kazutoshi
024a0dc704 Hold value of I18n.locale with ApplicationHelperSpec (#3474) 2017-05-31 17:09:28 +02:00
Yamagishi Kazutoshi
b652c52992 Keep ENV['LOCAL_HTTPS'] with ApplicationControllerSpec (fix random fail) (#3479)
* Keep ENV['LOCAL_HTTPS'] with ApplicationControllerSpec (fix random fail)

* use climate_control
2017-05-31 17:09:17 +02:00
Matt Jankowski
8b69d8d46d Improve spec coverage and clean up api/v1/media controller (#3467) 2017-05-30 21:11:29 -04:00
Matt Jankowski
5a54a5ba69 Clean up api/subscriptions controller (#3448) 2017-05-31 02:15:09 +02:00
Jack Jennings
4e75c71b3e Add status destroy authorization to policy (#3453)
* Add status destroy authorization to policy

* Create explicit unreblog status authorization
2017-05-30 22:56:31 +02:00
Matt Jankowski
4e89226243 Improve api oembed controller (#3450)
* Add StreamEntryFinder class to parse URLs

* Use StreamEntryFinder and clean up api/oembed controller
2017-05-30 16:30:06 -04:00
Matt Jankowski
0ee8f559ec Clean up api/salmon controller (#3449) 2017-05-30 16:28:58 -04:00
Jack Jennings
c4bf180a77 Fix incorrect visibility setter in StatusPolicySpec (#3456) 2017-05-30 22:14:32 +02:00
Matt Jankowski
18c81c74e5 Remove exports/base controller in favor of shared concern (#3444) 2017-05-30 19:06:01 +02:00
Akihiko Odaki
20d016f430 Refactor User and spec (#3431)
* Protect send_devise_notification of User

* Improve spec for User
2017-05-30 15:28:56 +02:00
Jack Jennings
877b82f63e Move status reblog authorization into policy (#3425) 2017-05-30 15:16:14 +02:00
Akihiko Odaki
27811b2b39 Use around hook to restore context in InstancePresenter spec (#3430) 2017-05-30 09:03:35 -04:00
Akihiko Odaki
80598c9190 Use around hook to restore context in InstanceHelper spec (#3429) 2017-05-30 06:32:33 -04:00
Akihiko Odaki
6f064d0e65 Use around hook to restore context in Admin::SettingsController spec (#3428) 2017-05-30 06:32:09 -04:00
Akihiko Odaki
1b74047c4f Use around hook to restore context in ApplicationHelper spec (#3432) 2017-05-30 06:29:48 -04:00
Yamagishi Kazutoshi
53d9bebaf3 Update dependencies of Node.js (#3067)
* Update autoprefixer to version 7.1.0

* Update axios to version 0.16.1

* Update babel-cli to version 6.24.1

* Update babel-eslint to version 7.2.3

* Update babel-preset-react to version 6.24.1

* Update coffee-script to version 1.12.6

* Update css-loader to version 0.28.1

* Update express to version 4.15.2

* Update js-yaml to version 3.8.4

* Update mocha to version 3.4.1

* Update jsdom to version 10.1.0

* Update npmlog to version 4.1.0

* Update postcss-smart-import to version 0.7.0

* Update postcss-loader to version 2.0.5

* Update pg to version 6.1.5

* Update prop-types to version 15.5.10

* Update rails-ujs to version 5.1.1

* Update redis to version 2.7.1

* Update sass-loader to version 6.0.5

* Update stringz to version 0.2.0

* Update webpack to version 2.5.1

* Update style-loader to version 0.17.0

* Update sinon to version 2.2.0

* Update react-motion to version 0.5.0

* Update react-notification to version 6.7.0

* Update react-redux-loading-bar to version 2.9.0

* Update react-intl to version 2.3.0

* Update redux-immutable to version 4.0.0

* Update reselect to version 3.0.1

* Update react-toggle to version 3.0.1

* Update react-simple-dropdown to version 3.0.0

* Update react-router to version 3.0.5

* Update react-router-scroll to version 0.4.2

* Update react-toggle to version 4.0.1

* yarn upgrade
2017-05-29 18:23:28 +02:00
Jack Jennings
faf53a5a3e Extract authorization policy for viewing statuses (#3150) 2017-05-29 18:22:22 +02:00
Akihiko Odaki
ff2bfea0b3 Remove methods from ObfuscateFilename and spec (#3347)
* Remove methods from ObfuscateFilename

* Spec ObfuscateFilename
2017-05-29 18:14:16 +02:00
Akihiko Odaki
587f2d0b1f Spec Auth::ConfirmationsController (#3348) 2017-05-29 18:13:11 +02:00
Akihiko Odaki
a111604729 Spec AccountControllerConcern (#3349) 2017-05-29 18:12:54 +02:00
Akihiko Odaki
e41d7875d7 Cover Admin::ReportsController more (#3346) 2017-05-29 18:12:34 +02:00
Akihiko Odaki
9481b20b23 Cover InstancesController more (#3342) 2017-05-29 18:12:17 +02:00
Akihiko Odaki
46a60e19ef Cover Admin::SuspensionsController more (#3350) 2017-05-29 18:11:16 +02:00
Akihiko Odaki
503298d89b Spec Auth::PasswordsController (#3352) 2017-05-29 18:10:50 +02:00
Akihiko Odaki
1cb145578b Spec Admin::ResetsController calls send_reset_password_instructions (#3354) 2017-05-29 18:09:17 +02:00
Akihiko Odaki
2b299efbbc Spec #destroy feature of Admin::TwoFactorAuthenticationsController (#3355) 2017-05-29 18:08:58 +02:00
Akihiko Odaki
07c90d0b0d Coever Admin::SilencesController more (#3356) 2017-05-29 18:08:42 +02:00
Akihiko Odaki
e48d2dfd87 Cover Settings::ExportsController more (#3358) 2017-05-29 18:08:26 +02:00
Akihiko Odaki
43722d1add Cover Oauth::AuthorizedApplicationsController (#3359) 2017-05-29 18:08:05 +02:00
Akihiko Odaki
cdcb550ce9 Cover Oauth::AuthorizationsController more (#3361) 2017-05-29 18:07:39 +02:00
Akihiko Odaki
013825b7b0 Cover Settings::TwoFactorAuthenticationsController more (#3376) 2017-05-29 18:07:07 +02:00
Akihiko Odaki
bbaa9dda56 Cover WellKnown::HostMetaController more (#3382) 2017-05-29 18:05:30 +02:00
Akihiko Odaki
bb41a6ad80 Cover Settings::TwoFactorAuthentication::RecoveryCodesController more (#3383) 2017-05-29 18:05:20 +02:00
Akihiko Odaki
f868fcb154 Spec Settings::Exports (#3384)
* Spec output of Settings::Exports::BlockedAccountsController

* Spec output of Settings::Exports::FollowingAccountsController

* Spec output of Settings::Exports::MutedAccountsController

* Spec Settings::Exports::BaseController

This commit removes duplicate specs in classes inheriting
Settings::Exports::BaseController as well.
2017-05-29 18:05:01 +02:00
Akihiko Odaki
7300379c81 Cover WellKnown::WebfingerController more (#3385) 2017-05-29 18:04:37 +02:00
Akihiko Odaki
820e97af27 Cover Settings::TwoFactorAuthentication::ConfirmationsController more (#3386) 2017-05-29 18:04:00 +02:00
Akihiko Odaki
53bc24a6ae Cover ApplicationHelper more (#3412) 2017-05-29 18:03:37 +02:00
Akihiko Odaki
8192f0d981 Spec Activitystreams2BuilderHelper (#3413) 2017-05-29 18:03:15 +02:00
Akihiko Odaki
9389058544 Spec FlashesHelper (#3414) 2017-05-29 18:03:02 +02:00
Akihiko Odaki
b0c3722eea Spec HomeHelper (#3415) 2017-05-29 18:02:44 +02:00
Akihiko Odaki
5f935f5da8 Spec HttpHelper (#3416) 2017-05-29 18:02:32 +02:00
Akihiko Odaki
437ef29afd Cover Admin::PubSubHubHubController more (#3343) 2017-05-29 18:01:37 +02:00
Atsushi Yamamoto
4547e3585a Add preference setting for delete toot modal (#3368)
* Set delete_modal preference to true by default
* Does not show confirmation modal if delete_modal is false
* Add ja translation for preference setting page
2017-05-29 17:56:13 +02:00
happycoloredbanana
a867644030 Avoid comparing domains when looking for an exact match of a local account (#3336) 2017-05-27 00:55:08 +02:00
Akinori MUSHA
8a5d3b2e5d Go to root after login in single user mode (#3289)
In single user mode, visitors are redirected to the single user's
profile page.  So, if you are the owner without a session, you start
from that page, click the login button and authenticate yourself
expecting you'll soon get started with the home page, but in reality
you'll get redirected back to where you started from -- your own
profile page.

This fixes the behavior by redirecting you home after login if you
have started from your own profile page.
2017-05-26 14:14:03 +02:00
Akihiko Odaki
5d9513aaea Spec Admin::BaseController (#3328) 2017-05-26 14:13:26 +02:00
Akihiko Odaki
9efb491366 Cover TagsController more (#3306) 2017-05-25 16:07:38 +02:00
abcang
4c889085d2 Add test when hashtag and URL are concatenated (#3279) 2017-05-24 15:36:10 +02:00
Yamagishi Kazutoshi
d54273621b Skip formatting for cashtag in status text (#3275)
Resolve #3270
2017-05-24 14:32:53 +02:00
Akihiko Odaki
6af012aae4 Cover HomeController more (#3258) 2017-05-23 23:37:24 +02:00
Eugen Rochko
b886ecea5c Fix Devise destroy method being available to delete user record (#3266)
(You may think that we need account deletions, but this way would've just orphaned the db records)
2017-05-23 21:32:42 +02:00
alpaca-tc
46ca699d72 Toggle sensitive from admin page (#3261) 2017-05-23 19:45:43 +02:00
Matt Jankowski
fecc2c2f47 Refactor of API timeline actions (#3263)
- Increase coverage to exercise all parts of each action
- Move into namespace to share common code
- Misc refactor of each action for smaller methods, simpler code
2017-05-23 18:11:39 +02:00
Akihiko Odaki
855082f935 Cover StatusesController more (#3259) 2017-05-23 16:34:28 +02:00
Akihiko Odaki
d670290506 Cover StreamEntriesController more and remove redundant instructions (#3257)
* Cover StreamEntriesController more

* Remove redundant instructions in StreamEntriesController
2017-05-23 15:04:23 +02:00
Akihiko Odaki
1cf385646a Spec AuthorizeFollowsController assigns @account (#3246)
@account is used by the view.
2017-05-23 13:12:45 +02:00
Akihiko Odaki
4f023cb001 Introduce recent to Follow (#3247)
Introduce recent to Follow, as Account and other models have.
This change also adds specs for the scope and the dependents.
2017-05-23 13:12:19 +02:00
Akihiko Odaki
9cc7a3c209 Cover AccountsController more in spec (#3229)
* Introduce recent scope to Status and StreamEntry

Introduce recent scope to Status and StreamEntry as Account has.

* Cover AccountsController more in AccountsController
2017-05-23 02:53:01 +02:00
Matt Jankowski
4f2a0f546d Coverage for api/web/settings controller (#3238) 2017-05-22 23:12:28 +02:00
Matt Jankowski
8c11626321 Add coverage for ReportFilter and AccountFilter (#3236) 2017-05-22 15:50:58 -04:00
Matt Jankowski
0eb8b00cc9 Specs for cleanup workers (#3235)
* Add spec files for feed and media cleanup workers

* Add coverage for feed and media cleanup schedulers

* Clean up feed and media cleanup workers
2017-05-22 19:36:21 +02:00
Akihiko Odaki
c0aba00ae1 Cover ApplicationController more in spec (#3230) 2017-05-22 17:58:57 +02:00
Akihiko Odaki
387e8449a8 Cover AccountUnfollowController more in spec (#3228) 2017-05-22 17:58:49 +02:00
Akihiko Odaki
4f061bc92c Cover AccountFollowController more in spec (#3227) 2017-05-22 16:29:48 +02:00
Akihiko Odaki
680f357380 Cover AboutController more in spec (#3226) 2017-05-22 16:27:19 +02:00
Immae
ae917bfb23 Allow alternate domains for mastodon handlers (#3187) 2017-05-22 15:40:04 +02:00
Akihiko Odaki
cbd9804857 single_user_mode? always returns boolean (#3215)
This change also adds a specification for the method.
2017-05-22 06:00:06 +02:00
Matt Jankowski
1122249e51 Filter languages with opt out (#3175)
* Remove allowed_languages and add filtered_languages

* Use filtered_languages instead of allowed_languages
2017-05-20 17:32:44 +02:00
Eugen Rochko
c8c80650bc Ensure unique identifiers in fabricated objects in tests (#3173) 2017-05-20 17:09:40 +02:00
Eugen Rochko
a55019c3bf More tests for public timeline method (#3171) 2017-05-20 16:56:30 +02:00
Sorin Davidoi
fcf794aebe Performance improvements (#3168)
* refactor(components/status_list): Avoid quering scrollTop if not necessary

* refactor(components/dropdown_menu): Do not render items if not expanded

* refactor: Cherry-pick react-motion imports

* refactor(compose/privacy_dropdown): Do not render options if not open

* refactor(components/column_collapsable): Do not render children if collapsed
2017-05-20 14:58:13 +02:00
Jack Jennings
2d7cf07852 Enable and fill in JSON response test in API::AccountsController (#3157) 2017-05-20 00:27:13 +02:00
Matt Jankowski
c3f8cd26c7 Add coverage for api/v1 controllers (#3155) 2017-05-19 23:32:37 +02:00
Eugen Rochko
5695449335 Add buttons to block and unblock domain (#3127)
* Add buttons to block and unblock domain

* Relationship API now returns "domain_blocking" status for accounts,
rename "block entire domain" to "hide entire domain", fix unblocking domain,
do not block notifications from domain-blocked-but-followed people, do
not send Salmons to domain blocked users

* Add test

* Personal domain blocks shouldn't affect Salmon after all, since in this
direction of communication the control is very thin when it comes to
public stuff. Best stay consistent and not affect federation in this way

* Ignore followers and follow request from domain blocked folks,
ensure account domain blocks are not created for empty domain,
and avoid duplicates in validation

* Purge followers when blocking domain (without soft-blocks, since they
are useless here)

* Add tests, fix local timeline being empty when having any domain blocks
2017-05-19 21:05:32 +02:00
Matt Jankowski
c8f8629c7e Add spec coverage for regeneration worker (#3143) 2017-05-19 16:55:09 +02:00
Matt Jankowski
4423ed3557 Specs for precompute feed service (#3142)
* Add spec for precompute feed service

* Refactor PrecomputeFeedService

* spec wip
2017-05-19 16:21:52 +02:00
Yamagishi Kazutoshi
2ad6edded5 Replace inline styles with stylesheet (#3115) 2017-05-19 11:42:54 +02:00
Eugen Rochko
0cafe62561 Account domain blocks (#2381)
* Add <ostatus:conversation /> tag to Atom input/output

Only uses ref attribute (not href) because href would be
the alternate link that's always included also.

Creates new conversation for every non-reply status. Carries
over conversation for every reply. Keeps remote URIs verbatim,
generates local URIs on the fly like the rest of them.

* Conversation muting - prevents notifications that reference a conversation
(including replies, favourites, reblogs) from being created. API endpoints
/api/v1/statuses/:id/mute and /api/v1/statuses/:id/unmute

Currently no way to tell when a status/conversation is muted, so the web UI
only has a "disable notifications" button, doesn't work as a toggle

* Display "Dismiss notifications" on all statuses in notifications column, not just own

* Add "muted" as a boolean attribute on statuses JSON

For now always false on contained reblogs, since it's only relevant for
statuses returned from the notifications endpoint, which are not nested

Remove "Disable notifications" from detailed status view, since it's
only relevant in the notifications column

* Up max class length

* Remove pending test for conversation mute

* Add tests, clean up

* Rename to "mute conversation" and "unmute conversation"

* Raise validation error when trying to mute/unmute status without conversation

* Adding account domain blocks that filter notifications and public timelines

* Add tests for domain blocks in notifications, public timelines
Filter reblogs of blocked domains from home

* Add API for listing and creating account domain blocks

* API for creating/deleting domain blocks, tests for Status#ancestors
and Status#descendants, filter domain blocks from them

* Filter domains in streaming API

* Update account_domain_block_spec.rb
2017-05-19 01:14:30 +02:00
Matt Jankowski
b3f0e6f7b9 After remote follow worker specs (#3098)
* Add specs for the after remote follow workers

* Refactor the after remote follow workers
2017-05-18 21:10:41 +02:00
Eugen Rochko
93e9f8b3ed Fix #2572 - Resolve preview cards for remote statuses as well as local ones (#3088) 2017-05-17 00:41:15 +02:00
alpaca-tc
b8b7be1e4a Improve Account#triadic_closures (#3079) 2017-05-16 12:06:38 +02:00
alpaca-tc
0d7dc192c3 Optimize Status#permitted_for 24x (#3069)
* Build query with arel node

* Add spec for current Status#permitted_for implementation

* Refactor status.rb

* Order by visibility to optimize query
2017-05-16 02:54:17 +02:00
Eugen Rochko
bd7dc50186 Fix change of status callbacks not setting in_reply_to_account_id and (#3072)
possibly others when expected. Add some tests for it
2017-05-15 21:20:55 +02:00
Eugen Rochko
5039bc93d5 Feature conversations muting (#3017)
* Add <ostatus:conversation /> tag to Atom input/output

Only uses ref attribute (not href) because href would be
the alternate link that's always included also.

Creates new conversation for every non-reply status. Carries
over conversation for every reply. Keeps remote URIs verbatim,
generates local URIs on the fly like the rest of them.

* Conversation muting - prevents notifications that reference a conversation
(including replies, favourites, reblogs) from being created. API endpoints
/api/v1/statuses/:id/mute and /api/v1/statuses/:id/unmute

Currently no way to tell when a status/conversation is muted, so the web UI
only has a "disable notifications" button, doesn't work as a toggle

* Display "Dismiss notifications" on all statuses in notifications column, not just own

* Add "muted" as a boolean attribute on statuses JSON

For now always false on contained reblogs, since it's only relevant for
statuses returned from the notifications endpoint, which are not nested

Remove "Disable notifications" from detailed status view, since it's
only relevant in the notifications column

* Up max class length

* Remove pending test for conversation mute

* Add tests, clean up

* Rename to "mute conversation" and "unmute conversation"

* Raise validation error when trying to mute/unmute status without conversation
2017-05-15 03:04:13 +02:00
Matt Jankowski
4752edb183 Update rails to version 5.0.3 (#3054)
* Update rails to version 5.0.3

* Update aws-sdk to version 2.9.17

* Update capistrano to version 3.8.1

* Update climate_control to version 0.2.0

* Update debug_inspector to version 0.0.3

* Update et-orbi to version 1.0.4

* Update oj to version 3.0.7

* Update microformats2 to version 2.9.0

* Update lograge to version 0.5.1

* Clean up usage of Microformats2 object to remove deprecation warnings

* Update microformats2 to version 3.0.1
2017-05-14 16:30:14 +02:00
Eugen Rochko
83b444ddab Do not cancel PuSH subscriptions after encountering "permanent" error… (#3046)
* Do not cancel PuSH subscriptions after encountering "permanent" error response

After talking with MMN about it, turns out some servers/php setups do
return 4xx errors while rebooting, so this anti-feature that was meant
to take load off of the hub is doing more harm than good in terms of
breaking subscriptions

* Update delivery_worker.rb
2017-05-14 03:22:48 +02:00
alpaca-tc
4b5edb0d16 Fixes NoMethodError: undefined method 'first' for nil:NilClass (#3036) 2017-05-13 15:56:12 +02:00
Matt Jankowski
dc59643e73 Spec coverage and refactor for pubsub/delivery worker (#3021)
* Framework for delivery worker spec

* Refactor of pubsub delivery worker
2017-05-12 20:35:36 +02:00
Eugen Rochko
12ef99556e Add conversation model, <ostatus:conversation /> (#3016)
* Add <ostatus:conversation /> tag to Atom input/output

Only uses ref attribute (not href) because href would be
the alternate link that's always included also.

Creates new conversation for every non-reply status. Carries
over conversation for every reply. Keeps remote URIs verbatim,
generates local URIs on the fly like the rest of them.

* Fix conversation migration

* More spec coverage for status before_create

* Prevent n+1 query when generating Atom with the new conversations

* Improve code style

* Remove redundant local variable
2017-05-12 19:09:21 +02:00
Eugen Rochko
7b5af13d19 Prepend reblogs' wrapper content with "RT @original_author", (#3013)
so that when a reblog parse fails on another instance, it doesn't
look like a misattributed/stolen text
2017-05-12 17:46:44 +02:00
Matt Jankowski
213c146d87 Coverage for domain block worker (#2997) 2017-05-11 22:25:15 +02:00
Eugen Rochko
7c20c80ed7 Fix #2027 - Accept own ID for remote follow with and without preceding @ (#2991)
* Fix #2027 - Accept own ID for remote follow with and without preceding @
Fix #2177 - Omit leading "acct:" in remote follow redirect template expansion

* Fix test
2017-05-11 21:54:30 +02:00
Jack Jennings
cd56acd184 Add specs for current Status#permitted? implementation (#2982) 2017-05-11 11:25:10 +02:00
Eugen Rochko
3702e2c924 Fix regressions from #2683 (#2970)
* Fix regressions from #2683

Properly format spoiler text HTML, while keeping old logic for blankness intact
Process hashtags and mentions in spoiler text
Format spoiler text for Atom
Change "show more" toggle into a button instead of anchor
Fix style regression on dropdowns for detailed statuses

* Fix lint issue

* Convert spoiler text to plaintext in desktop notifications
2017-05-11 00:28:10 +02:00
Matt Jankowski
60fa69399c Pubsub confirmation worker spec (#2974)
* Add specs for valid challenge response from pubsub confirmation worker

* Refactor the pubsub confirmation worker
2017-05-11 00:19:37 +02:00
Eugen Rochko
ee7719d540 Fix #2955 - Send HEAD request ahead of GET when fetching URL previews (#2972) 2017-05-10 23:30:07 +02:00
Matt Jankowski
cd8fc8031f Spec feed insert worker (#2965)
* Spec for feed insert worker when missing records

* more specs!

* Refactor feed insert worker
2017-05-10 20:32:05 +02:00
Matt Jankowski
6658a20e98 Disable sidekiq logging in test mode (#2962) 2017-05-10 16:16:42 +02:00
Matt Jankowski
cd3630e3d1 Spec coverage and refactor of digest mailer worker (#2961) 2017-05-10 15:45:43 +02:00
Eugen Rochko
221898cc90 Add spec for Pubsubhubbub::DistributionWorker. PuSH-deliver public items (#2954)
to all subscribers. IDN-normalize callback URLs for subscriptions on insert.
2017-05-10 02:55:43 +02:00
Matt Jankowski
b188aeb0e7 Specs for pubsub subscribe service (#2951)
* Add spec for pubsubhubbub/subscribe

* Refactor pubsubhubbub/subscribe service
2017-05-09 20:48:30 +02:00
Matt Jankowski
87ef624429 Spec and refactor for pubsubhubbub/unsubscribe service (#2946)
* Add coverage for pubsub unsubscribe service

* Refactor pubsub unsubscribe service
2017-05-09 19:58:18 +02:00
Akihiko Odaki
a1abb11656 Use CLD3 (#2949)
Compact Language Detector v3 (CLD3) is the successor of CLD2, which was
used in the previous implementation. CLD3 includes improvements since CLD2,
and supports newer compilers. On the other hand, it has additional
requirements and cld3-ruby, the FFI of CLD3 for Ruby, is still new and may
be still inmature.

Though CLD3 is named after CLD2, it is implemented with a neural network
model, different from the old implementation, which is based on a Naïve
Bayesian classifier.

CLD3 supports newer compilers, such as GCC 6. CLD2 is not compatible with
GCC 6 because it assigns negative values to varibales typed unsigned.
(see internal/cld_generated_cjk_uni_prop_80.cc) The support for GCC 6 and
newer compilers are essential today, when some server operating system
such as Ubuntu Server 16.10 has GCC 6 by default.

On the one hand, CLD3 requires C++11 support. Environments with old
compilers such as Ubuntu Server 14.04 needs to update the system or install
a newer compiler.

CLD3 needs protocol buffers as a new dependency. However,it is not
considered problematic because major server operating systems, CentOS and
Ubuntu Server provide them.

The FFI cld3-ruby was written by me (Akihiko Odaki) for use in Mastodon.
It is still new and may be inmature, but confirmed to pass existing tests.
2017-05-09 19:58:03 +02:00
abcang
5b538cde50 Fixed bug that timeline can not be displayed by InvalidURIError (#2947) 2017-05-09 18:17:41 +02:00
Matt Jankowski
c298bcbb49 Services specs for subscribe and unsubscribe (#2928)
* Add specs for unsubscribe service

* Fix non existent methods in unsubscribe service

* Clean up status handling in subscribe service
2017-05-09 00:45:02 +02:00
Matt Jankowski
735c5098c5 Specs for API push controller, with refactor (#2926)
* Coverage for api push controller

* Refactor the api/push controller
2017-05-09 00:44:30 +02:00
Matt Jankowski
23f6ec856f Delegate Account#user_locale method and allow nil (#2927) 2017-05-08 23:10:50 +02:00
Matt Jankowski
bddb03a089 Improve allowed language handling (#2897)
* Dont allow empty value in user allowed languages

* Sanitize language input to reject blank values in array
2017-05-08 03:32:52 +02:00
Matt Jankowski
d2aae2c2e0 Unblock domain service specs/refactor (#2867)
* Add spec for unblock domain service

* Refactor UnblockDomainService
2017-05-07 14:44:28 +02:00
alpaca-tc
ea2e2f4857 Hotfix convert string from symbol (#2856)
* Convert key to string from symbol

* Prefer :public_send instead of
2017-05-06 23:06:52 +02:00
alpaca-tc
de0f63e0ac Fixes slow query in notifications api (#2851)
* Add inclusion validation for activity_type

* When excluding all types with `WHERE NOT IN`, the query search all rows.
2017-05-06 21:55:40 +02:00
alpaca-tc
0f6ae77634 Optimize MuteService and AfterBlockService (#2836) 2017-05-06 16:31:07 +02:00
Eugen Rochko
78b890cc81 Fix Scheduler::SubscriptionsScheduler (#2834)
* Fix Scheduler::SubscriptionsScheduler, add worker test for it

* Change production log level of Sidekiq to "warn" instead of "info"
2017-05-06 13:05:03 +02:00
Matt Jankowski
05b8a950c6 Stream entry specs and refactor to use delegate (#2827)
* Add coverage for stream entry delegated methods

* Use delegate with allow_nil to clean up stream entry
2017-05-06 04:00:21 +02:00
Matt Jankowski
5393dbf4a2 Misc spec coverage improvements (#2821)
* Dont use raise_error by itself (avoids warning)

* Add coverage for AccountFilter

* Improve coverage and refactor for Subscription#lease_seconds

* Improve coverage and refactor for NotificationMailer

* Simplify assignment of min/max threshold on subscription
2017-05-05 14:56:00 -04:00
masarakki
23bf01bcd6 use Twitter::Extractor for creating links (#2502) 2017-05-05 19:48:22 +02:00
Matt Jankowski
dc50a231de Add specs (and refactor) of FetchRemoteResourceService and SearchService (#2812)
* Coverage for fetch remote resource service

* Refactor fetch remote resource service

* Coverage for search service

* Refactor search service
2017-05-05 17:26:04 +02:00
Eugen Rochko
c32e312061 More robust PuSH subscription refreshes (#2799)
* Fix #2473 - Use sidekiq scheduler to refresh PuSH subscriptions instead of cron

Fix an issue where / in domain would raise exception in TagManager#normalize_domain

PuSH subscriptions refresh done in a round-robin way to avoid hammering a single
server's hub in sequence. Correct handling of failures/retries through Sidekiq (see
also #2613). Optimize Account#with_followers scope. Also, since subscriptions
are now delegated to Sidekiq jobs, an uncaught exception will not stop the entire
refreshing operation halfway through

Fix #2702 - Correct user agent header on outgoing http requests

* Add test for SubscribeService

* Extract #expiring_accounts into method

* Make mastodon:push:refresh no-op

* Queues are now defined in sidekiq.yml

* Queues are now in sidekiq.yml
2017-05-05 02:23:01 +02:00
Matt Jankowski
d30ae1bc44 Remove the react-rails gem (#2801)
* Remove react-rails gem

* Fix broken view spec
2017-05-04 23:45:18 +02:00
alpaca-tc
a206fa8037 Delete records in smaller transaction (#2802) 2017-05-04 23:44:39 +02:00
Matt Jankowski
ca59622a87 Remove redis clear from before in spec (#2795) 2017-05-04 20:53:39 +02:00
Matt Jankowski
232b822d86 Replace best_in_place editor on admin settings page (#2789)
* Remove best_in_place

* Replace best_in_place usage with rails helpers

* Move admin/settings#index to #edit action

* Remove click_to__edit from i18n
2017-05-04 18:12:44 +02:00
Eugen Rochko
78df86a7c8 Likely fix #2458, fix #2031 - handle out-of-order deletes for statuses (#2734)
* Likely fix #2458, fix #2031 - handle out-of-order deletes for statuses

If a delete arrives before the original status, cache that information
for 6h, and if the original status arrives in that window, ignore it

* Add test case
2017-05-04 04:34:57 +02:00
ThibG
f1d96e40a3 Additional specs for URI handling (#2759) 2017-05-03 20:40:14 +02:00
Eugen Rochko
91a8a5eab8 Fix #2706 - Always respond with 200 to PuSH payloads (#2733)
Fix #2196 - Respond with 201 when Salmon accepted, 400 when unverified
Fix #2629 - Correctly handle confirm_domain? for local accounts
Unify rules for extracting author acct from XML, prefer <email>, fall back
to <name> + <uri> (see also #2017, #2172)
2017-05-03 17:02:18 +02:00
Matt Jankowski
9db85085a9 Language improvements, replace whatlanguage with CLD (#2753)
* add failing en specs

* add cld2 gem

* Replace WhatLanguage with CLD
2017-05-03 16:59:31 +02:00
Eugen Rochko
ef2af79a48 Replace sprockets/browserify with Webpack (#2617)
* Replace browserify with webpack

* Add react-intl-translations-manager

* Do not minify in development, add offline-plugin for ServiceWorker background cache updates

* Adjust tests and dependencies

* Fix production deployments

* Fix tests

* More optimizations

* Improve travis cache for npm stuff

* Re-run travis

* Add back support for custom.scss as before

* Remove offline-plugin and babili

* Fix issue with Immutable.List().unshift(...values) not working as expected

* Make travis load schema instead of running all migrations in sequence

* Fix missing React import in WarningContainer. Optimize rendering performance by using ImmutablePureComponent instead of
React.PureComponent. ImmutablePureComponent uses Immutable.is() to compare props. Replace dynamic callback bindings in
<UI />

* Add react definitions to places that use JSX

* Add Procfile.dev for running rails, webpack and streaming API at the same time
2017-05-03 02:04:16 +02:00
Matt Jankowski
129e06f0b3 Auth sign out (#2511)
* Add a spec for signing out

* Add spec showing that suspended user gets a 403 forbidden on sign out

* Allow suspended account users to sign out
2017-05-02 23:37:58 +02:00
ThibG
d50fcad917 Add rspec to further specify FollowRemoteAccountService (#2414) 2017-05-02 23:37:26 +02:00
Kaylee
fae1ed48e3 Add option to disable two factor auth in admin accounts panel. (#2584)
* Add option to disable two factor auth in admin accounts panel.
Closes #2578

* Add @mjankowski's suggestions.
* Moves destroy actions behind User#disable_two_factor!
* Adds spec coverage for Admin:TwoFactorAuthenticationsController and User#disable_two_factor!
2017-05-02 21:07:12 +02:00
Yamagishi Kazutoshi
70f9858e6f Fix subscription expiration condition (#2715)
* Fix subscription expiration condition

* dry and add spec
2017-05-02 18:21:22 +02:00
Matt Jankowski
fdd7ee1cc7 Coverage for remote follows (#2694)
* Add coverage for create with empty acct value

* Add coverage for create with webfinger failure

* Add coverage for create with webfinger providing bad values

* Add coverage for create when webfinger is good

* Add coverage for session[:remote_follow] having data

* Simplify how remote follow pulls acct from session

* Remote follow behaves more like model

* Move the discovery portions of remote follow out of controller

* Check for suspended accounts
2017-05-02 00:44:23 +02:00
Matt Jankowski
c44ef23e50 Filter on allowed user language preferences (#2361)
* Naive approached to timeline filtering

* Convert allowed_languages into a db column

* Allow users to choose languages to see statuses in

* Style list items as two columns

* Add a hint to explain language filtering preference
2017-05-01 17:42:13 +02:00
Matt Jankowski
4f155a3c86 Reports spec for media attachments, clean up method (#2660)
* Add coverage for Report#media_attachments

* Direct query on media attachment
2017-04-30 18:15:49 +02:00
Akihiko Odaki
a28863bf95 Revert "Use PostgreSQL inheritance for blocks and mutes (#2520)" (#2634)
This reverts commit 7cd8e61b5a.
2017-04-30 08:49:24 -04:00
Matt Jankowski
a5c8ffcaf9 Extract user tracking into concern (#2600) 2017-04-30 00:28:16 +02:00
Akihiko Odaki
7cd8e61b5a Use PostgreSQL inheritance for blocks and mutes (#2520) 2017-04-30 00:27:31 +02:00
Matt Jankowski
54450f75d3 More coverage yes more even more (#2627)
* Add coverage for admin/confirmations controller

* Coverage for statuses controller show action

* Add coverage for admin/domain_blocks controller

* Add coverage for settings/profiles#update
2017-04-30 00:25:38 +02:00
Joël Quenneville
e9ebe097ec Add tests to the Feed model (#2594)
This adds a test for the `Feed#get` method. While the data
transformations in `Feed#get` may seem redundant, they are important to
maintain the order from Redis. The tests I wrote will fail if someone
tries to refactor away this "redundancy" (as I tried to do in the first
iteration of this change).
2017-04-29 00:21:35 +02:00
Patrick Figel
ef6f02c173 Set correct attachment type for rejected media (#2599)
In #2110, a new attachment type "unknown" was introduced for
attachments that were rejected due to a domain being blocked using
reject_media. However, the "type" field was never set to "unknown"
because a default value of "0" (image) is set for that column,
causing the `type.blank?` expression to always equal false.

This version uses type_changed? instead, causing the type to be set
to "unknown" unless a type has been explicitly set. This introduces
a small change in behaviour causing the type to be set to unknown
before paperclip calls `before_post_process`. Presumably this
behaviour is more appropriate than the current one because the
attachment type has not been determined by that point.

Included are new tests for `ProcessFeedService` and
`UpdateRemoteProfileService` which now check that remote media is
downloaded for non-blocked domains and is rejected for others.
2017-04-29 00:18:32 +02:00
Matt Jankowski
ea15b2a359 Fix broken view spec for about/links (#2591) 2017-04-28 17:54:03 +02:00
Matt Jankowski
e1d3dc89fa Fix broken spec for about/links view partial (#2586) 2017-04-28 16:45:24 +02:00
Yamagishi Kazutoshi
af18c8ab28 Fix spec for hide get started (#2585) 2017-04-28 16:44:57 +02:00
Matt Jankowski
442d4cfb4a More controller specs (#2561)
* Add render_views in more places

* Delegate methods from account to user with allow nil true, so that admin accounts show view renders when missing a user

* Use actual account instances in authorize follow controller spec
2017-04-28 15:12:37 +02:00
Matt Jankowski
7662a68de1 Return missing page when tag does not exist (#2563) 2017-04-28 15:11:21 +02:00
Matt Jankowski
3e819257ab More status specs (#2564)
* Add rough outline of coverage needed for public timeline

* Specs for visibility, replies, boosts

* Specs for silenced account

* Specs for local_only option

* Specs for blocks and mutes

* Add tentative spec around including other silenced account statuses

* Add with_public_visibility scope

* Add simple coverage for tag_timeline

* Tag timeline includes replies

* Replace tag.statuses with a tagged_with scope in tag timeline method

* Use with_public_visibility in tag timeline

* Extract common scope between public and tag timelines to method

* Extract local domain check to local_only scope

* Extract local_only check to starting scope method

* Move list of excluded from timeline account ids to account model

* Simplify excluded accounts list on account model

* Only join accounts when needed

* Rename method for account specific filtering

* Extract method for account exclusions

* Fix bug where silenced accounts were not including statuses from other silenced accounts

* DRY up filter application from account or no account

* timeline_scope can be private

* Add spec showing that account can find its excluded accounts ids

* Add spec which fails if local_only does not have a left outer join

* rubocop
2017-04-28 15:10:41 +02:00
Matt Jankowski
01b7056b81 Add render_views to admin/reports controller spec (#2548) 2017-04-28 01:21:38 +02:00
Matt Jankowski
65ad682e81 Fix remote follow (#2547)
* Add coverage for remote_follow#new

* Fix reference to authorize_follows/card partial
2017-04-28 00:16:44 +02:00
Eugen Rochko
97dff125a8 Improve shared status verification (#2525)
* Instead of parsing shared status contents verbatim, make roundtrip
to purported original URL. Confirm that the "original" URL is from the
same domain as the author it claims to be from.

* Fix obvious typo, add comment

* Use URI look-up first

* Add test, update Goldfinger dependency to make less useless HTTP requests per Webfinger lookup
2017-04-27 17:06:47 +02:00
Matt Jankowski
b17d7a1f85 Catch error when server decryption fails on 2FA (#2512) 2017-04-27 15:18:21 +02:00
Eugen Rochko
4a7dc4fadc OEmbed support for PreviewCard (#2337)
* OEmbed support for PreviewCard

* Improve ProviderDiscovery code failure treatment

* Do not crawl links if there is a content warning, since those
don't display a link card anyway

* Reset db schema

* Fresh migrate

* Fix rubocop style issues
Fix #1681 - return existing access token when applicable instead of creating new

* Fix test

* Extract http client to helper

* Improve oembed controller
2017-04-27 14:42:22 +02:00
Matt Jankowski
6712d98c58 Add spec coverage and refactor authorize_follows controller (#2505) 2017-04-27 00:19:53 +02:00
Matt Jankowski
89255db0d8 Domain block service cleanup (#2490)
* Add coverage for domain block service with silence

* Get rid of warning about find_each and order

* Move domain_block to attr_reader

* Move optional clear_media into silence_accounts method

* Use blocked_domain method to reduce passed vars

* Extract blocked_domain_accounts method to find accounts on the domain

* Extract media_from_blocked_domain method to find relevant attachments

* Separate destruction of account images and account attachments
2017-04-26 20:09:01 +02:00
Matt Jankowski
d120a7ac99 Add spec for Status.as_home_timeline (#2451) 2017-04-26 20:08:30 +02:00
Francis Chong
395608442d Add media dimensions (#2448)
* Fixes #1985

- add migration AddMediaAttachmentMeta, which add meta field to media_attachments
- before saving attachment, set file meta if needed
- add meta in api

* add spec

* align the “size” format for image and video

* fix code climate

* fixes media_attachment_spec.rb
2017-04-26 03:48:12 +02:00
Guillaume Lo Re
9e955fe688 Stricter whitelist rules (#2213)
* Stricter whitelist rules

* Linting

* Added spec for blacklisting

* Test subdomain blacklist on domain whitelist

* No need to split

* Change spec name
2017-04-26 01:22:51 +02:00
alpaca-tc
31d49716c6 Localize with i18n for Devise::FailureApp (#2309)
This PR fixes I18n.locale for rake middlewares. Mastodon uses Devise that depends on Warden.
Warden::Manager can be found in rake middleware. It is outside of the controller.

In the case of authentication failed, warden calls throw(:warden). At the time Warden::Manager
delegates request to failure_app to generate response and flash[:alert] after catching it.
Unfortunately, I18n.locale is already reset then because I18n.with_locale is enabled only
inside the controller. If we used I18n.locale=, Devise::FailureApp could get the current locale.
2017-04-25 15:06:41 +02:00
Evan Minto
1b5966ad07 Change ActivityPub paging to match spec. Clean up ActivityPub outbox changes. (#2410)
* Change ActivityPub paging to match spec. Clean up ActivityPub outbox changes.

* Fix code style and test failures for OutboxController.

* Attempt to fix CI errors.
2017-04-25 15:06:06 +02:00
Eugen Rochko
553d6a1ea6 Fix #2402 - Add Idempotency-Key header to PostStatusService that prevents (#2419)
duplicates. Web UI regenerates UUID for that header every time the compose
form is changed or successfully submitted

Also, fix Farsi i18n overwriting the English one
2017-04-25 15:04:49 +02:00
Yamagishi Kazutoshi
7f02d09887 Decodes URL containing IDN (#2436) 2017-04-25 15:03:51 +02:00
178inaba
1b6e534850 Optimize account search (#2421) 2017-04-25 04:44:43 +02:00
Eugen
87f7a3922c Punycode URI normalization (#2370)
* Fix #2119 - Whenever about to send a HTTP request, normalize the URI

* Add test for IDN request in FetchLinkCardService

* Perform IDN normalization on domains before they are stored in the DB
2017-04-25 02:47:31 +02:00
Eugen
cc9714a4d0 Add more FeedManager#filter? tests (#2413) 2017-04-24 22:37:24 +02:00
Matt Jankowski
8940fe2b4e Fix flashes partial render error for controllers which don't inherit from application controller (#2400)
* Add failing spec for oauth/authorized_applications controller

* Use explicit reference to flashes partial from admin layout

Because some of the controllers which use the admin layout do not inherit from
application controller, this partial is not in their view path.
2017-04-24 17:30:30 +02:00
Eugen
f89e26bdd5 Followers-only post federation (#2111)
* Make private toots get PuSHed to subscription URLs that belong to domains where you have approved followers

* Authorized followers controller, stub for bulk action

* Soft block in the background

* Add simple test for new controller

* Rename Settings::FollowersController to Settings::FollowerDomainsController, paginate results,
rename "private" post setting to "followers-only", fix pagination style, improve post privacy
preferences style, improve warning style

* Extract compose form warnings into own container, show warning when posting to followers-only with unlocked account
2017-04-24 00:38:37 +02:00
Matt Jankowski
7355f7f8f4 Fix deprecation warnings on activitypub controller specs (#2357) 2017-04-23 19:10:04 +02:00
Matt Jankowski
8c0108a4e8 Ensure i18n matches options (#2358)
* Add failing spec showing that human_locales does not match what i18n knows about

* Add missing `ar` key for arabic to human locales

* Remove duplicate `id` key from available locales

* Sort keys in human locales list

* Add spec for human_locale helper
2017-04-23 19:09:52 +02:00
Joël Quenneville
eff1644d22 Test embedded_view related code in a helper (#2282)
The two methods `StreamEntriesHelper#stream_link_target` and
`StreamEntriesHelper#acct` are based on checking whether we are running
in an embedded view.

This adds some test helper code to make the testing easier. We extracted
some "magic strings" to constants to lower the coupling in the specs.
2017-04-23 06:05:52 +02:00
Joël Quenneville
e8a3a00068 Add test coverage to CSS class generation (#2285)
The code that generates CSS is based on a lot of boolean conditions.
The possible combinations of these grows exponentially as we add more
conditions.

Since most of the code is conditional on a single boolean, we tested the
following:

1. All `false`
2. All `true`
3. Each individual flag set to `true`

The methods tested are:

* `StreamEntriesHelper#style_classes`
* `StreamEntriesHelper#microformats_classes`
* `StreamEntriesHelper#microformats_h_class`
2017-04-23 06:04:32 +02:00
Evan Minto
227f45402d ActivityPub: Add basic, read-only support for Outboxes, Notes, and Create/Announce Activities (#2197)
* Clean up collapsible components

* Expose user Outboxes and AS2 representations of statuses

* Save work thus far.

* Fix bad merge.

* Save my work

* Clean up pagination.

* First test working.

* Add tests.

* Add Forbidden error template.

* Revert yarn.lock changes.

* Fix code style deviations and use localized instead of hardcoded English text.
2017-04-23 05:21:10 +02:00
Ash Furrow
eb7db6f8da Admin UI for confirming users (#2245)
* Shows confirmed status in list.

* Adds ability to confirm users in admin UI.

* Added new english translations.

* Addresses feedback from #2245.

* More feedback.
2017-04-23 04:43:42 +02:00
Wesley Ellis
4a6ccafa2f Add og:url tags (#2298)
* add og:url to stream_item view

* add og:url and text to about/show

* add og:url to profile, followers and follows pages
2017-04-22 17:29:46 +02:00
Matt Jankowski
1a0960ba88 [WIP] Html lang on statuses (#2297)
* Add html lang attributes around statuses

* Remove urls from language detection
2017-04-22 04:26:25 +02:00
Matt Jankowski
502eef6f8d 2FA controller cleanup (#2296)
* Add spec coverage for settings/two_factor_auth area

* extract setup method for qr code

* Move otp required check to before action

* Merge method only used once

* Remove duplicate view

* Consolidate creation of @codes for backup

* Move settings/2fq#recovery_codes to settings/recovery_codes#create

* Rename settings/two_factor_auth#disable to #destroy

* Add coverage for the otp required path on 2fa#show

* Clean up the recovery codes list styles

* Move settings/two_factor_auth to settings/two_factor_authentication

* Reorganize the settings two factor auth area

Updated to use a flow like:

- settings/two_factor_authentication goes to a #show view which has a button
  either enable or disable 2fa on the account
- the disable button turns off the otp requirement for the user
- the enable button cycles the user secret and redirects to a confirmation page
- the confirmation page is a #new view which shows the QR code for user
- that page posts to #create which verifies the code, and creates the recovery
  codes
- that create action shares a view with a recovery codes controller which can be
  used separately to reset codes if needed
2017-04-22 04:23:17 +02:00
Joël Quenneville
3ed49b5854 Add tests for StreamEntriesHelper#rtl? (#2286)
We used some random Arabic characters to test that the various RTL
conditions got triggered.
2017-04-22 00:13:37 +02:00
Matt Jankowski
e5407e62b2 User settings mutation (#2270)
* Add user spec for settings, highlight global default mutation issue

* Fix mutation issue caused by settings/preferences spec
2017-04-21 18:07:17 +02:00
Matt Jankowski
be1ecde300 Clean up settings/preferences controller (#2237)
* Add missing fields group on preferences page

* Clean up settings/preferences controller

* Extract a UserSettingsDecorator
2017-04-21 03:26:52 +02:00
Matt Jankowski
696f4794fd Admin settings controller refactor, add specs, cleanup (#2225)
* Add render_views for admin/settings spec

* Add coverage for admin/settings#update

* Add coverage for admin/settings typecasting open_registrations setting

* Simplify how admin/settings finds the value for updating

* Rely on activerecord to not update a value that hasnt changed

* Add coverage for non-existent setting

* Use a constant for boolean settings
2017-04-20 17:18:09 +02:00
Marcin Cieślak
0ab0db9fe7 Add static gif for upload testing (#2168) 2017-04-19 23:47:50 +02:00
Eugen
5849eac93e Fix #2108 - Fix gif uploads (#2171)
* Fix #2108 - Fix gif uploads
Add specs for media attachment gifv conversion

* Add ffmpeg to travis

* Make travis install ffmpeg, not libav

* Switch travis to trusty
2017-04-19 23:21:00 +02:00
Eugen
a38b05112e Fix possibility of unrightful webfinger redirect (#2147)
* Fix possibility of unrightful webfinger redirect

* Add more tests for FollowRemoteAccountService
2017-04-19 17:28:35 +02:00
abcang
815462feff Fix html escape characters in the URL (#2138)
* fix character escaping in URL

* add tests

* put a comma after the last item

* add HTML escape test
2017-04-19 14:52:18 +02:00
Matt Jankowski
9a8134cea6 Restful refactor of accounts/ routes (#2133)
* Add routing specs for accounts followers and following actions

* Use more restful route naming for public account follow pages

Moves two actions:
- accounts#followers to accounts/follower_accounts#index
- accounts#following to accounts/following_accounts#index

Adds routing spec to ensure prior URLs are preserved.
2017-04-19 13:52:37 +02:00
Eugen
64c57b7465 Fix wording on admin instances view, fix typos in language detector spec (#2106) 2017-04-18 23:39:51 +02:00
Eugen
2f4657aac7 Add more tests for AtomSerializer (#2096) 2017-04-18 22:30:17 +02:00
Matt Jankowski
4591c7dbdb Language detection refactor (#2099)
* Extract detect_language to separate class

* Use default locale, not just en

* Add spec to confirm that whatlanguage cant identify empty string

* Allow account locale to override default in language detector

* PostStatusService supplies an account to detect language
2017-04-18 22:20:12 +02:00
happycoloredbanana
9026426b4d Remove API authentication for public statuses (after review) (#1919) 2017-04-18 21:58:57 +02:00
Matt Jankowski
ce5cb2b81c Instances list in admin (#2095)
* Add admin/instances index action

* Add link to instances admin page

* View lists instances

* Instances, grouped by domain, ordered by count

* Use Account.remote scope

* Extract method: Account.by_domain_accounts
2017-04-18 21:09:07 +02:00
Matt Jankowski
4b6be67e85 Admin reports with accounts (#2092)
* Add a ReportFilter class

* Add reports and targeted_reports relationships to Account

* Use ReportFilter from admin/reports controller

* Link to admin/reports filtered views from admin account show view

* Add indexes to reports.account_id and reports.target_account_id
2017-04-18 19:36:18 +02:00
Matt Jankowski
7a7a2d591f Remove unused methods (#1730)
* Remove unused method #set_counters_maps from api controller

* Remove unused method #set_account_counters_maps from api controller

* Remove unused method Account#followers_domains

* Remove unused User.prolific scope

* Add mastodon:users:admins task to list all admin emails

* Use interpolated query style in Account.triadic_closures

* Coverage for Account.triadic_closures
2017-04-18 01:21:55 +02:00
Eugen
d907fdbdfc Add forgotten AtomSerializer spec (#2047) 2017-04-18 01:07:46 +02:00
Matt Jankowski
d087c95aa7 Instance helper to replace site title helper (#2038)
* Move site title helper to instance helper (name change only)

* Remove newline in <title> tag

* Add site_hostname helper method to wrap up local_domain value

* Use site_hostname helper in places that need local_domain value
2017-04-18 00:16:32 +02:00
Matt Jankowski
3cdfdaf013 Remove i18n_spec file (#2014)
These were being skipped with `xit`, but we also just added the `unused` task to
the CI server, and it will be caught there. That seems good enough for this,
instead of requiring a full keys check on every local spec suite run.
2017-04-17 20:02:29 +02:00
Matt Jankowski
47cc8462e3 Media controller specs (#2022)
* Add spec for media controller

* Add MediaAttachment.attached scope

* Simplify methods in media controller
2017-04-17 20:02:00 +02:00
Matt Jankowski
fd9d43b6e2 Improve handling of HTTP_ACCEPT for webfinger (#2008)
This change includes:

- Improve the spec coverage for incoming request to the webfinger action
- For requests without an accept header (ie, what a browser might look like),
  return a JSON response.
- For requests with an explicit format of xml or json, return that format.
- For requests using an accept header, return that format.

Also adds failing spec showing webfinger does not return xml, which covers the
issue described in: https://github.com/tootsuite/mastodon/issues/1983
2017-04-17 19:58:03 +02:00
Tomohiro Suwa
a9f4feb735 Fix nil query_username (#2013) 2017-04-17 19:57:02 +02:00
saturday06
abf70be71e Assign user locale on signup (#1982) 2017-04-17 10:29:08 +02:00
Eugen
ba6b4c6e62 Make file attachment on MediaAttachment optional (#1865)
Create MediaAttachment but without actual file download when domain is blocked with reject_media set to true
Clean up old media files when creating a new domain block with reject_media set to true
Return remote_url in media attachments API if local file is not present
Undo domain block action in admin UI
Ability to enable reject_media from admin UI
2017-04-16 12:51:30 +02:00
Matt Jankowski
d4d5135559 View spec fix (#1888)
* Add option to disable verify partial doubles

* Add show_landing_strip? helper method

* Use show_landing_strip? helper in accounts and stream entries views

* Fix naming in view specs
2017-04-16 03:40:33 +02:00
Matt Jankowski
c0f0bcf17d Add password reset for users from admin accounts area (#1841) 2017-04-15 16:44:59 +02:00
Matt Jankowski
f382a4f746 About page contact email (#1839)
* Correct site_contact_email typo

* Separate about more page into partials, add specs
2017-04-15 13:33:25 +02:00
Matt Jankowski
e50dfef6a8 Silence Devise deprecation warning about TestHelpers (#1840)
Devise changed their approach -
3f3ec236bb

This change silences a deprecation warning about `TestHelpers`
2017-04-15 13:30:57 +02:00
Patrick Figel
15b393201e Add recovery code support for two-factor auth (#1773)
* Add recovery code support for two-factor auth

When users enable two-factor auth, the app now generates ten
single-use recovery codes. Users are encouraged to print the codes
and store them in a safe place.

The two-factor prompt during login now accepts both OTP codes and
recovery codes.

The two-factor settings UI allows users to regenerated lost
recovery codes. Users who have set up two-factor auth prior to
this feature being added can use it to generate recovery codes
for the first time.

Fixes #563 and fixes #987

* Set OTP_SECRET in test enviroment

* add missing .html to view file names
2017-04-15 13:26:03 +02:00
Matt Jankowski
0d9e4aaaaf Account search service refactor (#1791)
* Begin coverage for account search service

* Coverage for hashtag query

* Coverage for calling local vs remote find based on domain presence

* Spec to check that exact matches are not duped

* Coverage of resolve option

* Coverage for account being provided

* Start to refactor account search service

* Isolate query username and domain methods

* Isolate exact_match method

* Extract methods for local and remote results

* Simplify local vs remote and account isoliation

* Extract methods for local and remote results

* Simplify de-dupe of exact match

* Simplify logic to check for non exact remotes

* Cache some methods

* Remove nil from exact_match from results array

* Return exact matches first

* Use find_remote even with no domain

Account.find_local is just an alias for Account.find_remote(user, nil) - so we
can not bother with the conditional here, and call find_remote directly.
2017-04-15 03:17:07 +02:00
ThibG
626e7d1548 Refresh webfinger (#1323)
* Refresh local info for remote accounts when webfinger returns new values

It only refreshes account info if one of the URLs or the public-key changes,
in which cases it refreshes the full info, re-downloading the feeds from that
user.

Some special handling should probably be done when the public key changes,
but I have been unable to find any use for it in Mastodon yet.

* Re-fetch remote users we aren't subscribed to.

This might induce performance issues, we might want to only do that for users
we explicitly attempted to subscribe but failed to.

* Refactor changes

* Do not refresh existing remote account details more than once a day

* Avoid re-fetching webfinger info in tests unless otherwise specified
2017-04-15 03:16:05 +02:00
Matt Jankowski
2a7120b5ff Rename admin/accounts_helper to admin/filter_helper (#1816)
- Add some spec coverage for both methods
- Add explicit constant call-outs for where the params are from
2017-04-15 03:09:55 +02:00
Ben Roberts
90aa720d2e add basic microformats tests (#1803)
as suggested, moving to view tests rather than a controller test
replaces https://github.com/tootsuite/mastodon/pull/1786 which i will
close momentary
2017-04-15 02:37:00 +02:00
Chad Pytel
29f169f7f0 Introduce capybara and first feature spec (#1801)
This commit introduces Capybara and the first feature spec.

I focused on coverage for log in for the first feature spec because that would
have prevented a1c63cb01d causing #1236.
2017-04-15 01:21:02 +02:00
Matt Jankowski
20170cb4f3 Improve i18n chooser (#1804)
* Add locale spec with failing locale plus region check

* Use a more accurate locale when supplied by browser headers

Previously we were using a matching option which would use the first locale
available which matched the locale portion, even if a region was specified.

This changes to first try to find an exact match, and then fall back to the
region, and then fall back to the  default.

* Clean up default_locale method
2017-04-15 01:12:39 +02:00
Matt Jankowski
22f1d9099a Admin reports controller improvements (#1714)
* Simplify admin/reports controller filtering for index

* Rename parameter to resolved

* Fix issue where reports view could not access filter_link_to

* Add coverage for admin/reports controller

* DRY up resolution of related reports for target account

* Clean up admin/reports routes

* Add Report#statuses method

* DRY up current account action taken params

* Rubocop styles
2017-04-14 11:10:28 +02:00
Matt Jankowski
49c3000d62 Organize coverage dirs (#1695)
* Add `Presenters` group to SimpleCov configuration

* Move validators to app/validators, add to simplecov config
2017-04-13 16:00:31 +02:00
Matt Jankowski
3beb0c0bef Fix csv export coverage in export spec (#1691) 2017-04-13 15:29:30 +02:00
Matt Jankowski
d8789124ae Clean up well-known routes/controllers (#1649)
* Add request spec for host meta route returning xml

* Add routing spec for xrd routes

* Update well-known routes

* Move webfinger and host-meta actions to their own controllers
2017-04-13 13:09:07 +02:00
Matt Jankowski
26ea2f70bd Admin accounts controller cleanup (#1664)
* Remove unused account_params method in admin/accounts controller

* Introduce AccountFilter to find accounts

* Use AccountFilter in admin/accounts controller

* Use more restful routes admin silence and suspension area

* Add admin/silences and admin/suspensions controllers
2017-04-13 13:04:23 +02:00
Matt Jankowski
0a6cfca260 Settings export refactor (#1646)
* Refactor Export to take an account and know about the export types

* Use Export instance in settings/exports#show
2017-04-13 13:02:02 +02:00
Matt Jankowski
bcf45f1783 Remove pending specs for methods that dont exist (#1658)
* Remote spec for non-existent entry_classes helper method

This method no longer exists, and is handled by a local variable in a partial
instead.

* Remove spec for non-existent Account#ping! method
2017-04-13 02:51:13 +02:00
Eugen
4df3f306fe Fix tests issue introduced in #1607 (#1639) 2017-04-13 02:17:40 +02:00
Matt Jankowski
9d1d8797fd Webfinger resource to extract username from resource string (#1607)
* Add WebfingerResource class to extract usernames

* Use WebfingerResource in xrd#webfinger
2017-04-12 18:22:38 +02:00
Matt Jankowski
bf7306636e Allow import/export of mutes list (#1541)
* Allow export of mutes list

* Allow importing of mutes list

* Refactor to use Settings::Exports::BaseController and DRY up exports code
2017-04-12 18:20:44 +02:00
Henry Smith
23d4232961 Add unit tests for https://github.com/tootsuite/mastodon/pull/1574 (#1584) 2017-04-12 17:16:59 +02:00
Matt Jankowski
ca70fe5ae7 Simplify the way the embed view is created (#1590)
* Add coverage for embedded status view

* Refactor embed view to eliminate @external_links variable
2017-04-12 16:12:42 +02:00
Matt Jankowski
c27253cbb6 Default to json type for webfinger requests (#1583) 2017-04-12 16:03:37 +02:00
Matt Jankowski
9e940ae9a9 Fix issue with intermittent api/v1/notifications failure (#1606)
The spec was checking the activity_id of the activities held in notifications
within the controller.

Because the activities are different models, it is possible that they are
created with the same database IDs, and when they are this spec fails because an
activity which should not count as a match is counted as one.
2017-04-12 15:53:54 +02:00
Matt Jankowski
63e46e27f0 Refactor exports controller (#1567)
* Add basic coverage for settings/exports controller

* Remove unused @account variable from settings/exports controller

* Add coverage for download export actions

* Remove deprecated `render :text` in favor of `send_data` for csv downloads

* Add model to handle exports

* Use Export class in settings/exports controller

* Simplify settings/exports controller methods

* Move settings/export to more restful routes
2017-04-11 22:00:43 +02:00
Matt Jankowski
db979ebcee Imports controller errors (#1553)
* Add spec for settings/imports controller

* Add failing spec for settings/imports#create

* Fix broken imports

* Refactor ImportWorker
2017-04-11 21:40:14 +02:00
Matt Jankowski
a9886524d9 Remove unused methods from StreamEntriesHelper (#1365)
Removes:

- avatar_for_status_url
- relative_time
- reblogged_by_me_class
- favourited_by_me_class
2017-04-11 19:32:02 +02:00
Matt Jankowski
9a30c564b6 Remove unused AtomBuilderHelper (#1364)
This was used in the views/atom/user_stream.xml.ruby file, which no longer is
used.
2017-04-11 19:31:22 +02:00
Matt Jankowski
fcec9fcd99 Pagination improvements (#1445)
* Replace will_paginate with kaminari

* Use #page instead of #paginate in controllers

* Replace will_paginate.page_gap with pagination.truncate in i18n

* Customize kaminari views to match prior styles

* Set kaminari options to match prior behavior

* Replace will_paginate with paginate in views
2017-04-11 01:11:41 +02:00
Eugen
e0977567e7 When avatar/header are GIF, generate static versions (#1428)
* When avatar/header are GIF, generate static versions.
Account API returns "avatar"/"avatar_static", "header"/"header_static"
Static version is the same as original for other cases
Web UI de-animates avatars in toots, lists of users

Fix #441, fix #596, prerequisite for #1064

* Fix JS test

* Add rake task to generate static avatars/headers from GIF ones, add test
2017-04-11 00:38:58 +02:00
Eugen
68f3ce7d0c API param to exclude notification types from response (#1341)
* Add exclude_types param to /api/v1/notifications

* Exclude notification types in web UI through exclude_types in the API
2017-04-10 23:45:29 +02:00
Matt Jankowski
874a240a58 Clean up generation of account webfinger string (#1477)
* Consolidate webfinger string creation under Account#to_webfinger_s

* Introduce Account#local_username_and_domain for consolidation
2017-04-10 22:58:06 +02:00
Matt Jankowski
8d0a4b7200 Admin base controller (#1465)
* Add Admin::BaseController to wrap admin area

Extracts the setting of the `admin` layout and verifying that users are admins
to a common base class for the admin/ controllers.

* Add basic coverage for admin/reports and admin/settings controllers
2017-04-10 21:27:03 +02:00
Eugen
47a3702db4 Fix /api/v1/accounts/update_credentials tests (#1357) 2017-04-09 20:23:14 +02:00
David Celis
6e0b4032b3 Allow users to update their Account in the API (#1179)
* Allow users to update their Account in the API

It would be nice for API clients to be able to allow users to update
their accounts without having to wrap Mastodon in a web view. This patch
adds an API endpoint to let users submit a PATCH for their account.

Signed-off-by: David Celis <me@davidcel.is>

* Add /api/v1/accounts/update_credentials to the API docs

Signed-off-by: David Celis <me@davidcel.is>
2017-04-09 18:33:40 +02:00
Matt Jankowski
3b3671c797 Helper cleanup (#1348)
* Remove unused helper files

* Add coverage for application helper

* Add coverage for StreamEntriesHelper #display_name
2017-04-09 17:11:37 +02:00
Matt Jankowski
41b79ae693 Clean up about page (#1282)
* Add InstancePresenter to expose site details

* Clean up about controller, use instance presenter
2017-04-09 14:47:25 +02:00
Matt Jankowski
641e809eaf Search cleanup (#1333)
* Clean up SQL output in Tag and Account search methods

* Add basic coverage for Tag.search_for

* Add coverage for Account.search_for

* Add coverage for Account.advanced_search_for
2017-04-09 14:45:01 +02:00
Matt Jankowski
27ab3d3d36 Ignore implied formats for catch all route requests (#1340)
A request to `/test` would show the custom 404 page, but a request to
`/test.test` would return a 404 with an empty body.

This change ignores the format on incoming catch all route requests, so that the
html 404 page is returned on these requests.
2017-04-09 14:39:41 +02:00
Eugen
697e4da6cb Fix #1141, fix #1126 - Avatar/profile info fetching (#1215)
* Fix #1141, fix #1126 - Work through UpdateRemoteProfileService for both <feed> and <entry> top-level tags

* Improve code quality, remove line unrelated to fix
2017-04-08 13:26:03 +02:00
Matt Jankowski
0d9a420c1a Use Setting.site_title value for og:site_name occurrences (#1194)
* Add helper method to return Setting.site_title

* Use site_title helper in application layout

* Use site_title value for og:site_name
2017-04-08 12:15:40 +02:00
Eugen
c3f2975fa4 Merge pull request #1183 from thoughtbot/cp-post-status-service-specs
Add specs for PostStatusService
2017-04-07 22:30:28 +02:00
Eugen
29a092d6c4 Merge pull request #1184 from thoughtbot/extract-proper-status
DRY up reblog vs original status check
2017-04-07 22:28:15 +02:00
Joël Quenneville
9fda277193 Implement pending specs on Status
Implement the two pending specs on `Status`: `reblogs_count` and
`favourites_count`.
2017-04-07 15:36:06 -04:00
Chad Pytel
1a82d2bde6 Use I18n for media attachment validation errors
These are currently user facing errors, but are not localized. This adds the
ability for these messages to be localized.
2017-04-07 14:23:18 -04:00
Chad Pytel
6a69f9ac65 Add specs for PostStatusService
This implements all pending specs, and adds additional coverage for the
following functionality:

* Normal status creation
* Creating a reply status
* Creating a sensitive status
* Creating a status with spoiler text
* A status with no spoiler text gets an empty string for spoiler text
* Creating a status with custom visibility
* Creating a status for an application
* Processing mentions
* Processing Hashtags
* Pinging PuSH hubs
* Crawling links
* Attaching media
2017-04-07 14:21:16 -04:00
Joël Quenneville
8a67fb5eca DRY up reblog vs original status check
Checking reblog vs original status was happening in multiple places
across the app. For views, this logic was encapsulated in a helper
method named `proper_status` but in the other layers of the app, the
logic was duplicated.

Because the logic is used at all layers of the app, we extracted it into
a `Status#proper` method on the model and changed all uses of the logic
to use this method. There is now a single source of truth for this
condition.

We added test coverage to untested methods that got refactored.
2017-04-07 14:18:30 -04:00
Chad Pytel
31f01ad592 Add specs for media attachment validations
There are currently not specs for the two media validations that are performed
by `PostStatusService`. This adds specs for the validations that ensure that you
cannot attach more than four files, and that a status cannot have both image and
video attachments.
2017-04-07 12:50:43 -04:00
Eugen Rochko
cd68e54a7d Split SalmonWorker into smaller parts, move profile updating into another job 2017-04-05 21:43:10 +02:00
Eugen Rochko
d9f1e07c6c Fix spec 2017-04-05 03:31:26 +02:00
Eugen
744456cb55 Merge branch 'master' into add_more_tests_to_models 2017-04-05 03:27:38 +02:00
Eugen
4751381a24 Merge pull request #852 from peterkeen/email-whitelist-817
[#817] Add email whitelist
2017-04-05 03:04:58 +02:00
Samy KACIMI
3c5248ca20 Merge branch 'master' into add_more_tests_to_models 2017-04-05 00:43:10 +02:00
Samy KACIMI
630ba25d44 update account_spec 2017-04-05 00:37:23 +02:00
Samy KACIMI
9f80b2dd4e add more tests to models 2017-04-05 00:29:56 +02:00
Kurtis Rainbolt-Greene
d6c7d89053 Quick attempt to get pull requests passing 2017-04-04 12:14:44 -07:00
Pete Keen
f28fcf9080 [#817] Add email whitelist
This adds the ability to filter user signup with a whitelist
instead of or in addition to a blacklist.

Fixes #817
2017-04-04 11:20:15 -04:00
Eugen Rochko
c9ffa7ab1d Add basic logging of who resolved report 2017-04-03 19:35:00 +02:00
Eugen Rochko
244da78105 Import feature for following/blocking lists (addresses #62, #177, #201, #454) 2017-03-30 19:42:33 +02:00
Eugen Rochko
a4ce8b09fb Fix #690 - Webfinger should handle new shortform profile URLs now (nice) 2017-03-28 11:25:43 +02:00
Eugen Rochko
b3cb765a94 Prettier account and stream entry URLs 2017-03-22 19:55:14 +01:00
Eugen Rochko
9adf6e8736 Fix wrongful matching of last period in extended usernames
Fix anchor tags in some wikipedia URLs being matches as a hashtag
2017-03-05 18:08:19 +01:00
Eugen Rochko
0cb5a2a0a7 Add digest e-mails 2017-03-04 00:00:48 +01:00
Kit Redgrave
4554ccd5d0 Mute button progress so far. WIP, doesn't entirely work correctly. 2017-03-01 22:31:21 -06:00
Eugen Rochko
7413dcb28a Fix #555 - Use a better URL parser 2017-02-22 19:35:11 +01:00
Eugen Rochko
2cc31b3194 Adding POST /api/v1/reports API, and a UI for submitting reports 2017-02-14 20:59:26 +01:00
Eugen Rochko
8550d32165 Adding more unit tests. Fixing Salmon slaps XML 2017-02-12 17:30:15 +01:00
Eugen Rochko
f9f8f52fe9 Stop trying to shoehorn all Salmon updates into the poor database-connected
StreamEntry model. Simply render Salmon slaps as they are needed
2017-02-12 01:19:14 +01:00
Eugen Rochko
db53bfd30e Fix semantics of follow requests another slaps 2017-02-11 17:13:11 +01:00
Eugen Rochko
f05f545a58 Improve docs 2017-02-11 16:21:07 +01:00
Eugen Rochko
1e66536797 After FollowService, re-fetch remote account asynchronously, do nothing
if account lock info was up to date, otherwise re-do the FollowService
with now updated information
2017-02-11 14:25:01 +01:00
Eugen Rochko
59c8c2b28a Make follow requests federate 2017-02-11 02:58:00 +01:00
Eugen
7037774d6e Merge pull request #603 from evanminto/activitypub-account
Expose ActivityStreams 2.0 representation of accounts
2017-02-07 02:08:40 +01:00
Evan Minto
8dfcb02bde Add test 2017-02-06 12:14:02 -08:00
Eugen Rochko
d384845469 Fix #61 - Add list of blocked users to the UI; clean up failed push notifications API
Try to fix Travis CI setup
2017-02-05 19:18:11 +01:00
Eugen Rochko
2f126b1225 Removing failed push notification API, make context loads use cache 2017-02-05 17:51:44 +01:00
Eugen Rochko
727d236fcc Cleaning up format of broadcast real-time messages, removing
redis-backed "mentions" timeline as redundant (given notifications)
2017-02-02 00:03:31 +01:00
Eugen Rochko
546423df0f Add tests for new API 2017-01-29 12:35:55 +01:00
Eugen Rochko
d5a3d0aca2 Fix tests 2017-01-29 12:25:10 +01:00
Eugen Rochko
9327d05bf7 API for apps to register for push notifications 2017-01-29 01:30:32 +01:00
Eugen Rochko
cb33960e0e Fix #204, fix #515 - URL truncating is now a style so copypasting is not
affected, replaced onClick handler with onMouseUp/Down to detect text
selection not trigger onClick handler then
2017-01-24 17:05:44 +01:00
Eugen Rochko
77a76d5171 Domain blocks now have varying severity - auto-suspend vs auto-silence 2017-01-23 17:38:38 +01:00
Eugen Rochko
a75cada495 Fix oembed controller test 2017-01-23 13:36:17 +01:00
Eugen Rochko
f748a91ec7 Fix #463 - Fetch and display previews of URLs using OpenGraph tags 2017-01-20 01:00:14 +01:00
Eugen Rochko
5d88ef90c9 Fix tests, add applications to eager loading/cache for statuses, fix
application website validation, don't link to app website if website isn't set,
also comment out animated boost icon from #464 until it's consistent with non-animated version
2017-01-15 14:01:33 +01:00
Eugen Rochko
f2667139ad Adding about/more page with extended information that can be set up by an admin 2017-01-13 20:16:38 +01:00
Effy Elden
ad61a21752 Update the Mastodon repository URL from Gargron/ to tootsuite/ in various places 2017-01-11 13:24:14 +11:00
Eugen Rochko
4293e132d1 Persist UI settings, add missing localizations for German 2017-01-09 14:00:55 +01:00
Jessica Stokes
b115602bb7 "Reblog" -> "boost" in more places
A couple of places were using "reblog" rather than "boost" - this updates them to match the web UI
2017-01-04 18:00:50 -08:00
Eugen Rochko
800f6cf6a3 Fix #390 - fix redirect after sign-up (to login page instead of homepage) 2017-01-04 15:31:25 +01:00
Eugen Rochko
6a20c13009 Add API for retrieving favourites 2016-12-29 20:33:26 +01:00
Eugen Rochko
2bc6e7c96e Add API for retrieving blocked accounts 2016-12-29 20:12:32 +01:00
Eugen Rochko
57f6f80838 Add ability to use remote follow function on other sites 2016-12-29 16:54:54 +01:00
Eugen Rochko
6a54df90c8 Replacing follow requests in the settings area with in-UI column 2016-12-26 21:33:51 +01:00
Eugen Rochko
ef9e827c54 Adding follow requests API 2016-12-26 19:30:45 +01:00
Eugen Rochko
1ed3bb3a02 Don't show statuses to blocked users 2016-12-26 19:13:56 +01:00
Eugen Rochko
302051ffcb Add page for authorizing/rejecting follow requests 2016-12-23 00:04:52 +01:00
Eugen Rochko
238233440f Follow call on locked account creates follow request instead
Reflect "requested" relationship in API and UI
Reflect inability of private posts to be reblogged in the UI
Disable Webfinger for locked accounts
2016-12-22 23:03:57 +01:00
Eugen Rochko
d417da7d3a Private visibility on statuses prevents non-followers from seeing those
Filters out hidden stream entries from Atom feed
Blocks now generate hidden stream entries, can be used to federate blocks
Private statuses cannot be reblogged (generates generic 422 error for now)
POST /api/v1/statuses now takes visibility=(public|unlisted|private) param instead of unlisted boolean
Statuses JSON now contains visibility=(public|unlisted|private) field
2016-12-21 20:04:13 +01:00
Eugen Rochko
9d4f96f440 Removing external hub completely, fix #333 fixing digit-only hashtags,
removing web app capability from non-webapp pages
2016-12-18 12:24:37 +01:00
Eugen Rochko
66e08d880c Improved admin UI 2016-12-13 13:42:10 +01:00
Eugen Rochko
88218c83d5 Add suspend account functionality to admin UI 2016-12-06 18:22:59 +01:00
Eugen Rochko
595f592304 Do not autoplay videos, display play button instead. Use expiring links when using S3. Do not keep originals
for avatars/headers, resize avatars down to 120x120 instead of 300x300. Set cache headers on S3 stuff, also
make it private (aka only accessible via expiring links to prevent hotlinking)
2016-12-04 12:28:10 +01:00
Eugen Rochko
047bdf6fe0 Improve notification model 2016-12-03 20:04:19 +01:00
Eugen Rochko
a8814a19dc Add basic OEmbed provider API, fix #247 2016-11-30 23:01:03 +01:00
Eugen Rochko
8d4ef0b6c3 Per-status control for unlisted mode, also federation for unlisted mode
Fix #233, fix #268
2016-11-30 21:34:59 +01:00
Eugen Rochko
329b2a326d Normalize localizations, add stub for admin/accounts 2016-11-30 15:32:26 +01:00
Eugen Rochko
bee7aeaea5 Unify collection caching code 2016-11-29 15:49:39 +01:00
Eugen Rochko
f37efe8e56 Add simple admin overview of PuSH subscriptions 2016-11-28 18:45:13 +01:00
Eugen Rochko
b5ad0eb4ea Adding embedded PuSH server 2016-11-28 13:36:47 +01:00
Eugen Rochko
23162b2893 Update hub URL and re-subscribe if hub URL changes 2016-11-26 15:18:21 +01:00
Eugen Rochko
30f9e9e624 Remove Neo4J 2016-11-24 23:46:27 +01:00
Eugen Rochko
38025dfea3 Adding unified streamable notifications 2016-11-20 19:39:58 +01:00
Eugen Rochko
c282e5be19 Fix OAuth authorization page and add a spec for it 2016-11-18 23:10:44 +01:00
Eugen Rochko
08ed85b3cf Fix #148 - Devise mailer fixed, test spec added so it won't slip past again 2016-11-17 12:29:11 +01:00
Eugen Rochko
f06907eadd Remove broken test 2016-11-16 19:11:30 +01:00
Eugen Rochko
1b61e404b4 Localizations for most server-side strings 2016-11-16 00:55:33 +01:00
Eugen Rochko
733b5415be Fix #142 - Escape ILIKE special characters from Account.find_remote 2016-11-13 11:27:13 +01:00
Eugen Rochko
a6cd0711b0 Upgrade ruby to 2.3.1 2016-11-12 01:55:33 +01:00
Eugen Rochko
0b03bbad67 Discard misattributed remote statuses, improve timelines filter 2016-11-10 00:15:49 +01:00
Eugen Rochko
d98b43cf56 Move timelines API from statuses to its own controller, add a check for
resources that require a user context vs those that don't (such as public timeline)

/api/v1/statuses/public   -> /api/v1/timelines/public
/api/v1/statuses/home     -> /api/v1/timelines/home
/api/v1/statuses/mentions -> /api/v1/timelines/mentions
/api/v1/statuses/tag/:tag -> /api/v1/timelines/tag/:tag
2016-11-08 23:29:08 +01:00
Eugen Rochko
5ad6611101 Add test for FanOutOnWriteService 2016-11-06 15:56:34 +01:00
Eugen Rochko
f0956d434c Improve @mention regex 2016-11-05 22:09:51 +01:00
Eugen Rochko
cb22dce970 Adding hashtags 2016-11-05 17:13:14 +01:00
Eugen Rochko
082e57fc13 Adding hashtag model 2016-11-04 19:12:59 +01:00
Eugen Rochko
834f3c4388 Add tests for recently added API end points 2016-11-03 18:53:37 +01:00
Kai Schaper
c8cab8f1c9 add/update specs for Button, DisplayName, DropdownMenu 2016-10-20 17:34:44 +02:00
Eugen Rochko
02613aef3b Adding application/jrd+json webfinger resource 2016-10-18 02:54:49 +02:00
Eugen Rochko
2614a47a15 Exclude blocked users from follow suggestions, fix tests 2016-10-15 13:48:38 +02:00
Eugen Rochko
2f130a59ba Merge branch 'kschaper-avatar-component-testing' into development 2016-10-14 02:30:08 +02:00
Eugen Rochko
f74201faf0 Merge branch 'avatar-component-testing' of https://github.com/kschaper/mastodon into kschaper-avatar-component-testing 2016-10-14 02:29:51 +02:00
Eugen Rochko
588cb77e9f E-mail preferences page 2016-10-14 02:28:49 +02:00
Eugen Rochko
84495615b4 Shorten rendered links (strip protocol and www, truncate to 30 chars), redirect
to sign in page after sign up instead of root path which redirects to /about
2016-10-13 16:51:34 +02:00
Kai Schaper
89ccac4522 actually use const 2016-10-12 18:18:55 +02:00
Kai Schaper
ec8754f22e add chai-enzyme; add/improve Avatar component specs 2016-10-12 18:06:18 +02:00
Kai Schaper
4a1e0dbfde use ES6 arrow functions 2016-10-10 22:48:58 +02:00
Kai Schaper
7698f1e833 add sinon; add basic Button component test 2016-10-10 22:48:58 +02:00
Kai Schaper
56f381d45c add jsdom; add basic Avatar component test 2016-10-10 22:48:58 +02:00
Kai Schaper
7d3382f755 set up Mocha/Chai/Enzyme for React component unit testing 2016-10-10 22:48:58 +02:00
Eugen Rochko
33668b91f8 Adding test for ProcessFeedService 2016-10-10 16:03:38 +02:00
Eugen Rochko
9bf5a73968 Adding domain blocks 2016-10-09 14:48:59 +02:00
Eugen Rochko
c3f5dfeabb Adding public timeline 2016-10-07 16:00:11 +02:00
Eugen Rochko
7b9a4af311 API for blocking and unblocking 2016-10-03 18:17:06 +02:00
Eugen Rochko
9d59d7b463 Adding a block model and filter mentions from blocked users (fix #60) 2016-10-03 17:12:13 +02:00
Eugen Rochko
fc198a8b4c Adding e-mail confirmations 2016-10-03 16:51:00 +02:00
Eugen Rochko
6d7290f47c Add API for getting info about authenticated user: /api/v1/accounts/verify_credentials 2016-10-02 16:14:21 +02:00
Eugen Rochko
be86d4e0a3 Preparing feeds for better filtering 2016-10-02 15:28:47 +02:00
Eugen Rochko
b36197c307 Fix tests, update goldfinger to catch XML errors 2016-09-30 23:08:29 +02:00
Eugen Rochko
2603c7e69f Fix tests 2016-09-27 23:23:23 +02:00
Eugen Rochko
e63aebff7a Adding landing page 2016-09-27 23:12:33 +02:00
Eugen Rochko
4f9b7432dd Fix #52 - Add API versioning (v1) 2016-09-27 16:59:08 +02:00
Eugen Rochko
7e14eefc81 Replace logo, fix #57 - delete/unreblog/unfavourite API, fix #45 - app
registration API
2016-09-26 23:56:53 +02:00
Eugen Rochko
c6b0311b86 Fix #54 - Fetch remote accounts by URL from mentions
Fetching atom extracted from FetchRemoteAccountService and FetchRemoteStatusService
into FetchAtomService. Mentions of the constant "http://activityschema.org/collection/public"
skipped as it's not a real URL/user.
2016-09-26 16:44:40 +02:00
Eugen Rochko
e46abc71ca Fix notifications in UI, added new API for fetching account relationships 2016-09-21 22:07:18 +02:00
Eugen Rochko
4bec613897 Fix #24 - Thread resolving for remote statuses
This is a big one, so let me enumerate:

Accounts as well as stream entry pages now contain Link headers that
reference the Atom feed and Webfinger URL for the former and Atom entry
for the latter. So you only need to HEAD those resources to get that
information, no need to download and parse HTML <link>s.

ProcessFeedService will now queue ThreadResolveWorker for each remote
status that it cannot find otherwise. Furthermore, entries are now
processed in reverse order (from bottom to top) in case a newer entry
references a chronologically previous one.

ThreadResolveWorker uses FetchRemoteStatusService to obtain a status
and attach the child status it was queued for to it.

FetchRemoteStatusService looks up the URL, first with a HEAD, tests
if it's an Atom feed, in which case it processes it directly. Next
for Link headers to the Atom feed, in which case that is fetched
and processed. Lastly if it's HTML, it is checked for <link>s to the Atom
feed, and if such is found, that is fetched and processed. The account for
the status is derived from author/name attribute in the XML and the hostname
in the URL (domain). FollowRemoteAccountService and ProcessFeedService
are used.

This means that potentially threads are resolved recursively until a dead-end
is encountered, however it is performed asynchronously over background jobs,
so it should be ok.
2016-09-21 01:50:31 +02:00
Eugen Rochko
608a2bfffc Upgrade to PubSubHubbub 0.4 (removing verify_token) 2016-09-20 02:43:20 +02:00
Eugen Rochko
9863196f7b Small rubocop offences removed 2016-09-18 13:42:24 +02:00
Eugen Rochko
adffc7a495 Fix #43 2016-09-18 12:28:49 +02:00
Eugen Rochko
4f25e3d042 Test case for new api endpoint 2016-09-16 00:27:09 +02:00
Eugen Rochko
ce29624c6d Fixing image upload limits, allowing webm, merge/unmerge events trigger
timeline reload in UI, other small fixes
2016-09-12 18:22:43 +02:00
Eugen Rochko
3cc47beb6e Refactored generation of unique tags, URIs and object URLs into own classes,
as well as formatting of content
2016-09-09 20:04:34 +02:00
Eugen Rochko
85d89b472d Optimized n+1 queries in accounts Atom and HTML views
Added stack trace for SQL queries in development
Removed badly thought out accounts/lookup API
2016-09-08 20:36:01 +02:00
Eugen Rochko
a4cc966476 Removing default avatars 2016-09-08 18:23:59 +02:00
Eugen Rochko
509c18eb13 Fix local follows, 404 in logs 2016-09-08 02:40:51 +02:00
Eugen Rochko
87576e1ab1 Fixing atom feeds for accounts, adding tests that would catch such bugs in future 2016-09-08 00:33:07 +02:00
Eugen Rochko
1d9c886518 Adding enclosures to Atom and statuses JSON 2016-09-05 18:57:59 +02:00
Eugen Rochko
eec0dc46a6 PostStatusService can attach media to status, ProcessFeedService likewise 2016-09-05 18:39:53 +02:00
Eugen Rochko
ae1fac0062 Add API to upload media attachments 2016-09-05 17:46:36 +02:00
Eugen Rochko
413e700fe0 Enhancing test suite but I think the problem might have been caching setting 2016-09-05 01:26:08 +02:00
Eugen Rochko
a289c1d52f Handle delete Salmons, todo: clean up timelines 2016-09-04 14:44:16 +02:00
Eugen Rochko
1022d682dc Normalized data in Redux, fix for asset URLs when rendered outside request 2016-09-04 14:04:26 +02:00
Eugen Rochko
8985f8e66c Fixing more configuration issues with ActionCable 2016-08-18 18:39:35 +02:00
Eugen Rochko
10ba09f546 Upgrade to Rails 5.0.0.1 2016-08-17 17:58:00 +02:00
Eugen Rochko
ff2cbc0753 Remember me enabled by default 2016-03-28 00:06:52 +02:00
Eugen Rochko
c38ad208c3 Fix mentions regex to exclude usernames in Medium URLs 2016-03-27 23:51:18 +02:00
Eugen Rochko
039e6f951c Fix issue with unresolvable usernames 2016-03-27 23:38:46 +02:00
Eugen Rochko
85b00d19b8 Moving Salmon notifications to background processing, fixing mini-profiler
behaviour with Turbolinks enabled, optimizing Rabl for production
2016-03-26 13:42:10 +01:00
Eugen Rochko
a08e724476 Fix subscriptions:clear task, refactor feeds, refactor streamable activites
and atom feed generation to some extent, as well as the way mentions are
stored
2016-03-25 02:13:30 +01:00
Eugen Rochko
c8999a116e Sanitize remote html in atom feeds, API (not just UI), use cached mention
relations on Status#mentions
2016-03-24 12:43:55 +01:00
Eugen Rochko
02e4fb2e06 Only re-download avatar if URL changed (fix #19) 2016-03-22 21:05:23 +01:00
Eugen Rochko
f14f462eaf Adding Turbolinks, adding status posting form on homepage 2016-03-21 18:26:47 +01:00
Eugen Rochko
c28971c70c Fixed the lookup API method 2016-03-21 17:33:04 +01:00
Eugen Rochko
9d55529318 Fix text color in dashboard inputs, sanitize remote status content in UI,
simplify FanOutOnWriteService, add /api/accounts/lookup method
2016-03-21 17:02:16 +01:00
Eugen Rochko
35aafdba96 Ancestors and descendants of statuses 2016-03-21 11:43:21 +01:00
Eugen Rochko
253ab26520 Adding more tests for helpers 2016-03-20 13:50:22 +01:00
Eugen Rochko
b640f35621 Writing out more tests, fixed some bugs 2016-03-20 13:03:06 +01:00
Eugen Rochko
2b116131d7 Adding e-mail notifications about mentions, follows, favourites and reblogs. Fixing another mention recording bug 2016-03-19 19:20:07 +01:00
Eugen Rochko
71ae4dd3d2 Adding public following and followers pages, fix #3 2016-03-19 14:02:30 +01:00
Eugen Rochko
c349200761 Removing some unused gems, adding pagination to profiles, fix #10 2016-03-19 12:49:34 +01:00
Eugen Rochko
d4892ace62 Adding more test stubs 2016-03-19 12:13:47 +01:00
Eugen Rochko
071f64d317 Fix #13, Atom feeds now contain formatted post content 2016-03-19 00:16:04 +01:00
Eugen Rochko
7837afbb5f Removing autogenerated test stubs that were not needed 2016-03-18 23:36:54 +01:00
Eugen Rochko
6fec8afc3f Bind oauth applications to users 2016-03-14 17:49:13 +01:00
Eugen Rochko
3441361568 Adding simple_form, adding profile settings, header image 2016-03-12 20:47:22 +01:00
Eugen Rochko
ab6696e855 Adding doorkeeper, adding a REST API
POST /api/statuses                  Params: status (text contents), in_reply_to_id (optional)
GET  /api/statuses/:id
POST /api/statuses/:id/reblog

GET  /api/accounts/:id
GET  /api/accounts/:id/following
GET  /api/accounts/:id/followers
POST /api/accounts/:id/follow
POST /api/accounts/:id/unfollow

POST /api/follows                  Params: uri (e.g. user@domain)

OAuth authentication is currently disabled, but the API can be used with HTTP Auth.
2016-03-07 12:42:33 +01:00
Eugen Rochko
3824c58853 Adding GNU Public license, adding home timeline, reblog/favourite counters 2016-03-06 17:52:23 +01:00
Eugen Rochko
3b4e04dc32 Fixing some bugs, adding pending test examples 2016-03-05 12:50:59 +01:00
Eugen Rochko
23d08c6749 Changing the use of config constants to the Rails configuration object 2016-02-29 20:06:39 +01:00
Eugen Rochko
0e8f59c16f Refactoring Grape API methods into normal controllers & other things 2016-02-29 19:42:08 +01:00
Eugen Rochko
11ff92c9d7 Adding a test for ReblogService, fixing mentions for remote statuses 2016-02-28 21:22:56 +01:00
Eugen Rochko
47d50b0e39 A lot of fixes from a live test 2016-02-28 15:46:29 +01:00
Eugen Rochko
8fada4fae2 When posting a status or reblogging one, ping hubs about the feed update 2016-02-28 14:33:13 +01:00
Eugen Rochko
ad5ae3f60e Update profile information and download avatar of remote accounts 2016-02-28 14:26:26 +01:00
Eugen Rochko
f16b31f077 Adding a bunch of tests 2016-02-26 15:28:08 +01:00
Eugen Rochko
71fe24096c Adding a Mention model, test stubs 2016-02-25 00:17:01 +01:00
Eugen Rochko
fa33750105 Adding reblogs, favourites, improving atom generation 2016-02-23 19:17:37 +01:00
Eugen Rochko
709c6685a9 Made some progress 2016-02-22 16:00:20 +01:00
Eugen Rochko
9c4856bdb1 Initial commit 2016-02-20 22:53:20 +01:00