Currently we're using a list of MIME types for `accept` attribute on `input[type="file"]` for filter options of file picker, and actual file extensions will be infered by browsers. However, infered extensions may not include our expected items. For example, "image/jpeg" seems to be infered to
only ".jfif" extension in Firefox.
To ensure common file extensions are in the list, this PR adds file extensions in addition to MIME types. Also having items in both format is encouraged by HTML5 spec.
https://www.w3.org/TR/html5/forms.html#file-upload-state-(type=file)
Currently we're using a list of MIME types for `accept` attribute on `input[type="file"]` for filter options of file picker, and actual file extensions will be infered by browsers. However, infered extensions may not include our expected items. For example, "image/jpeg" seems to be infered to
only ".jfif" extension in Firefox.
To ensure common file extensions are in the list, this PR adds file extensions in addition to MIME types. Also having items in both format is encouraged by HTML5 spec.
https://www.w3.org/TR/html5/forms.html#file-upload-state-(type=file)
* Fix JavaScript interface with long IDs
Somewhat predictably, the JS interface handled IDs as numbers, which in
JS are IEEE double-precision floats. This loses some precision when
working with numbers as large as those generated by the new ID scheme,
so we instead handle them here as strings. This is relatively simple,
and doesn't appear to have caused any problems, but should definitely
be tested more thoroughly than the built-in tests. Several days of use
appear to support this working properly.
BREAKING CHANGE:
The major(!) change here is that IDs are now returned as strings by the
REST endpoints, rather than as integers. In practice, relatively few
changes were required to make the existing JS UI work with this change,
but it will likely hit API clients pretty hard: it's an entirely
different type to consume. (The one API client I tested, Tusky, handles
this with no problems, however.)
Twitter ran into this issue when introducing Snowflake IDs, and decided
to instead introduce an `id_str` field in JSON responses. I have opted
to *not* do that, and instead force all IDs to 64-bit integers
represented by strings in one go. (I believe Twitter exacerbated their
problem by rolling out the changes three times: once for statuses, once
for DMs, and once for user IDs, as well as by leaving an integer ID
value in JSON. As they said, "If you’re using the `id` field with JSON
in a Javascript-related language, there is a very high likelihood that
the integers will be silently munged by Javascript interpreters. In most
cases, this will result in behavior such as being unable to load or
delete a specific direct message, because the ID you're sending to the
API is different than the actual identifier associated with the
message." [1]) However, given that this is a significant change for API
users, alternatives or a transition time may be appropriate.
1: https://blog.twitter.com/developer/en_us/a/2011/direct-messages-going-snowflake-on-sep-30-2011.html
* Additional fixes for stringified IDs in JSON
These should be the last two. These were identified using eslint to try
to identify any plain casts to JavaScript numbers. (Some such casts are
legitimate, but these were not.)
Adding the following to .eslintrc.yml will identify casts to numbers:
~~~
no-restricted-syntax:
- warn
- selector: UnaryExpression[operator='+'] > :not(Literal)
message: Avoid the use of unary +
- selector: CallExpression[callee.name='Number']
message: Casting with Number() may coerce string IDs to numbers
~~~
The remaining three casts appear legitimate: two casts to array indices,
one in a server to turn an environment variable into a number.
* Back out RelationshipsController Change
This was made to make a test a bit less flakey, but has nothing to
do with this branch.
* Change internal streaming payloads to stringified IDs as well
Per
https://github.com/tootsuite/mastodon/pull/5019#issuecomment-330736452
we need these changes to send deleted status IDs as strings, not
integers.
* Fix JavaScript interface with long IDs
Somewhat predictably, the JS interface handled IDs as numbers, which in
JS are IEEE double-precision floats. This loses some precision when
working with numbers as large as those generated by the new ID scheme,
so we instead handle them here as strings. This is relatively simple,
and doesn't appear to have caused any problems, but should definitely
be tested more thoroughly than the built-in tests. Several days of use
appear to support this working properly.
BREAKING CHANGE:
The major(!) change here is that IDs are now returned as strings by the
REST endpoints, rather than as integers. In practice, relatively few
changes were required to make the existing JS UI work with this change,
but it will likely hit API clients pretty hard: it's an entirely
different type to consume. (The one API client I tested, Tusky, handles
this with no problems, however.)
Twitter ran into this issue when introducing Snowflake IDs, and decided
to instead introduce an `id_str` field in JSON responses. I have opted
to *not* do that, and instead force all IDs to 64-bit integers
represented by strings in one go. (I believe Twitter exacerbated their
problem by rolling out the changes three times: once for statuses, once
for DMs, and once for user IDs, as well as by leaving an integer ID
value in JSON. As they said, "If you’re using the `id` field with JSON
in a Javascript-related language, there is a very high likelihood that
the integers will be silently munged by Javascript interpreters. In most
cases, this will result in behavior such as being unable to load or
delete a specific direct message, because the ID you're sending to the
API is different than the actual identifier associated with the
message." [1]) However, given that this is a significant change for API
users, alternatives or a transition time may be appropriate.
1: https://blog.twitter.com/developer/en_us/a/2011/direct-messages-going-snowflake-on-sep-30-2011.html
* Additional fixes for stringified IDs in JSON
These should be the last two. These were identified using eslint to try
to identify any plain casts to JavaScript numbers. (Some such casts are
legitimate, but these were not.)
Adding the following to .eslintrc.yml will identify casts to numbers:
~~~
no-restricted-syntax:
- warn
- selector: UnaryExpression[operator='+'] > :not(Literal)
message: Avoid the use of unary +
- selector: CallExpression[callee.name='Number']
message: Casting with Number() may coerce string IDs to numbers
~~~
The remaining three casts appear legitimate: two casts to array indices,
one in a server to turn an environment variable into a number.
* Back out RelationshipsController Change
This was made to make a test a bit less flakey, but has nothing to
do with this branch.
* Change internal streaming payloads to stringified IDs as well
Per
https://github.com/tootsuite/mastodon/pull/5019#issuecomment-330736452
we need these changes to send deleted status IDs as strings, not
integers.
* Custom emoji
- In OStatus: `<link rel="emoji" name="coolcat" href="http://..." />`
- In ActivityPub: `{ type: "Emoji", name: ":coolcat:", href: "http://..." }`
- In REST API: Status object includes `emojis` array (`shortcode`, `url`)
- Domain blocks with reject media stop emojis
- Emoji file up to 50KB
- Web UI handles custom emojis
- Static pages render custom emojis as `<img />` tags
Side effects:
- Undo #4500 optimization, as I needed to modify it to restore
shortcode handling in emojify()
- Formatter#plaintext should now make sure stripped out line-breaks
and paragraphs are replaced with newlines
* Fix emoji at the start not being converted
* Custom emoji
- In OStatus: `<link rel="emoji" name="coolcat" href="http://..." />`
- In ActivityPub: `{ type: "Emoji", name: ":coolcat:", href: "http://..." }`
- In REST API: Status object includes `emojis` array (`shortcode`, `url`)
- Domain blocks with reject media stop emojis
- Emoji file up to 50KB
- Web UI handles custom emojis
- Static pages render custom emojis as `<img />` tags
Side effects:
- Undo #4500 optimization, as I needed to modify it to restore
shortcode handling in emojify()
- Formatter#plaintext should now make sure stripped out line-breaks
and paragraphs are replaced with newlines
* Fix emoji at the start not being converted
`Status#reply?` may returns true even if the thread is missing.
e.g. the replied status was deleted or couldn't be fetched.
Then it raises NoMethodError on various AP json serialization.
This issue won't happen on Atom serialization because it checks thread
existence using `StreamEntry#threaded?` instead.
`Status#reply?` may returns true even if the thread is missing.
e.g. the replied status was deleted or couldn't be fetched.
Then it raises NoMethodError on various AP json serialization.
This issue won't happen on Atom serialization because it checks thread
existence using `StreamEntry#threaded?` instead.
* Adjust status embeds
Adjust styles of embed code. Adjust styles of embed pages. Fix overflow of embed-modal.
* Remove trailing whitespace
* Using width from the variable
* Adjust status embeds
Adjust styles of embed code. Adjust styles of embed pages. Fix overflow of embed-modal.
* Remove trailing whitespace
* Using width from the variable
- Fix assumption that `url` is always a string. Handle it if it's an
array of strings, array of objects, object, or string, both for
accounts and for objects
- `sharedInbox` is actually supposed to be under `endpoints`, handle
both cases and adjust the serializer
- Fix assumption that `url` is always a string. Handle it if it's an
array of strings, array of objects, object, or string, both for
accounts and for objects
- `sharedInbox` is actually supposed to be under `endpoints`, handle
both cases and adjust the serializer
Using _: property names is discouraged, as in the future,
canonicalization may throw an error when encountering that instead
of discarding it silently like it does now.
We are defining some ActivityStreams properties which we expect
to land in ActivityStreams eventually, to ensure that future versions
of Mastodon will remain compatible with this even once that happens.
Those would be `locked`, `sensitive` and `Hashtag`
We are defining a custom context inline for some properties which we
do not expect to land in any other context. `atomUri`, `inReplyToAtomUri`
and `conversation` are part of the custom defined OStatus context.
Using _: property names is discouraged, as in the future,
canonicalization may throw an error when encountering that instead
of discarding it silently like it does now.
We are defining some ActivityStreams properties which we expect
to land in ActivityStreams eventually, to ensure that future versions
of Mastodon will remain compatible with this even once that happens.
Those would be `locked`, `sensitive` and `Hashtag`
We are defining a custom context inline for some properties which we
do not expect to land in any other context. `atomUri`, `inReplyToAtomUri`
and `conversation` are part of the custom defined OStatus context.
* Refactor Web::PushSubscription, remove welcome message
* Add missing helper
* Use locale of the receiver on push notifications (#4519)
* Remove unused translations
* Fix dir on notifications
* Refactor Web::PushSubscription, remove welcome message
* Add missing helper
* Use locale of the receiver on push notifications (#4519)
* Remove unused translations
* Fix dir on notifications
- Use statuses controller for embeds instead of stream entries controller
- Prefer /@:username/:id/embed URL for embeds
- Use /@:username as author_url in OEmbed
- Add follow link to embeds which opens web intent in new window
- Use redis cache in development
- Cache entire embed
- Use statuses controller for embeds instead of stream entries controller
- Prefer /@:username/:id/embed URL for embeds
- Use /@:username as author_url in OEmbed
- Add follow link to embeds which opens web intent in new window
- Use redis cache in development
- Cache entire embed
* Process Create / Announce activity in FetchRemoteStatusService
* Use activity URL in ActivityPub for reblogs
* Redirect to the original status on StatusesController#show
* Process Create / Announce activity in FetchRemoteStatusService
* Use activity URL in ActivityPub for reblogs
* Redirect to the original status on StatusesController#show
*Note: OStatus URIs are invalid for ActivityPub. But we have them for
as long as we want to keep old OStatus-sourced content and as long as
we remain OStatus-compatible.*
- In Announce handling, if object URI is not a URL, fallback to object URL
- Do not use specialized ThreadResolveWorker, rely on generalized handling
- When serializing notes, if parent's URI is not a URL, use parent's URL
*Note: OStatus URIs are invalid for ActivityPub. But we have them for
as long as we want to keep old OStatus-sourced content and as long as
we remain OStatus-compatible.*
- In Announce handling, if object URI is not a URL, fallback to object URL
- Do not use specialized ThreadResolveWorker, rely on generalized handling
- When serializing notes, if parent's URI is not a URL, use parent's URL
To be able to pull some basic statistics from a Mastodon instance, include the
user, status and connected domain counters in the `/api/v1/instance` response.
Fixes#3570.
Signed-off-by: Gergely Nagy <algernon@madhouse-project.org>
To be able to pull some basic statistics from a Mastodon instance, include the
user, status and connected domain counters in the `/api/v1/instance` response.
Fixes#3570.
Signed-off-by: Gergely Nagy <algernon@madhouse-project.org>
* Add ActivityPub inbox
* Handle ActivityPub deletes
* Handle ActivityPub creates
* Handle ActivityPub announces
* Stubs for handling all activities that need to be handled
* Add ActivityPub actor resolving
* Handle conversation URI passing in ActivityPub
* Handle content language in ActivityPub
* Send accept header when fetching actor, handle JSON parse errors
* Test for ActivityPub::FetchRemoteAccountService
* Handle public key and icon/image when embedded/as array/as resolvable URI
* Implement ActivityPub::FetchRemoteStatusService
* Add stubs for more interactions
* Undo activities implemented
* Handle out of order activities
* Hook up ActivityPub to ResolveRemoteAccountService, handle
Update Account activities
* Add fragment IDs to all transient activity serializers
* Add tests and fixes
* Add stubs for missing tests
* Add more tests
* Add more tests
* Add ActivityPub inbox
* Handle ActivityPub deletes
* Handle ActivityPub creates
* Handle ActivityPub announces
* Stubs for handling all activities that need to be handled
* Add ActivityPub actor resolving
* Handle conversation URI passing in ActivityPub
* Handle content language in ActivityPub
* Send accept header when fetching actor, handle JSON parse errors
* Test for ActivityPub::FetchRemoteAccountService
* Handle public key and icon/image when embedded/as array/as resolvable URI
* Implement ActivityPub::FetchRemoteStatusService
* Add stubs for more interactions
* Undo activities implemented
* Handle out of order activities
* Hook up ActivityPub to ResolveRemoteAccountService, handle
Update Account activities
* Add fragment IDs to all transient activity serializers
* Add tests and fixes
* Add stubs for missing tests
* Add more tests
* Add more tests
* Improve webfinger templates and make tests more flexible
* Clean up AS2 representation of actor
* Refactor outbox
* Create activities representation
* Add representations of followers/following collections, do not redirect /users/:username route if format is empty
* Remove unused translations
* ActivityPub endpoint for single statuses, add ActivityPub::TagManager for better
URL/URI generation
* Add ActivityPub::TagManager#to
* Represent all attachments as Document instead of Image/Video specifically
(Because for remote ones we may not know for sure)
Add mentions and hashtags representation to AP notes
* Add AP-resolvable hashtag URIs
* Use ActiveModelSerializers for ActivityPub
* Clean up unused translations
* Separate route for object and activity
* Adjust cc/to matrices
* Add to/cc to activities, ensure announce activity embeds target status and
not the wrapper status, add "id" to all collections
* Improve webfinger templates and make tests more flexible
* Clean up AS2 representation of actor
* Refactor outbox
* Create activities representation
* Add representations of followers/following collections, do not redirect /users/:username route if format is empty
* Remove unused translations
* ActivityPub endpoint for single statuses, add ActivityPub::TagManager for better
URL/URI generation
* Add ActivityPub::TagManager#to
* Represent all attachments as Document instead of Image/Video specifically
(Because for remote ones we may not know for sure)
Add mentions and hashtags representation to AP notes
* Add AP-resolvable hashtag URIs
* Use ActiveModelSerializers for ActivityPub
* Clean up unused translations
* Separate route for object and activity
* Adjust cc/to matrices
* Add to/cc to activities, ensure announce activity embeds target status and
not the wrapper status, add "id" to all collections
* Redesign the landing page, mount public timeline on it
* Adjust the standalone mounted component to the lacking of router
* Adjust auth layout pages to new design
* Fix tests
* Standalone public timeline polling every 5 seconds
* Remove now obsolete translations
* Add responsive design for new landing page
* Address reviews
* Add floating clouds behind frontpage form
* Use access token from public page when available
* Fix mentions and hashtags links, cursor on status content in standalone mode
* Add footer link to source code
* Fix errors on pages that don't embed the component, use classnames
* Fix tests
* Change anonymous autoPlayGif default to false
* When gif autoplay is disabled, hover to play
* Add option to hide the timeline preview
* Slightly improve alt layout
* Add elephant friend to new frontpage
* Display "back to mastodon" in place of "login" when logged in on frontpage
* Change polling time to 3s
* Redesign the landing page, mount public timeline on it
* Adjust the standalone mounted component to the lacking of router
* Adjust auth layout pages to new design
* Fix tests
* Standalone public timeline polling every 5 seconds
* Remove now obsolete translations
* Add responsive design for new landing page
* Address reviews
* Add floating clouds behind frontpage form
* Use access token from public page when available
* Fix mentions and hashtags links, cursor on status content in standalone mode
* Add footer link to source code
* Fix errors on pages that don't embed the component, use classnames
* Fix tests
* Change anonymous autoPlayGif default to false
* When gif autoplay is disabled, hover to play
* Add option to hide the timeline preview
* Slightly improve alt layout
* Add elephant friend to new frontpage
* Display "back to mastodon" in place of "login" when logged in on frontpage
* Change polling time to 3s
