Compact Language Detector v3 (CLD3) is the successor of CLD2, which was
used in the previous implementation. CLD3 includes improvements since CLD2,
and supports newer compilers. On the other hand, it has additional
requirements and cld3-ruby, the FFI of CLD3 for Ruby, is still new and may
be still inmature.
Though CLD3 is named after CLD2, it is implemented with a neural network
model, different from the old implementation, which is based on a Naïve
Bayesian classifier.
CLD3 supports newer compilers, such as GCC 6. CLD2 is not compatible with
GCC 6 because it assigns negative values to varibales typed unsigned.
(see internal/cld_generated_cjk_uni_prop_80.cc) The support for GCC 6 and
newer compilers are essential today, when some server operating system
such as Ubuntu Server 16.10 has GCC 6 by default.
On the one hand, CLD3 requires C++11 support. Environments with old
compilers such as Ubuntu Server 14.04 needs to update the system or install
a newer compiler.
CLD3 needs protocol buffers as a new dependency. However,it is not
considered problematic because major server operating systems, CentOS and
Ubuntu Server provide them.
The FFI cld3-ruby was written by me (Akihiko Odaki) for use in Mastodon.
It is still new and may be inmature, but confirmed to pass existing tests.
* Travis: Fix bundler_args
`bundler_args` is not functional if using custom install script
in `.travis.yml`. Directly attach the argument to the install script.
* Travis: Run i18n-tasks through bundle exec
Parallel to similar scripts in the same travis config.
More resiliant to changes in configuration.
* Travis: bundler vendor directory
Travis + rvm seems to ignore `.bundle/config` and hence use rvm
global vendor directory by default. Adding `--path` will fix this
and hence make `cache.bundler = true` really functional.
* Travis: disable bundler cache
No significant change in build time. Huge cache. Best disabled.
* Replace browserify with webpack
* Add react-intl-translations-manager
* Do not minify in development, add offline-plugin for ServiceWorker background cache updates
* Adjust tests and dependencies
* Fix production deployments
* Fix tests
* More optimizations
* Improve travis cache for npm stuff
* Re-run travis
* Add back support for custom.scss as before
* Remove offline-plugin and babili
* Fix issue with Immutable.List().unshift(...values) not working as expected
* Make travis load schema instead of running all migrations in sequence
* Fix missing React import in WarningContainer. Optimize rendering performance by using ImmutablePureComponent instead of
React.PureComponent. ImmutablePureComponent uses Immutable.is() to compare props. Replace dynamic callback bindings in
<UI />
* Add react definitions to places that use JSX
* Add Procfile.dev for running rails, webpack and streaming API at the same time
* Convert to "container-based" travis build
Since all ppa sources used are currently whitelisted by Travis, why don't we convert to a "sudo-less" structure that promises a shorter build start time?
* Explicitly disable sudo in Travis
* Fix#2108 - Fix gif uploads
Add specs for media attachment gifv conversion
* Add ffmpeg to travis
* Make travis install ffmpeg, not libav
* Switch travis to trusty
* Rename admin.domain_block to admin.domain_blocks in prep for i18n improvement
* Use implicit controller/action path for i18n in admin/domain_blocks
* Add DomainBlock#accounts has_many
* Avoid i18n health warning for `en` locale by using symbol scope with :count
* Remove unused i18n key: plaintext_secret_html
* Remove unused i18n key two_factor_auth.warning
* Remove final will_paginate i18n keys
* Remove unused key two_factor_auth.recovery_codes
* Remove unused key: admin.reports.comment.none
* Remove unused reports. i18n namespace (moved to admin.reports)
* Ignore keys from locales which override activemodel and activerecord errors
* Revert "Remove unused key: admin.reports.comment.none"
This reverts commit 350ef2685fadc069e619bb6d1066190de195d942.
* Update i18n key reference to match moved location
* Add missing `en` keys to i18n
* Tell i18n-tasks to ignore missing attributes that dont need overwriting
* Add i18n-tasks unused to travis
* Update rspec-rails to version 3.5.2
* Update addressable to version 2.5.1
* Update autoprefixer-rails to version 6.7.7.1
* Update bullet to version 5.5.1
* Update domain_name to version 0.5.20170404
* Update letter_opener_web to version 1.3.1
* Upate redis-rails to version 5.0.2
* Update active_record_query_trace to version 1.5.4
* Update capistrano-rails to version 1.2.3
* Update dotenv-rails to version 2.2.0
* Update pg to version 0.20.0
* Update tilt to version 2.0.7
* Update warden to version 1.2.7
* Update tins to version 1.13.2
* Update terminal-table to version 1.7.3
* Update oj to version 2.18.5
* Update simplecov to version 0.14.1
* Update uglifier to version 3.1.13
* Update hashdiff to version 0.3.2
* Update webmock to version 2.3.2
* Update devise to version 4.2.1
* Use ruby version 2.4.1
* Update sass to version 3.4.23
* Update puma to version 3.8.2
* Update will_paginate to version 3.1.5
* Update font-awesome-rails to version 4.7.0.1
* Update fuubar to version 2.2.0
* Update pry-rails to version 0.3.6
* Update simple-navigation to version 4.0.5
* Update rubocop to version 0.48.1
* Update doorkeeper to version 4.2.5
* Update faker to version 1.7.3
* Update aws-sdk to version 2.9.5
* Update fabrication to version 2.16.1
* Update hamlit-rails to version 0.2.0
* Update http to version 2.2.1
* Update httplog to version 0.99.2
* Update sidekiq to version 4.2.10
* Update rspec-sidekiq to version 3.0.0
* Update pghero to version 1.6.4
* Update rack-cors to version 0.4.1
* Update i18n-tasks to version 0.9.13
* Update ruby-oembed to version 0.12.0
* Update jquery-rails to version 4.3.1
* Update simple_form to version 3.4.0
* Update react-rails to version 1.11.0
* Update aws-sdk to version 2.9.6
* Update sidekiq-unique-jobs to version 5.0.0
* Update uglifier to version 3.2.0
Akihiko Odaki
Koala Yeung
Keiji Matsuzaki
Eugen Rochko
Mingye Wang
Matt Jankowski
Alex Dunn
Andrea Faulds