pulibrary / bibdata / 01e1c98d-4173-433d-bcfe-5c9df63f1652

90%

push

circleci

Don't index values into cjk_subject field unless they truly have CJK characters (#2936)

* Don't index values into cjk_subject field unless they truly have CJK characters

Our CJK transliteration routines rely on the icu4j RuleBasedTransliterator
class, which can only be run in one thread at a time.  Therefore, we
should not send data through these routines that don't truly need it, or
indexing will be quite slow (especially on Solr 9).

This pr moves cjk_subject indexing to Rust, and uses the nice unicode-blocks
crate to confirm that the cjk_subject values each have at least on CJK
character.

* Create a mock of the figgy mms ids translation map before the first indexing test

Previously we were creating it in config_spec.rb, which works well as long as
we run the tests alphabetically and don't have anything that comes before
config_spec.rb alphabetically that needs an IndexerService.

However, this PR adds a test file called cjk_spec.rb that needs an IndexerService,
so we move the fake translation map creation to be before running that test.

I initially tried using RSpec's when_first_matching_example_defined on the
:indexing tag, but it would not load WebMock within the config block, so I
just went with a before :all.

0 of 79 new or added lines in 2 files covered. (0.0%)

7801 of 8698 relevant lines covered (89.69%)

329.94 hits per line