pulibrary / bibdata / 03ba0a89-dbf2-455d-b783-c916d773ed37

91%

main: 90%

Build Type

push

circleci

Committed by sandbergja

Commit Message

Use a Rust function to provide ISO 639-3 language data

Previously, we got this data from the human_languages
gem, which is excellent but slow.  This Rust implementation
is much faster: while indexing a file of ~50,000 records,
the ruby implementation took 19.289 seconds, while the
rust implementation took 0.244 seconds!

This commit also includes a rake task to refresh the
Rust function with new language data as needed.

The new Rust implementation also fixes a small bug in the current
language indexing.  Previously, if a Marc record contained
the ISO 639-3 language code ell (Modern Greek), its language
would be indexed as "Modern Greek (1453-)", but if it
contained the equivalent ISO 639-2b code (much more likely),
it would be indexed as "Greek, Modern (1453-)", leading
materials to be split across the two facets.  In the new
implementation, both are indexed as "Greek, Modern (1453-)".

One tradeoff is that compile times are noticably slower.

Run Details

61 of 69 new or added lines in 4 files covered. (88.41%)

1 existing line in 1 file now uncovered.

7519 of 8249 relevant lines covered (91.15%)

413.83 hits per line