stanford-oval / genie-toolkit / 1634

54%

master: 56%

Build Type

push

travis-ci-com

Committed by web-flow

Commit Message

Refactor how entities are generated in agent sentences (#453)

The goal of this refactor is two fold:
- We need to generate delexicalized agent utterances at inference
  time to pass to the NLG
- We want the actual text of the entity to be generated by the I18n
  code during postprocessing, so consistent postprocessing is
  applied to both synthetic and NLG sentences, and we can use
  locale-specific code.

To apply this refactoring, the system of inference time constants
is modified to make use of a SequentialEntityAllocator, which allocates
entity tokens (NUMBER_0, DATE_1, etc.) for each value that appears
in the context. This entity allocator is also passed to the Describe
module when that module extracts a value from the program.

At training time, we have only one SequentialEntityAllocator,
which is primed with the fake entities used during generation,
plus any entity that is generated by the simulator and observed
by Describe. At inference time, we reset the entity allocator
immediately prior to processing a new context.

The entities thus allocated are later passed to the I18n module for
postprocessing. This module gained code to automatically select
the most appropriate unit for a given size.

Fixes #447

Run Details

10800 of 23233 branches covered (46.49%)

Branch coverage included in aggregate %.

167 of 213 new or added lines in 13 files covered. (78.4%)

3 existing lines in 3 files now uncovered.

25374 of 43928 relevant lines covered (57.76%)

546053.0 hits per line