Language models keep track of complex linguistic information about the preceding context -including, e.g., syntactic relations in a sentence. We investigate whether they also capture information beneficial for resolving pronominal anaphora in English. We analyze two state of the art models with LSTM and Transformer architectures, respectively, using probe tasks on a coreference annotated corpus.
No takes yet. Share an insight, caveat, or question.
Sorodoc et al. (2020) studied this question.