图 书 馆
模型建筑应该反映语言结构吗？Should Model Architecture Reflect Linguistic Structure? |
|
| 课程网址: | http://videolectures.net/iclr2016_dyer_model_architecture/ |
| 主讲教师: | Chris Dyer |
| 开课单位: | 卡内基梅隆大学 |
| 开课时间: | 2016-05-27 |
| 课程语种: | 英语 |
| 中文简介: |  |
| 课程简介: | Sequential recurrent neural networks (RNNs) over finite alphabets are remarkably effective models of natural language. RNNs now obtain language modeling results that substantially improve over long-standing state-of-the-art baselines, as well as in various conditional language modeling tasks such as machine translation, image caption generation, and dialogue generation. Despite these impressive results, such models are a priori inappropriate models of language. One point of criticism is that language users create and understand new words all the time, challenging the finite vocabulary assumption. A second is that relationships among words are computed in terms of latent nested structures rather than sequential surface order (Chomsky, 1957; Everaert, Huybregts, Chomsky, Berwick, and Bolhuis, 2015). In this talk I discuss two models that explore the hypothesis that more (a priori) appropriate models of language will lead to better performance on real-world language processing tasks. The first composes sub word units (bytes, characters, or morphemes) into lexical representations, enabling more naturalistic interpretation and generation of novel word forms. The second, which we call recurrent neural network grammars (RNNGs), is a new generative model of sentences that explicitly models nested, hierarchical relationships among words and phrases. RNNGs operate via a recursive syntactic process reminiscent of probabilistic context-free grammar generation, but decisions are parameterized using RNNs that condition on the entire (top-down, left-to-right) syntactic derivation history, greatly relaxing context-free independence assumptions. Experimental results show that RNNGs obtain better results in generating language than models that don’t exploit linguistic structures. |
| 关 键 词: | 模型建筑; 语言结构 |
| 课程来源: | 视频讲座网 |
| 数据采集: | 2020-11-29：yxd |
| 最后编审: | 2020-11-29：yxd |
| 阅读次数: | 22 |