目录
一、什么是 word2vec？ .......................................................................................................... 2 二、快速入门 ........................................................................................................................... 3 三、作者八卦 ........................................................................................................................... 4 四、背景知识 ........................................................................................................................... 5 4.1 词向量 ........................................................................................................................ 5 4.2 统计语言模型 ............................................................................................................ 5 4.3 NNLM .......................................................................................................................... 7 4.4 其他 NNLM ................................................................................................................ 9 4.5 Log-Linear 模型 ........................................................................................................... 9 4.6 Log-Bilinear 模型 ...................................................................................................... 10 4.6 层次化 Log-Bilinear 模型......................................................................................... 10 五、模型 ................................................................................................................................. 11 5.1 CBOW ........................................................................................................................ 11 5.2 Skip-Gram .................................................................................................................. 13 5.3 为什么要使用 Hierarchical Softmax 或 Negative Sampling .................................... 16 六、Tricks................................................................................................................................ 17 6.1 指数运算 .................................................................................................................. 17 6.2 按 word 分布随机抽样 ........................................................................................... 18 6.3 哈希编码 .................................................................................................................. 20 6.4 随机数 ...................................................................................................................... 20 6.5 回车符 ...................................................................................................................... 20 6.6 高频词亚采样 .......................................................................................................... 21 七、分布式实现 ..................................................................................................................... 21 八、总结 ................................................................................................................................. 22 参考代码 ................................................................................................................................. 22 参考文献 ................................................................................................................................. 23