> For the complete documentation index, see [llms.txt](https://inwt233.gitbook.io/ai-learning/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://inwt233.gitbook.io/ai-learning/di-yi-bu-fen-nlp-ji-chu/week-02.md).

# 第 2 周：统计语言模型与文本检索

本周从“相邻 Token 的频率”出发，建立概率语言模型与传统文本检索的基础。

## 每日安排

1. [N-gram 语言模型](/ai-learning/di-yi-bu-fen-nlp-ji-chu/week-02/day-008.md)
2. [数据稀疏、平滑与未知事件概率](/ai-learning/di-yi-bu-fen-nlp-ji-chu/week-02/day-009.md)
3. [交叉熵与困惑度](/ai-learning/di-yi-bu-fen-nlp-ji-chu/week-02/day-010.md)
4. [Bag of Words 与 TF-IDF](/ai-learning/di-yi-bu-fen-nlp-ji-chu/week-02/day-011.md)
5. [深入：交叉熵为什么能评价语言模型](/ai-learning/di-yi-bu-fen-nlp-ji-chu/week-02/day-012.md)
6. [深入：BM25 为什么优于简单 TF-IDF](/ai-learning/di-yi-bu-fen-nlp-ji-chu/week-02/day-013.md)
7. [实验：实现一个带 BM25 的小型文档搜索器](/ai-learning/di-yi-bu-fen-nlp-ji-chu/week-02/day-014.md)

## 本周完成标准

* 能从计数推导 N-gram 的最大似然概率。
* 能解释未见事件为什么需要平滑、回退或插值。
* 能计算并解释交叉熵与困惑度。
* 能区分词袋表示、TF-IDF 与 BM25 的核心假设。
* 能实现并评价一个小型文档搜索器。


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://inwt233.gitbook.io/ai-learning/di-yi-bu-fen-nlp-ji-chu/week-02.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.