init: edu-content 内容库骨架（L1-L4 四层数据模型 + B类迁移 + lint/CI 工具）

2026-08-22 00:16:49 +08:00
--- a/sources/SCHEMA.md
+++ b/sources/SCHEMA.md
@@ -0,0 +1,66 @@
+# sources/ — L1 资源清单（全库唯一锚点）
+
+`sources/` 回答：**数据从哪来、什么 license、原始文件在 OSS 哪里**。任何原始文件（PDF/ZIP/MP3/原始 JSON）进 OSS 前，必须在此先登记。
+
+## 文件组织
+
+- 一个 YAML 文件 = 一个**域**，内含多条 `source`。
+- 域分三类目录：`china/`（国内课内 + 开源数据集）、`exams/`（考试来源）、`internal/`（自制无来源课件）。
+- 文件名用 kebab-case，如 `cepoc.yaml`、`cambridge-official.yaml`、`smartedu.yaml`。
+
+## 两级结构
+
+```
+source（引用单位，血缘根 id）
+  └─ items[]（具体文件 = OSS 对象）
+```
+
+- L3 结构化记录引用 `source_id`（必填）+ 可选 `source_item_id`（精确到文件）。
+- `internal` 类 source：`url: null`，`license: internal-only`，`oss_key` 指向归档对象。
+
+## 字段（详见 `source.schema.json`）
+
+| 字段 | 必填 | 说明 |
+|---|---|---|
+| `id` | ✅ | 全局唯一 kebab-case，血缘链根 id |
+| `title` | ✅ | 人类可读 |
+| `publisher` | ✅ | 发布方 |
+| `type` | ✅ | `dataset`/`website`/`book`/`internal`/`tool` |
+| `url` | ⚠️ | internal 可 null |
+| `license` | ✅ | 见 `../LICENSES.md` 枚举 |
+| `license_note` | ❌ | 许可说明（如"研究/教育用途"） |
+| `fetch_method` | ❌ | `curl`/`git-clone`/`manual-download`/`manual-export`/`generated` |
+| `status` | ✅ | `planned`→`fetching`→`fetched`/`failed`；`archived` 保留合规记录 |
+| `fetched_at` | ❌ | 下载日期 `YYYY-MM-DD` |
+| `items[]` | ❌ | 具体文件，每项含 `id`/`title`/`oss_key`/`sha256`/`size`/`media_type`/`fetch_date` |
+
+`items[].oss_key` 在 `fetched` 后必填；`sha256`/`size` 由 `tools/oss_sync.py` 上传时自动回填。
+
+## 示例
+
+```yaml
+sources:
+  - id: cepoc
+    title: CEPOC 剑桥机读完形题库
+    publisher: Cambridge ALTA, University of Cambridge
+    type: dataset
+    url: https://github.com/CambridgeALTA/cepoc
+    license: cc-nc-education
+    license_note: "研究/教育用途，非商业（见仓库 licence.txt）"
+    fetch_method: git-clone
+    status: fetched
+    fetched_at: "2026-08-21"
+    items:
+      - id: cepoc-ket-json
+        title: KET.json（6套68空）
+        oss_key: edu-raw/exams/ket/questions/cepoc/KET.json
+        sha256: "a1b2c3..."   # oss_sync.py 自动回填
+        size: 12345
+        media_type: application/json
+        fetch_date: "2026-08-21"
+        status: fetched
+```
+
+## 状态生命周期
+
+`planned`（调研发现，未下载）→ `fetching` → `fetched`/`failed`。调研报告迁移时大量条目初始为 `planned`。`unknown` 状态超过 30 天 lint 报 warning。
--- a/sources/china/ceval.yaml
+++ b/sources/china/ceval.yaml
@@ -0,0 +1,31 @@
+sources:
+  - id: ceval
+    title: C-Eval 中文综合评测集（含 K12 学科）
+    publisher: 上海交大等
+    type: dataset
+    url: https://cevalbenchmark.com/
+    license: cc-nc-education
+    license_note: "⚠️ 仅限非商业研究/教育用途，禁止商用"
+    fetch_method: manual-download
+    status: planned
+    items: []
+  - id: phet
+    title: PhET 互动仿真（科学/数学）
+    publisher: University of Colorado Boulder
+    type: website
+    url: https://phet.colorado.edu/
+    license: cc-by
+    license_note: "CC-BY-4.0，署名即可，可商用"
+    fetch_method: manual-download
+    status: planned
+    items: []
+  - id: web-papers
+    title: 第一试卷网 / 教习网等公开试卷（版权灰）
+    publisher: 各第三方站点（非官方）
+    type: website
+    url: null
+    license: unknown
+    license_note: "版权状态不明，集中登记、默认按最严格处理；未经核查不得进 structured/derived"
+    fetch_method: manual-download
+    status: planned
+    items: []
--- a/sources/china/smartedu.yaml
+++ b/sources/china/smartedu.yaml
@@ -0,0 +1,51 @@
+sources:
+  - id: smartedu
+    title: 国家中小学智慧教育平台（含组卷/课件导出）
+    publisher: 教育部 国家智慧教育公共服务平台
+    type: website
+    url: https://basic.smartedu.cn/
+    license: official-free
+    license_note: "官方平台，教学使用；具体导出内容须遵守平台服务条款"
+    fetch_method: manual-export
+    status: planned
+    items: []
+  - id: tal-scq5k
+    title: TAL-SCQ5K 小学应用题数据集
+    publisher: 好未来
+    type: dataset
+    url: null
+    license: cc-nc-education
+    license_note: "研究/教育用途，非商业（见仓库许可）"
+    fetch_method: manual-download
+    status: planned
+    items: []
+  - id: ape210k
+    title: Ape210K 小学应用题数据集（21万题）
+    publisher: 猿辅导
+    type: dataset
+    url: https://github.com/studentdc/Ape210K
+    license: cc-nc-education
+    license_note: "研究/教育用途，非商业"
+    fetch_method: git-clone
+    status: planned
+    items: []
+  - id: math23k
+    title: Math23K 初中数学应用题数据集
+    publisher: 学术开源
+    type: dataset
+    url: https://github.com/chauncyeu/math23k
+    license: cc-nc-education
+    license_note: "研究/教育用途，非商业"
+    fetch_method: git-clone
+    status: planned
+    items: []
+  - id: cmath
+    title: CMATH 中文小学数学数据集
+    publisher: 学术开源
+    type: dataset
+    url: null
+    license: cc-nc-education
+    license_note: "研究/教育用途，非商业"
+    fetch_method: manual-download
+    status: planned
+    items: []
--- a/sources/exams/amc-past.yaml
+++ b/sources/exams/amc-past.yaml
@@ -0,0 +1,21 @@
+sources:
+  - id: amc-past
+    title: AMC 8 历年真题（含答案）
+    publisher: 美国数学协会 (MAA)
+    type: dataset
+    url: https://artofproblemsolving.com/wiki/index.php/AMC_8_Problems_and_Solutions
+    license: unknown
+    license_note: "AMC 真题版权归 MAA，公开传播受限；内部教研参考，禁止再分发/商用。待核查后定级"
+    fetch_method: manual-download
+    status: planned
+    items: []
+  - id: ccf-csp
+    title: CCF CSP-J/S 历年真题
+    publisher: 中国计算机学会 (CCF)
+    type: dataset
+    url: https://www.csp.thusaac.com/
+    license: official-free
+    license_note: "CCF 官方公开，允许教学使用"
+    fetch_method: manual-download
+    status: planned
+    items: []
--- a/sources/exams/cambridge-official.yaml
+++ b/sources/exams/cambridge-official.yaml
@@ -0,0 +1,45 @@
+sources:
+  - id: cambridge-official
+    title: 剑桥官方 KET/PET 样卷、词表与听力音频
+    publisher: Cambridge Assessment English / UCLES
+    type: dataset
+    url: https://www.cambridgeenglish.org/Images/
+    license: official-free
+    license_note: "官方免费发布，允许教学使用（样卷/词表/听力音频）"
+    fetch_method: manual-download
+    status: fetched
+    fetched_at: "2026-08-21"
+    items:
+      - id: a2-key-vocab
+        title: A2 Key 2020 词汇表 PDF
+        oss_key: edu-raw/exams/ket/vocab/cambridge-official/a2-key-vocabulary-list.pdf
+        sha256: null
+        size: null
+        media_type: application/pdf
+        fetch_date: "2026-08-21"
+        status: planned
+        note: "vocab JSONL (structured/exams/ket/vocab/) 由此清洗；待 oss_sync 上传"
+      - id: b1-preliminary-vocab
+        title: B1 Preliminary 词汇表 PDF
+        oss_key: edu-raw/exams/pet/vocab/cambridge-official/b1-preliminary-vocabulary-list.pdf
+        sha256: null
+        size: null
+        media_type: application/pdf
+        fetch_date: "2026-08-21"
+        status: planned
+      - id: a2-key-listening
+        title: A2 Key 听力音频
+        oss_key: edu-raw/exams/ket/audio/cambridge-official/a2-key-listening.mp3
+        sha256: null
+        size: null
+        media_type: audio/mpeg
+        fetch_date: "2026-08-21"
+        status: planned
+      - id: sample-papers
+        title: 官方样卷包（PDF/MP3/ZIP）
+        oss_key: edu-raw/exams/ket/sample_papers/cambridge-official/
+        sha256: null
+        size: null
+        media_type: application/zip
+        fetch_date: "2026-08-21"
+        status: planned
--- a/sources/exams/cepoc.yaml
+++ b/sources/exams/cepoc.yaml
@@ -0,0 +1,30 @@
+sources:
+  - id: cepoc
+    title: CEPOC 剑桥英语机读完形填空题库（KET/PET/FCE）
+    publisher: Cambridge ALTA, University of Cambridge
+    type: dataset
+    url: https://github.com/CambridgeALTA/cepoc
+    license: cc-nc-education
+    license_note: "研究/教育用途，非商业（见仓库 licence.txt）"
+    fetch_method: git-clone
+    status: fetched
+    fetched_at: "2026-08-21"
+    items:
+      - id: cepoc-ket-json
+        title: KET.json（6套68空）
+        oss_key: edu-raw/exams/ket/questions/cepoc/KET.json
+        sha256: null
+        size: null
+        media_type: application/json
+        fetch_date: "2026-08-21"
+        status: planned
+        note: "latin1 编码；清洗为 qb-1.1 落 structured/exams/ket/questions/cepoc-a2-key.jsonl"
+      - id: cepoc-b1-json
+        title: B1 Preliminary.json（含 FCE 36套360空）
+        oss_key: edu-raw/exams/pet/questions/cepoc/B1-Preliminary.json
+        sha256: null
+        size: null
+        media_type: application/json
+        fetch_date: "2026-08-21"
+        status: planned
+        note: "清洗为 qb-1.1 落 structured/exams/pet/questions/cepoc-b1-preliminary.jsonl"
--- a/sources/internal/qihang-v1.yaml
+++ b/sources/internal/qihang-v1.yaml
@@ -0,0 +1,19 @@
+sources:
+  - id: qihang-v1
+    title: 启航班 V1 课件与素材（pptx/pdf/wav/png）
+    publisher: 内部自制（版权未清）
+    type: internal
+    url: null
+    license: internal-only
+    license_note: "自制课件，仅内部参考，禁止再分发/商用"
+    fetch_method: manual-export
+    status: archived
+    items:
+      - id: qihang-v1-all
+        title: 启航班V1 整包归档
+        oss_key: edu-raw/internal/qihang-v1/all/
+        sha256: null
+        size: null
+        media_type: application/octet-stream
+        fetch_date: null
+        status: planned
--- a/sources/internal/qihang-v2.yaml
+++ b/sources/internal/qihang-v2.yaml
@@ -0,0 +1,19 @@
+sources:
+  - id: qihang-v2
+    title: 启航班 V2 课件与素材
+    publisher: 内部自制（版权未清）
+    type: internal
+    url: null
+    license: internal-only
+    license_note: "自制课件，仅内部参考，禁止再分发/商用"
+    fetch_method: manual-export
+    status: archived
+    items:
+      - id: qihang-v2-all
+        title: 启航班V2 整包归档
+        oss_key: edu-raw/internal/qihang-v2/all/
+        sha256: null
+        size: null
+        media_type: application/octet-stream
+        fetch_date: null
+        status: planned
--- a/sources/internal/qihang-v3.yaml
+++ b/sources/internal/qihang-v3.yaml
@@ -0,0 +1,19 @@
+sources:
+  - id: qihang-v3
+    title: 启航班 V3 课程作品展示站（js/ts 代码工程）
+    publisher: 内部自制（版权未清）
+    type: internal
+    url: null
+    license: internal-only
+    license_note: "自制工程，仅内部参考；若持续演进可未来拆独立 repo"
+    fetch_method: manual-export
+    status: archived
+    items:
+      - id: qihang-v3-all
+        title: 启航班V3 整包归档
+        oss_key: edu-raw/internal/qihang-v3/all/
+        sha256: null
+        size: null
+        media_type: application/octet-stream
+        fetch_date: null
+        status: planned
--- a/sources/internal/zhizaoying-v1.yaml
+++ b/sources/internal/zhizaoying-v1.yaml
@@ -0,0 +1,19 @@
+sources:
+  - id: zhizaoying-v1
+    title: 智造营 V1 录播课与课件（mp4/xlsx/pptx）
+    publisher: 内部自制（版权未清）
+    type: internal
+    url: null
+    license: internal-only
+    license_note: "自制课件，仅内部参考，禁止再分发/商用"
+    fetch_method: manual-export
+    status: archived
+    items:
+      - id: zhizaoying-v1-all
+        title: 智造营V1 整包归档
+        oss_key: edu-raw/internal/zhizaoying-v1/all/
+        sha256: null
+        size: null
+        media_type: application/octet-stream
+        fetch_date: null
+        status: planned
--- a/sources/source.schema.json
+++ b/sources/source.schema.json
@@ -0,0 +1,50 @@
+{
+  "$schema": "http://json-schema.org/draft-07/schema#",
+  "title": "edu-content source registry",
+  "description": "机器可读的 L1 资源清单。一个 YAML 文件 = 一个域，内含多条 source。source 是四层血缘链的根 id。",
+  "type": "object",
+  "required": ["sources"],
+  "additionalProperties": false,
+  "properties": {
+    "sources": {
+      "type": "array",
+      "items": { "$ref": "#/definitions/source" }
+    }
+  },
+  "definitions": {
+    "source": {
+      "type": "object",
+      "required": ["id", "title", "publisher", "type", "license", "status"],
+      "additionalProperties": false,
+      "properties": {
+        "id": { "type": "string", "pattern": "^[a-z0-9][a-z0-9-]*$", "description": "全局唯一 kebab-case，血缘链根 id" },
+        "title": { "type": "string" },
+        "publisher": { "type": "string" },
+        "type": { "type": "string", "enum": ["dataset", "website", "book", "internal", "tool"] },
+        "url": { "type": ["string", "null"], "description": "internal 可为 null" },
+        "license": { "type": "string", "enum": ["official-free", "cc-by", "cc-by-sa", "cc-nc-education", "restricted", "internal-only", "llm-generated", "unknown"] },
+        "license_note": { "type": "string" },
+        "fetch_method": { "type": "string", "enum": ["curl", "git-clone", "manual-download", "manual-export", "generated"], "description": "unknown 允许暂存" },
+        "status": { "type": "string", "enum": ["planned", "fetching", "fetched", "failed", "archived"] },
+        "fetched_at": { "type": ["string", "null"], "pattern": "^\\d{4}-\\d{2}-\\d{2}$|^$" },
+        "items": { "type": "array", "items": { "$ref": "#/definitions/item" } }
+      }
+    },
+    "item": {
+      "type": "object",
+      "required": ["id", "title"],
+      "additionalProperties": false,
+      "properties": {
+        "id": { "type": "string", "pattern": "^[a-z0-9][a-z0-9-]*$", "description": "source 内唯一" },
+        "title": { "type": "string" },
+        "oss_key": { "type": ["string", "null"], "description": "fetched 后必填，形如 edu-raw/exams/ket/vocab/cambridge-official/a2-key-vocab.pdf" },
+        "sha256": { "type": ["string", "null"], "pattern": "^[a-f0-9]{64}$|^$" },
+        "size": { "type": ["integer", "null"], "minimum": 0 },
+        "media_type": { "type": "string" },
+        "fetch_date": { "type": ["string", "null"], "pattern": "^\\d{4}-\\d{2}-\\d{2}$|^$" },
+        "status": { "type": "string", "enum": ["planned", "fetching", "fetched", "failed", "archived"] },
+        "note": { "type": "string" }
+      }
+    }
+  }
+}