---
phase: 01-poc
plan: 02
type: execute
wave: 2
depends_on:
  - 01-01
files_modified:
  - govcrawler/utils/__init__.py
  - govcrawler/utils/url_norm.py
  - govcrawler/utils/simhash_utils.py
  - govcrawler/parser/__init__.py
  - govcrawler/parser/cleaner.py
  - govcrawler/parser/list_parser.py
  - govcrawler/parser/detail_parser.py
  - govcrawler/parser/extractor.py
  - govcrawler/sites/__init__.py
  - govcrawler/sites/gdqy.py
  - tests/__init__.py
  - tests/conftest.py
  - tests/fixtures/README.md
  - tests/test_url_norm.py
  - tests/test_simhash.py
  - tests/test_cleaner.py
  - tests/test_list_parser.py
  - tests/test_detail_parser.py
  - tests/test_gne_fallback.py
autonomous: true
requirements:
  - PARSE-01
  - PARSE-02
  - PARSE-03
  - PARSE-04
tags:
  - parser
  - dedup
  - parsel
  - gne
  - trafilatura
  - simhash

must_haves:
  truths:
    - "给定一个 gdqy 详情页 HTML，parser 能抽出 title / publish_time / source / content_html / content_text / attachment_urls 六个字段"
    - "当主 XPath 的 title 选择器被换成失效选择器时，GNE 兜底仍能抽出正文（>50 字）"
    - "当 GNE 也失败（content 仍然 <50 字）时，trafilatura 作为二级兜底能返回正文"
    - "content_text 不包含 HTML 标签、script/style 块，且保留段落换行（\\n）"
    - "normalize_url 对 scheme/host/path/query 归一化幂等：同一 URL 归一化两次结果一致"
    - "url_hash 是 normalize_url 后 SHA256 十六进制，64 字符"
    - "content_simhash 对中文 2-gram 产生 16 字符十六进制（64-bit）"
    - "列表页 parser 能从 gdqy szfwj 列表 HTML 中抽出一批 (url, title, publish_time) 三元组"
    - "sites/gdqy.py 导出 SZFWJ_LIST_URL、SZFWJ_SELECTORS、SZFWJ_ATTACHMENT_CSS 供 Plan 03 消费"
  artifacts:
    - path: "govcrawler/utils/url_norm.py"
      provides: "normalize_url + url_hash + sha256_bytes"
      contains: "def normalize_url"
    - path: "govcrawler/utils/simhash_utils.py"
      provides: "content_simhash (text -> 16-hex 64-bit)"
      contains: "def content_simhash"
    - path: "govcrawler/parser/cleaner.py"
      provides: "html_to_text — lxml 块级标签换行"
      contains: "def html_to_text"
    - path: "govcrawler/parser/list_parser.py"
      provides: "parse_list(html, base_url, selectors) -> list[ListItem]"
      contains: "def parse_list"
    - path: "govcrawler/parser/detail_parser.py"
      provides: "parse_detail(html, base_url, selectors) -> DetailFields（含 GNE/trafilatura 兜底）"
      contains: "def parse_detail"
    - path: "govcrawler/parser/extractor.py"
      provides: "gne_extract + trafilatura_extract — 兜底抽取"
      contains: "def gne_extract"
    - path: "govcrawler/sites/gdqy.py"
      provides: "gdqy 站点硬编码 selectors + list_url"
      contains: "SZFWJ_SELECTORS"
    - path: "tests/test_gne_fallback.py"
      provides: "主 XPath 失效时 GNE/trafilatura 兜底的断言"
      contains: "used_fallback"
  key_links:
    - from: "govcrawler/parser/detail_parser.py"
      to: "govcrawler/parser/extractor.py"
      via: "XPath 抽空或正文过短时调用 gne_extract，再失败调用 trafilatura_extract"
      pattern: "gne_extract|trafilatura_extract"
    - from: "govcrawler/parser/detail_parser.py"
      to: "govcrawler/parser/cleaner.py"
      via: "content_html -> html_to_text -> content_text"
      pattern: "html_to_text"
    - from: "govcrawler/parser/list_parser.py"
      to: "govcrawler/sites/gdqy.py"
      via: "Plan 03 把 SZFWJ_SELECTORS 注入 parse_list"
      pattern: "SZFWJ_SELECTORS|selectors"
    - from: "tests/test_gne_fallback.py"
      to: "govcrawler/parser/detail_parser.py"
      via: "把 selectors.title 换成 'h1.nonexistent' 断言 used_fallback=True"
      pattern: "used_fallback"
---

<objective>
交付"纯函数"层：URL 归一化/哈希、SimHash、HTML 清洗、列表页解析、详情页解析（含 GNE 主 + trafilatura 次双兜底）、gdqy 站点硬编码配置。**本 plan 不碰网络、不碰浏览器、不碰 DB、不碰文件系统**，纯函数 + 单元测试。

Purpose: Plan 03 的 Fetcher/Storage/CLI 要把"抓到的 HTML → 结构化字段 → 落盘"连起来。把解析与去重做成无副作用纯函数，Plan 03 只管粘合，单元测试可在 CI 秒级跑完。对应 REQ PARSE-01 / PARSE-02 / PARSE-03 / PARSE-04。

Output: `govcrawler/utils/`、`govcrawler/parser/`、`govcrawler/sites/gdqy.py`、完整 pytest 覆盖。
</objective>

<execution_context>
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
@$HOME/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/01-poc/01-CONTEXT.md
@.planning/phases/01-poc/01-RESEARCH.md

<!-- 不引用 01-01-SUMMARY 因为本 plan 不依赖 Plan 01 的任何运行时产物；仅 Plan 03 会用到 Plan 01 的 SQLAlchemy models/Settings。 -->

<interfaces>
<!-- 本 plan 产出的契约（Plan 03 / 测试会消费）：-->

```python
# govcrawler/utils/url_norm.py
def normalize_url(u: str) -> str: ...       # 强制 https、lower host、去 fragment、sort query、去尾 /
def url_hash(u: str) -> str: ...            # sha256 hex of normalize_url(u)，64 字符
def sha256_bytes(data: bytes) -> str: ...   # Plan 03 附件用
```

```python
# govcrawler/utils/simhash_utils.py
def content_simhash(text: str) -> str: ...  # 16 个 hex 字符（64-bit）；空串返回 "0"*16
```

```python
# govcrawler/parser/cleaner.py
def html_to_text(html_str: str) -> str: ...  # 块级标签后加 \n；entity 解码；去除 script/style
```

```python
# govcrawler/parser/list_parser.py
from dataclasses import dataclass

@dataclass
class ListItem:
    url: str
    title: str
    publish_time_raw: str   # 原始串，不在本 plan 里 parse

def parse_list(html: str, base_url: str, selectors: dict) -> list[ListItem]:
    """selectors keys: row, href, title, date (CSS selectors)."""
```

```python
# govcrawler/parser/detail_parser.py
from dataclasses import dataclass

@dataclass
class DetailFields:
    title: str
    publish_time_raw: str
    source: str
    content_html: str
    content_text: str
    attachment_urls: list[str]
    used_fallback: bool        # True if GNE or trafilatura triggered
    fallback_engine: str | None  # "gne" | "trafilatura" | None

def parse_detail(html: str, base_url: str, selectors: dict) -> DetailFields:
    """selectors keys: title, publish_time, source, content, attachment_css"""
```

```python
# govcrawler/parser/extractor.py
def gne_extract(html: str, base_url: str) -> dict: ...
# returns {"title": str, "publish_time": str, "content": str (HTML-ish)}

def trafilatura_extract(html: str, base_url: str) -> dict: ...
# returns same shape; secondary fallback per additional_locked_decisions
```

```python
# govcrawler/sites/gdqy.py
SZFWJ_LIST_URL = "https://www.gdqy.gov.cn/gdqy/newxxgk/fgwj/szfwj/"
SZFWJ_SITE_ID = "gdqy"
SZFWJ_COLUMN_ID = "szfwj"

SZFWJ_LIST_SELECTORS: dict = { ... }     # row/href/title/date
SZFWJ_DETAIL_SELECTORS: dict = { ... }   # title/publish_time/source/content/attachment_css
```
</interfaces>
</context>

<tasks>

<task type="auto">
  <name>Task 1: utils + cleaner 纯函数 + 单测</name>
  <files>
    govcrawler/utils/__init__.py,
    govcrawler/utils/url_norm.py,
    govcrawler/utils/simhash_utils.py,
    govcrawler/parser/__init__.py,
    govcrawler/parser/cleaner.py,
    tests/__init__.py,
    tests/conftest.py,
    tests/fixtures/README.md,
    tests/test_url_norm.py,
    tests/test_simhash.py,
    tests/test_cleaner.py
  </files>
  <read_first>
    - /Users/alphaPlanet/vibecoding/website/.planning/phases/01-poc/01-RESEARCH.md  (§Code Examples · SimHash / §Common Pitfalls #3 URL 归一化 / §Code Examples · HTML → 纯文本)
    - /Users/alphaPlanet/vibecoding/website/.planning/phases/01-poc/01-CONTEXT.md  (§去重 / §Parser 正文清洗)
    - /Users/alphaPlanet/vibecoding/website/.planning/phases/01-poc/01-01-PLAN.md  (frontmatter 的 files_modified，确认 Plan 01 未创建 utils/ 或 parser/ 目录)
  </read_first>
  <action>
  创建三个纯函数模块 + 各自单测。全部用 Write 工具。

  **1. `govcrawler/utils/__init__.py`**：空文件。

  **2. `govcrawler/utils/url_norm.py`** — 照搬 RESEARCH §Code Examples 的 `normalize_url`，加 `url_hash` 和 `sha256_bytes`：
  ```python
  from __future__ import annotations
  import hashlib
  from urllib.parse import urlparse, urlunparse, urlencode, parse_qsl

  ALLOWED_SCHEMES = {"http", "https"}

  def normalize_url(u: str) -> str:
      """强制 https（原 scheme 必须在白名单）、lower host、去 fragment、sort query、去尾 /。
      对非 http/https scheme 抛 ValueError — 防 SRE: file://、javascript:。"""
      u = (u or "").strip()
      p = urlparse(u)
      if p.scheme.lower() not in ALLOWED_SCHEMES:
          raise ValueError(f"refuse non-http scheme: {p.scheme!r}")
      scheme = "https"
      host = p.netloc.lower()
      if not host:
          raise ValueError(f"url missing host: {u!r}")
      path = p.path.rstrip("/") or "/"
      q = urlencode(sorted(parse_qsl(p.query, keep_blank_values=True)))
      return urlunparse((scheme, host, path, "", q, ""))

  def url_hash(u: str) -> str:
      """sha256 hex of normalize_url(u) (64 chars)."""
      return hashlib.sha256(normalize_url(u).encode("utf-8")).hexdigest()

  def sha256_bytes(data: bytes) -> str:
      """For attachment file_hash — Plan 03 consumer."""
      return hashlib.sha256(data).hexdigest()
  ```

  **3. `govcrawler/utils/simhash_utils.py`** — 照搬 RESEARCH §Code Examples · SimHash。用中文 2-gram，不引入 jieba：
  ```python
  from __future__ import annotations
  from simhash import Simhash

  def content_simhash(text: str) -> str:
      """64-bit SimHash, returned as 16 hex chars. 中文 2-gram, 见 RESEARCH #Dont Hand-Roll."""
      if not text:
          return "0" * 16
      # 同时保留英文 word 切分 + 中文 2-gram
      features: list[str] = []
      # 2-gram over full string (works for CJK and fine for mixed text)
      features.extend(text[i:i+2] for i in range(len(text) - 1) if text[i:i+2].strip())
      if not features:
          return "0" * 16
      sh = Simhash(features, f=64)
      return f"{sh.value:016x}"
  ```

  **4. `govcrawler/parser/__init__.py`**：空文件。

  **5. `govcrawler/parser/cleaner.py`** — 基于 RESEARCH §Code Examples · HTML → 纯文本，并显式去除 script/style：
  ```python
  from __future__ import annotations
  from lxml import html as lxml_html

  BLOCK_TAGS = {"p", "div", "br", "li", "tr", "h1", "h2", "h3", "h4", "h5", "h6",
                "section", "article", "blockquote"}
  REMOVE_TAGS = {"script", "style", "noscript", "iframe"}

  def html_to_text(html_str: str) -> str:
      """Convert HTML fragment to paragraph-preserving plain text.
      - Remove script/style/noscript/iframe
      - Insert '\\n' after every block-level tag's tail
      - Collapse runs of blank lines, strip per-line whitespace
      """
      if not html_str or not html_str.strip():
          return ""
      try:
          doc = lxml_html.fragment_fromstring(html_str, create_parent="div")
      except Exception:
          # Fallback — wrap in minimal html; lxml is tolerant
          doc = lxml_html.fromstring(f"<div>{html_str}</div>")
      # Drop script/style subtrees
      for el in list(doc.iter()):
          if el.tag in REMOVE_TAGS:
              parent = el.getparent()
              if parent is not None:
                  parent.remove(el)
      # Tail-inject newlines on block tags
      for el in doc.iter():
          if el.tag in BLOCK_TAGS:
              el.tail = ("\n" + el.tail) if el.tail else "\n"
      text = doc.text_content()
      lines = [ln.strip() for ln in text.splitlines()]
      return "\n".join(ln for ln in lines if ln)
  ```

  **6. `tests/__init__.py`**：空文件。

  **7. `tests/conftest.py`**：
  ```python
  from pathlib import Path
  import pytest

  FIXTURES = Path(__file__).parent / "fixtures"

  @pytest.fixture
  def fixtures_dir() -> Path:
      return FIXTURES
  ```

  **8. `tests/fixtures/README.md`**：说明 gdqy_post_2136593.html 由 Plan 03 的 smoke test 实抓后落盘，本 plan 的测试使用内联 HTML 字符串（不依赖网络）。

  **9. `tests/test_url_norm.py`**：
  ```python
  import pytest
  from govcrawler.utils.url_norm import normalize_url, url_hash, sha256_bytes

  class TestNormalize:
      def test_idempotent(self):
          u = "HTTP://Www.Gdqy.Gov.CN/a/b/?b=2&a=1#frag"
          once = normalize_url(u)
          assert normalize_url(once) == once

      def test_scheme_forced_https(self):
          assert normalize_url("http://www.gdqy.gov.cn/").startswith("https://")

      def test_host_lowered(self):
          assert "www.gdqy.gov.cn" in normalize_url("HTTPS://WWW.GDQY.GOV.CN/x")

      def test_trailing_slash_stripped(self):
          assert normalize_url("https://a.com/x/") == "https://a.com/x"

      def test_root_slash_preserved(self):
          assert normalize_url("https://a.com/") == "https://a.com/"

      def test_fragment_dropped(self):
          assert "#" not in normalize_url("https://a.com/x#frag")

      def test_query_sorted(self):
          assert normalize_url("https://a.com/?b=2&a=1") == normalize_url("https://a.com/?a=1&b=2")

      def test_refuse_file_scheme(self):
          with pytest.raises(ValueError):
              normalize_url("file:///etc/passwd")

      def test_refuse_javascript_scheme(self):
              with pytest.raises(ValueError):
                  normalize_url("javascript:alert(1)")

      def test_refuse_empty_host(self):
          with pytest.raises(ValueError):
              normalize_url("https:///x")

  class TestUrlHash:
      def test_64_hex(self):
          h = url_hash("https://www.gdqy.gov.cn/a/b")
          assert len(h) == 64
          assert all(c in "0123456789abcdef" for c in h)

      def test_same_after_normalization(self):
          assert url_hash("http://Www.GDQY.gov.cn/a/b/?b=2&a=1") == \
                 url_hash("https://www.gdqy.gov.cn/a/b?a=1&b=2")

  class TestSha256Bytes:
      def test_known_vector(self):
          # sha256("") = e3b0c44...
          assert sha256_bytes(b"") == \
                 "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
  ```

  **10. `tests/test_simhash.py`**：
  ```python
  from govcrawler.utils.simhash_utils import content_simhash

  def test_empty_returns_zeros():
      assert content_simhash("") == "0" * 16

  def test_hex_16_chars():
      h = content_simhash("这是一段测试正文，用于生成 64-bit SimHash。")
      assert len(h) == 16
      assert all(c in "0123456789abcdef" for c in h)

  def test_same_text_same_hash():
      t = "广州市人民政府关于发布无人驾驶航空器安全管控的公告"
      assert content_simhash(t) == content_simhash(t)

  def test_different_text_usually_different_hash():
      a = content_simhash("完全不同的一段文字 A，讨论的是天气")
      b = content_simhash("完全不同的一段文字 B，讨论的是体育")
      assert a != b  # 2-gram SimHash, 不同主题极不可能碰撞
  ```

  **11. `tests/test_cleaner.py`**：
  ```python
  from govcrawler.parser.cleaner import html_to_text

  def test_empty():
      assert html_to_text("") == ""
      assert html_to_text("   ") == ""

  def test_paragraph_newlines():
      h = "<div><p>第一段</p><p>第二段</p></div>"
      out = html_to_text(h)
      assert "第一段" in out and "第二段" in out
      assert "\n" in out  # 段落分隔

  def test_strip_tags():
      assert "<b>" not in html_to_text("<p>hello <b>world</b></p>")

  def test_drop_script():
      h = "<div><p>keep</p><script>alert(1)</script></div>"
      out = html_to_text(h)
      assert "keep" in out
      assert "alert" not in out

  def test_drop_style():
      h = "<div><style>.x{color:red}</style><p>kept</p></div>"
      out = html_to_text(h)
      assert "kept" in out
      assert "color:red" not in out

  def test_entity_decoded():
      assert "&nbsp;" not in html_to_text("<p>a&nbsp;b</p>")
  ```

  约束：
  - 不引入任何额外运行时依赖（simhash/lxml 已在 Plan 01 pyproject.toml）。
  - `normalize_url` 对非 http/https scheme 抛 `ValueError` — 防御 `file://`, `javascript:`（RESEARCH §Security Domain 中的 Tampering 缓解）。
  </action>
  <verify>
    <automated>python -c "import ast; [ast.parse(open(f).read()) for f in ['govcrawler/utils/url_norm.py','govcrawler/utils/simhash_utils.py','govcrawler/parser/cleaner.py','tests/test_url_norm.py','tests/test_simhash.py','tests/test_cleaner.py']]" && uv run pytest tests/test_url_norm.py tests/test_simhash.py tests/test_cleaner.py -x -q</automated>
  </verify>
  <acceptance_criteria>
    - `uv run pytest tests/test_url_norm.py -x` 全绿（10 个 case）
    - `uv run pytest tests/test_simhash.py -x` 全绿（4 个 case）
    - `uv run pytest tests/test_cleaner.py -x` 全绿（6 个 case）
    - `normalize_url("http://Www.GDQY.gov.cn/a/b/?b=2&a=1")` 返回 `https://www.gdqy.gov.cn/a/b?a=1&b=2`
    - `url_hash(x)` 长度恒为 64
    - `content_simhash("")` 返回 `"0000000000000000"`
    - `content_simhash(text)` 对非空 text 返回 16 个 hex 字符
    - `html_to_text` 能丢掉 `<script>`/`<style>`，保留段落换行
    - `normalize_url("file:///etc/passwd")` 抛 ValueError
  </acceptance_criteria>
  <done>三个纯函数模块可被 import、10+4+6=20 个单测全绿。</done>
</task>

<task type="auto">
  <name>Task 2: list_parser + detail_parser + GNE/trafilatura 兜底 + gdqy selectors</name>
  <files>
    govcrawler/parser/extractor.py,
    govcrawler/parser/list_parser.py,
    govcrawler/parser/detail_parser.py,
    govcrawler/sites/__init__.py,
    govcrawler/sites/gdqy.py,
    tests/test_list_parser.py,
    tests/test_detail_parser.py,
    tests/test_gne_fallback.py
  </files>
  <read_first>
    - /Users/alphaPlanet/vibecoding/website/.planning/phases/01-poc/01-RESEARCH.md  (§Code Examples · parsel 列表/详情 / §Pattern 2 Parser 分层 / §Anti-Patterns 附件 selector)
    - /Users/alphaPlanet/vibecoding/website/.planning/phases/01-poc/01-CONTEXT.md  (§Parser 附件识别 / §specifics 成功标准 4)
    - https://github.com/GeneralNewsExtractor/GeneralNewsExtractor  (GNE API: GeneralNewsExtractor().extract(html, host=...))
    - https://trafilatura.readthedocs.io/  (trafilatura.extract(html, url=..., include_comments=False))
  </read_first>
  <action>
  交付 Parser 主干 + 双兜底 + gdqy 站点配置 + 单测。全部 Write。

  **1. `govcrawler/parser/extractor.py`** — GNE 主兜底 + trafilatura 次兜底：
  ```python
  from __future__ import annotations
  from typing import Any
  import logging

  log = logging.getLogger(__name__)

  # GNE 懒加载 — import 昂贵且首次调用要加载模型文件
  _GNE = None
  def _get_gne():
      global _GNE
      if _GNE is None:
          from gne import GeneralNewsExtractor
          _GNE = GeneralNewsExtractor()
      return _GNE

  def gne_extract(html: str, base_url: str) -> dict[str, Any]:
      """主兜底：GNE 抽正文。失败返回 {}。
      Returns keys: title, publish_time, content (HTML-ish plain from GNE), author."""
      if not html:
          return {}
      try:
          res = _get_gne().extract(html, host=base_url)
          # GNE returns {'title','publish_time','content','author','images'}
          return {
              "title": (res.get("title") or "").strip(),
              "publish_time": (res.get("publish_time") or "").strip(),
              "content": res.get("content") or "",
              "author": (res.get("author") or "").strip(),
          }
      except Exception as e:
          log.warning("gne_extract failed: %s", e)
          return {}

  def trafilatura_extract(html: str, base_url: str) -> dict[str, Any]:
      """次兜底（per user additional_locked_decisions）：trafilatura 抽正文。失败返回 {}。
      trafilatura.extract returns plain text (we use output_format='txt')."""
      if not html:
          return {}
      try:
          import trafilatura
          txt = trafilatura.extract(
              html,
              url=base_url,
              include_comments=False,
              include_tables=True,
              favor_recall=True,
          )
          # metadata
          meta = trafilatura.extract_metadata(html) if hasattr(trafilatura, "extract_metadata") else None
          title = (getattr(meta, "title", "") or "") if meta else ""
          pub = (getattr(meta, "date", "") or "") if meta else ""
          return {
              "title": title.strip() if title else "",
              "publish_time": pub.strip() if pub else "",
              "content": txt or "",  # trafilatura returns plain text; we treat it as content_text
              "author": "",
          }
      except Exception as e:
          log.warning("trafilatura_extract failed: %s", e)
          return {}
  ```

  **2. `govcrawler/parser/list_parser.py`**：
  ```python
  from __future__ import annotations
  from dataclasses import dataclass
  from urllib.parse import urljoin
  from parsel import Selector

  @dataclass
  class ListItem:
      url: str
      title: str
      publish_time_raw: str  # 原始串；datetime 解析留给 Plan 03 的 storage 层

  def parse_list(html: str, base_url: str, selectors: dict) -> list[ListItem]:
      """Extract list rows from a column index page.
      selectors keys:
        row   : CSS for each row element
        href  : CSS relative to row, ending with ::attr(href)
        title : CSS relative to row, ending with ::text
        date  : CSS relative to row, ending with ::text
      """
      sel = Selector(text=html or "")
      items: list[ListItem] = []
      rows = sel.css(selectors["row"])
      for row in rows:
          href = row.css(selectors["href"]).get()
          if not href:
              continue
          title = (row.css(selectors["title"]).get() or "").strip()
          date = (row.css(selectors["date"]).get() or "").strip()
          items.append(ListItem(
              url=urljoin(base_url, href),
              title=title,
              publish_time_raw=date,
          ))
      return items
  ```

  **3. `govcrawler/parser/detail_parser.py`** — 主 XPath → GNE → trafilatura 三级：
  ```python
  from __future__ import annotations
  from dataclasses import dataclass
  from urllib.parse import urljoin
  from parsel import Selector

  from govcrawler.parser.cleaner import html_to_text
  from govcrawler.parser.extractor import gne_extract, trafilatura_extract

  MIN_CONTENT_CHARS = 100   # 主 XPath content_html 短于此值 → 触发 GNE
  MIN_FALLBACK_CHARS = 50   # GNE/trafilatura 正文短于此值 → 认为失败

  @dataclass
  class DetailFields:
      title: str
      publish_time_raw: str
      source: str
      content_html: str
      content_text: str
      attachment_urls: list[str]
      used_fallback: bool
      fallback_engine: str | None  # "gne" | "trafilatura" | None

  def parse_detail(html: str, base_url: str, selectors: dict) -> DetailFields:
      """selectors keys (all CSS):
        title           : ::text
        publish_time    : ::text
        source          : ::text
        content         : root element of article body
        attachment_css  : CSS for <a href> elements inside content
      """
      sel = Selector(text=html or "")
      title = (sel.css(selectors["title"]).get() or "").strip()
      pub = (sel.css(selectors["publish_time"]).get() or "").strip()
      source = (sel.css(selectors.get("source", "nonexistent-do-not-match")).get() or "").strip()
      content_root = sel.css(selectors["content"])
      content_html = content_root.get() or ""

      # 附件抽取（在 content_html 内找链接）
      attach_css = selectors.get("attachment_css") or \
          "a[href$='.pdf'], a[href$='.doc'], a[href$='.docx'], a[href$='.xls'], a[href$='.xlsx'], a[href$='.zip']"
      attach_nodes = content_root.css(f"{attach_css}::attr(href)").getall() if content_root else []

      used_fallback = False
      fallback_engine: str | None = None

      # Decision point 2 (RESEARCH §System Architecture Diagram):
      # 主 XPath content 过短或 title 为空 → GNE
      if len(content_html) < MIN_CONTENT_CHARS or not title:
          gne = gne_extract(html, base_url)
          gne_content = gne.get("content", "") or ""
          if len(gne_content) >= MIN_FALLBACK_CHARS:
              content_html = gne_content
              used_fallback = True
              fallback_engine = "gne"
              if not title:
                  title = gne.get("title", "") or title
              if not pub:
                  pub = gne.get("publish_time", "") or pub
          else:
              # GNE 也失败 → trafilatura（二级兜底 per user locked decision）
              tra = trafilatura_extract(html, base_url)
              tra_content = tra.get("content", "") or ""
              if len(tra_content) >= MIN_FALLBACK_CHARS:
                  content_html = tra_content
                  used_fallback = True
                  fallback_engine = "trafilatura"
                  if not title:
                      title = tra.get("title", "") or title
                  if not pub:
                      pub = tra.get("publish_time", "") or pub

      content_text = html_to_text(content_html)

      attachment_urls: list[str] = []
      seen: set[str] = set()
      for a in attach_nodes:
          if not a:
              continue
          abs_u = urljoin(base_url, a)
          if abs_u in seen:
              continue
          seen.add(abs_u)
          attachment_urls.append(abs_u)

      return DetailFields(
          title=title,
          publish_time_raw=pub,
          source=source,
          content_html=content_html,
          content_text=content_text,
          attachment_urls=attachment_urls,
          used_fallback=used_fallback,
          fallback_engine=fallback_engine,
      )
  ```

  **4. `govcrawler/sites/__init__.py`**：空文件。

  **5. `govcrawler/sites/gdqy.py`** — 硬编码 gdqy szfwj 栏目 selectors（Plan 03 直接 import）：
  ```python
  """Hard-coded selectors for gdqy.gov.cn's 市政府文件 (szfwj) column.
  Phase 2 will migrate to YAML; for Phase 1 PoC we accept hardcoding per CONTEXT.md LOCKED.
  """
  from __future__ import annotations

  SITE_ID = "gdqy"
  SITE_NAME = "清远市人民政府"
  SITE_BASE_URL = "https://www.gdqy.gov.cn"

  COLUMN_ID = "szfwj"
  COLUMN_NAME = "市政府文件"
  COLUMN_CATEGORY = "法规文件"
  COLUMN_LIST_URL = "https://www.gdqy.gov.cn/gdqy/newxxgk/fgwj/szfwj/"

  # 已知测试 URL（无附件）
  TARGET_ARTICLE_URL = "https://www.gdqy.gov.cn/gdqy/newxxgk/fgwj/szfwj/content/post_2136593.html"
  TARGET_ARTICLE_KEY = "post_2136593"

  # 列表页 selectors —— parsel CSS
  # Phase 1 执行者若实测 gdqy 列表结构与下面不符，需要更新（这是 PoC 环节预期的唯一手工调整点）
  LIST_SELECTORS: dict = {
      "row":   "ul.news_list li, ul.list_news li, div.list li",  # 多候选；政务站常用模板
      "href":  "a::attr(href)",
      "title": "a::text",
      "date":  "span.date::text, span.time::text, em::text",
  }

  # 详情页 selectors
  DETAIL_SELECTORS: dict = {
      "title":        "h1.article-title::text, h1::text, div.article-title::text",
      "publish_time": "span.time::text, div.info span::text, em.time::text",
      "source":       "span.source::text, div.info em::text",
      "content":      "div.article-content, div.content, div.TRS_Editor, div.article",
      "attachment_css": (
          "a[href$='.pdf'], a[href$='.doc'], a[href$='.docx'], "
          "a[href$='.xls'], a[href$='.xlsx'], a[href$='.zip'], "
          "a[href*='.pdf?'], a[href*='.doc?'], a[href*='.docx?']"
      ),
  }
  ```

  **6. `tests/test_list_parser.py`**：
  ```python
  from govcrawler.parser.list_parser import parse_list

  HTML = """
  <html><body>
    <ul class="list_news">
      <li><a href="/x/post_1.html">标题 1</a><span class="date">2026-04-10</span></li>
      <li><a href="/x/post_2.html">标题 2</a><span class="date">2026-04-09</span></li>
      <li><!-- no anchor --></li>
    </ul>
  </body></html>
  """

  SEL = {
      "row":   "ul.list_news li",
      "href":  "a::attr(href)",
      "title": "a::text",
      "date":  "span.date::text",
  }

  def test_parse_list_extracts_rows():
      items = parse_list(HTML, "https://www.gdqy.gov.cn/x/", SEL)
      assert len(items) == 2
      assert items[0].url == "https://www.gdqy.gov.cn/x/post_1.html"
      assert items[0].title == "标题 1"
      assert items[0].publish_time_raw == "2026-04-10"

  def test_parse_list_skips_anchorless_row():
      items = parse_list(HTML, "https://www.gdqy.gov.cn/x/", SEL)
      assert all(it.url for it in items)

  def test_parse_list_empty_html():
      assert parse_list("", "https://x/", SEL) == []
  ```

  **7. `tests/test_detail_parser.py`** — 测主 XPath 正常路径：
  ```python
  from govcrawler.parser.detail_parser import parse_detail
  from govcrawler.sites.gdqy import DETAIL_SELECTORS

  SAMPLE_HTML = """
  <html><body>
    <h1 class="article-title">关于无人驾驶航空器安全管控的公告</h1>
    <div class="info">
      <span class="time">2026-04-10 16:34:22</span>
      <span class="source">清远市人民政府</span>
    </div>
    <div class="article-content">
      <p>第一段：为保障 2026 年第四届全国轻型飞机锦标赛航空嘉年华活动期间空域安全，现就相关事项公告如下。</p>
      <p>第二段：管控时间自 2026 年 5 月 1 日起至 5 月 10 日止。</p>
      <p>第三段：管控区域为清远市清城区、清新区以北半径 30 公里内空域。</p>
      <p>附件：<a href="/attach/notice.pdf">公告全文.pdf</a></p>
    </div>
  </body></html>
  """

  def test_main_xpath_path():
      r = parse_detail(SAMPLE_HTML, "https://www.gdqy.gov.cn/x/post_1.html", DETAIL_SELECTORS)
      assert "无人驾驶航空器安全管控" in r.title
      assert "2026-04-10" in r.publish_time_raw
      assert "清远市人民政府" in r.source
      assert "第一段" in r.content_text
      assert r.used_fallback is False
      assert r.fallback_engine is None

  def test_attachment_absolute():
      r = parse_detail(SAMPLE_HTML, "https://www.gdqy.gov.cn/x/post_1.html", DETAIL_SELECTORS)
      assert len(r.attachment_urls) == 1
      assert r.attachment_urls[0].startswith("https://www.gdqy.gov.cn/")
      assert r.attachment_urls[0].endswith("notice.pdf")

  def test_content_text_no_html_tags():
      r = parse_detail(SAMPLE_HTML, "https://www.gdqy.gov.cn/x/post_1.html", DETAIL_SELECTORS)
      assert "<p>" not in r.content_text
      assert "\n" in r.content_text  # paragraph preserved
  ```

  **8. `tests/test_gne_fallback.py`** — 主 XPath 失效 → GNE 兜底；GNE 也不行 → trafilatura（用 monkeypatch 强制模拟双兜底路径）：
  ```python
  import pytest
  from govcrawler.parser import detail_parser
  from govcrawler.parser.detail_parser import parse_detail

  # 注意：这里使用与 SAMPLE_HTML 相似的 HTML，但 detail selectors 指向一个不存在的 class
  BROKEN_SELECTORS = {
      "title":          "h1.nonexistent-class::text",
      "publish_time":   "span.nonexistent-time::text",
      "source":         "span.nonexistent-source::text",
      "content":        "div.nonexistent-content",  # 这会强制触发兜底
      "attachment_css": "a[href$='.pdf']",
  }

  RICH_HTML = """
  <html><head><title>清远市政府公告</title></head><body>
    <article>
      <h1>关于无人驾驶航空器安全管控的公告</h1>
      <div class="post-meta">2026-04-10 清远市人民政府</div>
      <div class="post-body">
        <p>为保障 2026 年第四届全国轻型飞机锦标赛航空嘉年华活动期间空域安全，现就相关事项公告如下。
        管控时间自 2026 年 5 月 1 日起至 5 月 10 日止。管控区域为清远市清城区、清新区以北半径 30 公里内空域。
        请广大无人机爱好者提前做好飞行计划调整，感谢配合。如有疑问请联系清远市公安局治安支队。</p>
        <p>本公告自发布之日起施行，解释权归清远市人民政府办公室。</p>
      </div>
    </article>
  </body></html>
  """

  @pytest.mark.network_free
  def test_gne_fallback_triggered_when_xpath_fails():
      r = parse_detail(RICH_HTML, "https://www.gdqy.gov.cn/x/post_x.html", BROKEN_SELECTORS)
      assert r.used_fallback is True
      assert r.fallback_engine in {"gne", "trafilatura"}
      assert len(r.content_text) > 50
      # GNE/trafilatura 至少应该抽到"无人驾驶航空器"几个字
      assert "无人" in r.content_text or "公告" in r.content_text

  def test_trafilatura_fallback_when_gne_fails(monkeypatch):
      # 强制让 gne_extract 返回空，确保 trafilatura 分支被走到
      monkeypatch.setattr(detail_parser, "gne_extract", lambda html, base_url: {})
      r = parse_detail(RICH_HTML, "https://www.gdqy.gov.cn/x/post_x.html", BROKEN_SELECTORS)
      assert r.used_fallback is True
      assert r.fallback_engine == "trafilatura"
      assert len(r.content_text) > 50

  def test_both_fallbacks_empty_returns_short_content(monkeypatch):
      monkeypatch.setattr(detail_parser, "gne_extract", lambda html, base_url: {})
      monkeypatch.setattr(detail_parser, "trafilatura_extract", lambda html, base_url: {})
      r = parse_detail(RICH_HTML, "https://www.gdqy.gov.cn/x/post_x.html", BROKEN_SELECTORS)
      # Both fallbacks empty — used_fallback stays False because neither "succeeded"
      assert r.used_fallback is False
      assert r.fallback_engine is None
  ```

  约束：
  - GNE 的 import 在模块顶层会拖慢 pytest 收集 — 用懒加载（`_GNE` 全局 + `_get_gne()`）。
  - trafilatura 同样懒导（`import trafilatura` 写在函数里）。
  - detail_parser 的 MIN_CONTENT_CHARS=100、MIN_FALLBACK_CHARS=50 与 RESEARCH §System Architecture Diagram Decision point 2 对齐。
  - 附件抽取用**多候选 CSS**（后缀 + 带 query），对应 RESEARCH §Anti-Patterns 第 2 条。
  </action>
  <verify>
    <automated>python -c "import ast; [ast.parse(open(f).read()) for f in ['govcrawler/parser/extractor.py','govcrawler/parser/list_parser.py','govcrawler/parser/detail_parser.py','govcrawler/sites/gdqy.py','tests/test_list_parser.py','tests/test_detail_parser.py','tests/test_gne_fallback.py']]" && uv run pytest tests/test_list_parser.py tests/test_detail_parser.py tests/test_gne_fallback.py -x -q</automated>
  </verify>
  <acceptance_criteria>
    - `uv run pytest tests/test_list_parser.py -x -q` 3 个 case 全绿
    - `uv run pytest tests/test_detail_parser.py -x -q` 3 个 case 全绿
    - `uv run pytest tests/test_gne_fallback.py -x -q` 3 个 case 全绿，其中至少一个证明 `used_fallback=True`、`fallback_engine in {"gne","trafilatura"}`、`len(content_text) > 50`
    - `govcrawler/sites/gdqy.py` 可 import，且导出 `SITE_ID="gdqy"`, `COLUMN_ID="szfwj"`, `COLUMN_LIST_URL`, `TARGET_ARTICLE_URL`, `LIST_SELECTORS`, `DETAIL_SELECTORS`
    - `parse_detail` 对 SAMPLE_HTML 返回：title 含 "无人驾驶航空器"、publish_time_raw 含 "2026-04-10"、source="清远市人民政府"、attachment_urls=["https://www.gdqy.gov.cn/attach/notice.pdf"]、used_fallback=False
    - `parse_detail` 对 RICH_HTML+BROKEN_SELECTORS 返回 used_fallback=True 且 content_text>50 字（Phase 1 成功标准第 4 条）
  </acceptance_criteria>
  <done>Parser 三层（主 XPath + GNE + trafilatura）就位；gdqy selectors 硬编码可被 Plan 03 import；9 个 pytest case 全绿。</done>
</task>

</tasks>

<threat_model>
## Trust Boundaries

| Boundary | Description |
|----------|-------------|
| Fetched HTML → parser | 对方站点的 HTML 视为不可信输入 |
| Parser output → Plan 03 Storage | parser 的返回视为半信任（URL 可能含非法路径/协议） |

## STRIDE Threat Register

| Threat ID | Category | Component | Disposition | Mitigation Plan |
|-----------|----------|-----------|-------------|-----------------|
| T-01-02-01 | Tampering | `normalize_url` 输入恶意 scheme（`file://`, `javascript:`, `data:`） | mitigate | 显式 scheme 白名单 `{http, https}`，其它抛 ValueError（在 Task 1 的 tests/test_url_norm.py 覆盖） |
| T-01-02-02 | Tampering | `html_to_text` 吞掉 `<script>`/`<style>` 前被 XSS payload 污染 content_text | mitigate | 本阶段 content_text 只落盘 + 写 DB，不会被前端渲染；但仍显式 drop `<script>/<style>/<noscript>/<iframe>`（Task 1 测试覆盖） |
| T-01-02-03 | DoS | lxml 解析巨大 HTML（百 MB）耗尽内存 | accept | Plan 03 的 Fetcher 在 `page.content()` 层面已被 Chromium 的渲染上限限制；本 plan 的 parser 是纯 CPU，不做额外限流（Phase 2 再加） |
| T-01-02-04 | Information Disclosure | GNE/trafilatura 抽取过程中可能把 HTML 内的脚本内容当正文抽出 | mitigate | detail_parser 最后一步始终 `html_to_text(content_html)` 再 drop script/style |
| T-01-02-05 | Tampering | 附件 URL 被构造为 `javascript:` → urljoin 保留原 scheme | mitigate | Plan 03 storage.files 下载前必须调 `normalize_url`，ValueError 则跳过（下一 plan 的 threat model 会复核） |
</threat_model>

<verification>
- Task 1: utils + cleaner 20 个单测全绿
- Task 2: list/detail/fallback 9 个单测全绿
- 全量：`uv run pytest tests/ -x -q` 应报告 29 个 PASSED
- 手工确认：`python -c "from govcrawler.sites.gdqy import *; print(SITE_ID, COLUMN_ID, TARGET_ARTICLE_URL)"` 不报错
</verification>

<success_criteria>
- [ ] `uv run pytest tests/ -x -q` 全绿（≥29 个 case）
- [ ] `govcrawler.parser.detail_parser.parse_detail` 能处理"主 XPath 成功"与"主 XPath 失败 → GNE 兜底"两条路径
- [ ] 当 GNE 返回空，trafilatura 作为第二兜底被触发（monkeypatch 测试证明）
- [ ] `govcrawler.sites.gdqy` 导出足够的常量让 Plan 03 直接 wire CLI
- [ ] 所有公开函数有 type hint；所有模块可 `python -c "import X"` 成功
- [ ] **REQ 覆盖自检**：
  - PARSE-01 → `list_parser.parse_list` + `tests/test_list_parser.py`
  - PARSE-02 → `detail_parser.parse_detail` 主路径 + `tests/test_detail_parser.py`
  - PARSE-03 → `detail_parser.parse_detail` GNE 兜底 + `tests/test_gne_fallback.py`（CONTEXT §specifics 成功标准 4）
  - PARSE-04 → `cleaner.html_to_text` + detail_parser 末尾 `html_to_text(content_html)` + `tests/test_cleaner.py`
</success_criteria>

<output>
完成后创建 `.planning/phases/01-poc/01-02-SUMMARY.md`，记录：
- 实际 pytest 通过数
- GNE 与 trafilatura 在 monkeypatch 下触发路径的验证
- gdqy selectors 的来源（若实际 smoke 时发现 selectors 不匹配真实 gdqy DOM，Plan 03 执行者会调整——记录调整）
- 任何与 RESEARCH §Code Examples 的偏离
</output>
