diff --git a/.github/FUNDING.yml b/.github/FUNDING.yml new file mode 100644 index 0000000..eb7b21e --- /dev/null +++ b/.github/FUNDING.yml @@ -0,0 +1 @@ +github: NanmiCoder \ No newline at end of file diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..cc26f98 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,45 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## 项目概述 + +这是一个 Python 爬虫教程仓库，包含从入门到高级的爬虫技术教学内容。作者是 MediaCrawler 开源项目的作者。 + +## 常用命令 + +```bash +# 安装依赖 +npm install + +# 启动文档开发服务器 +npm run docs:dev + +# 构建文档 +npm run docs:build + +# 预览构建后的文档 +npm run docs:preview +``` + +## 项目结构 + +- `docs/` - VitePress 文档源文件（Markdown 格式） + - `docs/.vitepress/` - VitePress 配置和主题 + - `docs/爬虫入门/` - 入门教程文档 + - `docs/爬虫进价/` - 进阶教程文档（待完善） + - `docs/高级爬虫/` - 高级教程文档（待完善） +- `源代码/` - 教程对应的 Python 示例代码 + - 每个实战章节有独立目录，包含同步和异步两种实现版本 + +## Python 示例代码依赖 + +示例代码使用以下主要库： +- `httpx` - HTTP 请求库 +- `aiomysql` - 异步 MySQL 客户端 +- `aiofiles` - 异步文件操作 +- `pydantic` - 数据验证 + +## 文档站点 + +在线文档: https://nanmicoder.github.io/CrawlerTutorial/ diff --git a/README.md b/README.md index ed1ed85..54d3111 100644 --- a/README.md +++ b/README.md @@ -11,8 +11,39 @@ 对应的视频链接近期也会同步更新出来，[查看B站合集地址](https://space.bilibili.com/434377496/channel/collectiondetail?sid=4035213&ctype=0) +## 爬虫入门 +- [✔] [01_为什么要写这个爬虫教程](docs/爬虫入门/01_为什么要写这个爬虫教程.md) +- [✔] [02_个人学会爬虫能赚钱吗](docs/爬虫入门/02_个人学会爬虫能赚钱吗.md) +- [✔] [03_网络爬虫到底是什么](docs/爬虫入门/03_网络爬虫到底是什么.md) +- [✔] [04_爬虫的基本工作原理](docs/爬虫入门/04_爬虫的基本工作原理.md) +- [✔] [05_常用的抓包工具有那些](docs/爬虫入门/05_常用的抓包工具有那些.md) +- [✔] [06_为什么说用Python写爬虫有天生优势](docs/爬虫入门/06_为什么说用Python写爬虫有天生优势.md) +- [✔] [07_Python常见的网络请求库](docs/爬虫入门/07_Python常见的网络请求库.md) +- [✔] [08_爬虫入门实战1_静态网页数据提取](docs/爬虫入门/08_爬虫入门实战1_静态网页数据提取.md) +- [✔] [09_爬虫入门实战2_动态数据提取](docs/爬虫入门/09_爬虫入门实战2_动态数据提取.md) +- [✔] [10_爬虫入门实战3_数据存储实现](docs/爬虫入门/10_爬虫入门实战3_数据存储实现.md) +- [✔] [11_爬虫入门实战4_高效率的爬虫实现](docs/爬虫入门/11_爬虫入门实战4_高效率的爬虫实现.md) + +## 爬虫进阶 +- [✔] [01_工程化爬虫开发规范](docs/爬虫进价/01_工程化爬虫开发规范.md) +- [✔] [02_反爬虫对抗基础_请求伪装](docs/爬虫进价/02_反爬虫对抗基础_请求伪装.md) +- [✔] [03_代理IP的使用与管理](docs/爬虫进价/03_代理IP的使用与管理.md) +- [✔] [04_Playwright浏览器自动化入门](docs/爬虫进价/04_Playwright浏览器自动化入门.md) +- [✔] [05_Playwright进阶_反检测与性能优化](docs/爬虫进价/05_Playwright进阶_反检测与性能优化.md) +- [✔] [06_登录认证_Cookie与Session管理](docs/爬虫进价/06_登录认证_Cookie与Session管理.md) +- [✔] [07_登录认证_扫码与短信登录实现](docs/爬虫进价/07_登录认证_扫码与短信登录实现.md) +- [✔] [08_验证码识别与处理](docs/爬虫进价/08_验证码识别与处理.md) +- [✔] [09_数据清洗与预处理](docs/爬虫进价/09_数据清洗与预处理.md) +- [✔] [10_数据分析与可视化](docs/爬虫进价/10_数据分析与可视化.md) +- [✔] [11_进阶综合实战项目](docs/爬虫进价/11_进阶综合实战项目.md) + +## 高级爬虫 +- [✖] 待更新... + ## 爬虫交流群 -可以加作者wx拉进群: yzglan，备注来自github爬虫教程. +扫码加作者企微拉进群，备注来自github爬虫教程 + +企微二维码 ## 免责声明 diff --git a/docs/.vitepress/config.mjs b/docs/.vitepress/config.mjs index a835e40..b9ad440 100644 --- a/docs/.vitepress/config.mjs +++ b/docs/.vitepress/config.mjs @@ -1,7 +1,8 @@ import {defineConfig} from 'vitepress' +import {withMermaid} from 'vitepress-plugin-mermaid' // https://vitepress.dev/reference/site-config -export default defineConfig({ +export default withMermaid(defineConfig({ base: '/CrawlerTutorial/', title: "程序员阿江-Relakkes的爬虫教程", description: "程序员阿江-Relakkes的爬虫教程", @@ -63,6 +64,23 @@ export default defineConfig({ link: '/爬虫入门/12_爬虫入门实战5_编写易于维护的爬虫代码' } ] + }, + { + text: 'Python爬虫进阶', + collapsed: false, + items: [ + {text: '01_工程化爬虫开发规范', link: '/爬虫进价/01_工程化爬虫开发规范'}, + {text: '02_反爬虫对抗基础_请求伪装', link: '/爬虫进价/02_反爬虫对抗基础_请求伪装'}, + {text: '03_代理IP的使用与管理', link: '/爬虫进价/03_代理IP的使用与管理'}, + {text: '04_Playwright浏览器自动化入门', link: '/爬虫进价/04_Playwright浏览器自动化入门'}, + {text: '05_Playwright进阶_反检测与性能优化', link: '/爬虫进价/05_Playwright进阶_反检测与性能优化'}, + {text: '06_登录认证_Cookie与Session管理', link: '/爬虫进价/06_登录认证_Cookie与Session管理'}, + {text: '07_登录认证_扫码与短信登录实现', link: '/爬虫进价/07_登录认证_扫码与短信登录实现'}, + {text: '08_验证码识别与处理', link: '/爬虫进价/08_验证码识别与处理'}, + {text: '09_数据清洗与预处理', link: '/爬虫进价/09_数据清洗与预处理'}, + {text: '10_数据分析与可视化', link: '/爬虫进价/10_数据分析与可视化'}, + {text: '11_进阶综合实战项目', link: '/爬虫进价/11_进阶综合实战项目'} + ] } ], @@ -71,4 +89,4 @@ export default defineConfig({ ] }, -}) +})) diff --git a/docs/.vitepress/theme/DynamicAds.vue b/docs/.vitepress/theme/DynamicAds.vue index a36f427..3a3cbeb 100644 --- a/docs/.vitepress/theme/DynamicAds.vue +++ b/docs/.vitepress/theme/DynamicAds.vue @@ -11,9 +11,9 @@ const fetchAds = async () => { return [ { id: 1, - imageUrl: 'https://nm.zizhi1.com/static/img/40097e36a617f58db1dd132b5841cb1e.ad_pla1.webp', - landingUrl: 'https://nanmicoder.github.io/MediaCrawler/%E4%BD%9C%E8%80%85%E4%BB%8B%E7%BB%8D.html', - text: '⚡️【广告位招租】⚡️投放广告请联系微信:yzglan' + imageUrl: 'https://github.com/NanmiCoder/MediaCrawler/raw/main/docs/static/images/MediaCrawlerPro.jpg', + landingUrl: 'https://github.com/MediaCrawlerPro', + text: '👏欢迎大家来订阅MediaCrawlerPro源代码' } ] } @@ -63,7 +63,8 @@ onUnmounted(() => { } .ad-image { - max-width: 130px; + max-width: 100%; + width: 280px; height: auto; margin-bottom: 0.5rem; } diff --git a/docs/.vitepress/theme/custom.css b/docs/.vitepress/theme/custom.css index fcc8e90..3179728 100644 --- a/docs/.vitepress/theme/custom.css +++ b/docs/.vitepress/theme/custom.css @@ -6,4 +6,5 @@ :root { --vp-sidebar-width: 285px; --vp-sidebar-bg-color: var(--vp-c-bg-alt); -} \ No newline at end of file + --vp-aside-width: 300px; +} diff --git a/docs/index.md b/docs/index.md index cca5c0f..e08ace8 100644 --- a/docs/index.md +++ b/docs/index.md @@ -22,10 +22,27 @@ - [✖] [12_入门实战5_编写易于维护的爬虫代码](爬虫入门/12_爬虫入门实战5_编写易于维护的爬虫代码.md) ## 爬虫进阶 +### 爬虫进阶教程目录大纲 +- [✔] [01_工程化爬虫开发规范](爬虫进价/01_工程化爬虫开发规范.md) +- [✔] [02_反爬虫对抗基础_请求伪装](爬虫进价/02_反爬虫对抗基础_请求伪装.md) +- [✔] [03_代理IP的使用与管理](爬虫进价/03_代理IP的使用与管理.md) +- [✔] [04_Playwright浏览器自动化入门](爬虫进价/04_Playwright浏览器自动化入门.md) +- [✔] [05_Playwright进阶_反检测与性能优化](爬虫进价/05_Playwright进阶_反检测与性能优化.md) +- [✔] [06_登录认证_Cookie与Session管理](爬虫进价/06_登录认证_Cookie与Session管理.md) +- [✔] [07_登录认证_扫码与短信登录实现](爬虫进价/07_登录认证_扫码与短信登录实现.md) +- [✔] [08_验证码识别与处理](爬虫进价/08_验证码识别与处理.md) +- [✔] [09_数据清洗与预处理](爬虫进价/09_数据清洗与预处理.md) +- [✔] [10_数据分析与可视化](爬虫进价/10_数据分析与可视化.md) +- [✔] [11_进阶综合实战项目](爬虫进价/11_进阶综合实战项目.md) + ## 高级爬虫 +### 高级爬虫教程目录大纲 +- [✖] 待更新... ## 爬虫交流群 -可以加作者wx拉进群: yzglan，备注来自github爬虫教程. +扫码加作者企微拉进群，备注来自github爬虫教程 + +企微二维码 ## 免责声明 diff --git a/docs/static/images/QIWEI.png b/docs/static/images/QIWEI.png new file mode 100644 index 0000000..4c33fba Binary files /dev/null and b/docs/static/images/QIWEI.png differ diff --git "a/docs/\347\210\254\350\231\253\350\277\233\344\273\267/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203.md" "b/docs/\347\210\254\350\231\253\350\277\233\344\273\267/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203.md" new file mode 100644 index 0000000..bbee559 --- /dev/null +++ "b/docs/\347\210\254\350\231\253\350\277\233\344\273\267/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203.md" @@ -0,0 +1,1013 @@ +# 工程化爬虫开发规范 + +> 在入门教程中，我们学会了如何写一个能用的爬虫。但在实际生产环境中，一个"能用"的爬虫远远不够。本章将带你从工程化的角度重新审视爬虫开发，让你的爬虫代码更加健壮、可维护、易于调试。 + +> **学习目标**：掌握日志系统、配置管理、异常处理和项目结构设计，为后续进阶学习打下坚实基础。 + +## 为什么需要工程化 + +当你的爬虫从简单的脚本演变为需要长期运行、多人协作的项目时，以下问题会逐渐浮现： + +- **调试困难**：程序出错了，但不知道错在哪里 +- **配置混乱**：API密钥、数据库连接等硬编码在代码中 +- **脆弱性高**：网络波动就导致整个程序崩溃 +- **可读性差**：几个月后自己都看不懂当初写的代码 + +工程化开发规范正是为了解决这些问题而存在的。本章我们将重点学习： + +1. 使用 `loguru` 构建专业的日志系统 +2. 使用 `pydantic-settings` 进行配置管理 +3. 实现统一的异常处理和重试机制 +4. 项目目录结构的最佳实践 + +### 工程化核心模块 + +```mermaid +graph TB + subgraph 工程化核心 + log[日志系统
loguru] + config[配置管理
pydantic-settings] + exception[异常处理
自定义异常+重试] + structure[项目结构
模块化设计] + end + + subgraph 解决的问题 + p1[调试困难] + p2[配置混乱] + p3[脆弱性高] + p4[可读性差] + end + + log -->|解决| p1 + config -->|解决| p2 + exception -->|解决| p3 + structure -->|解决| p4 +``` + +--- + +## 日志系统设计 + +### 为什么需要专业的日志系统 + +在入门教程中，我们大量使用 `print()` 来输出信息。这种方式在调试时很方便，但存在以下问题： + +- 无法区分信息的重要程度（调试信息和错误信息混在一起） +- 无法持久化保存日志 +- 无法在生产环境中动态调整日志级别 +- 无法记录时间、文件位置等上下文信息 + +`loguru` 是一个优秀的 Python 日志库，它用极简的 API 解决了以上所有问题。 + +### loguru 基本使用 + +首先安装 loguru： + +```bash +pip install loguru +``` + +最简单的使用方式： + +```python +from loguru import logger + +logger.debug("这是一条调试信息") +logger.info("这是一条普通信息") +logger.warning("这是一条警告信息") +logger.error("这是一条错误信息") +logger.critical("这是一条严重错误信息") +``` + +运行后你会看到彩色的、带时间戳和文件位置的日志输出： + +``` +2024-03-28 10:30:00.123 | DEBUG | __main__::3 - 这是一条调试信息 +2024-03-28 10:30:00.124 | INFO | __main__::4 - 这是一条普通信息 +2024-03-28 10:30:00.125 | WARNING | __main__::5 - 这是一条警告信息 +2024-03-28 10:30:00.126 | ERROR | __main__::6 - 这是一条错误信息 +2024-03-28 10:30:00.127 | CRITICAL | __main__::7 - 这是一条严重错误信息 +``` + +### 日志分级策略 + +在爬虫项目中，建议按以下方式使用日志级别： + +```mermaid +graph LR + subgraph 日志级别从低到高 + DEBUG[DEBUG
调试信息] --> INFO[INFO
正常信息] + INFO --> WARNING[WARNING
警告信息] + WARNING --> ERROR[ERROR
错误信息] + ERROR --> CRITICAL[CRITICAL
致命错误] + end + + subgraph 爬虫使用场景 + d1[请求参数/响应片段] -.-> DEBUG + d2[爬取第N页/保存N条] -.-> INFO + d3[请求超时重试] -.-> WARNING + d4[单页解析失败] -.-> ERROR + d5[数据库连接失败] -.-> CRITICAL + end +``` + +| 级别 | 使用场景 | 爬虫示例 | +|------|----------|----------| +| DEBUG | 详细的调试信息，如请求参数、响应内容片段 | 请求URL、响应状态码、响应内容片段 | +| INFO | 正常的运行信息，如"开始爬取第N页"、"成功保存N条数据" | 爬取进度、数据保存成功 | +| WARNING | 可恢复的异常，如"请求超时，准备重试" | 请求超时、触发频率限制、需要重试 | +| ERROR | 严重错误但程序可继续，如"单个页面解析失败" | 单个页面解析失败、数据格式异常 | +| CRITICAL | 致命错误，程序无法继续，如"数据库连接失败" | 数据库连接失败、认证失效 | + +### 日志持久化和轮转 + +在生产环境中，我们需要将日志保存到文件以便后续分析。loguru 提供了强大的日志文件管理功能： + +```python +from loguru import logger +import sys + +# 移除默认的控制台输出（可选） +logger.remove() + +# 添加控制台输出，只显示 INFO 及以上级别 +logger.add( + sys.stderr, + level="INFO", + format="{time:YYYY-MM-DD HH:mm:ss} | {level: <8} | {name}:{function}:{line} - {message}" +) + +# 添加文件输出，记录所有级别，按日期轮转 +logger.add( + "logs/crawler_{time:YYYY-MM-DD}.log", + rotation="00:00", # 每天午夜轮转 + retention="7 days", # 保留7天的日志 + compression="zip", # 旧日志压缩 + level="DEBUG", + encoding="utf-8" +) + +# 单独的错误日志文件 +logger.add( + "logs/error_{time:YYYY-MM-DD}.log", + rotation="00:00", + retention="30 days", + level="ERROR", + encoding="utf-8" +) +``` + +### 在爬虫中使用日志的最佳实践 + +#### 通用示例 + +```python +from loguru import logger + +async def fetch_page(client, url: str) -> str: + """获取页面内容""" + logger.debug(f"准备请求: {url}") + + try: + response = await client.get(url) + response.raise_for_status() + logger.info(f"请求成功: {url}, 状态码: {response.status_code}") + return response.text + except httpx.TimeoutException: + logger.warning(f"请求超时: {url}") + raise + except httpx.HTTPStatusError as e: + logger.error(f"HTTP错误: {url}, 状态码: {e.response.status_code}") + raise + except Exception as e: + logger.exception(f"未知错误: {url}") # exception() 会自动记录堆栈信息 + raise +``` + +#### 爬虫日志实战示例 + +以下是一个名言网站爬虫的日志使用示例： + +```python +# quotes_crawler.py - 名言网站爬虫日志示例 +from loguru import logger +import httpx + +async def fetch_quotes(client: httpx.AsyncClient, page: int = 1): + """ + 获取名言列表 + + 展示在实际爬虫中如何使用日志 + """ + url = f"https://quotes.toscrape.com/page/{page}/" + + logger.debug(f"[QuotesCrawler] 请求参数: page={page}") + + try: + response = await client.get(url) + + # HTTP状态码处理 + if response.status_code == 200: + logger.info(f"[QuotesCrawler] 获取成功: 第{page}页") + return response.text + elif response.status_code == 404: + logger.warning(f"[QuotesCrawler] 页面不存在: 第{page}页") + return None + elif response.status_code == 429: + logger.warning(f"[QuotesCrawler] 触发频率限制，需要等待") + raise Exception("触发频率限制") + else: + logger.error(f"[QuotesCrawler] HTTP错误: status_code={response.status_code}") + return None + + except httpx.TimeoutException: + logger.warning(f"[QuotesCrawler] 请求超时: {url}") + raise + except Exception as e: + logger.exception(f"[QuotesCrawler] 未知错误: {e}") + raise +``` + +> **日志前缀约定**：在爬虫项目中，我们使用 `[模块名]` 作为日志前缀，便于在日志文件中快速定位问题来源，如 `[QuotesCrawler]`、`[DataParser]`、`[DataStore]`。 + +--- + +## 配置管理 + +### 为什么需要配置管理 + +硬编码的配置散落在代码各处是一种糟糕的实践： + +```python +# 反面示例 - 不要这样做 +client = httpx.AsyncClient(timeout=30) +db_url = "mysql://root:password123@localhost/crawler" +API_KEY = "sk-xxxxxxxxxxxx" +``` + +这种做法的问题： +- 敏感信息（密码、API密钥）暴露在代码中 +- 不同环境（开发、测试、生产）需要修改代码 +- 配置分散，难以统一管理 + +### pydantic-settings 简介 + +`pydantic-settings` 是 Pydantic 的扩展，专门用于处理应用配置。它支持： + +- 从环境变量读取配置 +- 从 `.env` 文件读取配置 +- 配置值的类型验证 +- 敏感信息的安全处理 + +安装： + +```bash +pip install pydantic-settings +``` + +### 配置管理流程 + +```mermaid +flowchart LR + subgraph 配置来源 + env[环境变量
CRAWLER_XXX] + dotenv[.env 文件
敏感信息] + default[代码默认值] + end + + subgraph pydantic-settings + validate[类型验证] + merge[配置合并] + end + + subgraph 使用 + settings[settings 单例] + app[应用程序] + end + + env --> validate + dotenv --> validate + default --> validate + validate --> merge + merge --> settings + settings --> app +``` + +### 基本使用 + +创建配置类： + +```python +# config/settings.py +from pydantic_settings import BaseSettings +from pydantic import Field +from typing import Optional + + +class CrawlerSettings(BaseSettings): + """爬虫配置""" + + # 基础配置 + debug: bool = Field(default=False, description="调试模式") + log_level: str = Field(default="INFO", description="日志级别") + + # 请求配置 + request_timeout: int = Field(default=30, description="请求超时时间(秒)") + max_retries: int = Field(default=3, description="最大重试次数") + retry_delay: float = Field(default=1.0, description="重试延迟(秒)") + + # 并发配置 + max_concurrent: int = Field(default=10, description="最大并发数") + + # 数据库配置 + db_host: str = Field(default="localhost", description="数据库主机") + db_port: int = Field(default=3306, description="数据库端口") + db_user: str = Field(default="root", description="数据库用户") + db_password: str = Field(default="", description="数据库密码") + db_name: str = Field(default="crawler", description="数据库名") + + # 代理配置 + proxy_url: Optional[str] = Field(default=None, description="代理地址") + + class Config: + env_file = ".env" + env_file_encoding = "utf-8" + # 环境变量前缀，如 CRAWLER_DEBUG=true + env_prefix = "CRAWLER_" + + +# 全局配置实例 +settings = CrawlerSettings() +``` + +### 爬虫配置实战示例 + +以下是名言网站爬虫的配置示例，展示如何组织配置文件： + +```python +# config/settings.py - 爬虫配置 +from pydantic_settings import BaseSettings +from pydantic import Field +from typing import Optional +from enum import Enum + + +class StorageType(str, Enum): + """存储类型""" + JSON = "json" + CSV = "csv" + + +class CrawlerSettings(BaseSettings): + """爬虫配置""" + + # 基础配置 + app_name: str = "QuotesCrawler" + debug: bool = False + log_level: str = "INFO" + + # 请求配置 + request_timeout: int = 30 + max_retries: int = 3 + retry_delay: float = 1.0 + + # 爬虫配置 + base_url: str = "https://quotes.toscrape.com" + max_pages: int = 10 + max_concurrency: int = 3 + crawl_delay_min: float = 0.5 + crawl_delay_max: float = 1.5 + + # 存储配置 + storage_type: StorageType = StorageType.JSON + output_dir: str = "./output" + + class Config: + env_file = ".env" + env_prefix = "CRAWLER_" + + +# 全局配置实例 +settings = CrawlerSettings() +``` + +```python +# config/constants.py - 常量配置 +"""爬虫常量配置""" + +# 默认请求头 +DEFAULT_HEADERS = { + "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) " + "AppleWebKit/537.36 (KHTML, like Gecko) " + "Chrome/120.0.0.0 Safari/537.36", + "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", + "Accept-Language": "zh-CN,zh;q=0.9,en;q=0.8", +} + +# HTTP 状态码 +HTTP_OK = 200 +HTTP_NOT_FOUND = 404 +HTTP_TOO_MANY_REQUESTS = 429 +HTTP_SERVER_ERROR = 500 +``` + +### 使用 .env 文件 + +创建 `.env` 文件（记得添加到 `.gitignore`）： + +```env +# .env +CRAWLER_DEBUG=true +CRAWLER_LOG_LEVEL=DEBUG +CRAWLER_REQUEST_TIMEOUT=60 +CRAWLER_DB_PASSWORD=your_secret_password +CRAWLER_PROXY_URL=http://127.0.0.1:7890 +``` + +在代码中使用配置： + +```python +from config.settings import settings +from loguru import logger + +# 根据配置设置日志级别 +logger.add("logs/app.log", level=settings.log_level) + +# 使用配置创建客户端 +client = httpx.AsyncClient( + timeout=settings.request_timeout, + proxies=settings.proxy_url +) + +logger.info(f"爬虫启动，调试模式: {settings.debug}") +``` + +### 多环境配置 + +对于需要区分开发、测试、生产环境的项目，可以这样组织： + +```python +# config/settings.py +import os +from pydantic_settings import BaseSettings +from pydantic import Field + + +class BaseConfig(BaseSettings): + """基础配置""" + env: str = Field(default="development", description="运行环境") + debug: bool = False + log_level: str = "INFO" + + class Config: + env_file = ".env" + env_prefix = "CRAWLER_" + + +class DevelopmentConfig(BaseConfig): + """开发环境配置""" + debug: bool = True + log_level: str = "DEBUG" + + +class ProductionConfig(BaseConfig): + """生产环境配置""" + debug: bool = False + log_level: str = "WARNING" + + +def get_settings() -> BaseConfig: + """根据环境变量返回对应的配置""" + env = os.getenv("CRAWLER_ENV", "development") + config_map = { + "development": DevelopmentConfig, + "production": ProductionConfig, + } + config_class = config_map.get(env, DevelopmentConfig) + return config_class() + + +settings = get_settings() +``` + +--- + +## 异常处理与重试机制 + +### 异常处理流程 + +```mermaid +flowchart TD + request[发起请求] --> response{响应状态} + + response -->|成功| parse[解析数据] + response -->|超时| retry{重试?} + response -->|HTTP错误| check_code{检查错误码} + + retry -->|是| request + retry -->|否| fail[记录失败] + + check_code -->|429| rate_limit[触发限流
等待后重试] + check_code -->|412| risk_control[触发风控
切换IP/降速] + check_code -->|401/403| auth_error[认证失败
重新登录] + check_code -->|其他| fail + + parse --> save[保存数据] + + rate_limit --> request + risk_control --> request +``` + +### 自定义异常类 + +为爬虫项目定义专门的异常类，便于区分和处理不同类型的错误： + +```python +# exceptions.py + +class CrawlerException(Exception): + """爬虫基础异常类""" + pass + + +class RequestException(CrawlerException): + """请求相关异常""" + pass + + +class ParseException(CrawlerException): + """解析相关异常""" + pass + + +class StorageException(CrawlerException): + """存储相关异常""" + pass + + +class RateLimitException(RequestException): + """触发速率限制""" + pass + + +class IPBlockedException(RequestException): + """IP被封禁""" + pass + + +class LoginRequiredException(CrawlerException): + """需要登录""" + pass +``` + +### 爬虫异常定义实战 + +以下是通用爬虫的异常类设计示例： + +```python +# exceptions/crawler_exceptions.py +"""爬虫异常类""" + + +class CrawlerException(Exception): + """爬虫基础异常""" + pass + + +class RequestException(CrawlerException): + """请求相关异常""" + pass + + +class ParseException(CrawlerException): + """解析相关异常""" + pass + + +class StorageException(CrawlerException): + """存储相关异常""" + pass + + +class RateLimitException(RequestException): + """触发速率限制""" + pass + + +class IPBlockedException(RequestException): + """IP被封禁""" + pass + + +class PageNotFoundException(RequestException): + """页面不存在""" + pass + + +# HTTP 状态码映射 +HTTP_ERROR_MESSAGES = { + 400: "请求参数错误", + 401: "未授权访问", + 403: "禁止访问", + 404: "页面不存在", + 429: "请求过于频繁", + 500: "服务器内部错误", + 502: "网关错误", + 503: "服务暂时不可用", +} +``` + +### 使用 tenacity 实现重试 + +`tenacity` 是一个强大的重试库，比手写重试逻辑更加优雅和可靠。 + +安装： + +```bash +pip install tenacity +``` + +基本使用： + +```python +from tenacity import ( + retry, + stop_after_attempt, + wait_exponential, + retry_if_exception_type, + before_sleep_log +) +from loguru import logger +import httpx + +from exceptions import RequestException, RateLimitException + + +@retry( + stop=stop_after_attempt(3), # 最多重试3次 + wait=wait_exponential(multiplier=1, max=10), # 指数退避，最长等待10秒 + retry=retry_if_exception_type(( # 只对特定异常重试 + httpx.TimeoutException, + httpx.ConnectError, + RequestException + )), + before_sleep=before_sleep_log(logger, "WARNING") # 重试前记录日志 +) +async def fetch_with_retry(client: httpx.AsyncClient, url: str) -> str: + """带重试的请求函数""" + response = await client.get(url) + + # 检查是否触发速率限制 + if response.status_code == 429: + raise RateLimitException("触发速率限制") + + response.raise_for_status() + return response.text +``` + +### 更复杂的重试策略 + +```python +from tenacity import ( + retry, + stop_after_attempt, + wait_random_exponential, + retry_if_exception_type, + RetryError +) + + +def create_retry_decorator(max_attempts: int = 3, max_wait: int = 60): + """创建可配置的重试装饰器""" + return retry( + stop=stop_after_attempt(max_attempts), + wait=wait_random_exponential(multiplier=1, max=max_wait), + retry=retry_if_exception_type(( + httpx.TimeoutException, + httpx.ConnectError, + httpx.HTTPStatusError, + )), + reraise=True # 重试用尽后重新抛出异常 + ) + + +# 使用 +@create_retry_decorator(max_attempts=5, max_wait=30) +async def fetch_important_data(client, url): + """重要数据获取，使用更多重试次数""" + response = await client.get(url) + response.raise_for_status() + return response.json() +``` + +### 全局异常处理 + +在爬虫主程序中实现全局异常处理： + +```python +import asyncio +from loguru import logger +from exceptions import CrawlerException, IPBlockedException + + +async def run_crawler(): + """爬虫主程序""" + try: + # 爬虫逻辑 + await crawl_all_pages() + except IPBlockedException as e: + logger.critical(f"IP被封禁，程序终止: {e}") + # 可以在这里触发告警通知 + raise + except CrawlerException as e: + logger.error(f"爬虫异常: {e}") + raise + except asyncio.CancelledError: + logger.warning("任务被取消") + raise + except Exception as e: + logger.exception(f"未预期的异常: {e}") + raise + finally: + logger.info("爬虫任务结束，执行清理工作...") + # 清理资源 + await cleanup() + + +if __name__ == "__main__": + try: + asyncio.run(run_crawler()) + except KeyboardInterrupt: + logger.info("用户中断程序") + except Exception as e: + logger.critical(f"程序异常退出: {e}") + exit(1) +``` + +--- + +## 项目目录结构 + +### 项目架构图 + +```mermaid +graph TB + subgraph 入口层 + main[main.py
程序入口] + end + + subgraph 核心模块 + config[config/
配置管理] + crawler[crawler/
爬虫逻辑] + store[store/
数据存储] + end + + subgraph 功能模块 + login[login/
登录认证] + client[client/
API客户端] + analysis[analysis/
数据分析] + end + + subgraph 基础模块 + core[core/
浏览器管理] + tools[tools/
工具函数] + models[models/
数据模型] + exceptions[exceptions/
异常定义] + end + + main --> config + main --> crawler + main --> store + main --> analysis + + crawler --> login + crawler --> client + + login --> core + client --> tools + tools --> models +``` + +### 推荐的目录结构 + +一个工程化的爬虫项目应该有清晰的目录结构： + +``` +my_crawler/ +├── config/ # 配置模块 +│ ├── __init__.py +│ └── settings.py # 配置定义 +├── core/ # 核心模块 +│ ├── __init__.py +│ ├── client.py # HTTP客户端封装 +│ └── retry.py # 重试策略 +├── crawler/ # 爬虫模块 +│ ├── __init__.py +│ ├── base.py # 爬虫基类 +│ └── xxx_crawler.py # 具体爬虫实现 +├── parser/ # 解析模块 +│ ├── __init__.py +│ └── xxx_parser.py # 页面解析器 +├── store/ # 存储模块 +│ ├── __init__.py +│ ├── base.py # 存储基类 +│ ├── mysql.py # MySQL存储 +│ └── json_store.py # JSON文件存储 +├── models/ # 数据模型 +│ ├── __init__.py +│ └── xxx_model.py # Pydantic模型定义 +├── exceptions/ # 异常定义 +│ ├── __init__.py +│ └── crawler_exceptions.py +├── utils/ # 工具函数 +│ ├── __init__.py +│ └── helpers.py +├── logs/ # 日志目录 +├── data/ # 数据输出目录 +├── tests/ # 测试目录 +│ └── test_xxx.py +├── .env # 环境变量（不提交到git） +├── .env.example # 环境变量示例 +├── .gitignore +├── requirements.txt # 依赖列表 +├── main.py # 程序入口 +└── README.md # 项目说明 +``` + +### 模块划分原则 + +1. **单一职责**：每个模块只负责一件事 +2. **高内聚低耦合**：相关代码放在一起，模块间依赖最小化 +3. **抽象与实现分离**：定义基类/接口，方便扩展和替换 + +--- + +## 实战案例：工程化改造 + +让我们将入门教程中的爬虫进行工程化改造。 + +### 改造前（入门教程版本） + +```python +# 入门教程的简单版本 +import httpx +from parsel import Selector + +async def crawl(): + async with httpx.AsyncClient() as client: + response = await client.get("https://example.com") + print(f"状态码: {response.status_code}") + # 解析... + # 存储... +``` + +### 改造后（工程化版本） + +项目结构： + +``` +refactored_crawler/ +├── config/ +│ ├── __init__.py +│ └── settings.py +├── exceptions.py +├── logger_config.py +├── client.py +├── crawler.py +└── main.py +``` + +核心代码示例： + +```python +# logger_config.py +import sys +from loguru import logger +from config.settings import settings + + +def setup_logger(): + """配置日志系统""" + # 移除默认处理器 + logger.remove() + + # 控制台输出 + logger.add( + sys.stderr, + level=settings.log_level, + format="{time:YYYY-MM-DD HH:mm:ss} | " + "{level: <8} | " + "{name}:{function}:{line} - " + "{message}" + ) + + # 文件输出 + logger.add( + f"logs/crawler_{settings.env}.log", + rotation="00:00", + retention="7 days", + level="DEBUG", + encoding="utf-8" + ) + + return logger +``` + +```python +# client.py +import httpx +from tenacity import retry, stop_after_attempt, wait_exponential +from loguru import logger +from config.settings import settings +from exceptions import RequestException + + +class CrawlerClient: + """封装的HTTP客户端""" + + def __init__(self): + self.client = httpx.AsyncClient( + timeout=settings.request_timeout, + headers={ + "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36" + } + ) + + @retry( + stop=stop_after_attempt(settings.max_retries), + wait=wait_exponential(multiplier=1, max=10) + ) + async def get(self, url: str) -> httpx.Response: + """发送GET请求，带重试""" + logger.debug(f"请求: {url}") + response = await self.client.get(url) + response.raise_for_status() + logger.info(f"请求成功: {url}") + return response + + async def close(self): + """关闭客户端""" + await self.client.aclose() + logger.debug("HTTP客户端已关闭") +``` + +```python +# main.py +import asyncio +from loguru import logger +from logger_config import setup_logger +from crawler import BBSCrawler +from config.settings import settings + + +async def main(): + """主程序入口""" + # 初始化日志 + setup_logger() + + logger.info(f"爬虫启动 - 环境: {settings.env}, 调试模式: {settings.debug}") + + crawler = BBSCrawler() + try: + await crawler.run() + except Exception as e: + logger.exception(f"爬虫运行异常: {e}") + raise + finally: + await crawler.cleanup() + logger.info("爬虫运行结束") + + +if __name__ == "__main__": + try: + asyncio.run(main()) + except KeyboardInterrupt: + logger.info("用户中断") +``` + +--- + +## 本章小结 + +本章我们学习了爬虫工程化开发的核心内容： + +1. **日志系统**：使用 loguru 替代 print，实现分级日志、日志轮转和持久化 +2. **配置管理**：使用 pydantic-settings 统一管理配置，支持环境变量和 .env 文件 +3. **异常处理**：自定义异常类，使用 tenacity 实现优雅的重试机制 +4. **项目结构**：遵循单一职责原则，合理划分模块 + +这些工程化实践将贯穿整个进阶教程，是后续学习的基础。 + +--- + +## 下一章预告 + +下一章我们将学习「反爬虫对抗基础：请求伪装」。主要内容包括： + +- 常见的反爬虫检测手段 +- User-Agent 随机轮换 +- 完整的请求头伪装 +- 使用 curl_cffi 模拟浏览器指纹 +- 智能速率控制 + +这将是我们与反爬虫斗争的第一步，让我们的爬虫不再那么容易被识别和封禁。 diff --git "a/docs/\347\210\254\350\231\253\350\277\233\344\273\267/02_\345\217\215\347\210\254\350\231\253\345\257\271\346\212\227\345\237\272\347\241\200_\350\257\267\346\261\202\344\274\252\350\243\205.md" "b/docs/\347\210\254\350\231\253\350\277\233\344\273\267/02_\345\217\215\347\210\254\350\231\253\345\257\271\346\212\227\345\237\272\347\241\200_\350\257\267\346\261\202\344\274\252\350\243\205.md" new file mode 100644 index 0000000..7b73e7b --- /dev/null +++ "b/docs/\347\210\254\350\231\253\350\277\233\344\273\267/02_\345\217\215\347\210\254\350\231\253\345\257\271\346\212\227\345\237\272\347\241\200_\350\257\267\346\261\202\344\274\252\350\243\205.md" @@ -0,0 +1,1009 @@ +# 反爬虫对抗基础：请求伪装 + +> 当你开始爬取一些有价值的网站时，很快就会发现：不是所有网站都欢迎爬虫。反爬虫与反反爬虫的对抗，是爬虫工程师必须面对的课题。本章我们将学习最基础也最重要的反爬对抗技术——请求伪装。 + +> **学习目标**：掌握 User-Agent 轮换、请求头伪装、TLS 指纹模拟和速率控制等核心技术，让你的爬虫不易被识别。 + +## 反爬虫机制概述 + +### 反爬检测流程 + +```mermaid +flowchart TD + request[爬虫发起请求] --> check1{User-Agent检测} + check1 -->|异常UA| block1[直接拒绝] + check1 -->|正常UA| check2{请求头检测} + + check2 -->|缺少必要头| block2[返回错误] + check2 -->|请求头正常| check3{频率检测} + + check3 -->|频率过高| block3[触发限流 429] + check3 -->|频率正常| check4{Cookie检测} + + check4 -->|无Cookie/过期| block4[要求登录 401] + check4 -->|Cookie有效| check5{其他检测} + + check5 -->|检测失败| block5[返回 403] + check5 -->|检测通过| success[返回数据] + + style block1 fill:#f66 + style block2 fill:#f66 + style block3 fill:#f96 + style block4 fill:#f96 + style block5 fill:#f66 + style success fill:#6f6 +``` + +### 为什么网站要反爬虫 + +在开始学习反爬技术之前，我们需要理解网站为什么要反爬虫： + +1. **保护数据资产**：数据是有价值的，网站不希望被批量获取 +2. **保护服务器资源**：爬虫会消耗服务器带宽和计算资源 +3. **防止恶意行为**：如价格监控、竞品分析、数据倒卖等 +4. **合规要求**：某些数据有法律保护要求 + +### 常见的反爬虫检测手段 + +```mermaid +graph LR + subgraph 低难度 + ua[UA检测] + headers[请求头检测] + end + + subgraph 中难度 + freq[频率检测] + cookie[Cookie检测] + sign[API签名检测] + end + + subgraph 高难度 + tls[TLS指纹检测] + js[JS环境检测] + captcha[验证码] + end + + ua --> headers --> freq --> cookie --> sign --> tls --> js --> captcha +``` + +| 检测类型 | 检测方式 | 难度 | 常见应对 | +|---------|---------|------|---------| +| 请求特征检测 | User-Agent、请求头完整性 | 低 | 伪装请求头 | +| 行为特征检测 | 访问频率、访问路径 | 中 | 速率控制、随机延迟 | +| API签名检测 | 参数签名验证 | 中 | 逆向签名算法 | +| Cookie检测 | 登录状态验证 | 中 | 登录获取Cookie | +| 浏览器指纹检测 | JS 环境、Canvas、WebGL | 高 | 使用真实浏览器 | +| 验证码检测 | 图片验证码、滑块验证码 | 高 | OCR、打码平台 | + +本章主要聚焦于**请求特征检测**的对抗，这是最基础的反爬手段。 + +--- + +## User-Agent 策略 + +### 什么是 User-Agent + +User-Agent（简称 UA）是 HTTP 请求头中的一个字段，用于标识发起请求的客户端类型。例如： + +``` +Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36 +``` + +这个字符串包含了： +- 浏览器类型（Chrome） +- 浏览器版本（120.0.0.0） +- 操作系统（Windows 10 64位） +- 渲染引擎（AppleWebKit/537.36） + +### 为什么要轮换 User-Agent + +如果你的爬虫始终使用同一个 UA，会有以下风险： + +1. **特征明显**：Python 默认的 UA 是 `python-requests/2.x.x`，一眼就能识别 +2. **容易被追踪**：同一 UA 的大量请求会被关联分析 +3. **容易被封禁**：一旦被识别，可以按 UA 封禁 + +### 实现 User-Agent 轮换 + +#### 方法一：手动维护 UA 列表 + +```python +import random + +# 桌面浏览器 UA 列表 +DESKTOP_USER_AGENTS = [ + # Chrome Windows + "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36", + "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36", + # Chrome Mac + "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36", + # Firefox Windows + "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:121.0) Gecko/20100101 Firefox/121.0", + # Firefox Mac + "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:121.0) Gecko/20100101 Firefox/121.0", + # Safari Mac + "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Safari/605.1.15", + # Edge Windows + "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36 Edg/120.0.0.0", +] + +# 移动端 UA 列表 +MOBILE_USER_AGENTS = [ + # iPhone Safari + "Mozilla/5.0 (iPhone; CPU iPhone OS 17_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Mobile/15E148 Safari/604.1", + # Android Chrome + "Mozilla/5.0 (Linux; Android 14; Pixel 8) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Mobile Safari/537.36", + # Android Samsung + "Mozilla/5.0 (Linux; Android 14; SM-S918B) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Mobile Safari/537.36", +] + + +def get_random_ua(mobile: bool = False) -> str: + """获取随机 User-Agent""" + ua_list = MOBILE_USER_AGENTS if mobile else DESKTOP_USER_AGENTS + return random.choice(ua_list) +``` + +#### 方法二：使用 fake-useragent 库 + +```bash +pip install fake-useragent +``` + +```python +from fake_useragent import UserAgent + +# 创建 UserAgent 对象 +ua = UserAgent() + +# 获取随机 UA +print(ua.random) # 完全随机 +print(ua.chrome) # 随机 Chrome UA +print(ua.firefox) # 随机 Firefox UA +print(ua.safari) # 随机 Safari UA +``` + +#### 实现 UA 轮换器 + +```python +import random +from typing import List, Optional +from fake_useragent import UserAgent + + +class UARotator: + """User-Agent 轮换器""" + + def __init__(self, use_fake_ua: bool = True, custom_uas: Optional[List[str]] = None): + """ + 初始化 UA 轮换器 + + Args: + use_fake_ua: 是否使用 fake-useragent 库 + custom_uas: 自定义 UA 列表 + """ + self.use_fake_ua = use_fake_ua + self.custom_uas = custom_uas or [] + + if use_fake_ua: + try: + self._fake_ua = UserAgent() + except Exception: + self.use_fake_ua = False + + def get_random(self) -> str: + """获取随机 UA""" + # 优先使用自定义列表 + if self.custom_uas: + return random.choice(self.custom_uas) + + # 使用 fake-useragent + if self.use_fake_ua: + return self._fake_ua.random + + # 默认返回 Chrome UA + return ( + "Mozilla/5.0 (Windows NT 10.0; Win64; x64) " + "AppleWebKit/537.36 (KHTML, like Gecko) " + "Chrome/120.0.0.0 Safari/537.36" + ) + + def get_chrome(self) -> str: + """获取 Chrome UA""" + if self.use_fake_ua: + return self._fake_ua.chrome + return self.get_random() + + def get_mobile(self) -> str: + """获取移动端 UA""" + mobile_uas = [ + "Mozilla/5.0 (iPhone; CPU iPhone OS 17_2 like Mac OS X) AppleWebKit/605.1.15", + "Mozilla/5.0 (Linux; Android 14; Pixel 8) AppleWebKit/537.36", + ] + return random.choice(mobile_uas) +``` + +--- + +## 请求头完整伪装 + +### 为什么仅有 User-Agent 不够 + +很多网站不仅检测 UA，还会检测其他请求头字段。真实浏览器的请求头是非常丰富的： + +```http +GET /api/data HTTP/1.1 +Host: www.example.com +User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 +Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 +Accept-Language: zh-CN,zh;q=0.9,en;q=0.8 +Accept-Encoding: gzip, deflate, br +Connection: keep-alive +Upgrade-Insecure-Requests: 1 +Sec-Fetch-Dest: document +Sec-Fetch-Mode: navigate +Sec-Fetch-Site: none +Sec-Fetch-User: ?1 +Cache-Control: max-age=0 +``` + +而 Python 默认的请求头非常简陋，很容易被识别。 + +### 构建完整的请求头 + +```python +from typing import Dict, Optional + + +class HeadersBuilder: + """请求头构建器""" + + # 基础请求头模板 + BASE_HEADERS = { + "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8", + "Accept-Language": "zh-CN,zh;q=0.9,en;q=0.8", + "Accept-Encoding": "gzip, deflate, br", + "Connection": "keep-alive", + "Upgrade-Insecure-Requests": "1", + "Sec-Fetch-Dest": "document", + "Sec-Fetch-Mode": "navigate", + "Sec-Fetch-Site": "none", + "Sec-Fetch-User": "?1", + "Cache-Control": "max-age=0", + } + + # API 请求头模板 + API_HEADERS = { + "Accept": "application/json, text/plain, */*", + "Accept-Language": "zh-CN,zh;q=0.9,en;q=0.8", + "Accept-Encoding": "gzip, deflate, br", + "Connection": "keep-alive", + "Sec-Fetch-Dest": "empty", + "Sec-Fetch-Mode": "cors", + "Sec-Fetch-Site": "same-origin", + } + + def __init__(self, ua_rotator: Optional['UARotator'] = None): + self.ua_rotator = ua_rotator or UARotator() + + def build( + self, + referer: Optional[str] = None, + origin: Optional[str] = None, + extra_headers: Optional[Dict[str, str]] = None, + is_api: bool = False + ) -> Dict[str, str]: + """ + 构建请求头 + + Args: + referer: Referer 地址 + origin: Origin 地址 + extra_headers: 额外的请求头 + is_api: 是否是 API 请求 + + Returns: + 完整的请求头字典 + """ + # 选择基础模板 + headers = self.API_HEADERS.copy() if is_api else self.BASE_HEADERS.copy() + + # 添加 User-Agent + headers["User-Agent"] = self.ua_rotator.get_random() + + # 添加 Referer + if referer: + headers["Referer"] = referer + # 如果有 Referer，通常 Sec-Fetch-Site 应该是 same-origin 或 cross-site + headers["Sec-Fetch-Site"] = "same-origin" + + # 添加 Origin（通常用于 POST 请求或 CORS 请求） + if origin: + headers["Origin"] = origin + + # 合并额外请求头 + if extra_headers: + headers.update(extra_headers) + + return headers + + def build_for_ajax( + self, + referer: str, + x_requested_with: bool = True + ) -> Dict[str, str]: + """ + 构建 AJAX 请求头 + + Args: + referer: Referer 地址 + x_requested_with: 是否添加 X-Requested-With 头 + + Returns: + AJAX 请求头 + """ + headers = self.build(referer=referer, is_api=True) + + if x_requested_with: + headers["X-Requested-With"] = "XMLHttpRequest" + + return headers +``` + +### Referer 的正确设置 + +Referer 表示当前请求是从哪个页面发起的。正确设置 Referer 很重要： + +```python +async def crawl_with_referer(client, list_url: str, detail_urls: list): + """演示正确设置 Referer""" + headers_builder = HeadersBuilder() + + # 访问列表页 + list_headers = headers_builder.build() + response = await client.get(list_url, headers=list_headers) + + # 访问详情页时，Referer 应该是列表页 + for detail_url in detail_urls: + detail_headers = headers_builder.build(referer=list_url) + response = await client.get(detail_url, headers=detail_headers) +``` + +### 请求头实战配置 + +以下是一个通用的请求头构建器，可以根据目标网站进行定制： + +```python +# headers_builder.py - 通用请求头构建器 +"""通用请求头配置""" + +class SiteHeadersBuilder: + """站点请求头构建器""" + + def __init__(self, base_url: str): + """ + 初始化构建器 + + Args: + base_url: 目标网站基础URL + """ + self.base_url = base_url + self.base_headers = { + "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) " + "AppleWebKit/537.36 (KHTML, like Gecko) " + "Chrome/120.0.0.0 Safari/537.36", + "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", + "Accept-Language": "zh-CN,zh;q=0.9,en;q=0.8", + "Accept-Encoding": "gzip, deflate, br", + "Connection": "keep-alive", + "Sec-Fetch-Dest": "document", + "Sec-Fetch-Mode": "navigate", + "Sec-Fetch-Site": "none", + } + + def build_for_page(self, referer: str = "") -> dict: + """ + 构建页面请求头 + + Args: + referer: Referer 地址 + + Returns: + 完整的请求头字典 + """ + headers = self.base_headers.copy() + if referer: + headers["Referer"] = referer + headers["Sec-Fetch-Site"] = "same-origin" + return headers + + def build_for_api(self, referer: str = "") -> dict: + """构建 API 请求头""" + headers = self.base_headers.copy() + headers["Accept"] = "application/json, text/plain, */*" + headers["Sec-Fetch-Dest"] = "empty" + headers["Sec-Fetch-Mode"] = "cors" + if referer: + headers["Referer"] = referer + headers["Sec-Fetch-Site"] = "same-origin" + return headers +``` + +> **请求头配置要点**： +> 1. **Referer**：很多网站会检查，应设置为合理的来源页面 +> 2. **Sec-Fetch 系列**：现代浏览器标准头，建议保持完整 +> 3. **Accept**：页面请求和 API 请求的 Accept 头不同 + +--- + +## 使用 curl_cffi 模拟浏览器指纹 + +### 什么是 TLS 指纹 + +除了 HTTP 请求头，服务器还可以通过 TLS（HTTPS 握手）的特征来识别客户端。不同的客户端（浏览器、Python requests、curl 等）在 TLS 握手时会展现不同的特征： + +- 支持的加密套件顺序 +- 支持的 TLS 扩展 +- 椭圆曲线参数 + +Python 的 `requests` 和 `httpx` 使用的 TLS 指纹与真实浏览器差异很大，容易被识别。 + +### curl_cffi 简介 + +`curl_cffi` 是一个可以模拟各种浏览器 TLS 指纹的 HTTP 客户端库。 + +安装： + +```bash +pip install curl_cffi +``` + +### 基本使用 + +```python +from curl_cffi import requests + +# 模拟 Chrome 浏览器 +response = requests.get( + "https://tls.browserleaks.com/json", + impersonate="chrome120" # 模拟 Chrome 120 +) +print(response.json()) + +# 支持的浏览器指纹 +# chrome99, chrome100, chrome101, ..., chrome120 +# chrome99_android +# edge99, edge101 +# safari15_3, safari15_5, safari17_0 +``` + +### 异步使用 + +```python +from curl_cffi.requests import AsyncSession + +async def fetch_with_curl_cffi(): + """使用 curl_cffi 的异步请求""" + async with AsyncSession(impersonate="chrome120") as session: + response = await session.get("https://httpbin.org/headers") + print(response.json()) +``` + +### 封装 curl_cffi 客户端 + +```python +from curl_cffi.requests import AsyncSession +from typing import Optional, Dict, Any +import random + + +class BrowserClient: + """ + 模拟浏览器的 HTTP 客户端 + + 使用 curl_cffi 模拟真实浏览器的 TLS 指纹 + """ + + BROWSER_VERSIONS = [ + "chrome119", + "chrome120", + "edge99", + "edge101", + "safari15_5", + "safari17_0", + ] + + def __init__( + self, + impersonate: Optional[str] = None, + proxy: Optional[str] = None, + timeout: int = 30 + ): + """ + 初始化客户端 + + Args: + impersonate: 模拟的浏览器，如 "chrome120"，None 表示随机 + proxy: 代理地址 + timeout: 超时时间 + """ + self.impersonate = impersonate + self.proxy = proxy + self.timeout = timeout + self._session: Optional[AsyncSession] = None + + async def __aenter__(self): + await self.start() + return self + + async def __aexit__(self, *args): + await self.close() + + async def start(self): + """启动会话""" + browser = self.impersonate or random.choice(self.BROWSER_VERSIONS) + self._session = AsyncSession( + impersonate=browser, + proxy=self.proxy, + timeout=self.timeout + ) + + async def close(self): + """关闭会话""" + if self._session: + await self._session.close() + self._session = None + + async def get( + self, + url: str, + headers: Optional[Dict[str, str]] = None, + **kwargs + ) -> Any: + """发送 GET 请求""" + if not self._session: + await self.start() + return await self._session.get(url, headers=headers, **kwargs) + + async def post( + self, + url: str, + data: Optional[Dict] = None, + json: Optional[Dict] = None, + headers: Optional[Dict[str, str]] = None, + **kwargs + ) -> Any: + """发送 POST 请求""" + if not self._session: + await self.start() + return await self._session.post( + url, data=data, json=json, headers=headers, **kwargs + ) +``` + +--- + +## 速率控制 + +### 为什么需要速率控制 + +即使你的请求伪装得再好，如果访问频率过高，也会触发反爬机制。人类的浏览行为是有一定节奏的，而机器的请求往往过于规律或过于密集。 + +### 基本延迟策略 + +```python +import asyncio +import random + + +async def crawl_with_delay(urls: list, min_delay: float = 1.0, max_delay: float = 3.0): + """ + 带随机延迟的爬取 + + Args: + urls: URL 列表 + min_delay: 最小延迟（秒） + max_delay: 最大延迟（秒） + """ + for url in urls: + # 发送请求 + response = await fetch(url) + + # 随机延迟 + delay = random.uniform(min_delay, max_delay) + await asyncio.sleep(delay) +``` + +### 令牌桶限速器 + +令牌桶算法是一种更精确的限速方式，它允许一定程度的突发请求，同时保持长期平均速率。 + +```python +import asyncio +import time +from typing import Optional + + +class TokenBucket: + """ + 令牌桶限速器 + + 工作原理： + - 桶有固定容量（最大令牌数） + - 以固定速率向桶中添加令牌 + - 每次请求消耗一个令牌 + - 桶空时请求需要等待 + """ + + def __init__( + self, + rate: float, + capacity: Optional[int] = None + ): + """ + 初始化令牌桶 + + Args: + rate: 每秒添加的令牌数（即每秒最多请求数） + capacity: 桶容量，默认等于 rate + """ + self.rate = rate + self.capacity = capacity or int(rate) + self.tokens = self.capacity + self.last_time = time.monotonic() + self._lock = asyncio.Lock() + + async def acquire(self, tokens: int = 1) -> float: + """ + 获取令牌 + + Args: + tokens: 需要的令牌数 + + Returns: + 实际等待的时间 + """ + async with self._lock: + now = time.monotonic() + + # 计算从上次到现在应该添加的令牌数 + elapsed = now - self.last_time + self.tokens = min( + self.capacity, + self.tokens + elapsed * self.rate + ) + self.last_time = now + + # 如果令牌不足，计算需要等待的时间 + if self.tokens < tokens: + wait_time = (tokens - self.tokens) / self.rate + await asyncio.sleep(wait_time) + self.tokens = 0 + return wait_time + else: + self.tokens -= tokens + return 0 + + async def __aenter__(self): + await self.acquire() + return self + + async def __aexit__(self, *args): + pass + + +# 使用示例 +async def crawl_with_rate_limit(): + """使用令牌桶限速""" + # 每秒最多 2 个请求 + limiter = TokenBucket(rate=2.0) + + urls = ["https://example.com/page/{}".format(i) for i in range(10)] + + for url in urls: + async with limiter: # 自动限速 + response = await fetch(url) + print(f"Fetched: {url}") +``` + +### 使用 asyncio.Semaphore 控制并发 + +```python +import asyncio + + +class ConcurrencyLimiter: + """并发限制器""" + + def __init__(self, max_concurrent: int = 10): + """ + 初始化并发限制器 + + Args: + max_concurrent: 最大并发数 + """ + self.semaphore = asyncio.Semaphore(max_concurrent) + + async def run_with_limit(self, coro): + """在并发限制下执行协程""" + async with self.semaphore: + return await coro + + +async def crawl_concurrent(urls: list, max_concurrent: int = 5): + """ + 带并发限制的批量爬取 + + Args: + urls: URL 列表 + max_concurrent: 最大并发数 + """ + semaphore = asyncio.Semaphore(max_concurrent) + + async def fetch_with_limit(url): + async with semaphore: + return await fetch(url) + + # 并发执行，但同时最多 max_concurrent 个请求 + tasks = [fetch_with_limit(url) for url in urls] + results = await asyncio.gather(*tasks, return_exceptions=True) + + return results +``` + +--- + +## HTTP 错误处理 + +### HTTP 状态码处理 + +在爬取网站时，需要处理各种 HTTP 状态码： + +```mermaid +flowchart LR + response[HTTP响应] --> check{检查状态码} + + check -->|200| success[成功
正常处理数据] + check -->|401| login[需要认证
登录获取凭证] + check -->|403| forbidden[禁止访问
检查请求头] + check -->|429| ratelimit[频率限制
降低速率] + check -->|404| notfound[资源不存在] + + login --> relogin[获取认证凭证] + ratelimit --> wait[等待后重试] + forbidden --> fixheaders[修复请求头] +``` + +```python +# error_handler.py - HTTP错误处理 +from loguru import logger + + +async def handle_response(response, url: str): + """ + 处理 HTTP 响应 + + Args: + response: HTTP 响应对象 + url: 请求的 URL + """ + status_code = response.status_code + + if status_code == 200: + logger.info(f"请求成功: {url}") + return response + + elif status_code == 401: + logger.warning(f"需要认证: {url}") + raise Exception("需要登录或认证凭证已过期") + + elif status_code == 403: + logger.error(f"禁止访问: {url}") + raise Exception("访问被禁止，请检查请求头配置") + + elif status_code == 429: + logger.warning(f"触发频率限制: {url}") + raise Exception("请求过于频繁，请降低访问速率") + + elif status_code == 404: + logger.warning(f"资源不存在: {url}") + return None + + else: + logger.error(f"HTTP错误 {status_code}: {url}") + raise Exception(f"HTTP错误: {status_code}") +``` + +--- + +## 实战案例：完整的请求伪装爬虫 + +让我们把本章学到的技术整合成一个完整的通用爬虫示例： + +```python +# -*- coding: utf-8 -*- +""" +完整的请求伪装爬虫示例 +结合 UA 轮换、请求头伪装、速率控制 +""" + +import asyncio +import random +from typing import List, Dict, Optional +from loguru import logger + +# 如果安装了 curl_cffi，优先使用 +try: + from curl_cffi.requests import AsyncSession + USE_CURL_CFFI = True +except ImportError: + import httpx + USE_CURL_CFFI = False + + +class AntiDetectionCrawler: + """反检测爬虫""" + + DESKTOP_UAS = [ + "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36", + "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36", + "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:121.0) Gecko/20100101 Firefox/121.0", + ] + + def __init__( + self, + max_concurrent: int = 5, + min_delay: float = 1.0, + max_delay: float = 3.0 + ): + self.max_concurrent = max_concurrent + self.min_delay = min_delay + self.max_delay = max_delay + self.semaphore = asyncio.Semaphore(max_concurrent) + self._session = None + + async def __aenter__(self): + await self.start() + return self + + async def __aexit__(self, *args): + await self.close() + + async def start(self): + """启动客户端""" + if USE_CURL_CFFI: + self._session = AsyncSession(impersonate="chrome120") + else: + self._session = httpx.AsyncClient(timeout=30) + logger.info(f"客户端启动 (curl_cffi: {USE_CURL_CFFI})") + + async def close(self): + """关闭客户端""" + if self._session: + await self._session.close() if USE_CURL_CFFI else await self._session.aclose() + + def _build_headers(self, referer: Optional[str] = None) -> Dict[str, str]: + """构建请求头""" + headers = { + "User-Agent": random.choice(self.DESKTOP_UAS), + "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", + "Accept-Language": "zh-CN,zh;q=0.9,en;q=0.8", + "Accept-Encoding": "gzip, deflate, br", + "Connection": "keep-alive", + "Upgrade-Insecure-Requests": "1", + } + if referer: + headers["Referer"] = referer + return headers + + async def fetch( + self, + url: str, + referer: Optional[str] = None + ) -> Optional[str]: + """ + 获取页面内容 + + Args: + url: 目标 URL + referer: Referer 地址 + + Returns: + 页面内容或 None + """ + async with self.semaphore: + try: + headers = self._build_headers(referer) + + logger.debug(f"请求: {url}") + response = await self._session.get(url, headers=headers) + + if USE_CURL_CFFI: + response.raise_for_status() + content = response.text + else: + response.raise_for_status() + content = response.text + + logger.info(f"成功: {url}") + + # 随机延迟 + delay = random.uniform(self.min_delay, self.max_delay) + await asyncio.sleep(delay) + + return content + + except Exception as e: + logger.error(f"失败: {url} - {e}") + return None + + async def crawl_batch( + self, + urls: List[str], + referer: Optional[str] = None + ) -> List[Optional[str]]: + """ + 批量爬取 + + Args: + urls: URL 列表 + referer: Referer 地址 + + Returns: + 内容列表 + """ + tasks = [self.fetch(url, referer) for url in urls] + return await asyncio.gather(*tasks) + + +async def main(): + """主函数""" + logger.remove() + logger.add(lambda m: print(m, end=""), level="DEBUG") + + urls = [ + "https://httpbin.org/headers", + "https://httpbin.org/user-agent", + "https://httpbin.org/ip", + ] + + async with AntiDetectionCrawler(max_concurrent=2, min_delay=1, max_delay=2) as crawler: + results = await crawler.crawl_batch(urls) + for url, content in zip(urls, results): + if content: + print(f"\n{'='*50}") + print(f"URL: {url}") + print(content[:500]) + + +if __name__ == "__main__": + asyncio.run(main()) +``` + +--- + +## 本章小结 + +本章我们学习了反爬虫对抗的基础技术——请求伪装： + +1. **User-Agent 轮换**：使用真实浏览器 UA，随机轮换避免被追踪 +2. **请求头完整伪装**：构建与真实浏览器一致的完整请求头 +3. **TLS 指纹模拟**：使用 curl_cffi 模拟浏览器的 TLS 指纹 +4. **速率控制**：使用随机延迟和令牌桶算法控制请求频率 +5. **HTTP 错误处理**：正确处理各种 HTTP 状态码 + +这些技术可以应对大部分基于请求特征的反爬检测。 + +--- + +## 下一章预告 + +下一章我们将学习「代理 IP 的使用与管理」。主要内容包括： + +- 代理 IP 的类型和选择 +- 代理池的设计与实现 +- 代理的有效性检测和淘汰机制 +- 代理与爬虫的集成 + +代理 IP 是突破 IP 封禁的重要手段，也是大规模爬虫必不可少的基础设施。 diff --git "a/docs/\347\210\254\350\231\253\350\277\233\344\273\267/03_\344\273\243\347\220\206IP\347\232\204\344\275\277\347\224\250\344\270\216\347\256\241\347\220\206.md" "b/docs/\347\210\254\350\231\253\350\277\233\344\273\267/03_\344\273\243\347\220\206IP\347\232\204\344\275\277\347\224\250\344\270\216\347\256\241\347\220\206.md" new file mode 100644 index 0000000..cd42ebc --- /dev/null +++ "b/docs/\347\210\254\350\231\253\350\277\233\344\273\267/03_\344\273\243\347\220\206IP\347\232\204\344\275\277\347\224\250\344\270\216\347\256\241\347\220\206.md" @@ -0,0 +1,1139 @@ +# 代理 IP 的使用与管理 + +> 当你的爬虫遭遇 IP 封禁时，代理 IP 就成了必不可少的工具。本章将深入讲解代理 IP 的原理、类型选择，以及如何设计和实现一个实用的代理池管理系统。 + +## 代理 IP 基础 + +### 什么是代理 IP + +代理服务器（Proxy Server）是一种位于客户端和目标服务器之间的中间服务器。当你通过代理发送请求时： + +1. 你的请求先发送到代理服务器 +2. 代理服务器代替你向目标服务器发送请求 +3. 目标服务器将响应返回给代理服务器 +4. 代理服务器再将响应转发给你 + +这样，目标服务器看到的是代理服务器的 IP，而不是你的真实 IP。 + +### 代理类型详解 + +#### 按协议分类 + +| 类型 | 特点 | 使用场景 | +|------|------|----------| +| HTTP 代理 | 只支持 HTTP 协议 | 普通网页爬取 | +| HTTPS 代理 | 支持 HTTPS 协议 | 加密网站爬取 | +| SOCKS4 代理 | 支持 TCP 连接 | 需要更底层控制 | +| SOCKS5 代理 | 支持 TCP/UDP，可认证 | 最灵活的代理类型 | + +#### 按匿名度分类 + +| 类型 | 特点 | 识别难度 | +|------|------|----------| +| 透明代理 | 目标服务器能看到你的真实 IP | 容易被识别 | +| 匿名代理 | 隐藏真实 IP，但暴露代理身份 | 中等 | +| 高匿代理 | 完全隐藏真实 IP 和代理身份 | 较难识别 | + +对于爬虫来说，**高匿代理**是首选。 + +#### 按来源分类 + +| 类型 | 优点 | 缺点 | +|------|------|------| +| 免费代理 | 成本为零 | 不稳定、速度慢、可用率低 | +| 付费代理 | 稳定、速度快、可用率高 | 需要成本 | +| 自建代理 | 完全可控 | 需要服务器资源 | + +### 代理提供商选择指南 + +选择代理提供商时需要考虑： + +1. **IP 质量**：是否是高匿代理，是否已被目标网站封禁 +2. **IP 数量**：IP 池的规模 +3. **地理分布**：是否覆盖目标地区 +4. **稳定性**：连接成功率和响应速度 +5. **价格**：按流量计费还是按 IP 数计费 +6. **API 支持**：是否提供便捷的 API 获取代理 + +常见的代理类型： + +- **API 提取型**：通过 API 获取代理 IP 列表 +- **隧道代理型**：固定入口，自动轮换 IP +- **动态转发型**：每次请求自动更换 IP + +--- + +## 代理池设计 + +### 为什么需要代理池 + +直接使用单个代理存在以下问题： + +- 代理可能随时失效 +- 单个 IP 容易被封禁 +- 无法动态切换和管理 + +代理池可以解决这些问题： + +- 统一管理多个代理 +- 自动检测代理有效性 +- 智能分配和轮换代理 +- 记录代理质量评分 + +### 代理池架构设计 + +```mermaid +graph TB + subgraph ProxyPoolManager["代理池管理器"] + direction TB + + subgraph Components["核心组件"] + direction LR + Fetcher["代理获取器
(Fetcher)"] + Checker["代理检测器
(Checker)"] + Allocator["代理分配器
(Allocator)"] + end + + Storage[("代理存储
(内存 / Redis)")] + + Fetcher --> Storage + Checker --> Storage + Allocator --> Storage + end + + style ProxyPoolManager fill:#e8f4f8,stroke:#0288d1 + style Storage fill:#fff3e0,stroke:#ff9800 +``` + +**代理池工作流程**： + +```mermaid +flowchart LR + subgraph 获取阶段 + API["代理API"] --> Fetcher["获取器"] + Free["免费代理"] --> Fetcher + end + + subgraph 检测阶段 + Fetcher --> Checker["检测器"] + Checker -->|有效| Pool[("代理池")] + Checker -->|无效| Discard["丢弃"] + end + + subgraph 分配阶段 + Pool --> Allocator["分配器"] + Allocator --> Crawler["爬虫请求"] + Crawler -->|成功| Score["评分+1"] + Crawler -->|失败| Penalty["评分-1"] + Score --> Pool + Penalty --> Pool + end + + style Pool fill:#c8e6c9,stroke:#4caf50 + style Discard fill:#ffcdd2,stroke:#f44336 +``` + +### 核心接口设计 + +```python +from abc import ABC, abstractmethod +from typing import Optional, List +from dataclasses import dataclass +from enum import Enum + + +class ProxyProtocol(Enum): + """代理协议""" + HTTP = "http" + HTTPS = "https" + SOCKS5 = "socks5" + + +@dataclass +class ProxyInfo: + """代理信息""" + host: str + port: int + protocol: ProxyProtocol = ProxyProtocol.HTTP + username: Optional[str] = None + password: Optional[str] = None + + # 质量指标 + success_count: int = 0 + fail_count: int = 0 + avg_response_time: float = 0.0 + last_check_time: float = 0.0 + + @property + def url(self) -> str: + """构建代理 URL""" + auth = "" + if self.username and self.password: + auth = f"{self.username}:{self.password}@" + return f"{self.protocol.value}://{auth}{self.host}:{self.port}" + + @property + def score(self) -> float: + """计算代理评分""" + total = self.success_count + self.fail_count + if total == 0: + return 0.5 # 未测试的代理给中等分数 + success_rate = self.success_count / total + # 考虑响应时间，越快分数越高 + time_score = max(0, 1 - self.avg_response_time / 10) + return success_rate * 0.7 + time_score * 0.3 + + +class IProxyFetcher(ABC): + """代理获取器接口""" + + @abstractmethod + async def fetch(self) -> List[ProxyInfo]: + """获取代理列表""" + pass + + +class IProxyChecker(ABC): + """代理检测器接口""" + + @abstractmethod + async def check(self, proxy: ProxyInfo) -> bool: + """检测代理是否可用""" + pass + + +class IProxyPool(ABC): + """代理池接口""" + + @abstractmethod + async def get_proxy(self) -> Optional[ProxyInfo]: + """获取一个可用代理""" + pass + + @abstractmethod + async def return_proxy(self, proxy: ProxyInfo, success: bool): + """归还代理并报告使用结果""" + pass + + @abstractmethod + async def add_proxy(self, proxy: ProxyInfo): + """添加代理""" + pass + + @abstractmethod + async def remove_proxy(self, proxy: ProxyInfo): + """移除代理""" + pass +``` + +--- + +## 代理获取器实现 + +### 免费代理获取（仅供学习） + +```python +import httpx +from typing import List +from loguru import logger + + +class FreeProxyFetcher(IProxyFetcher): + """ + 免费代理获取器 + + 注意：免费代理质量较差，仅供学习测试使用 + """ + + async def fetch(self) -> List[ProxyInfo]: + """从免费代理网站获取代理""" + proxies = [] + + # 示例：从 API 获取（这里用一个示例 API） + try: + async with httpx.AsyncClient(timeout=10) as client: + # 这是一个示例 URL，实际使用时替换为真实的代理 API + response = await client.get( + "https://api.proxyscrape.com/v2/" + "?request=getproxies&protocol=http&timeout=10000&country=all" + ) + + if response.status_code == 200: + lines = response.text.strip().split("\n") + for line in lines: + try: + host, port = line.strip().split(":") + proxies.append(ProxyInfo( + host=host, + port=int(port), + protocol=ProxyProtocol.HTTP + )) + except ValueError: + continue + + logger.info(f"获取到 {len(proxies)} 个免费代理") + + except Exception as e: + logger.error(f"获取免费代理失败: {e}") + + return proxies +``` + +### API 代理获取器 + +```python +class APIProxyFetcher(IProxyFetcher): + """ + API 代理获取器 + + 从付费代理服务商的 API 获取代理 + """ + + def __init__( + self, + api_url: str, + api_key: Optional[str] = None, + count: int = 10 + ): + """ + 初始化 API 代理获取器 + + Args: + api_url: API 地址 + api_key: API 密钥 + count: 每次获取数量 + """ + self.api_url = api_url + self.api_key = api_key + self.count = count + + async def fetch(self) -> List[ProxyInfo]: + """从 API 获取代理""" + proxies = [] + + try: + params = {"num": self.count} + if self.api_key: + params["key"] = self.api_key + + async with httpx.AsyncClient(timeout=10) as client: + response = await client.get(self.api_url, params=params) + data = response.json() + + # 根据实际 API 返回格式解析 + # 这里假设返回 {"data": [{"ip": "x.x.x.x", "port": 8080}, ...]} + for item in data.get("data", []): + proxies.append(ProxyInfo( + host=item["ip"], + port=item["port"], + protocol=ProxyProtocol(item.get("protocol", "http")) + )) + + logger.info(f"从 API 获取到 {len(proxies)} 个代理") + + except Exception as e: + logger.error(f"从 API 获取代理失败: {e}") + + return proxies +``` + +--- + +## 代理检测器实现 + +```python +import time +import httpx +from typing import Optional + + +class ProxyChecker(IProxyChecker): + """ + 代理检测器 + + 检测代理的可用性和响应速度 + """ + + # 用于检测的 URL 列表 + CHECK_URLS = [ + "https://httpbin.org/ip", + "https://api.ipify.org?format=json", + ] + + def __init__(self, timeout: int = 10): + """ + 初始化检测器 + + Args: + timeout: 检测超时时间 + """ + self.timeout = timeout + + async def check(self, proxy: ProxyInfo) -> bool: + """ + 检测代理是否可用 + + Args: + proxy: 代理信息 + + Returns: + 代理是否可用 + """ + start_time = time.time() + + try: + async with httpx.AsyncClient( + proxies=proxy.url, + timeout=self.timeout + ) as client: + for url in self.CHECK_URLS: + try: + response = await client.get(url) + if response.status_code == 200: + # 更新响应时间 + response_time = time.time() - start_time + proxy.avg_response_time = ( + proxy.avg_response_time * 0.7 + + response_time * 0.3 + ) + proxy.last_check_time = time.time() + logger.debug( + f"代理可用: {proxy.host}:{proxy.port}, " + f"响应时间: {response_time:.2f}s" + ) + return True + except Exception: + continue + + except Exception as e: + logger.debug(f"代理检测失败: {proxy.host}:{proxy.port} - {e}") + + return False + + async def check_batch( + self, + proxies: List[ProxyInfo], + concurrency: int = 20 + ) -> List[ProxyInfo]: + """ + 批量检测代理 + + Args: + proxies: 代理列表 + concurrency: 并发数 + + Returns: + 可用的代理列表 + """ + import asyncio + + semaphore = asyncio.Semaphore(concurrency) + valid_proxies = [] + + async def check_one(proxy: ProxyInfo): + async with semaphore: + if await self.check(proxy): + valid_proxies.append(proxy) + + tasks = [check_one(p) for p in proxies] + await asyncio.gather(*tasks, return_exceptions=True) + + logger.info(f"检测完成: {len(valid_proxies)}/{len(proxies)} 可用") + return valid_proxies +``` + +--- + +## 代理池实现 + +```python +import asyncio +import random +import time +from typing import Optional, List, Dict +from collections import defaultdict +from loguru import logger + + +class ProxyPool(IProxyPool): + """ + 代理池实现 + + 特性： + - 自动获取和检测代理 + - 基于评分的智能分配 + - 自动淘汰失效代理 + - 支持代理预热 + """ + + def __init__( + self, + fetcher: IProxyFetcher, + checker: IProxyChecker, + min_proxies: int = 10, + max_proxies: int = 100, + check_interval: int = 300, + max_fail_count: int = 3 + ): + """ + 初始化代理池 + + Args: + fetcher: 代理获取器 + checker: 代理检测器 + min_proxies: 最小代理数量 + max_proxies: 最大代理数量 + check_interval: 检测间隔（秒） + max_fail_count: 最大失败次数 + """ + self.fetcher = fetcher + self.checker = checker + self.min_proxies = min_proxies + self.max_proxies = max_proxies + self.check_interval = check_interval + self.max_fail_count = max_fail_count + + # 代理存储 + self._proxies: Dict[str, ProxyInfo] = {} + self._lock = asyncio.Lock() + + # 后台任务 + self._refresh_task: Optional[asyncio.Task] = None + self._running = False + + def _proxy_key(self, proxy: ProxyInfo) -> str: + """生成代理唯一标识""" + return f"{proxy.host}:{proxy.port}" + + async def start(self): + """启动代理池""" + self._running = True + + # 初始获取代理 + await self._refresh_proxies() + + # 启动后台刷新任务 + self._refresh_task = asyncio.create_task(self._refresh_loop()) + + logger.info("代理池已启动") + + async def stop(self): + """停止代理池""" + self._running = False + + if self._refresh_task: + self._refresh_task.cancel() + try: + await self._refresh_task + except asyncio.CancelledError: + pass + + logger.info("代理池已停止") + + async def _refresh_loop(self): + """后台刷新循环""" + while self._running: + try: + await asyncio.sleep(self.check_interval) + await self._refresh_proxies() + except asyncio.CancelledError: + break + except Exception as e: + logger.error(f"代理刷新异常: {e}") + + async def _refresh_proxies(self): + """刷新代理""" + async with self._lock: + # 检查是否需要补充代理 + if len(self._proxies) >= self.min_proxies: + return + + logger.info(f"代理不足 ({len(self._proxies)}/{self.min_proxies})，开始获取...") + + # 获取新代理 + new_proxies = await self.fetcher.fetch() + + # 检测代理 + valid_proxies = await self.checker.check_batch(new_proxies) + + # 添加到池中 + for proxy in valid_proxies: + key = self._proxy_key(proxy) + if key not in self._proxies and len(self._proxies) < self.max_proxies: + self._proxies[key] = proxy + + logger.info(f"代理池更新完成，当前数量: {len(self._proxies)}") + + async def get_proxy(self) -> Optional[ProxyInfo]: + """ + 获取一个可用代理 + + 使用加权随机选择，评分高的代理被选中概率更大 + """ + async with self._lock: + if not self._proxies: + logger.warning("代理池为空") + return None + + # 计算权重 + proxies = list(self._proxies.values()) + weights = [max(p.score, 0.1) for p in proxies] + + # 加权随机选择 + selected = random.choices(proxies, weights=weights, k=1)[0] + + logger.debug(f"分配代理: {selected.host}:{selected.port} (评分: {selected.score:.2f})") + return selected + + async def return_proxy(self, proxy: ProxyInfo, success: bool): + """ + 归还代理并报告使用结果 + + Args: + proxy: 代理信息 + success: 使用是否成功 + """ + async with self._lock: + key = self._proxy_key(proxy) + + if key not in self._proxies: + return + + stored_proxy = self._proxies[key] + + if success: + stored_proxy.success_count += 1 + else: + stored_proxy.fail_count += 1 + + # 检查是否需要淘汰 + if stored_proxy.fail_count >= self.max_fail_count: + total = stored_proxy.success_count + stored_proxy.fail_count + if total > 5 and stored_proxy.score < 0.3: + del self._proxies[key] + logger.info(f"淘汰低质量代理: {proxy.host}:{proxy.port}") + + async def add_proxy(self, proxy: ProxyInfo): + """添加代理""" + async with self._lock: + key = self._proxy_key(proxy) + if key not in self._proxies and len(self._proxies) < self.max_proxies: + self._proxies[key] = proxy + + async def remove_proxy(self, proxy: ProxyInfo): + """移除代理""" + async with self._lock: + key = self._proxy_key(proxy) + if key in self._proxies: + del self._proxies[key] + + @property + def size(self) -> int: + """代理池大小""" + return len(self._proxies) + + def get_stats(self) -> Dict: + """获取统计信息""" + if not self._proxies: + return {"total": 0} + + proxies = list(self._proxies.values()) + scores = [p.score for p in proxies] + + return { + "total": len(proxies), + "avg_score": sum(scores) / len(scores), + "max_score": max(scores), + "min_score": min(scores), + } +``` + +--- + +## 代理与爬虫集成 + +### 使用 httpx 设置代理 + +```python +import httpx + +async def fetch_with_proxy(url: str, proxy_url: str) -> str: + """使用代理发送请求""" + async with httpx.AsyncClient(proxies=proxy_url, timeout=30) as client: + response = await client.get(url) + return response.text + + +# 使用示例 +proxy = "http://user:pass@127.0.0.1:8080" +content = await fetch_with_proxy("https://httpbin.org/ip", proxy) +``` + +### 集成代理池的爬虫 + +```python +class ProxiedCrawler: + """ + 集成代理池的爬虫 + + 自动管理代理的获取、轮换和报告 + """ + + def __init__(self, proxy_pool: ProxyPool): + self.proxy_pool = proxy_pool + + async def fetch(self, url: str) -> Optional[str]: + """使用代理获取页面""" + proxy = await self.proxy_pool.get_proxy() + + if not proxy: + logger.warning("无可用代理") + return None + + try: + async with httpx.AsyncClient( + proxies=proxy.url, + timeout=30 + ) as client: + response = await client.get(url) + response.raise_for_status() + + # 报告成功 + await self.proxy_pool.return_proxy(proxy, success=True) + return response.text + + except Exception as e: + logger.warning(f"请求失败: {url} - {e}") + # 报告失败 + await self.proxy_pool.return_proxy(proxy, success=False) + return None +``` + +--- + +## 隧道代理使用 + +隧道代理是一种特殊的代理模式，你只需连接到固定的代理入口，每次请求自动分配不同的 IP。 + +```python +class TunnelProxyClient: + """ + 隧道代理客户端 + + 特点：固定入口，自动轮换 IP + """ + + def __init__( + self, + host: str, + port: int, + username: str, + password: str + ): + self.proxy_url = f"http://{username}:{password}@{host}:{port}" + + async def get(self, url: str, **kwargs) -> httpx.Response: + """发送请求（自动使用隧道代理）""" + async with httpx.AsyncClient( + proxies=self.proxy_url, + timeout=30 + ) as client: + return await client.get(url, **kwargs) + + +# 使用示例 +tunnel = TunnelProxyClient( + host="tunnel.example.com", + port=12345, + username="your_username", + password="your_password" +) + +# 每次请求自动使用不同 IP +response1 = await tunnel.get("https://httpbin.org/ip") +response2 = await tunnel.get("https://httpbin.org/ip") +``` + +--- + +## 代理使用最佳实践 + +### IP 封禁机制分析 + +大多数网站都有反爬虫的 IP 封禁机制，常见的触发条件和处理方式： + +```mermaid +flowchart TD + Request["发起请求"] --> RateCheck{"频率检测"} + RateCheck -->|正常| UACheck{"UA检测"} + RateCheck -->|过快| Block429["429 限流"] + + UACheck -->|正常| BehaviorCheck{"行为检测"} + UACheck -->|异常| Block412["412 风控"] + + BehaviorCheck -->|正常| Success["正常响应"] + BehaviorCheck -->|异常| Block403["403 封禁"] + + Block429 --> IPMark["IP标记"] + Block412 --> IPMark + Block403 --> IPMark + + IPMark --> BlackList["IP黑名单"] + + style Success fill:#c8e6c9,stroke:#4caf50 + style Block429 fill:#fff3e0,stroke:#ff9800 + style Block412 fill:#ffcdd2,stroke:#f44336 + style Block403 fill:#ffcdd2,stroke:#f44336 + style BlackList fill:#ffcdd2,stroke:#f44336 +``` + +**常见 IP 封禁特点**： + +| 触发条件 | 响应码 | 封禁时长 | 解封方式 | +|---------|--------|---------|---------| +| 请求频率过高 | 429 | 几分钟~1小时 | 降低频率后自动解封 | +| 风控检测触发 | 403 | 数小时~1天 | 需更换IP | +| 严重违规 | 403/IP拉黑 | 数天~永久 | 需更换IP | + +### 代理有效性检测器 + +使用 httpbin.org 等测试服务来验证代理的可用性： + +```python +import time +import httpx +from typing import Optional +from loguru import logger + + +class SiteProxyChecker(IProxyChecker): + """ + 通用代理检测器 + + 使用 httpbin.org 检测代理可用性和匿名度 + """ + + # 使用 httpbin.org 检测代理IP + CHECK_URL = "https://httpbin.org/ip" + + # 通用请求头 + HEADERS = { + "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) " + "AppleWebKit/537.36 (KHTML, like Gecko) " + "Chrome/131.0.0.0 Safari/537.36", + "Accept": "application/json" + } + + def __init__(self, timeout: int = 10): + self.timeout = timeout + + async def check(self, proxy: ProxyInfo) -> bool: + """ + 检测代理是否可用 + + 判断标准： + - 请求成功（状态码200） + - 响应包含有效JSON + - 返回的IP与代理IP一致（验证代理生效） + """ + start_time = time.time() + + try: + async with httpx.AsyncClient( + proxies=proxy.url, + timeout=self.timeout, + headers=self.HEADERS + ) as client: + response = await client.get(self.CHECK_URL) + + if response.status_code != 200: + logger.debug(f"代理状态码异常: {proxy.host}:{proxy.port} - {response.status_code}") + return False + + data = response.json() + + # 验证返回的IP（httpbin.org 返回 {"origin": "x.x.x.x"}） + origin_ip = data.get("origin", "") + if not origin_ip: + logger.debug(f"代理响应异常: {proxy.host}:{proxy.port}") + return False + + # 更新响应时间 + response_time = time.time() - start_time + proxy.avg_response_time = ( + proxy.avg_response_time * 0.7 + response_time * 0.3 + ) + proxy.last_check_time = time.time() + + logger.debug( + f"代理可用: {proxy.host}:{proxy.port}, " + f"出口IP: {origin_ip}, 响应时间: {response_time:.2f}s" + ) + return True + + except Exception as e: + logger.debug(f"代理检测失败: {proxy.host}:{proxy.port} - {e}") + return False +``` + +### 代理爬虫完整示例 + +下面展示一个完整的代理爬虫示例，使用 httpbin.org 作为测试目标： + +```python +import asyncio +import httpx +from typing import Optional, Dict, Any +from loguru import logger +from dataclasses import dataclass + + +@dataclass +class ProxyCrawlerConfig: + """代理爬虫配置""" + # 代理池配置 + min_proxies: int = 10 + max_proxies: int = 50 + + # 请求配置 + request_timeout: int = 30 + max_retries: int = 3 + retry_delay: float = 1.0 + + # 频率控制 + request_interval: float = 0.5 # 请求间隔(秒) + + +class ProxyCrawler: + """ + 代理爬虫 + + 特性： + - 自动代理轮换 + - 智能重试 + - 频率控制 + - 错误处理 + """ + + # 通用请求头 + DEFAULT_HEADERS = { + "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) " + "AppleWebKit/537.36 (KHTML, like Gecko) " + "Chrome/131.0.0.0 Safari/537.36", + "Accept": "application/json, text/plain, */*", + "Accept-Language": "zh-CN,zh;q=0.9,en;q=0.8", + } + + def __init__( + self, + proxy_pool: ProxyPool, + config: Optional[ProxyCrawlerConfig] = None + ): + self.proxy_pool = proxy_pool + self.config = config or ProxyCrawlerConfig() + self._last_request_time = 0.0 + + async def _wait_for_rate_limit(self): + """频率控制""" + import time + elapsed = time.time() - self._last_request_time + if elapsed < self.config.request_interval: + await asyncio.sleep(self.config.request_interval - elapsed) + self._last_request_time = time.time() + + async def _request( + self, + url: str, + params: Optional[Dict] = None, + headers: Optional[Dict] = None + ) -> Optional[Dict[str, Any]]: + """ + 发送带代理的请求 + + 自动处理代理轮换和重试 + """ + await self._wait_for_rate_limit() + + merged_headers = {**self.DEFAULT_HEADERS, **(headers or {})} + + for attempt in range(self.config.max_retries): + proxy = await self.proxy_pool.get_proxy() + if not proxy: + logger.warning("无可用代理，使用直连") + proxy_url = None + else: + proxy_url = proxy.url + + try: + async with httpx.AsyncClient( + proxies=proxy_url, + timeout=self.config.request_timeout, + headers=merged_headers + ) as client: + response = await client.get(url, params=params) + + # 处理响应 + if response.status_code == 200: + if proxy: + await self.proxy_pool.return_proxy(proxy, success=True) + return response.json() + + # HTTP错误 + if response.status_code == 429: + logger.warning("请求频率过高，等待后重试") + if proxy: + await self.proxy_pool.return_proxy(proxy, success=False) + await asyncio.sleep(self.config.retry_delay * 2) + continue + + if response.status_code in (403, 412): + logger.warning(f"IP被封禁 ({response.status_code})，切换代理") + if proxy: + await self.proxy_pool.return_proxy(proxy, success=False) + continue + + # 其他错误 + logger.warning(f"HTTP错误: {response.status_code}") + if proxy: + await self.proxy_pool.return_proxy(proxy, success=True) + return None + + except httpx.TimeoutException: + logger.warning(f"请求超时，切换代理重试 (尝试 {attempt + 1})") + if proxy: + await self.proxy_pool.return_proxy(proxy, success=False) + except Exception as e: + logger.error(f"请求异常: {e}") + if proxy: + await self.proxy_pool.return_proxy(proxy, success=False) + + logger.error(f"请求失败，已达最大重试次数: {url}") + return None + + async def get_with_proxy(self, url: str) -> Optional[Dict[str, Any]]: + """ + 使用代理获取URL + + Args: + url: 目标URL + + Returns: + 响应数据 + """ + return await self._request(url) + + +# 使用示例 +async def main(): + # 创建代理获取器 + fetcher = APIProxyFetcher( + api_url="https://your-proxy-api.com/get", + api_key="your_api_key", + count=20 + ) + + # 创建代理检测器 + checker = SiteProxyChecker(timeout=10) + + # 创建代理池 + pool = ProxyPool( + fetcher=fetcher, + checker=checker, + min_proxies=10, + max_proxies=50 + ) + + # 启动代理池 + await pool.start() + + try: + # 创建爬虫 + crawler = ProxyCrawler(pool) + + # 测试请求（使用 httpbin.org 验证代理生效） + result = await crawler.get_with_proxy("https://httpbin.org/ip") + if result: + print(f"当前出口IP: {result.get('origin')}") + + # 测试 headers + result = await crawler.get_with_proxy("https://httpbin.org/headers") + if result: + headers = result.get("headers", {}) + print(f"User-Agent: {headers.get('User-Agent', 'N/A')}") + + # 获取代理池统计 + stats = pool.get_stats() + logger.info(f"代理池统计: {stats}") + + finally: + await pool.stop() + + +if __name__ == "__main__": + asyncio.run(main()) +``` + +### 代理使用最佳实践 + +代理使用的一些通用建议： + +```mermaid +graph LR + subgraph 代理选择 + A1["高匿代理"] --> A2["住宅IP优先"] + A2 --> A3["国内节点"] + end + + subgraph 请求策略 + B1["控制频率"] --> B2["随机延迟"] + B2 --> B3["失败轮换"] + end + + subgraph 风控规避 + C1["完整请求头"] --> C2["Cookie携带"] + C2 --> C3["行为模拟"] + end + + A3 --> B1 + B3 --> C1 + + style A1 fill:#e3f2fd,stroke:#2196f3 + style B1 fill:#fff3e0,stroke:#ff9800 + style C1 fill:#e8f5e9,stroke:#4caf50 +``` + +**关键建议**： + +1. **代理类型**：大型网站对代理检测严格，推荐使用高匿住宅代理 +2. **请求频率**：单IP建议 0.5-1 秒/请求，避免触发频率限制 +3. **完整请求头**：必须携带 User-Agent、Accept 等头信息 +4. **Cookie 携带**：部分 API 需要登录态，代理请求也要携带 Cookie +5. **失败处理**：遇到 403/429 立即切换代理，避免 IP 被永久封禁 + +--- + +## 本章小结 + +本章我们学习了代理 IP 的完整知识体系： + +1. **代理基础**：代理类型、匿名度、来源选择 +2. **代理池设计**：获取器、检测器、分配器的接口设计 +3. **核心实现**：代理获取、有效性检测、智能分配 +4. **爬虫集成**：httpx 代理设置、自动轮换、隧道代理 +5. **最佳实践**：代理检测、错误处理、使用建议 + +代理 IP 是大规模爬虫的基础设施，合理使用可以有效应对 IP 封禁。 + +--- + +## 下一章预告 + +下一章我们将学习「Playwright 浏览器自动化入门」。主要内容包括： + +- Playwright 的安装和基本使用 +- 页面导航和元素定位 +- 等待策略和超时处理 +- 截图和 PDF 导出 +- 爬取 JavaScript 渲染的页面 + +浏览器自动化是应对复杂反爬的利器，让我们一起探索！ diff --git "a/docs/\347\210\254\350\231\253\350\277\233\344\273\267/04_Playwright\346\265\217\350\247\210\345\231\250\350\207\252\345\212\250\345\214\226\345\205\245\351\227\250.md" "b/docs/\347\210\254\350\231\253\350\277\233\344\273\267/04_Playwright\346\265\217\350\247\210\345\231\250\350\207\252\345\212\250\345\214\226\345\205\245\351\227\250.md" new file mode 100644 index 0000000..dbb598d --- /dev/null +++ "b/docs/\347\210\254\350\231\253\350\277\233\344\273\267/04_Playwright\346\265\217\350\247\210\345\231\250\350\207\252\345\212\250\345\214\226\345\205\245\351\227\250.md" @@ -0,0 +1,969 @@ +# Playwright 浏览器自动化入门 + +> 当网页内容需要 JavaScript 渲染，或者需要模拟复杂的用户交互时，传统的 HTTP 请求方式就无能为力了。这时候，浏览器自动化工具就成了我们的利器。本章将带你入门 Playwright——一个现代、强大的浏览器自动化库。 + +## 为什么选择 Playwright + +### 浏览器自动化工具对比 + +| 特性 | Selenium | Puppeteer | Playwright | +|------|----------|-----------|------------| +| 语言支持 | 多语言 | JavaScript/TypeScript | 多语言 | +| 浏览器支持 | 多浏览器 | 仅 Chromium | Chromium/Firefox/WebKit | +| 自动等待 | 手动 | 部分自动 | 完全自动 | +| 性能 | 较慢 | 快 | 快 | +| API 设计 | 较旧 | 现代 | 最现代 | +| 维护状态 | 活跃 | 活跃 | 微软维护，非常活跃 | + +### Playwright 的优势 + +1. **跨浏览器支持**：一套代码支持 Chromium、Firefox、WebKit +2. **自动等待机制**：智能等待元素可交互，减少 flaky 测试 +3. **网络拦截**：可以拦截和修改网络请求 +4. **移动端模拟**：内置多种设备配置 +5. **同步/异步 API**：Python 版本同时支持同步和异步模式 + +--- + +## 安装与配置 + +### 安装 Playwright + +```bash +pip install playwright +``` + +### 安装浏览器 + +```bash +# 安装所有支持的浏览器 +playwright install + +# 或者只安装 Chromium +playwright install chromium + +# 安装 Firefox +playwright install firefox + +# 安装 WebKit（Safari 内核） +playwright install webkit +``` + +### 验证安装 + +```python +from playwright.sync_api import sync_playwright + +with sync_playwright() as p: + browser = p.chromium.launch(headless=False) + page = browser.new_page() + page.goto("https://example.com") + print(page.title()) + browser.close() +``` + +--- + +## 核心概念 + +### Browser/Context/Page 三层模型 + +```mermaid +graph TB + subgraph Browser["Browser 浏览器实例"] + direction TB + + subgraph Context1["Browser Context 1
(登录态环境)"] + Page1["Page 1
网站首页"] + Page2["Page 2
详情页面"] + end + + subgraph Context2["Browser Context 2
(匿名环境)"] + Page3["Page 3
搜索页面"] + end + end + + style Browser fill:#e3f2fd,stroke:#1976d2 + style Context1 fill:#e8f5e9,stroke:#4caf50 + style Context2 fill:#fff3e0,stroke:#ff9800 + style Page1 fill:#f3e5f5,stroke:#9c27b0 + style Page2 fill:#f3e5f5,stroke:#9c27b0 + style Page3 fill:#f3e5f5,stroke:#9c27b0 +``` + +**三层架构说明**： + +```mermaid +flowchart LR + subgraph Browser层 + B["Browser
浏览器进程"] + end + + subgraph Context层 + C1["Context
Cookie/Storage"] + C2["Context
独立环境"] + end + + subgraph Page层 + P1["Page
页面操作"] + P2["Page
DOM/Network"] + end + + B --> C1 & C2 + C1 --> P1 + C2 --> P2 + + style B fill:#e3f2fd,stroke:#2196f3 + style C1 fill:#e8f5e9,stroke:#4caf50 + style C2 fill:#e8f5e9,stroke:#4caf50 + style P1 fill:#fff3e0,stroke:#ff9800 + style P2 fill:#fff3e0,stroke:#ff9800 +``` + +- **Browser**：浏览器实例，对应一个浏览器进程 +- **Browser Context**：独立的浏览器环境，有独立的 Cookie、localStorage 等 +- **Page**：一个标签页，在这里进行实际的页面操作 + +> **应用场景**：可以用不同 Context 分别处理登录态和匿名访问，互不干扰。 + +### 同步 vs 异步 API + +Playwright Python 版本提供两套 API： + +```python +# 同步 API +from playwright.sync_api import sync_playwright + +with sync_playwright() as p: + browser = p.chromium.launch() + page = browser.new_page() + page.goto("https://example.com") + browser.close() + +# 异步 API +from playwright.async_api import async_playwright +import asyncio + +async def main(): + async with async_playwright() as p: + browser = await p.chromium.launch() + page = await browser.new_page() + await page.goto("https://example.com") + await browser.close() + +asyncio.run(main()) +``` + +对于爬虫项目，推荐使用**异步 API**，可以更好地与其他异步代码配合。 + +--- + +## 页面导航 + +### 基本导航 + +```python +# 打开页面 +await page.goto("https://example.com") + +# 等待特定状态 +await page.goto("https://example.com", wait_until="domcontentloaded") # DOM 加载完成 +await page.goto("https://example.com", wait_until="load") # 页面完全加载 +await page.goto("https://example.com", wait_until="networkidle") # 网络空闲 + +# 刷新页面 +await page.reload() + +# 前进/后退 +await page.go_back() +await page.go_forward() +``` + +### wait_until 参数说明 + +| 值 | 说明 | 使用场景 | +|---|------|---------| +| `domcontentloaded` | DOM 解析完成 | 快速获取页面结构 | +| `load` | 所有资源加载完成 | 需要图片等资源 | +| `networkidle` | 网络空闲（500ms 无请求） | SPA 应用，动态加载 | +| `commit` | 收到第一个响应字节 | 最快，不常用 | + +--- + +## 元素定位 + +Playwright 提供多种元素定位方式，推荐优先使用语义化的定位器。 + +### 推荐：使用 Locator + +```python +# 通过文本定位 +page.get_by_text("登录") +page.get_by_text("登录", exact=True) # 精确匹配 + +# 通过角色定位（推荐用于按钮、链接等） +page.get_by_role("button", name="提交") +page.get_by_role("link", name="首页") +page.get_by_role("textbox", name="用户名") + +# 通过标签定位 +page.get_by_label("用户名") +page.get_by_label("密码") + +# 通过占位符定位 +page.get_by_placeholder("请输入用户名") + +# 通过 test-id 定位（需要在 HTML 中添加 data-testid 属性） +page.get_by_test_id("submit-button") +``` + +### CSS 选择器 + +```python +# CSS 选择器 +page.locator("div.container") +page.locator("#login-form") +page.locator("input[type='text']") +page.locator("div.item:nth-child(2)") +``` + +### XPath + +```python +# XPath +page.locator("//div[@class='container']") +page.locator("//button[contains(text(), '提交')]") +page.locator("//input[@id='username']") +``` + +### 组合定位 + +```python +# 链式定位 +page.locator("div.container").locator("button.submit") + +# 过滤 +page.locator("div.item").filter(has_text="特价") +page.locator("div.item").filter(has=page.get_by_role("button")) + +# 第 N 个元素 +page.locator("div.item").nth(0) # 第一个 +page.locator("div.item").first # 第一个 +page.locator("div.item").last # 最后一个 +``` + +--- + +## 交互操作 + +### 点击 + +```python +# 基本点击 +await page.click("button.submit") +await page.locator("button.submit").click() + +# 双击 +await page.dblclick("div.item") + +# 右键点击 +await page.click("div.item", button="right") + +# 点击位置 +await page.click("div.map", position={"x": 100, "y": 200}) + +# 强制点击（忽略可见性检查） +await page.click("button.hidden", force=True) +``` + +### 输入 + +```python +# 输入文本 +await page.fill("input#username", "myuser") +await page.locator("input#password").fill("mypassword") + +# 逐字符输入（模拟真实打字） +await page.type("input#search", "hello world", delay=100) + +# 清空后输入 +await page.fill("input#search", "") +await page.fill("input#search", "new text") +``` + +### 选择 + +```python +# 下拉选择（select 元素） +await page.select_option("select#country", "china") +await page.select_option("select#country", value="cn") +await page.select_option("select#country", label="中国") + +# 多选 +await page.select_option("select#tags", ["tag1", "tag2"]) + +# 复选框/单选框 +await page.check("input#agree") +await page.uncheck("input#newsletter") +await page.set_checked("input#agree", True) +``` + +### 键盘操作 + +```python +# 按键 +await page.keyboard.press("Enter") +await page.keyboard.press("Tab") +await page.keyboard.press("Control+A") +await page.keyboard.press("Control+C") + +# 输入文本 +await page.keyboard.type("Hello World") + +# 组合键 +await page.keyboard.down("Shift") +await page.keyboard.press("ArrowDown") +await page.keyboard.up("Shift") +``` + +### 鼠标操作 + +```python +# 移动鼠标 +await page.mouse.move(100, 200) + +# 点击 +await page.mouse.click(100, 200) +await page.mouse.dblclick(100, 200) + +# 拖拽 +await page.mouse.move(100, 200) +await page.mouse.down() +await page.mouse.move(300, 400) +await page.mouse.up() + +# 悬停 +await page.hover("div.dropdown") +``` + +--- + +## 等待策略 + +### 自动等待 + +Playwright 的一大优势是**自动等待**。当你执行操作时，它会自动等待元素满足条件： + +```python +# 点击时自动等待元素可见、可交互 +await page.click("button.submit") + +# fill 时自动等待元素可编辑 +await page.fill("input#name", "value") +``` + +### 显式等待 + +有时需要显式等待某些条件： + +```python +# 等待元素出现 +await page.wait_for_selector("div.content") +await page.wait_for_selector("div.content", state="visible") +await page.wait_for_selector("div.content", state="hidden") + +# 等待页面状态 +await page.wait_for_load_state("networkidle") +await page.wait_for_load_state("domcontentloaded") + +# 等待 URL 变化 +await page.wait_for_url("**/success") +await page.wait_for_url(lambda url: "success" in url) + +# 等待特定时间 +await page.wait_for_timeout(1000) # 等待 1 秒（尽量避免使用） + +# 等待函数返回 True +await page.wait_for_function("document.querySelector('.loaded') !== null") +``` + +### 超时设置 + +```python +# 全局超时 +browser = await p.chromium.launch() +context = await browser.new_context() +page = await context.new_page() +page.set_default_timeout(30000) # 30 秒 + +# 单次操作超时 +await page.click("button.submit", timeout=5000) +await page.wait_for_selector("div.result", timeout=10000) +``` + +--- + +## 页面内容提取 + +### 获取文本 + +```python +# 获取元素文本 +text = await page.locator("h1.title").text_content() +texts = await page.locator("div.item").all_text_contents() + +# 获取可见文本 +visible_text = await page.locator("div.content").inner_text() +``` + +### 获取属性 + +```python +# 获取属性 +href = await page.locator("a.link").get_attribute("href") +src = await page.locator("img.avatar").get_attribute("src") + +# 获取多个元素的属性 +hrefs = await page.locator("a").evaluate_all("els => els.map(el => el.href)") +``` + +### 获取 HTML + +```python +# 获取元素 HTML +html = await page.locator("div.content").inner_html() + +# 获取整个页面 HTML +full_html = await page.content() +``` + +### 执行 JavaScript + +```python +# 执行 JS 并获取返回值 +title = await page.evaluate("document.title") + +# 传递参数 +result = await page.evaluate("(x, y) => x + y", 1, 2) + +# 在元素上执行 JS +text = await page.locator("div.item").evaluate("el => el.textContent") +``` + +--- + +## 网络请求拦截 + +Playwright 可以拦截和修改网络请求，这对于爬虫非常有用。 + +### 监听请求 + +```python +# 监听请求 +def on_request(request): + print(f"Request: {request.method} {request.url}") + +page.on("request", on_request) + +# 监听响应 +def on_response(response): + print(f"Response: {response.status} {response.url}") + +page.on("response", on_response) + +# 等待特定请求 +async with page.expect_request("**/api/data") as request_info: + await page.click("button.load") +request = await request_info.value + +# 等待特定响应 +async with page.expect_response("**/api/data") as response_info: + await page.click("button.load") +response = await response_info.value +data = await response.json() +``` + +### 拦截请求 + +```python +# 拦截并修改请求 +async def handle_route(route): + # 修改请求头 + headers = {**route.request.headers, "X-Custom": "value"} + await route.continue_(headers=headers) + +await page.route("**/*", handle_route) + +# 阻止某些资源加载（提高性能） +await page.route("**/*.{png,jpg,jpeg,gif}", lambda route: route.abort()) +await page.route("**/analytics.js", lambda route: route.abort()) + +# 返回模拟响应 +async def mock_api(route): + await route.fulfill( + status=200, + content_type="application/json", + body='{"data": "mocked"}' + ) + +await page.route("**/api/data", mock_api) +``` + +--- + +## 截图与 PDF + +### 截图 + +```python +# 页面截图 +await page.screenshot(path="screenshot.png") + +# 全页面截图 +await page.screenshot(path="full.png", full_page=True) + +# 元素截图 +await page.locator("div.chart").screenshot(path="chart.png") + +# 截图选项 +await page.screenshot( + path="screenshot.png", + type="png", # png 或 jpeg + quality=80, # jpeg 质量（0-100） + full_page=True, # 全页面 + clip={"x": 0, "y": 0, "width": 800, "height": 600} # 裁剪区域 +) +``` + +### 导出 PDF + +```python +# 导出 PDF（仅 Chromium 支持） +await page.pdf(path="page.pdf") + +# PDF 选项 +await page.pdf( + path="page.pdf", + format="A4", + print_background=True, + margin={"top": "1cm", "bottom": "1cm"} +) +``` + +--- + +## 浏览器配置 + +### 有头/无头模式 + +```python +# 有头模式（显示浏览器窗口） +browser = await p.chromium.launch(headless=False) + +# 无头模式（不显示窗口，默认） +browser = await p.chromium.launch(headless=True) +``` + +### 视口设置 + +```python +# 设置视口大小 +context = await browser.new_context( + viewport={"width": 1920, "height": 1080} +) + +# 模拟移动设备 +from playwright.async_api import async_playwright + +async with async_playwright() as p: + iphone = p.devices["iPhone 13"] + browser = await p.webkit.launch() + context = await browser.new_context(**iphone) +``` + +### 代理设置 + +```python +# 设置代理 +browser = await p.chromium.launch( + proxy={ + "server": "http://proxy.example.com:8080", + "username": "user", + "password": "pass" + } +) +``` + +### 用户数据目录 + +```python +# 使用持久化的用户数据目录（保存 Cookie 等） +context = await p.chromium.launch_persistent_context( + user_data_dir="./user_data", + headless=False +) +``` + +--- + +## Playwright 实战演练 + +### 为什么需要浏览器自动化 + +当网页使用 JavaScript 动态渲染内容时，传统的 HTTP 请求无法获取渲染后的数据。Playwright 可以完整模拟浏览器行为，等待页面渲染完成后再提取数据。 + +```mermaid +flowchart TD + subgraph 适用场景 + SPA["SPA应用
动态路由"] + Lazy["懒加载
滚动触发"] + JS["JS渲染
动态内容"] + end + + subgraph Playwright优势 + Wait["智能等待
自动判断"] + Interact["模拟交互
点击/输入"] + Network["网络拦截
获取API数据"] + end + + SPA --> Wait + Lazy --> Interact + JS --> Network + + style SPA fill:#e3f2fd,stroke:#2196f3 + style Lazy fill:#fff3e0,stroke:#ff9800 + style JS fill:#e8f5e9,stroke:#4caf50 +``` + +### 实战：爬取 JS 渲染的名言网站 + +quotes.toscrape.com/js 是一个专门用于学习的网站，其内容通过 JavaScript 动态渲染，非常适合作为 Playwright 入门练习。 + +```python +# -*- coding: utf-8 -*- +""" +使用 Playwright 爬取 JS 渲染的名言网站 +""" + +import asyncio +from playwright.async_api import async_playwright +from loguru import logger +from typing import List, Dict, Any + + +async def scrape_quotes_js() -> List[Dict[str, Any]]: + """爬取 JS 渲染的名言网站""" + async with async_playwright() as p: + # 启动浏览器 + browser = await p.chromium.launch(headless=True) + + # 创建上下文，设置视口 + context = await browser.new_context( + viewport={"width": 1920, "height": 1080}, + locale="en-US" + ) + page = await context.new_page() + + try: + logger.info("正在访问名言网站...") + await page.goto( + "https://quotes.toscrape.com/js/", + wait_until="networkidle" + ) + + # 等待名言加载（JS 渲染需要时间） + await page.wait_for_selector(".quote", timeout=10000) + + # 提取名言 + quote_elements = await page.locator(".quote").all() + logger.info(f"找到 {len(quote_elements)} 条名言") + + results = [] + for quote_el in quote_elements: + try: + # 提取名言文本 + text_el = quote_el.locator(".text") + text = await text_el.text_content() + + # 提取作者 + author_el = quote_el.locator(".author") + author = await author_el.text_content() + + # 提取标签 + tag_elements = await quote_el.locator(".tag").all() + tags = [await tag.text_content() for tag in tag_elements] + + results.append({ + "text": text.strip() if text else "", + "author": author.strip() if author else "", + "tags": tags + }) + except Exception as e: + logger.debug(f"提取名言失败: {e}") + continue + + # 输出结果 + logger.info(f"成功提取 {len(results)} 条名言") + return results + + finally: + await browser.close() + + +async def main(): + quotes = await scrape_quotes_js() + + print("\n=== JS 渲染页面爬取结果 ===\n") + for i, quote in enumerate(quotes[:5], 1): + print(f"{i}. {quote['text'][:60]}...") + print(f" 作者: {quote['author']}") + print(f" 标签: {', '.join(quote['tags'])}") + print() + + +if __name__ == "__main__": + asyncio.run(main()) +``` + +### 实战：分页爬取与滚动加载 + +```python +# -*- coding: utf-8 -*- +""" +使用 Playwright 处理分页和滚动加载 +""" + +import asyncio +from playwright.async_api import async_playwright +from loguru import logger +from typing import List, Dict, Any + + +async def scrape_multiple_pages(max_pages: int = 3) -> List[Dict[str, Any]]: + """ + 爬取多页名言 + + Args: + max_pages: 最大页数 + + Returns: + 名言列表 + """ + all_quotes = [] + + async with async_playwright() as p: + browser = await p.chromium.launch(headless=True) + context = await browser.new_context( + viewport={"width": 1920, "height": 1080} + ) + page = await context.new_page() + + try: + for page_num in range(1, max_pages + 1): + url = f"https://quotes.toscrape.com/js/page/{page_num}/" + logger.info(f"正在爬取第 {page_num} 页...") + + await page.goto(url, wait_until="networkidle") + await page.wait_for_selector(".quote", timeout=10000) + + # 提取当前页名言 + quote_elements = await page.locator(".quote").all() + + for quote_el in quote_elements: + try: + text = await quote_el.locator(".text").text_content() + author = await quote_el.locator(".author").text_content() + + all_quotes.append({ + "text": text.strip() if text else "", + "author": author.strip() if author else "", + "page": page_num + }) + except Exception as e: + logger.debug(f"提取失败: {e}") + continue + + logger.info(f"第 {page_num} 页爬取完成，当前共 {len(all_quotes)} 条") + + # 适当延迟，避免请求过快 + await page.wait_for_timeout(500) + + finally: + await browser.close() + + return all_quotes + + +async def main(): + quotes = await scrape_multiple_pages(max_pages=3) + + print(f"\n=== 共爬取 {len(quotes)} 条名言 ===\n") + for i, quote in enumerate(quotes[:10], 1): + print(f"{i}. [{quote['page']}页] {quote['text'][:50]}...") + print(f" 作者: {quote['author']}") + print() + + +if __name__ == "__main__": + asyncio.run(main()) +``` + +### 实战：拦截网络请求 + +Playwright 的网络拦截功能可以直接获取 API 返回的 JSON 数据，比解析 DOM 更高效： + +```python +# -*- coding: utf-8 -*- +""" +使用 Playwright 拦截网络请求 +""" + +import asyncio +from playwright.async_api import async_playwright +from loguru import logger +from typing import List, Dict, Any + + +async def intercept_api_requests() -> List[Dict[str, Any]]: + """ + 拦截 API 请求获取数据 + + 通过网络拦截直接获取 JSON 数据 + """ + api_responses = [] + + async with async_playwright() as p: + browser = await p.chromium.launch(headless=True) + context = await browser.new_context() + page = await context.new_page() + + # 监听所有响应 + async def handle_response(response): + url = response.url + # 过滤 API 请求 + if response.request.resource_type == "xhr" or response.request.resource_type == "fetch": + try: + if response.status == 200: + content_type = response.headers.get("content-type", "") + if "json" in content_type: + data = await response.json() + api_responses.append({ + "url": url, + "data": data + }) + logger.info(f"拦截到 API 响应: {url[:60]}...") + except Exception as e: + logger.debug(f"解析响应失败: {e}") + + page.on("response", handle_response) + + # 访问页面触发请求 + await page.goto("https://quotes.toscrape.com/js/", wait_until="networkidle") + + # 等待数据拦截完成 + await page.wait_for_timeout(2000) + + await browser.close() + + return api_responses + + +async def demo_block_resources(): + """演示阻止资源加载以提升性能""" + async with async_playwright() as p: + browser = await p.chromium.launch(headless=True) + page = await browser.new_page() + + # 阻止图片和字体加载 + await page.route("**/*.{png,jpg,jpeg,gif,svg}", lambda route: route.abort()) + await page.route("**/*.{woff,woff2,ttf}", lambda route: route.abort()) + + logger.info("已设置资源拦截，图片和字体将不会加载") + + await page.goto("https://quotes.toscrape.com/") + await page.wait_for_selector(".quote") + + quotes_count = await page.locator(".quote").count() + logger.info(f"页面加载完成，找到 {quotes_count} 条名言（无图片模式）") + + await browser.close() + + +async def main(): + print("=== 拦截 API 请求示例 ===") + responses = await intercept_api_requests() + print(f"共拦截到 {len(responses)} 个 API 响应\n") + + print("=== 阻止资源加载示例 ===") + await demo_block_resources() + + +if __name__ == "__main__": + asyncio.run(main()) +``` + +### Playwright 使用注意事项 + +```mermaid +graph LR + subgraph 反检测 + A1["隐藏自动化特征"] + A2["模拟真实UA"] + A3["设置合理视口"] + end + + subgraph 性能优化 + B1["禁用图片加载"] + B2["拦截无关资源"] + B3["复用浏览器实例"] + end + + subgraph 稳定性 + C1["适当等待时间"] + C2["处理弹窗"] + C3["异常重试机制"] + end + + A1 --> B1 --> C1 + + style A1 fill:#e3f2fd,stroke:#2196f3 + style B1 fill:#fff3e0,stroke:#ff9800 + style C1 fill:#e8f5e9,stroke:#4caf50 +``` + +**关键注意点**： + +1. **反自动化检测**：部分网站会检测 `navigator.webdriver`，需要使用反检测技术（下一章详解） +2. **请求频率**：避免频繁刷新页面，建议间隔 1-2 秒 +3. **Cookie 管理**：登录态通过 Context 的 `storage_state` 保存和恢复 +4. **资源优化**：禁用图片/字体加载可大幅提升速度 +5. **网络拦截**：直接拦截 API 响应比解析 DOM 更稳定高效 + +--- + +## 本章小结 + +本章我们学习了 Playwright 浏览器自动化的基础知识： + +1. **核心概念**：Browser/Context/Page 三层模型，同步/异步 API +2. **页面导航**：goto、等待策略、导航控制 +3. **元素定位**：语义化定位器、CSS、XPath +4. **交互操作**：点击、输入、选择、键盘、鼠标 +5. **等待策略**：自动等待、显式等待、超时设置 +6. **内容提取**：文本、属性、HTML、执行 JavaScript +7. **网络拦截**：监听请求响应、拦截修改请求 +8. **实战演练**：爬取 JS 渲染页面、分页处理、资源拦截 + +--- + +## 下一章预告 + +下一章我们将学习「Playwright 进阶：反检测与性能优化」。主要内容包括： + +- 浏览器指纹检测原理 +- stealth.min.js 反检测注入 +- CDP 模式的使用 +- 性能优化技巧（禁用资源、复用上下文） +- 异常处理和资源管理 + +这些进阶技巧将帮助你的爬虫更好地应对严格的反爬检测。 diff --git "a/docs/\347\210\254\350\231\253\350\277\233\344\273\267/05_Playwright\350\277\233\351\230\266_\345\217\215\346\243\200\346\265\213\344\270\216\346\200\247\350\203\275\344\274\230\345\214\226.md" "b/docs/\347\210\254\350\231\253\350\277\233\344\273\267/05_Playwright\350\277\233\351\230\266_\345\217\215\346\243\200\346\265\213\344\270\216\346\200\247\350\203\275\344\274\230\345\214\226.md" new file mode 100644 index 0000000..4f549cb --- /dev/null +++ "b/docs/\347\210\254\350\231\253\350\277\233\344\273\267/05_Playwright\350\277\233\351\230\266_\345\217\215\346\243\200\346\265\213\344\270\216\346\200\247\350\203\275\344\274\230\345\214\226.md" @@ -0,0 +1,958 @@ +# Playwright 进阶：反检测与性能优化 + +> 掌握了 Playwright 的基础操作后，你会发现在实际爬虫场景中，很多网站能够检测到自动化行为并进行封禁。本章将深入讲解浏览器指纹检测原理，以及如何使用 stealth.js 等技术绕过检测，同时介绍性能优化技巧。 + +## 浏览器指纹检测原理 + +### 什么是浏览器指纹 + +浏览器指纹是通过收集浏览器的各种特征（如 User-Agent、屏幕分辨率、插件列表、字体列表等）来唯一标识一个浏览器的技术。即使没有 Cookie，网站也可以通过指纹追踪用户。 + +### 自动化浏览器的检测点 + +```mermaid +flowchart TD + subgraph 检测层级 + L1["Level 1: 基础检测"] + L2["Level 2: 特征检测"] + L3["Level 3: 行为检测"] + end + + subgraph 基础检测项 + D1["navigator.webdriver"] + D2["window.chrome"] + D3["navigator.plugins"] + end + + subgraph 特征检测项 + D4["Canvas指纹"] + D5["WebGL指纹"] + D6["Audio指纹"] + end + + subgraph 行为检测项 + D7["鼠标轨迹"] + D8["键盘节奏"] + D9["页面停留"] + end + + L1 --> D1 & D2 & D3 + L2 --> D4 & D5 & D6 + L3 --> D7 & D8 & D9 + + style L1 fill:#e8f5e9,stroke:#4caf50 + style L2 fill:#fff3e0,stroke:#ff9800 + style L3 fill:#ffebee,stroke:#f44336 +``` + +| 检测类型 | 检测方法 | 说明 | +|---------|---------|------| +| WebDriver 标志 | `navigator.webdriver` | Playwright 默认为 true | +| Chrome 特征 | `window.chrome` | 自动化环境可能缺失 | +| 插件检测 | `navigator.plugins` | 自动化环境插件列表异常 | +| 语言检测 | `navigator.languages` | 配置不当可能暴露 | +| 权限 API | `navigator.permissions` | 行为与真实浏览器不同 | +| Canvas 指纹 | `canvas.toDataURL()` | 渲染结果可能有差异 | +| WebGL 指纹 | WebGL 参数 | 可能暴露虚拟化环境 | +| 时区/语言 | 系统设置 | 与声称的地区不符 | + +### 常见的检测脚本 + +```javascript +// 检测 WebDriver +if (navigator.webdriver) { + console.log("检测到自动化浏览器"); +} + +// 检测 Chrome 特征 +if (!window.chrome) { + console.log("可能是自动化环境"); +} + +// 检测 plugins +if (navigator.plugins.length === 0) { + console.log("插件列表为空，可能是自动化"); +} + +// 检测 permissions +navigator.permissions.query({name: "notifications"}).then(result => { + if (result.state === "prompt") { + // 正常行为 + } +}); +``` + +--- + +## stealth.js 反检测技术 + +### 什么是 stealth.js + +`stealth.js` 是 puppeteer-extra-plugin-stealth 项目提取出的反检测脚本，它通过修改浏览器的各种属性来伪装自动化浏览器，使其看起来像真实用户操作的浏览器。 + +这也是 MediaCrawler 项目使用的核心反检测技术。 + +### stealth.min.js 的工作原理 + +stealth.js 主要做以下修改： + +1. **隐藏 WebDriver 标志**：将 `navigator.webdriver` 设置为 `undefined` +2. **模拟 Chrome 特征**：添加 `window.chrome` 对象 +3. **修改 plugins**：模拟正常的插件列表 +4. **修改 permissions**：使权限 API 行为正常 +5. **修复 iframe 检测**：处理 contentWindow 问题 +6. **其他特征修复**：语言、时区等 + +### 在 Playwright 中使用 stealth.js + +```python +import asyncio +from playwright.async_api import async_playwright + +# stealth.min.js 脚本内容（实际使用时从文件加载） +STEALTH_JS = """ +// 隐藏 webdriver 标志 +Object.defineProperty(navigator, 'webdriver', { + get: () => undefined +}); + +// 模拟 chrome 对象 +window.chrome = { + runtime: {} +}; + +// 模拟 plugins +Object.defineProperty(navigator, 'plugins', { + get: () => [1, 2, 3, 4, 5] +}); + +// 模拟 languages +Object.defineProperty(navigator, 'languages', { + get: () => ['zh-CN', 'zh', 'en'] +}); + +// 修复 permissions +const originalQuery = window.navigator.permissions.query; +window.navigator.permissions.query = (parameters) => ( + parameters.name === 'notifications' ? + Promise.resolve({ state: Notification.permission }) : + originalQuery(parameters) +); +""" + + +async def create_stealth_page(browser): + """创建带反检测的页面""" + context = await browser.new_context() + + # 在每个新页面创建时注入 stealth 脚本 + await context.add_init_script(STEALTH_JS) + + page = await context.new_page() + return page + + +async def main(): + async with async_playwright() as p: + browser = await p.chromium.launch(headless=True) + page = await create_stealth_page(browser) + + # 测试反检测效果 + await page.goto("https://bot.sannysoft.com/") + await page.screenshot(path="stealth_test.png", full_page=True) + + await browser.close() + + +asyncio.run(main()) +``` + +### 完整的 stealth.min.js + +实际项目中，建议使用完整的 stealth.min.js 文件。你可以从以下来源获取： + +1. [puppeteer-extra-plugin-stealth](https://github.com/berstend/puppeteer-extra/tree/master/packages/puppeteer-extra-plugin-stealth) +2. MediaCrawler 项目中的 stealth.min.js + +加载方式： + +```python +async def create_stealth_context(browser, stealth_js_path: str): + """创建带完整反检测的上下文""" + context = await browser.new_context() + + # 从文件加载 stealth.js + with open(stealth_js_path, "r") as f: + stealth_js = f.read() + + await context.add_init_script(stealth_js) + return context +``` + +--- + +## CDP 模式 + +### 什么是 CDP + +CDP（Chrome DevTools Protocol）是 Chrome 浏览器提供的调试协议，允许程序直接与浏览器通信。Playwright 内部就是使用 CDP 与 Chromium 通信的。 + +### 标准模式 vs CDP 模式 + +| 特性 | 标准模式 | CDP 模式 | +|------|---------|---------| +| API | Playwright 封装的 API | 原始 CDP 命令 | +| 功能 | 常用功能 | 完整的浏览器控制 | +| 复杂度 | 简单 | 较复杂 | +| 使用场景 | 大多数场景 | 需要底层控制时 | + +### 使用 CDP 连接已有浏览器 + +```python +import asyncio +from playwright.async_api import async_playwright + + +async def connect_existing_browser(): + """连接到已经运行的浏览器""" + async with async_playwright() as p: + # 首先启动一个带调试端口的浏览器 + # chrome --remote-debugging-port=9222 + + # 通过 CDP 连接 + browser = await p.chromium.connect_over_cdp( + "http://localhost:9222" + ) + + # 获取所有页面 + contexts = browser.contexts + if contexts: + pages = contexts[0].pages + if pages: + page = pages[0] + print(f"当前页面: {page.url}") + + await browser.close() +``` + +### 使用 CDP 命令 + +```python +async def use_cdp_commands(page): + """直接使用 CDP 命令""" + # 获取 CDP session + client = await page.context.new_cdp_session(page) + + # 执行 CDP 命令 + # 例如：获取性能指标 + metrics = await client.send("Performance.getMetrics") + print(metrics) + + # 例如：模拟网络条件 + await client.send("Network.emulateNetworkConditions", { + "offline": False, + "downloadThroughput": 1000000, # 1 MB/s + "uploadThroughput": 500000, + "latency": 100 + }) + + # 例如：截取完整页面 + result = await client.send("Page.captureScreenshot", { + "format": "png", + "captureBeyondViewport": True + }) +``` + +--- + +## 性能优化 + +### 禁用不必要的资源加载 + +加载图片、字体、CSS 等资源会消耗大量时间和带宽。对于只需要获取数据的爬虫，可以禁用这些资源： + +```python +async def create_optimized_context(browser): + """创建优化性能的上下文""" + context = await browser.new_context() + + # 拦截并阻止不必要的资源 + await context.route("**/*.{png,jpg,jpeg,gif,svg,ico}", lambda route: route.abort()) + await context.route("**/*.{woff,woff2,ttf,otf}", lambda route: route.abort()) + await context.route("**/*.css", lambda route: route.abort()) + + # 阻止分析和追踪脚本 + await context.route("**/analytics.js", lambda route: route.abort()) + await context.route("**/gtag/**", lambda route: route.abort()) + await context.route("**/facebook.com/**", lambda route: route.abort()) + + return context + + +async def selective_blocking(page): + """选择性阻止资源""" + async def handle_route(route): + resource_type = route.request.resource_type + + # 只允许 document、script、xhr、fetch + if resource_type in ["document", "script", "xhr", "fetch"]: + await route.continue_() + else: + await route.abort() + + await page.route("**/*", handle_route) +``` + +### 浏览器上下文复用 + +创建新的浏览器上下文比创建新页面消耗更多资源。合理复用上下文可以提高性能： + +```python +class BrowserPool: + """浏览器上下文池""" + + def __init__(self, browser, pool_size: int = 5): + self.browser = browser + self.pool_size = pool_size + self._contexts = [] + self._available = asyncio.Queue() + + async def initialize(self): + """初始化上下文池""" + for _ in range(self.pool_size): + context = await self.browser.new_context() + self._contexts.append(context) + await self._available.put(context) + + async def get_context(self): + """获取可用上下文""" + return await self._available.get() + + async def return_context(self, context): + """归还上下文""" + # 清理 cookies 和 storage + await context.clear_cookies() + await self._available.put(context) + + async def close_all(self): + """关闭所有上下文""" + for context in self._contexts: + await context.close() +``` + +### 多页面并发管理 + +```python +import asyncio + + +class ConcurrentCrawler: + """并发爬虫""" + + def __init__(self, browser, max_concurrent: int = 5): + self.browser = browser + self.max_concurrent = max_concurrent + self.semaphore = asyncio.Semaphore(max_concurrent) + + async def crawl_url(self, context, url: str) -> dict: + """爬取单个 URL""" + async with self.semaphore: + page = await context.new_page() + try: + await page.goto(url, wait_until="domcontentloaded") + title = await page.title() + return {"url": url, "title": title, "success": True} + except Exception as e: + return {"url": url, "error": str(e), "success": False} + finally: + await page.close() + + async def crawl_batch(self, urls: list) -> list: + """批量爬取""" + context = await self.browser.new_context() + try: + tasks = [self.crawl_url(context, url) for url in urls] + return await asyncio.gather(*tasks) + finally: + await context.close() +``` + +### 内存和资源监控 + +```python +async def monitor_resources(browser): + """监控浏览器资源使用""" + # 获取所有上下文 + contexts = browser.contexts + print(f"活跃上下文数: {len(contexts)}") + + total_pages = 0 + for ctx in contexts: + pages = ctx.pages + total_pages += len(pages) + for page in pages: + print(f" 页面: {page.url[:50]}...") + + print(f"总页面数: {total_pages}") + + +async def cleanup_stale_pages(context, max_age_seconds: int = 300): + """清理闲置页面""" + # 这里需要自己记录页面的创建时间 + # 示例仅展示逻辑 + for page in context.pages: + # 关闭空白页或闲置页 + if page.url == "about:blank": + await page.close() +``` + +--- + +## 异常处理 + +### 页面崩溃检测和恢复 + +```python +async def safe_navigate(page, url: str, max_retries: int = 3): + """安全的页面导航，带重试""" + for attempt in range(max_retries): + try: + await page.goto(url, timeout=30000) + return True + except Exception as e: + error_msg = str(e).lower() + + if "crash" in error_msg or "target closed" in error_msg: + # 页面崩溃，需要重新创建 + print(f"页面崩溃，尝试恢复 (尝试 {attempt + 1}/{max_retries})") + # 这里应该创建新页面 + continue + + elif "timeout" in error_msg: + # 超时，可以重试 + print(f"超时，重试 (尝试 {attempt + 1}/{max_retries})") + continue + + else: + # 其他错误 + print(f"导航错误: {e}") + raise + + return False +``` + +### 浏览器进程管理 + +```python +import signal +import atexit + + +class BrowserManager: + """浏览器进程管理器""" + + def __init__(self): + self._browser = None + self._playwright = None + + # 注册清理函数 + atexit.register(self._cleanup_sync) + signal.signal(signal.SIGTERM, self._signal_handler) + signal.signal(signal.SIGINT, self._signal_handler) + + async def start(self, playwright): + """启动浏览器""" + self._playwright = playwright + self._browser = await playwright.chromium.launch(headless=True) + return self._browser + + async def stop(self): + """停止浏览器""" + if self._browser: + await self._browser.close() + self._browser = None + + def _cleanup_sync(self): + """同步清理""" + if self._browser: + # 强制关闭 + try: + import asyncio + loop = asyncio.get_event_loop() + loop.run_until_complete(self.stop()) + except Exception: + pass + + def _signal_handler(self, signum, frame): + """信号处理""" + print(f"收到信号 {signum}，正在清理...") + self._cleanup_sync() + exit(0) +``` + +--- + +## 反检测实战 + +### 网站的反自动化检测机制 + +大型网站通常有较为完善的反自动化检测： + +```mermaid +flowchart LR + subgraph 检测机制 + Check1["WebDriver检测"] + Check2["请求头验证"] + Check3["行为分析"] + Check4["频率限制"] + end + + subgraph 检测结果 + Pass["正常访问"] + Block["触发风控"] + Captcha["触发验证码"] + end + + Check1 -->|通过| Check2 + Check1 -->|失败| Block + Check2 -->|通过| Check3 + Check2 -->|异常| Block + Check3 -->|正常| Pass + Check3 -->|异常| Captcha + Check4 -->|超限| Block + + style Pass fill:#c8e6c9,stroke:#4caf50 + style Block fill:#ffcdd2,stroke:#f44336 + style Captcha fill:#fff3e0,stroke:#ff9800 +``` + +### 完整反检测配置 + +```python +# -*- coding: utf-8 -*- +""" +Playwright 反检测配置 +""" + +import asyncio +from playwright.async_api import async_playwright, Browser, BrowserContext, Page +from loguru import logger +from typing import Optional + + +# 通用 stealth 脚本 +STEALTH_JS = """ +// 隐藏 webdriver 标志 +Object.defineProperty(navigator, 'webdriver', { + get: () => undefined +}); + +// 模拟 Chrome 对象 +window.chrome = { + runtime: {}, + loadTimes: function() {}, + csi: function() {}, + app: {} +}; + +// 模拟正常的插件列表 +Object.defineProperty(navigator, 'plugins', { + get: () => { + const plugins = [ + { + name: 'Chrome PDF Plugin', + description: 'Portable Document Format', + filename: 'internal-pdf-viewer' + }, + { + name: 'Chrome PDF Viewer', + description: '', + filename: 'mhjfbmdgcfjbbpaeojofohoefgiehjai' + }, + { + name: 'Native Client', + description: '', + filename: 'internal-nacl-plugin' + } + ]; + plugins.item = (i) => plugins[i]; + plugins.namedItem = (name) => plugins.find(p => p.name === name); + plugins.refresh = () => {}; + return plugins; + } +}); + +// 模拟语言设置 +Object.defineProperty(navigator, 'languages', { + get: () => ['zh-CN', 'zh', 'en-US', 'en'] +}); + +// 修复 permissions API +const originalQuery = window.navigator.permissions.query; +window.navigator.permissions.query = (parameters) => ( + parameters.name === 'notifications' + ? Promise.resolve({ state: Notification.permission }) + : originalQuery(parameters) +); + +// 模拟硬件并发数 +Object.defineProperty(navigator, 'hardwareConcurrency', { + get: () => 8 +}); + +// 模拟设备内存 +Object.defineProperty(navigator, 'deviceMemory', { + get: () => 8 +}); + +// 隐藏自动化相关属性 +delete window.cdc_adoQpoasnfa76pfcZLmcfl_Array; +delete window.cdc_adoQpoasnfa76pfcZLmcfl_Promise; +delete window.cdc_adoQpoasnfa76pfcZLmcfl_Symbol; +""" + + +class StealthBrowser: + """ + 反检测浏览器封装 + + 特性： + - 自动注入 stealth 脚本 + - 模拟真实浏览器环境 + - 支持资源优化 + - Cookie 管理 + """ + + def __init__(self, headless: bool = True): + self.headless = headless + self._playwright = None + self._browser: Optional[Browser] = None + self._context: Optional[BrowserContext] = None + + async def start(self) -> BrowserContext: + """启动浏览器并创建反检测上下文""" + from playwright.async_api import async_playwright + + self._playwright = await async_playwright().start() + + # 启动浏览器，添加反检测参数 + self._browser = await self._playwright.chromium.launch( + headless=self.headless, + args=[ + "--disable-blink-features=AutomationControlled", + "--disable-dev-shm-usage", + "--no-sandbox", + "--disable-setuid-sandbox", + "--disable-infobars", + "--window-size=1920,1080", + "--start-maximized", + ] + ) + + # 创建上下文 + self._context = await self._browser.new_context( + viewport={"width": 1920, "height": 1080}, + locale="zh-CN", + timezone_id="Asia/Shanghai", + user_agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) " + "AppleWebKit/537.36 (KHTML, like Gecko) " + "Chrome/131.0.0.0 Safari/537.36", + extra_http_headers={ + "Accept-Language": "zh-CN,zh;q=0.9,en;q=0.8", + } + ) + + # 注入 stealth 脚本 + await self._context.add_init_script(STEALTH_JS) + + logger.info("反检测浏览器已启动") + return self._context + + async def create_optimized_page(self) -> Page: + """创建性能优化的页面""" + if not self._context: + raise RuntimeError("浏览器未启动，请先调用 start()") + + page = await self._context.new_page() + + # 禁用不必要的资源 + await page.route("**/*.{png,jpg,jpeg,gif,webp,svg,ico}", lambda r: r.abort()) + await page.route("**/*.{woff,woff2,ttf,otf,eot}", lambda r: r.abort()) + await page.route("**/analytics**", lambda r: r.abort()) + await page.route("**/tracking**", lambda r: r.abort()) + + return page + + async def save_cookies(self, path: str): + """保存 Cookie 到文件""" + if self._context: + await self._context.storage_state(path=path) + logger.info(f"Cookie 已保存到: {path}") + + async def load_cookies(self, path: str): + """从文件加载 Cookie""" + import os + if os.path.exists(path): + # 需要重新创建 context + if self._context: + await self._context.close() + + self._context = await self._browser.new_context( + storage_state=path, + viewport={"width": 1920, "height": 1080}, + locale="zh-CN" + ) + await self._context.add_init_script(STEALTH_JS) + logger.info(f"Cookie 已从 {path} 加载") + + async def stop(self): + """关闭浏览器""" + if self._context: + await self._context.close() + if self._browser: + await self._browser.close() + if self._playwright: + await self._playwright.stop() + logger.info("浏览器已关闭") + + +async def test_stealth_browser(): + """测试反检测效果（使用 bot.sannysoft.com）""" + browser = StealthBrowser(headless=True) + context = await browser.start() + + try: + page = await browser.create_optimized_page() + + # 访问 WebDriver 检测网站 + logger.info("访问 bot.sannysoft.com 测试反检测效果...") + await page.goto("https://bot.sannysoft.com/", wait_until="networkidle") + + # 检查反检测效果 + webdriver = await page.evaluate("navigator.webdriver") + chrome = await page.evaluate("!!window.chrome") + plugins = await page.evaluate("navigator.plugins.length") + + logger.info(f"反检测检查:") + logger.info(f" - navigator.webdriver: {webdriver}") + logger.info(f" - window.chrome 存在: {chrome}") + logger.info(f" - plugins 数量: {plugins}") + + # 截图保存测试结果 + await page.screenshot(path="stealth_test.png", full_page=True) + logger.info("截图已保存到 stealth_test.png") + + finally: + await browser.stop() + + +if __name__ == "__main__": + asyncio.run(test_stealth_browser()) +``` + +### 性能优化爬虫配置 + +```python +# -*- coding: utf-8 -*- +""" +Playwright 性能优化配置 +""" + +import asyncio +from playwright.async_api import async_playwright, Route +from loguru import logger +from typing import Set + + +class OptimizedCrawler: + """ + 性能优化爬虫 + + 优化策略： + - 禁用图片/字体/CSS加载 + - 拦截广告和追踪脚本 + - 复用浏览器上下文 + - 智能等待策略 + """ + + # 需要阻止的资源类型 + BLOCKED_RESOURCE_TYPES: Set[str] = { + "image", + "font", + "stylesheet", + "media", + } + + # 需要阻止的URL模式 + BLOCKED_URL_PATTERNS = [ + "**/analytics**", + "**/tracking**", + "**/ads**", + "**/*.gif", + "**/*.png", + "**/*.jpg", + "**/*.jpeg", + "**/*.webp", + ] + + def __init__(self, headless: bool = True): + self.headless = headless + self._browser = None + self._context = None + + async def _route_handler(self, route: Route): + """路由处理器 - 决定是否阻止请求""" + request = route.request + + # 检查资源类型 + if request.resource_type in self.BLOCKED_RESOURCE_TYPES: + await route.abort() + return + + # 检查URL模式（广告和追踪） + url = request.url + for pattern in ["analytics", "tracking", "ads"]: + if pattern in url: + await route.abort() + return + + await route.continue_() + + async def start(self): + """启动优化后的浏览器""" + self._playwright = await async_playwright().start() + + self._browser = await self._playwright.chromium.launch( + headless=self.headless, + args=["--disable-blink-features=AutomationControlled"] + ) + + self._context = await self._browser.new_context( + viewport={"width": 1920, "height": 1080}, + locale="zh-CN" + ) + + # 设置路由拦截 + await self._context.route("**/*", self._route_handler) + + logger.info("性能优化浏览器已启动") + + async def crawl_page(self, url: str) -> dict: + """ + 爬取页面 + + Args: + url: 目标URL + + Returns: + 页面信息 + """ + page = await self._context.new_page() + + try: + logger.info(f"爬取页面: {url}") + + # 访问页面 + await page.goto(url, wait_until="domcontentloaded") + + # 获取页面标题 + title = await page.title() + + # 获取页面内容 + content = await page.content() + + return { + "url": url, + "title": title.strip() if title else "", + "content_length": len(content) + } + + finally: + await page.close() + + async def stop(self): + """关闭浏览器""" + if self._context: + await self._context.close() + if self._browser: + await self._browser.close() + if self._playwright: + await self._playwright.stop() + + +async def benchmark_optimization(): + """性能对比测试""" + import time + + # 测试URL列表 + urls = [ + "https://quotes.toscrape.com/", + "https://quotes.toscrape.com/page/2/", + "https://quotes.toscrape.com/page/3/", + ] + + crawler = OptimizedCrawler(headless=True) + await crawler.start() + + try: + start = time.time() + + for url in urls: + result = await crawler.crawl_page(url) + logger.info(f"页面: {result['title']} | 大小: {result['content_length']} bytes") + + elapsed = time.time() - start + logger.info(f"总耗时: {elapsed:.2f}s | 平均: {elapsed/len(urls):.2f}s/页") + + finally: + await crawler.stop() + + +if __name__ == "__main__": + asyncio.run(benchmark_optimization()) +``` + +### 反检测效果验证流程 + +```mermaid +flowchart LR + Start["启动浏览器"] --> Inject["注入stealth.js"] + Inject --> Visit["访问测试网站"] + Visit --> Check{"检测验证"} + + Check -->|webdriver=undefined| Pass1["✓ 通过"] + Check -->|chrome存在| Pass2["✓ 通过"] + Check -->|plugins正常| Pass3["✓ 通过"] + + Pass1 & Pass2 & Pass3 --> Result["反检测成功"] + Result --> Crawl["正常爬取"] + + style Result fill:#c8e6c9,stroke:#4caf50 + style Crawl fill:#e3f2fd,stroke:#2196f3 +``` + +--- + +## 本章小结 + +本章我们学习了 Playwright 的进阶技术： + +1. **浏览器指纹检测**：了解网站如何检测自动化浏览器 +2. **stealth.js 反检测**：使用脚本注入绕过检测 +3. **CDP 模式**：直接使用 Chrome DevTools Protocol +4. **性能优化**：禁用资源加载、上下文复用、并发管理 +5. **异常处理**：页面崩溃恢复、资源清理 +6. **实战演练**：反检测配置、性能优化爬虫 + +--- + +## 下一章预告 + +下一章我们将学习「登录认证：Cookie 与 Session 管理」。主要内容包括： + +- Cookie 和 Session 的深入理解 +- Cookie 的提取、存储和注入 +- 登录状态检测和自动刷新 +- 多账号 Cookie 轮换 + +这些技术是爬取需要登录的网站的基础。 diff --git "a/docs/\347\210\254\350\231\253\350\277\233\344\273\267/06_\347\231\273\345\275\225\350\256\244\350\257\201_Cookie\344\270\216Session\347\256\241\347\220\206.md" "b/docs/\347\210\254\350\231\253\350\277\233\344\273\267/06_\347\231\273\345\275\225\350\256\244\350\257\201_Cookie\344\270\216Session\347\256\241\347\220\206.md" new file mode 100644 index 0000000..a6e4b69 --- /dev/null +++ "b/docs/\347\210\254\350\231\253\350\277\233\344\273\267/06_\347\231\273\345\275\225\350\256\244\350\257\201_Cookie\344\270\216Session\347\256\241\347\220\206.md" @@ -0,0 +1,1245 @@ +# 06_登录认证：Cookie 与 Session 管理 + +在爬虫开发中，很多数据需要登录后才能获取。本章将深入讲解 Cookie 和 Session 的工作机制，以及如何在爬虫中高效管理登录状态。 + +## 一、认证机制深入理解 + +### 1.1 Cookie 的工作原理 + +Cookie 是服务器发送给浏览器的小型数据片段，浏览器会保存并在后续请求中自动携带。 + +```mermaid +sequenceDiagram + participant Client as 客户端 + participant Server as 服务器 + + Client->>Server: HTTP 请求 (首次访问) + Server-->>Client: Set-Cookie: session=xxx + Note over Client: 保存 Cookie 到本地 + + Client->>Server: HTTP 请求 + Cookie: session=xxx + Server-->>Client: 返回用户数据 + Note over Server: 识别用户身份 +``` + +### 1.2 Cookie 属性详解 + +每个 Cookie 都有多个属性，理解这些属性对于正确使用 Cookie 至关重要： + +| 属性 | 说明 | 爬虫影响 | +|-----|------|---------| +| **Name/Value** | Cookie 的名称和值 | 核心数据，必须正确提取 | +| **Domain** | Cookie 生效的域名 | 决定请求哪些域名时携带 | +| **Path** | Cookie 生效的路径 | 决定请求哪些路径时携带 | +| **Expires/Max-Age** | 过期时间 | 决定 Cookie 何时失效 | +| **HttpOnly** | 禁止 JavaScript 访问 | 只能通过 HTTP 响应获取 | +| **Secure** | 仅 HTTPS 传输 | 注意协议匹配 | +| **SameSite** | 跨站请求限制 | 影响第三方请求 | + +```python +# Cookie 属性示例 +cookie_example = { + "name": "session_id", + "value": "abc123xyz", + "domain": ".example.com", # 包含子域名 + "path": "/", # 所有路径 + "expires": 1735689600, # Unix 时间戳 + "httpOnly": True, # 不能通过 JS 访问 + "secure": True, # 仅 HTTPS + "sameSite": "Lax" # 跨站限制 +} +``` + +### 1.3 Session 机制 + +Session 是服务端的会话状态存储机制，通常通过 Cookie 中的 Session ID 关联： + +```python +# 服务端 Session 工作流程（伪代码） +class SessionFlow: + """ + 1. 用户登录 -> 服务器创建 Session + 2. 服务器返回 Session ID (通过 Set-Cookie) + 3. 浏览器携带 Session ID 访问 + 4. 服务器根据 Session ID 查找用户状态 + """ + + def login(self, username, password): + if self.verify(username, password): + session_id = generate_session_id() + self.sessions[session_id] = { + "user_id": user.id, + "login_time": time.time(), + "expires": time.time() + 3600 * 24 + } + return session_id + return None +``` + +### 1.4 Token 认证机制 + +现代应用越来越多使用 Token 认证（如 JWT），与 Cookie/Session 有所不同： + +| 特性 | Cookie/Session | Token (JWT) | +|-----|----------------|-------------| +| 状态存储 | 服务端 | 客户端（Token 自包含） | +| 跨域支持 | 受限（SameSite） | 天然支持 | +| 传输方式 | Cookie 头 | Authorization 头 | +| 服务端开销 | 需要存储 Session | 无状态 | + +```python +# Token 认证示例 +import httpx + +headers = { + "Authorization": "Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..." +} + +async with httpx.AsyncClient() as client: + resp = await client.get("https://api.example.com/data", headers=headers) +``` + +## 二、Cookie 的提取与存储 + +### 2.1 从浏览器手动提取 + +最简单的方式是从浏览器开发者工具中手动复制 Cookie： + +1. 打开浏览器，登录目标网站 +2. 按 F12 打开开发者工具 +3. 切换到 Application/Storage -> Cookies +4. 复制需要的 Cookie 值 + +```python +# 手动提取的 Cookie 使用示例 +import httpx + +cookies = { + "session_id": "abc123", + "user_token": "xyz789", + "preferences": "theme=dark" +} + +async with httpx.AsyncClient(cookies=cookies) as client: + resp = await client.get("https://example.com/user/profile") +``` + +### 2.2 使用 Playwright 自动提取 + +Playwright 可以自动化登录流程并提取 Cookie： + +```python +from playwright.async_api import async_playwright +import json + +async def extract_cookies_after_login(): + """登录后自动提取 Cookie""" + async with async_playwright() as p: + browser = await p.chromium.launch(headless=False) # 有头模式便于手动登录 + context = await browser.new_context() + page = await context.new_page() + + # 导航到登录页 + await page.goto("https://example.com/login") + + # 等待用户手动登录（或自动填写表单） + # await page.fill("#username", "your_username") + # await page.fill("#password", "your_password") + # await page.click("#login-button") + + # 等待登录成功（检测特定元素或 URL 变化） + await page.wait_for_url("**/dashboard**", timeout=120000) + + # 提取所有 Cookie + cookies = await context.cookies() + + # 保存到文件 + with open("cookies.json", "w") as f: + json.dump(cookies, f, indent=2) + + print(f"提取了 {len(cookies)} 个 Cookie") + + await browser.close() + return cookies +``` + +### 2.3 Cookie 序列化格式 + +Cookie 可以用多种格式存储，常见的有 JSON 和 Netscape 格式： + +```python +import json +from http.cookiejar import MozillaCookieJar +from datetime import datetime + +class CookieSerializer: + """Cookie 序列化工具""" + + @staticmethod + def to_json(cookies: list, filepath: str): + """保存为 JSON 格式""" + with open(filepath, "w", encoding="utf-8") as f: + json.dump(cookies, f, indent=2, ensure_ascii=False) + + @staticmethod + def from_json(filepath: str) -> list: + """从 JSON 加载""" + with open(filepath, "r", encoding="utf-8") as f: + return json.load(f) + + @staticmethod + def to_netscape(cookies: list, filepath: str): + """保存为 Netscape 格式（兼容 curl/wget）""" + lines = ["# Netscape HTTP Cookie File"] + for c in cookies: + # 格式: domain, flag, path, secure, expiry, name, value + domain = c.get("domain", "") + flag = "TRUE" if domain.startswith(".") else "FALSE" + path = c.get("path", "/") + secure = "TRUE" if c.get("secure", False) else "FALSE" + expiry = str(int(c.get("expires", 0))) + name = c.get("name", "") + value = c.get("value", "") + + lines.append(f"{domain}\t{flag}\t{path}\t{secure}\t{expiry}\t{name}\t{value}") + + with open(filepath, "w") as f: + f.write("\n".join(lines)) + + @staticmethod + def to_dict(cookies: list) -> dict: + """转换为简单字典格式（name: value）""" + return {c["name"]: c["value"] for c in cookies} +``` + +### 2.4 Cookie 加密存储 + +对于敏感的 Cookie，建议加密存储： + +```python +import json +import base64 +from cryptography.fernet import Fernet + +class SecureCookieStorage: + """加密 Cookie 存储""" + + def __init__(self, key: bytes = None): + # 如果没有提供密钥，生成新密钥 + self.key = key or Fernet.generate_key() + self.cipher = Fernet(self.key) + + def save_key(self, filepath: str): + """保存密钥（请妥善保管）""" + with open(filepath, "wb") as f: + f.write(self.key) + + @classmethod + def load_key(cls, filepath: str) -> "SecureCookieStorage": + """从文件加载密钥""" + with open(filepath, "rb") as f: + return cls(f.read()) + + def encrypt_cookies(self, cookies: list, filepath: str): + """加密并保存 Cookie""" + data = json.dumps(cookies).encode("utf-8") + encrypted = self.cipher.encrypt(data) + with open(filepath, "wb") as f: + f.write(encrypted) + + def decrypt_cookies(self, filepath: str) -> list: + """解密并加载 Cookie""" + with open(filepath, "rb") as f: + encrypted = f.read() + decrypted = self.cipher.decrypt(encrypted) + return json.loads(decrypted.decode("utf-8")) +``` + +## 三、Cookie 注入与使用 + +### 3.1 httpx 中使用 Cookie + +httpx 提供了灵活的 Cookie 管理方式： + +```python +import httpx + +# 方式1: 直接传入字典 +cookies_dict = {"session": "abc123", "token": "xyz"} +async with httpx.AsyncClient(cookies=cookies_dict) as client: + resp = await client.get("https://example.com/api") + +# 方式2: 使用 Cookies 对象 +from httpx import Cookies + +cookies = Cookies() +cookies.set("session", "abc123", domain="example.com") +cookies.set("token", "xyz", domain="example.com") + +async with httpx.AsyncClient(cookies=cookies) as client: + resp = await client.get("https://example.com/api") + +# 方式3: 从响应中自动获取 +async with httpx.AsyncClient() as client: + # 登录请求会设置 Cookie + login_resp = await client.post( + "https://example.com/login", + data={"username": "user", "password": "pass"} + ) + # 后续请求自动携带 Cookie + profile_resp = await client.get("https://example.com/profile") +``` + +### 3.2 Playwright 中注入 Cookie + +Playwright 可以在访问页面前注入 Cookie： + +```python +from playwright.async_api import async_playwright +import json + +async def inject_cookies_and_browse(): + """注入 Cookie 并访问页面""" + async with async_playwright() as p: + browser = await p.chromium.launch(headless=True) + context = await browser.new_context() + + # 从文件加载 Cookie + with open("cookies.json", "r") as f: + cookies = json.load(f) + + # 注入 Cookie + await context.add_cookies(cookies) + + # 现在可以直接访问需要登录的页面 + page = await context.new_page() + await page.goto("https://example.com/dashboard") + + # 检查是否登录成功 + if await page.locator(".user-avatar").is_visible(): + print("Cookie 注入成功，已登录") + else: + print("Cookie 可能已过期") + + await browser.close() +``` + +### 3.3 跨工具共享 Cookie + +将 Playwright 提取的 Cookie 转换为 httpx 可用的格式： + +```python +def playwright_cookies_to_httpx(playwright_cookies: list) -> dict: + """ + 将 Playwright 格式的 Cookie 转换为 httpx 格式 + + Playwright 格式: [{"name": "x", "value": "y", "domain": "...", ...}, ...] + httpx 格式: {"name": "value", ...} + """ + return {c["name"]: c["value"] for c in playwright_cookies} + +def httpx_cookies_to_playwright(cookies_dict: dict, domain: str) -> list: + """ + 将简单字典格式转换为 Playwright 格式 + """ + return [ + { + "name": name, + "value": value, + "domain": domain, + "path": "/" + } + for name, value in cookies_dict.items() + ] +``` + +### 3.4 多账号 Cookie 轮换 + +在需要多账号爬取的场景，可以实现 Cookie 轮换： + +```python +import asyncio +import random +from typing import Dict, List, Optional +from dataclasses import dataclass, field +from datetime import datetime + +@dataclass +class AccountCookie: + """账号 Cookie 信息""" + account_id: str + cookies: dict + last_used: Optional[datetime] = None + use_count: int = 0 + is_valid: bool = True + +class CookieRotator: + """Cookie 轮换器""" + + def __init__(self, min_interval: float = 5.0): + self._accounts: Dict[str, AccountCookie] = {} + self._min_interval = min_interval # 同一账号最小使用间隔（秒） + self._lock = asyncio.Lock() + + def add_account(self, account_id: str, cookies: dict): + """添加账号""" + self._accounts[account_id] = AccountCookie( + account_id=account_id, + cookies=cookies + ) + + async def get_cookies(self) -> Optional[dict]: + """获取一个可用的 Cookie""" + async with self._lock: + now = datetime.now() + available = [] + + for acc in self._accounts.values(): + if not acc.is_valid: + continue + + # 检查使用间隔 + if acc.last_used: + elapsed = (now - acc.last_used).total_seconds() + if elapsed < self._min_interval: + continue + + available.append(acc) + + if not available: + return None + + # 选择使用次数最少的账号（负载均衡） + selected = min(available, key=lambda x: x.use_count) + selected.last_used = now + selected.use_count += 1 + + return selected.cookies + + def mark_invalid(self, cookies: dict): + """标记 Cookie 失效""" + for acc in self._accounts.values(): + if acc.cookies == cookies: + acc.is_valid = False + break + + @property + def valid_count(self) -> int: + """有效账号数量""" + return sum(1 for acc in self._accounts.values() if acc.is_valid) +``` + +## 四、登录状态管理 + +### 4.1 登录状态检测 + +检测 Cookie 是否仍然有效是关键步骤： + +```python +import httpx +from typing import Callable, Awaitable +from loguru import logger + +class LoginStateChecker: + """登录状态检测器""" + + def __init__( + self, + check_url: str, + success_indicator: Callable[[httpx.Response], bool] + ): + """ + Args: + check_url: 用于检测登录状态的 URL + success_indicator: 判断是否登录成功的函数 + """ + self.check_url = check_url + self.success_indicator = success_indicator + + async def is_logged_in(self, cookies: dict) -> bool: + """检查是否已登录""" + try: + async with httpx.AsyncClient(cookies=cookies) as client: + resp = await client.get(self.check_url, timeout=10) + return self.success_indicator(resp) + except Exception as e: + logger.warning(f"登录状态检测失败: {e}") + return False + + @classmethod + def create_json_checker(cls, check_url: str, success_field: str) -> "LoginStateChecker": + """ + 创建 JSON 响应检测器 + + Args: + check_url: API 地址 + success_field: 成功时 JSON 中存在的字段 + """ + def indicator(resp: httpx.Response) -> bool: + try: + data = resp.json() + return success_field in data + except: + return False + + return cls(check_url, indicator) + + @classmethod + def create_redirect_checker(cls, check_url: str, login_url: str) -> "LoginStateChecker": + """ + 创建重定向检测器 + + Args: + check_url: 需要登录的页面 + login_url: 登录页 URL（被重定向到此说明未登录） + """ + def indicator(resp: httpx.Response) -> bool: + # 如果没有被重定向到登录页，说明已登录 + return login_url not in str(resp.url) + + return cls(check_url, indicator) +``` + +### 4.2 Cookie 过期检测 + +监控 Cookie 的过期时间： + +```python +import time +from datetime import datetime, timedelta +from typing import List, Tuple + +class CookieExpiryMonitor: + """Cookie 过期监控""" + + @staticmethod + def check_expiry(cookies: list) -> Tuple[bool, List[dict]]: + """ + 检查 Cookie 是否过期 + + Returns: + (是否全部有效, 已过期的 Cookie 列表) + """ + now = time.time() + expired = [] + + for cookie in cookies: + expires = cookie.get("expires", 0) + if expires > 0 and expires < now: + expired.append(cookie) + + return len(expired) == 0, expired + + @staticmethod + def get_earliest_expiry(cookies: list) -> datetime: + """获取最早过期的时间""" + expiry_times = [ + c.get("expires", float("inf")) + for c in cookies + if c.get("expires", 0) > 0 + ] + + if not expiry_times: + return datetime.max + + return datetime.fromtimestamp(min(expiry_times)) + + @staticmethod + def will_expire_soon(cookies: list, threshold_hours: int = 1) -> bool: + """检查 Cookie 是否即将过期""" + earliest = CookieExpiryMonitor.get_earliest_expiry(cookies) + return earliest < datetime.now() + timedelta(hours=threshold_hours) +``` + +### 4.3 完整的 Cookie 管理器 + +整合以上功能的完整管理器： + +```python +import json +import asyncio +from pathlib import Path +from typing import Optional, Callable, Awaitable +from datetime import datetime +from loguru import logger + +class CookieManager: + """完整的 Cookie 管理器""" + + def __init__( + self, + storage_path: str, + check_url: str, + login_checker: Callable[[dict], Awaitable[bool]], + auto_refresh_callback: Optional[Callable[[], Awaitable[dict]]] = None + ): + """ + Args: + storage_path: Cookie 存储路径 + check_url: 登录状态检测 URL + login_checker: 登录状态检测函数 + auto_refresh_callback: 自动刷新回调（如重新登录） + """ + self.storage_path = Path(storage_path) + self.check_url = check_url + self.login_checker = login_checker + self.auto_refresh_callback = auto_refresh_callback + + self._cookies: Optional[list] = None + self._last_check: Optional[datetime] = None + self._check_interval = 300 # 5分钟检测一次 + + async def load(self) -> bool: + """加载 Cookie""" + if not self.storage_path.exists(): + logger.warning(f"Cookie 文件不存在: {self.storage_path}") + return False + + try: + with open(self.storage_path, "r") as f: + self._cookies = json.load(f) + logger.info(f"加载了 {len(self._cookies)} 个 Cookie") + return True + except Exception as e: + logger.error(f"加载 Cookie 失败: {e}") + return False + + async def save(self): + """保存 Cookie""" + if self._cookies: + self.storage_path.parent.mkdir(parents=True, exist_ok=True) + with open(self.storage_path, "w") as f: + json.dump(self._cookies, f, indent=2) + logger.info(f"保存了 {len(self._cookies)} 个 Cookie") + + def update(self, cookies: list): + """更新 Cookie""" + self._cookies = cookies + self._last_check = datetime.now() + + async def get_valid_cookies(self) -> Optional[dict]: + """获取有效的 Cookie（自动检测和刷新）""" + # 首次加载 + if self._cookies is None: + await self.load() + + if not self._cookies: + if self.auto_refresh_callback: + await self._refresh_cookies() + return None + + # 检查是否需要验证 + if self._need_check(): + is_valid = await self._validate() + if not is_valid: + if self.auto_refresh_callback: + await self._refresh_cookies() + else: + return None + + # 返回简单字典格式 + return {c["name"]: c["value"] for c in self._cookies} + + def _need_check(self) -> bool: + """是否需要检测""" + if self._last_check is None: + return True + elapsed = (datetime.now() - self._last_check).total_seconds() + return elapsed > self._check_interval + + async def _validate(self) -> bool: + """验证 Cookie 是否有效""" + logger.debug("验证 Cookie 有效性...") + cookies_dict = {c["name"]: c["value"] for c in self._cookies} + is_valid = await self.login_checker(cookies_dict) + self._last_check = datetime.now() + + if is_valid: + logger.info("Cookie 验证通过") + else: + logger.warning("Cookie 已失效") + + return is_valid + + async def _refresh_cookies(self): + """刷新 Cookie""" + if not self.auto_refresh_callback: + return + + logger.info("开始刷新 Cookie...") + try: + new_cookies = await self.auto_refresh_callback() + if new_cookies: + self._cookies = new_cookies + await self.save() + logger.info("Cookie 刷新成功") + except Exception as e: + logger.error(f"Cookie 刷新失败: {e}") +``` + +## 五、B站 Cookie 管理实战 + +### 5.1 B站 Cookie 结构分析 + +B站使用多个关键 Cookie 来管理用户登录状态： + +```mermaid +graph TB + subgraph B站核心Cookie + SESSDATA["SESSDATA
会话凭证"] + DedeUserID["DedeUserID
用户ID"] + bili_jct["bili_jct
CSRF Token"] + end + + subgraph 辅助Cookie + buvid3["buvid3
设备标识"] + buvid4["buvid4
设备标识"] + sid["sid
短会话ID"] + end + + SESSDATA --> API["API请求认证"] + DedeUserID --> API + bili_jct --> POST["POST请求CSRF"] + + style SESSDATA fill:#e8f5e9,stroke:#4caf50 + style DedeUserID fill:#e8f5e9,stroke:#4caf50 + style bili_jct fill:#fff3e0,stroke:#ff9800 +``` + +| Cookie 名称 | 作用 | 有效期 | 说明 | +|------------|------|-------|------| +| **SESSDATA** | 会话凭证 | ~1个月 | 最重要的登录凭证 | +| **DedeUserID** | 用户ID | ~1个月 | 用于部分API请求 | +| **bili_jct** | CSRF Token | ~1个月 | POST请求必需 | +| buvid3 | 设备标识 | ~1年 | 追踪设备 | +| buvid4 | 设备标识 | ~1年 | 追踪设备 | +| sid | 短会话 | 会话 | 短期会话标识 | + +### 5.2 B站 Cookie 提取器 + +```python +# -*- coding: utf-8 -*- +""" +B站 Cookie 提取和管理 +""" + +import json +import asyncio +from pathlib import Path +from typing import Optional, Dict, List +from dataclasses import dataclass +from datetime import datetime +from loguru import logger + + +@dataclass +class BilibiliCookies: + """B站 Cookie 数据类""" + sessdata: str + dede_user_id: str + bili_jct: str + buvid3: str = "" + buvid4: str = "" + sid: str = "" + raw_cookies: List[dict] = None + + @classmethod + def from_playwright_cookies(cls, cookies: List[dict]) -> "BilibiliCookies": + """从 Playwright 格式的 Cookie 创建""" + cookie_dict = {c["name"]: c["value"] for c in cookies} + + return cls( + sessdata=cookie_dict.get("SESSDATA", ""), + dede_user_id=cookie_dict.get("DedeUserID", ""), + bili_jct=cookie_dict.get("bili_jct", ""), + buvid3=cookie_dict.get("buvid3", ""), + buvid4=cookie_dict.get("buvid4", ""), + sid=cookie_dict.get("sid", ""), + raw_cookies=cookies + ) + + @classmethod + def from_browser_string(cls, cookie_string: str) -> "BilibiliCookies": + """ + 从浏览器复制的 Cookie 字符串创建 + + 格式: "SESSDATA=xxx; DedeUserID=xxx; bili_jct=xxx" + """ + cookie_dict = {} + for item in cookie_string.split(";"): + item = item.strip() + if "=" in item: + key, value = item.split("=", 1) + cookie_dict[key.strip()] = value.strip() + + return cls( + sessdata=cookie_dict.get("SESSDATA", ""), + dede_user_id=cookie_dict.get("DedeUserID", ""), + bili_jct=cookie_dict.get("bili_jct", ""), + buvid3=cookie_dict.get("buvid3", ""), + buvid4=cookie_dict.get("buvid4", ""), + sid=cookie_dict.get("sid", "") + ) + + def to_httpx_cookies(self) -> Dict[str, str]: + """转换为 httpx 可用的格式""" + cookies = { + "SESSDATA": self.sessdata, + "DedeUserID": self.dede_user_id, + "bili_jct": self.bili_jct, + } + if self.buvid3: + cookies["buvid3"] = self.buvid3 + if self.buvid4: + cookies["buvid4"] = self.buvid4 + if self.sid: + cookies["sid"] = self.sid + return cookies + + def to_playwright_cookies(self, domain: str = ".bilibili.com") -> List[dict]: + """转换为 Playwright 可用的格式""" + if self.raw_cookies: + return self.raw_cookies + + cookies = [] + for name, value in self.to_httpx_cookies().items(): + cookies.append({ + "name": name, + "value": value, + "domain": domain, + "path": "/" + }) + return cookies + + def is_valid(self) -> bool: + """检查核心 Cookie 是否存在""" + return bool(self.sessdata and self.dede_user_id and self.bili_jct) + + def to_header_string(self) -> str: + """转换为请求头格式""" + return "; ".join(f"{k}={v}" for k, v in self.to_httpx_cookies().items()) + + +class BilibiliCookieManager: + """ + B站 Cookie 管理器 + + 功能： + - Cookie 加载/保存 + - 登录状态检测 + - Cookie 有效性验证 + """ + + # 登录状态检测 API + CHECK_URL = "https://api.bilibili.com/x/web-interface/nav" + + def __init__(self, storage_path: str = "bilibili_cookies.json"): + self.storage_path = Path(storage_path) + self._cookies: Optional[BilibiliCookies] = None + self._last_check: Optional[datetime] = None + + async def load(self) -> bool: + """从文件加载 Cookie""" + if not self.storage_path.exists(): + logger.warning(f"Cookie 文件不存在: {self.storage_path}") + return False + + try: + with open(self.storage_path, "r", encoding="utf-8") as f: + data = json.load(f) + + if isinstance(data, list): + # Playwright 格式 + self._cookies = BilibiliCookies.from_playwright_cookies(data) + elif isinstance(data, dict): + # 自定义格式 + self._cookies = BilibiliCookies( + sessdata=data.get("SESSDATA", ""), + dede_user_id=data.get("DedeUserID", ""), + bili_jct=data.get("bili_jct", ""), + buvid3=data.get("buvid3", ""), + buvid4=data.get("buvid4", ""), + sid=data.get("sid", "") + ) + + logger.info(f"Cookie 加载成功，用户ID: {self._cookies.dede_user_id}") + return True + + except Exception as e: + logger.error(f"加载 Cookie 失败: {e}") + return False + + async def save(self, cookies: BilibiliCookies): + """保存 Cookie 到文件""" + self._cookies = cookies + self.storage_path.parent.mkdir(parents=True, exist_ok=True) + + data = { + "SESSDATA": cookies.sessdata, + "DedeUserID": cookies.dede_user_id, + "bili_jct": cookies.bili_jct, + "buvid3": cookies.buvid3, + "buvid4": cookies.buvid4, + "sid": cookies.sid, + "save_time": datetime.now().isoformat() + } + + with open(self.storage_path, "w", encoding="utf-8") as f: + json.dump(data, f, indent=2, ensure_ascii=False) + + logger.info(f"Cookie 已保存到: {self.storage_path}") + + async def verify(self) -> bool: + """验证 Cookie 是否有效""" + if not self._cookies or not self._cookies.is_valid(): + return False + + import httpx + + try: + async with httpx.AsyncClient( + cookies=self._cookies.to_httpx_cookies(), + timeout=10 + ) as client: + resp = await client.get(self.CHECK_URL) + data = resp.json() + + if data.get("code") == 0: + user_info = data.get("data", {}) + if user_info.get("isLogin"): + logger.info(f"Cookie 有效，用户: {user_info.get('uname')}") + self._last_check = datetime.now() + return True + + logger.warning("Cookie 已失效") + return False + + except Exception as e: + logger.error(f"验证 Cookie 失败: {e}") + return False + + async def get_valid_cookies(self) -> Optional[BilibiliCookies]: + """获取有效的 Cookie""" + if self._cookies is None: + await self.load() + + if self._cookies and await self.verify(): + return self._cookies + + return None + + @property + def cookies(self) -> Optional[BilibiliCookies]: + """获取当前 Cookie（不验证）""" + return self._cookies +``` + +### 5.3 B站 Cookie 使用示例 + +```python +# -*- coding: utf-8 -*- +""" +B站 Cookie 使用示例 +""" + +import asyncio +import httpx +from loguru import logger + + +async def demo_bilibili_cookie(): + """演示如何使用 B站 Cookie""" + + # 方式1: 从浏览器复制的字符串 + cookie_string = "SESSDATA=xxx; DedeUserID=123456; bili_jct=xxx" + cookies = BilibiliCookies.from_browser_string(cookie_string) + + # 方式2: 使用 Cookie 管理器 + manager = BilibiliCookieManager("data/bilibili_cookies.json") + await manager.load() + + if not await manager.verify(): + logger.error("Cookie 无效，请重新登录") + return + + cookies = manager.cookies + + # 使用 Cookie 请求 API + async with httpx.AsyncClient( + cookies=cookies.to_httpx_cookies(), + headers={ + "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) " + "AppleWebKit/537.36 (KHTML, like Gecko) " + "Chrome/131.0.0.0 Safari/537.36", + "Referer": "https://www.bilibili.com/" + } + ) as client: + # 获取用户信息 + resp = await client.get("https://api.bilibili.com/x/web-interface/nav") + data = resp.json() + + if data.get("code") == 0: + user = data.get("data", {}) + print(f"用户名: {user.get('uname')}") + print(f"等级: {user.get('level_info', {}).get('current_level')}") + print(f"硬币: {user.get('money')}") + + # 获取收藏夹（需要登录） + resp = await client.get( + "https://api.bilibili.com/x/v3/fav/folder/created/list-all", + params={"up_mid": cookies.dede_user_id} + ) + fav_data = resp.json() + + if fav_data.get("code") == 0: + folders = fav_data.get("data", {}).get("list", []) + print(f"\n收藏夹列表 ({len(folders)}个):") + for folder in folders[:5]: + print(f" - {folder.get('title')} ({folder.get('media_count')}个)") + + +if __name__ == "__main__": + asyncio.run(demo_bilibili_cookie()) +``` + +### 5.4 B站 Cookie 与 Playwright 集成 + +```python +# -*- coding: utf-8 -*- +""" +B站 Cookie 与 Playwright 集成 +""" + +import asyncio +from playwright.async_api import async_playwright +from loguru import logger + + +async def bilibili_with_playwright(): + """使用 Playwright 注入 B站 Cookie""" + + # 加载 Cookie + manager = BilibiliCookieManager("data/bilibili_cookies.json") + await manager.load() + + if not manager.cookies or not manager.cookies.is_valid(): + logger.error("Cookie 无效") + return + + async with async_playwright() as p: + browser = await p.chromium.launch(headless=True) + context = await browser.new_context( + viewport={"width": 1920, "height": 1080}, + locale="zh-CN" + ) + + # 注入 Cookie + await context.add_cookies( + manager.cookies.to_playwright_cookies() + ) + + page = await context.new_page() + + # 访问个人主页 + await page.goto("https://space.bilibili.com/", wait_until="networkidle") + + # 检查是否登录成功 + avatar = page.locator(".bili-avatar") + if await avatar.count() > 0: + logger.info("Cookie 注入成功，已登录") + + # 获取用户名 + username = await page.locator(".h-name").text_content() + print(f"当前用户: {username}") + else: + logger.warning("Cookie 可能已过期") + + await browser.close() + + +if __name__ == "__main__": + asyncio.run(bilibili_with_playwright()) +``` + +--- + +## 六、完整的登录会话管理 + +下面是一个完整的实战示例，展示如何在爬虫中管理登录状态： + +```python +# session_crawler.py +import asyncio +import httpx +from typing import Optional +from loguru import logger + +from cookie_manager import CookieManager +from login_state_checker import LoginStateChecker + +class SessionCrawler: + """带登录状态管理的爬虫""" + + def __init__(self, base_url: str, cookie_path: str): + self.base_url = base_url + self.cookie_path = cookie_path + + # 初始化登录检测器 + self.checker = LoginStateChecker.create_json_checker( + check_url=f"{base_url}/api/user/info", + success_field="user_id" + ) + + # 初始化 Cookie 管理器 + self.cookie_manager = CookieManager( + storage_path=cookie_path, + check_url=f"{base_url}/api/user/info", + login_checker=self.checker.is_logged_in + ) + + self._client: Optional[httpx.AsyncClient] = None + + async def __aenter__(self): + await self.start() + return self + + async def __aexit__(self, *args): + await self.close() + + async def start(self): + """启动爬虫""" + cookies = await self.cookie_manager.get_valid_cookies() + if not cookies: + raise RuntimeError("无法获取有效的 Cookie，请先登录") + + self._client = httpx.AsyncClient( + base_url=self.base_url, + cookies=cookies, + timeout=30 + ) + logger.info("爬虫启动成功") + + async def close(self): + """关闭爬虫""" + if self._client: + await self._client.aclose() + await self.cookie_manager.save() + logger.info("爬虫已关闭") + + async def fetch(self, endpoint: str) -> dict: + """获取数据""" + try: + resp = await self._client.get(endpoint) + resp.raise_for_status() + return resp.json() + except httpx.HTTPStatusError as e: + if e.response.status_code == 401: + logger.warning("登录状态失效，需要重新登录") + # 可以在这里触发重新登录逻辑 + raise + + +async def main(): + """使用示例""" + async with SessionCrawler( + base_url="https://example.com", + cookie_path="data/cookies.json" + ) as crawler: + # 获取用户信息 + user_info = await crawler.fetch("/api/user/info") + print(f"用户: {user_info.get('username')}") + + # 获取数据列表 + data_list = await crawler.fetch("/api/data/list") + print(f"获取了 {len(data_list)} 条数据") + + +if __name__ == "__main__": + asyncio.run(main()) +``` + +## 六、最佳实践 + +### 6.1 Cookie 安全存储建议 + +1. **不要硬编码 Cookie**：将 Cookie 存储在配置文件或环境变量中 +2. **敏感 Cookie 加密**：使用加密存储敏感的登录凭证 +3. **设置合适的文件权限**：Cookie 文件应限制访问权限 +4. **定期轮换**：定期重新登录获取新 Cookie + +### 6.2 登录状态管理建议 + +1. **检测频率控制**：不要频繁检测，设置合理的检测间隔 +2. **优雅降级**：Cookie 失效时应有备用方案 +3. **多账号轮换**：大规模爬取时使用多账号 +4. **遵守频率限制**：避免触发反爬机制 + +### 6.3 常见问题排查 + +| 问题 | 可能原因 | 解决方案 | +|-----|---------|---------| +| Cookie 无效 | 已过期/被踢出 | 检查过期时间，重新登录 | +| 请求被拒绝 | 缺少必要 Cookie | 检查 Cookie 完整性 | +| 跨域问题 | Domain 不匹配 | 检查 Cookie 的 Domain 属性 | +| HTTPS 问题 | Secure 属性 | 确保使用 HTTPS 协议 | + +## 本章小结 + +本章深入讲解了 Cookie 和 Session 的工作机制，以及在爬虫中如何有效管理登录状态： + +1. **理解认证机制**：Cookie 属性、Session 原理、Token 认证 +2. **Cookie 提取存储**：手动提取、自动化提取、序列化格式、加密存储 +3. **Cookie 注入使用**：httpx 和 Playwright 中的使用方式、跨工具共享 +4. **登录状态管理**：状态检测、过期监控、自动刷新机制 +5. **B站实战**：Cookie 结构分析、专用管理器、验证与使用 + +掌握这些技术后，你就能够处理大多数需要登录的爬虫场景。 + +--- + +## 与第11章实战项目的关联 + +本章 Cookie 管理技术在第11章 B站综合实战项目中有核心应用： + +| 本章内容 | 第11章对应实现 | 文件位置 | +|---------|--------------|---------| +| BilibiliCookies 数据类 | Cookie 模型定义 | `models/cookies.py` | +| BilibiliCookieManager | 登录管理器 | `login/auth.py` | +| Cookie 验证逻辑 | 登录状态检测 | `login/auth.py` | +| Playwright 集成 | 扫码登录实现 | `login/auth.py` | + +```mermaid +graph LR + subgraph 本章知识点 + A1["Cookie结构"] + A2["Cookie管理器"] + A3["登录验证"] + end + + subgraph 第11章实战应用 + B1["扫码登录"] + B2["登录态保持"] + B3["API请求认证"] + end + + A1 --> B1 + A2 --> B2 + A3 --> B3 + + style A1 fill:#e3f2fd,stroke:#2196f3 + style A2 fill:#e3f2fd,stroke:#2196f3 + style A3 fill:#e3f2fd,stroke:#2196f3 + style B1 fill:#c8e6c9,stroke:#4caf50 + style B2 fill:#c8e6c9,stroke:#4caf50 + style B3 fill:#c8e6c9,stroke:#4caf50 +``` + +**学习建议**： + +1. 理解 B站三大核心 Cookie（SESSDATA、DedeUserID、bili_jct）的作用 +2. 掌握 Cookie 的提取、存储、验证完整流程 +3. 在第11章中，`login/auth.py` 是 Cookie 管理的核心实现 + +## 下一章预告 + +下一章我们将学习更复杂的登录方式——**扫码登录和短信验证码登录**的实现。我们会参考 MediaCrawler 的登录模块设计，学习如何处理二维码扫码流程、短信验证码接收等高级场景。 diff --git "a/docs/\347\210\254\350\231\253\350\277\233\344\273\267/07_\347\231\273\345\275\225\350\256\244\350\257\201_\346\211\253\347\240\201\344\270\216\347\237\255\344\277\241\347\231\273\345\275\225\345\256\236\347\216\260.md" "b/docs/\347\210\254\350\231\253\350\277\233\344\273\267/07_\347\231\273\345\275\225\350\256\244\350\257\201_\346\211\253\347\240\201\344\270\216\347\237\255\344\277\241\347\231\273\345\275\225\345\256\236\347\216\260.md" new file mode 100644 index 0000000..ba042e8 --- /dev/null +++ "b/docs/\347\210\254\350\231\253\350\277\233\344\273\267/07_\347\231\273\345\275\225\350\256\244\350\257\201_\346\211\253\347\240\201\344\270\216\347\237\255\344\277\241\347\231\273\345\275\225\345\256\236\347\216\260.md" @@ -0,0 +1,1313 @@ +# 07_登录认证：扫码与短信登录实现 + +上一章我们学习了 Cookie 和 Session 的基础管理，本章将深入探讨更复杂的登录场景——扫码登录和短信验证码登录。这些登录方式在社交媒体、电商等平台中广泛使用，也是 MediaCrawler 等项目的核心技术之一。 + +## 一、扫码登录原理 + +### 1.1 扫码登录流程 + +扫码登录是一种安全便捷的认证方式，其工作流程如下： + +```mermaid +sequenceDiagram + participant PC as PC浏览器 + participant Server as 服务器 + participant APP as 手机APP + + PC->>Server: 1. 请求二维码 + Server-->>PC: 2. 返回二维码+UUID + + loop 轮询登录状态 + PC->>Server: 3. 查询状态 + Server-->>PC: 状态: 待扫描 + end + + APP->>Server: 4. 用户扫码 + Server-->>PC: 5. 状态: 已扫描 + + APP->>Server: 6. 用户确认 + Server-->>PC: 7. 返回登录凭证(Cookie) + + Note over PC: 8. 登录成功! +``` + +### 1.2 关键技术点 + +1. **二维码生成**：服务器生成唯一 UUID，编码到二维码中 +2. **状态轮询**：PC 端定时查询登录状态（长轮询或 WebSocket） +3. **状态流转**：待扫描 → 已扫描 → 已确认 → 已过期 +4. **凭证下发**：登录成功后通过 Cookie 或 Token 返回凭证 + +### 1.3 常见扫码登录状态 + +```python +from enum import Enum + +class QRCodeStatus(Enum): + """二维码状态枚举""" + WAITING = "waiting" # 等待扫描 + SCANNED = "scanned" # 已扫描，等待确认 + CONFIRMED = "confirmed" # 已确认登录 + EXPIRED = "expired" # 二维码已过期 + CANCELED = "canceled" # 用户取消 +``` + +## 二、Playwright 实现扫码登录 + +### 2.1 基础框架 + +```python +import asyncio +from playwright.async_api import async_playwright, Page, BrowserContext +from typing import Optional, Callable, Awaitable +from loguru import logger +from enum import Enum + +class QRCodeLoginBase: + """扫码登录基础类""" + + def __init__( + self, + timeout: int = 120, + poll_interval: float = 2.0, + on_status_change: Optional[Callable[[str], Awaitable[None]]] = None + ): + """ + Args: + timeout: 登录超时时间（秒） + poll_interval: 状态轮询间隔（秒） + on_status_change: 状态变化回调 + """ + self.timeout = timeout + self.poll_interval = poll_interval + self.on_status_change = on_status_change + + self._page: Optional[Page] = None + self._context: Optional[BrowserContext] = None + self._current_status: Optional[str] = None + + async def _notify_status_change(self, status: str): + """通知状态变化""" + if status != self._current_status: + self._current_status = status + logger.info(f"登录状态变化: {status}") + if self.on_status_change: + await self.on_status_change(status) + + async def get_qrcode_image(self) -> bytes: + """获取二维码图片，子类实现""" + raise NotImplementedError + + async def check_login_status(self) -> str: + """检查登录状态，子类实现""" + raise NotImplementedError + + async def extract_cookies(self) -> list: + """提取登录后的 Cookie""" + if self._context: + return await self._context.cookies() + return [] +``` + +### 2.2 完整扫码登录实现 + +以下是一个通用的扫码登录实现示例： + +```python +import asyncio +import base64 +from pathlib import Path +from playwright.async_api import async_playwright, Page, BrowserContext +from typing import Optional, Callable, Awaitable +from loguru import logger + +class QRCodeLogin: + """通用扫码登录实现""" + + def __init__( + self, + login_url: str, + qrcode_selector: str, + success_url_pattern: str, + timeout: int = 120, + poll_interval: float = 2.0 + ): + """ + Args: + login_url: 登录页面 URL + qrcode_selector: 二维码元素选择器 + success_url_pattern: 登录成功后的 URL 特征 + timeout: 超时时间 + poll_interval: 轮询间隔 + """ + self.login_url = login_url + self.qrcode_selector = qrcode_selector + self.success_url_pattern = success_url_pattern + self.timeout = timeout + self.poll_interval = poll_interval + + self._browser = None + self._context = None + self._page = None + + async def start(self, playwright, headless: bool = False): + """启动浏览器""" + self._browser = await playwright.chromium.launch(headless=headless) + self._context = await self._browser.new_context() + self._page = await self._context.new_page() + logger.info("浏览器已启动") + + async def close(self): + """关闭浏览器""" + if self._browser: + await self._browser.close() + logger.info("浏览器已关闭") + + async def navigate_to_login(self): + """导航到登录页面""" + await self._page.goto(self.login_url, wait_until="networkidle") + logger.info(f"已打开登录页面: {self.login_url}") + + async def save_qrcode(self, filepath: str = "qrcode.png"): + """保存二维码图片""" + # 等待二维码出现 + await self._page.wait_for_selector(self.qrcode_selector, timeout=10000) + + # 截取二维码 + qrcode_element = self._page.locator(self.qrcode_selector) + await qrcode_element.screenshot(path=filepath) + logger.info(f"二维码已保存: {filepath}") + return filepath + + async def wait_for_login( + self, + on_qrcode_ready: Optional[Callable[[str], Awaitable[None]]] = None + ) -> bool: + """ + 等待用户扫码登录 + + Args: + on_qrcode_ready: 二维码准备好后的回调 + + Returns: + 是否登录成功 + """ + # 保存二维码 + qrcode_path = await self.save_qrcode() + + # 通知二维码已准备好 + if on_qrcode_ready: + await on_qrcode_ready(qrcode_path) + + # 等待登录成功（URL 变化或特定元素出现） + try: + await self._page.wait_for_url( + f"**{self.success_url_pattern}**", + timeout=self.timeout * 1000 + ) + logger.info("登录成功！") + return True + except Exception as e: + logger.warning(f"登录超时或失败: {e}") + return False + + async def get_cookies(self) -> list: + """获取登录后的 Cookie""" + return await self._context.cookies() + + async def login( + self, + on_qrcode_ready: Optional[Callable[[str], Awaitable[None]]] = None + ) -> Optional[list]: + """ + 执行完整的扫码登录流程 + + Returns: + 成功返回 Cookie 列表，失败返回 None + """ + await self.navigate_to_login() + success = await self.wait_for_login(on_qrcode_ready) + + if success: + cookies = await self.get_cookies() + logger.info(f"获取到 {len(cookies)} 个 Cookie") + return cookies + return None +``` + +### 2.3 终端显示二维码 + +为了在无头模式下也能扫码，可以在终端显示二维码： + +```python +try: + import qrcode + HAS_QRCODE = True +except ImportError: + HAS_QRCODE = False + +def display_qrcode_in_terminal(data: str): + """在终端显示二维码""" + if not HAS_QRCODE: + print("提示: 安装 qrcode 库可在终端显示二维码: pip install qrcode") + return + + qr = qrcode.QRCode( + version=1, + error_correction=qrcode.constants.ERROR_CORRECT_L, + box_size=1, + border=1 + ) + qr.add_data(data) + qr.make(fit=True) + + # 使用字符绘制 + qr.print_ascii(invert=True) + +def display_qrcode_image_in_terminal(image_path: str): + """ + 将图片二维码转换为终端可显示的文本 + 需要安装: pip install pillow + """ + try: + from PIL import Image + + img = Image.open(image_path) + img = img.convert('L') # 转为灰度 + + # 缩小图片 + width = 60 + ratio = width / img.width + height = int(img.height * ratio * 0.5) # 0.5 补偿终端字符高宽比 + img = img.resize((width, height)) + + # 转换为 ASCII + chars = " .:-=+*#%@" + pixels = img.getdata() + ascii_img = "" + for i, pixel in enumerate(pixels): + if i > 0 and i % width == 0: + ascii_img += "\n" + ascii_img += chars[pixel * len(chars) // 256] + + print(ascii_img) + except ImportError: + print(f"二维码已保存到: {image_path}") +``` + +## 三、短信验证码登录 + +### 3.1 短信登录流程 + +```mermaid +sequenceDiagram + participant User as 用户 + participant Frontend as 前端 + participant Server as 服务器 + + User->>Frontend: 1. 输入手机号 + Frontend->>Server: 2. 请求发送验证码 + Server-->>Frontend: 3. 返回发送结果 + + User->>Frontend: 4. 输入验证码 + Frontend->>Server: 5. 提交登录请求 + Server-->>Frontend: 6. 返回登录结果+凭证 + + Note over User,Server: 登录成功，获取Cookie +``` + +### 3.2 Playwright 实现短信登录 + +```python +import asyncio +from playwright.async_api import async_playwright, Page +from typing import Optional, Callable, Awaitable +from loguru import logger + +class SMSLogin: + """短信验证码登录""" + + def __init__( + self, + login_url: str, + phone_input_selector: str, + send_code_btn_selector: str, + code_input_selector: str, + submit_btn_selector: str, + success_url_pattern: str + ): + """ + Args: + login_url: 登录页 URL + phone_input_selector: 手机号输入框选择器 + send_code_btn_selector: 发送验证码按钮选择器 + code_input_selector: 验证码输入框选择器 + submit_btn_selector: 登录按钮选择器 + success_url_pattern: 登录成功 URL 特征 + """ + self.login_url = login_url + self.phone_input_selector = phone_input_selector + self.send_code_btn_selector = send_code_btn_selector + self.code_input_selector = code_input_selector + self.submit_btn_selector = submit_btn_selector + self.success_url_pattern = success_url_pattern + + self._browser = None + self._context = None + self._page = None + + async def start(self, playwright, headless: bool = False): + """启动浏览器""" + self._browser = await playwright.chromium.launch(headless=headless) + self._context = await self._browser.new_context() + self._page = await self._context.new_page() + + async def close(self): + """关闭浏览器""" + if self._browser: + await self._browser.close() + + async def input_phone(self, phone: str): + """输入手机号""" + await self._page.goto(self.login_url, wait_until="networkidle") + await self._page.fill(self.phone_input_selector, phone) + logger.info(f"已输入手机号: {phone[:3]}****{phone[-4:]}") + + async def send_verification_code(self) -> bool: + """发送验证码""" + try: + await self._page.click(self.send_code_btn_selector) + logger.info("验证码发送请求已提交") + # 等待一小段时间确保请求发出 + await asyncio.sleep(1) + return True + except Exception as e: + logger.error(f"发送验证码失败: {e}") + return False + + async def input_code_and_login( + self, + code: str, + timeout: int = 30 + ) -> bool: + """输入验证码并登录""" + try: + await self._page.fill(self.code_input_selector, code) + await self._page.click(self.submit_btn_selector) + + # 等待登录成功 + await self._page.wait_for_url( + f"**{self.success_url_pattern}**", + timeout=timeout * 1000 + ) + logger.info("登录成功！") + return True + except Exception as e: + logger.error(f"登录失败: {e}") + return False + + async def get_cookies(self) -> list: + """获取 Cookie""" + return await self._context.cookies() + + async def login_with_manual_code( + self, + phone: str, + get_code_callback: Callable[[], Awaitable[str]] + ) -> Optional[list]: + """ + 使用手动输入验证码的方式登录 + + Args: + phone: 手机号 + get_code_callback: 获取验证码的回调（如等待用户输入） + + Returns: + 成功返回 Cookie，失败返回 None + """ + await self.input_phone(phone) + await self.send_verification_code() + + # 获取验证码（手动输入或从接码平台获取） + code = await get_code_callback() + + if await self.input_code_and_login(code): + return await self.get_cookies() + return None +``` + +### 3.3 验证码获取方式 + +```python +import asyncio + +async def get_code_from_user() -> str: + """从控制台获取用户输入的验证码""" + print("\n请输入收到的验证码: ", end="", flush=True) + # 在异步环境中等待用户输入 + loop = asyncio.get_event_loop() + code = await loop.run_in_executor(None, input) + return code.strip() + + +class SMSCodeReceiver: + """ + 短信接码平台接口（示意） + + 注意：实际使用需要接入具体的接码平台 API + """ + + def __init__(self, api_key: str, api_url: str): + self.api_key = api_key + self.api_url = api_url + + async def get_phone_number(self) -> str: + """获取手机号""" + # 调用接码平台 API 获取手机号 + raise NotImplementedError("需要实现具体的接码平台接口") + + async def wait_for_code(self, phone: str, timeout: int = 60) -> Optional[str]: + """等待接收验证码""" + # 轮询接码平台获取验证码 + raise NotImplementedError("需要实现具体的接码平台接口") + + async def release_phone(self, phone: str): + """释放手机号""" + raise NotImplementedError("需要实现具体的接码平台接口") +``` + +## 四、登录模块统一封装 + +### 4.1 使用工厂模式 + +```python +from abc import ABC, abstractmethod +from typing import Optional, Callable, Awaitable +from enum import Enum + +class LoginMethod(Enum): + """登录方式枚举""" + QRCODE = "qrcode" # 扫码登录 + SMS = "sms" # 短信验证码 + PASSWORD = "password" # 账号密码 + COOKIE = "cookie" # Cookie 注入 + +class LoginResult: + """登录结果""" + + def __init__( + self, + success: bool, + cookies: list = None, + error: str = None + ): + self.success = success + self.cookies = cookies or [] + self.error = error + +class BaseLogin(ABC): + """登录基类""" + + @abstractmethod + async def login(self) -> LoginResult: + """执行登录""" + pass + + @abstractmethod + async def close(self): + """清理资源""" + pass + +class LoginFactory: + """登录工厂""" + + _registry = {} + + @classmethod + def register(cls, method: LoginMethod): + """注册登录实现""" + def decorator(login_class): + cls._registry[method] = login_class + return login_class + return decorator + + @classmethod + def create( + cls, + method: LoginMethod, + **kwargs + ) -> BaseLogin: + """创建登录实例""" + if method not in cls._registry: + raise ValueError(f"不支持的登录方式: {method}") + return cls._registry[method](**kwargs) +``` + +### 4.2 注册具体实现 + +```python +@LoginFactory.register(LoginMethod.COOKIE) +class CookieLogin(BaseLogin): + """Cookie 注入登录""" + + def __init__(self, cookies: list, check_url: str = None): + self.cookies = cookies + self.check_url = check_url + + async def login(self) -> LoginResult: + """Cookie 注入不需要实际登录，直接返回""" + return LoginResult(success=True, cookies=self.cookies) + + async def close(self): + pass + + +@LoginFactory.register(LoginMethod.QRCODE) +class QRCodeLoginImpl(BaseLogin): + """扫码登录实现""" + + def __init__( + self, + login_url: str, + qrcode_selector: str, + success_url_pattern: str, + on_qrcode_ready: Callable[[str], Awaitable[None]] = None, + **kwargs + ): + self.login_url = login_url + self.qrcode_selector = qrcode_selector + self.success_url_pattern = success_url_pattern + self.on_qrcode_ready = on_qrcode_ready + self._qrcode_login = None + self._playwright = None + + async def login(self) -> LoginResult: + from playwright.async_api import async_playwright + + self._playwright = await async_playwright().start() + self._qrcode_login = QRCodeLogin( + login_url=self.login_url, + qrcode_selector=self.qrcode_selector, + success_url_pattern=self.success_url_pattern + ) + + await self._qrcode_login.start(self._playwright, headless=False) + + try: + cookies = await self._qrcode_login.login(self.on_qrcode_ready) + if cookies: + return LoginResult(success=True, cookies=cookies) + return LoginResult(success=False, error="登录超时") + except Exception as e: + return LoginResult(success=False, error=str(e)) + + async def close(self): + if self._qrcode_login: + await self._qrcode_login.close() + if self._playwright: + await self._playwright.stop() +``` + +### 4.3 统一登录管理器 + +```python +import json +from pathlib import Path +from loguru import logger + +class LoginManager: + """统一登录管理器""" + + def __init__( + self, + platform: str, + cookie_path: str, + preferred_method: LoginMethod = LoginMethod.COOKIE + ): + """ + Args: + platform: 平台名称 + cookie_path: Cookie 存储路径 + preferred_method: 首选登录方式 + """ + self.platform = platform + self.cookie_path = Path(cookie_path) + self.preferred_method = preferred_method + self._cookies: list = [] + + async def ensure_login(self, **login_kwargs) -> bool: + """ + 确保已登录 + + 优先使用已保存的 Cookie，如果无效则使用指定方式登录 + """ + # 1. 尝试加载已保存的 Cookie + if await self._try_load_cookies(): + logger.info(f"[{self.platform}] 使用已保存的 Cookie") + return True + + # 2. 执行登录 + logger.info(f"[{self.platform}] 开始 {self.preferred_method.value} 登录") + login = LoginFactory.create(self.preferred_method, **login_kwargs) + + try: + result = await login.login() + if result.success: + self._cookies = result.cookies + await self._save_cookies() + logger.info(f"[{self.platform}] 登录成功") + return True + else: + logger.error(f"[{self.platform}] 登录失败: {result.error}") + return False + finally: + await login.close() + + async def _try_load_cookies(self) -> bool: + """尝试加载 Cookie""" + if not self.cookie_path.exists(): + return False + + try: + with open(self.cookie_path, "r") as f: + self._cookies = json.load(f) + + # TODO: 验证 Cookie 有效性 + return len(self._cookies) > 0 + except Exception as e: + logger.warning(f"加载 Cookie 失败: {e}") + return False + + async def _save_cookies(self): + """保存 Cookie""" + self.cookie_path.parent.mkdir(parents=True, exist_ok=True) + with open(self.cookie_path, "w") as f: + json.dump(self._cookies, f, indent=2) + logger.info(f"Cookie 已保存: {self.cookie_path}") + + def get_cookies(self) -> list: + """获取 Cookie""" + return self._cookies + + def get_cookies_dict(self) -> dict: + """获取字典格式的 Cookie""" + return {c["name"]: c["value"] for c in self._cookies} +``` + +## 五、实战示例 + +### 5.1 完整的扫码登录示例 + +```python +import asyncio +from playwright.async_api import async_playwright +import json + +async def qrcode_login_demo(): + """扫码登录完整示例""" + + # 二维码准备好后的回调 + async def on_qrcode_ready(path: str): + print(f"\n{'='*40}") + print(f"请使用手机扫描二维码: {path}") + print(f"{'='*40}\n") + # 可以在这里显示二维码到终端 + display_qrcode_image_in_terminal(path) + + async with async_playwright() as p: + # 创建扫码登录实例（以示例网站为例） + qr_login = QRCodeLogin( + login_url="https://example.com/login", + qrcode_selector="img.qrcode", # 二维码选择器 + success_url_pattern="/dashboard", + timeout=120 + ) + + await qr_login.start(p, headless=False) + + try: + cookies = await qr_login.login(on_qrcode_ready) + + if cookies: + # 保存 Cookie + with open("login_cookies.json", "w") as f: + json.dump(cookies, f, indent=2) + print(f"登录成功！获取到 {len(cookies)} 个 Cookie") + else: + print("登录失败或超时") + finally: + await qr_login.close() + + +if __name__ == "__main__": + asyncio.run(qrcode_login_demo()) +``` + +### 5.2 多平台登录管理 + +```python +async def multi_platform_demo(): + """多平台登录管理示例""" + + # 配置多个平台 + platforms = { + "platform_a": { + "cookie_path": "data/platform_a_cookies.json", + "login_url": "https://a.example.com/login", + "qrcode_selector": "#qrcode-img", + "success_url": "/home" + }, + "platform_b": { + "cookie_path": "data/platform_b_cookies.json", + "login_url": "https://b.example.com/login", + "qrcode_selector": ".login-qrcode", + "success_url": "/dashboard" + } + } + + for name, config in platforms.items(): + print(f"\n处理平台: {name}") + + manager = LoginManager( + platform=name, + cookie_path=config["cookie_path"], + preferred_method=LoginMethod.QRCODE + ) + + # 定义二维码回调 + async def on_qrcode(path: str): + print(f"[{name}] 请扫描二维码: {path}") + + success = await manager.ensure_login( + login_url=config["login_url"], + qrcode_selector=config["qrcode_selector"], + success_url_pattern=config["success_url"], + on_qrcode_ready=on_qrcode + ) + + if success: + cookies = manager.get_cookies_dict() + print(f"[{name}] 登录成功，Cookie 数量: {len(cookies)}") +``` + +## 六、最佳实践 + +### 6.1 安全建议 + +1. **不要频繁登录**：频繁登录可能触发风控 +2. **保护登录凭证**：Cookie 应加密存储 +3. **遵守服务条款**：了解平台的使用限制 +4. **处理敏感信息**：手机号等信息不要硬编码 + +### 6.2 稳定性建议 + +1. **超时处理**：设置合理的超时时间 +2. **重试机制**：登录失败时适当重试 +3. **状态监控**：监控登录状态，及时发现问题 +4. **优雅降级**：提供多种登录方式作为备选 + +### 6.3 代码组织建议 + +``` +login/ +├── __init__.py +├── base.py # 基类定义 +├── factory.py # 登录工厂 +├── qrcode.py # 扫码登录实现 +├── sms.py # 短信登录实现 +├── cookie.py # Cookie 登录实现 +└── manager.py # 统一管理器 +``` + +## 七、B站扫码登录实战 + +本节以 B站 为实战平台，演示完整的扫码登录实现。B站 是国内最大的二次元视频平台，其登录系统具有代表性。 + +### 7.1 B站扫码登录流程分析 + +B站扫码登录涉及以下API： + +```mermaid +sequenceDiagram + participant PC as PC浏览器 + participant Server as B站服务器 + participant APP as B站APP + + PC->>Server: 1. GET /qrcode/generate + Server-->>PC: 2. 返回 qrcode_key + url + + Note over PC: 生成二维码图片 + + loop 轮询登录状态 (间隔2秒) + PC->>Server: 3. GET /qrcode/poll?qrcode_key=xxx + Server-->>PC: 4. 返回状态码 + Note right of Server: 86101=未扫描
86090=已扫描
86038=已过期
0=已确认 + end + + APP->>Server: 5. 用户扫码 + Server-->>PC: 6. 状态: 86090 已扫描 + + APP->>Server: 7. 用户确认登录 + Server-->>PC: 8. 状态: 0 + Set-Cookie + + Note over PC: 9. 登录成功！保存Cookie +``` + +### 7.2 B站登录状态码 + +| 状态码 | 含义 | 说明 | +|-------|------|------| +| 0 | 成功 | 登录成功，响应中包含Cookie | +| 86101 | 未扫描 | 等待用户扫描二维码 | +| 86090 | 已扫描 | 用户已扫描，等待确认 | +| 86038 | 已过期 | 二维码已过期，需重新获取 | + +### 7.3 B站扫码登录完整实现 + +```python +import asyncio +import httpx +import qrcode +from io import BytesIO +from dataclasses import dataclass +from typing import Optional, Callable, Awaitable +from enum import IntEnum +from loguru import logger + + +class BilibiliQRStatus(IntEnum): + """B站扫码状态码""" + SUCCESS = 0 # 登录成功 + NOT_SCANNED = 86101 # 未扫描 + SCANNED = 86090 # 已扫描，待确认 + EXPIRED = 86038 # 已过期 + + +@dataclass +class BilibiliCookies: + """B站Cookie数据类""" + sessdata: str + dede_user_id: str + bili_jct: str + buvid3: str = "" + buvid4: str = "" + sid: str = "" + + def to_dict(self) -> dict: + """转换为httpx可用的字典格式""" + return { + "SESSDATA": self.sessdata, + "DedeUserID": self.dede_user_id, + "bili_jct": self.bili_jct, + "buvid3": self.buvid3, + "buvid4": self.buvid4, + "sid": self.sid, + } + + +class BilibiliQRCodeLogin: + """B站扫码登录实现""" + + # B站登录相关API + QRCODE_GENERATE_URL = "https://passport.bilibili.com/x/passport-login/web/qrcode/generate" + QRCODE_POLL_URL = "https://passport.bilibili.com/x/passport-login/web/qrcode/poll" + + def __init__( + self, + timeout: int = 180, + poll_interval: float = 2.0, + on_status_change: Optional[Callable[[int, str], Awaitable[None]]] = None + ): + """ + Args: + timeout: 登录超时时间（秒） + poll_interval: 状态轮询间隔（秒） + on_status_change: 状态变化回调 (status_code, message) + """ + self.timeout = timeout + self.poll_interval = poll_interval + self.on_status_change = on_status_change + + self._client: Optional[httpx.AsyncClient] = None + self._qrcode_key: str = "" + self._current_status: int = -1 + + async def __aenter__(self): + """异步上下文管理器入口""" + self._client = httpx.AsyncClient( + headers={ + "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) " + "AppleWebKit/537.36 (KHTML, like Gecko) " + "Chrome/120.0.0.0 Safari/537.36", + "Referer": "https://www.bilibili.com/", + } + ) + return self + + async def __aexit__(self, exc_type, exc_val, exc_tb): + """异步上下文管理器出口""" + if self._client: + await self._client.aclose() + + async def _notify_status(self, code: int, message: str): + """通知状态变化""" + if code != self._current_status: + self._current_status = code + logger.info(f"B站登录状态: {message} ({code})") + if self.on_status_change: + await self.on_status_change(code, message) + + async def generate_qrcode(self) -> tuple[str, bytes]: + """ + 生成登录二维码 + + Returns: + (qrcode_url, qrcode_image_bytes) + """ + resp = await self._client.get(self.QRCODE_GENERATE_URL) + data = resp.json() + + if data["code"] != 0: + raise Exception(f"获取二维码失败: {data['message']}") + + self._qrcode_key = data["data"]["qrcode_key"] + qrcode_url = data["data"]["url"] + + # 生成二维码图片 + qr = qrcode.QRCode( + version=1, + error_correction=qrcode.constants.ERROR_CORRECT_L, + box_size=10, + border=2 + ) + qr.add_data(qrcode_url) + qr.make(fit=True) + + img = qr.make_image(fill_color="black", back_color="white") + buffer = BytesIO() + img.save(buffer, format="PNG") + + logger.info("B站登录二维码已生成") + return qrcode_url, buffer.getvalue() + + def print_qrcode_to_terminal(self, url: str): + """在终端打印二维码""" + qr = qrcode.QRCode( + version=1, + error_correction=qrcode.constants.ERROR_CORRECT_L, + box_size=1, + border=1 + ) + qr.add_data(url) + qr.make(fit=True) + qr.print_ascii(invert=True) + + async def poll_status(self) -> tuple[int, Optional[BilibiliCookies]]: + """ + 轮询登录状态 + + Returns: + (status_code, cookies_if_success) + """ + resp = await self._client.get( + self.QRCODE_POLL_URL, + params={"qrcode_key": self._qrcode_key} + ) + data = resp.json() + + code = data["data"]["code"] + message = data["data"]["message"] + + await self._notify_status(code, message) + + if code == BilibiliQRStatus.SUCCESS: + # 登录成功，从响应中提取Cookie + cookies = self._extract_cookies(resp) + return code, cookies + + return code, None + + def _extract_cookies(self, resp: httpx.Response) -> BilibiliCookies: + """从响应中提取B站Cookie""" + cookies = resp.cookies + + # 同时从响应体获取refresh_token等信息 + data = resp.json()["data"] + + return BilibiliCookies( + sessdata=cookies.get("SESSDATA", ""), + dede_user_id=cookies.get("DedeUserID", ""), + bili_jct=cookies.get("bili_jct", ""), + buvid3=cookies.get("buvid3", ""), + buvid4=cookies.get("buvid4", ""), + sid=cookies.get("sid", ""), + ) + + async def login( + self, + save_qrcode_path: str = "bilibili_qrcode.png", + show_in_terminal: bool = True + ) -> Optional[BilibiliCookies]: + """ + 执行完整的扫码登录流程 + + Args: + save_qrcode_path: 二维码图片保存路径 + show_in_terminal: 是否在终端显示二维码 + + Returns: + 登录成功返回Cookie，失败返回None + """ + # 1. 生成二维码 + url, image_bytes = await self.generate_qrcode() + + # 保存二维码图片 + with open(save_qrcode_path, "wb") as f: + f.write(image_bytes) + logger.info(f"二维码已保存至: {save_qrcode_path}") + + # 在终端显示 + if show_in_terminal: + print("\n请使用B站APP扫描以下二维码登录:\n") + self.print_qrcode_to_terminal(url) + print(f"\n二维码图片也已保存至: {save_qrcode_path}\n") + + # 2. 轮询登录状态 + start_time = asyncio.get_event_loop().time() + + while True: + elapsed = asyncio.get_event_loop().time() - start_time + if elapsed > self.timeout: + logger.warning("登录超时") + return None + + code, cookies = await self.poll_status() + + if code == BilibiliQRStatus.SUCCESS: + logger.info("B站登录成功！") + return cookies + + if code == BilibiliQRStatus.EXPIRED: + logger.warning("二维码已过期") + return None + + await asyncio.sleep(self.poll_interval) + + +async def bilibili_qrcode_login_demo(): + """B站扫码登录演示""" + import json + from pathlib import Path + + # 状态变化回调 + async def on_status(code: int, message: str): + status_emoji = { + BilibiliQRStatus.NOT_SCANNED: "⏳", + BilibiliQRStatus.SCANNED: "📱", + BilibiliQRStatus.SUCCESS: "✅", + BilibiliQRStatus.EXPIRED: "❌", + } + emoji = status_emoji.get(code, "❓") + print(f"{emoji} {message}") + + async with BilibiliQRCodeLogin( + timeout=180, + poll_interval=2.0, + on_status_change=on_status + ) as login: + cookies = await login.login( + save_qrcode_path="bilibili_qrcode.png", + show_in_terminal=True + ) + + if cookies: + # 保存Cookie + cookie_path = Path("data/bilibili_cookies.json") + cookie_path.parent.mkdir(parents=True, exist_ok=True) + + with open(cookie_path, "w") as f: + json.dump(cookies.to_dict(), f, indent=2, ensure_ascii=False) + + print(f"\n登录成功！Cookie已保存至: {cookie_path}") + print(f"SESSDATA: {cookies.sessdata[:20]}...") + print(f"DedeUserID: {cookies.dede_user_id}") + else: + print("\n登录失败或超时") + + +if __name__ == "__main__": + asyncio.run(bilibili_qrcode_login_demo()) +``` + +### 7.4 使用已登录Cookie访问B站API + +```python +import httpx +import json +from pathlib import Path + + +async def use_bilibili_cookies(): + """使用已保存的Cookie访问B站API""" + + # 1. 加载Cookie + cookie_path = Path("data/bilibili_cookies.json") + if not cookie_path.exists(): + print("请先执行扫码登录获取Cookie") + return + + with open(cookie_path) as f: + cookies = json.load(f) + + # 2. 创建带Cookie的客户端 + async with httpx.AsyncClient( + cookies=cookies, + headers={ + "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) " + "AppleWebKit/537.36 (KHTML, like Gecko) " + "Chrome/120.0.0.0 Safari/537.36", + "Referer": "https://www.bilibili.com/", + } + ) as client: + # 3. 获取用户信息 + resp = await client.get( + "https://api.bilibili.com/x/web-interface/nav" + ) + data = resp.json() + + if data["code"] == 0: + user_info = data["data"] + print(f"登录用户: {user_info['uname']}") + print(f"用户ID: {user_info['mid']}") + print(f"等级: LV{user_info['level_info']['current_level']}") + print(f"硬币: {user_info['money']}") + else: + print(f"获取用户信息失败: {data['message']}") + + +if __name__ == "__main__": + import asyncio + asyncio.run(use_bilibili_cookies()) +``` + +### 7.5 Cookie有效性验证 + +```python +import httpx +from typing import Optional + + +async def verify_bilibili_cookies(cookies: dict) -> Optional[dict]: + """ + 验证B站Cookie是否有效 + + Args: + cookies: Cookie字典 + + Returns: + 有效返回用户信息，无效返回None + """ + async with httpx.AsyncClient( + cookies=cookies, + headers={ + "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) " + "AppleWebKit/537.36 Chrome/120.0.0.0 Safari/537.36", + "Referer": "https://www.bilibili.com/", + } + ) as client: + resp = await client.get( + "https://api.bilibili.com/x/web-interface/nav" + ) + data = resp.json() + + if data["code"] == 0 and data["data"]["isLogin"]: + return { + "mid": data["data"]["mid"], + "uname": data["data"]["uname"], + "level": data["data"]["level_info"]["current_level"], + "vip_type": data["data"]["vipType"], + } + + return None + + +async def refresh_or_relogin(): + """检查Cookie，过期则重新登录""" + import json + from pathlib import Path + + cookie_path = Path("data/bilibili_cookies.json") + + # 1. 尝试加载并验证现有Cookie + if cookie_path.exists(): + with open(cookie_path) as f: + cookies = json.load(f) + + user_info = await verify_bilibili_cookies(cookies) + if user_info: + print(f"Cookie有效，当前用户: {user_info['uname']}") + return cookies + + print("Cookie已失效，需要重新登录") + + # 2. 执行扫码登录 + async with BilibiliQRCodeLogin() as login: + result = await login.login() + if result: + with open(cookie_path, "w") as f: + json.dump(result.to_dict(), f, indent=2) + print("重新登录成功") + return result.to_dict() + + return None +``` + +--- + +## 八、与第11章的关联 + +本章介绍的B站扫码登录技术在第11章综合实战项目中有完整应用： + +### 代码位置 +- **登录模块**：`源代码/爬虫进阶/11_进阶综合实战项目/login/auth.py` +- **Cookie管理**：`源代码/爬虫进阶/11_进阶综合实战项目/client/bilibili_client.py` + +### 技术要点对应 +| 本章内容 | 第11章实现 | +|---------|-----------| +| `BilibiliQRCodeLogin` 类 | `login/auth.py` 中的登录逻辑 | +| `BilibiliCookies` 数据类 | `models/` 中的数据模型 | +| Cookie有效性验证 | `client/` 中的请求拦截器 | +| 状态码枚举 | `config/bilibili_config.py` 中的常量定义 | + +### 学习路径 + +```mermaid +graph LR + A[第06章
Cookie管理] --> B[第07章
扫码登录] + B --> C[第08章
验证码处理] + C --> D[第11章
综合实战] + + style B fill:#e1f5fe + style D fill:#fff9c4 +``` + +掌握本章的扫码登录技术后，你已经具备了B站爬虫的核心认证能力，可以直接应用到第11章的综合项目中。 + +--- + +## 本章小结 + +本章深入讲解了扫码登录和短信验证码登录的实现： + +1. **扫码登录原理**：二维码生成、状态轮询、凭证下发的完整流程 +2. **Playwright 实现**：使用浏览器自动化实现扫码登录 +3. **短信登录实现**：手机号输入、验证码获取、登录提交 +4. **统一封装**：使用工厂模式封装多种登录方式 +5. **B站实战**：完整的B站扫码登录实现，包括二维码生成、状态轮询、Cookie提取 + +掌握这些技术后，你可以应对大多数需要登录的爬虫场景。 + +## 下一章预告 + +下一章我们将学习**验证码识别与处理**，包括图片验证码 OCR 识别、滑块验证码轨迹模拟等技术。这些技术在登录和爬取过程中经常遇到，是爬虫进阶的重要内容。 diff --git "a/docs/\347\210\254\350\231\253\350\277\233\344\273\267/08_\351\252\214\350\257\201\347\240\201\350\257\206\345\210\253\344\270\216\345\244\204\347\220\206.md" "b/docs/\347\210\254\350\231\253\350\277\233\344\273\267/08_\351\252\214\350\257\201\347\240\201\350\257\206\345\210\253\344\270\216\345\244\204\347\220\206.md" new file mode 100644 index 0000000..99685d5 --- /dev/null +++ "b/docs/\347\210\254\350\231\253\350\277\233\344\273\267/08_\351\252\214\350\257\201\347\240\201\350\257\206\345\210\253\344\270\216\345\244\204\347\220\206.md" @@ -0,0 +1,1180 @@ +# 08_验证码识别与处理 + +验证码是网站用于区分人类和机器的重要手段。本章将介绍常见的验证码类型，以及如何在爬虫中处理这些验证码。 + +## 一、验证码类型概览 + +### 1.1 常见验证码类型 + +| 类型 | 特点 | 难度 | 常见场景 | +|-----|------|------|---------| +| 图片字符验证码 | 数字/字母/汉字 | ⭐⭐ | 登录、注册 | +| 数学运算验证码 | 简单算术题 | ⭐ | 评论、提交 | +| 滑块拼图验证码 | 拖动滑块到缺口 | ⭐⭐⭐ | 登录、支付 | +| 点选验证码 | 点击指定图片/文字 | ⭐⭐⭐⭐ | 登录、敏感操作 | +| 行为验证码 | 分析用户行为 | ⭐⭐⭐⭐⭐ | 全站防护 | + +### 1.2 验证码的工作原理 + +```mermaid +sequenceDiagram + participant User as 用户 + participant Frontend as 前端 + participant Server as 服务器 + + User->>Frontend: 1. 请求验证码 + Frontend->>Server: 2. 请求验证码 + Server-->>Frontend: 3. 返回验证码图片+ID + Frontend-->>User: 4. 显示验证码 + + User->>Frontend: 5. 输入验证码 + Frontend->>Server: 6. 提交验证 + Server-->>Frontend: 7. 返回验证结果 + + Note over User,Server: 验证通过后继续操作 +``` + +## 二、图片验证码识别 + +### 2.1 ddddocr 库介绍 + +ddddocr 是一个开源的 OCR 库，专门用于识别验证码： + +```bash +pip install ddddocr +``` + +### 2.2 基本使用 + +```python +import ddddocr + +# 创建 OCR 实例 +ocr = ddddocr.DdddOcr() + +# 读取验证码图片 +with open("captcha.png", "rb") as f: + image_bytes = f.read() + +# 识别 +result = ocr.classification(image_bytes) +print(f"识别结果: {result}") +``` + +### 2.3 图片预处理 + +对于复杂验证码，预处理可以提高识别率： + +```python +from PIL import Image +import io + +class CaptchaPreprocessor: + """验证码图片预处理""" + + @staticmethod + def to_grayscale(image_bytes: bytes) -> bytes: + """转为灰度图""" + img = Image.open(io.BytesIO(image_bytes)) + gray = img.convert('L') + + buffer = io.BytesIO() + gray.save(buffer, format='PNG') + return buffer.getvalue() + + @staticmethod + def binarize(image_bytes: bytes, threshold: int = 127) -> bytes: + """ + 二值化 + + Args: + image_bytes: 图片字节 + threshold: 阈值，小于阈值变黑，大于变白 + """ + img = Image.open(io.BytesIO(image_bytes)) + gray = img.convert('L') + + # 二值化 + binary = gray.point(lambda x: 255 if x > threshold else 0) + + buffer = io.BytesIO() + binary.save(buffer, format='PNG') + return buffer.getvalue() + + @staticmethod + def remove_noise(image_bytes: bytes) -> bytes: + """ + 去噪点（简单的中值滤波） + + 需要安装: pip install pillow + """ + from PIL import ImageFilter + + img = Image.open(io.BytesIO(image_bytes)) + # 中值滤波去噪 + denoised = img.filter(ImageFilter.MedianFilter(size=3)) + + buffer = io.BytesIO() + denoised.save(buffer, format='PNG') + return buffer.getvalue() + + @staticmethod + def enhance_contrast(image_bytes: bytes, factor: float = 2.0) -> bytes: + """ + 增强对比度 + + Args: + factor: 对比度因子，>1 增强，<1 降低 + """ + from PIL import ImageEnhance + + img = Image.open(io.BytesIO(image_bytes)) + enhancer = ImageEnhance.Contrast(img) + enhanced = enhancer.enhance(factor) + + buffer = io.BytesIO() + enhanced.save(buffer, format='PNG') + return buffer.getvalue() +``` + +### 2.4 封装的 OCR 识别器 + +```python +import ddddocr +from typing import Optional +from loguru import logger + +class OCRCaptchaSolver: + """OCR 验证码识别器""" + + def __init__(self, preprocess: bool = True): + """ + Args: + preprocess: 是否进行预处理 + """ + self.ocr = ddddocr.DdddOcr(show_ad=False) + self.preprocess = preprocess + self.preprocessor = CaptchaPreprocessor() + + def solve(self, image_bytes: bytes) -> Optional[str]: + """ + 识别验证码 + + Args: + image_bytes: 图片字节 + + Returns: + 识别结果 + """ + try: + # 预处理 + if self.preprocess: + image_bytes = self._preprocess(image_bytes) + + # 识别 + result = self.ocr.classification(image_bytes) + logger.debug(f"验证码识别结果: {result}") + return result + + except Exception as e: + logger.error(f"验证码识别失败: {e}") + return None + + def _preprocess(self, image_bytes: bytes) -> bytes: + """预处理流程""" + # 灰度化 + image_bytes = self.preprocessor.to_grayscale(image_bytes) + # 增强对比度 + image_bytes = self.preprocessor.enhance_contrast(image_bytes) + # 二值化 + image_bytes = self.preprocessor.binarize(image_bytes, threshold=150) + return image_bytes + + def solve_with_retry( + self, + image_bytes: bytes, + max_retries: int = 3 + ) -> Optional[str]: + """ + 带重试的识别 + + 尝试不同的预处理参数 + """ + # 尝试不同阈值 + thresholds = [127, 100, 150] + + for threshold in thresholds: + try: + processed = self.preprocessor.to_grayscale(image_bytes) + processed = self.preprocessor.binarize(processed, threshold) + result = self.ocr.classification(processed) + + if result and len(result) >= 4: # 假设验证码至少 4 位 + return result + + except Exception: + continue + + # 最后尝试原图 + return self.ocr.classification(image_bytes) +``` + +## 三、滑块验证码处理 + +滑块验证码需要识别缺口位置并模拟人类拖拽行为。 + +### 3.1 缺口位置识别 + +```python +import cv2 +import numpy as np +from typing import Tuple, Optional +from loguru import logger + +class SliderGapDetector: + """滑块缺口位置检测器""" + + @staticmethod + def detect_gap_by_edge( + background_bytes: bytes, + slider_bytes: bytes + ) -> Optional[int]: + """ + 通过边缘检测找缺口位置 + + Args: + background_bytes: 背景图片字节 + slider_bytes: 滑块图片字节 + + Returns: + 缺口 x 坐标 + """ + # 读取图片 + bg = cv2.imdecode( + np.frombuffer(background_bytes, np.uint8), + cv2.IMREAD_COLOR + ) + slider = cv2.imdecode( + np.frombuffer(slider_bytes, np.uint8), + cv2.IMREAD_COLOR + ) + + # 转灰度 + bg_gray = cv2.cvtColor(bg, cv2.COLOR_BGR2GRAY) + slider_gray = cv2.cvtColor(slider, cv2.COLOR_BGR2GRAY) + + # 边缘检测 + bg_edges = cv2.Canny(bg_gray, 100, 200) + slider_edges = cv2.Canny(slider_gray, 100, 200) + + # 模板匹配 + result = cv2.matchTemplate(bg_edges, slider_edges, cv2.TM_CCOEFF_NORMED) + _, _, _, max_loc = cv2.minMaxLoc(result) + + gap_x = max_loc[0] + logger.debug(f"检测到缺口位置: x={gap_x}") + return gap_x + + @staticmethod + def detect_gap_by_color( + background_bytes: bytes, + target_color: Tuple[int, int, int] = (0, 0, 0) + ) -> Optional[int]: + """ + 通过颜色差异找缺口位置 + + 某些滑块验证码的缺口有特定颜色 + + Args: + background_bytes: 背景图片 + target_color: 缺口颜色 (B, G, R) + + Returns: + 缺口 x 坐标 + """ + bg = cv2.imdecode( + np.frombuffer(background_bytes, np.uint8), + cv2.IMREAD_COLOR + ) + + # 转 HSV + hsv = cv2.cvtColor(bg, cv2.COLOR_BGR2HSV) + + # 创建颜色掩码 + lower = np.array([0, 0, 0]) + upper = np.array([180, 255, 50]) # 暗色区域 + mask = cv2.inRange(hsv, lower, upper) + + # 找轮廓 + contours, _ = cv2.findContours( + mask, + cv2.RETR_EXTERNAL, + cv2.CHAIN_APPROX_SIMPLE + ) + + if contours: + # 找最大轮廓 + largest = max(contours, key=cv2.contourArea) + x, y, w, h = cv2.boundingRect(largest) + return x + + return None +``` + +### 3.2 人类轨迹模拟 + +模拟人类拖拽行为是绕过行为检测的关键： + +```python +import random +import math +from typing import List, Tuple + +class HumanTrajectoryGenerator: + """人类轨迹生成器""" + + @staticmethod + def generate_trajectory( + distance: int, + duration: float = 0.5 + ) -> List[Tuple[int, int, float]]: + """ + 生成模拟人类的拖拽轨迹 + + Args: + distance: 需要移动的距离 + duration: 预期持续时间（秒） + + Returns: + 轨迹点列表 [(x, y, time), ...] + """ + trajectory = [] + current_x = 0 + current_time = 0 + + # 使用缓动函数模拟加速-匀速-减速 + steps = random.randint(20, 30) + step_time = duration / steps + + for i in range(steps): + # 使用 ease-out 缓动 + progress = i / steps + eased = HumanTrajectoryGenerator._ease_out_quad(progress) + + target_x = int(distance * eased) + move_x = target_x - current_x + + # 添加随机偏移 + offset_y = random.randint(-3, 3) + + current_x = target_x + current_time += step_time + random.uniform(-0.01, 0.01) + + trajectory.append((current_x, offset_y, current_time)) + + # 确保最后到达目标 + trajectory.append((distance, 0, duration)) + + return trajectory + + @staticmethod + def _ease_out_quad(t: float) -> float: + """二次缓出函数""" + return t * (2 - t) + + @staticmethod + def _ease_out_cubic(t: float) -> float: + """三次缓出函数""" + return 1 - pow(1 - t, 3) + + @staticmethod + def generate_bezier_trajectory( + distance: int, + duration: float = 0.5 + ) -> List[Tuple[int, int, float]]: + """ + 使用贝塞尔曲线生成更自然的轨迹 + + Args: + distance: 移动距离 + duration: 持续时间 + + Returns: + 轨迹点列表 + """ + trajectory = [] + + # 控制点 + p0 = (0, 0) + p1 = (distance * 0.3, random.randint(-10, 10)) + p2 = (distance * 0.7, random.randint(-5, 5)) + p3 = (distance, 0) + + steps = random.randint(25, 35) + + for i in range(steps + 1): + t = i / steps + + # 三阶贝塞尔曲线 + x = (1-t)**3 * p0[0] + 3*(1-t)**2*t * p1[0] + 3*(1-t)*t**2 * p2[0] + t**3 * p3[0] + y = (1-t)**3 * p0[1] + 3*(1-t)**2*t * p1[1] + 3*(1-t)*t**2 * p2[1] + t**3 * p3[1] + + time_point = duration * t + random.uniform(-0.005, 0.005) + trajectory.append((int(x), int(y), max(0, time_point))) + + return trajectory +``` + +### 3.3 Playwright 实现滑块拖拽 + +```python +import asyncio +from playwright.async_api import Page +from typing import List, Tuple +from loguru import logger + +class SliderCaptchaSolver: + """滑块验证码解决器""" + + def __init__(self, page: Page): + self.page = page + self.gap_detector = SliderGapDetector() + self.trajectory_generator = HumanTrajectoryGenerator() + + async def solve( + self, + slider_selector: str, + background_selector: str, + slider_image_selector: str + ) -> bool: + """ + 解决滑块验证码 + + Args: + slider_selector: 滑块按钮选择器 + background_selector: 背景图选择器 + slider_image_selector: 滑块图片选择器 + + Returns: + 是否成功 + """ + try: + # 获取背景和滑块图片 + bg_element = self.page.locator(background_selector) + slider_element = self.page.locator(slider_image_selector) + + bg_bytes = await bg_element.screenshot() + slider_bytes = await slider_element.screenshot() + + # 检测缺口位置 + gap_x = self.gap_detector.detect_gap_by_edge(bg_bytes, slider_bytes) + if not gap_x: + logger.error("无法检测缺口位置") + return False + + logger.info(f"缺口位置: {gap_x}") + + # 执行拖拽 + await self._drag_slider(slider_selector, gap_x) + + # 等待验证结果 + await asyncio.sleep(1) + + return True + + except Exception as e: + logger.error(f"滑块验证码处理失败: {e}") + return False + + async def _drag_slider(self, selector: str, distance: int): + """ + 执行拖拽操作 + + Args: + selector: 滑块选择器 + distance: 拖拽距离 + """ + slider = self.page.locator(selector) + box = await slider.bounding_box() + + if not box: + raise Exception("无法获取滑块位置") + + # 起始位置（滑块中心） + start_x = box['x'] + box['width'] / 2 + start_y = box['y'] + box['height'] / 2 + + # 生成轨迹 + trajectory = self.trajectory_generator.generate_bezier_trajectory(distance) + + # 鼠标移动到滑块 + await self.page.mouse.move(start_x, start_y) + await asyncio.sleep(random.uniform(0.1, 0.2)) + + # 按下鼠标 + await self.page.mouse.down() + await asyncio.sleep(random.uniform(0.05, 0.1)) + + # 沿轨迹移动 + last_time = 0 + for x, y, time_point in trajectory: + # 计算时间间隔 + delay = time_point - last_time + if delay > 0: + await asyncio.sleep(delay) + last_time = time_point + + # 移动鼠标 + await self.page.mouse.move(start_x + x, start_y + y) + + # 松开鼠标 + await asyncio.sleep(random.uniform(0.05, 0.1)) + await self.page.mouse.up() + + logger.info(f"滑块拖拽完成，距离: {distance}px") + + +# 需要导入 random +import random +``` + +## 四、第三方打码平台 + +对于复杂验证码，可以使用第三方打码平台。 + +### 4.1 打码平台接口封装 + +```python +import httpx +import asyncio +from abc import ABC, abstractmethod +from typing import Optional +from loguru import logger + +class CaptchaServiceBase(ABC): + """打码平台基类""" + + @abstractmethod + async def solve_image(self, image_bytes: bytes) -> Optional[str]: + """识别图片验证码""" + pass + + @abstractmethod + async def report_error(self, task_id: str): + """报告识别错误（退款）""" + pass + + @abstractmethod + async def get_balance(self) -> float: + """获取账户余额""" + pass + + +class GenericCaptchaService(CaptchaServiceBase): + """ + 通用打码平台接口 + + 注意：这是一个示例实现，实际使用需要根据具体平台的 API 调整 + """ + + def __init__( + self, + api_key: str, + api_url: str, + timeout: int = 30 + ): + """ + Args: + api_key: API 密钥 + api_url: API 地址 + timeout: 超时时间 + """ + self.api_key = api_key + self.api_url = api_url + self.timeout = timeout + self._last_task_id: Optional[str] = None + + async def solve_image( + self, + image_bytes: bytes, + captcha_type: str = "default" + ) -> Optional[str]: + """ + 识别图片验证码 + + Args: + image_bytes: 图片字节 + captcha_type: 验证码类型 + + Returns: + 识别结果 + """ + import base64 + + try: + async with httpx.AsyncClient(timeout=self.timeout) as client: + # 提交任务 + image_base64 = base64.b64encode(image_bytes).decode() + + resp = await client.post( + f"{self.api_url}/create_task", + json={ + "api_key": self.api_key, + "image": image_base64, + "type": captcha_type + } + ) + + data = resp.json() + task_id = data.get("task_id") + + if not task_id: + logger.error(f"创建任务失败: {data}") + return None + + self._last_task_id = task_id + + # 轮询获取结果 + result = await self._poll_result(client, task_id) + return result + + except Exception as e: + logger.error(f"打码平台请求失败: {e}") + return None + + async def _poll_result( + self, + client: httpx.AsyncClient, + task_id: str, + max_attempts: int = 30 + ) -> Optional[str]: + """轮询获取结果""" + for _ in range(max_attempts): + try: + resp = await client.get( + f"{self.api_url}/get_result", + params={"task_id": task_id} + ) + + data = resp.json() + status = data.get("status") + + if status == "ready": + return data.get("result") + elif status == "error": + logger.error(f"识别错误: {data.get('error')}") + return None + + await asyncio.sleep(1) + + except Exception as e: + logger.warning(f"轮询异常: {e}") + await asyncio.sleep(1) + + logger.error("识别超时") + return None + + async def report_error(self, task_id: str = None): + """报告识别错误""" + task_id = task_id or self._last_task_id + if not task_id: + return + + try: + async with httpx.AsyncClient() as client: + await client.post( + f"{self.api_url}/report_error", + json={ + "api_key": self.api_key, + "task_id": task_id + } + ) + logger.info(f"已报告错误: {task_id}") + except Exception as e: + logger.warning(f"报告错误失败: {e}") + + async def get_balance(self) -> float: + """获取余额""" + try: + async with httpx.AsyncClient() as client: + resp = await client.get( + f"{self.api_url}/balance", + params={"api_key": self.api_key} + ) + data = resp.json() + return data.get("balance", 0.0) + except Exception as e: + logger.error(f"获取余额失败: {e}") + return 0.0 +``` + +### 4.2 带重试的验证码处理 + +```python +from typing import Callable, Awaitable + +class CaptchaSolverWithRetry: + """带重试的验证码处理器""" + + def __init__( + self, + primary_solver: Callable[[bytes], Awaitable[Optional[str]]], + fallback_solver: Callable[[bytes], Awaitable[Optional[str]]] = None, + max_retries: int = 3 + ): + """ + Args: + primary_solver: 主要识别方法 + fallback_solver: 备用识别方法（如打码平台） + max_retries: 最大重试次数 + """ + self.primary_solver = primary_solver + self.fallback_solver = fallback_solver + self.max_retries = max_retries + + async def solve( + self, + get_image: Callable[[], Awaitable[bytes]], + verify: Callable[[str], Awaitable[bool]] + ) -> Optional[str]: + """ + 解决验证码 + + Args: + get_image: 获取验证码图片的方法 + verify: 验证结果的方法 + + Returns: + 成功的验证码结果 + """ + for attempt in range(self.max_retries): + # 获取验证码图片 + image = await get_image() + + # 尝试主要识别方法 + result = await self.primary_solver(image) + if result and await verify(result): + logger.info(f"验证码识别成功 (尝试 {attempt + 1})") + return result + + # 尝试备用方法 + if self.fallback_solver: + result = await self.fallback_solver(image) + if result and await verify(result): + logger.info(f"备用方法识别成功 (尝试 {attempt + 1})") + return result + + logger.warning(f"验证码验证失败，重试 ({attempt + 1}/{self.max_retries})") + + logger.error("验证码处理失败，已达最大重试次数") + return None +``` + +## 五、合规与伦理 + +### 5.1 法律边界 + +1. **遵守服务条款**：了解目标网站的使用条款 +2. **合理使用**：仅用于学习研究或授权的测试 +3. **不用于恶意目的**：不进行攻击、欺诈等行为 +4. **保护用户隐私**：不获取、存储他人隐私信息 + +### 5.2 替代方案 + +在某些场景下，可以考虑替代方案： + +1. **官方 API**：很多平台提供开放 API，无需处理验证码 +2. **登录态复用**：使用已登录的 Cookie，减少登录次数 +3. **降低频率**：控制请求频率，避免触发验证码 +4. **联系网站**：对于数据研究，可联系网站获取授权 + +### 5.3 成本控制 + +使用打码平台时的成本控制建议： + +```python +class CostController: + """打码成本控制器""" + + def __init__( + self, + daily_budget: float, + cost_per_captcha: float = 0.01 + ): + self.daily_budget = daily_budget + self.cost_per_captcha = cost_per_captcha + self._daily_spent = 0.0 + self._last_reset = None + + def can_use_service(self) -> bool: + """是否可以使用打码服务""" + self._check_reset() + return self._daily_spent < self.daily_budget + + def record_usage(self): + """记录一次使用""" + self._daily_spent += self.cost_per_captcha + + def _check_reset(self): + """检查是否需要重置""" + from datetime import date + today = date.today() + if self._last_reset != today: + self._daily_spent = 0.0 + self._last_reset = today + + @property + def remaining_budget(self) -> float: + """剩余预算""" + self._check_reset() + return max(0, self.daily_budget - self._daily_spent) +``` + +## 六、实战示例 + +### 6.1 完整的验证码处理流程 + +```python +import asyncio +from playwright.async_api import async_playwright + +async def demo_captcha_solving(): + """验证码处理完整演示""" + + # 初始化 OCR 识别器 + ocr_solver = OCRCaptchaSolver(preprocess=True) + + async with async_playwright() as p: + browser = await p.chromium.launch(headless=False) + page = await browser.new_page() + + await page.goto("https://example.com/login") + + # 获取验证码图片 + captcha_element = page.locator("#captcha-image") + captcha_bytes = await captcha_element.screenshot() + + # 识别验证码 + code = ocr_solver.solve(captcha_bytes) + print(f"识别结果: {code}") + + # 输入验证码 + if code: + await page.fill("#captcha-input", code) + await page.click("#submit-btn") + + await asyncio.sleep(3) + await browser.close() +``` + +## 七、验证码处理实战 + +大型网站在特定场景下会触发验证码，本节介绍常见验证码的特点和处理方法。 + +### 7.1 验证码触发场景 + +```mermaid +flowchart TD + request[发起请求] --> check{风控检测} + check -->|正常| success[返回数据] + check -->|异常| trigger{触发验证码} + + trigger -->|高频请求| slider[滑块验证码] + trigger -->|IP异常| geetest[极验验证码] + trigger -->|敏感操作| click[点选验证码] + + slider --> verify{验证} + geetest --> verify + click --> verify + + verify -->|成功| success + verify -->|失败| block[临时封禁] +``` + +### 7.2 常见验证码类型 + +| 场景 | 验证码类型 | 触发条件 | 处理难度 | +|-----|-----------|---------|---------| +| 登录保护 | 滑块验证码 | 异地登录、频繁登录 | ⭐⭐⭐ | +| 接口防护 | 极验验证码 | 请求频率过高 | ⭐⭐⭐⭐ | +| 敏感操作 | 点选验证码 | 短时间大量操作 | ⭐⭐⭐⭐⭐ | +| 批量操作 | 简单确认 | 批量提交 | ⭐ | + +### 7.3 滑块验证码处理 + +```python +import asyncio +import httpx +from playwright.async_api import async_playwright, Page +from loguru import logger + + +class SliderCaptcha: + """滑块验证码处理器""" + + def __init__(self, page: Page): + self.page = page + + async def detect_and_solve(self) -> bool: + """ + 检测并解决滑块验证码 + + Returns: + 是否成功解决 + """ + try: + # 检测是否出现滑块验证码 + slider_frame = self.page.frame_locator("iframe[src*='captcha']") + + # 等待滑块出现（最多5秒） + try: + await slider_frame.locator(".geetest_slider_button").wait_for( + timeout=5000 + ) + except Exception: + # 没有验证码，正常情况 + return True + + logger.info("检测到滑块验证码") + + # 获取滑块和背景图 + bg_element = slider_frame.locator(".geetest_canvas_bg") + slider_element = slider_frame.locator(".geetest_canvas_slice") + + bg_bytes = await bg_element.screenshot() + slider_bytes = await slider_element.screenshot() + + # 检测缺口位置 + gap_x = self._detect_gap(bg_bytes, slider_bytes) + + if not gap_x: + logger.error("无法检测缺口位置") + return False + + # 执行拖拽 + await self._drag_slider(slider_frame, gap_x) + + # 等待验证结果 + await asyncio.sleep(2) + + # 检查是否成功 + try: + await slider_frame.locator(".geetest_success").wait_for( + timeout=3000 + ) + logger.info("滑块验证码通过") + return True + except Exception: + logger.warning("滑块验证码验证失败") + return False + + except Exception as e: + logger.error(f"滑块验证码处理异常: {e}") + return False + + def _detect_gap(self, bg_bytes: bytes, slider_bytes: bytes) -> int: + """检测缺口位置""" + import cv2 + import numpy as np + + bg = cv2.imdecode(np.frombuffer(bg_bytes, np.uint8), cv2.IMREAD_COLOR) + slider = cv2.imdecode(np.frombuffer(slider_bytes, np.uint8), cv2.IMREAD_COLOR) + + # 边缘检测 + bg_edges = cv2.Canny(cv2.cvtColor(bg, cv2.COLOR_BGR2GRAY), 100, 200) + slider_edges = cv2.Canny(cv2.cvtColor(slider, cv2.COLOR_BGR2GRAY), 100, 200) + + # 模板匹配 + result = cv2.matchTemplate(bg_edges, slider_edges, cv2.TM_CCOEFF_NORMED) + _, _, _, max_loc = cv2.minMaxLoc(result) + + return max_loc[0] + + async def _drag_slider(self, frame, distance: int): + """拖拽滑块""" + import random + + slider_btn = frame.locator(".geetest_slider_button") + box = await slider_btn.bounding_box() + + if not box: + raise Exception("无法获取滑块位置") + + start_x = box['x'] + box['width'] / 2 + start_y = box['y'] + box['height'] / 2 + + # 生成人类轨迹 + trajectory = self._generate_human_trajectory(distance) + + await self.page.mouse.move(start_x, start_y) + await asyncio.sleep(random.uniform(0.1, 0.2)) + + await self.page.mouse.down() + + for x, y, delay in trajectory: + await asyncio.sleep(delay) + await self.page.mouse.move(start_x + x, start_y + y) + + await asyncio.sleep(random.uniform(0.05, 0.1)) + await self.page.mouse.up() + + def _generate_human_trajectory(self, distance: int): + """生成人类轨迹""" + import random + + trajectory = [] + current_x = 0 + steps = random.randint(20, 30) + + for i in range(steps): + progress = i / steps + # 缓动函数 + eased = progress * (2 - progress) + target_x = int(distance * eased) + + x = target_x + y = random.randint(-3, 3) + delay = random.uniform(0.01, 0.03) + + trajectory.append((x, y, delay)) + + trajectory.append((distance, 0, 0.05)) + return trajectory + + +async def demo_captcha_handling(): + """带验证码处理的页面访问示例""" + + async with async_playwright() as p: + browser = await p.chromium.launch(headless=False) + context = await browser.new_context() + page = await context.new_page() + + # 访问测试页面 + await page.goto("https://example.com") + + # 模拟一些操作... + # 如果触发验证码，自动处理 + captcha_handler = SliderCaptcha(page) + + # 在关键操作后检查验证码 + success = await captcha_handler.detect_and_solve() + + if success: + print("操作成功，无验证码或验证码已处理") + else: + print("验证码处理失败") + + await browser.close() + + +if __name__ == "__main__": + asyncio.run(demo_captcha_handling()) +``` + +### 7.4 避免触发验证码的策略 + +在爬虫开发中，预防优于处理： + +```mermaid +flowchart LR + subgraph 预防策略 + A[控制请求频率] --> B[模拟真实行为] + B --> C[使用登录态] + C --> D[IP轮换] + end + + subgraph 建议配置 + E[每分钟<30请求] + F[随机延迟2-5秒] + G[保持Cookie有效] + H[高匿代理池] + end + + A --> E + B --> F + C --> G + D --> H +``` + +```python +import asyncio +import random +from typing import Optional + + +class RateLimiter: + """请求频率控制器""" + + def __init__( + self, + requests_per_minute: int = 20, + min_delay: float = 2.0, + max_delay: float = 5.0 + ): + self.requests_per_minute = requests_per_minute + self.min_delay = min_delay + self.max_delay = max_delay + self._last_request_time: Optional[float] = None + self._request_count = 0 + self._minute_start: Optional[float] = None + + async def wait(self): + """等待直到可以发送下一个请求""" + now = asyncio.get_event_loop().time() + + # 重置分钟计数 + if self._minute_start is None or now - self._minute_start > 60: + self._minute_start = now + self._request_count = 0 + + # 检查是否超过频率限制 + if self._request_count >= self.requests_per_minute: + wait_time = 60 - (now - self._minute_start) + if wait_time > 0: + await asyncio.sleep(wait_time) + self._minute_start = asyncio.get_event_loop().time() + self._request_count = 0 + + # 随机延迟 + if self._last_request_time: + elapsed = now - self._last_request_time + if elapsed < self.min_delay: + delay = random.uniform(self.min_delay, self.max_delay) + await asyncio.sleep(delay - elapsed) + + self._last_request_time = asyncio.get_event_loop().time() + self._request_count += 1 + + +# 使用示例 +rate_limiter = RateLimiter( + requests_per_minute=20, + min_delay=2.0, + max_delay=5.0 +) + +async def safe_request(client, url): + """安全的请求（带频率控制）""" + await rate_limiter.wait() + return await client.get(url) +``` + +--- + +## 本章小结 + +本章介绍了验证码识别与处理的核心技术： + +1. **验证码类型**：图片字符、滑块、点选等多种类型 +2. **OCR 识别**：使用 ddddocr 进行图片验证码识别 +3. **滑块处理**：缺口检测和人类轨迹模拟 +4. **打码平台**：第三方服务的接入和成本控制 +5. **合规考虑**：法律边界和替代方案 +6. **实战演练**：验证码触发场景和处理策略 + +验证码处理是爬虫进阶的重要技能，但务必在合法合规的前提下使用。 + +## 下一章预告 + +下一章我们将学习**数据清洗与预处理**，包括文本清洗、正则表达式应用、数据去重等技术。爬取的数据往往需要清洗才能使用，这是数据处理流程中的关键环节。 diff --git "a/docs/\347\210\254\350\231\253\350\277\233\344\273\267/09_\346\225\260\346\215\256\346\270\205\346\264\227\344\270\216\351\242\204\345\244\204\347\220\206.md" "b/docs/\347\210\254\350\231\253\350\277\233\344\273\267/09_\346\225\260\346\215\256\346\270\205\346\264\227\344\270\216\351\242\204\345\244\204\347\220\206.md" new file mode 100644 index 0000000..d0fade2 --- /dev/null +++ "b/docs/\347\210\254\350\231\253\350\277\233\344\273\267/09_\346\225\260\346\215\256\346\270\205\346\264\227\344\270\216\351\242\204\345\244\204\347\220\206.md" @@ -0,0 +1,1363 @@ +# 09_数据清洗与预处理 + +爬取的原始数据往往包含噪声、冗余和格式不统一等问题。本章将介绍数据清洗的核心技术，帮助你将"脏数据"转换为可用的"干净数据"。 + +## 一、数据清洗概述 + +### 1.1 为什么需要数据清洗 + +爬取的原始数据常见问题： + +| 问题类型 | 示例 | 影响 | +|---------|------|------| +| HTML 残留 | `

文本

` | 数据不纯净 | +| 空白字符 | 多余空格、换行 | 格式混乱 | +| 编码问题 | 乱码、特殊字符 | 无法正常显示 | +| 重复数据 | 相同内容多次出现 | 数据冗余 | +| 格式不统一 | 日期格式各异 | 难以分析 | +| 缺失值 | 空字段 | 数据不完整 | + +### 1.2 数据清洗流程 + +```mermaid +flowchart LR + raw[原始数据] --> format[格式清洗] + format --> content[内容清洗] + content --> dedup[去重处理] + dedup --> normalize[标准化] + normalize --> clean[干净数据] + + subgraph 格式清洗 + html[HTML标签移除] + encode[编码修复] + end + + subgraph 内容清洗 + whitespace[空白处理] + special[特殊字符] + end +``` + +### 1.3 数据质量评估 + +在清洗前后，评估数据质量： + +```python +from dataclasses import dataclass +from typing import List, Dict, Any + +@dataclass +class DataQualityReport: + """数据质量报告""" + total_records: int + valid_records: int + duplicate_count: int + missing_fields: Dict[str, int] + format_issues: Dict[str, int] + + @property + def validity_rate(self) -> float: + """有效率""" + return self.valid_records / self.total_records if self.total_records > 0 else 0 + + @property + def duplicate_rate(self) -> float: + """重复率""" + return self.duplicate_count / self.total_records if self.total_records > 0 else 0 + + def __str__(self) -> str: + return f""" +数据质量报告: + 总记录数: {self.total_records} + 有效记录: {self.valid_records} ({self.validity_rate:.1%}) + 重复记录: {self.duplicate_count} ({self.duplicate_rate:.1%}) + 缺失字段: {self.missing_fields} + 格式问题: {self.format_issues} + """.strip() +``` + +## 二、文本清洗 + +### 2.1 HTML 标签移除 + +```python +import re +from typing import Optional + +class HTMLCleaner: + """HTML 清洗器""" + + # 需要完全移除的标签（包括内容） + REMOVE_TAGS = ['script', 'style', 'head', 'meta', 'link'] + + @staticmethod + def remove_tags(html: str) -> str: + """移除所有 HTML 标签""" + # 先移除特定标签及其内容 + for tag in HTMLCleaner.REMOVE_TAGS: + pattern = f'<{tag}[^>]*>.*?' + html = re.sub(pattern, '', html, flags=re.DOTALL | re.IGNORECASE) + + # 移除所有标签 + html = re.sub(r'<[^>]+>', '', html) + + return html + + @staticmethod + def remove_tags_keep_text(html: str) -> str: + """移除标签但保留文本内容""" + # 处理常见的块级元素，添加换行 + html = re.sub(r'', '\n', html, flags=re.IGNORECASE) + # 移除其他标签 + html = re.sub(r'<[^>]+>', '', html) + return html + + @staticmethod + def decode_entities(text: str) -> str: + """解码 HTML 实体""" + import html + return html.unescape(text) + + +# 使用 BeautifulSoup（更可靠） +def clean_html_with_bs4(html: str) -> str: + """使用 BeautifulSoup 清洗 HTML""" + try: + from bs4 import BeautifulSoup + soup = BeautifulSoup(html, 'html.parser') + + # 移除脚本和样式 + for script in soup(['script', 'style']): + script.decompose() + + # 获取文本 + text = soup.get_text(separator='\n') + return text + except ImportError: + return HTMLCleaner.remove_tags(html) +``` + +### 2.2 空白字符处理 + +```python +class WhitespaceCleaner: + """空白字符清洗器""" + + @staticmethod + def normalize_whitespace(text: str) -> str: + """标准化空白字符""" + # 将所有空白字符转为普通空格 + text = re.sub(r'[\t\r\f\v]', ' ', text) + # 合并多个空格 + text = re.sub(r' +', ' ', text) + # 合并多个换行 + text = re.sub(r'\n+', '\n', text) + # 去除首尾空白 + return text.strip() + + @staticmethod + def remove_all_whitespace(text: str) -> str: + """移除所有空白字符""" + return re.sub(r'\s+', '', text) + + @staticmethod + def trim_lines(text: str) -> str: + """去除每行首尾空白""" + lines = text.split('\n') + return '\n'.join(line.strip() for line in lines) + + @staticmethod + def remove_empty_lines(text: str) -> str: + """移除空行""" + lines = text.split('\n') + return '\n'.join(line for line in lines if line.strip()) +``` + +### 2.3 特殊字符清理 + +```python +import unicodedata + +class SpecialCharCleaner: + """特殊字符清洗器""" + + @staticmethod + def remove_control_chars(text: str) -> str: + """移除控制字符""" + return ''.join( + char for char in text + if unicodedata.category(char) != 'Cc' + ) + + @staticmethod + def normalize_unicode(text: str, form: str = 'NFKC') -> str: + """ + Unicode 标准化 + + Args: + text: 输入文本 + form: 标准化形式 (NFC/NFD/NFKC/NFKD) + + Returns: + 标准化后的文本 + """ + return unicodedata.normalize(form, text) + + @staticmethod + def remove_emojis(text: str) -> str: + """移除 emoji""" + emoji_pattern = re.compile( + "[" + "\U0001F600-\U0001F64F" # 表情 + "\U0001F300-\U0001F5FF" # 符号和象形文字 + "\U0001F680-\U0001F6FF" # 交通和地图 + "\U0001F1E0-\U0001F1FF" # 旗帜 + "\U00002702-\U000027B0" # 装饰符号 + "\U000024C2-\U0001F251" # 封闭字符 + "]+", + flags=re.UNICODE + ) + return emoji_pattern.sub('', text) + + @staticmethod + def to_halfwidth(text: str) -> str: + """全角转半角""" + result = [] + for char in text: + code = ord(char) + # 全角空格 + if code == 0x3000: + result.append(' ') + # 其他全角字符 + elif 0xFF01 <= code <= 0xFF5E: + result.append(chr(code - 0xFEE0)) + else: + result.append(char) + return ''.join(result) +``` + +### 2.4 编码问题处理 + +```python +import chardet + +class EncodingFixer: + """编码问题修复器""" + + @staticmethod + def detect_encoding(data: bytes) -> str: + """检测编码""" + result = chardet.detect(data) + return result['encoding'] or 'utf-8' + + @staticmethod + def fix_encoding(text: str, source_encoding: str = None) -> str: + """修复编码问题""" + try: + # 如果是乱码，尝试重新编码 + if source_encoding: + return text.encode('latin1').decode(source_encoding) + else: + # 尝试常见编码 + for encoding in ['utf-8', 'gbk', 'gb2312', 'big5']: + try: + return text.encode('latin1').decode(encoding) + except (UnicodeDecodeError, UnicodeEncodeError): + continue + except Exception: + pass + return text + + @staticmethod + def safe_decode(data: bytes, fallback: str = 'utf-8') -> str: + """安全解码""" + # 检测编码 + detected = EncodingFixer.detect_encoding(data) + try: + return data.decode(detected) + except (UnicodeDecodeError, TypeError): + return data.decode(fallback, errors='ignore') +``` + +## 三、正则表达式高级应用 + +### 3.1 常用提取模式 + +```python +class RegexPatterns: + """常用正则表达式模式""" + + # 基础模式 + CHINESE = r'[\u4e00-\u9fa5]+' # 中文 + EMAIL = r'[\w.+-]+@[\w-]+\.[\w.-]+' # 邮箱 + PHONE = r'1[3-9]\d{9}' # 中国手机号 + URL = r'https?://[^\s<>"{}|\\^`\[\]]+' # URL + IP = r'\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}' # IP 地址 + + # 日期时间模式 + DATE_YMD = r'\d{4}[-/年]\d{1,2}[-/月]\d{1,2}日?' # 年月日 + TIME_HMS = r'\d{1,2}:\d{2}(:\d{2})?' # 时分秒 + DATETIME = r'\d{4}[-/]\d{1,2}[-/]\d{1,2}\s+\d{1,2}:\d{2}(:\d{2})?' + + # 数值模式 + INTEGER = r'-?\d+' + FLOAT = r'-?\d+\.?\d*' + PRICE = r'[¥$￥]\s*\d+\.?\d*' # 价格 + PERCENTAGE = r'\d+\.?\d*%' # 百分比 + + +class RegexExtractor: + """正则表达式提取器""" + + @staticmethod + def extract_all(text: str, pattern: str) -> list: + """提取所有匹配""" + return re.findall(pattern, text) + + @staticmethod + def extract_first(text: str, pattern: str) -> str: + """提取第一个匹配""" + match = re.search(pattern, text) + return match.group() if match else '' + + @staticmethod + def extract_groups(text: str, pattern: str) -> dict: + """提取命名分组""" + match = re.search(pattern, text) + return match.groupdict() if match else {} + + @staticmethod + def extract_between(text: str, start: str, end: str) -> list: + """提取两个标记之间的内容""" + pattern = f'{re.escape(start)}(.*?){re.escape(end)}' + return re.findall(pattern, text, re.DOTALL) +``` + +### 3.2 高级替换 + +```python +class RegexReplacer: + """正则表达式替换器""" + + @staticmethod + def replace_with_callback( + text: str, + pattern: str, + callback + ) -> str: + """使用回调函数替换""" + return re.sub(pattern, callback, text) + + @staticmethod + def mask_sensitive(text: str) -> str: + """脱敏处理""" + # 手机号脱敏 + text = re.sub( + r'(1[3-9]\d)\d{4}(\d{4})', + r'\1****\2', + text + ) + # 邮箱脱敏 + text = re.sub( + r'([\w.+-]{1,3})[\w.+-]*(@[\w-]+\.[\w.-]+)', + r'\1***\2', + text + ) + # 身份证脱敏 + text = re.sub( + r'(\d{6})\d{8}(\d{4})', + r'\1********\2', + text + ) + return text + + @staticmethod + def clean_url_params(url: str, keep_params: list = None) -> str: + """清理 URL 参数""" + from urllib.parse import urlparse, parse_qs, urlencode, urlunparse + + parsed = urlparse(url) + params = parse_qs(parsed.query) + + if keep_params: + params = {k: v for k, v in params.items() if k in keep_params} + else: + params = {} + + new_query = urlencode(params, doseq=True) + return urlunparse(parsed._replace(query=new_query)) +``` + +## 四、数据去重 + +### 4.1 精确去重 + +```python +from typing import List, Dict, Any, Callable +import hashlib + +class ExactDeduplicator: + """精确去重器""" + + @staticmethod + def dedupe_list(items: List[str]) -> List[str]: + """列表去重（保持顺序）""" + seen = set() + result = [] + for item in items: + if item not in seen: + seen.add(item) + result.append(item) + return result + + @staticmethod + def dedupe_dicts( + items: List[Dict], + key_field: str + ) -> List[Dict]: + """字典列表去重""" + seen = set() + result = [] + for item in items: + key = item.get(key_field) + if key not in seen: + seen.add(key) + result.append(item) + return result + + @staticmethod + def dedupe_by_hash( + items: List[Dict], + fields: List[str] + ) -> List[Dict]: + """ + 根据多个字段计算哈希去重 + + Args: + items: 数据列表 + fields: 用于计算哈希的字段 + + Returns: + 去重后的列表 + """ + seen = set() + result = [] + + for item in items: + # 计算哈希 + key_str = '|'.join(str(item.get(f, '')) for f in fields) + key_hash = hashlib.md5(key_str.encode()).hexdigest() + + if key_hash not in seen: + seen.add(key_hash) + result.append(item) + + return result +``` + +### 4.2 模糊去重 + +```python +class FuzzyDeduplicator: + """模糊去重器""" + + @staticmethod + def levenshtein_distance(s1: str, s2: str) -> int: + """计算编辑距离""" + if len(s1) < len(s2): + return FuzzyDeduplicator.levenshtein_distance(s2, s1) + + if len(s2) == 0: + return len(s1) + + previous_row = range(len(s2) + 1) + for i, c1 in enumerate(s1): + current_row = [i + 1] + for j, c2 in enumerate(s2): + insertions = previous_row[j + 1] + 1 + deletions = current_row[j] + 1 + substitutions = previous_row[j] + (c1 != c2) + current_row.append(min(insertions, deletions, substitutions)) + previous_row = current_row + + return previous_row[-1] + + @staticmethod + def similarity(s1: str, s2: str) -> float: + """计算相似度 (0-1)""" + if not s1 or not s2: + return 0.0 + distance = FuzzyDeduplicator.levenshtein_distance(s1, s2) + max_len = max(len(s1), len(s2)) + return 1 - distance / max_len + + @staticmethod + def dedupe_fuzzy( + items: List[str], + threshold: float = 0.8 + ) -> List[str]: + """ + 模糊去重 + + Args: + items: 字符串列表 + threshold: 相似度阈值 + + Returns: + 去重后的列表 + """ + if not items: + return [] + + result = [items[0]] + + for item in items[1:]: + is_duplicate = False + for existing in result: + if FuzzyDeduplicator.similarity(item, existing) >= threshold: + is_duplicate = True + break + if not is_duplicate: + result.append(item) + + return result +``` + +## 五、数据标准化 + +### 5.1 日期时间标准化 + +```python +from datetime import datetime +from typing import Optional + +class DateTimeNormalizer: + """日期时间标准化器""" + + # 常见日期格式 + DATE_FORMATS = [ + '%Y-%m-%d', + '%Y/%m/%d', + '%Y年%m月%d日', + '%Y.%m.%d', + '%d-%m-%Y', + '%m/%d/%Y', + ] + + # 常见日期时间格式 + DATETIME_FORMATS = [ + '%Y-%m-%d %H:%M:%S', + '%Y-%m-%d %H:%M', + '%Y/%m/%d %H:%M:%S', + '%Y年%m月%d日 %H:%M:%S', + '%Y年%m月%d日 %H时%M分', + ] + + @classmethod + def parse_date(cls, text: str) -> Optional[datetime]: + """解析日期""" + text = text.strip() + + for fmt in cls.DATE_FORMATS + cls.DATETIME_FORMATS: + try: + return datetime.strptime(text, fmt) + except ValueError: + continue + + return None + + @classmethod + def normalize_date( + cls, + text: str, + output_format: str = '%Y-%m-%d' + ) -> str: + """标准化日期格式""" + dt = cls.parse_date(text) + if dt: + return dt.strftime(output_format) + return text + + @staticmethod + def parse_relative_time(text: str) -> Optional[datetime]: + """解析相对时间（如"3小时前"）""" + import re + from datetime import timedelta + + now = datetime.now() + patterns = [ + (r'(\d+)\s*秒前', lambda m: now - timedelta(seconds=int(m.group(1)))), + (r'(\d+)\s*分钟前', lambda m: now - timedelta(minutes=int(m.group(1)))), + (r'(\d+)\s*小时前', lambda m: now - timedelta(hours=int(m.group(1)))), + (r'(\d+)\s*天前', lambda m: now - timedelta(days=int(m.group(1)))), + (r'刚刚', lambda m: now), + (r'昨天', lambda m: now - timedelta(days=1)), + (r'前天', lambda m: now - timedelta(days=2)), + ] + + for pattern, handler in patterns: + match = re.search(pattern, text) + if match: + return handler(match) + + return None +``` + +### 5.2 数值标准化 + +```python +class NumberNormalizer: + """数值标准化器""" + + @staticmethod + def parse_number(text: str) -> float: + """ + 解析数字（支持中文单位） + + Examples: + "1.5万" -> 15000 + "3.2亿" -> 320000000 + "1,234.56" -> 1234.56 + """ + text = text.strip() + + # 中文单位映射 + units = { + '万': 10000, + '亿': 100000000, + 'k': 1000, + 'K': 1000, + 'm': 1000000, + 'M': 1000000, + 'b': 1000000000, + 'B': 1000000000, + } + + multiplier = 1 + for unit, value in units.items(): + if unit in text: + multiplier = value + text = text.replace(unit, '') + break + + # 移除逗号 + text = text.replace(',', '') + + # 提取数字 + match = re.search(r'-?\d+\.?\d*', text) + if match: + return float(match.group()) * multiplier + + return 0.0 + + @staticmethod + def format_number( + value: float, + precision: int = 2, + use_units: bool = True + ) -> str: + """格式化数字""" + if not use_units: + return f'{value:.{precision}f}' + + if value >= 100000000: + return f'{value/100000000:.{precision}f}亿' + elif value >= 10000: + return f'{value/10000:.{precision}f}万' + else: + return f'{value:.{precision}f}' +``` + +### 5.3 文本标准化 + +```python +class TextNormalizer: + """文本标准化器""" + + @staticmethod + def normalize(text: str) -> str: + """完整的文本标准化流程""" + # 1. Unicode 标准化 + text = unicodedata.normalize('NFKC', text) + # 2. 全角转半角 + text = SpecialCharCleaner.to_halfwidth(text) + # 3. 移除控制字符 + text = SpecialCharCleaner.remove_control_chars(text) + # 4. 标准化空白 + text = WhitespaceCleaner.normalize_whitespace(text) + return text + + @staticmethod + def normalize_punctuation(text: str) -> str: + """标准化标点符号""" + # 中文标点转英文 + mapping = { + '，': ', ', + '。': '. ', + '！': '! ', + '？': '? ', + '；': '; ', + '：': ': ', + '"': '"', + '"': '"', + ''': "'", + ''': "'", + } + for cn, en in mapping.items(): + text = text.replace(cn, en) + return text +``` + +## 六、综合数据清洗器 + +```python +from dataclasses import dataclass, field +from typing import List, Dict, Any, Callable + +@dataclass +class CleaningConfig: + """清洗配置""" + remove_html: bool = True + normalize_whitespace: bool = True + normalize_unicode: bool = True + remove_emojis: bool = False + to_halfwidth: bool = True + +class DataCleaner: + """综合数据清洗器""" + + def __init__(self, config: CleaningConfig = None): + self.config = config or CleaningConfig() + + def clean_text(self, text: str) -> str: + """清洗文本""" + if not text: + return '' + + # HTML 清洗 + if self.config.remove_html: + text = HTMLCleaner.remove_tags(text) + text = HTMLCleaner.decode_entities(text) + + # Unicode 标准化 + if self.config.normalize_unicode: + text = SpecialCharCleaner.normalize_unicode(text) + + # 全角转半角 + if self.config.to_halfwidth: + text = SpecialCharCleaner.to_halfwidth(text) + + # 移除 emoji + if self.config.remove_emojis: + text = SpecialCharCleaner.remove_emojis(text) + + # 空白处理 + if self.config.normalize_whitespace: + text = WhitespaceCleaner.normalize_whitespace(text) + + return text + + def clean_dict(self, data: Dict, text_fields: List[str] = None) -> Dict: + """清洗字典中的文本字段""" + result = data.copy() + text_fields = text_fields or list(data.keys()) + + for field in text_fields: + if field in result and isinstance(result[field], str): + result[field] = self.clean_text(result[field]) + + return result + + def clean_list( + self, + items: List[Dict], + text_fields: List[str] = None + ) -> List[Dict]: + """清洗字典列表""" + return [self.clean_dict(item, text_fields) for item in items] +``` + +## 七、实战示例 + +```python +import asyncio + +async def demo_data_cleaning(): + """数据清洗演示""" + + # 模拟爬取的原始数据 + raw_data = [ + { + "title": "

Python　爬虫教程

\n\n", + "content": "

这是一篇&关于爬虫的教程。

", + "date": "2024年1月15日", + "price": "￥99.00", + "views": "1.5万" + }, + { + "title": "

Python　爬虫教程

", # 重复 + "content": "

这是另一篇关于爬虫的教程。

", + "date": "2024-01-15", + "price": "99元", + "views": "15000" + } + ] + + print("原始数据:") + for item in raw_data: + print(f" {item}") + + # 1. 文本清洗 + cleaner = DataCleaner(CleaningConfig( + remove_html=True, + normalize_whitespace=True, + to_halfwidth=True + )) + + cleaned_data = cleaner.clean_list(raw_data, ["title", "content"]) + + print("\n清洗后数据:") + for item in cleaned_data: + print(f" {item}") + + # 2. 日期标准化 + for item in cleaned_data: + item["date"] = DateTimeNormalizer.normalize_date(item["date"]) + + print("\n日期标准化后:") + for item in cleaned_data: + print(f" date: {item['date']}") + + # 3. 数值标准化 + for item in cleaned_data: + item["views_num"] = NumberNormalizer.parse_number(item["views"]) + + print("\n数值标准化后:") + for item in cleaned_data: + print(f" views: {item['views']} -> {item['views_num']}") + + # 4. 去重 + deduped = ExactDeduplicator.dedupe_by_hash(cleaned_data, ["title"]) + print(f"\n去重后记录数: {len(deduped)}") + + +if __name__ == "__main__": + asyncio.run(demo_data_cleaning()) +``` + +## 八、数据清洗实战 + +本节以网页数据为例，演示完整的数据清洗流程。 + +### 8.1 数据特点分析 + +```mermaid +flowchart TD + subgraph 原始数据问题 + title_em["标题含HTML标签
(搜索高亮)"] + view_unit["数值含单位
(1.5万、3.2亿)"] + duration_format["时长格式不一
(02:30、1:23:45)"] + pubdate_format["发布时间多样
(刚刚、3天前、2024-01-15)"] + desc_html["描述含HTML
(换行、链接)"] + end + + subgraph 清洗目标 + title_clean[纯文本标题] + view_num[数值] + duration_sec[秒数时长] + pubdate_std[标准日期] + desc_plain[纯文本描述] + end + + title_em --> title_clean + view_unit --> view_num + duration_format --> duration_sec + pubdate_format --> pubdate_std + desc_html --> desc_plain +``` + +### 8.2 数据清洗器 + +```python +import re +from dataclasses import dataclass +from datetime import datetime, timedelta +from typing import Optional, List, Dict, Any +from loguru import logger + + +@dataclass +class WebContent: + """网页内容数据模型（清洗后）""" + bvid: str # 视频BV号 + title: str # 标题（纯文本） + description: str # 简介（纯文本） + owner_name: str # UP主名称 + owner_mid: int # UP主ID + view_count: int # 播放量 + like_count: int # 点赞数 + coin_count: int # 投币数 + favorite_count: int # 收藏数 + share_count: int # 分享数 + danmaku_count: int # 弹幕数 + comment_count: int # 评论数 + duration_seconds: int # 时长（秒） + publish_time: datetime # 发布时间 + tags: List[str] # 标签列表 + + +class DataCleaner: + """网页数据清洗器""" + + @staticmethod + def clean_title(title: str) -> str: + """ + 清洗视频标题 + + 处理： + - 移除搜索高亮标签 ... + - 移除其他HTML标签 + - 标准化空白字符 + """ + if not title: + return "" + + # 移除 标签但保留内容 + title = re.sub(r']*>([^<]*)', r'\1', title) + # 移除其他HTML标签 + title = re.sub(r'<[^>]+>', '', title) + # HTML实体解码 + import html + title = html.unescape(title) + # 标准化空白 + title = re.sub(r'\s+', ' ', title).strip() + + return title + + @staticmethod + def clean_description(desc: str) -> str: + """ + 清洗视频简介 + + 处理： + - 移除HTML标签 + - 保留换行结构 + - 移除过多空行 + """ + if not desc: + return "" + + # 移除HTML标签 + desc = re.sub(r'<[^>]+>', '', desc) + # HTML实体解码 + import html + desc = html.unescape(desc) + # 合并多个换行 + desc = re.sub(r'\n{3,}', '\n\n', desc) + # 去除首尾空白 + desc = desc.strip() + + return desc + + @staticmethod + def parse_view_count(view_str: str) -> int: + """ + 解析播放量 + + 支持格式： + - "15000" -> 15000 + - "1.5万" -> 15000 + - "3.2亿" -> 320000000 + - "1,234,567" -> 1234567 + """ + if not view_str: + return 0 + + view_str = str(view_str).strip() + + # 移除逗号 + view_str = view_str.replace(',', '') + + # 处理中文单位 + if '亿' in view_str: + num = float(view_str.replace('亿', '')) + return int(num * 100000000) + elif '万' in view_str: + num = float(view_str.replace('万', '')) + return int(num * 10000) + else: + # 尝试直接转换 + try: + return int(float(view_str)) + except ValueError: + return 0 + + @staticmethod + def parse_duration(duration_str: str) -> int: + """ + 解析视频时长为秒数 + + 支持格式： + - "02:30" -> 150 + - "1:23:45" -> 5025 + - 150 (已是秒数) -> 150 + """ + if not duration_str: + return 0 + + # 如果已经是数字，直接返回 + if isinstance(duration_str, (int, float)): + return int(duration_str) + + duration_str = str(duration_str).strip() + + # 尝试直接转换（API返回的可能已是秒数） + try: + return int(duration_str) + except ValueError: + pass + + # 解析时:分:秒格式 + parts = duration_str.split(':') + try: + if len(parts) == 2: + # MM:SS + minutes, seconds = int(parts[0]), int(parts[1]) + return minutes * 60 + seconds + elif len(parts) == 3: + # HH:MM:SS + hours, minutes, seconds = int(parts[0]), int(parts[1]), int(parts[2]) + return hours * 3600 + minutes * 60 + seconds + except ValueError: + pass + + return 0 + + @staticmethod + def parse_publish_time(pubdate: Any) -> Optional[datetime]: + """ + 解析发布时间 + + 支持格式： + - Unix时间戳 (int) + - "刚刚", "3分钟前", "2小时前", "3天前" + - "2024-01-15" + - "2024-01-15 10:30:00" + - "2024年1月15日" + """ + if not pubdate: + return None + + # Unix时间戳 + if isinstance(pubdate, (int, float)): + if pubdate > 1000000000000: # 毫秒 + pubdate = pubdate / 1000 + return datetime.fromtimestamp(pubdate) + + pubdate_str = str(pubdate).strip() + now = datetime.now() + + # 相对时间 + relative_patterns = [ + (r'刚刚', lambda m: now), + (r'(\d+)\s*秒前', lambda m: now - timedelta(seconds=int(m.group(1)))), + (r'(\d+)\s*分钟前', lambda m: now - timedelta(minutes=int(m.group(1)))), + (r'(\d+)\s*小时前', lambda m: now - timedelta(hours=int(m.group(1)))), + (r'(\d+)\s*天前', lambda m: now - timedelta(days=int(m.group(1)))), + (r'昨天', lambda m: now - timedelta(days=1)), + (r'前天', lambda m: now - timedelta(days=2)), + ] + + for pattern, handler in relative_patterns: + match = re.search(pattern, pubdate_str) + if match: + return handler(match) + + # 绝对时间格式 + date_formats = [ + '%Y-%m-%d %H:%M:%S', + '%Y-%m-%d %H:%M', + '%Y-%m-%d', + '%Y/%m/%d %H:%M:%S', + '%Y/%m/%d', + '%Y年%m月%d日 %H:%M', + '%Y年%m月%d日', + ] + + for fmt in date_formats: + try: + return datetime.strptime(pubdate_str, fmt) + except ValueError: + continue + + return None + + @classmethod + def clean_video_data(cls, raw_data: Dict[str, Any]) -> Optional[WebContent]: + """ + 清洗单条视频数据 + + Args: + raw_data: API返回的原始数据 + + Returns: + 清洗后的视频数据对象 + """ + try: + # 提取并清洗各字段 + return WebContent( + bvid=raw_data.get('bvid', ''), + title=cls.clean_title(raw_data.get('title', '')), + description=cls.clean_description(raw_data.get('desc', '')), + owner_name=raw_data.get('owner', {}).get('name', ''), + owner_mid=raw_data.get('owner', {}).get('mid', 0), + view_count=cls.parse_view_count( + raw_data.get('stat', {}).get('view', 0) + ), + like_count=raw_data.get('stat', {}).get('like', 0), + coin_count=raw_data.get('stat', {}).get('coin', 0), + favorite_count=raw_data.get('stat', {}).get('favorite', 0), + share_count=raw_data.get('stat', {}).get('share', 0), + danmaku_count=raw_data.get('stat', {}).get('danmaku', 0), + comment_count=raw_data.get('stat', {}).get('reply', 0), + duration_seconds=cls.parse_duration(raw_data.get('duration', 0)), + publish_time=cls.parse_publish_time(raw_data.get('pubdate', 0)), + tags=raw_data.get('tags', []) if isinstance(raw_data.get('tags'), list) else [], + ) + except Exception as e: + logger.error(f"清洗视频数据失败: {e}") + return None + + @classmethod + def clean_video_list( + cls, + raw_list: List[Dict[str, Any]] + ) -> List[WebContent]: + """ + 清洗视频列表数据 + + Args: + raw_list: 原始数据列表 + + Returns: + 清洗后的视频列表 + """ + results = [] + for raw_data in raw_list: + video = cls.clean_video_data(raw_data) + if video: + results.append(video) + return results + + +async def data_cleaning_demo(): + """数据清洗演示""" + + # 模拟API返回的原始数据 + raw_videos = [ + { + "bvid": "BV1xx411c7mD", + "title": "Python爬虫教程 - 从入门到精通", + "desc": "本视频介绍Python爬虫的基础知识。\n\n包含以下内容：\n1. 环境搭建\n2. 请求发送\n3. 数据解析", + "owner": {"name": "技术UP主", "mid": 12345678}, + "stat": { + "view": "15.6万", + "like": 8500, + "coin": 3200, + "favorite": 12000, + "share": 450, + "danmaku": 2300, + "reply": 680 + }, + "duration": "15:30", + "pubdate": 1705286400 # Unix时间戳 + }, + { + "bvid": "BV1yy411c8nM", + "title": "数据分析实战 - Python项目", + "desc": "使用Python分析热门视频数据。", + "owner": {"name": "数据分析师", "mid": 87654321}, + "stat": { + "view": "3.2万", + "like": 2100, + "coin": 890, + "favorite": 4500, + "share": 180, + "danmaku": 560, + "reply": 230 + }, + "duration": "1:05:20", + "pubdate": "2024-01-10" + } + ] + + print("原始数据:") + for video in raw_videos: + print(f" 标题: {video['title']}") + print(f" 播放: {video['stat']['view']}") + print() + + # 清洗数据 + cleaner = DataCleaner() + cleaned_videos = cleaner.clean_video_list(raw_videos) + + print("清洗后数据:") + for video in cleaned_videos: + print(f" BV号: {video.bvid}") + print(f" 标题: {video.title}") + print(f" 播放量: {video.view_count:,}") + print(f" 时长: {video.duration_seconds}秒") + print(f" 发布时间: {video.publish_time}") + print() + + +if __name__ == "__main__": + import asyncio + asyncio.run(data_cleaning_demo()) +``` + +### 8.3 数据去重 + +```python +import hashlib +from typing import List, Set + + +class DataDeduplicator: + """数据去重器""" + + def __init__(self): + self._seen_bvids: Set[str] = set() + self._seen_hashes: Set[str] = set() + + def is_duplicate_by_bvid(self, bvid: str) -> bool: + """通过BV号判断是否重复""" + if bvid in self._seen_bvids: + return True + self._seen_bvids.add(bvid) + return False + + def is_duplicate_by_content(self, title: str, owner_mid: int) -> bool: + """ + 通过内容哈希判断是否重复 + （用于检测同一UP主发布的相似标题视频） + """ + content = f"{title}|{owner_mid}" + content_hash = hashlib.md5(content.encode()).hexdigest() + + if content_hash in self._seen_hashes: + return True + self._seen_hashes.add(content_hash) + return False + + def dedupe_videos( + self, + videos: List[WebContent] + ) -> List[WebContent]: + """ + 去重视频列表 + + 优先使用BV号去重，同时检测内容重复 + """ + results = [] + + for video in videos: + # BV号去重 + if self.is_duplicate_by_bvid(video.bvid): + continue + + # 内容去重（可选） + if self.is_duplicate_by_content(video.title, video.owner_mid): + continue + + results.append(video) + + return results + + def reset(self): + """重置去重状态""" + self._seen_bvids.clear() + self._seen_hashes.clear() +``` + +### 8.4 数据质量报告 + +```python +from dataclasses import dataclass +from typing import List + + +@dataclass +class DataQualityReport: + """数据质量报告""" + total_count: int # 总记录数 + valid_count: int # 有效记录数 + duplicate_count: int # 重复记录数 + missing_title: int # 缺失标题数 + missing_owner: int # 缺失UP主数 + zero_views: int # 零播放数 + invalid_duration: int # 无效时长数 + invalid_pubdate: int # 无效发布时间数 + + @property + def valid_rate(self) -> float: + """有效率""" + return self.valid_count / self.total_count if self.total_count > 0 else 0 + + def __str__(self) -> str: + return f""" +数据质量报告: + 总记录数: {self.total_count} + 有效记录数: {self.valid_count} ({self.valid_rate:.1%}) + 重复记录: {self.duplicate_count} + 缺失标题: {self.missing_title} + 缺失UP主: {self.missing_owner} + 零播放: {self.zero_views} + 无效时长: {self.invalid_duration} + 无效发布时间: {self.invalid_pubdate} + """.strip() + + +def generate_quality_report(videos: List[WebContent]) -> DataQualityReport: + """生成数据质量报告""" + missing_title = sum(1 for v in videos if not v.title) + missing_owner = sum(1 for v in videos if not v.owner_name) + zero_views = sum(1 for v in videos if v.view_count == 0) + invalid_duration = sum(1 for v in videos if v.duration_seconds <= 0) + invalid_pubdate = sum(1 for v in videos if v.publish_time is None) + + valid_count = sum( + 1 for v in videos + if v.title and v.owner_name and v.view_count > 0 + and v.duration_seconds > 0 and v.publish_time + ) + + return DataQualityReport( + total_count=len(videos), + valid_count=valid_count, + duplicate_count=0, # 去重后为0 + missing_title=missing_title, + missing_owner=missing_owner, + zero_views=zero_views, + invalid_duration=invalid_duration, + invalid_pubdate=invalid_pubdate, + ) +``` + +--- + +## 本章小结 + +本章介绍了数据清洗与预处理的核心技术： + +1. **文本清洗**：HTML 移除、空白处理、特殊字符、编码修复 +2. **正则表达式**：常用模式、提取与替换、脱敏处理 +3. **数据去重**：精确去重和模糊去重 +4. **数据标准化**：日期、数值、文本的统一格式 +5. **综合实战**：视频数据清洗、播放量解析、时长转换、发布时间标准化 + +数据清洗是数据处理流程中的关键环节，干净的数据才能产生有价值的分析结果。 + +## 下一章预告 + +下一章我们将学习**数据分析与可视化**，包括使用 pandas 进行数据统计、生成词云、绑制图表等技术。这些技术可以帮助我们从爬取的数据中提取有价值的洞察。 diff --git "a/docs/\347\210\254\350\231\253\350\277\233\344\273\267/10_\346\225\260\346\215\256\345\210\206\346\236\220\344\270\216\345\217\257\350\247\206\345\214\226.md" "b/docs/\347\210\254\350\231\253\350\277\233\344\273\267/10_\346\225\260\346\215\256\345\210\206\346\236\220\344\270\216\345\217\257\350\247\206\345\214\226.md" new file mode 100644 index 0000000..cd7c314 --- /dev/null +++ "b/docs/\347\210\254\350\231\253\350\277\233\344\273\267/10_\346\225\260\346\215\256\345\210\206\346\236\220\344\270\216\345\217\257\350\247\206\345\214\226.md" @@ -0,0 +1,1585 @@ +# 第十章：数据分析与可视化 + +> 本章将学习如何使用 Python 数据分析和可视化工具处理爬取的数据。我们将掌握 pandas 进行数据统计分析、使用词云展示文本数据、利用 matplotlib 和 pyecharts 生成各类图表，最终实现自动化的数据分析报告生成。 + +## 10.1 数据分析概述 + +### 为什么需要数据分析 + +爬虫获取的原始数据只是第一步，真正的价值在于从数据中提取有用的信息和洞察： + +- **发现数据规律**：了解数据的分布、趋势和特征 +- **验证假设**：用数据验证业务假设和决策 +- **支持决策**：为业务决策提供数据依据 +- **数据可视化**：将复杂数据转化为直观的图表 + +### 数据分析流程 + +```mermaid +flowchart LR + subgraph 数据准备 + A[原始数据] --> B[数据清洗] + B --> C[数据转换] + end + + subgraph 数据分析 + C --> D[统计分析] + C --> E[文本分析] + C --> F[时间序列] + end + + subgraph 结果输出 + D --> G[图表可视化] + E --> H[词云生成] + F --> G + G --> I[分析报告] + H --> I + end +``` + +### 主要工具介绍 + +| 工具 | 用途 | 特点 | +|------|------|------| +| pandas | 数据处理和分析 | 功能强大，生态完善 | +| jieba | 中文分词 | 准确率高，易于使用 | +| wordcloud | 词云生成 | 可定制性强 | +| matplotlib | 静态图表 | 功能全面，输出清晰 | +| pyecharts | 交互式图表 | 效果炫酷，支持 Web | + +## 10.2 pandas 数据分析 + +### DataFrame 基础操作 + +pandas 是 Python 数据分析的核心库，DataFrame 是其最重要的数据结构： + +```python +import pandas as pd + +# 从爬取的数据创建 DataFrame +data = [ + {"title": "Python 教程", "views": 15000, "likes": 320, "date": "2024-01-15"}, + {"title": "爬虫入门", "views": 12000, "likes": 280, "date": "2024-01-16"}, + {"title": "数据分析", "views": 18000, "likes": 450, "date": "2024-01-17"}, +] +df = pd.DataFrame(data) + +# 基础信息 +print(df.info()) # 列类型和非空数量 +print(df.describe()) # 数值列统计摘要 + +# 数据选择 +print(df['title']) # 选择单列 +print(df[['title', 'views']]) # 选择多列 +print(df[df['views'] > 14000]) # 条件筛选 + +# 排序 +print(df.sort_values('views', ascending=False)) + +# 新增列 +df['engagement'] = df['likes'] / df['views'] * 100 +``` + +### 数据聚合与分组统计 + +分组统计是数据分析中最常用的操作： + +```python +# 假设有更多数据 +data = [ + {"category": "技术", "title": "Python 教程", "views": 15000}, + {"category": "技术", "title": "爬虫入门", "views": 12000}, + {"category": "生活", "title": "美食推荐", "views": 8000}, + {"category": "生活", "title": "旅行日记", "views": 10000}, + {"category": "技术", "title": "数据分析", "views": 18000}, +] +df = pd.DataFrame(data) + +# 按分类分组统计 +grouped = df.groupby('category') +print(grouped['views'].sum()) # 各分类总浏览量 +print(grouped['views'].mean()) # 各分类平均浏览量 +print(grouped['views'].agg(['sum', 'mean', 'max', 'min'])) # 多个统计量 + +# 多列聚合 +print(df.groupby('category').agg({ + 'views': 'sum', + 'title': 'count' +}).rename(columns={'title': 'article_count'})) +``` + +### 数据透视表 + +数据透视表可以快速进行多维度分析： + +```python +# 更复杂的数据 +data = [ + {"date": "2024-01", "category": "技术", "platform": "PC", "views": 15000}, + {"date": "2024-01", "category": "技术", "platform": "Mobile", "views": 8000}, + {"date": "2024-01", "category": "生活", "platform": "PC", "views": 6000}, + {"date": "2024-02", "category": "技术", "platform": "PC", "views": 18000}, + {"date": "2024-02", "category": "生活", "platform": "Mobile", "views": 9000}, +] +df = pd.DataFrame(data) + +# 创建透视表：按日期和分类统计各平台浏览量 +pivot = pd.pivot_table( + df, + values='views', + index='date', + columns='category', + aggfunc='sum', + fill_value=0 +) +print(pivot) + +# 带边距的透视表 +pivot_with_margins = pd.pivot_table( + df, + values='views', + index='date', + columns='category', + aggfunc='sum', + margins=True, + margins_name='总计' +) +print(pivot_with_margins) +``` + +### 时间序列分析 + +爬虫数据通常包含时间维度，时间序列分析很有价值： + +```python +# 创建时间序列数据 +dates = pd.date_range('2024-01-01', periods=30, freq='D') +views = [1000 + i * 50 + (i % 7) * 200 for i in range(30)] +df = pd.DataFrame({'date': dates, 'views': views}) +df.set_index('date', inplace=True) + +# 滚动统计（7天移动平均） +df['rolling_avg'] = df['views'].rolling(window=7).mean() + +# 按周统计 +weekly = df.resample('W').agg({ + 'views': ['sum', 'mean'] +}) +print(weekly) + +# 环比增长率 +df['growth_rate'] = df['views'].pct_change() * 100 +``` + +## 10.3 词云生成 + +词云是展示文本数据的直观方式，在分析评论、标题等文本数据时非常有用。 + +### jieba 中文分词 + +jieba 是最流行的中文分词库： + +```python +import jieba +import jieba.analyse + +# 基础分词 +text = "Python爬虫教程帮助你快速入门数据采集技术" +words = jieba.lcut(text) +print(words) # ['Python', '爬虫', '教程', '帮助', '你', '快速', '入门', '数据', '采集', '技术'] + +# 提取关键词（TF-IDF） +keywords = jieba.analyse.extract_tags( + text, + topK=5, + withWeight=True +) +for word, weight in keywords: + print(f"{word}: {weight:.4f}") + +# 提取关键词（TextRank） +keywords_tr = jieba.analyse.textrank( + text, + topK=5, + withWeight=True +) +``` + +### 停用词过滤 + +停用词是指对文本分析没有实际意义的词（如"的"、"是"、"在"等）： + +```python +# 常用中文停用词 +STOPWORDS = { + '的', '是', '在', '了', '和', '与', '或', '有', '个', '人', + '这', '那', '就', '都', '也', '为', '对', '到', '从', '把', + '被', '让', '给', '向', '往', '于', '及', '以', '等', '不', + '很', '会', '能', '可', '要', '我', '你', '他', '她', '它', +} + +def filter_stopwords(words: list) -> list: + """过滤停用词和单字""" + return [ + w for w in words + if w not in STOPWORDS and len(w) > 1 + ] + +# 使用 +text = "这是一个关于Python爬虫的教程，帮助你快速入门" +words = jieba.lcut(text) +filtered = filter_stopwords(words) +print(filtered) # ['Python', '爬虫', '教程', '帮助', '快速', '入门'] +``` + +### 词云生成 + +使用 wordcloud 库生成词云： + +```python +from wordcloud import WordCloud +import matplotlib.pyplot as plt + +def generate_wordcloud( + text: str, + output_path: str = "wordcloud.png", + width: int = 800, + height: int = 600, + background_color: str = "white", + font_path: str = None +): + """ + 生成词云图片 + + Args: + text: 空格分隔的词语文本 + output_path: 输出图片路径 + width: 图片宽度 + height: 图片高度 + background_color: 背景颜色 + font_path: 中文字体路径（必须指定才能显示中文） + """ + # 创建词云对象 + wc = WordCloud( + width=width, + height=height, + background_color=background_color, + font_path=font_path, # 中文需要指定字体 + max_words=200, + max_font_size=100, + random_state=42 + ) + + # 生成词云 + wc.generate(text) + + # 保存图片 + wc.to_file(output_path) + + # 显示词云 + plt.figure(figsize=(10, 8)) + plt.imshow(wc, interpolation='bilinear') + plt.axis('off') + plt.tight_layout() + plt.savefig(output_path.replace('.png', '_display.png'), dpi=150) + plt.close() + + return output_path + + +# 完整流程：从文本到词云 +def text_to_wordcloud(texts: list, output_path: str, font_path: str = None): + """从文本列表生成词云""" + # 1. 分词 + all_words = [] + for text in texts: + words = jieba.lcut(text) + all_words.extend(words) + + # 2. 过滤停用词 + filtered_words = filter_stopwords(all_words) + + # 3. 统计词频 + word_freq = {} + for word in filtered_words: + word_freq[word] = word_freq.get(word, 0) + 1 + + # 4. 生成词云文本 + word_text = ' '.join(filtered_words) + + # 5. 生成词云 + return generate_wordcloud(word_text, output_path, font_path=font_path) +``` + +### 自定义词云形状 + +可以使用图片作为词云的形状蒙版： + +```python +import numpy as np +from PIL import Image +from wordcloud import WordCloud, ImageColorGenerator + +def generate_shaped_wordcloud( + text: str, + mask_image_path: str, + output_path: str, + font_path: str = None, + use_mask_colors: bool = True +): + """ + 生成自定义形状的词云 + + Args: + text: 词语文本 + mask_image_path: 形状蒙版图片路径 + output_path: 输出路径 + font_path: 字体路径 + use_mask_colors: 是否使用蒙版图片的颜色 + """ + # 读取蒙版图片 + mask = np.array(Image.open(mask_image_path)) + + # 创建词云 + wc = WordCloud( + mask=mask, + background_color="white", + font_path=font_path, + max_words=500, + max_font_size=80, + random_state=42, + contour_width=1, + contour_color='steelblue' + ) + + wc.generate(text) + + # 使用蒙版颜色 + if use_mask_colors: + image_colors = ImageColorGenerator(mask) + wc.recolor(color_func=image_colors) + + wc.to_file(output_path) + return output_path +``` + +## 10.4 数据可视化 + +### matplotlib 基础图表 + +matplotlib 是 Python 最基础的可视化库： + +```python +import matplotlib.pyplot as plt +import matplotlib +matplotlib.rcParams['font.sans-serif'] = ['SimHei', 'Arial Unicode MS'] +matplotlib.rcParams['axes.unicode_minus'] = False + +def plot_line_chart( + x_data: list, + y_data: list, + title: str, + xlabel: str, + ylabel: str, + output_path: str +): + """绘制折线图""" + plt.figure(figsize=(10, 6)) + plt.plot(x_data, y_data, marker='o', linewidth=2, markersize=6) + plt.title(title, fontsize=14) + plt.xlabel(xlabel, fontsize=12) + plt.ylabel(ylabel, fontsize=12) + plt.grid(True, alpha=0.3) + plt.tight_layout() + plt.savefig(output_path, dpi=150) + plt.close() + + +def plot_bar_chart( + categories: list, + values: list, + title: str, + xlabel: str, + ylabel: str, + output_path: str, + horizontal: bool = False +): + """绘制柱状图""" + plt.figure(figsize=(10, 6)) + + if horizontal: + plt.barh(categories, values, color='steelblue') + plt.xlabel(ylabel) + plt.ylabel(xlabel) + else: + plt.bar(categories, values, color='steelblue') + plt.xlabel(xlabel) + plt.ylabel(ylabel) + + plt.title(title, fontsize=14) + plt.tight_layout() + plt.savefig(output_path, dpi=150) + plt.close() + + +def plot_pie_chart( + labels: list, + sizes: list, + title: str, + output_path: str +): + """绘制饼图""" + plt.figure(figsize=(10, 8)) + + # 突出最大的一块 + max_idx = sizes.index(max(sizes)) + explode = [0.05 if i == max_idx else 0 for i in range(len(sizes))] + + plt.pie( + sizes, + labels=labels, + explode=explode, + autopct='%1.1f%%', + startangle=90, + colors=plt.cm.Set3.colors[:len(labels)] + ) + plt.title(title, fontsize=14) + plt.axis('equal') + plt.tight_layout() + plt.savefig(output_path, dpi=150) + plt.close() + + +def plot_multi_line_chart( + x_data: list, + y_data_dict: dict, + title: str, + xlabel: str, + ylabel: str, + output_path: str +): + """绘制多条折线图""" + plt.figure(figsize=(12, 6)) + + for label, y_data in y_data_dict.items(): + plt.plot(x_data, y_data, marker='o', label=label, linewidth=2) + + plt.title(title, fontsize=14) + plt.xlabel(xlabel, fontsize=12) + plt.ylabel(ylabel, fontsize=12) + plt.legend(loc='best') + plt.grid(True, alpha=0.3) + plt.tight_layout() + plt.savefig(output_path, dpi=150) + plt.close() +``` + +### pyecharts 交互式图表 + +pyecharts 可以生成交互式的 HTML 图表： + +```python +from pyecharts.charts import Bar, Line, Pie, WordCloud as PyechartsWordCloud +from pyecharts import options as opts +from pyecharts.globals import ThemeType + +def create_bar_chart( + categories: list, + values: list, + title: str, + output_path: str +): + """创建交互式柱状图""" + bar = ( + Bar(init_opts=opts.InitOpts(theme=ThemeType.LIGHT)) + .add_xaxis(categories) + .add_yaxis("数量", values) + .set_global_opts( + title_opts=opts.TitleOpts(title=title), + toolbox_opts=opts.ToolboxOpts(is_show=True), + datazoom_opts=opts.DataZoomOpts(is_show=True) + ) + ) + bar.render(output_path) + return output_path + + +def create_line_chart( + x_data: list, + y_data_dict: dict, + title: str, + output_path: str +): + """创建交互式折线图""" + line = Line(init_opts=opts.InitOpts(theme=ThemeType.LIGHT)) + line.add_xaxis(x_data) + + for name, y_data in y_data_dict.items(): + line.add_yaxis( + name, + y_data, + is_smooth=True, + markpoint_opts=opts.MarkPointOpts( + data=[ + opts.MarkPointItem(type_="max", name="最大值"), + opts.MarkPointItem(type_="min", name="最小值"), + ] + ), + markline_opts=opts.MarkLineOpts( + data=[opts.MarkLineItem(type_="average", name="平均值")] + ) + ) + + line.set_global_opts( + title_opts=opts.TitleOpts(title=title), + tooltip_opts=opts.TooltipOpts(trigger="axis"), + toolbox_opts=opts.ToolboxOpts(is_show=True), + legend_opts=opts.LegendOpts(is_show=True) + ) + line.render(output_path) + return output_path + + +def create_pie_chart( + data: list, + title: str, + output_path: str +): + """ + 创建交互式饼图 + + Args: + data: [(name, value), ...] 格式的数据 + title: 图表标题 + output_path: 输出路径 + """ + pie = ( + Pie(init_opts=opts.InitOpts(theme=ThemeType.LIGHT)) + .add( + "", + data, + radius=["30%", "70%"], + rosetype="radius" + ) + .set_global_opts( + title_opts=opts.TitleOpts(title=title), + legend_opts=opts.LegendOpts( + orient="vertical", + pos_top="15%", + pos_left="2%" + ) + ) + .set_series_opts( + label_opts=opts.LabelOpts(formatter="{b}: {d}%") + ) + ) + pie.render(output_path) + return output_path + + +def create_wordcloud_chart( + words: list, + title: str, + output_path: str +): + """ + 创建交互式词云 + + Args: + words: [(word, count), ...] 格式的数据 + title: 图表标题 + output_path: 输出路径 + """ + wc = ( + PyechartsWordCloud(init_opts=opts.InitOpts(theme=ThemeType.LIGHT)) + .add( + "", + words, + word_size_range=[20, 100], + shape="circle" + ) + .set_global_opts( + title_opts=opts.TitleOpts(title=title), + toolbox_opts=opts.ToolboxOpts(is_show=True) + ) + ) + wc.render(output_path) + return output_path +``` + +## 10.5 数据分析报告生成 + +### Markdown 报告模板 + +自动生成 Markdown 格式的分析报告： + +```python +from datetime import datetime +from typing import Dict, List, Any + +class ReportGenerator: + """数据分析报告生成器""" + + def __init__(self, title: str, author: str = "数据分析团队"): + self.title = title + self.author = author + self.sections = [] + self.images = [] + + def add_section(self, title: str, content: str): + """添加章节""" + self.sections.append({ + "title": title, + "content": content + }) + + def add_image(self, path: str, caption: str): + """添加图片""" + self.images.append({ + "path": path, + "caption": caption + }) + + def add_table(self, headers: List[str], rows: List[List[Any]], title: str = ""): + """添加表格""" + content = "" + if title: + content += f"**{title}**\n\n" + + # 表头 + content += "| " + " | ".join(headers) + " |\n" + content += "| " + " | ".join(["---"] * len(headers)) + " |\n" + + # 数据行 + for row in rows: + content += "| " + " | ".join(str(cell) for cell in row) + " |\n" + + self.sections.append({ + "title": "", + "content": content + }) + + def add_summary(self, metrics: Dict[str, Any]): + """添加数据摘要""" + content = "| 指标 | 数值 |\n| --- | --- |\n" + for key, value in metrics.items(): + content += f"| {key} | {value} |\n" + + self.sections.append({ + "title": "数据概览", + "content": content + }) + + def generate(self, output_path: str) -> str: + """生成报告""" + report = [] + + # 标题 + report.append(f"# {self.title}") + report.append("") + + # 元信息 + report.append(f"> 生成时间：{datetime.now().strftime('%Y-%m-%d %H:%M:%S')}") + report.append(f"> 作者：{self.author}") + report.append("") + report.append("---") + report.append("") + + # 目录 + report.append("## 目录") + report.append("") + for i, section in enumerate(self.sections, 1): + if section["title"]: + report.append(f"{i}. [{section['title']}](#{section['title'].lower().replace(' ', '-')})") + report.append("") + report.append("---") + report.append("") + + # 内容 + for section in self.sections: + if section["title"]: + report.append(f"## {section['title']}") + report.append("") + report.append(section["content"]) + report.append("") + + # 图表 + if self.images: + report.append("## 图表") + report.append("") + for img in self.images: + report.append(f"### {img['caption']}") + report.append("") + report.append(f"![{img['caption']}]({img['path']})") + report.append("") + + # 写入文件 + content = "\n".join(report) + with open(output_path, 'w', encoding='utf-8') as f: + f.write(content) + + return output_path +``` + +### 完整分析流程示例 + +```python +import pandas as pd +from typing import List, Dict + +class DataAnalyzer: + """数据分析器""" + + def __init__(self, data: List[Dict]): + self.df = pd.DataFrame(data) + self.report = ReportGenerator("数据分析报告") + + def basic_statistics(self) -> Dict: + """基础统计""" + stats = { + "总记录数": len(self.df), + "时间范围": f"{self.df['date'].min()} ~ {self.df['date'].max()}" if 'date' in self.df else "N/A", + } + + # 数值列统计 + numeric_cols = self.df.select_dtypes(include=['number']).columns + for col in numeric_cols: + stats[f"{col}_总计"] = self.df[col].sum() + stats[f"{col}_平均"] = round(self.df[col].mean(), 2) + stats[f"{col}_最大"] = self.df[col].max() + stats[f"{col}_最小"] = self.df[col].min() + + return stats + + def category_analysis(self, category_col: str, value_col: str) -> pd.DataFrame: + """分类分析""" + return self.df.groupby(category_col)[value_col].agg([ + 'count', 'sum', 'mean', 'max', 'min' + ]).round(2) + + def time_series_analysis(self, date_col: str, value_col: str, freq: str = 'D') -> pd.DataFrame: + """时间序列分析""" + df_ts = self.df.copy() + df_ts[date_col] = pd.to_datetime(df_ts[date_col]) + df_ts.set_index(date_col, inplace=True) + + return df_ts.resample(freq)[value_col].agg(['sum', 'mean', 'count']).round(2) + + def generate_report(self, output_dir: str) -> str: + """生成完整分析报告""" + import os + os.makedirs(output_dir, exist_ok=True) + + # 1. 基础统计 + stats = self.basic_statistics() + self.report.add_summary(stats) + + # 2. 添加描述性文字 + self.report.add_section( + "分析说明", + f"本报告对 {len(self.df)} 条数据进行了分析，" + f"包含 {len(self.df.columns)} 个字段。" + ) + + # 3. 生成图表（如果有相应字段） + numeric_cols = self.df.select_dtypes(include=['number']).columns.tolist() + if numeric_cols: + # 柱状图 + col = numeric_cols[0] + if len(self.df) <= 20: + bar_path = os.path.join(output_dir, "bar_chart.png") + plot_bar_chart( + self.df.index.tolist() if self.df.index.dtype != 'int64' else list(range(len(self.df))), + self.df[col].tolist(), + f"{col} 分布", + "索引", + col, + bar_path + ) + self.report.add_image(bar_path, f"{col} 分布柱状图") + + # 4. 生成报告 + report_path = os.path.join(output_dir, "report.md") + return self.report.generate(report_path) +``` + +## 10.6 实战案例：社交媒体评论分析 + +让我们综合运用本章所学，对爬取的社交媒体评论数据进行完整分析： + +```python +""" +社交媒体评论数据分析 +""" +import os +import json +from datetime import datetime +from typing import List, Dict +from collections import Counter + +import pandas as pd +import jieba +from wordcloud import WordCloud +import matplotlib.pyplot as plt +import matplotlib +matplotlib.rcParams['font.sans-serif'] = ['SimHei', 'Arial Unicode MS'] +matplotlib.rcParams['axes.unicode_minus'] = False + + +class CommentAnalyzer: + """评论数据分析器""" + + STOPWORDS = { + '的', '是', '在', '了', '和', '与', '或', '有', '个', '人', + '这', '那', '就', '都', '也', '为', '对', '到', '从', '把', + '被', '让', '给', '向', '往', '于', '及', '以', '等', '不', + '很', '会', '能', '可', '要', '我', '你', '他', '她', '它', + '啊', '吧', '呢', '呀', '哦', '嗯', '哈', '嘿', + } + + def __init__(self, comments: List[Dict], output_dir: str = "./analysis_output"): + """ + Args: + comments: 评论数据列表，每条评论包含 content, user, time, likes 等字段 + output_dir: 输出目录 + """ + self.comments = comments + self.df = pd.DataFrame(comments) + self.output_dir = output_dir + os.makedirs(output_dir, exist_ok=True) + + def analyze_basic_stats(self) -> Dict: + """基础统计分析""" + stats = { + "total_comments": len(self.df), + "unique_users": self.df['user'].nunique() if 'user' in self.df else 0, + "total_likes": self.df['likes'].sum() if 'likes' in self.df else 0, + "avg_likes": round(self.df['likes'].mean(), 2) if 'likes' in self.df else 0, + "avg_length": round(self.df['content'].str.len().mean(), 2), + } + + # 时间分析 + if 'time' in self.df: + self.df['time'] = pd.to_datetime(self.df['time'], errors='coerce') + valid_times = self.df['time'].dropna() + if len(valid_times) > 0: + stats["time_range"] = f"{valid_times.min()} ~ {valid_times.max()}" + + return stats + + def analyze_word_frequency(self, top_n: int = 50) -> List[tuple]: + """词频分析""" + all_words = [] + + for content in self.df['content']: + words = jieba.lcut(str(content)) + # 过滤停用词和单字 + words = [ + w for w in words + if w not in self.STOPWORDS and len(w) > 1 + ] + all_words.extend(words) + + word_freq = Counter(all_words) + return word_freq.most_common(top_n) + + def generate_wordcloud(self, font_path: str = None) -> str: + """生成词云""" + word_freq = self.analyze_word_frequency(200) + + # 转换为空格分隔的文本 + text = ' '.join([word for word, _ in word_freq for _ in range(word_freq[0][1])]) + + # 直接使用词频 + freq_dict = dict(word_freq) + + wc = WordCloud( + width=1200, + height=800, + background_color='white', + font_path=font_path, + max_words=200, + max_font_size=150, + random_state=42 + ) + + wc.generate_from_frequencies(freq_dict) + + output_path = os.path.join(self.output_dir, "wordcloud.png") + wc.to_file(output_path) + + return output_path + + def analyze_time_distribution(self) -> str: + """时间分布分析""" + if 'time' not in self.df: + return None + + df = self.df.copy() + df['time'] = pd.to_datetime(df['time'], errors='coerce') + df = df.dropna(subset=['time']) + + if len(df) == 0: + return None + + # 按小时统计 + df['hour'] = df['time'].dt.hour + hour_dist = df.groupby('hour').size() + + # 绘图 + plt.figure(figsize=(12, 6)) + plt.bar(hour_dist.index, hour_dist.values, color='steelblue') + plt.title('评论时间分布（按小时）', fontsize=14) + plt.xlabel('小时', fontsize=12) + plt.ylabel('评论数', fontsize=12) + plt.xticks(range(0, 24)) + plt.grid(axis='y', alpha=0.3) + plt.tight_layout() + + output_path = os.path.join(self.output_dir, "time_distribution.png") + plt.savefig(output_path, dpi=150) + plt.close() + + return output_path + + def analyze_user_activity(self, top_n: int = 10) -> str: + """用户活跃度分析""" + if 'user' not in self.df: + return None + + user_counts = self.df['user'].value_counts().head(top_n) + + plt.figure(figsize=(12, 6)) + plt.barh(range(len(user_counts)), user_counts.values, color='steelblue') + plt.yticks(range(len(user_counts)), user_counts.index) + plt.title(f'评论最多的 {top_n} 位用户', fontsize=14) + plt.xlabel('评论数', fontsize=12) + plt.ylabel('用户', fontsize=12) + plt.gca().invert_yaxis() + plt.tight_layout() + + output_path = os.path.join(self.output_dir, "user_activity.png") + plt.savefig(output_path, dpi=150) + plt.close() + + return output_path + + def generate_report(self, font_path: str = None) -> str: + """生成完整分析报告""" + report_lines = [] + + # 标题 + report_lines.append("# 社交媒体评论分析报告") + report_lines.append("") + report_lines.append(f"> 生成时间：{datetime.now().strftime('%Y-%m-%d %H:%M:%S')}") + report_lines.append("") + report_lines.append("---") + report_lines.append("") + + # 基础统计 + stats = self.analyze_basic_stats() + report_lines.append("## 1. 基础统计") + report_lines.append("") + report_lines.append("| 指标 | 数值 |") + report_lines.append("| --- | --- |") + report_lines.append(f"| 评论总数 | {stats['total_comments']} |") + report_lines.append(f"| 独立用户数 | {stats['unique_users']} |") + report_lines.append(f"| 点赞总数 | {stats['total_likes']} |") + report_lines.append(f"| 平均点赞数 | {stats['avg_likes']} |") + report_lines.append(f"| 平均评论长度 | {stats['avg_length']} 字 |") + if 'time_range' in stats: + report_lines.append(f"| 时间范围 | {stats['time_range']} |") + report_lines.append("") + + # 词频分析 + report_lines.append("## 2. 热门词汇 TOP 20") + report_lines.append("") + word_freq = self.analyze_word_frequency(20) + report_lines.append("| 排名 | 词汇 | 出现次数 |") + report_lines.append("| --- | --- | --- |") + for i, (word, count) in enumerate(word_freq, 1): + report_lines.append(f"| {i} | {word} | {count} |") + report_lines.append("") + + # 生成图表 + report_lines.append("## 3. 可视化图表") + report_lines.append("") + + # 词云 + wordcloud_path = self.generate_wordcloud(font_path) + report_lines.append("### 3.1 词云图") + report_lines.append("") + report_lines.append(f"![词云图]({os.path.basename(wordcloud_path)})") + report_lines.append("") + + # 时间分布 + time_path = self.analyze_time_distribution() + if time_path: + report_lines.append("### 3.2 评论时间分布") + report_lines.append("") + report_lines.append(f"![时间分布]({os.path.basename(time_path)})") + report_lines.append("") + + # 用户活跃度 + user_path = self.analyze_user_activity() + if user_path: + report_lines.append("### 3.3 用户活跃度") + report_lines.append("") + report_lines.append(f"![用户活跃度]({os.path.basename(user_path)})") + report_lines.append("") + + # 写入报告 + report_content = "\n".join(report_lines) + report_path = os.path.join(self.output_dir, "analysis_report.md") + with open(report_path, 'w', encoding='utf-8') as f: + f.write(report_content) + + print(f"分析报告已生成: {report_path}") + return report_path + + +def demo(): + """演示分析功能""" + # 模拟评论数据 + comments = [ + {"content": "Python真的很好学，推荐这个教程！", "user": "user1", "time": "2024-01-15 10:30:00", "likes": 15}, + {"content": "爬虫入门很有用，学到了很多知识", "user": "user2", "time": "2024-01-15 11:20:00", "likes": 8}, + {"content": "数据分析这章讲得很详细，点赞", "user": "user3", "time": "2024-01-15 14:45:00", "likes": 12}, + {"content": "希望能出更多进阶内容，继续学习Python", "user": "user1", "time": "2024-01-15 16:00:00", "likes": 5}, + {"content": "词云效果很酷，学会了！", "user": "user4", "time": "2024-01-15 18:30:00", "likes": 20}, + {"content": "教程质量很高，适合入门学习", "user": "user5", "time": "2024-01-15 20:15:00", "likes": 10}, + {"content": "Python数据分析真的很强大", "user": "user6", "time": "2024-01-16 09:00:00", "likes": 7}, + {"content": "爬虫技术学起来很有趣", "user": "user7", "time": "2024-01-16 10:30:00", "likes": 9}, + ] + + # 创建分析器 + analyzer = CommentAnalyzer(comments, "./demo_output") + + # 生成报告 + report_path = analyzer.generate_report() + print(f"报告已保存到: {report_path}") + + +if __name__ == "__main__": + demo() +``` + +## 10.7 视频数据分析实战 + +本节以视频数据为例，演示完整的数据分析流程。 + +### 数据分析目标 + +```mermaid +flowchart TD + subgraph 输入数据 + videos[视频数据] + comments[弹幕/评论数据] + end + + subgraph 分析维度 + stat[播放量/互动统计] + up[UP主分析] + time[发布时间分析] + keyword[标题关键词] + end + + subgraph 输出 + rank[热门视频排行] + trend[发布趋势图] + cloud[标题词云] + report[分析报告] + end + + videos --> stat --> rank + videos --> up --> rank + videos --> time --> trend + videos --> keyword --> cloud + rank --> report + trend --> report + cloud --> report +``` + +### 视频数据分析器 + +```python +import os +import pandas as pd +import jieba +from collections import Counter +from datetime import datetime +from typing import List, Dict, Any, Optional +from dataclasses import dataclass +from wordcloud import WordCloud +import matplotlib.pyplot as plt +import matplotlib +matplotlib.rcParams['font.sans-serif'] = ['SimHei', 'Arial Unicode MS', 'PingFang SC'] +matplotlib.rcParams['axes.unicode_minus'] = False + + +@dataclass +class VideoData: + """视频数据模型""" + bvid: str + title: str + owner_name: str + owner_mid: int + view_count: int + like_count: int + coin_count: int + favorite_count: int + share_count: int + danmaku_count: int + comment_count: int + duration_seconds: int + publish_time: datetime + tags: List[str] + + +class VideoAnalyzer: + """视频数据分析器""" + + # 常用停用词 + STOPWORDS = { + '的', '是', '在', '了', '和', '与', '或', '有', '个', '人', + '这', '那', '就', '都', '也', '为', '对', '到', '从', '把', + '被', '让', '给', '向', '往', '于', '及', '以', '等', '不', + '很', '会', '能', '可', '要', '我', '你', '他', '她', '它', + '视频', '合集', '第一', '第二', '第三', '更新', '最新', + } + + def __init__(self, videos: List[VideoData], output_dir: str = "./video_analysis"): + """ + Args: + videos: 视频数据列表 + output_dir: 输出目录 + """ + self.videos = videos + self.output_dir = output_dir + os.makedirs(output_dir, exist_ok=True) + + # 转换为DataFrame便于分析 + self.df = pd.DataFrame([ + { + 'bvid': v.bvid, + 'title': v.title, + 'owner_name': v.owner_name, + 'owner_mid': v.owner_mid, + 'view_count': v.view_count, + 'like_count': v.like_count, + 'coin_count': v.coin_count, + 'favorite_count': v.favorite_count, + 'share_count': v.share_count, + 'danmaku_count': v.danmaku_count, + 'comment_count': v.comment_count, + 'duration_seconds': v.duration_seconds, + 'publish_time': v.publish_time, + } + for v in videos + ]) + + def basic_statistics(self) -> Dict[str, Any]: + """基础统计分析""" + stats = { + "总视频数": len(self.df), + "独立UP主数": self.df['owner_mid'].nunique(), + "总播放量": f"{self.df['view_count'].sum():,}", + "平均播放量": f"{self.df['view_count'].mean():,.0f}", + "最高播放量": f"{self.df['view_count'].max():,}", + "总点赞数": f"{self.df['like_count'].sum():,}", + "总投币数": f"{self.df['coin_count'].sum():,}", + "总收藏数": f"{self.df['favorite_count'].sum():,}", + "平均时长": f"{self.df['duration_seconds'].mean() / 60:.1f} 分钟", + } + + # 计算互动率 + if self.df['view_count'].sum() > 0: + engagement_rate = ( + self.df['like_count'].sum() + + self.df['coin_count'].sum() + + self.df['favorite_count'].sum() + ) / self.df['view_count'].sum() * 100 + stats["平均互动率"] = f"{engagement_rate:.2f}%" + + return stats + + def top_videos_by_views(self, top_n: int = 10) -> pd.DataFrame: + """播放量TOP N视频""" + return self.df.nlargest(top_n, 'view_count')[ + ['title', 'owner_name', 'view_count', 'like_count', 'publish_time'] + ] + + def top_uploaders(self, top_n: int = 10) -> pd.DataFrame: + """最活跃UP主（视频数量）""" + uploader_stats = self.df.groupby(['owner_mid', 'owner_name']).agg({ + 'bvid': 'count', + 'view_count': 'sum', + 'like_count': 'sum' + }).reset_index() + + uploader_stats.columns = ['owner_mid', 'owner_name', 'video_count', 'total_views', 'total_likes'] + return uploader_stats.nlargest(top_n, 'video_count') + + def analyze_publish_time(self) -> Dict[str, pd.Series]: + """发布时间分析""" + df = self.df.copy() + df['publish_time'] = pd.to_datetime(df['publish_time']) + + # 按小时分布 + df['hour'] = df['publish_time'].dt.hour + hourly = df.groupby('hour')['bvid'].count() + + # 按星期分布 + df['weekday'] = df['publish_time'].dt.dayofweek + weekday_names = ['周一', '周二', '周三', '周四', '周五', '周六', '周日'] + daily = df.groupby('weekday')['bvid'].count() + daily.index = [weekday_names[i] for i in daily.index] + + # 按日期趋势 + df['date'] = df['publish_time'].dt.date + date_trend = df.groupby('date')['bvid'].count() + + return { + 'hourly': hourly, + 'daily': daily, + 'date_trend': date_trend + } + + def analyze_title_keywords(self, top_n: int = 50) -> List[tuple]: + """标题关键词分析""" + all_words = [] + + for title in self.df['title']: + words = jieba.lcut(str(title)) + words = [ + w.strip() for w in words + if w.strip() and w not in self.STOPWORDS and len(w) > 1 + ] + all_words.extend(words) + + return Counter(all_words).most_common(top_n) + + def generate_views_distribution_chart(self) -> str: + """生成播放量分布图""" + fig, axes = plt.subplots(1, 2, figsize=(14, 5)) + + # 播放量直方图 + axes[0].hist(self.df['view_count'], bins=30, color='steelblue', edgecolor='white') + axes[0].set_title('播放量分布', fontsize=14) + axes[0].set_xlabel('播放量') + axes[0].set_ylabel('视频数') + + # 播放量箱线图（对数尺度） + import numpy as np + log_views = np.log10(self.df['view_count'] + 1) + axes[1].boxplot(log_views) + axes[1].set_title('播放量分布（对数）', fontsize=14) + axes[1].set_ylabel('log10(播放量)') + + plt.tight_layout() + output_path = os.path.join(self.output_dir, 'views_distribution.png') + plt.savefig(output_path, dpi=150) + plt.close() + + return output_path + + def generate_publish_time_chart(self) -> str: + """生成发布时间分析图""" + time_data = self.analyze_publish_time() + + fig, axes = plt.subplots(2, 2, figsize=(14, 10)) + + # 小时分布 + axes[0, 0].bar(time_data['hourly'].index, time_data['hourly'].values, color='steelblue') + axes[0, 0].set_title('发布时间分布（按小时）', fontsize=12) + axes[0, 0].set_xlabel('小时') + axes[0, 0].set_ylabel('视频数') + axes[0, 0].set_xticks(range(0, 24, 2)) + + # 星期分布 + axes[0, 1].bar(time_data['daily'].index, time_data['daily'].values, color='coral') + axes[0, 1].set_title('发布时间分布（按星期）', fontsize=12) + axes[0, 1].set_xlabel('星期') + axes[0, 1].set_ylabel('视频数') + + # 日期趋势 + axes[1, 0].plot(time_data['date_trend'].index, time_data['date_trend'].values, + marker='o', markersize=3, linewidth=1, color='green') + axes[1, 0].set_title('发布趋势', fontsize=12) + axes[1, 0].set_xlabel('日期') + axes[1, 0].set_ylabel('视频数') + axes[1, 0].tick_params(axis='x', rotation=45) + + # 互动数据对比 + interaction_data = { + '点赞': self.df['like_count'].sum(), + '投币': self.df['coin_count'].sum(), + '收藏': self.df['favorite_count'].sum(), + '分享': self.df['share_count'].sum(), + } + axes[1, 1].bar(interaction_data.keys(), interaction_data.values(), color='purple') + axes[1, 1].set_title('互动数据汇总', fontsize=12) + axes[1, 1].set_ylabel('总数') + + plt.tight_layout() + output_path = os.path.join(self.output_dir, 'publish_time_analysis.png') + plt.savefig(output_path, dpi=150) + plt.close() + + return output_path + + def generate_title_wordcloud(self, font_path: Optional[str] = None) -> str: + """生成标题词云""" + keywords = self.analyze_title_keywords(200) + freq_dict = dict(keywords) + + wc = WordCloud( + width=1200, + height=800, + background_color='white', + font_path=font_path, + max_words=200, + max_font_size=150, + random_state=42, + colormap='viridis' + ) + + wc.generate_from_frequencies(freq_dict) + + output_path = os.path.join(self.output_dir, 'title_wordcloud.png') + wc.to_file(output_path) + + return output_path + + def generate_up_ranking_chart(self, top_n: int = 15) -> str: + """生成UP主排行图""" + top_ups = self.top_uploaders(top_n) + + fig, axes = plt.subplots(1, 2, figsize=(14, 6)) + + # 视频数量排行 + axes[0].barh(range(len(top_ups)), top_ups['video_count'].values, color='steelblue') + axes[0].set_yticks(range(len(top_ups))) + axes[0].set_yticklabels(top_ups['owner_name'].values) + axes[0].set_title(f'视频数量 TOP {top_n} UP主', fontsize=12) + axes[0].set_xlabel('视频数') + axes[0].invert_yaxis() + + # 总播放量排行 + top_by_views = self.df.groupby(['owner_mid', 'owner_name'])['view_count'].sum().reset_index() + top_by_views = top_by_views.nlargest(top_n, 'view_count') + + axes[1].barh(range(len(top_by_views)), top_by_views['view_count'].values, color='coral') + axes[1].set_yticks(range(len(top_by_views))) + axes[1].set_yticklabels(top_by_views['owner_name'].values) + axes[1].set_title(f'总播放量 TOP {top_n} UP主', fontsize=12) + axes[1].set_xlabel('总播放量') + axes[1].invert_yaxis() + + plt.tight_layout() + output_path = os.path.join(self.output_dir, 'up_ranking.png') + plt.savefig(output_path, dpi=150) + plt.close() + + return output_path + + def generate_report(self, font_path: Optional[str] = None) -> str: + """生成完整分析报告""" + report_lines = [] + + # 标题 + report_lines.append("# 视频数据分析报告") + report_lines.append("") + report_lines.append(f"> 生成时间：{datetime.now().strftime('%Y-%m-%d %H:%M:%S')}") + report_lines.append(f"> 数据量：{len(self.videos)} 条视频") + report_lines.append("") + report_lines.append("---") + report_lines.append("") + + # 基础统计 + stats = self.basic_statistics() + report_lines.append("## 1. 基础统计") + report_lines.append("") + report_lines.append("| 指标 | 数值 |") + report_lines.append("| --- | --- |") + for key, value in stats.items(): + report_lines.append(f"| {key} | {value} |") + report_lines.append("") + + # 热门视频排行 + report_lines.append("## 2. 播放量 TOP 10 视频") + report_lines.append("") + top_videos = self.top_videos_by_views(10) + report_lines.append("| 排名 | 标题 | UP主 | 播放量 | 点赞数 |") + report_lines.append("| --- | --- | --- | --- | --- |") + for i, (_, row) in enumerate(top_videos.iterrows(), 1): + title = row['title'][:30] + '...' if len(row['title']) > 30 else row['title'] + report_lines.append( + f"| {i} | {title} | {row['owner_name']} | " + f"{row['view_count']:,} | {row['like_count']:,} |" + ) + report_lines.append("") + + # 关键词排行 + report_lines.append("## 3. 标题热门关键词 TOP 20") + report_lines.append("") + keywords = self.analyze_title_keywords(20) + report_lines.append("| 排名 | 关键词 | 出现次数 |") + report_lines.append("| --- | --- | --- |") + for i, (word, count) in enumerate(keywords, 1): + report_lines.append(f"| {i} | {word} | {count} |") + report_lines.append("") + + # 生成图表 + report_lines.append("## 4. 数据可视化") + report_lines.append("") + + # 播放量分布 + views_chart = self.generate_views_distribution_chart() + report_lines.append("### 4.1 播放量分布") + report_lines.append(f"![播放量分布]({os.path.basename(views_chart)})") + report_lines.append("") + + # 发布时间分析 + time_chart = self.generate_publish_time_chart() + report_lines.append("### 4.2 发布时间分析") + report_lines.append(f"![发布时间分析]({os.path.basename(time_chart)})") + report_lines.append("") + + # UP主排行 + up_chart = self.generate_up_ranking_chart() + report_lines.append("### 4.3 UP主排行") + report_lines.append(f"![UP主排行]({os.path.basename(up_chart)})") + report_lines.append("") + + # 标题词云 + wordcloud_path = self.generate_title_wordcloud(font_path) + report_lines.append("### 4.4 标题词云") + report_lines.append(f"![标题词云]({os.path.basename(wordcloud_path)})") + report_lines.append("") + + # 写入报告 + report_content = "\n".join(report_lines) + report_path = os.path.join(self.output_dir, "video_analysis_report.md") + with open(report_path, 'w', encoding='utf-8') as f: + f.write(report_content) + + print(f"数据分析报告已生成: {report_path}") + return report_path + + +async def video_analysis_demo(): + """视频数据分析演示""" + from datetime import timedelta + import random + + # 模拟视频数据 + sample_titles = [ + "Python爬虫从入门到精通", + "数据分析实战教程", + "机器学习入门指南", + "数据可视化技巧分享", + "Web开发最佳实践", + "深度学习PyTorch教程", + "数据清洗与预处理", + "API接口设计规范", + ] + + sample_ups = [ + ("技术UP主A", 12345678), + ("数据分析师B", 23456789), + ("Python教学C", 34567890), + ("编程达人D", 45678901), + ] + + videos = [] + base_time = datetime.now() - timedelta(days=30) + + for i in range(50): + up_name, up_mid = random.choice(sample_ups) + video = VideoData( + bvid=f"BV1{''.join(random.choices('abcdefghijklmnopqrstuvwxyz0123456789', k=10))}", + title=f"{random.choice(sample_titles)} 第{i+1}集", + owner_name=up_name, + owner_mid=up_mid, + view_count=random.randint(1000, 500000), + like_count=random.randint(100, 20000), + coin_count=random.randint(50, 5000), + favorite_count=random.randint(100, 10000), + share_count=random.randint(10, 1000), + danmaku_count=random.randint(50, 5000), + comment_count=random.randint(20, 2000), + duration_seconds=random.randint(60, 3600), + publish_time=base_time + timedelta( + days=random.randint(0, 30), + hours=random.randint(0, 23) + ), + tags=[] + ) + videos.append(video) + + # 创建分析器并生成报告 + analyzer = VideoAnalyzer(videos, "./video_demo_output") + report_path = analyzer.generate_report() + + print(f"演示报告已生成: {report_path}") + + +if __name__ == "__main__": + import asyncio + asyncio.run(video_analysis_demo()) +``` + +--- + +## 本章小结 + +本章我们学习了数据分析与可视化的核心技术： + +1. **pandas 数据分析** + - DataFrame 是数据分析的核心数据结构 + - groupby 实现分组聚合统计 + - pivot_table 进行多维分析 + - 时间序列支持滚动统计和重采样 + +2. **词云生成** + - jieba 实现中文分词 + - 停用词过滤提升分析质量 + - wordcloud 库生成静态词云 + - 支持自定义形状和颜色 + +3. **数据可视化** + - matplotlib 生成静态图表 + - pyecharts 生成交互式图表 + - 选择合适的图表类型展示数据 + +4. **自动化报告** + - Markdown 格式便于阅读和分享 + - 模板化生成提高效率 + - 图表嵌入增强可读性 + +5. **综合实战** + - 视频数据多维统计分析 + - 发布时间规律分析 + - UP主活跃度排名 + - 标题关键词词云 + +**关键要点：** +- 数据分析的目标是发现洞察，而非展示技术 +- 选择合适的可视化方式，让数据"说话" +- 自动化报告生成可以大大提高效率 +- 中文分析需要注意字体和编码问题 + +--- + +## 下一章预告 + +在最后一章「进阶综合实战项目」中，我们将综合运用整个进阶教程所学的所有技术，实现一个完整的社交媒体数据采集与分析工具。该项目将包含： + +- 多种登录方式支持 +- 反检测浏览器自动化 +- 代理 IP 轮换 +- 数据清洗和存储 +- 词云和图表分析 + +敬请期待！ diff --git "a/docs/\347\210\254\350\231\253\350\277\233\344\273\267/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256.md" "b/docs/\347\210\254\350\231\253\350\277\233\344\273\267/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256.md" new file mode 100644 index 0000000..07ec32e --- /dev/null +++ "b/docs/\347\210\254\350\231\253\350\277\233\344\273\267/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256.md" @@ -0,0 +1,2075 @@ +# 第十一章：进阶综合实战项目 + +> 本章将综合运用整个进阶教程所学的所有技术，以 **B站（bilibili.com）** 为目标，实现一个完整的视频数据采集与分析工具。该项目将包含登录认证、API 签名、浏览器自动化、数据存储、分析报告等完整功能链路。 + +## 11.1 项目概述 + +### 项目目标 + +构建一个类似 [MediaCrawler](https://github.com/NanmiCoder/MediaCrawler) 简化版的视频数据采集工具，以 **B站** 为目标平台，具备以下能力： + +- **登录认证**：支持扫码登录和 Cookie 登录 +- **API 签名**：实现 B站 WBI 签名算法 +- **视频搜索**：按关键词搜索视频 +- **视频详情**：获取完整视频信息（播放量、点赞、收藏等） +- **数据存储**：支持 JSON、CSV 两种存储方式 +- **数据分析**：自动生成词云和统计报告 + +> **目标网站说明：** +> - 网站：https://www.bilibili.com +> - 类型：国内最大的视频社区平台 +> - 特点：需要登录获取完整数据，API 有 WBI 签名保护 +> - 数据：视频标题、UP主、播放量、点赞、收藏、弹幕数等 + +### 整体架构图 + +在开始编码之前，让我们先从宏观角度理解整个项目的架构： + +```mermaid +graph TB + subgraph 入口层 + main["main.py
程序入口"] + end + + subgraph 核心模块 + config["config 配置
settings.py
bilibili_config.py"] + crawler["crawler 爬虫
spider.py
核心调度"] + store["store 存储
backend.py
JSON/CSV"] + end + + subgraph 功能模块 + login["login 登录
auth.py
扫码/Cookie"] + client["client 客户端
bilibili_client.py
API请求"] + analysis["analysis 分析
report.py
词云/统计"] + end + + subgraph 基础模块 + core["core 浏览器
browser.py
Playwright封装"] + tools["tools 工具
sign.py
WBI签名"] + models["models 模型
bilibili.py
数据结构"] + end + + main --> config + main --> crawler + main --> store + main --> analysis + + crawler --> login + crawler --> client + + login --> core + client --> tools + tools --> models + + config -.->|配置注入| crawler + crawler -->|数据| store +``` + +### 爬虫执行流程 + +整个爬虫从启动到完成的完整流程如下： + +```mermaid +flowchart TD + start([程序启动 main.py]) --> step1 + + subgraph step1 [步骤1: 初始化浏览器] + browser["BrowserManager.start()
启动 Playwright
创建 BrowserContext"] + end + + step1 --> step2 + + subgraph step2 [步骤2: 登录认证] + check{检查登录状态} + check -->|已登录| skip[跳过登录] + check -->|未登录| login_flow + subgraph login_flow [执行登录] + qrcode["扫码登录: 显示二维码-等待扫码-获取Cookie"] + cookie["Cookie登录: 注入Cookie-验证有效性"] + end + end + + step2 --> step3 + + subgraph step3 [步骤3: 初始化API客户端] + init["同步Cookie到httpx
获取WBI签名密钥
初始化签名器"] + end + + step3 --> step4 + + subgraph step4 [步骤4: 执行爬取任务] + mode{爬取模式} + mode -->|SEARCH| search["关键词搜索-翻页获取-获取详情"] + mode -->|DETAIL| detail[直接获取指定视频详情] + search --> request + detail --> request + request["每次请求:
构造参数-WBI签名-发送请求-解析响应-随机延迟"] + end + + step4 --> step5 + + subgraph step5 [步骤5: 数据存储] + save["BilibiliVideo对象列表
转换为字典
保存为 JSON/CSV"] + end + + step5 --> step6 + + subgraph step6 [步骤6: 生成分析报告] + report["统计分析-词频统计
生成词云-输出Markdown报告"] + end + + step6 --> step7 + + subgraph step7 [步骤7: 清理资源] + cleanup["关闭浏览器
保存登录状态
程序退出"] + end + + step7 --> finish([完成]) +``` + +### 数据流向图 + +理解数据在各模块之间如何流转： + +```mermaid +flowchart LR + subgraph input [用户输入] + keyword["关键词
配置文件"] + end + + subgraph process [系统处理] + search["搜索API
(WBI签名)"] + detail["详情API
(获取详情)"] + validate["Pydantic数据验证
BilibiliVideo"] + end + + subgraph output [最终输出] + json["JSON文件
(结构化)"] + csv["CSV文件
(表格化)"] + report["分析报告
(Markdown)"] + wordcloud["词云图片
(PNG)"] + end + + keyword --> search + search -->|视频列表| detail + detail --> validate + validate --> json + validate --> csv + validate --> report + report --> wordcloud +``` + +### 参考项目 + +本项目参考 [MediaCrawler](https://github.com/NanmiCoder/MediaCrawler) 的 B站实现： + +| 文件 | 说明 | +|------|------| +| `media_platform/bilibili/core.py` | 爬虫核心逻辑 | +| `media_platform/bilibili/client.py` | API 客户端 | +| `media_platform/bilibili/login.py` | 登录认证 | +| `media_platform/bilibili/help.py` | WBI 签名算法 | + +### 技术栈 + +| 模块 | 技术选型 | 作用 | +|------|----------|------| +| 配置管理 | pydantic-settings | 类型安全的配置，支持环境变量 | +| 日志系统 | loguru | 优雅的日志记录和轮转 | +| 浏览器自动化 | Playwright | 处理登录、获取Cookie和签名密钥 | +| HTTP 客户端 | httpx | 异步HTTP请求，高性能 | +| 数据验证 | Pydantic | 数据模型定义和验证 | +| 数据分析 | pandas + jieba + wordcloud | 统计分析和可视化 | + +### 项目结构 + +``` +11_进阶综合实战项目/ +├── config/ # 配置模块 +│ ├── __init__.py +│ ├── settings.py # 通用配置 +│ └── bilibili_config.py # B站特定配置 +├── core/ # 核心模块 +│ ├── __init__.py +│ └── browser.py # 浏览器管理 +├── login/ # 登录模块 +│ ├── __init__.py +│ └── auth.py # B站登录认证 +├── client/ # API 客户端模块 +│ ├── __init__.py +│ └── bilibili_client.py # B站 API 客户端 +├── crawler/ # 爬虫模块 +│ ├── __init__.py +│ └── spider.py # B站爬虫实现 +├── store/ # 存储模块 +│ ├── __init__.py +│ └── backend.py # 存储后端 +├── proxy/ # 代理模块（可选） +│ ├── __init__.py +│ └── pool.py # 代理池 +├── models/ # 数据模型模块 +│ ├── __init__.py +│ └── bilibili.py # B站数据模型 +├── tools/ # 工具模块 +│ ├── __init__.py +│ └── sign.py # WBI 签名工具 +├── analysis/ # 分析模块 +│ ├── __init__.py +│ └── report.py # 报告生成 +└── main.py # 入口文件 +``` + +## 11.2 配置模块设计 + +### 通用配置 + +使用 pydantic-settings 实现类型安全的配置管理： + +```python +# config/settings.py +from pydantic_settings import BaseSettings +from pydantic import Field +from typing import Optional, List +from enum import Enum + + +class StorageType(str, Enum): + """存储类型""" + JSON = "json" + CSV = "csv" + + +class LoginType(str, Enum): + """登录类型""" + COOKIE = "cookie" + QRCODE = "qrcode" + + +class CrawlerType(str, Enum): + """爬取类型""" + SEARCH = "search" # 关键词搜索 + DETAIL = "detail" # 指定视频详情 + + +class Settings(BaseSettings): + """项目配置""" + + # 基础配置 + app_name: str = "BilibiliCrawler" + debug: bool = False + + # 浏览器配置 + browser_headless: bool = False # B站扫码登录需要显示浏览器 + browser_timeout: int = 30000 + browser_user_data_dir: Optional[str] = "./browser_data" + save_login_state: bool = True + + # 登录配置 + login_type: LoginType = LoginType.QRCODE + cookie_str: str = "" + + # 爬虫配置 + crawler_type: CrawlerType = CrawlerType.SEARCH + keywords: str = "Python教程" # 搜索关键词，多个用逗号分隔 + specified_id_list: List[str] = [] # 指定视频列表 + max_video_count: int = 20 + max_concurrency: int = 3 + crawl_delay_min: float = 1.0 + crawl_delay_max: float = 3.0 + + # 存储配置 + storage_type: StorageType = StorageType.JSON + storage_output_dir: str = "./output" + + class Config: + env_file = ".env" + env_prefix = "CRAWLER_" + + +# 全局配置实例 +settings = Settings() +``` + +### B站特定配置 + +```python +# config/bilibili_config.py +"""B站 API 配置""" + +# API 地址 +SEARCH_URL = "https://api.bilibili.com/x/web-interface/wbi/search/type" +VIDEO_INFO_URL = "https://api.bilibili.com/x/web-interface/view" +NAV_URL = "https://api.bilibili.com/x/web-interface/nav" + +# 请求配置 +SEARCH_PAGE_SIZE = 20 +REQUEST_TIMEOUT = 30 + +# 默认请求头 +DEFAULT_HEADERS = { + "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) " + "AppleWebKit/537.36 (KHTML, like Gecko) " + "Chrome/120.0.0.0 Safari/537.36", + "Referer": "https://www.bilibili.com", + "Origin": "https://www.bilibili.com", +} + +# WBI 签名密钥混淆表 +WBI_MIXIN_KEY_ENC_TAB = [ + 46, 47, 18, 2, 53, 8, 23, 32, 15, 50, 10, 31, 58, 3, 45, 35, + 27, 43, 5, 49, 33, 9, 42, 19, 29, 28, 14, 39, 12, 38, 41, 13, + 37, 48, 7, 16, 24, 55, 40, 61, 26, 17, 0, 1, 60, 51, 30, 4, + 22, 25, 54, 21, 56, 59, 6, 63, 57, 62, 11, 36, 20, 34, 44, 52, +] + +# 登录相关 +LOGIN_BUTTON_SELECTOR = "xpath=//div[@class='right-entry__outside go-login-btn']//div" +QRCODE_SELECTOR = "//div[@class='login-scan-box']//img" +LOGIN_COOKIE_KEYS = ["SESSDATA", "DedeUserID", "bili_jct"] +``` + +## 11.3 WBI 签名算法 + +B站使用 WBI 签名保护 API 请求，需要实现签名算法。 + +### WBI 签名原理 + +WBI（Web Bilibili Interface）签名是 B站用来保护 API 接口的一种机制，防止接口被恶意调用。 + +**为什么需要签名？** + +- 防止请求被篡改 +- 防止接口被滥用 +- 增加爬虫难度 + +### WBI 签名流程图 + +```mermaid +flowchart TD + subgraph step1 ["第一步: 获取签名密钥"] + api["调用 /x/web-interface/nav API"] + api --> response["响应中包含 wbi_img:
{img_url, sub_url}"] + response --> extract["提取文件名作为密钥:
img_key, sub_key"] + end + + step1 --> step2 + + subgraph step2 ["第二步: 生成盐值 Salt"] + concat["raw_key = img_key + sub_key"] + concat --> mixin["使用混淆表重排字符:
WBI_MIXIN_KEY_ENC_TAB"] + mixin --> salt["salt = 重排后取前32位"] + end + + step2 --> step3 + + subgraph step3 ["第三步: 计算签名"] + params["原始参数:
{keyword, page}"] + params --> addwts["添加时间戳 wts"] + addwts --> sort["按 key 排序并URL编码"] + sort --> append["拼接盐值: query + salt"] + append --> md5["计算 MD5 得到 w_rid"] + md5 --> final["最终参数:
{...原参数, wts, w_rid}"] + end +``` + +> **💡 关于 JS 逆向** +> +> 你可能会好奇：这个 WBI 签名算法是怎么逆向分析出来的？混淆表 `WBI_MIXIN_KEY_ENC_TAB` 又是从哪里找到的？ +> +> 别着急！这部分涉及到 **JavaScript 逆向**技术，我会在后面的 **「高级爬虫 - JS 逆向」** 章节中详细讲解。届时会带你一步步分析 B站的前端代码，找出签名算法的实现细节。 +> +> 本章的重点是让你理解**如何使用**这个签名算法，以及整个项目的工程化架构。签名算法的逆向分析过程，我们后面再深入探讨。 + +### 签名器实现 + +```python +# tools/sign.py +import hashlib +import time +import urllib.parse +from typing import Dict, Tuple +from functools import reduce + +from ..config import bilibili_config + + +class BilibiliSign: + """ + B站 WBI 签名器 + + WBI 签名算法用于保护 B站 API 请求。 + 签名流程： + 1. 从 wbi_img_urls 中提取 img_key 和 sub_key + 2. 使用混淆表生成 salt + 3. 对请求参数进行签名 + """ + + def __init__(self, img_key: str, sub_key: str): + """ + 初始化签名器 + + Args: + img_key: 从 img_url 中提取的密钥 + sub_key: 从 sub_url 中提取的密钥 + """ + self.img_key = img_key + self.sub_key = sub_key + + def get_salt(self) -> str: + """ + 生成盐值 + + 通过混淆表对 img_key + sub_key 进行重排。 + """ + raw_wbi_key = self.img_key + self.sub_key + return reduce( + lambda s, i: s + raw_wbi_key[i], + bilibili_config.WBI_MIXIN_KEY_ENC_TAB, + '' + )[:32] + + def sign(self, req_data: Dict) -> Dict: + """ + 对请求参数进行签名 + + Args: + req_data: 原始请求参数 + + Returns: + Dict: 签名后的请求参数（包含 wts 和 w_rid） + """ + salt = self.get_salt() + + # 添加时间戳 + req_data['wts'] = int(time.time()) + + # 按 key 排序并编码 + params = dict(sorted(req_data.items())) + query = urllib.parse.urlencode(params) + + # 计算签名 + text_to_sign = query + salt + w_rid = hashlib.md5(text_to_sign.encode()).hexdigest() + + req_data['w_rid'] = w_rid + return req_data + + +def extract_wbi_keys_from_urls(img_url: str, sub_url: str) -> Tuple[str, str]: + """ + 从 URL 中提取 WBI 密钥 + + Args: + img_url: wbi_img 的 img_url + sub_url: wbi_img 的 sub_url + + Returns: + Tuple[str, str]: (img_key, sub_key) + """ + def extract_key(url: str) -> str: + # 从 URL 中提取文件名（不含扩展名） + # 例如：https://xxx/bfs/wbi/xxx.png -> xxx + filename = url.rsplit('/', 1)[-1] + return filename.split('.')[0] + + return extract_key(img_url), extract_key(sub_url) +``` + +## 11.4 数据模型定义 + +使用 Pydantic 定义视频数据模型： + +```python +# models/bilibili.py +from typing import Optional, List +from datetime import datetime +from pydantic import BaseModel, Field + + +class BilibiliVideo(BaseModel): + """B站视频信息模型""" + + # 视频标识 + video_id: str = Field(default="", description="视频 aid") + bvid: str = Field(default="", description="视频 BV 号") + + # 视频信息 + title: str = Field(default="", description="视频标题") + desc: str = Field(default="", description="视频描述") + cover_url: str = Field(default="", description="封面 URL") + duration: int = Field(default=0, description="时长（秒）") + create_time: int = Field(default=0, description="发布时间戳") + + # UP主信息 + user_id: int = Field(default=0, description="UP主 ID") + nickname: str = Field(default="", description="UP主昵称") + avatar: str = Field(default="", description="UP主头像") + + # 互动数据 + play_count: int = Field(default=0, description="播放量") + liked_count: int = Field(default=0, description="点赞数") + coin_count: int = Field(default=0, description="投币数") + favorite_count: int = Field(default=0, description="收藏数") + share_count: int = Field(default=0, description="分享数") + danmaku_count: int = Field(default=0, description="弹幕数") + comment_count: int = Field(default=0, description="评论数") + + # 爬取信息 + source_keyword: str = Field(default="", description="搜索关键词") + crawl_time: str = Field( + default_factory=lambda: datetime.now().strftime("%Y-%m-%d %H:%M:%S"), + description="爬取时间" + ) + + @classmethod + def from_api_response(cls, data: dict, source_keyword: str = "") -> "BilibiliVideo": + """从视频详情 API 响应构建模型""" + stat = data.get("stat", {}) + owner = data.get("owner", {}) + + return cls( + video_id=str(data.get("aid", "")), + bvid=data.get("bvid", ""), + title=data.get("title", ""), + desc=data.get("desc", ""), + cover_url=data.get("pic", ""), + duration=data.get("duration", 0), + create_time=data.get("pubdate", 0), + user_id=owner.get("mid", 0), + nickname=owner.get("name", ""), + avatar=owner.get("face", ""), + play_count=stat.get("view", 0), + liked_count=stat.get("like", 0), + coin_count=stat.get("coin", 0), + favorite_count=stat.get("favorite", 0), + share_count=stat.get("share", 0), + danmaku_count=stat.get("danmaku", 0), + comment_count=stat.get("reply", 0), + source_keyword=source_keyword, + ) + + @classmethod + def from_search_result(cls, data: dict, keyword: str = "") -> "BilibiliVideo": + """从搜索结果构建模型""" + return cls( + video_id=str(data.get("aid", "")), + bvid=data.get("bvid", ""), + title=data.get("title", "").replace("", "").replace("", ""), + desc=data.get("description", ""), + cover_url="https:" + data.get("pic", "") if data.get("pic", "").startswith("//") else data.get("pic", ""), + duration=data.get("duration", 0) if isinstance(data.get("duration"), int) else 0, + user_id=data.get("mid", 0), + nickname=data.get("author", ""), + avatar=data.get("upic", ""), + play_count=data.get("play", 0), + liked_count=data.get("like", 0), + danmaku_count=data.get("danmaku", 0), + source_keyword=keyword, + ) + + def to_dict(self) -> dict: + """转换为字典""" + return self.model_dump() +``` + +## 11.5 登录认证模块 + +登录认证是爬虫获取完整数据的关键步骤。B站对未登录用户有很多数据限制，登录后可以获取更多信息。 + +### 为什么需要登录？ + +| 数据项 | 未登录 | 已登录 | +|--------|--------|--------| +| 搜索结果 | 有限制 | 完整 | +| 视频详情 | 基础信息 | 完整信息 | +| 用户数据 | 部分隐藏 | 可见 | +| API调用频率 | 严格限制 | 相对宽松 | + +### 登录方式对比 + +| 方式 | 优点 | 缺点 | 适用场景 | +|------|------|------|----------| +| 扫码登录 | 安全、无需处理复杂逻辑 | 需要手机APP配合 | 首次登录、开发调试 | +| Cookie登录 | 快速、可自动化 | Cookie会过期 | 批量部署、定时任务 | + +### 扫码登录流程图 + +```mermaid +sequenceDiagram + participant PC as PC浏览器 + participant Server as B站服务器 + participant APP as B站APP + + PC->>Server: 1. 访问B站首页 + PC->>Server: 2. 点击登录按钮 + PC->>Server: 3. 请求二维码+UUID + Server-->>PC: 4. 返回二维码图片 + Note over PC: 5. 显示二维码 + + loop 轮询登录状态 + PC->>Server: 6. 检查登录状态 + APP->>Server: 7. 用户扫描二维码 + Server-->>PC: 8. 状态: 已扫描 + PC->>Server: 9. 继续轮询 + APP->>Server: 10. 用户点击确认 + Server-->>PC: 11. 返回登录凭证(Set-Cookie) + end + + Note over PC: 12. 保存Cookie
登录成功! +``` + +### Cookie 登录流程图 + +```mermaid +flowchart LR + subgraph input [用户输入] + cookie["用户提供Cookie
(从浏览器复制)"] + end + + subgraph process [处理流程] + parse["解析Cookie字符串
提取键值对"] + inject["注入到BrowserContext
add_cookies()"] + visit["访问B站首页
加载页面"] + check{"检查关键Cookie
SESSDATA存在?"} + end + + subgraph result [结果] + success["登录成功!
开始爬取"] + fail["Cookie已过期
需要重新登录"] + end + + cookie --> parse --> inject --> visit --> check + check -->|存在| success + check -->|不存在| fail +``` + +**关键Cookie说明：** + +| Cookie名称 | 说明 | +|-----------|------| +| SESSDATA | 会话凭证，最重要的登录标识 | +| DedeUserID | 用户ID | +| bili_jct | CSRF Token，某些操作需要 | + +### 登录实现代码 + +```python +# login/auth.py +import asyncio +import base64 +from abc import ABC, abstractmethod +from pathlib import Path +from typing import Optional, List, Dict +from loguru import logger + +from playwright.async_api import BrowserContext, Page + +# 登录相关常量 +BILIBILI_URL = "https://www.bilibili.com" +LOGIN_BUTTON_SELECTOR = "xpath=//div[@class='right-entry__outside go-login-btn']//div" +QRCODE_SELECTOR = "//div[@class='login-scan-box']//img" + + +class BilibiliLogin: + """ + B站登录类 + + 支持扫码登录和 Cookie 登录两种方式。 + """ + + def __init__( + self, + login_type: str, + browser_context: BrowserContext, + context_page: Page, + cookie_str: str = "", + ): + self.login_type = login_type + self.browser_context = browser_context + self.context_page = context_page + self.cookie_str = cookie_str + + async def begin(self) -> bool: + """开始登录流程""" + logger.info(f"[BilibiliLogin] 开始登录，方式: {self.login_type}") + + if self.login_type == "qrcode": + return await self.login_by_qrcode() + elif self.login_type == "cookie": + return await self.login_by_cookies() + else: + logger.error(f"[BilibiliLogin] 不支持的登录类型: {self.login_type}") + return False + + async def login_by_qrcode(self) -> bool: + """ + 扫码登录 + + 流程： + 1. 访问 B站首页 + 2. 点击登录按钮 + 3. 获取二维码图片并显示 + 4. 等待用户扫码 + 5. 检查登录状态 + """ + logger.info("[BilibiliLogin] 开始扫码登录...") + + try: + # 1. 访问 B站首页 + await self.context_page.goto(BILIBILI_URL) + await asyncio.sleep(2) + + # 2. 点击登录按钮 + try: + login_button = await self.context_page.wait_for_selector( + LOGIN_BUTTON_SELECTOR, + timeout=10000 + ) + if login_button: + await login_button.click() + await asyncio.sleep(1) + except Exception as e: + logger.warning(f"[BilibiliLogin] 点击登录按钮失败: {e}") + + # 3. 获取并显示二维码 + qrcode_img = await self._find_login_qrcode() + if qrcode_img: + await self._show_qrcode(qrcode_img) + + # 4. 等待登录成功 + logger.info("[BilibiliLogin] 请使用 B站 APP 扫描二维码登录...") + logger.info("[BilibiliLogin] 等待登录成功（最长等待 120 秒）...") + + for _ in range(120): + if await self.check_login_state(): + logger.info("[BilibiliLogin] 扫码登录成功！") + await asyncio.sleep(2) + return True + await asyncio.sleep(1) + + logger.error("[BilibiliLogin] 扫码登录超时") + return False + + except Exception as e: + logger.error(f"[BilibiliLogin] 扫码登录失败: {e}") + return False + + async def login_by_cookies(self) -> bool: + """Cookie 登录""" + logger.info("[BilibiliLogin] 开始 Cookie 登录...") + + if not self.cookie_str: + logger.error("[BilibiliLogin] Cookie 字符串为空") + return False + + try: + cookies = self._parse_cookie_str(self.cookie_str) + await self.browser_context.add_cookies(cookies) + logger.info(f"[BilibiliLogin] 成功注入 {len(cookies)} 个 Cookie") + + await self.context_page.goto(BILIBILI_URL) + await asyncio.sleep(2) + + if await self.check_login_state(): + logger.info("[BilibiliLogin] Cookie 登录成功！") + return True + else: + logger.error("[BilibiliLogin] Cookie 登录失败，Cookie 可能已过期") + return False + + except Exception as e: + logger.error(f"[BilibiliLogin] Cookie 登录失败: {e}") + return False + + async def check_login_state(self) -> bool: + """检查登录状态""" + try: + cookies = await self.browser_context.cookies() + cookie_dict = {c['name']: c['value'] for c in cookies} + + for key in ["SESSDATA", "DedeUserID"]: + if key in cookie_dict and cookie_dict[key]: + return True + return False + except Exception: + return False + + async def _find_login_qrcode(self) -> Optional[str]: + """查找登录二维码""" + try: + qrcode_element = await self.context_page.wait_for_selector( + QRCODE_SELECTOR, + timeout=10000 + ) + if qrcode_element: + qrcode_src = await qrcode_element.get_attribute("src") + if qrcode_src and qrcode_src.startswith("data:image"): + return qrcode_src.split(",")[1] + return None + except Exception as e: + logger.error(f"[BilibiliLogin] 获取二维码失败: {e}") + return None + + async def _show_qrcode(self, qrcode_base64: str): + """显示二维码""" + try: + qrcode_bytes = base64.b64decode(qrcode_base64) + qrcode_path = Path("qrcode.png") + with open(qrcode_path, 'wb') as f: + f.write(qrcode_bytes) + logger.info(f"[BilibiliLogin] 二维码已保存到: {qrcode_path.absolute()}") + + print("\n" + "=" * 60) + print(" 请使用 B站 APP 扫描二维码登录") + print(f" 二维码文件: {qrcode_path.absolute()}") + print(" 等待登录中...") + print("=" * 60 + "\n") + except Exception as e: + logger.error(f"[BilibiliLogin] 显示二维码失败: {e}") + + def _parse_cookie_str(self, cookie_str: str) -> List[Dict]: + """解析 Cookie 字符串""" + cookies = [] + for item in cookie_str.split(";"): + item = item.strip() + if not item or "=" not in item: + continue + parts = item.split("=", 1) + name = parts[0].strip() + value = parts[1].strip() if len(parts) > 1 else "" + if name: + cookies.append({ + "name": name, + "value": value, + "domain": ".bilibili.com", + "path": "/" + }) + return cookies +``` + +## 11.6 API 客户端 + +API 客户端是爬虫与 B站服务器交互的核心模块，负责发送请求、处理签名、解析响应。 + +### 客户端职责 + +```mermaid +graph TB + subgraph 核心功能 + cookie["Cookie管理
• 从浏览器同步
• 注入到请求头
• 验证有效性"] + wbi["WBI签名
• 获取密钥
• 参数签名
• 自动刷新"] + http["HTTP请求
• GET/POST请求
• 超时处理
• 错误重试"] + end + + subgraph API方法 + api["BilibiliClient API
• search_video_by_keyword
• get_video_info
• pong (登录检查)"] + end + + cookie --> api + wbi --> api + http --> api +``` + +### B站 API 列表 + +| API | 地址 | 功能 | 是否需要签名 | +|-----|------|------|-------------| +| 用户信息 | `/x/web-interface/nav` | 获取登录用户信息和WBI密钥 | 否 | +| 视频搜索 | `/x/web-interface/wbi/search/type` | 按关键词搜索视频 | **是** | +| 视频详情 | `/x/web-interface/view` | 获取视频完整信息 | 否 | + +### 客户端实现代码 + +```python +# client/bilibili_client.py +import json +from typing import Dict, Optional, List +from loguru import logger +import httpx + +from playwright.async_api import BrowserContext, Page + +from ..tools.sign import BilibiliSign, extract_wbi_keys_from_urls +from ..models.bilibili import BilibiliVideo +from ..config import bilibili_config + + +class BilibiliClient: + """ + B站 API 客户端 + + 封装 B站的 API 请求，支持 WBI 签名。 + """ + + def __init__(self): + self.headers = bilibili_config.DEFAULT_HEADERS.copy() + self.cookie_dict: Dict[str, str] = {} + self._signer: Optional[BilibiliSign] = None + self._timeout = bilibili_config.REQUEST_TIMEOUT + + async def update_cookies(self, browser_context: BrowserContext): + """从浏览器上下文更新 Cookie""" + cookies = await browser_context.cookies() + cookie_str = "; ".join([f"{c['name']}={c['value']}" for c in cookies]) + self.headers["Cookie"] = cookie_str + self.cookie_dict = {c['name']: c['value'] for c in cookies} + logger.info(f"[BilibiliClient] 更新了 {len(cookies)} 个 Cookie") + + async def init_wbi_sign(self, page: Page): + """ + 初始化 WBI 签名器 + + 从浏览器的 localStorage 中获取 WBI 密钥。 + """ + try: + wbi_img_urls = await page.evaluate(""" + () => { + return localStorage.getItem('wbi_img_urls'); + } + """) + + if not wbi_img_urls: + logger.warning("[BilibiliClient] 未找到 wbi_img_urls，尝试从 API 获取") + await self._fetch_wbi_keys() + return + + wbi_data = json.loads(wbi_img_urls) + img_url = wbi_data.get("imgUrl", "") + sub_url = wbi_data.get("subUrl", "") + + if img_url and sub_url: + img_key, sub_key = extract_wbi_keys_from_urls(img_url, sub_url) + self._signer = BilibiliSign(img_key, sub_key) + logger.info("[BilibiliClient] WBI 签名器初始化成功") + else: + await self._fetch_wbi_keys() + + except Exception as e: + logger.error(f"[BilibiliClient] 初始化 WBI 签名器失败: {e}") + await self._fetch_wbi_keys() + + async def _fetch_wbi_keys(self): + """从 API 获取 WBI 密钥（备用方案）""" + try: + async with httpx.AsyncClient(timeout=self._timeout) as client: + response = await client.get( + "https://api.bilibili.com/x/web-interface/nav", + headers=self.headers + ) + data = response.json() + + if data.get("code") == 0: + wbi_img = data.get("data", {}).get("wbi_img", {}) + img_url = wbi_img.get("img_url", "") + sub_url = wbi_img.get("sub_url", "") + + if img_url and sub_url: + img_key, sub_key = extract_wbi_keys_from_urls(img_url, sub_url) + self._signer = BilibiliSign(img_key, sub_key) + logger.info("[BilibiliClient] 从 API 获取 WBI 密钥成功") + return + + logger.error("[BilibiliClient] 无法获取 WBI 密钥") + + except Exception as e: + logger.error(f"[BilibiliClient] 获取 WBI 密钥失败: {e}") + + async def _request( + self, + method: str, + url: str, + params: Optional[Dict] = None, + enable_sign: bool = False + ) -> Optional[Dict]: + """发送 HTTP 请求""" + try: + if enable_sign and self._signer and params: + params = self._signer.sign(params) + + async with httpx.AsyncClient(timeout=self._timeout) as client: + if method.upper() == "GET": + response = await client.get(url, params=params, headers=self.headers) + else: + response = await client.post(url, params=params, headers=self.headers) + + if response.status_code == 200: + return response.json() + else: + logger.error(f"[BilibiliClient] 请求失败: {response.status_code}") + return None + + except Exception as e: + logger.error(f"[BilibiliClient] 请求出错: {e}") + return None + + async def search_video_by_keyword( + self, + keyword: str, + page: int = 1, + page_size: int = 20, + ) -> List[BilibiliVideo]: + """ + 按关键词搜索视频 + + Args: + keyword: 搜索关键词 + page: 页码 + page_size: 每页数量 + + Returns: + List[BilibiliVideo]: 视频列表 + """ + logger.info(f"[BilibiliClient] 搜索视频: {keyword}, 第 {page} 页") + + params = { + "keyword": keyword, + "search_type": "video", + "page": page, + "page_size": page_size, + } + + data = await self._request( + "GET", + bilibili_config.SEARCH_URL, + params=params, + enable_sign=True + ) + + if not data or data.get("code") != 0: + logger.error(f"[BilibiliClient] 搜索失败: {data.get('message') if data else 'No response'}") + return [] + + result = data.get("data", {}) + video_list = result.get("result", []) + + videos = [] + for item in video_list: + try: + video = BilibiliVideo.from_search_result(item, keyword) + videos.append(video) + except Exception as e: + logger.debug(f"[BilibiliClient] 解析视频失败: {e}") + + logger.info(f"[BilibiliClient] 搜索到 {len(videos)} 个视频") + return videos + + async def get_video_info( + self, + aid: Optional[str] = None, + bvid: Optional[str] = None + ) -> Optional[BilibiliVideo]: + """ + 获取视频详情 + + Args: + aid: 视频 aid + bvid: 视频 BV 号 + + Returns: + BilibiliVideo: 视频信息 + """ + if not aid and not bvid: + logger.error("[BilibiliClient] aid 和 bvid 至少提供一个") + return None + + params = {} + if bvid: + params["bvid"] = bvid + elif aid: + params["aid"] = aid + + logger.info(f"[BilibiliClient] 获取视频详情: {bvid or aid}") + + data = await self._request( + "GET", + bilibili_config.VIDEO_INFO_URL, + params=params, + enable_sign=False + ) + + if not data or data.get("code") != 0: + logger.error(f"[BilibiliClient] 获取视频详情失败") + return None + + video_data = data.get("data", {}) + return BilibiliVideo.from_api_response(video_data) + + async def pong(self) -> bool: + """检查登录状态""" + try: + data = await self._request( + "GET", + "https://api.bilibili.com/x/web-interface/nav", + enable_sign=False + ) + + if data and data.get("code") == 0: + user_data = data.get("data", {}) + if user_data.get("isLogin"): + username = user_data.get("uname", "未知用户") + logger.info(f"[BilibiliClient] 已登录: {username}") + return True + + return False + except Exception: + return False +``` + +## 11.7 爬虫模块 + +爬虫模块是整个项目的核心调度器，负责协调浏览器、登录、API客户端等组件完成数据采集任务。 + +### 爬虫类设计 + +```mermaid +graph LR + subgraph BilibiliCrawler + subgraph 属性 + attr1["browser_manager"] --> desc1["管理Playwright浏览器"] + attr2["browser_context"] --> desc2["浏览器上下文(Cookie容器)"] + attr3["context_page"] --> desc3["页面实例"] + attr4["bili_client"] --> desc4["API客户端"] + attr5["_results"] --> desc5["爬取结果列表"] + end + + subgraph 方法 + m1["start()"] --> d1["启动爬虫(主入口)"] + m2["_init_browser()"] --> d2["初始化浏览器"] + m3["_do_login()"] --> d3["执行登录"] + m4["_init_client()"] --> d4["初始化API客户端"] + m5["search_by_keywords()"] --> d5["关键词搜索"] + m6["get_specified_videos()"] --> d6["获取指定视频"] + m7["close()"] --> d7["清理资源"] + end + end +``` + +### 两种爬取模式 + +#### SEARCH 模式（关键词搜索视频） + +```mermaid +flowchart LR + subgraph input [关键词列表] + k1["Python"] + k2["教程"] + k3["数据分析"] + end + + subgraph search [搜索分页] + p1["第1页"] + p2["第2页"] + p3["..."] + end + + subgraph result [视频列表] + v1["BV1xxx"] + v2["BV2xxx"] + v3["BV3xxx"] + end + + detail["获取每个视频详情
(完整播放量等)"] + + input --> search --> result --> detail +``` + +#### DETAIL 模式（获取指定视频详情） + +```mermaid +flowchart LR + subgraph input [指定BV号列表] + bv1["BV1abc"] + bv2["BV2def"] + bv3["BV3ghi"] + end + + api["遍历列表
逐个调用详情API"] + + input --> api +``` + +### 反爬策略 + +为了避免被 B站 封禁，爬虫采用了以下策略： + +| 策略 | 实现方式 | 配置项 | +|------|----------|--------| +| 随机延迟 | 每次请求后随机等待 1-3 秒 | `crawl_delay_min`, `crawl_delay_max` | +| 频率控制 | 限制最大爬取数量 | `max_video_count` | +| 登录态 | 使用真实登录Cookie | `login_type` | +| 完整请求头 | User-Agent、Referer等 | `DEFAULT_HEADERS` | + +### 爬虫实现代码 + +```python +# crawler/spider.py +import asyncio +import random +from typing import List, Optional +from loguru import logger + +from playwright.async_api import BrowserContext, Page + +from ..config import settings, CrawlerType +from ..core.browser import BrowserManager +from ..login.auth import BilibiliLogin +from ..client.bilibili_client import BilibiliClient +from ..models.bilibili import BilibiliVideo + + +class BilibiliCrawler: + """ + B站爬虫类 + + 整合浏览器管理、登录认证、API客户端，实现完整的爬取流程。 + """ + + def __init__(self): + self.browser_manager: Optional[BrowserManager] = None + self.browser_context: Optional[BrowserContext] = None + self.context_page: Optional[Page] = None + self.bili_client: Optional[BilibiliClient] = None + self._results: List[BilibiliVideo] = [] + + # 配置 + self.max_video_count = settings.max_video_count + self.delay_min = settings.crawl_delay_min + self.delay_max = settings.crawl_delay_max + + async def start(self) -> List[BilibiliVideo]: + """ + 启动爬虫 + + 完整流程： + 1. 启动浏览器 + 2. 执行登录 + 3. 初始化 API 客户端 + 4. 根据配置执行爬取 + 5. 关闭浏览器 + """ + logger.info(f"[BilibiliCrawler] 启动爬虫，类型: {settings.crawler_type}") + + try: + # 1. 启动浏览器 + await self._init_browser() + + # 2. 执行登录 + login_success = await self._do_login() + if not login_success: + logger.error("[BilibiliCrawler] 登录失败，退出") + return [] + + # 3. 初始化 API 客户端 + await self._init_client() + + # 4. 根据配置执行爬取 + if settings.crawler_type == CrawlerType.SEARCH: + await self.search_by_keywords() + elif settings.crawler_type == CrawlerType.DETAIL: + await self.get_specified_videos() + + logger.info(f"[BilibiliCrawler] 爬取完成，共 {len(self._results)} 个视频") + return self._results + + except Exception as e: + logger.exception(f"[BilibiliCrawler] 爬取出错: {e}") + return self._results + + finally: + await self.close() + + async def _init_browser(self): + """初始化浏览器""" + logger.info("[BilibiliCrawler] 初始化浏览器...") + + self.browser_manager = BrowserManager( + headless=settings.browser_headless, + timeout=settings.browser_timeout, + user_data_dir=settings.browser_user_data_dir if settings.save_login_state else None + ) + + self.browser_context = await self.browser_manager.start() + self.context_page = await self.browser_manager.new_page() + + async def _do_login(self) -> bool: + """执行登录""" + self.bili_client = BilibiliClient() + await self.bili_client.update_cookies(self.browser_context) + + if await self.bili_client.pong(): + logger.info("[BilibiliCrawler] 已有登录状态，跳过登录") + return True + + login = BilibiliLogin( + login_type=settings.login_type.value, + browser_context=self.browser_context, + context_page=self.context_page, + cookie_str=settings.cookie_str + ) + + success = await login.begin() + + if success: + await self.bili_client.update_cookies(self.browser_context) + + return success + + async def _init_client(self): + """初始化 API 客户端""" + await self.bili_client.init_wbi_sign(self.context_page) + + async def search_by_keywords(self) -> List[BilibiliVideo]: + """按关键词搜索视频""" + keywords = [kw.strip() for kw in settings.keywords.split(",") if kw.strip()] + + if not keywords: + logger.warning("[BilibiliCrawler] 未配置搜索关键词") + return [] + + logger.info(f"[BilibiliCrawler] 开始搜索，关键词: {keywords}") + + for keyword in keywords: + await self._search_single_keyword(keyword) + if len(self._results) >= self.max_video_count: + break + + return self._results + + async def _search_single_keyword(self, keyword: str): + """搜索单个关键词""" + page = 1 + + while len(self._results) < self.max_video_count: + logger.info(f"[BilibiliCrawler] 搜索 '{keyword}'，第 {page} 页") + + videos = await self.bili_client.search_video_by_keyword( + keyword=keyword, + page=page, + ) + + if not videos: + break + + for video in videos: + if len(self._results) >= self.max_video_count: + break + + # 获取完整视频详情 + video_detail = await self.bili_client.get_video_info(bvid=video.bvid) + if video_detail: + video_detail.source_keyword = keyword + self._results.append(video_detail) + logger.info(f"[BilibiliCrawler] 获取视频: {video_detail.title[:30]}...") + else: + self._results.append(video) + + await self._random_delay() + + page += 1 + if page > 50: + break + + async def get_specified_videos(self) -> List[BilibiliVideo]: + """获取指定视频列表的详情""" + video_list = settings.specified_id_list + + if not video_list: + logger.warning("[BilibiliCrawler] 未配置指定视频列表") + return [] + + logger.info(f"[BilibiliCrawler] 获取 {len(video_list)} 个指定视频") + + for video_id in video_list: + if len(self._results) >= self.max_video_count: + break + + video = await self.bili_client.get_video_info(bvid=video_id) + if video: + self._results.append(video) + logger.info(f"[BilibiliCrawler] 获取视频: {video.title[:30]}...") + + await self._random_delay() + + return self._results + + async def _random_delay(self): + """随机延迟""" + delay = random.uniform(self.delay_min, self.delay_max) + await asyncio.sleep(delay) + + async def close(self): + """关闭浏览器""" + if self.browser_manager: + await self.browser_manager.close() +``` + +## 11.8 数据存储模块 + +数据存储模块负责将爬取到的数据持久化保存，支持多种存储格式。 + +### 存储架构设计 + +采用**策略模式**设计，方便扩展新的存储方式： + +```mermaid +graph TB + subgraph manager [存储管理器] + sm["StorageManager
• save(data)
• load()
• filepath"] + end + + subgraph base [抽象基类] + bs["BaseStorage (ABC)
• save() 抽象方法
• load() 抽象方法"] + end + + subgraph impl [具体实现] + json["JSONStorage
• 保存为JSON文件
• 保持数据结构
• 便于程序处理"] + csv["CSVStorage
• 保存为CSV文件
• 适合Excel打开
• 便于数据分析"] + end + + sm -->|根据配置选择| json + sm -->|根据配置选择| csv + json -->|继承| bs + csv -->|继承| bs +``` + +### 存储格式对比 + +| 格式 | 优点 | 缺点 | 适用场景 | +|------|------|------|----------| +| JSON | 保持嵌套结构、程序易读取 | 文件较大、不便人工查看 | 后续程序处理、API接口 | +| CSV | Excel可打开、便于分析 | 无法保存嵌套结构 | 数据分析、报表制作 | + +### 存储实现代码 + +```python +# store/backend.py +import json +import csv +from abc import ABC, abstractmethod +from datetime import datetime +from pathlib import Path +from typing import List, Dict, Any +from loguru import logger + + +class BaseStorage(ABC): + """存储基类""" + + @abstractmethod + async def save(self, data: List[Dict]) -> bool: + pass + + @abstractmethod + async def load(self) -> List[Dict]: + pass + + +class JSONStorage(BaseStorage): + """JSON 存储""" + + def __init__(self, output_dir: str, filename: str = None): + self.output_dir = Path(output_dir) + self.output_dir.mkdir(parents=True, exist_ok=True) + + if filename: + self.filepath = self.output_dir / filename + else: + timestamp = datetime.now().strftime('%Y%m%d_%H%M%S') + self.filepath = self.output_dir / f"data_{timestamp}.json" + + async def save(self, data: List[Dict]) -> bool: + try: + with open(self.filepath, 'w', encoding='utf-8') as f: + json.dump(data, f, ensure_ascii=False, indent=2) + logger.info(f"数据已保存到: {self.filepath} ({len(data)} 条)") + return True + except Exception as e: + logger.error(f"保存失败: {e}") + return False + + async def load(self) -> List[Dict]: + if not self.filepath.exists(): + return [] + try: + with open(self.filepath, 'r', encoding='utf-8') as f: + return json.load(f) + except Exception as e: + logger.error(f"加载失败: {e}") + return [] + + +class CSVStorage(BaseStorage): + """CSV 存储""" + + def __init__(self, output_dir: str, filename: str = None, fields: List[str] = None): + self.output_dir = Path(output_dir) + self.output_dir.mkdir(parents=True, exist_ok=True) + + if filename: + self.filepath = self.output_dir / filename + else: + timestamp = datetime.now().strftime('%Y%m%d_%H%M%S') + self.filepath = self.output_dir / f"data_{timestamp}.csv" + + self.fields = fields + + async def save(self, data: List[Dict]) -> bool: + if not data: + logger.warning("没有数据需要保存") + return True + + try: + fields = self.fields or list(data[0].keys()) + + with open(self.filepath, 'w', encoding='utf-8-sig', newline='') as f: + writer = csv.DictWriter(f, fieldnames=fields, extrasaction='ignore') + writer.writeheader() + writer.writerows(data) + + logger.info(f"数据已保存到: {self.filepath} ({len(data)} 条)") + return True + except Exception as e: + logger.error(f"保存失败: {e}") + return False + + async def load(self) -> List[Dict]: + if not self.filepath.exists(): + return [] + try: + with open(self.filepath, 'r', encoding='utf-8-sig') as f: + reader = csv.DictReader(f) + return list(reader) + except Exception as e: + logger.error(f"加载失败: {e}") + return [] + + +class StorageManager: + """存储管理器""" + + def __init__(self, storage_type: str, output_dir: str, **kwargs): + self.output_dir = output_dir + + if storage_type == 'json': + self._storage = JSONStorage(output_dir, **kwargs) + elif storage_type == 'csv': + self._storage = CSVStorage(output_dir, **kwargs) + else: + raise ValueError(f"不支持的存储类型: {storage_type}") + + async def save(self, data: List[Dict]) -> bool: + return await self._storage.save(data) + + async def load(self) -> List[Dict]: + return await self._storage.load() + + @property + def filepath(self) -> Path: + return self._storage.filepath +``` + +## 11.9 分析报告模块 + +分析报告模块负责对爬取的数据进行统计分析，生成可视化报告。 + +### 分析功能概览 + +```mermaid +flowchart LR + subgraph input [输入数据] + videos["BilibiliVideo
对象列表"] + end + + subgraph process [分析处理] + stats["视频指标统计
• 播放量统计
• 点赞数统计
• 收藏数统计"] + top["热门视频排名
TOP 10"] + up["UP主分布统计
词频分析
(jieba分词)"] + end + + subgraph output [输出结果] + md["Markdown 报告
• 数据表格
• TOP排名
• UP主分布"] + img["词云图片
(PNG)"] + end + + videos --> stats --> md + stats --> top --> img + top --> up +``` + +### 报告内容结构 + +生成的 Markdown 报告包含以下章节： + +| 章节 | 内容 | 分析维度 | +|------|------|----------| +| 视频指标统计 | 播放量、点赞、投币等指标的汇总统计 | 总计、平均、最高、最低 | +| 热门视频 TOP 10 | 按播放量排序的前10个视频 | 标题、UP主、播放量、点赞 | +| UP主分布 TOP 10 | 出现频率最高的UP主 | UP主名称、视频数量 | +| 标题热词 TOP 20 | 视频标题中出现最多的词汇 | 词汇、出现频次 | +| 标题词云 | 可视化展示热门词汇 | 词云图片 | + +### 可选依赖说明 + +分析模块使用了可选依赖，即使没有安装也不会报错： + +```mermaid +graph TB + subgraph deps [可选依赖] + jieba["jieba
中文分词
(词频统计)"] + wc["wordcloud
词云生成
(可视化)"] + pd["pandas
数据处理
(可选)"] + end + + subgraph fallback [未安装时的降级策略] + note["如果未安装:
• jieba 未安装 - 跳过词频统计和词云生成
• wordcloud 未安装 - 跳过词云生成
• pandas 未安装 - 使用纯Python实现统计"] + end + + jieba --> note + wc --> note + pd --> note +``` + +### 分析实现代码 + +```python +# analysis/report.py +from typing import List, Dict, Union +from datetime import datetime +from collections import Counter +from pathlib import Path +from loguru import logger + +# 可选依赖 +try: + import jieba + HAS_JIEBA = True +except ImportError: + HAS_JIEBA = False + +try: + from wordcloud import WordCloud + HAS_WORDCLOUD = True +except ImportError: + HAS_WORDCLOUD = False + + +class BilibiliAnalyzer: + """B站视频数据分析器""" + + STOPWORDS = { + '的', '是', '在', '了', '和', '与', '或', '有', '个', '人', + '这', '那', '就', '都', '也', '为', '对', '到', '从', '把', + } + + def __init__(self, videos: List[Union[Dict, any]], output_dir: str = "./output"): + # 转换为字典列表 + self.data = [] + for video in videos: + if hasattr(video, 'to_dict'): + self.data.append(video.to_dict()) + elif hasattr(video, 'model_dump'): + self.data.append(video.model_dump()) + elif isinstance(video, dict): + self.data.append(video) + + self.output_dir = Path(output_dir) + self.output_dir.mkdir(parents=True, exist_ok=True) + + def video_metrics_stats(self) -> Dict: + """视频指标统计""" + metrics = { + 'play_count': [], + 'liked_count': [], + 'coin_count': [], + 'favorite_count': [], + 'share_count': [], + 'danmaku_count': [], + 'comment_count': [], + } + + for item in self.data: + for key in metrics.keys(): + value = item.get(key, 0) or 0 + metrics[key].append(int(value)) + + stats = {} + for key, values in metrics.items(): + if values: + stats[key] = { + 'total': sum(values), + 'avg': sum(values) / len(values), + 'max': max(values), + 'min': min(values), + } + return stats + + def top_videos(self, metric: str = 'play_count', top_n: int = 10) -> List[Dict]: + """获取排名前 N 的视频""" + sorted_data = sorted( + self.data, + key=lambda x: x.get(metric, 0) or 0, + reverse=True + ) + return sorted_data[:top_n] + + def up_distribution(self, top_n: int = 10) -> List[tuple]: + """UP主分布统计""" + counter = Counter() + for item in self.data: + nickname = item.get('nickname', '未知UP主') + if nickname: + counter[nickname] += 1 + return counter.most_common(top_n) + + def word_frequency(self, text_field: str, top_n: int = 20) -> List[tuple]: + """词频统计""" + if not HAS_JIEBA: + return [] + + all_words = [] + for item in self.data: + text = item.get(text_field, '') + if text: + words = jieba.lcut(str(text)) + words = [w for w in words if w not in self.STOPWORDS and len(w) > 1] + all_words.extend(words) + + return Counter(all_words).most_common(top_n) + + def generate_wordcloud(self, text_field: str, output_file: str = "wordcloud.png") -> str: + """生成词云""" + if not HAS_WORDCLOUD: + return "" + + word_freq = self.word_frequency(text_field, 200) + if not word_freq: + return "" + + wc = WordCloud( + width=1200, + height=800, + background_color='white', + max_words=200, + ) + wc.generate_from_frequencies(dict(word_freq)) + + output_path = self.output_dir / output_file + wc.to_file(str(output_path)) + return str(output_path) + + +class ReportGenerator: + """报告生成器""" + + def __init__(self, videos: List, output_dir: str = "./output"): + self.videos = videos + self.output_dir = Path(output_dir) + self.output_dir.mkdir(parents=True, exist_ok=True) + self.analyzer = BilibiliAnalyzer(videos, output_dir) + + def generate(self, title: str = "B站视频数据分析报告") -> str: + """生成完整分析报告""" + lines = [] + + # 标题 + lines.append(f"# {title}") + lines.append("") + lines.append(f"> 生成时间: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}") + lines.append(f"> 数据量: {len(self.analyzer.data)} 条") + lines.append("") + lines.append("---") + lines.append("") + + # 1. 视频指标统计 + lines.append("## 1. 视频指标统计") + lines.append("") + metrics_stats = self.analyzer.video_metrics_stats() + if metrics_stats: + lines.append("| 指标 | 总计 | 平均 | 最高 | 最低 |") + lines.append("| --- | ---: | ---: | ---: | ---: |") + + metric_names = { + 'play_count': '播放量', + 'liked_count': '点赞数', + 'coin_count': '投币数', + 'favorite_count': '收藏数', + 'share_count': '分享数', + 'danmaku_count': '弹幕数', + 'comment_count': '评论数', + } + + for key, name in metric_names.items(): + if key in metrics_stats: + stat = metrics_stats[key] + lines.append( + f"| {name} | {stat['total']:,} | " + f"{stat['avg']:,.0f} | {stat['max']:,} | {stat['min']:,} |" + ) + lines.append("") + + # 2. 热门视频 TOP 10 + lines.append("## 2. 热门视频 TOP 10") + lines.append("") + top_videos = self.analyzer.top_videos('play_count', 10) + if top_videos: + lines.append("| 排名 | 标题 | UP主 | 播放量 | 点赞 |") + lines.append("| --- | --- | --- | ---: | ---: |") + for i, video in enumerate(top_videos, 1): + title_short = video.get('title', '')[:30] + '...' + lines.append( + f"| {i} | {title_short} | {video.get('nickname', '未知')} | " + f"{video.get('play_count', 0):,} | {video.get('liked_count', 0):,} |" + ) + lines.append("") + + # 3. UP主分布 + lines.append("## 3. UP主分布 TOP 10") + lines.append("") + up_dist = self.analyzer.up_distribution(10) + if up_dist: + lines.append("| 排名 | UP主 | 视频数 |") + lines.append("| --- | --- | ---: |") + for i, (name, count) in enumerate(up_dist, 1): + lines.append(f"| {i} | {name} | {count} |") + lines.append("") + + # 4. 标题热词 + if HAS_JIEBA: + lines.append("## 4. 标题热词 TOP 20") + lines.append("") + word_freq = self.analyzer.word_frequency('title', 20) + if word_freq: + lines.append("| 排名 | 词汇 | 频次 |") + lines.append("| --- | --- | ---: |") + for i, (word, count) in enumerate(word_freq, 1): + lines.append(f"| {i} | {word} | {count} |") + lines.append("") + + if HAS_WORDCLOUD: + wordcloud_path = self.analyzer.generate_wordcloud('title', 'title_wordcloud.png') + if wordcloud_path: + lines.append("### 标题词云") + lines.append("") + lines.append("![标题词云](title_wordcloud.png)") + lines.append("") + + # 保存报告 + report_content = '\n'.join(lines) + report_path = self.output_dir / "report.md" + with open(report_path, 'w', encoding='utf-8') as f: + f.write(report_content) + + logger.info(f"报告已保存: {report_path}") + return str(report_path) + + +def generate_report(videos: List, output_dir: str = "./output") -> str: + """生成分析报告（便捷函数）""" + generator = ReportGenerator(videos, output_dir) + return generator.generate() +``` + +## 11.10 主程序入口 + +整合所有模块： + +```python +# main.py +import asyncio +import sys +from pathlib import Path +from typing import List +from loguru import logger + +# 导入各模块 +from config import settings, CrawlerType +from crawler.spider import BilibiliCrawler +from store.backend import StorageManager +from analysis.report import generate_report +from models.bilibili import BilibiliVideo + + +def setup_logger(): + """配置日志""" + logger.remove() + logger.add( + sys.stderr, + format="{time:YYYY-MM-DD HH:mm:ss} | " + "{level: <8} | " + "{message}", + level="INFO" + ) + logger.add( + "logs/bilibili_{time:YYYY-MM-DD}.log", + rotation="1 day", + retention="7 days", + level="DEBUG", + ) + + +async def main(): + """主函数""" + + print(""" + ╔══════════════════════════════════════════════════════════╗ + ║ B站视频数据采集与分析工具 v2.0 ║ + ║ ║ + ║ 功能： ║ + ║ - 视频搜索与详情获取 ║ + ║ - 扫码登录 / Cookie 登录 ║ + ║ - JSON / CSV 数据存储 ║ + ║ - 词云和统计分析报告 ║ + ║ ║ + ║ 参考项目：MediaCrawler ║ + ╚══════════════════════════════════════════════════════════╝ + """) + + logger.info(f"启动 {settings.app_name}") + logger.info(f"爬取类型: {settings.crawler_type.value}") + logger.info(f"登录方式: {settings.login_type.value}") + logger.info(f"最大数量: {settings.max_video_count}") + + try: + # 1. 运行爬虫 + logger.info("开始爬取数据...") + crawler = BilibiliCrawler() + videos = await crawler.start() + logger.info(f"爬取完成: {len(videos)} 条视频") + + if not videos: + logger.warning("没有爬取到数据，退出") + return + + # 2. 保存数据 + data = [video.to_dict() for video in videos] + storage = StorageManager( + storage_type=settings.storage_type.value, + output_dir=settings.storage_output_dir + ) + await storage.save(data) + logger.info(f"数据已保存: {storage.filepath}") + + # 3. 生成报告 + report_path = generate_report(videos, settings.storage_output_dir) + logger.info(f"报告已生成: {report_path}") + + logger.info("=" * 50) + logger.info("任务完成！") + logger.info(f"数据文件: {storage.filepath}") + logger.info(f"分析报告: {report_path}") + logger.info("=" * 50) + + except KeyboardInterrupt: + logger.warning("用户中断执行") + except Exception as e: + logger.exception(f"执行出错: {e}") + + +if __name__ == "__main__": + setup_logger() + Path("logs").mkdir(exist_ok=True) + Path(settings.storage_output_dir).mkdir(parents=True, exist_ok=True) + asyncio.run(main()) +``` + +## 11.11 运行与测试 + +### 依赖安装 + +```bash +# 安装依赖 +pip install playwright httpx pydantic pydantic-settings loguru + +# 安装 Playwright 浏览器 +playwright install chromium + +# 可选：数据分析依赖 +pip install pandas jieba wordcloud +``` + +### 配置修改 + +修改 `config/settings.py` 中的配置： + +```python +# 爬取类型 +crawler_type = CrawlerType.SEARCH # 或 CrawlerType.DETAIL + +# 搜索关键词 +keywords = "Python教程,数据分析" + +# 最大爬取数量 +max_video_count = 20 + +# 登录方式 +login_type = LoginType.QRCODE # 首次使用扫码登录 +``` + +### 运行程序 + +```bash +cd 源代码/爬虫进阶/11_进阶综合实战项目 +python main.py +``` + +首次运行会弹出浏览器窗口，使用 B站 APP 扫码登录后，程序会自动开始爬取。 + +## 本章小结 + +本章我们完成了一个完整的 B站视频数据采集与分析项目，综合运用了： + +1. **登录认证** + - 扫码登录 + - Cookie 登录 + - 登录状态持久化 + +2. **API 签名** + - WBI 签名算法 + - 签名密钥获取 + +3. **数据爬取** + - 关键词搜索 + - 视频详情获取 + - 并发控制 + +4. **数据处理** + - Pydantic 数据模型 + - JSON/CSV 存储 + - 数据分析报告 + +5. **工程化设计** + - 模块化架构 + - 配置管理 + - 日志系统 + +**关键要点：** + +- B站 API 需要 WBI 签名保护 +- 登录状态可以持久化，避免重复扫码 +- 注意控制爬取频率，避免触发限制 +- 做好异常处理和日志记录 +- 遵守 B站使用条款和法律法规 + +--- + +## 教程总结 + +恭喜你完成了 Python 爬虫进阶教程的全部内容！在这 11 章的学习中，你掌握了： + +1. **工程化开发**：日志、配置、异常处理 +2. **反爬对抗**：请求伪装、代理 IP、浏览器指纹 +3. **浏览器自动化**：Playwright 基础和进阶 +4. **登录认证**：Cookie 管理、扫码登录 +5. **验证码处理**：OCR 识别、滑块验证 +6. **数据处理**：清洗、去重、分析、可视化 + +这些技术都源自 [MediaCrawler](https://github.com/NanmiCoder/MediaCrawler) 等实际项目的生产实践，希望能帮助你在爬虫开发领域更进一步。 + +**最后的建议：** + +- 持续关注反爬技术的演进 +- 遵守法律法规和网站规则 +- 参与开源项目，与社区共同成长 +- 将爬虫作为数据获取的手段，关注数据本身的价值 + +祝你在数据采集的道路上越走越远！ diff --git "a/docs/\347\210\254\350\231\253\350\277\233\344\273\267/README.md" "b/docs/\347\210\254\350\231\253\350\277\233\344\273\267/README.md" index e69de29..544941e 100644 --- "a/docs/\347\210\254\350\231\253\350\277\233\344\273\267/README.md" +++ "b/docs/\347\210\254\350\231\253\350\277\233\344\273\267/README.md" @@ -0,0 +1,318 @@ +# Python 爬虫进阶教程 + +> 本教程是「Python 爬虫入门教程」的延续，面向已掌握基础爬虫技术的学习者。教程将从工程化实践入手，逐步深入反爬虫对抗、浏览器自动化、登录认证等进阶主题，最终通过综合实战项目巩固所学知识。 + +## 适合人群 + +- 已完成入门教程 11 篇内容的学习者 +- 了解 Python 异步编程基础（async/await） +- 希望提升爬虫工程化能力和反爬对抗能力的开发者 + +## 技术栈概览 + +| 类别 | 技术/工具 | +|------|----------| +| HTTP 客户端 | httpx (异步) | +| 浏览器自动化 | Playwright | +| 数据验证 | Pydantic | +| 日志系统 | loguru | +| 配置管理 | pydantic-settings | +| 代理管理 | 自研代理池 | +| 验证码识别 | ddddocr / 第三方打码平台 | +| 数据分析 | pandas / wordcloud / jieba | + +--- + +## 教程目录 + +### [01_工程化爬虫开发规范](./01_工程化爬虫开发规范.md) + +**学习目标：** +- 理解爬虫项目的工程化重要性 +- 掌握 loguru 日志库的使用和日志分级策略 +- 学会使用 pydantic-settings 进行配置管理 +- 实现统一的异常处理和错误重试机制 +- 了解项目目录结构的最佳实践 + +**核心内容：** +- **日志系统设计**：loguru 基本使用、日志分级（DEBUG/INFO/WARNING/ERROR）、日志轮转和持久化、异步日志写入 +- **配置管理**：环境变量 vs 配置文件、pydantic-settings 实现配置验证、敏感信息的安全管理（.env 文件）、多环境配置切换 +- **异常处理**：自定义异常类设计、全局异常捕获、重试装饰器实现（tenacity 库）、优雅降级策略 +- **项目结构规范**：分层架构设计、模块划分原则、代码复用策略 + +**实战案例：** 将入门教程第 10 章的数据存储爬虫进行工程化改造，添加完善的日志系统、配置管理和异常处理机制。 + +--- + +### [02_反爬虫对抗基础_请求伪装](./02_反爬虫对抗基础_请求伪装.md) + +**学习目标：** +- 理解常见的反爬虫检测手段 +- 掌握 User-Agent 轮换策略 +- 学会构造完整的请求头伪装 +- 理解 Cookie 和 Session 的工作原理 +- 实现基础的速率控制 + +**核心内容：** +- **反爬虫机制概述**：基于请求特征的检测、基于行为特征的检测、基于内容的检测、常见的封禁策略 +- **User-Agent 策略**：真实浏览器 UA 收集、UA 随机轮换实现、移动端 vs 桌面端 UA 选择、fake-useragent 库的使用 +- **请求头完整伪装**：必要的请求头字段（Accept、Accept-Language、Accept-Encoding 等）、Referer 的正确设置、请求头一致性维护、使用 curl_cffi 模拟浏览器指纹 +- **速率控制**：固定延迟 vs 随机延迟、自适应速率调整、令牌桶算法实现、asyncio.Semaphore 控制并发数 + +**实战案例：** 爬取一个具有基础反爬的电商网站商品列表，实现完整的请求伪装和智能速率控制。 + +--- + +### [03_代理IP的使用与管理](./03_代理IP的使用与管理.md) + +**学习目标：** +- 理解代理 IP 的工作原理和类型 +- 学会评估代理 IP 的质量指标 +- 掌握代理池的设计与实现 +- 实现代理的自动检测和淘汰机制 +- 理解隧道代理和 API 代理的使用 + +**核心内容：** +- **代理 IP 基础**：HTTP 代理 vs HTTPS 代理 vs SOCKS5 代理、透明代理 vs 匿名代理 vs 高匿代理、免费代理 vs 付费代理的取舍、代理提供商选择指南 +- **代理池设计**（参考 MediaCrawler ProxyIpPool 设计）：代理池抽象接口设计、代理获取和存储、代理有效性检测、代理评分和淘汰机制、ProxyRefreshMixin 刷新机制 +- **代理使用模式**：每次请求更换代理、按 IP 绑定代理（Session 粘性）、隧道代理的使用、API 提取型代理的封装 +- **代理与异步请求集成**：httpx 设置代理、代理超时和重试策略、代理切换的优雅实现 + +**实战案例：** 实现一个完整的代理池管理类，支持从多个代理源获取代理、自动检测有效性、按需分配代理，并与爬虫主程序集成。 + +--- + +### [04_Playwright浏览器自动化入门](./04_Playwright浏览器自动化入门.md) + +**学习目标：** +- 理解浏览器自动化的应用场景 +- 掌握 Playwright 的安装和基本配置 +- 学会页面导航、元素定位和交互操作 +- 理解同步 API 和异步 API 的选择 +- 掌握截图、PDF 导出等实用功能 + +**核心内容：** +- **Playwright 概述**：与 Selenium/Puppeteer 的对比、支持的浏览器（Chromium/Firefox/WebKit）、同步 vs 异步 API 选择、安装和浏览器驱动管理 +- **页面基础操作**：Browser/Context/Page 三层模型、页面导航和等待策略、元素定位方式（CSS/XPath/Text/Role）、点击、输入、选择等交互操作 +- **等待策略**（重要）：自动等待机制、waitForSelector / waitForLoadState、自定义等待条件、超时处理 +- **页面内容提取**：获取元素文本和属性、执行 JavaScript 获取数据、拦截网络请求获取 API 数据、截图和 PDF 导出 +- **浏览器配置**：有头模式 vs 无头模式、视口大小设置、用户数据目录持久化、代理设置 + +**实战案例：** 使用 Playwright 爬取一个需要 JavaScript 渲染的 SPA 单页应用，提取动态加载的数据列表。 + +--- + +### [05_Playwright进阶_反检测与性能优化](./05_Playwright进阶_反检测与性能优化.md) + +**学习目标：** +- 理解浏览器指纹检测原理 +- 掌握 stealth.min.js 反检测注入 +- 学会 CDP 模式的使用场景 +- 掌握多浏览器实例管理和资源优化 +- 实现浏览器上下文复用策略 + +**核心内容：** +- **浏览器指纹检测**：navigator 属性检测、WebGL/Canvas 指纹、WebDriver 特征检测、行为特征检测 +- **stealth.js 反检测**（MediaCrawler 核心技术）：stealth.min.js 的工作原理、注入时机和方式、常见检测点的绕过、自定义反检测脚本 +- **CDP 模式深入**：什么是 Chrome DevTools Protocol、标准模式 vs CDP 模式对比、连接已有浏览器实例、CDP 命令直接调用 +- **性能优化**：禁用图片/CSS/字体加载、浏览器上下文复用、多页面并发管理、内存和资源监控、优雅关闭和资源释放 +- **异常处理**：页面崩溃检测和恢复、超时重试策略、浏览器进程管理 + +**实战案例：** 使用 Playwright + stealth.js 爬取一个具有严格反爬检测的网站，对比有无反检测的效果差异。 + +--- + +### [06_登录认证_Cookie与Session管理](./06_登录认证_Cookie与Session管理.md) + +**学习目标：** +- 深入理解 Cookie 和 Session 的工作机制 +- 掌握 Cookie 的提取、存储和注入 +- 学会检测登录状态和 Cookie 有效性 +- 实现 Cookie 的自动刷新机制 +- 理解不同登录场景的处理策略 + +**核心内容：** +- **认证机制深入**：Cookie 的属性详解（Domain/Path/Expires/HttpOnly/Secure）、Session 的服务端实现、Token 认证的工作流程、多设备登录和踢出策略 +- **Cookie 提取与存储**：从浏览器 DevTools 手动提取、使用 Playwright 自动提取、Cookie 的序列化格式（JSON/Netscape）、加密存储敏感 Cookie +- **Cookie 注入与使用**：httpx 中设置 Cookie、Playwright 中注入 Cookie、Cookie 的作用域管理、多账号 Cookie 轮换 +- **登录状态管理**：登录状态检测方法、Cookie 过期检测、自动重新登录机制、登录状态持久化 + +**实战案例：** 实现一个完整的 Cookie 管理模块，支持 Cookie 的存储、加载、验证和自动刷新，可用于需要登录的爬虫场景。 + +--- + +### [07_登录认证_扫码与短信登录实现](./07_登录认证_扫码与短信登录实现.md) + +**学习目标：** +- 理解扫码登录的技术原理 +- 掌握使用 Playwright 实现扫码登录流程 +- 理解短信验证码登录的实现方式 +- 学会登录状态的监控和回调机制 +- 实现多种登录方式的统一封装 + +**核心内容：** +- **扫码登录原理**（参考 MediaCrawler）：二维码生成和轮询机制、长轮询 vs WebSocket 状态推送、扫码状态（待扫描/已扫描/已确认/已过期）、登录成功后的 Cookie 获取 +- **Playwright 实现扫码登录**：定位二维码元素、二维码截图和保存、终端显示二维码（可选）、轮询检测登录状态、登录成功后 Cookie 提取 +- **短信验证码登录**：手机号输入和验证码发送、验证码输入（手动/API 接收）、滑块验证码的处理、登录失败的重试策略 +- **登录模块封装**：登录方式抽象（工厂模式）、统一的登录接口设计、登录状态回调机制、异常处理和超时控制 + +**实战案例：** 参考 MediaCrawler 的登录实现，完成一个支持扫码登录和 Cookie 注入的双模式登录模块。 + +--- + +### [08_验证码识别与处理](./08_验证码识别与处理.md) + +**学习目标：** +- 了解常见验证码类型和破解思路 +- 掌握图片验证码的 OCR 识别 +- 学会滑块验证码的轨迹模拟 +- 了解第三方打码平台的使用 +- 理解验证码处理的合规边界 + +**核心内容：** +- **验证码类型概览**：图片字符验证码、数学运算验证码、滑块拼图验证码、点选验证码、行为验证码（如 reCAPTCHA） +- **图片验证码识别**：ddddocr 库的使用、图片预处理（二值化/降噪）、本地模型 vs 云端 API、识别率优化策略 +- **滑块验证码处理**：缺口位置识别（OpenCV/图像差异）、人类拖拽轨迹模拟、速度和加速度曲线、Playwright 实现拖拽操作 +- **第三方打码平台**：平台选择和价格对比、API 调用封装、超时和失败重试、成本控制策略 +- **合规与伦理**：验证码绕过的法律边界、合理使用原则、替代方案考虑 + +**实战案例：** 实现一个验证码处理模块，支持图片验证码 OCR 识别和简单滑块验证码的自动处理。 + +--- + +### [09_数据清洗与预处理](./09_数据清洗与预处理.md) + +**学习目标：** +- 掌握常见的数据清洗技术 +- 学会使用正则表达式提取和清洗数据 +- 理解数据去重和合并策略 +- 掌握数据格式标准化方法 +- 学会处理缺失值和异常值 + +**核心内容：** +- **数据清洗概述**：脏数据的常见类型、数据质量评估指标、清洗流程设计 +- **文本清洗**：HTML 标签移除、空白字符处理、特殊字符清理、编码问题处理（Unicode 归一化） +- **正则表达式高级应用**：复杂模式匹配、命名分组提取、替换和转换、性能优化 +- **数据去重与合并**：精确去重 vs 模糊去重、相似度计算（编辑距离/余弦相似度）、多数据源合并策略、冲突解决规则 +- **数据标准化**：日期时间格式统一、数值单位换算、枚举值映射、字段命名规范 + +**实战案例：** 对前几章爬取的数据进行完整的清洗流程，包括文本清洗、格式标准化、去重处理，并输出干净的数据集。 + +--- + +### [10_数据分析与可视化](./10_数据分析与可视化.md) + +**学习目标：** +- 掌握 pandas 进行数据统计分析 +- 学会生成词云展示文本数据 +- 了解常用的数据可视化库（matplotlib/pyecharts） +- 实现简单的数据报告生成 +- 理解爬取数据的分析价值 + +**核心内容：** +- **pandas 数据分析**：DataFrame 基础操作、数据聚合与分组统计、数据透视表、时间序列分析 +- **词云生成**（参考 MediaCrawler）：jieba 中文分词、停用词过滤、wordcloud 库使用、自定义词云形状和配色 +- **数据可视化**：matplotlib 基础图表（折线图/柱状图/饼图）、pyecharts 交互式图表、数据仪表盘设计、图表导出和嵌入 +- **分析报告生成**：Markdown 报告模板、自动化报告生成、图表嵌入、定期报告推送 + +**实战案例：** 对爬取的社交媒体评论数据进行分析，生成词云、情感倾向统计图表，并输出一份完整的数据分析报告。 + +--- + +### [11_进阶综合实战项目](./11_进阶综合实战项目.md) + +**学习目标：** +- 综合运用进阶教程所学的所有技术 +- 实现一个完整的中等复杂度爬虫项目 +- 掌握爬虫项目的完整开发流程 +- 学会项目文档编写和部署 + +**核心内容：** +- **项目需求分析**：目标网站分析、数据需求定义、技术方案选型、风险评估 +- **项目架构设计**：模块划分、类图设计、数据流设计、接口定义 +- **核心功能实现**：登录模块（Cookie 注入 + 扫码登录）、爬取模块（Playwright + 反检测）、数据存储（多后端支持）、代理池集成、日志和监控 +- **项目打包与部署**：代码结构整理、依赖管理（requirements.txt / pyproject.toml）、Docker 容器化部署、定时任务配置 + +**实战案例：** 完成一个类似 MediaCrawler 简化版的社交媒体数据采集工具，支持： +- 多种登录方式 +- 反检测浏览器自动化 +- 代理 IP 轮换 +- 多格式数据存储 +- 词云生成 + +--- + +## 源代码目录结构 + +``` +源代码/爬虫进阶/ +├── 01_工程化爬虫开发规范/ +│ ├── config/ +│ ├── logger_demo.py +│ ├── exception_demo.py +│ └── refactored_crawler/ +├── 02_反爬虫对抗基础_请求伪装/ +│ ├── ua_rotator.py +│ ├── headers_builder.py +│ └── rate_limiter.py +├── 03_代理IP的使用与管理/ +│ ├── proxy_pool/ +│ ├── proxy_checker.py +│ └── proxy_demo.py +├── 04_Playwright浏览器自动化入门/ +│ ├── basic_operations.py +│ ├── wait_strategies.py +│ └── spa_crawler.py +├── 05_Playwright进阶_反检测与性能优化/ +│ ├── stealth_demo.py +│ ├── cdp_mode.py +│ └── performance_optimization.py +├── 06_登录认证_Cookie与Session管理/ +│ ├── cookie_manager.py +│ ├── session_demo.py +│ └── login_state_checker.py +├── 07_登录认证_扫码与短信登录实现/ +│ ├── qrcode_login.py +│ ├── sms_login.py +│ └── login_factory.py +├── 08_验证码识别与处理/ +│ ├── ocr_captcha.py +│ ├── slider_captcha.py +│ └── captcha_service.py +├── 09_数据清洗与预处理/ +│ ├── text_cleaner.py +│ ├── deduplication.py +│ └── data_normalizer.py +├── 10_数据分析与可视化/ +│ ├── pandas_analysis.py +│ ├── wordcloud_generator.py +│ └── chart_demo.py +└── 11_进阶综合实战项目/ + ├── config/ + ├── core/ + ├── login/ + ├── crawler/ + ├── store/ + ├── proxy/ + ├── analysis/ + └── main.py +``` + +--- + +## 教程特色 + +1. **与入门教程无缝衔接**：延续异步编程风格，逐步提升难度 +2. **源于实战项目**：核心技术均来自 MediaCrawler 项目的生产实践 +3. **每章都有代码**：提供完整可运行的示例代码和实战项目 +4. **注重工程规范**：从第一章就强调代码质量和可维护性 +5. **循序渐进**：从请求伪装到浏览器自动化，再到综合实战 + +--- + +## 学习建议 + +1. **按顺序学习**：每章内容都有前后依赖，建议按顺序学习 +2. **动手实践**：每章的实战案例务必亲自完成 +3. **阅读源码**：参考 MediaCrawler 项目源码加深理解 +4. **遵守法规**：爬虫技术仅用于学习研究，请遵守相关法律法规 diff --git a/package-lock.json b/package-lock.json index 3cfe448..89775f0 100644 --- a/package-lock.json +++ b/package-lock.json @@ -5,7 +5,9 @@ "packages": { "": { "devDependencies": { - "vitepress": "^1.3.4" + "mermaid": "^11.12.2", + "vitepress": "^1.3.4", + "vitepress-plugin-mermaid": "^2.0.17" } }, "node_modules/@algolia/autocomplete-core": { @@ -304,6 +306,20 @@ "@algolia/requester-common": "4.24.0" } }, + "node_modules/@antfu/install-pkg": { + "version": "1.1.0", + "resolved": "https://registry.npmmirror.com/@antfu/install-pkg/-/install-pkg-1.1.0.tgz", + "integrity": "sha512-MGQsmw10ZyI+EJo45CdSER4zEb+p31LpDAFp2Z3gkSd1yqVZGi0Ebx++YTEMonJy4oChEMLsxZ64j8FH6sSqtQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "package-manager-detector": "^1.3.0", + "tinyexec": "^1.0.1" + }, + "funding": { + "url": "https://github.com/sponsors/antfu" + } + }, "node_modules/@babel/helper-string-parser": { "version": "7.24.8", "resolved": "https://registry.npmmirror.com/@babel/helper-string-parser/-/helper-string-parser-7.24.8.tgz", @@ -351,6 +367,71 @@ "node": ">=6.9.0" } }, + "node_modules/@braintree/sanitize-url": { + "version": "7.1.1", + "resolved": "https://registry.npmmirror.com/@braintree/sanitize-url/-/sanitize-url-7.1.1.tgz", + "integrity": "sha512-i1L7noDNxtFyL5DmZafWy1wRVhGehQmzZaz1HiN5e7iylJMSZR7ekOV7NsIqa5qBldlLrsKv4HbgFUVlQrz8Mw==", + "dev": true, + "license": "MIT" + }, + "node_modules/@chevrotain/cst-dts-gen": { + "version": "11.0.3", + "resolved": "https://registry.npmmirror.com/@chevrotain/cst-dts-gen/-/cst-dts-gen-11.0.3.tgz", + "integrity": "sha512-BvIKpRLeS/8UbfxXxgC33xOumsacaeCKAjAeLyOn7Pcp95HiRbrpl14S+9vaZLolnbssPIUuiUd8IvgkRyt6NQ==", + "dev": true, + "license": "Apache-2.0", + "dependencies": { + "@chevrotain/gast": "11.0.3", + "@chevrotain/types": "11.0.3", + "lodash-es": "4.17.21" + } + }, + "node_modules/@chevrotain/cst-dts-gen/node_modules/lodash-es": { + "version": "4.17.21", + "resolved": "https://registry.npmmirror.com/lodash-es/-/lodash-es-4.17.21.tgz", + "integrity": "sha512-mKnC+QJ9pWVzv+C4/U3rRsHapFfHvQFoFB92e52xeyGMcX6/OlIl78je1u8vePzYZSkkogMPJ2yjxxsb89cxyw==", + "dev": true, + "license": "MIT" + }, + "node_modules/@chevrotain/gast": { + "version": "11.0.3", + "resolved": "https://registry.npmmirror.com/@chevrotain/gast/-/gast-11.0.3.tgz", + "integrity": "sha512-+qNfcoNk70PyS/uxmj3li5NiECO+2YKZZQMbmjTqRI3Qchu8Hig/Q9vgkHpI3alNjr7M+a2St5pw5w5F6NL5/Q==", + "dev": true, + "license": "Apache-2.0", + "dependencies": { + "@chevrotain/types": "11.0.3", + "lodash-es": "4.17.21" + } + }, + "node_modules/@chevrotain/gast/node_modules/lodash-es": { + "version": "4.17.21", + "resolved": "https://registry.npmmirror.com/lodash-es/-/lodash-es-4.17.21.tgz", + "integrity": "sha512-mKnC+QJ9pWVzv+C4/U3rRsHapFfHvQFoFB92e52xeyGMcX6/OlIl78je1u8vePzYZSkkogMPJ2yjxxsb89cxyw==", + "dev": true, + "license": "MIT" + }, + "node_modules/@chevrotain/regexp-to-ast": { + "version": "11.0.3", + "resolved": "https://registry.npmmirror.com/@chevrotain/regexp-to-ast/-/regexp-to-ast-11.0.3.tgz", + "integrity": "sha512-1fMHaBZxLFvWI067AVbGJav1eRY7N8DDvYCTwGBiE/ytKBgP8azTdgyrKyWZ9Mfh09eHWb5PgTSO8wi7U824RA==", + "dev": true, + "license": "Apache-2.0" + }, + "node_modules/@chevrotain/types": { + "version": "11.0.3", + "resolved": "https://registry.npmmirror.com/@chevrotain/types/-/types-11.0.3.tgz", + "integrity": "sha512-gsiM3G8b58kZC2HaWR50gu6Y1440cHiJ+i3JUvcp/35JchYejb2+5MVeJK0iKThYpAa/P2PYFV4hoi44HD+aHQ==", + "dev": true, + "license": "Apache-2.0" + }, + "node_modules/@chevrotain/utils": { + "version": "11.0.3", + "resolved": "https://registry.npmmirror.com/@chevrotain/utils/-/utils-11.0.3.tgz", + "integrity": "sha512-YslZMgtJUyuMbZ+aKvfF3x1f5liK4mWNxghFRv7jqRR9C3R3fAOGTTKvxXDa2Y1s9zSbcpuO0cAxDYsc9SrXoQ==", + "dev": true, + "license": "Apache-2.0" + }, "node_modules/@docsearch/css": { "version": "3.6.1", "resolved": "https://registry.npmmirror.com/@docsearch/css/-/css-3.6.1.tgz", @@ -767,12 +848,66 @@ "node": ">=12" } }, + "node_modules/@iconify/types": { + "version": "2.0.0", + "resolved": "https://registry.npmmirror.com/@iconify/types/-/types-2.0.0.tgz", + "integrity": "sha512-+wluvCrRhXrhyOmRDJ3q8mux9JkKy5SJ/v8ol2tu4FVjyYvtEzkc/3pK15ET6RKg4b4w4BmTk1+gsCUhf21Ykg==", + "dev": true, + "license": "MIT" + }, + "node_modules/@iconify/utils": { + "version": "3.1.0", + "resolved": "https://registry.npmmirror.com/@iconify/utils/-/utils-3.1.0.tgz", + "integrity": "sha512-Zlzem1ZXhI1iHeeERabLNzBHdOa4VhQbqAcOQaMKuTuyZCpwKbC2R4Dd0Zo3g9EAc+Y4fiarO8HIHRAth7+skw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@antfu/install-pkg": "^1.1.0", + "@iconify/types": "^2.0.0", + "mlly": "^1.8.0" + } + }, "node_modules/@jridgewell/sourcemap-codec": { "version": "1.5.0", "resolved": "https://registry.npmmirror.com/@jridgewell/sourcemap-codec/-/sourcemap-codec-1.5.0.tgz", "integrity": "sha512-gv3ZRaISU3fjPAgNsriBRqGWQL6quFx04YMPW/zD8XMLsU32mhCCbfbO6KZFLjvYpCZ8zyDEgqsgf+PwPaM7GQ==", "dev": true }, + "node_modules/@mermaid-js/mermaid-mindmap": { + "version": "9.3.0", + "resolved": "https://registry.npmmirror.com/@mermaid-js/mermaid-mindmap/-/mermaid-mindmap-9.3.0.tgz", + "integrity": "sha512-IhtYSVBBRYviH1Ehu8gk69pMDF8DSRqXBRDMWrEfHoaMruHeaP2DXA3PBnuwsMaCdPQhlUUcy/7DBLAEIXvCAw==", + "dev": true, + "license": "MIT", + "optional": true, + "dependencies": { + "@braintree/sanitize-url": "^6.0.0", + "cytoscape": "^3.23.0", + "cytoscape-cose-bilkent": "^4.1.0", + "cytoscape-fcose": "^2.1.0", + "d3": "^7.0.0", + "khroma": "^2.0.0", + "non-layered-tidy-tree-layout": "^2.0.2" + } + }, + "node_modules/@mermaid-js/mermaid-mindmap/node_modules/@braintree/sanitize-url": { + "version": "6.0.4", + "resolved": "https://registry.npmmirror.com/@braintree/sanitize-url/-/sanitize-url-6.0.4.tgz", + "integrity": "sha512-s3jaWicZd0pkP0jf5ysyHUI/RE7MHos6qlToFcGWXVp+ykHOy77OUMrfbgJ9it2C5bow7OIQwYYaHjk9XlBQ2A==", + "dev": true, + "license": "MIT", + "optional": true + }, + "node_modules/@mermaid-js/parser": { + "version": "0.6.3", + "resolved": "https://registry.npmmirror.com/@mermaid-js/parser/-/parser-0.6.3.tgz", + "integrity": "sha512-lnjOhe7zyHjc+If7yT4zoedx2vo4sHaTmtkl1+or8BRTnCtDmcTpAjpzDSfCZrshM5bCoz0GyidzadJAH1xobA==", + "dev": true, + "license": "MIT", + "dependencies": { + "langium": "3.3.1" + } + }, "node_modules/@rollup/rollup-android-arm-eabi": { "version": "4.21.1", "resolved": "https://registry.npmmirror.com/@rollup/rollup-android-arm-eabi/-/rollup-android-arm-eabi-4.21.1.tgz", @@ -999,12 +1134,303 @@ "shiki": "1.14.1" } }, + "node_modules/@types/d3": { + "version": "7.4.3", + "resolved": "https://registry.npmmirror.com/@types/d3/-/d3-7.4.3.tgz", + "integrity": "sha512-lZXZ9ckh5R8uiFVt8ogUNf+pIrK4EsWrx2Np75WvF/eTpJ0FMHNhjXk8CKEx/+gpHbNQyJWehbFaTvqmHWB3ww==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/d3-array": "*", + "@types/d3-axis": "*", + "@types/d3-brush": "*", + "@types/d3-chord": "*", + "@types/d3-color": "*", + "@types/d3-contour": "*", + "@types/d3-delaunay": "*", + "@types/d3-dispatch": "*", + "@types/d3-drag": "*", + "@types/d3-dsv": "*", + "@types/d3-ease": "*", + "@types/d3-fetch": "*", + "@types/d3-force": "*", + "@types/d3-format": "*", + "@types/d3-geo": "*", + "@types/d3-hierarchy": "*", + "@types/d3-interpolate": "*", + "@types/d3-path": "*", + "@types/d3-polygon": "*", + "@types/d3-quadtree": "*", + "@types/d3-random": "*", + "@types/d3-scale": "*", + "@types/d3-scale-chromatic": "*", + "@types/d3-selection": "*", + "@types/d3-shape": "*", + "@types/d3-time": "*", + "@types/d3-time-format": "*", + "@types/d3-timer": "*", + "@types/d3-transition": "*", + "@types/d3-zoom": "*" + } + }, + "node_modules/@types/d3-array": { + "version": "3.2.2", + "resolved": "https://registry.npmmirror.com/@types/d3-array/-/d3-array-3.2.2.tgz", + "integrity": "sha512-hOLWVbm7uRza0BYXpIIW5pxfrKe0W+D5lrFiAEYR+pb6w3N2SwSMaJbXdUfSEv+dT4MfHBLtn5js0LAWaO6otw==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-axis": { + "version": "3.0.6", + "resolved": "https://registry.npmmirror.com/@types/d3-axis/-/d3-axis-3.0.6.tgz", + "integrity": "sha512-pYeijfZuBd87T0hGn0FO1vQ/cgLk6E1ALJjfkC0oJ8cbwkZl3TpgS8bVBLZN+2jjGgg38epgxb2zmoGtSfvgMw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/d3-selection": "*" + } + }, + "node_modules/@types/d3-brush": { + "version": "3.0.6", + "resolved": "https://registry.npmmirror.com/@types/d3-brush/-/d3-brush-3.0.6.tgz", + "integrity": "sha512-nH60IZNNxEcrh6L1ZSMNA28rj27ut/2ZmI3r96Zd+1jrZD++zD3LsMIjWlvg4AYrHn/Pqz4CF3veCxGjtbqt7A==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/d3-selection": "*" + } + }, + "node_modules/@types/d3-chord": { + "version": "3.0.6", + "resolved": "https://registry.npmmirror.com/@types/d3-chord/-/d3-chord-3.0.6.tgz", + "integrity": "sha512-LFYWWd8nwfwEmTZG9PfQxd17HbNPksHBiJHaKuY1XeqscXacsS2tyoo6OdRsjf+NQYeB6XrNL3a25E3gH69lcg==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-color": { + "version": "3.1.3", + "resolved": "https://registry.npmmirror.com/@types/d3-color/-/d3-color-3.1.3.tgz", + "integrity": "sha512-iO90scth9WAbmgv7ogoq57O9YpKmFBbmoEoCHDB2xMBY0+/KVrqAaCDyCE16dUspeOvIxFFRI+0sEtqDqy2b4A==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-contour": { + "version": "3.0.6", + "resolved": "https://registry.npmmirror.com/@types/d3-contour/-/d3-contour-3.0.6.tgz", + "integrity": "sha512-BjzLgXGnCWjUSYGfH1cpdo41/hgdWETu4YxpezoztawmqsvCeep+8QGfiY6YbDvfgHz/DkjeIkkZVJavB4a3rg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/d3-array": "*", + "@types/geojson": "*" + } + }, + "node_modules/@types/d3-delaunay": { + "version": "6.0.4", + "resolved": "https://registry.npmmirror.com/@types/d3-delaunay/-/d3-delaunay-6.0.4.tgz", + "integrity": "sha512-ZMaSKu4THYCU6sV64Lhg6qjf1orxBthaC161plr5KuPHo3CNm8DTHiLw/5Eq2b6TsNP0W0iJrUOFscY6Q450Hw==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-dispatch": { + "version": "3.0.7", + "resolved": "https://registry.npmmirror.com/@types/d3-dispatch/-/d3-dispatch-3.0.7.tgz", + "integrity": "sha512-5o9OIAdKkhN1QItV2oqaE5KMIiXAvDWBDPrD85e58Qlz1c1kI/J0NcqbEG88CoTwJrYe7ntUCVfeUl2UJKbWgA==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-drag": { + "version": "3.0.7", + "resolved": "https://registry.npmmirror.com/@types/d3-drag/-/d3-drag-3.0.7.tgz", + "integrity": "sha512-HE3jVKlzU9AaMazNufooRJ5ZpWmLIoc90A37WU2JMmeq28w1FQqCZswHZ3xR+SuxYftzHq6WU6KJHvqxKzTxxQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/d3-selection": "*" + } + }, + "node_modules/@types/d3-dsv": { + "version": "3.0.7", + "resolved": "https://registry.npmmirror.com/@types/d3-dsv/-/d3-dsv-3.0.7.tgz", + "integrity": "sha512-n6QBF9/+XASqcKK6waudgL0pf/S5XHPPI8APyMLLUHd8NqouBGLsU8MgtO7NINGtPBtk9Kko/W4ea0oAspwh9g==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-ease": { + "version": "3.0.2", + "resolved": "https://registry.npmmirror.com/@types/d3-ease/-/d3-ease-3.0.2.tgz", + "integrity": "sha512-NcV1JjO5oDzoK26oMzbILE6HW7uVXOHLQvHshBUW4UMdZGfiY6v5BeQwh9a9tCzv+CeefZQHJt5SRgK154RtiA==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-fetch": { + "version": "3.0.7", + "resolved": "https://registry.npmmirror.com/@types/d3-fetch/-/d3-fetch-3.0.7.tgz", + "integrity": "sha512-fTAfNmxSb9SOWNB9IoG5c8Hg6R+AzUHDRlsXsDZsNp6sxAEOP0tkP3gKkNSO/qmHPoBFTxNrjDprVHDQDvo5aA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/d3-dsv": "*" + } + }, + "node_modules/@types/d3-force": { + "version": "3.0.10", + "resolved": "https://registry.npmmirror.com/@types/d3-force/-/d3-force-3.0.10.tgz", + "integrity": "sha512-ZYeSaCF3p73RdOKcjj+swRlZfnYpK1EbaDiYICEEp5Q6sUiqFaFQ9qgoshp5CzIyyb/yD09kD9o2zEltCexlgw==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-format": { + "version": "3.0.4", + "resolved": "https://registry.npmmirror.com/@types/d3-format/-/d3-format-3.0.4.tgz", + "integrity": "sha512-fALi2aI6shfg7vM5KiR1wNJnZ7r6UuggVqtDA+xiEdPZQwy/trcQaHnwShLuLdta2rTymCNpxYTiMZX/e09F4g==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-geo": { + "version": "3.1.0", + "resolved": "https://registry.npmmirror.com/@types/d3-geo/-/d3-geo-3.1.0.tgz", + "integrity": "sha512-856sckF0oP/diXtS4jNsiQw/UuK5fQG8l/a9VVLeSouf1/PPbBE1i1W852zVwKwYCBkFJJB7nCFTbk6UMEXBOQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/geojson": "*" + } + }, + "node_modules/@types/d3-hierarchy": { + "version": "3.1.7", + "resolved": "https://registry.npmmirror.com/@types/d3-hierarchy/-/d3-hierarchy-3.1.7.tgz", + "integrity": "sha512-tJFtNoYBtRtkNysX1Xq4sxtjK8YgoWUNpIiUee0/jHGRwqvzYxkq0hGVbbOGSz+JgFxxRu4K8nb3YpG3CMARtg==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-interpolate": { + "version": "3.0.4", + "resolved": "https://registry.npmmirror.com/@types/d3-interpolate/-/d3-interpolate-3.0.4.tgz", + "integrity": "sha512-mgLPETlrpVV1YRJIglr4Ez47g7Yxjl1lj7YKsiMCb27VJH9W8NVM6Bb9d8kkpG/uAQS5AmbA48q2IAolKKo1MA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/d3-color": "*" + } + }, + "node_modules/@types/d3-path": { + "version": "3.1.1", + "resolved": "https://registry.npmmirror.com/@types/d3-path/-/d3-path-3.1.1.tgz", + "integrity": "sha512-VMZBYyQvbGmWyWVea0EHs/BwLgxc+MKi1zLDCONksozI4YJMcTt8ZEuIR4Sb1MMTE8MMW49v0IwI5+b7RmfWlg==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-polygon": { + "version": "3.0.2", + "resolved": "https://registry.npmmirror.com/@types/d3-polygon/-/d3-polygon-3.0.2.tgz", + "integrity": "sha512-ZuWOtMaHCkN9xoeEMr1ubW2nGWsp4nIql+OPQRstu4ypeZ+zk3YKqQT0CXVe/PYqrKpZAi+J9mTs05TKwjXSRA==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-quadtree": { + "version": "3.0.6", + "resolved": "https://registry.npmmirror.com/@types/d3-quadtree/-/d3-quadtree-3.0.6.tgz", + "integrity": "sha512-oUzyO1/Zm6rsxKRHA1vH0NEDG58HrT5icx/azi9MF1TWdtttWl0UIUsjEQBBh+SIkrpd21ZjEv7ptxWys1ncsg==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-random": { + "version": "3.0.3", + "resolved": "https://registry.npmmirror.com/@types/d3-random/-/d3-random-3.0.3.tgz", + "integrity": "sha512-Imagg1vJ3y76Y2ea0871wpabqp613+8/r0mCLEBfdtqC7xMSfj9idOnmBYyMoULfHePJyxMAw3nWhJxzc+LFwQ==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-scale": { + "version": "4.0.9", + "resolved": "https://registry.npmmirror.com/@types/d3-scale/-/d3-scale-4.0.9.tgz", + "integrity": "sha512-dLmtwB8zkAeO/juAMfnV+sItKjlsw2lKdZVVy6LRr0cBmegxSABiLEpGVmSJJ8O08i4+sGR6qQtb6WtuwJdvVw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/d3-time": "*" + } + }, + "node_modules/@types/d3-scale-chromatic": { + "version": "3.1.0", + "resolved": "https://registry.npmmirror.com/@types/d3-scale-chromatic/-/d3-scale-chromatic-3.1.0.tgz", + "integrity": "sha512-iWMJgwkK7yTRmWqRB5plb1kadXyQ5Sj8V/zYlFGMUBbIPKQScw+Dku9cAAMgJG+z5GYDoMjWGLVOvjghDEFnKQ==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-selection": { + "version": "3.0.11", + "resolved": "https://registry.npmmirror.com/@types/d3-selection/-/d3-selection-3.0.11.tgz", + "integrity": "sha512-bhAXu23DJWsrI45xafYpkQ4NtcKMwWnAC/vKrd2l+nxMFuvOT3XMYTIj2opv8vq8AO5Yh7Qac/nSeP/3zjTK0w==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-shape": { + "version": "3.1.8", + "resolved": "https://registry.npmmirror.com/@types/d3-shape/-/d3-shape-3.1.8.tgz", + "integrity": "sha512-lae0iWfcDeR7qt7rA88BNiqdvPS5pFVPpo5OfjElwNaT2yyekbM0C9vK+yqBqEmHr6lDkRnYNoTBYlAgJa7a4w==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/d3-path": "*" + } + }, + "node_modules/@types/d3-time": { + "version": "3.0.4", + "resolved": "https://registry.npmmirror.com/@types/d3-time/-/d3-time-3.0.4.tgz", + "integrity": "sha512-yuzZug1nkAAaBlBBikKZTgzCeA+k1uy4ZFwWANOfKw5z5LRhV0gNA7gNkKm7HoK+HRN0wX3EkxGk0fpbWhmB7g==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-time-format": { + "version": "4.0.3", + "resolved": "https://registry.npmmirror.com/@types/d3-time-format/-/d3-time-format-4.0.3.tgz", + "integrity": "sha512-5xg9rC+wWL8kdDj153qZcsJ0FWiFt0J5RB6LYUNZjwSnesfblqrI/bJ1wBdJ8OQfncgbJG5+2F+qfqnqyzYxyg==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-timer": { + "version": "3.0.2", + "resolved": "https://registry.npmmirror.com/@types/d3-timer/-/d3-timer-3.0.2.tgz", + "integrity": "sha512-Ps3T8E8dZDam6fUyNiMkekK3XUsaUEik+idO9/YjPtfj2qruF8tFBXS7XhtE4iIXBLxhmLjP3SXpLhVf21I9Lw==", + "dev": true, + "license": "MIT" + }, + "node_modules/@types/d3-transition": { + "version": "3.0.9", + "resolved": "https://registry.npmmirror.com/@types/d3-transition/-/d3-transition-3.0.9.tgz", + "integrity": "sha512-uZS5shfxzO3rGlu0cC3bjmMFKsXv+SmZZcgp0KD22ts4uGXp5EVYGzu/0YdwZeKmddhcAccYtREJKkPfXkZuCg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/d3-selection": "*" + } + }, + "node_modules/@types/d3-zoom": { + "version": "3.0.8", + "resolved": "https://registry.npmmirror.com/@types/d3-zoom/-/d3-zoom-3.0.8.tgz", + "integrity": "sha512-iqMC4/YlFCSlO8+2Ii1GGGliCAY4XdeG748w5vQUbevlbDu0zSjH/+jojorQVBK/se0j6DUFNPBGSqD3YWYnDw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/d3-interpolate": "*", + "@types/d3-selection": "*" + } + }, "node_modules/@types/estree": { "version": "1.0.5", "resolved": "https://registry.npmmirror.com/@types/estree/-/estree-1.0.5.tgz", "integrity": "sha512-/kYRxGDLWzHOB7q+wtSUQlFrtcdUccpfy+X+9iMBpHK8QLLhx2wIPYuS5DYtR9Wa/YlZAbIovy7qVdB1Aq6Lyw==", "dev": true }, + "node_modules/@types/geojson": { + "version": "7946.0.16", + "resolved": "https://registry.npmmirror.com/@types/geojson/-/geojson-7946.0.16.tgz", + "integrity": "sha512-6C8nqWur3j98U6+lXDfTUWIfgvZU+EumvpHKcYjujKH7woYyLj2sUmff0tRhrqM7BohUw7Pz3ZB1jj2gW9Fvmg==", + "dev": true, + "license": "MIT" + }, "node_modules/@types/hast": { "version": "3.0.4", "resolved": "https://registry.npmmirror.com/@types/hast/-/hast-3.0.4.tgz", @@ -1036,6 +1462,14 @@ "integrity": "sha512-RGdgjQUZba5p6QEFAVx2OGb8rQDL/cPRG7GiedRzMcJ1tYnUANBncjbSB1NRGwbvjcPeikRABz2nshyPk1bhWg==", "dev": true }, + "node_modules/@types/trusted-types": { + "version": "2.0.7", + "resolved": "https://registry.npmmirror.com/@types/trusted-types/-/trusted-types-2.0.7.tgz", + "integrity": "sha512-ScaPdn1dQczgbl0QFTeTOmVHFULt394XJgOQNoyVhZ6r2vLnMLJfBPd53SB52T/3G36VI1/g2MZaX0cwDuXsfw==", + "dev": true, + "license": "MIT", + "optional": true + }, "node_modules/@types/unist": { "version": "3.0.3", "resolved": "https://registry.npmmirror.com/@types/unist/-/unist-3.0.3.tgz", @@ -1374,6 +1808,19 @@ } } }, + "node_modules/acorn": { + "version": "8.15.0", + "resolved": "https://registry.npmmirror.com/acorn/-/acorn-8.15.0.tgz", + "integrity": "sha512-NZyJarBfL7nWwIq+FDL6Zp/yHEhePMNnnJ0y3qfieCrmNvYct8uvtiV41UvlSe6apAfk0fY1FbWx+NwfmpvtTg==", + "dev": true, + "license": "MIT", + "bin": { + "acorn": "bin/acorn" + }, + "engines": { + "node": ">=0.4.0" + } + }, "node_modules/algoliasearch": { "version": "4.24.0", "resolved": "https://registry.npmmirror.com/algoliasearch/-/algoliasearch-4.24.0.tgz", @@ -1445,6 +1892,58 @@ "url": "https://github.com/sponsors/antfu" } }, + "node_modules/chevrotain": { + "version": "11.0.3", + "resolved": "https://registry.npmmirror.com/chevrotain/-/chevrotain-11.0.3.tgz", + "integrity": "sha512-ci2iJH6LeIkvP9eJW6gpueU8cnZhv85ELY8w8WiFtNjMHA5ad6pQLaJo9mEly/9qUyCpvqX8/POVUTf18/HFdw==", + "dev": true, + "license": "Apache-2.0", + "dependencies": { + "@chevrotain/cst-dts-gen": "11.0.3", + "@chevrotain/gast": "11.0.3", + "@chevrotain/regexp-to-ast": "11.0.3", + "@chevrotain/types": "11.0.3", + "@chevrotain/utils": "11.0.3", + "lodash-es": "4.17.21" + } + }, + "node_modules/chevrotain-allstar": { + "version": "0.3.1", + "resolved": "https://registry.npmmirror.com/chevrotain-allstar/-/chevrotain-allstar-0.3.1.tgz", + "integrity": "sha512-b7g+y9A0v4mxCW1qUhf3BSVPg+/NvGErk/dOkrDaHA0nQIQGAtrOjlX//9OQtRlSCy+x9rfB5N8yC71lH1nvMw==", + "dev": true, + "license": "MIT", + "dependencies": { + "lodash-es": "^4.17.21" + }, + "peerDependencies": { + "chevrotain": "^11.0.0" + } + }, + "node_modules/chevrotain/node_modules/lodash-es": { + "version": "4.17.21", + "resolved": "https://registry.npmmirror.com/lodash-es/-/lodash-es-4.17.21.tgz", + "integrity": "sha512-mKnC+QJ9pWVzv+C4/U3rRsHapFfHvQFoFB92e52xeyGMcX6/OlIl78je1u8vePzYZSkkogMPJ2yjxxsb89cxyw==", + "dev": true, + "license": "MIT" + }, + "node_modules/commander": { + "version": "7.2.0", + "resolved": "https://registry.npmmirror.com/commander/-/commander-7.2.0.tgz", + "integrity": "sha512-QrWXB+ZQSVPmIWIhtEO9H+gwHaMGYiF5ChvoJ+K9ZGHG/sVsa6yiesAD1GC/x46sET00Xlwo1u49RVVVzvcSkw==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">= 10" + } + }, + "node_modules/confbox": { + "version": "0.1.8", + "resolved": "https://registry.npmmirror.com/confbox/-/confbox-0.1.8.tgz", + "integrity": "sha512-RMtmw0iFkeR4YV+fUOSucriAQNb9g8zFR52MWCtl+cCZOFRNL6zeB395vPzFhEjjn4fMxXudmELnl/KF/WrK6w==", + "dev": true, + "license": "MIT" + }, "node_modules/copy-anything": { "version": "3.0.5", "resolved": "https://registry.npmmirror.com/copy-anything/-/copy-anything-3.0.5.tgz", @@ -1460,110 +1959,782 @@ "url": "https://github.com/sponsors/mesqueeb" } }, + "node_modules/cose-base": { + "version": "1.0.3", + "resolved": "https://registry.npmmirror.com/cose-base/-/cose-base-1.0.3.tgz", + "integrity": "sha512-s9whTXInMSgAp/NVXVNuVxVKzGH2qck3aQlVHxDCdAEPgtMKwc4Wq6/QKhgdEdgbLSi9rBTAcPoRa6JpiG4ksg==", + "dev": true, + "license": "MIT", + "dependencies": { + "layout-base": "^1.0.0" + } + }, "node_modules/csstype": { "version": "3.1.3", "resolved": "https://registry.npmmirror.com/csstype/-/csstype-3.1.3.tgz", "integrity": "sha512-M1uQkMl8rQK/szD0LNhtqxIPLpimGm8sOBwU7lLnCpSbTyY3yeU1Vc7l4KT5zT4s/yOxHH5O7tIuuLOCnLADRw==", "dev": true }, - "node_modules/entities": { - "version": "4.5.0", - "resolved": "https://registry.npmmirror.com/entities/-/entities-4.5.0.tgz", - "integrity": "sha512-V0hjH4dGPh9Ao5p0MoRY6BVqtwCjhz6vI5LT8AJ55H+4g9/4vbHx1I54fS0XuclLhDHArPQCiMjDxjaL8fPxhw==", + "node_modules/cytoscape": { + "version": "3.33.1", + "resolved": "https://registry.npmmirror.com/cytoscape/-/cytoscape-3.33.1.tgz", + "integrity": "sha512-iJc4TwyANnOGR1OmWhsS9ayRS3s+XQ185FmuHObThD+5AeJCakAAbWv8KimMTt08xCCLNgneQwFp+JRJOr9qGQ==", "dev": true, + "license": "MIT", "engines": { - "node": ">=0.12" - }, - "funding": { - "url": "https://github.com/fb55/entities?sponsor=1" + "node": ">=0.10" } }, - "node_modules/esbuild": { - "version": "0.21.5", - "resolved": "https://registry.npmmirror.com/esbuild/-/esbuild-0.21.5.tgz", - "integrity": "sha512-mg3OPMV4hXywwpoDxu3Qda5xCKQi+vCTZq8S9J/EpkhB2HzKXq4SNFZE3+NK93JYxc8VMSep+lOUSC/RVKaBqw==", + "node_modules/cytoscape-cose-bilkent": { + "version": "4.1.0", + "resolved": "https://registry.npmmirror.com/cytoscape-cose-bilkent/-/cytoscape-cose-bilkent-4.1.0.tgz", + "integrity": "sha512-wgQlVIUJF13Quxiv5e1gstZ08rnZj2XaLHGoFMYXz7SkNfCDOOteKBE6SYRfA9WxxI/iBc3ajfDoc6hb/MRAHQ==", "dev": true, - "hasInstallScript": true, - "bin": { - "esbuild": "bin/esbuild" - }, - "engines": { - "node": ">=12" + "license": "MIT", + "dependencies": { + "cose-base": "^1.0.0" }, - "optionalDependencies": { - "@esbuild/aix-ppc64": "0.21.5", - "@esbuild/android-arm": "0.21.5", - "@esbuild/android-arm64": "0.21.5", - "@esbuild/android-x64": "0.21.5", - "@esbuild/darwin-arm64": "0.21.5", - "@esbuild/darwin-x64": "0.21.5", - "@esbuild/freebsd-arm64": "0.21.5", - "@esbuild/freebsd-x64": "0.21.5", - "@esbuild/linux-arm": "0.21.5", - "@esbuild/linux-arm64": "0.21.5", - "@esbuild/linux-ia32": "0.21.5", - "@esbuild/linux-loong64": "0.21.5", - "@esbuild/linux-mips64el": "0.21.5", - "@esbuild/linux-ppc64": "0.21.5", - "@esbuild/linux-riscv64": "0.21.5", - "@esbuild/linux-s390x": "0.21.5", - "@esbuild/linux-x64": "0.21.5", - "@esbuild/netbsd-x64": "0.21.5", - "@esbuild/openbsd-x64": "0.21.5", - "@esbuild/sunos-x64": "0.21.5", - "@esbuild/win32-arm64": "0.21.5", - "@esbuild/win32-ia32": "0.21.5", - "@esbuild/win32-x64": "0.21.5" + "peerDependencies": { + "cytoscape": "^3.2.0" } }, - "node_modules/estree-walker": { - "version": "2.0.2", - "resolved": "https://registry.npmmirror.com/estree-walker/-/estree-walker-2.0.2.tgz", - "integrity": "sha512-Rfkk/Mp/DL7JVje3u18FxFujQlTNR2q6QfMSMB7AvCBx91NGj/ba3kCfza0f6dVDbw7YlRf/nDrn7pQrCCyQ/w==", - "dev": true - }, - "node_modules/focus-trap": { - "version": "7.5.4", - "resolved": "https://registry.npmmirror.com/focus-trap/-/focus-trap-7.5.4.tgz", - "integrity": "sha512-N7kHdlgsO/v+iD/dMoJKtsSqs5Dz/dXZVebRgJw23LDk+jMi/974zyiOYDziY2JPp8xivq9BmUGwIJMiuSBi7w==", + "node_modules/cytoscape-fcose": { + "version": "2.2.0", + "resolved": "https://registry.npmmirror.com/cytoscape-fcose/-/cytoscape-fcose-2.2.0.tgz", + "integrity": "sha512-ki1/VuRIHFCzxWNrsshHYPs6L7TvLu3DL+TyIGEsRcvVERmxokbf5Gdk7mFxZnTdiGtnA4cfSmjZJMviqSuZrQ==", "dev": true, + "license": "MIT", "dependencies": { - "tabbable": "^6.2.0" + "cose-base": "^2.2.0" + }, + "peerDependencies": { + "cytoscape": "^3.2.0" } }, - "node_modules/fsevents": { - "version": "2.3.3", - "resolved": "https://registry.npmmirror.com/fsevents/-/fsevents-2.3.3.tgz", - "integrity": "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==", + "node_modules/cytoscape-fcose/node_modules/cose-base": { + "version": "2.2.0", + "resolved": "https://registry.npmmirror.com/cose-base/-/cose-base-2.2.0.tgz", + "integrity": "sha512-AzlgcsCbUMymkADOJtQm3wO9S3ltPfYOFD5033keQn9NJzIbtnZj+UdBJe7DYml/8TdbtHJW3j58SOnKhWY/5g==", "dev": true, - "hasInstallScript": true, - "optional": true, - "os": [ - "darwin" - ], - "engines": { - "node": "^8.16.0 || ^10.6.0 || >=11.0.0" + "license": "MIT", + "dependencies": { + "layout-base": "^2.0.0" } }, - "node_modules/hookable": { - "version": "5.5.3", - "resolved": "https://registry.npmmirror.com/hookable/-/hookable-5.5.3.tgz", - "integrity": "sha512-Yc+BQe8SvoXH1643Qez1zqLRmbA5rCL+sSmk6TVos0LWVfNIB7PGncdlId77WzLGSIB5KaWgTaNTs2lNVEI6VQ==", - "dev": true + "node_modules/cytoscape-fcose/node_modules/layout-base": { + "version": "2.0.1", + "resolved": "https://registry.npmmirror.com/layout-base/-/layout-base-2.0.1.tgz", + "integrity": "sha512-dp3s92+uNI1hWIpPGH3jK2kxE2lMjdXdr+DH8ynZHpd6PUlH6x6cbuXnoMmiNumznqaNO31xu9e79F0uuZ0JFg==", + "dev": true, + "license": "MIT" }, - "node_modules/is-what": { - "version": "4.1.16", - "resolved": "https://registry.npmmirror.com/is-what/-/is-what-4.1.16.tgz", - "integrity": "sha512-ZhMwEosbFJkA0YhFnNDgTM4ZxDRsS6HqTo7qsZM08fehyRYIYa0yHu5R6mgo1n/8MgaPBXiPimPD77baVFYg+A==", + "node_modules/d3": { + "version": "7.9.0", + "resolved": "https://registry.npmmirror.com/d3/-/d3-7.9.0.tgz", + "integrity": "sha512-e1U46jVP+w7Iut8Jt8ri1YsPOvFpg46k+K8TpCb0P+zjCkjkPnV7WzfDJzMHy1LnA+wj5pLT1wjO901gLXeEhA==", "dev": true, - "engines": { - "node": ">=12.13" + "license": "ISC", + "dependencies": { + "d3-array": "3", + "d3-axis": "3", + "d3-brush": "3", + "d3-chord": "3", + "d3-color": "3", + "d3-contour": "4", + "d3-delaunay": "6", + "d3-dispatch": "3", + "d3-drag": "3", + "d3-dsv": "3", + "d3-ease": "3", + "d3-fetch": "3", + "d3-force": "3", + "d3-format": "3", + "d3-geo": "3", + "d3-hierarchy": "3", + "d3-interpolate": "3", + "d3-path": "3", + "d3-polygon": "3", + "d3-quadtree": "3", + "d3-random": "3", + "d3-scale": "4", + "d3-scale-chromatic": "3", + "d3-selection": "3", + "d3-shape": "3", + "d3-time": "3", + "d3-time-format": "4", + "d3-timer": "3", + "d3-transition": "3", + "d3-zoom": "3" }, - "funding": { - "url": "https://github.com/sponsors/mesqueeb" + "engines": { + "node": ">=12" } }, - "node_modules/magic-string": { + "node_modules/d3-array": { + "version": "3.2.4", + "resolved": "https://registry.npmmirror.com/d3-array/-/d3-array-3.2.4.tgz", + "integrity": "sha512-tdQAmyA18i4J7wprpYq8ClcxZy3SC31QMeByyCFyRt7BVHdREQZ5lpzoe5mFEYZUWe+oq8HBvk9JjpibyEV4Jg==", + "dev": true, + "license": "ISC", + "dependencies": { + "internmap": "1 - 2" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-axis": { + "version": "3.0.0", + "resolved": "https://registry.npmmirror.com/d3-axis/-/d3-axis-3.0.0.tgz", + "integrity": "sha512-IH5tgjV4jE/GhHkRV0HiVYPDtvfjHQlQfJHs0usq7M30XcSBvOotpmH1IgkcXsO/5gEQZD43B//fc7SRT5S+xw==", + "dev": true, + "license": "ISC", + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-brush": { + "version": "3.0.0", + "resolved": "https://registry.npmmirror.com/d3-brush/-/d3-brush-3.0.0.tgz", + "integrity": "sha512-ALnjWlVYkXsVIGlOsuWH1+3udkYFI48Ljihfnh8FZPF2QS9o+PzGLBslO0PjzVoHLZ2KCVgAM8NVkXPJB2aNnQ==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-dispatch": "1 - 3", + "d3-drag": "2 - 3", + "d3-interpolate": "1 - 3", + "d3-selection": "3", + "d3-transition": "3" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-chord": { + "version": "3.0.1", + "resolved": "https://registry.npmmirror.com/d3-chord/-/d3-chord-3.0.1.tgz", + "integrity": "sha512-VE5S6TNa+j8msksl7HwjxMHDM2yNK3XCkusIlpX5kwauBfXuyLAtNg9jCp/iHH61tgI4sb6R/EIMWCqEIdjT/g==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-path": "1 - 3" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-color": { + "version": "3.1.0", + "resolved": "https://registry.npmmirror.com/d3-color/-/d3-color-3.1.0.tgz", + "integrity": "sha512-zg/chbXyeBtMQ1LbD/WSoW2DpC3I0mpmPdW+ynRTj/x2DAWYrIY7qeZIHidozwV24m4iavr15lNwIwLxRmOxhA==", + "dev": true, + "license": "ISC", + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-contour": { + "version": "4.0.2", + "resolved": "https://registry.npmmirror.com/d3-contour/-/d3-contour-4.0.2.tgz", + "integrity": "sha512-4EzFTRIikzs47RGmdxbeUvLWtGedDUNkTcmzoeyg4sP/dvCexO47AaQL7VKy/gul85TOxw+IBgA8US2xwbToNA==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-array": "^3.2.0" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-delaunay": { + "version": "6.0.4", + "resolved": "https://registry.npmmirror.com/d3-delaunay/-/d3-delaunay-6.0.4.tgz", + "integrity": "sha512-mdjtIZ1XLAM8bm/hx3WwjfHt6Sggek7qH043O8KEjDXN40xi3vx/6pYSVTwLjEgiXQTbvaouWKynLBiUZ6SK6A==", + "dev": true, + "license": "ISC", + "dependencies": { + "delaunator": "5" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-dispatch": { + "version": "3.0.1", + "resolved": "https://registry.npmmirror.com/d3-dispatch/-/d3-dispatch-3.0.1.tgz", + "integrity": "sha512-rzUyPU/S7rwUflMyLc1ETDeBj0NRuHKKAcvukozwhshr6g6c5d8zh4c2gQjY2bZ0dXeGLWc1PF174P2tVvKhfg==", + "dev": true, + "license": "ISC", + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-drag": { + "version": "3.0.0", + "resolved": "https://registry.npmmirror.com/d3-drag/-/d3-drag-3.0.0.tgz", + "integrity": "sha512-pWbUJLdETVA8lQNJecMxoXfH6x+mO2UQo8rSmZ+QqxcbyA3hfeprFgIT//HW2nlHChWeIIMwS2Fq+gEARkhTkg==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-dispatch": "1 - 3", + "d3-selection": "3" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-dsv": { + "version": "3.0.1", + "resolved": "https://registry.npmmirror.com/d3-dsv/-/d3-dsv-3.0.1.tgz", + "integrity": "sha512-UG6OvdI5afDIFP9w4G0mNq50dSOsXHJaRE8arAS5o9ApWnIElp8GZw1Dun8vP8OyHOZ/QJUKUJwxiiCCnUwm+Q==", + "dev": true, + "license": "ISC", + "dependencies": { + "commander": "7", + "iconv-lite": "0.6", + "rw": "1" + }, + "bin": { + "csv2json": "bin/dsv2json.js", + "csv2tsv": "bin/dsv2dsv.js", + "dsv2dsv": "bin/dsv2dsv.js", + "dsv2json": "bin/dsv2json.js", + "json2csv": "bin/json2dsv.js", + "json2dsv": "bin/json2dsv.js", + "json2tsv": "bin/json2dsv.js", + "tsv2csv": "bin/dsv2dsv.js", + "tsv2json": "bin/dsv2json.js" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-ease": { + "version": "3.0.1", + "resolved": "https://registry.npmmirror.com/d3-ease/-/d3-ease-3.0.1.tgz", + "integrity": "sha512-wR/XK3D3XcLIZwpbvQwQ5fK+8Ykds1ip7A2Txe0yxncXSdq1L9skcG7blcedkOX+ZcgxGAmLX1FrRGbADwzi0w==", + "dev": true, + "license": "BSD-3-Clause", + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-fetch": { + "version": "3.0.1", + "resolved": "https://registry.npmmirror.com/d3-fetch/-/d3-fetch-3.0.1.tgz", + "integrity": "sha512-kpkQIM20n3oLVBKGg6oHrUchHM3xODkTzjMoj7aWQFq5QEM+R6E4WkzT5+tojDY7yjez8KgCBRoj4aEr99Fdqw==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-dsv": "1 - 3" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-force": { + "version": "3.0.0", + "resolved": "https://registry.npmmirror.com/d3-force/-/d3-force-3.0.0.tgz", + "integrity": "sha512-zxV/SsA+U4yte8051P4ECydjD/S+qeYtnaIyAs9tgHCqfguma/aAQDjo85A9Z6EKhBirHRJHXIgJUlffT4wdLg==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-dispatch": "1 - 3", + "d3-quadtree": "1 - 3", + "d3-timer": "1 - 3" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-format": { + "version": "3.1.2", + "resolved": "https://registry.npmmirror.com/d3-format/-/d3-format-3.1.2.tgz", + "integrity": "sha512-AJDdYOdnyRDV5b6ArilzCPPwc1ejkHcoyFarqlPqT7zRYjhavcT3uSrqcMvsgh2CgoPbK3RCwyHaVyxYcP2Arg==", + "dev": true, + "license": "ISC", + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-geo": { + "version": "3.1.1", + "resolved": "https://registry.npmmirror.com/d3-geo/-/d3-geo-3.1.1.tgz", + "integrity": "sha512-637ln3gXKXOwhalDzinUgY83KzNWZRKbYubaG+fGVuc/dxO64RRljtCTnf5ecMyE1RIdtqpkVcq0IbtU2S8j2Q==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-array": "2.5.0 - 3" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-hierarchy": { + "version": "3.1.2", + "resolved": "https://registry.npmmirror.com/d3-hierarchy/-/d3-hierarchy-3.1.2.tgz", + "integrity": "sha512-FX/9frcub54beBdugHjDCdikxThEqjnR93Qt7PvQTOHxyiNCAlvMrHhclk3cD5VeAaq9fxmfRp+CnWw9rEMBuA==", + "dev": true, + "license": "ISC", + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-interpolate": { + "version": "3.0.1", + "resolved": "https://registry.npmmirror.com/d3-interpolate/-/d3-interpolate-3.0.1.tgz", + "integrity": "sha512-3bYs1rOD33uo8aqJfKP3JWPAibgw8Zm2+L9vBKEHJ2Rg+viTR7o5Mmv5mZcieN+FRYaAOWX5SJATX6k1PWz72g==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-color": "1 - 3" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-path": { + "version": "3.1.0", + "resolved": "https://registry.npmmirror.com/d3-path/-/d3-path-3.1.0.tgz", + "integrity": "sha512-p3KP5HCf/bvjBSSKuXid6Zqijx7wIfNW+J/maPs+iwR35at5JCbLUT0LzF1cnjbCHWhqzQTIN2Jpe8pRebIEFQ==", + "dev": true, + "license": "ISC", + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-polygon": { + "version": "3.0.1", + "resolved": "https://registry.npmmirror.com/d3-polygon/-/d3-polygon-3.0.1.tgz", + "integrity": "sha512-3vbA7vXYwfe1SYhED++fPUQlWSYTTGmFmQiany/gdbiWgU/iEyQzyymwL9SkJjFFuCS4902BSzewVGsHHmHtXg==", + "dev": true, + "license": "ISC", + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-quadtree": { + "version": "3.0.1", + "resolved": "https://registry.npmmirror.com/d3-quadtree/-/d3-quadtree-3.0.1.tgz", + "integrity": "sha512-04xDrxQTDTCFwP5H6hRhsRcb9xxv2RzkcsygFzmkSIOJy3PeRJP7sNk3VRIbKXcog561P9oU0/rVH6vDROAgUw==", + "dev": true, + "license": "ISC", + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-random": { + "version": "3.0.1", + "resolved": "https://registry.npmmirror.com/d3-random/-/d3-random-3.0.1.tgz", + "integrity": "sha512-FXMe9GfxTxqd5D6jFsQ+DJ8BJS4E/fT5mqqdjovykEB2oFbTMDVdg1MGFxfQW+FBOGoB++k8swBrgwSHT1cUXQ==", + "dev": true, + "license": "ISC", + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-sankey": { + "version": "0.12.3", + "resolved": "https://registry.npmmirror.com/d3-sankey/-/d3-sankey-0.12.3.tgz", + "integrity": "sha512-nQhsBRmM19Ax5xEIPLMY9ZmJ/cDvd1BG3UVvt5h3WRxKg5zGRbvnteTyWAbzeSvlh3tW7ZEmq4VwR5mB3tutmQ==", + "dev": true, + "license": "BSD-3-Clause", + "dependencies": { + "d3-array": "1 - 2", + "d3-shape": "^1.2.0" + } + }, + "node_modules/d3-sankey/node_modules/d3-array": { + "version": "2.12.1", + "resolved": "https://registry.npmmirror.com/d3-array/-/d3-array-2.12.1.tgz", + "integrity": "sha512-B0ErZK/66mHtEsR1TkPEEkwdy+WDesimkM5gpZr5Dsg54BiTA5RXtYW5qTLIAcekaS9xfZrzBLF/OAkB3Qn1YQ==", + "dev": true, + "license": "BSD-3-Clause", + "dependencies": { + "internmap": "^1.0.0" + } + }, + "node_modules/d3-sankey/node_modules/d3-path": { + "version": "1.0.9", + "resolved": "https://registry.npmmirror.com/d3-path/-/d3-path-1.0.9.tgz", + "integrity": "sha512-VLaYcn81dtHVTjEHd8B+pbe9yHWpXKZUC87PzoFmsFrJqgFwDe/qxfp5MlfsfM1V5E/iVt0MmEbWQ7FVIXh/bg==", + "dev": true, + "license": "BSD-3-Clause" + }, + "node_modules/d3-sankey/node_modules/d3-shape": { + "version": "1.3.7", + "resolved": "https://registry.npmmirror.com/d3-shape/-/d3-shape-1.3.7.tgz", + "integrity": "sha512-EUkvKjqPFUAZyOlhY5gzCxCeI0Aep04LwIRpsZ/mLFelJiUfnK56jo5JMDSE7yyP2kLSb6LtF+S5chMk7uqPqw==", + "dev": true, + "license": "BSD-3-Clause", + "dependencies": { + "d3-path": "1" + } + }, + "node_modules/d3-sankey/node_modules/internmap": { + "version": "1.0.1", + "resolved": "https://registry.npmmirror.com/internmap/-/internmap-1.0.1.tgz", + "integrity": "sha512-lDB5YccMydFBtasVtxnZ3MRBHuaoE8GKsppq+EchKL2U4nK/DmEpPHNH8MZe5HkMtpSiTSOZwfN0tzYjO/lJEw==", + "dev": true, + "license": "ISC" + }, + "node_modules/d3-scale": { + "version": "4.0.2", + "resolved": "https://registry.npmmirror.com/d3-scale/-/d3-scale-4.0.2.tgz", + "integrity": "sha512-GZW464g1SH7ag3Y7hXjf8RoUuAFIqklOAq3MRl4OaWabTFJY9PN/E1YklhXLh+OQ3fM9yS2nOkCoS+WLZ6kvxQ==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-array": "2.10.0 - 3", + "d3-format": "1 - 3", + "d3-interpolate": "1.2.0 - 3", + "d3-time": "2.1.1 - 3", + "d3-time-format": "2 - 4" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-scale-chromatic": { + "version": "3.1.0", + "resolved": "https://registry.npmmirror.com/d3-scale-chromatic/-/d3-scale-chromatic-3.1.0.tgz", + "integrity": "sha512-A3s5PWiZ9YCXFye1o246KoscMWqf8BsD9eRiJ3He7C9OBaxKhAd5TFCdEx/7VbKtxxTsu//1mMJFrEt572cEyQ==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-color": "1 - 3", + "d3-interpolate": "1 - 3" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-selection": { + "version": "3.0.0", + "resolved": "https://registry.npmmirror.com/d3-selection/-/d3-selection-3.0.0.tgz", + "integrity": "sha512-fmTRWbNMmsmWq6xJV8D19U/gw/bwrHfNXxrIN+HfZgnzqTHp9jOmKMhsTUjXOJnZOdZY9Q28y4yebKzqDKlxlQ==", + "dev": true, + "license": "ISC", + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-shape": { + "version": "3.2.0", + "resolved": "https://registry.npmmirror.com/d3-shape/-/d3-shape-3.2.0.tgz", + "integrity": "sha512-SaLBuwGm3MOViRq2ABk3eLoxwZELpH6zhl3FbAoJ7Vm1gofKx6El1Ib5z23NUEhF9AsGl7y+dzLe5Cw2AArGTA==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-path": "^3.1.0" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-time": { + "version": "3.1.0", + "resolved": "https://registry.npmmirror.com/d3-time/-/d3-time-3.1.0.tgz", + "integrity": "sha512-VqKjzBLejbSMT4IgbmVgDjpkYrNWUYJnbCGo874u7MMKIWsILRX+OpX/gTk8MqjpT1A/c6HY2dCA77ZN0lkQ2Q==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-array": "2 - 3" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-time-format": { + "version": "4.1.0", + "resolved": "https://registry.npmmirror.com/d3-time-format/-/d3-time-format-4.1.0.tgz", + "integrity": "sha512-dJxPBlzC7NugB2PDLwo9Q8JiTR3M3e4/XANkreKSUxF8vvXKqm1Yfq4Q5dl8budlunRVlUUaDUgFt7eA8D6NLg==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-time": "1 - 3" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-timer": { + "version": "3.0.1", + "resolved": "https://registry.npmmirror.com/d3-timer/-/d3-timer-3.0.1.tgz", + "integrity": "sha512-ndfJ/JxxMd3nw31uyKoY2naivF+r29V+Lc0svZxe1JvvIRmi8hUsrMvdOwgS1o6uBHmiz91geQ0ylPP0aj1VUA==", + "dev": true, + "license": "ISC", + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-transition": { + "version": "3.0.1", + "resolved": "https://registry.npmmirror.com/d3-transition/-/d3-transition-3.0.1.tgz", + "integrity": "sha512-ApKvfjsSR6tg06xrL434C0WydLr7JewBB3V+/39RMHsaXTOG0zmt/OAXeng5M5LBm0ojmxJrpomQVZ1aPvBL4w==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-color": "1 - 3", + "d3-dispatch": "1 - 3", + "d3-ease": "1 - 3", + "d3-interpolate": "1 - 3", + "d3-timer": "1 - 3" + }, + "engines": { + "node": ">=12" + }, + "peerDependencies": { + "d3-selection": "2 - 3" + } + }, + "node_modules/d3-zoom": { + "version": "3.0.0", + "resolved": "https://registry.npmmirror.com/d3-zoom/-/d3-zoom-3.0.0.tgz", + "integrity": "sha512-b8AmV3kfQaqWAuacbPuNbL6vahnOJflOhexLzMMNLga62+/nh0JzvJ0aO/5a5MVgUFGS7Hu1P9P03o3fJkDCyw==", + "dev": true, + "license": "ISC", + "dependencies": { + "d3-dispatch": "1 - 3", + "d3-drag": "2 - 3", + "d3-interpolate": "1 - 3", + "d3-selection": "2 - 3", + "d3-transition": "2 - 3" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/dagre-d3-es": { + "version": "7.0.13", + "resolved": "https://registry.npmmirror.com/dagre-d3-es/-/dagre-d3-es-7.0.13.tgz", + "integrity": "sha512-efEhnxpSuwpYOKRm/L5KbqoZmNNukHa/Flty4Wp62JRvgH2ojwVgPgdYyr4twpieZnyRDdIH7PY2mopX26+j2Q==", + "dev": true, + "license": "MIT", + "dependencies": { + "d3": "^7.9.0", + "lodash-es": "^4.17.21" + } + }, + "node_modules/dayjs": { + "version": "1.11.19", + "resolved": "https://registry.npmmirror.com/dayjs/-/dayjs-1.11.19.tgz", + "integrity": "sha512-t5EcLVS6QPBNqM2z8fakk/NKel+Xzshgt8FFKAn+qwlD1pzZWxh0nVCrvFK7ZDb6XucZeF9z8C7CBWTRIVApAw==", + "dev": true, + "license": "MIT" + }, + "node_modules/delaunator": { + "version": "5.0.1", + "resolved": "https://registry.npmmirror.com/delaunator/-/delaunator-5.0.1.tgz", + "integrity": "sha512-8nvh+XBe96aCESrGOqMp/84b13H9cdKbG5P2ejQCh4d4sK9RL4371qou9drQjMhvnPmhWl5hnmqbEE0fXr9Xnw==", + "dev": true, + "license": "ISC", + "dependencies": { + "robust-predicates": "^3.0.2" + } + }, + "node_modules/dompurify": { + "version": "3.3.1", + "resolved": "https://registry.npmmirror.com/dompurify/-/dompurify-3.3.1.tgz", + "integrity": "sha512-qkdCKzLNtrgPFP1Vo+98FRzJnBRGe4ffyCea9IwHB1fyxPOeNTHpLKYGd4Uk9xvNoH0ZoOjwZxNptyMwqrId1Q==", + "dev": true, + "license": "(MPL-2.0 OR Apache-2.0)", + "optionalDependencies": { + "@types/trusted-types": "^2.0.7" + } + }, + "node_modules/entities": { + "version": "4.5.0", + "resolved": "https://registry.npmmirror.com/entities/-/entities-4.5.0.tgz", + "integrity": "sha512-V0hjH4dGPh9Ao5p0MoRY6BVqtwCjhz6vI5LT8AJ55H+4g9/4vbHx1I54fS0XuclLhDHArPQCiMjDxjaL8fPxhw==", + "dev": true, + "engines": { + "node": ">=0.12" + }, + "funding": { + "url": "https://github.com/fb55/entities?sponsor=1" + } + }, + "node_modules/esbuild": { + "version": "0.21.5", + "resolved": "https://registry.npmmirror.com/esbuild/-/esbuild-0.21.5.tgz", + "integrity": "sha512-mg3OPMV4hXywwpoDxu3Qda5xCKQi+vCTZq8S9J/EpkhB2HzKXq4SNFZE3+NK93JYxc8VMSep+lOUSC/RVKaBqw==", + "dev": true, + "hasInstallScript": true, + "bin": { + "esbuild": "bin/esbuild" + }, + "engines": { + "node": ">=12" + }, + "optionalDependencies": { + "@esbuild/aix-ppc64": "0.21.5", + "@esbuild/android-arm": "0.21.5", + "@esbuild/android-arm64": "0.21.5", + "@esbuild/android-x64": "0.21.5", + "@esbuild/darwin-arm64": "0.21.5", + "@esbuild/darwin-x64": "0.21.5", + "@esbuild/freebsd-arm64": "0.21.5", + "@esbuild/freebsd-x64": "0.21.5", + "@esbuild/linux-arm": "0.21.5", + "@esbuild/linux-arm64": "0.21.5", + "@esbuild/linux-ia32": "0.21.5", + "@esbuild/linux-loong64": "0.21.5", + "@esbuild/linux-mips64el": "0.21.5", + "@esbuild/linux-ppc64": "0.21.5", + "@esbuild/linux-riscv64": "0.21.5", + "@esbuild/linux-s390x": "0.21.5", + "@esbuild/linux-x64": "0.21.5", + "@esbuild/netbsd-x64": "0.21.5", + "@esbuild/openbsd-x64": "0.21.5", + "@esbuild/sunos-x64": "0.21.5", + "@esbuild/win32-arm64": "0.21.5", + "@esbuild/win32-ia32": "0.21.5", + "@esbuild/win32-x64": "0.21.5" + } + }, + "node_modules/estree-walker": { + "version": "2.0.2", + "resolved": "https://registry.npmmirror.com/estree-walker/-/estree-walker-2.0.2.tgz", + "integrity": "sha512-Rfkk/Mp/DL7JVje3u18FxFujQlTNR2q6QfMSMB7AvCBx91NGj/ba3kCfza0f6dVDbw7YlRf/nDrn7pQrCCyQ/w==", + "dev": true + }, + "node_modules/focus-trap": { + "version": "7.5.4", + "resolved": "https://registry.npmmirror.com/focus-trap/-/focus-trap-7.5.4.tgz", + "integrity": "sha512-N7kHdlgsO/v+iD/dMoJKtsSqs5Dz/dXZVebRgJw23LDk+jMi/974zyiOYDziY2JPp8xivq9BmUGwIJMiuSBi7w==", + "dev": true, + "dependencies": { + "tabbable": "^6.2.0" + } + }, + "node_modules/fsevents": { + "version": "2.3.3", + "resolved": "https://registry.npmmirror.com/fsevents/-/fsevents-2.3.3.tgz", + "integrity": "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==", + "dev": true, + "hasInstallScript": true, + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": "^8.16.0 || ^10.6.0 || >=11.0.0" + } + }, + "node_modules/hachure-fill": { + "version": "0.5.2", + "resolved": "https://registry.npmmirror.com/hachure-fill/-/hachure-fill-0.5.2.tgz", + "integrity": "sha512-3GKBOn+m2LX9iq+JC1064cSFprJY4jL1jCXTcpnfER5HYE2l/4EfWSGzkPa/ZDBmYI0ZOEj5VHV/eKnPGkHuOg==", + "dev": true, + "license": "MIT" + }, + "node_modules/hookable": { + "version": "5.5.3", + "resolved": "https://registry.npmmirror.com/hookable/-/hookable-5.5.3.tgz", + "integrity": "sha512-Yc+BQe8SvoXH1643Qez1zqLRmbA5rCL+sSmk6TVos0LWVfNIB7PGncdlId77WzLGSIB5KaWgTaNTs2lNVEI6VQ==", + "dev": true + }, + "node_modules/iconv-lite": { + "version": "0.6.3", + "resolved": "https://registry.npmmirror.com/iconv-lite/-/iconv-lite-0.6.3.tgz", + "integrity": "sha512-4fCk79wshMdzMp2rH06qWrJE4iolqLhCUH+OiuIgU++RB0+94NlDL81atO7GX55uUKueo0txHNtvEyI6D7WdMw==", + "dev": true, + "license": "MIT", + "dependencies": { + "safer-buffer": ">= 2.1.2 < 3.0.0" + }, + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/internmap": { + "version": "2.0.3", + "resolved": "https://registry.npmmirror.com/internmap/-/internmap-2.0.3.tgz", + "integrity": "sha512-5Hh7Y1wQbvY5ooGgPbDaL5iYLAPzMTUrjMulskHLH6wnv/A+1q5rgEaiuqEjB+oxGXIVZs1FF+R/KPN3ZSQYYg==", + "dev": true, + "license": "ISC", + "engines": { + "node": ">=12" + } + }, + "node_modules/is-what": { + "version": "4.1.16", + "resolved": "https://registry.npmmirror.com/is-what/-/is-what-4.1.16.tgz", + "integrity": "sha512-ZhMwEosbFJkA0YhFnNDgTM4ZxDRsS6HqTo7qsZM08fehyRYIYa0yHu5R6mgo1n/8MgaPBXiPimPD77baVFYg+A==", + "dev": true, + "engines": { + "node": ">=12.13" + }, + "funding": { + "url": "https://github.com/sponsors/mesqueeb" + } + }, + "node_modules/katex": { + "version": "0.16.27", + "resolved": "https://registry.npmmirror.com/katex/-/katex-0.16.27.tgz", + "integrity": "sha512-aeQoDkuRWSqQN6nSvVCEFvfXdqo1OQiCmmW1kc9xSdjutPv7BGO7pqY9sQRJpMOGrEdfDgF2TfRXe5eUAD2Waw==", + "dev": true, + "funding": [ + "https://opencollective.com/katex", + "https://github.com/sponsors/katex" + ], + "license": "MIT", + "dependencies": { + "commander": "^8.3.0" + }, + "bin": { + "katex": "cli.js" + } + }, + "node_modules/katex/node_modules/commander": { + "version": "8.3.0", + "resolved": "https://registry.npmmirror.com/commander/-/commander-8.3.0.tgz", + "integrity": "sha512-OkTL9umf+He2DZkUq8f8J9of7yL6RJKI24dVITBmNfZBmri9zYZQrKkuXiKhyfPSu8tUhnVBB1iKXevvnlR4Ww==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">= 12" + } + }, + "node_modules/khroma": { + "version": "2.1.0", + "resolved": "https://registry.npmmirror.com/khroma/-/khroma-2.1.0.tgz", + "integrity": "sha512-Ls993zuzfayK269Svk9hzpeGUKob/sIgZzyHYdjQoAdQetRKpOLj+k/QQQ/6Qi0Yz65mlROrfd+Ev+1+7dz9Kw==", + "dev": true + }, + "node_modules/langium": { + "version": "3.3.1", + "resolved": "https://registry.npmmirror.com/langium/-/langium-3.3.1.tgz", + "integrity": "sha512-QJv/h939gDpvT+9SiLVlY7tZC3xB2qK57v0J04Sh9wpMb6MP1q8gB21L3WIo8T5P1MSMg3Ep14L7KkDCFG3y4w==", + "dev": true, + "license": "MIT", + "dependencies": { + "chevrotain": "~11.0.3", + "chevrotain-allstar": "~0.3.0", + "vscode-languageserver": "~9.0.1", + "vscode-languageserver-textdocument": "~1.0.11", + "vscode-uri": "~3.0.8" + }, + "engines": { + "node": ">=16.0.0" + } + }, + "node_modules/layout-base": { + "version": "1.0.2", + "resolved": "https://registry.npmmirror.com/layout-base/-/layout-base-1.0.2.tgz", + "integrity": "sha512-8h2oVEZNktL4BH2JCOI90iD1yXwL6iNW7KcCKT2QZgQJR2vbqDsldCTPRU9NifTCqHZci57XvQQ15YTu+sTYPg==", + "dev": true, + "license": "MIT" + }, + "node_modules/lodash-es": { + "version": "4.17.22", + "resolved": "https://registry.npmmirror.com/lodash-es/-/lodash-es-4.17.22.tgz", + "integrity": "sha512-XEawp1t0gxSi9x01glktRZ5HDy0HXqrM0x5pXQM98EaI0NxO6jVM7omDOxsuEo5UIASAnm2bRp1Jt/e0a2XU8Q==", + "dev": true, + "license": "MIT" + }, + "node_modules/magic-string": { "version": "0.30.11", "resolved": "https://registry.npmmirror.com/magic-string/-/magic-string-0.30.11.tgz", "integrity": "sha512-+Wri9p0QHMy+545hKww7YAu5NyzF8iomPL/RQazugQ9+Ez4Ic3mERMd8ZTX5rfK944j+560ZJi8iAwgak1Ac7A==", @@ -1578,6 +2749,48 @@ "integrity": "sha512-1I+1qpDt4idfgLQG+BNWmrqku+7/2bi5nLf4YwF8y8zXvmfiTBY3PV3ZibfrjBueCByROpuBjLLFCajqkgYoLQ==", "dev": true }, + "node_modules/marked": { + "version": "16.4.2", + "resolved": "https://registry.npmmirror.com/marked/-/marked-16.4.2.tgz", + "integrity": "sha512-TI3V8YYWvkVf3KJe1dRkpnjs68JUPyEa5vjKrp1XEEJUAOaQc+Qj+L1qWbPd0SJuAdQkFU0h73sXXqwDYxsiDA==", + "dev": true, + "license": "MIT", + "bin": { + "marked": "bin/marked.js" + }, + "engines": { + "node": ">= 20" + } + }, + "node_modules/mermaid": { + "version": "11.12.2", + "resolved": "https://registry.npmmirror.com/mermaid/-/mermaid-11.12.2.tgz", + "integrity": "sha512-n34QPDPEKmaeCG4WDMGy0OT6PSyxKCfy2pJgShP+Qow2KLrvWjclwbc3yXfSIf4BanqWEhQEpngWwNp/XhZt6w==", + "dev": true, + "license": "MIT", + "dependencies": { + "@braintree/sanitize-url": "^7.1.1", + "@iconify/utils": "^3.0.1", + "@mermaid-js/parser": "^0.6.3", + "@types/d3": "^7.4.3", + "cytoscape": "^3.29.3", + "cytoscape-cose-bilkent": "^4.1.0", + "cytoscape-fcose": "^2.2.0", + "d3": "^7.9.0", + "d3-sankey": "^0.12.3", + "dagre-d3-es": "7.0.13", + "dayjs": "^1.11.18", + "dompurify": "^3.2.5", + "katex": "^0.16.22", + "khroma": "^2.1.0", + "lodash-es": "^4.17.21", + "marked": "^16.2.1", + "roughjs": "^4.6.6", + "stylis": "^4.3.6", + "ts-dedent": "^2.2.0", + "uuid": "^11.1.0" + } + }, "node_modules/minisearch": { "version": "7.1.0", "resolved": "https://registry.npmmirror.com/minisearch/-/minisearch-7.1.0.tgz", @@ -1590,6 +2803,19 @@ "integrity": "sha512-vKivATfr97l2/QBCYAkXYDbrIWPM2IIKEl7YPhjCvKlG3kE2gm+uBo6nEXK3M5/Ffh/FLpKExzOQ3JJoJGFKBw==", "dev": true }, + "node_modules/mlly": { + "version": "1.8.0", + "resolved": "https://registry.npmmirror.com/mlly/-/mlly-1.8.0.tgz", + "integrity": "sha512-l8D9ODSRWLe2KHJSifWGwBqpTZXIXTeo8mlKjY+E2HAakaTeNpqAyBZ8GSqLzHgw4XmHmC8whvpjJNMbFZN7/g==", + "dev": true, + "license": "MIT", + "dependencies": { + "acorn": "^8.15.0", + "pathe": "^2.0.3", + "pkg-types": "^1.3.1", + "ufo": "^1.6.1" + } + }, "node_modules/nanoid": { "version": "3.3.7", "resolved": "https://registry.npmmirror.com/nanoid/-/nanoid-3.3.7.tgz", @@ -1608,6 +2834,35 @@ "node": "^10 || ^12 || ^13.7 || ^14 || >=15.0.1" } }, + "node_modules/non-layered-tidy-tree-layout": { + "version": "2.0.2", + "resolved": "https://registry.npmmirror.com/non-layered-tidy-tree-layout/-/non-layered-tidy-tree-layout-2.0.2.tgz", + "integrity": "sha512-gkXMxRzUH+PB0ax9dUN0yYF0S25BqeAYqhgMaLUFmpXLEk7Fcu8f4emJuOAY0V8kjDICxROIKsTAKsV/v355xw==", + "dev": true, + "license": "MIT", + "optional": true + }, + "node_modules/package-manager-detector": { + "version": "1.6.0", + "resolved": "https://registry.npmmirror.com/package-manager-detector/-/package-manager-detector-1.6.0.tgz", + "integrity": "sha512-61A5ThoTiDG/C8s8UMZwSorAGwMJ0ERVGj2OjoW5pAalsNOg15+iQiPzrLJ4jhZ1HJzmC2PIHT2oEiH3R5fzNA==", + "dev": true, + "license": "MIT" + }, + "node_modules/path-data-parser": { + "version": "0.1.0", + "resolved": "https://registry.npmmirror.com/path-data-parser/-/path-data-parser-0.1.0.tgz", + "integrity": "sha512-NOnmBpt5Y2RWbuv0LMzsayp3lVylAHLPUTut412ZA3l+C4uw4ZVkQbjShYCQ8TCpUMdPapr4YjUqLYD6v68j+w==", + "dev": true, + "license": "MIT" + }, + "node_modules/pathe": { + "version": "2.0.3", + "resolved": "https://registry.npmmirror.com/pathe/-/pathe-2.0.3.tgz", + "integrity": "sha512-WUjGcAqP1gQacoQe+OBJsFA7Ld4DyXuUIjZ5cc75cLHvJ7dtNsTugphxIADwspS+AraAUePCKrSVtPLFj/F88w==", + "dev": true, + "license": "MIT" + }, "node_modules/perfect-debounce": { "version": "1.0.0", "resolved": "https://registry.npmmirror.com/perfect-debounce/-/perfect-debounce-1.0.0.tgz", @@ -1620,6 +2875,36 @@ "integrity": "sha512-anP1Z8qwhkbmu7MFP5iTt+wQKXgwzf7zTyGlcdzabySa9vd0Xt392U0rVmz9poOaBj0uHJKyyo9/upk0HrEQew==", "dev": true }, + "node_modules/pkg-types": { + "version": "1.3.1", + "resolved": "https://registry.npmmirror.com/pkg-types/-/pkg-types-1.3.1.tgz", + "integrity": "sha512-/Jm5M4RvtBFVkKWRu2BLUTNP8/M2a+UwuAX+ae4770q1qVGtfjG+WTCupoZixokjmHiry8uI+dlY8KXYV5HVVQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "confbox": "^0.1.8", + "mlly": "^1.7.4", + "pathe": "^2.0.1" + } + }, + "node_modules/points-on-curve": { + "version": "0.2.0", + "resolved": "https://registry.npmmirror.com/points-on-curve/-/points-on-curve-0.2.0.tgz", + "integrity": "sha512-0mYKnYYe9ZcqMCWhUjItv/oHjvgEsfKvnUTg8sAtnHr3GVy7rGkXCb6d5cSyqrWqL4k81b9CPg3urd+T7aop3A==", + "dev": true, + "license": "MIT" + }, + "node_modules/points-on-path": { + "version": "0.2.1", + "resolved": "https://registry.npmmirror.com/points-on-path/-/points-on-path-0.2.1.tgz", + "integrity": "sha512-25ClnWWuw7JbWZcgqY/gJ4FQWadKxGWk+3kR/7kD0tCaDtPPMj7oHu2ToLaVhfpnHrZzYby2w6tUA0eOIuUg8g==", + "dev": true, + "license": "MIT", + "dependencies": { + "path-data-parser": "0.1.0", + "points-on-curve": "0.2.0" + } + }, "node_modules/postcss": { "version": "8.4.41", "resolved": "https://registry.npmmirror.com/postcss/-/postcss-8.4.41.tgz", @@ -1664,6 +2949,13 @@ "integrity": "sha512-q1b3N5QkRUWUl7iyylaaj3kOpIT0N2i9MqIEQXP73GVsN9cw3fdx8X63cEmWhJGi2PPCF23Ijp7ktmd39rawIA==", "dev": true }, + "node_modules/robust-predicates": { + "version": "3.0.2", + "resolved": "https://registry.npmmirror.com/robust-predicates/-/robust-predicates-3.0.2.tgz", + "integrity": "sha512-IXgzBWvWQwE6PrDI05OvmXUIruQTcoMDzRsOd5CDvHCVLcLHMTSYvOK5Cm46kWqlV3yAbuSpBZdJ5oP5OUoStg==", + "dev": true, + "license": "Unlicense" + }, "node_modules/rollup": { "version": "4.21.1", "resolved": "https://registry.npmmirror.com/rollup/-/rollup-4.21.1.tgz", @@ -1699,6 +2991,33 @@ "fsevents": "~2.3.2" } }, + "node_modules/roughjs": { + "version": "4.6.6", + "resolved": "https://registry.npmmirror.com/roughjs/-/roughjs-4.6.6.tgz", + "integrity": "sha512-ZUz/69+SYpFN/g/lUlo2FXcIjRkSu3nDarreVdGGndHEBJ6cXPdKguS8JGxwj5HA5xIbVKSmLgr5b3AWxtRfvQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "hachure-fill": "^0.5.2", + "path-data-parser": "^0.1.0", + "points-on-curve": "^0.2.0", + "points-on-path": "^0.2.1" + } + }, + "node_modules/rw": { + "version": "1.3.3", + "resolved": "https://registry.npmmirror.com/rw/-/rw-1.3.3.tgz", + "integrity": "sha512-PdhdWy89SiZogBLaw42zdeqtRJ//zFd2PgQavcICDUgJT5oW10QCRKbJ6bg4r0/UY2M6BWd5tkxuGFRvCkgfHQ==", + "dev": true, + "license": "BSD-3-Clause" + }, + "node_modules/safer-buffer": { + "version": "2.1.2", + "resolved": "https://registry.npmmirror.com/safer-buffer/-/safer-buffer-2.1.2.tgz", + "integrity": "sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==", + "dev": true, + "license": "MIT" + }, "node_modules/search-insights": { "version": "2.17.0", "resolved": "https://registry.npmmirror.com/search-insights/-/search-insights-2.17.0.tgz", @@ -1734,6 +3053,13 @@ "node": ">=0.10.0" } }, + "node_modules/stylis": { + "version": "4.3.6", + "resolved": "https://registry.npmmirror.com/stylis/-/stylis-4.3.6.tgz", + "integrity": "sha512-yQ3rwFWRfwNUY7H5vpU0wfdkNSnvnJinhF9830Swlaxl03zsOjCfmX0ugac+3LtK0lYSgwL/KXc8oYL3mG4YFQ==", + "dev": true, + "license": "MIT" + }, "node_modules/superjson": { "version": "2.2.1", "resolved": "https://registry.npmmirror.com/superjson/-/superjson-2.2.1.tgz", @@ -1752,6 +3078,16 @@ "integrity": "sha512-Cat63mxsVJlzYvN51JmVXIgNoUokrIaT2zLclCXjRd8boZ0004U4KCs/sToJ75C6sdlByWxpYnb5Boif1VSFew==", "dev": true }, + "node_modules/tinyexec": { + "version": "1.0.2", + "resolved": "https://registry.npmmirror.com/tinyexec/-/tinyexec-1.0.2.tgz", + "integrity": "sha512-W/KYk+NFhkmsYpuHq5JykngiOCnxeVL8v8dFnqxSD8qEEdRfXk1SDM6JzNqcERbcGYj9tMrDQBYV9cjgnunFIg==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=18" + } + }, "node_modules/to-fast-properties": { "version": "2.0.0", "resolved": "https://registry.npmmirror.com/to-fast-properties/-/to-fast-properties-2.0.0.tgz", @@ -1761,6 +3097,37 @@ "node": ">=4" } }, + "node_modules/ts-dedent": { + "version": "2.2.0", + "resolved": "https://registry.npmmirror.com/ts-dedent/-/ts-dedent-2.2.0.tgz", + "integrity": "sha512-q5W7tVM71e2xjHZTlgfTDoPF/SmqKG5hddq9SzR49CH2hayqRKJtQ4mtRlSxKaJlR/+9rEM+mnBHf7I2/BQcpQ==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=6.10" + } + }, + "node_modules/ufo": { + "version": "1.6.3", + "resolved": "https://registry.npmmirror.com/ufo/-/ufo-1.6.3.tgz", + "integrity": "sha512-yDJTmhydvl5lJzBmy/hyOAA0d+aqCBuwl818haVdYCRrWV84o7YyeVm4QlVHStqNrrJSTb6jKuFAVqAFsr+K3Q==", + "dev": true, + "license": "MIT" + }, + "node_modules/uuid": { + "version": "11.1.0", + "resolved": "https://registry.npmmirror.com/uuid/-/uuid-11.1.0.tgz", + "integrity": "sha512-0/A9rDy9P7cJ+8w1c9WD9V//9Wj15Ce2MPz8Ri6032usz+NfePxx5AcN3bN+r6ZL6jEo066/yNYB3tn4pQEx+A==", + "dev": true, + "funding": [ + "https://github.com/sponsors/broofa", + "https://github.com/sponsors/ctavan" + ], + "license": "MIT", + "bin": { + "uuid": "dist/esm/bin/uuid" + } + }, "node_modules/vite": { "version": "5.4.2", "resolved": "https://registry.npmmirror.com/vite/-/vite-5.4.2.tgz", @@ -1859,6 +3226,75 @@ } } }, + "node_modules/vitepress-plugin-mermaid": { + "version": "2.0.17", + "resolved": "https://registry.npmmirror.com/vitepress-plugin-mermaid/-/vitepress-plugin-mermaid-2.0.17.tgz", + "integrity": "sha512-IUzYpwf61GC6k0XzfmAmNrLvMi9TRrVRMsUyCA8KNXhg/mQ1VqWnO0/tBVPiX5UoKF1mDUwqn5QV4qAJl6JnUg==", + "dev": true, + "license": "MIT", + "optionalDependencies": { + "@mermaid-js/mermaid-mindmap": "^9.3.0" + }, + "peerDependencies": { + "mermaid": "10 || 11", + "vitepress": "^1.0.0 || ^1.0.0-alpha" + } + }, + "node_modules/vscode-jsonrpc": { + "version": "8.2.0", + "resolved": "https://registry.npmmirror.com/vscode-jsonrpc/-/vscode-jsonrpc-8.2.0.tgz", + "integrity": "sha512-C+r0eKJUIfiDIfwJhria30+TYWPtuHJXHtI7J0YlOmKAo7ogxP20T0zxB7HZQIFhIyvoBPwWskjxrvAtfjyZfA==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=14.0.0" + } + }, + "node_modules/vscode-languageserver": { + "version": "9.0.1", + "resolved": "https://registry.npmmirror.com/vscode-languageserver/-/vscode-languageserver-9.0.1.tgz", + "integrity": "sha512-woByF3PDpkHFUreUa7Hos7+pUWdeWMXRd26+ZX2A8cFx6v/JPTtd4/uN0/jB6XQHYaOlHbio03NTHCqrgG5n7g==", + "dev": true, + "license": "MIT", + "dependencies": { + "vscode-languageserver-protocol": "3.17.5" + }, + "bin": { + "installServerIntoExtension": "bin/installServerIntoExtension" + } + }, + "node_modules/vscode-languageserver-protocol": { + "version": "3.17.5", + "resolved": "https://registry.npmmirror.com/vscode-languageserver-protocol/-/vscode-languageserver-protocol-3.17.5.tgz", + "integrity": "sha512-mb1bvRJN8SVznADSGWM9u/b07H7Ecg0I3OgXDuLdn307rl/J3A9YD6/eYOssqhecL27hK1IPZAsaqh00i/Jljg==", + "dev": true, + "license": "MIT", + "dependencies": { + "vscode-jsonrpc": "8.2.0", + "vscode-languageserver-types": "3.17.5" + } + }, + "node_modules/vscode-languageserver-textdocument": { + "version": "1.0.12", + "resolved": "https://registry.npmmirror.com/vscode-languageserver-textdocument/-/vscode-languageserver-textdocument-1.0.12.tgz", + "integrity": "sha512-cxWNPesCnQCcMPeenjKKsOCKQZ/L6Tv19DTRIGuLWe32lyzWhihGVJ/rcckZXJxfdKCFvRLS3fpBIsV/ZGX4zA==", + "dev": true, + "license": "MIT" + }, + "node_modules/vscode-languageserver-types": { + "version": "3.17.5", + "resolved": "https://registry.npmmirror.com/vscode-languageserver-types/-/vscode-languageserver-types-3.17.5.tgz", + "integrity": "sha512-Ld1VelNuX9pdF39h2Hgaeb5hEZM2Z3jUrrMgWQAu82jMtZp7p3vJT3BzToKtZI7NgQssZje5o0zryOrhQvzQAg==", + "dev": true, + "license": "MIT" + }, + "node_modules/vscode-uri": { + "version": "3.0.8", + "resolved": "https://registry.npmmirror.com/vscode-uri/-/vscode-uri-3.0.8.tgz", + "integrity": "sha512-AyFQ0EVmsOZOlAnxoFOGOq1SQDWAB7C6aqMGS23svWAllfOaxbuFvcT8D1i8z3Gyn8fraVeZNNmN6e9bxxXkKw==", + "dev": true, + "license": "MIT" + }, "node_modules/vue": { "version": "3.4.38", "resolved": "https://registry.npmmirror.com/vue/-/vue-3.4.38.tgz", diff --git a/package.json b/package.json index d7e5a5a..3ef7cb7 100644 --- a/package.json +++ b/package.json @@ -5,6 +5,8 @@ "docs:preview": "vitepress preview docs" }, "devDependencies": { - "vitepress": "^1.3.4" + "mermaid": "^11.12.2", + "vitepress": "^1.3.4", + "vitepress-plugin-mermaid": "^2.0.17" } } diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/README.md" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/README.md" new file mode 100644 index 0000000..937f08a --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/README.md" @@ -0,0 +1,33 @@ +# 第01章：工程化爬虫开发规范 + +展示日志系统、配置管理、异常处理等工程化实践。 + +## 快速开始 + +### 使用 uv 安装依赖 + +```bash +# 进入本章目录 +cd 01_工程化爬虫开发规范 + +# 安装依赖 +uv sync + +# 运行示例 +uv run python logger_demo.py +uv run python exception_demo.py +uv run python refactored_crawler/main.py +``` + +### 核心依赖 + +- `httpx` - 异步HTTP客户端 +- `pydantic` - 数据验证 +- `parsel` - HTML解析 +- `loguru` - 日志系统 + +## 主要文件 + +- `logger_demo.py` - 日志系统演示 +- `exception_demo.py` - 异常处理演示 +- `refactored_crawler/` - 工程化改造后的完整爬虫项目 diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/config/__init__.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/config/__init__.py" new file mode 100644 index 0000000..6be28b0 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/config/__init__.py" @@ -0,0 +1,5 @@ +# -*- coding: utf-8 -*- +# @Desc: 配置模块 +from .settings import settings, CrawlerSettings + +__all__ = ["settings", "CrawlerSettings"] diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/config/settings.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/config/settings.py" new file mode 100644 index 0000000..2c837e9 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/config/settings.py" @@ -0,0 +1,115 @@ +# -*- coding: utf-8 -*- +# @Desc: 爬虫配置管理模块 +# 使用 pydantic-settings 进行配置管理，支持环境变量和 .env 文件 + +from pydantic_settings import BaseSettings +from pydantic import Field +from typing import Optional +import os + + +class CrawlerSettings(BaseSettings): + """ + 爬虫项目配置类 + + 配置优先级（从高到低）： + 1. 环境变量 (CRAWLER_XXX) + 2. .env 文件 + 3. 默认值 + """ + + # 运行环境配置 + env: str = Field(default="development", description="运行环境: development/production") + debug: bool = Field(default=False, description="调试模式") + log_level: str = Field(default="INFO", description="日志级别: DEBUG/INFO/WARNING/ERROR") + + # HTTP 请求配置 + request_timeout: int = Field(default=30, description="请求超时时间(秒)") + max_retries: int = Field(default=3, description="最大重试次数") + retry_delay: float = Field(default=1.0, description="重试基础延迟(秒)") + retry_max_delay: float = Field(default=60.0, description="重试最大延迟(秒)") + + # 并发控制配置 + max_concurrent: int = Field(default=10, description="最大并发请求数") + request_delay: float = Field(default=0.5, description="请求间隔(秒)") + + # 数据库配置 + db_host: str = Field(default="localhost", description="数据库主机") + db_port: int = Field(default=3306, description="数据库端口") + db_user: str = Field(default="root", description="数据库用户名") + db_password: str = Field(default="", description="数据库密码") + db_name: str = Field(default="crawler_db", description="数据库名称") + + # 代理配置 + proxy_url: Optional[str] = Field(default=None, description="HTTP代理地址") + + # 存储配置 + output_dir: str = Field(default="./data", description="数据输出目录") + log_dir: str = Field(default="./logs", description="日志输出目录") + + class Config: + # .env 文件路径 + env_file = ".env" + env_file_encoding = "utf-8" + # 环境变量前缀 + env_prefix = "CRAWLER_" + # 额外字段处理 + extra = "ignore" + + @property + def db_url(self) -> str: + """构建数据库连接URL""" + return f"mysql://{self.db_user}:{self.db_password}@{self.db_host}:{self.db_port}/{self.db_name}" + + def ensure_dirs(self): + """确保必要的目录存在""" + os.makedirs(self.output_dir, exist_ok=True) + os.makedirs(self.log_dir, exist_ok=True) + + +# 全局配置实例（单例模式） +settings = CrawlerSettings() + + +# 开发环境配置 +class DevelopmentSettings(CrawlerSettings): + """开发环境配置""" + env: str = "development" + debug: bool = True + log_level: str = "DEBUG" + max_concurrent: int = 5 + + +# 生产环境配置 +class ProductionSettings(CrawlerSettings): + """生产环境配置""" + env: str = "production" + debug: bool = False + log_level: str = "INFO" + max_concurrent: int = 20 + + +def get_settings() -> CrawlerSettings: + """ + 根据环境变量返回对应的配置实例 + + 设置环境变量 CRAWLER_ENV=production 可切换到生产配置 + """ + env = os.getenv("CRAWLER_ENV", "development") + settings_map = { + "development": DevelopmentSettings, + "production": ProductionSettings, + } + settings_class = settings_map.get(env, DevelopmentSettings) + return settings_class() + + +if __name__ == "__main__": + # 测试配置 + config = get_settings() + print(f"当前环境: {config.env}") + print(f"调试模式: {config.debug}") + print(f"日志级别: {config.log_level}") + print(f"请求超时: {config.request_timeout}秒") + print(f"最大并发: {config.max_concurrent}") + print(f"数据库URL: {config.db_url}") diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/exception_demo.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/exception_demo.py" new file mode 100644 index 0000000..4abf6b9 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/exception_demo.py" @@ -0,0 +1,300 @@ +# -*- coding: utf-8 -*- +# @Desc: 异常处理与重试机制演示 +# 演示自定义异常类、tenacity 重试库的使用 + +import asyncio +import random +from typing import Optional + +from loguru import logger +from tenacity import ( + retry, + stop_after_attempt, + wait_exponential, + wait_random_exponential, + retry_if_exception_type, + before_sleep_log, + RetryError +) + + +# ==================== 自定义异常类 ==================== + +class CrawlerException(Exception): + """爬虫基础异常类""" + + def __init__(self, message: str, url: Optional[str] = None): + self.message = message + self.url = url + super().__init__(self.message) + + def __str__(self): + if self.url: + return f"{self.message} (URL: {self.url})" + return self.message + + +class RequestException(CrawlerException): + """HTTP 请求异常""" + pass + + +class TimeoutException(RequestException): + """请求超时异常""" + pass + + +class HTTPStatusException(RequestException): + """HTTP 状态码异常""" + + def __init__(self, message: str, status_code: int, url: Optional[str] = None): + self.status_code = status_code + super().__init__(message, url) + + def __str__(self): + return f"HTTP {self.status_code}: {self.message}" + + +class RateLimitException(RequestException): + """触发速率限制异常""" + pass + + +class IPBlockedException(RequestException): + """IP 被封禁异常""" + pass + + +class ParseException(CrawlerException): + """数据解析异常""" + pass + + +class StorageException(CrawlerException): + """数据存储异常""" + pass + + +class LoginRequiredException(CrawlerException): + """需要登录异常""" + pass + + +# ==================== 重试装饰器示例 ==================== + +@retry( + stop=stop_after_attempt(3), # 最多重试 3 次 + wait=wait_exponential(multiplier=1, max=10), # 指数退避: 1s, 2s, 4s...最长10s + retry=retry_if_exception_type(TimeoutException), # 只对超时异常重试 + before_sleep=before_sleep_log(logger, "WARNING") # 重试前记录日志 +) +async def fetch_with_basic_retry(url: str) -> str: + """基础重试示例""" + # 模拟随机超时 + if random.random() < 0.7: # 70% 概率超时 + logger.debug(f"模拟请求超时: {url}") + raise TimeoutException("请求超时", url) + + logger.info(f"请求成功: {url}") + return f"Response from {url}" + + +@retry( + stop=stop_after_attempt(5), + wait=wait_random_exponential(multiplier=1, min=1, max=30), # 随机指数退避 + retry=retry_if_exception_type((TimeoutException, RateLimitException)), + reraise=True # 重试用尽后重新抛出原始异常 +) +async def fetch_with_advanced_retry(url: str) -> str: + """高级重试示例 - 更多配置项""" + dice = random.random() + + if dice < 0.4: + raise TimeoutException("连接超时", url) + elif dice < 0.6: + raise RateLimitException("触发速率限制", url) + + return f"Success: {url}" + + +def create_retry_decorator( + max_attempts: int = 3, + base_delay: float = 1.0, + max_delay: float = 60.0 +): + """ + 工厂函数：创建可配置的重试装饰器 + + Args: + max_attempts: 最大重试次数 + base_delay: 基础延迟时间 + max_delay: 最大延迟时间 + """ + return retry( + stop=stop_after_attempt(max_attempts), + wait=wait_exponential(multiplier=base_delay, max=max_delay), + retry=retry_if_exception_type(( + TimeoutException, + RateLimitException + )), + reraise=True + ) + + +# 使用工厂函数创建装饰器 +@create_retry_decorator(max_attempts=4, base_delay=2.0) +async def fetch_important_data(url: str) -> dict: + """重要数据获取 - 使用更多重试次数""" + if random.random() < 0.6: + raise TimeoutException("超时", url) + return {"data": "important", "url": url} + + +# ==================== 异常处理最佳实践 ==================== + +async def crawl_page(url: str) -> Optional[dict]: + """ + 爬取单个页面的示例 - 展示异常处理最佳实践 + """ + try: + # 模拟请求 + result = await fetch_with_basic_retry(url) + return {"url": url, "content": result} + + except TimeoutException as e: + # 可恢复的异常 - 记录警告并返回 None + logger.warning(f"页面爬取超时，跳过: {e}") + return None + + except RateLimitException as e: + # 需要特殊处理的异常 + logger.warning(f"触发速率限制: {e}") + # 这里可以切换代理或增加延迟 + return None + + except IPBlockedException as e: + # 严重异常 - 向上抛出 + logger.error(f"IP被封禁: {e}") + raise + + except CrawlerException as e: + # 其他爬虫异常 + logger.error(f"爬虫异常: {e}") + return None + + except Exception as e: + # 未预期的异常 - 记录完整堆栈 + logger.exception(f"未知异常: {e}") + raise + + +async def run_crawler(urls: list): + """ + 爬虫主程序 - 展示全局异常处理 + """ + results = [] + + try: + for url in urls: + logger.info(f"开始爬取: {url}") + result = await crawl_page(url) + if result: + results.append(result) + # 请求间隔 + await asyncio.sleep(0.5) + + logger.info(f"爬取完成，成功: {len(results)}/{len(urls)}") + return results + + except IPBlockedException: + logger.critical("IP被封禁，终止爬虫") + raise + + except asyncio.CancelledError: + logger.warning("任务被取消") + raise + + except Exception as e: + logger.exception(f"爬虫运行异常: {e}") + raise + + finally: + # 清理工作 + logger.info("执行清理工作...") + + +# ==================== 演示函数 ==================== + +async def demo_basic_retry(): + """演示基础重试""" + print("\n" + "=" * 50) + print("演示 1: 基础重试机制") + print("=" * 50) + + url = "https://example.com/data" + try: + result = await fetch_with_basic_retry(url) + print(f"结果: {result}") + except TimeoutException as e: + print(f"重试用尽后仍然失败: {e}") + + +async def demo_advanced_retry(): + """演示高级重试""" + print("\n" + "=" * 50) + print("演示 2: 高级重试机制") + print("=" * 50) + + url = "https://example.com/important" + try: + result = await fetch_with_advanced_retry(url) + print(f"结果: {result}") + except (TimeoutException, RateLimitException) as e: + print(f"重试用尽: {e}") + + +async def demo_exception_handling(): + """演示异常处理""" + print("\n" + "=" * 50) + print("演示 3: 异常处理最佳实践") + print("=" * 50) + + urls = [ + "https://example.com/page1", + "https://example.com/page2", + "https://example.com/page3", + ] + + try: + results = await run_crawler(urls) + print(f"成功获取 {len(results)} 个页面") + except Exception as e: + print(f"爬虫异常终止: {e}") + + +async def main(): + """主函数""" + # 配置日志 + logger.remove() + logger.add( + lambda msg: print(msg, end=""), + format="{time:HH:mm:ss} | {level: <8} | {message}", + level="DEBUG" + ) + + print("=" * 50) + print("异常处理与重试机制演示") + print("=" * 50) + + # 运行演示 + await demo_basic_retry() + await demo_advanced_retry() + await demo_exception_handling() + + print("\n" + "=" * 50) + print("演示完成") + print("=" * 50) + + +if __name__ == "__main__": + asyncio.run(main()) diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/logger_demo.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/logger_demo.py" new file mode 100644 index 0000000..4c0e768 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/logger_demo.py" @@ -0,0 +1,180 @@ +# -*- coding: utf-8 -*- +# @Desc: loguru 日志库使用演示 +# 演示 loguru 的各种功能：日志级别、文件输出、日志轮转等 + +import sys +from loguru import logger + + +def basic_usage(): + """基础用法演示""" + print("=" * 50) + print("1. 基础日志输出") + print("=" * 50) + + # loguru 默认配置下的日志输出 + logger.debug("这是 DEBUG 级别 - 用于详细的调试信息") + logger.info("这是 INFO 级别 - 用于正常的运行信息") + logger.warning("这是 WARNING 级别 - 用于警告信息") + logger.error("这是 ERROR 级别 - 用于错误信息") + logger.critical("这是 CRITICAL 级别 - 用于严重错误") + + +def custom_format(): + """自定义日志格式""" + print("\n" + "=" * 50) + print("2. 自定义日志格式") + print("=" * 50) + + # 移除默认处理器 + logger.remove() + + # 添加自定义格式的处理器 + logger.add( + sys.stderr, + format="{time:YYYY-MM-DD HH:mm:ss.SSS} | " + "{level: <8} | " + "{name}:{function}:{line} | " + "{message}", + level="DEBUG", + colorize=True + ) + + logger.info("使用自定义格式的日志输出") + logger.debug("包含时间、级别、文件名、函数名、行号等信息") + + +def file_logging(): + """文件日志演示""" + print("\n" + "=" * 50) + print("3. 文件日志输出") + print("=" * 50) + + # 添加文件日志 + # rotation: 日志轮转策略 + # retention: 日志保留策略 + # compression: 旧日志压缩 + logger.add( + "logs/demo_{time:YYYY-MM-DD}.log", + rotation="100 MB", # 文件达到 100MB 时轮转 + retention="7 days", # 保留 7 天的日志 + compression="zip", # 压缩旧日志 + encoding="utf-8", + level="DEBUG" + ) + + logger.info("这条日志会同时输出到控制台和文件") + logger.debug("文件日志支持自动轮转和压缩") + + +def error_logging(): + """错误日志处理""" + print("\n" + "=" * 50) + print("4. 异常日志记录") + print("=" * 50) + + # 单独的错误日志文件 + logger.add( + "logs/error_{time:YYYY-MM-DD}.log", + rotation="00:00", # 每天午夜轮转 + retention="30 days", # 保留 30 天 + level="ERROR", # 只记录 ERROR 及以上级别 + encoding="utf-8" + ) + + try: + result = 1 / 0 + except ZeroDivisionError: + # 使用 exception() 方法会自动记录完整的堆栈信息 + logger.exception("捕获到除零错误") + + logger.error("这是一条普通的错误日志") + + +def context_logging(): + """上下文日志""" + print("\n" + "=" * 50) + print("5. 带上下文的日志") + print("=" * 50) + + # 使用 bind() 添加上下文信息 + context_logger = logger.bind(user_id="12345", request_id="abc-xyz") + context_logger.info("处理用户请求") + context_logger.info("请求处理完成") + + # 使用 opt() 添加额外信息 + logger.opt(colors=True).info("支持 彩色 文本") + + +def structured_logging(): + """结构化日志""" + print("\n" + "=" * 50) + print("6. 结构化日志（JSON格式）") + print("=" * 50) + + # 移除之前的处理器 + logger.remove() + + # 添加 JSON 格式的日志输出（适合日志分析系统） + logger.add( + "logs/structured.json", + serialize=True, # 输出 JSON 格式 + rotation="10 MB", + level="DEBUG" + ) + + # 同时保留控制台输出 + logger.add(sys.stderr, level="INFO") + + # 记录结构化数据 + logger.info("用户登录", user_id=12345, ip="192.168.1.1") + logger.info("请求完成", url="/api/data", status=200, duration_ms=150) + + +def crawler_logging_example(): + """爬虫场景的日志使用示例""" + print("\n" + "=" * 50) + print("7. 爬虫场景日志示例") + print("=" * 50) + + # 重置日志配置 + logger.remove() + logger.add(sys.stderr, level="DEBUG") + + # 模拟爬虫运行时的日志 + logger.info("爬虫任务启动") + logger.debug("配置加载完成: max_concurrent=10, timeout=30s") + + # 模拟爬取过程 + for page in range(1, 4): + logger.info(f"开始爬取第 {page} 页") + logger.debug(f"请求URL: https://example.com/page/{page}") + + # 模拟一些情况 + if page == 2: + logger.warning(f"第 {page} 页请求超时，准备重试") + logger.info(f"第 {page} 页重试成功") + + logger.info(f"第 {page} 页爬取完成，获取 20 条数据") + + logger.info("爬虫任务完成，共获取 60 条数据") + + +def main(): + """主函数""" + # 确保日志目录存在 + import os + os.makedirs("logs", exist_ok=True) + + # 运行所有演示 + basic_usage() + custom_format() + file_logging() + error_logging() + context_logging() + # structured_logging() # 取消注释以测试 JSON 日志 + crawler_logging_example() + + +if __name__ == "__main__": + main() diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/pyproject.toml" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/pyproject.toml" new file mode 100644 index 0000000..7cc8074 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/pyproject.toml" @@ -0,0 +1,19 @@ +[project] +name = "chapter01-engineering-standards" +version = "0.1.0" +description = "第01章：工程化爬虫开发规范 - 展示日志、配置管理、异常处理等工程化实践" +readme = "README.md" +requires-python = ">=3.11" +dependencies = [ + "httpx>=0.27.0", + "pydantic>=2.0.0", + "parsel>=1.9.0", + "loguru>=0.7.0", +] + +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[tool.uv] +dev-dependencies = [] diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/refactored_crawler/__init__.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/refactored_crawler/__init__.py" new file mode 100644 index 0000000..92d0381 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/refactored_crawler/__init__.py" @@ -0,0 +1,2 @@ +# -*- coding: utf-8 -*- +# @Desc: 工程化爬虫重构示例 diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/refactored_crawler/client.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/refactored_crawler/client.py" new file mode 100644 index 0000000..3f9b9b0 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/refactored_crawler/client.py" @@ -0,0 +1,108 @@ +# -*- coding: utf-8 -*- +# @Desc: HTTP 客户端封装 + +from typing import Optional + +import httpx +from loguru import logger +from tenacity import ( + retry, + stop_after_attempt, + wait_exponential, + retry_if_exception_type +) + +from .config import settings +from .exceptions import RequestException, TimeoutException + + +class CrawlerClient: + """ + 封装的 HTTP 客户端 + + 特性: + - 统一的请求头配置 + - 自动重试机制 + - 优雅的错误处理 + - 资源自动管理 + """ + + DEFAULT_HEADERS = { + "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) " + "AppleWebKit/537.36 (KHTML, like Gecko) " + "Chrome/131.0.0.0 Safari/537.36", + "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", + "Accept-Language": "zh-CN,zh;q=0.9,en;q=0.8", + } + + def __init__(self): + self._client: Optional[httpx.AsyncClient] = None + + async def __aenter__(self): + await self.start() + return self + + async def __aexit__(self, exc_type, exc_val, exc_tb): + await self.close() + + async def start(self): + """启动客户端""" + if self._client is None: + self._client = httpx.AsyncClient( + timeout=settings.request_timeout, + headers=self.DEFAULT_HEADERS, + follow_redirects=True + ) + logger.debug("HTTP 客户端已启动") + + async def close(self): + """关闭客户端""" + if self._client: + await self._client.aclose() + self._client = None + logger.debug("HTTP 客户端已关闭") + + @retry( + stop=stop_after_attempt(3), + wait=wait_exponential(multiplier=1, max=10), + retry=retry_if_exception_type((httpx.TimeoutException, httpx.ConnectError)) + ) + async def get(self, url: str, **kwargs) -> httpx.Response: + """ + 发送 GET 请求（带重试） + + Args: + url: 请求 URL + **kwargs: 传递给 httpx 的其他参数 + + Returns: + httpx.Response 响应对象 + + Raises: + RequestException: 请求失败 + """ + if not self._client: + await self.start() + + logger.debug(f"GET 请求: {url}") + + try: + response = await self._client.get(url, **kwargs) + response.raise_for_status() + logger.info(f"请求成功: {url} [{response.status_code}]") + return response + + except httpx.TimeoutException as e: + logger.warning(f"请求超时: {url}") + raise TimeoutException(f"请求超时: {url}") from e + + except httpx.HTTPStatusError as e: + logger.error(f"HTTP 错误: {url} [{e.response.status_code}]") + raise RequestException( + f"HTTP 状态码异常: {e.response.status_code}", + url=url + ) from e + + except Exception as e: + logger.exception(f"请求异常: {url}") + raise RequestException(f"请求异常: {e}", url=url) from e diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/refactored_crawler/config.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/refactored_crawler/config.py" new file mode 100644 index 0000000..a0fb362 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/refactored_crawler/config.py" @@ -0,0 +1,33 @@ +# -*- coding: utf-8 -*- +# @Desc: 配置管理 + +from pydantic_settings import BaseSettings +from pydantic import Field +from typing import Optional + + +class Settings(BaseSettings): + """爬虫配置""" + + # 环境配置 + env: str = Field(default="development", description="运行环境") + debug: bool = Field(default=True, description="调试模式") + log_level: str = Field(default="DEBUG", description="日志级别") + + # 爬虫配置 + first_n_page: int = Field(default=3, description="爬取前N页") + base_host: str = Field(default="https://www.ptt.cc", description="目标站点") + request_timeout: int = Field(default=30, description="请求超时") + max_retries: int = Field(default=3, description="最大重试次数") + request_delay: float = Field(default=0.5, description="请求间隔") + + # 输出配置 + output_dir: str = Field(default="./output", description="输出目录") + log_dir: str = Field(default="./logs", description="日志目录") + + class Config: + env_file = ".env" + env_prefix = "CRAWLER_" + + +settings = Settings() diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/refactored_crawler/crawler.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/refactored_crawler/crawler.py" new file mode 100644 index 0000000..e2f4fa3 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/refactored_crawler/crawler.py" @@ -0,0 +1,135 @@ +# -*- coding: utf-8 -*- +# @Desc: 爬虫核心逻辑 + +import asyncio +from typing import List +from loguru import logger + +from .config import settings +from .client import CrawlerClient +from .parser import BBSParser +from .models import NoteItem, NoteDetail, CrawlResult +from .exceptions import CrawlerException + + +class BBSCrawler: + """ + BBS 论坛爬虫 + + 工程化特性: + - 配置驱动 + - 完善的日志记录 + - 优雅的异常处理 + - 资源自动管理 + """ + + def __init__(self): + self.client = CrawlerClient() + self.parser = BBSParser() + self.result = CrawlResult() + + async def run(self) -> CrawlResult: + """ + 运行爬虫 + + Returns: + 爬取结果 + """ + logger.info(f"开始爬取任务 - 目标: 前 {settings.first_n_page} 页") + + async with self.client: + # Step 1: 获取分页信息 + previous_number = await self._get_previous_page_number() + + # Step 2: 爬取帖子列表 + note_items = await self._crawl_note_list(previous_number) + + # Step 3: 爬取帖子详情 + await self._crawl_note_details(note_items) + + logger.info( + f"爬取完成 - 成功: {self.result.success_count}, " + f"失败: {self.result.fail_count}" + ) + + return self.result + + async def _get_previous_page_number(self) -> int: + """获取上一页分页号""" + logger.info("获取分页信息...") + + url = f"{settings.base_host}/bbs/Stock/index.html" + response = await self.client.get(url) + page_number = self.parser.parse_previous_page_number(response.text) + + logger.info(f"当前最新分页号: {page_number + 1}") + return page_number + + async def _crawl_note_list(self, previous_number: int) -> List[NoteItem]: + """爬取帖子列表""" + logger.info(f"开始爬取帖子列表...") + + all_notes: List[NoteItem] = [] + start_page = previous_number + 1 + end_page = start_page - settings.first_n_page + + for page_num in range(start_page, end_page, -1): + try: + url = f"{settings.base_host}/bbs/Stock/index{page_num}.html" + logger.info(f"爬取第 {page_num} 页...") + + response = await self.client.get(url) + notes = self.parser.parse_note_list(response.text) + all_notes.extend(notes) + + logger.info(f"第 {page_num} 页获取 {len(notes)} 个帖子") + + # 请求间隔 + await asyncio.sleep(settings.request_delay) + + except CrawlerException as e: + logger.warning(f"第 {page_num} 页爬取失败: {e}") + continue + + self.result.total_pages = settings.first_n_page + self.result.total_notes = len(all_notes) + logger.info(f"帖子列表爬取完成, 共 {len(all_notes)} 个帖子") + + return all_notes + + async def _crawl_note_details(self, note_items: List[NoteItem]): + """爬取帖子详情""" + logger.info(f"开始爬取帖子详情, 共 {len(note_items)} 个...") + + for idx, note_item in enumerate(note_items, 1): + try: + url = f"{settings.base_host}{note_item.detail_link}" + logger.debug(f"[{idx}/{len(note_items)}] 爬取: {note_item.title[:30]}...") + + response = await self.client.get(url) + detail = self.parser.parse_note_detail(response.text, note_item) + + self.result.notes.append(detail) + self.result.success_count += 1 + + # 请求间隔 + await asyncio.sleep(settings.request_delay) + + except CrawlerException as e: + logger.warning(f"帖子详情爬取失败: {note_item.title[:20]}... - {e}") + self.result.fail_count += 1 + continue + + except Exception as e: + logger.exception(f"未知错误: {e}") + self.result.fail_count += 1 + continue + + # 每 10 个帖子输出一次进度 + if idx % 10 == 0: + logger.info(f"进度: {idx}/{len(note_items)}") + + async def cleanup(self): + """清理资源""" + logger.debug("执行清理...") + await self.client.close() diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/refactored_crawler/exceptions.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/refactored_crawler/exceptions.py" new file mode 100644 index 0000000..a682a24 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/refactored_crawler/exceptions.py" @@ -0,0 +1,46 @@ +# -*- coding: utf-8 -*- +# @Desc: 自定义异常类 + +from typing import Optional + + +class CrawlerException(Exception): + """爬虫基础异常类""" + + def __init__(self, message: str, url: Optional[str] = None): + self.message = message + self.url = url + super().__init__(self.message) + + def __str__(self): + if self.url: + return f"{self.message} (URL: {self.url})" + return self.message + + +class RequestException(CrawlerException): + """HTTP 请求异常""" + pass + + +class TimeoutException(RequestException): + """请求超时异常""" + pass + + +class HTTPStatusException(RequestException): + """HTTP 状态码异常""" + + def __init__(self, message: str, status_code: int, url: Optional[str] = None): + self.status_code = status_code + super().__init__(message, url) + + +class ParseException(CrawlerException): + """数据解析异常""" + pass + + +class StorageException(CrawlerException): + """数据存储异常""" + pass diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/refactored_crawler/logger.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/refactored_crawler/logger.py" new file mode 100644 index 0000000..f9e2a8a --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/refactored_crawler/logger.py" @@ -0,0 +1,51 @@ +# -*- coding: utf-8 -*- +# @Desc: 日志配置 + +import sys +import os +from loguru import logger +from config import settings + + +def setup_logger(): + """配置日志系统""" + # 确保日志目录存在 + os.makedirs(settings.log_dir, exist_ok=True) + + # 移除默认处理器 + logger.remove() + + # 控制台输出 - 彩色格式 + logger.add( + sys.stderr, + level=settings.log_level, + format="{time:YYYY-MM-DD HH:mm:ss} | " + "{level: <8} | " + "{name}:{function}:{line} | " + "{message}", + colorize=True + ) + + # 文件输出 - 所有日志 + logger.add( + f"{settings.log_dir}/crawler_{settings.env}.log", + rotation="00:00", # 每天午夜轮转 + retention="7 days", # 保留7天 + compression="zip", # 压缩旧日志 + level="DEBUG", + encoding="utf-8", + format="{time:YYYY-MM-DD HH:mm:ss} | {level: <8} | {name}:{function}:{line} | {message}" + ) + + # 错误日志单独文件 + logger.add( + f"{settings.log_dir}/error_{settings.env}.log", + rotation="00:00", + retention="30 days", + level="ERROR", + encoding="utf-8", + format="{time:YYYY-MM-DD HH:mm:ss} | {level: <8} | {name}:{function}:{line} | {message}" + ) + + logger.info(f"日志系统初始化完成 - 级别: {settings.log_level}") + return logger diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/refactored_crawler/main.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/refactored_crawler/main.py" new file mode 100644 index 0000000..1ab2369 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/refactored_crawler/main.py" @@ -0,0 +1,78 @@ +# -*- coding: utf-8 -*- +# @Desc: 爬虫主程序入口 + +import asyncio +import json +import os +from loguru import logger + +from .logger import setup_logger +from .config import settings +from .crawler import BBSCrawler +from .exceptions import CrawlerException + + +async def main(): + """主程序""" + # 初始化日志 + setup_logger() + + logger.info("=" * 50) + logger.info("工程化爬虫示例") + logger.info("=" * 50) + logger.info(f"运行环境: {settings.env}") + logger.info(f"调试模式: {settings.debug}") + logger.info(f"目标页数: {settings.first_n_page}") + + # 创建输出目录 + os.makedirs(settings.output_dir, exist_ok=True) + + # 创建爬虫实例 + crawler = BBSCrawler() + + try: + # 运行爬虫 + result = await crawler.run() + + # 保存结果 + output_file = f"{settings.output_dir}/crawl_result.json" + with open(output_file, "w", encoding="utf-8") as f: + json.dump(result.model_dump(), f, ensure_ascii=False, indent=2) + + logger.info(f"结果已保存到: {output_file}") + + # 输出统计 + logger.info("=" * 50) + logger.info("爬取统计") + logger.info("=" * 50) + logger.info(f"总页数: {result.total_pages}") + logger.info(f"总帖子: {result.total_notes}") + logger.info(f"成功数: {result.success_count}") + logger.info(f"失败数: {result.fail_count}") + + except CrawlerException as e: + logger.error(f"爬虫异常: {e}") + raise + + except Exception as e: + logger.exception(f"未预期的异常: {e}") + raise + + finally: + await crawler.cleanup() + logger.info("程序结束") + + +def run(): + """运行入口""" + try: + asyncio.run(main()) + except KeyboardInterrupt: + logger.info("用户中断程序") + except Exception as e: + logger.critical(f"程序异常退出: {e}") + exit(1) + + +if __name__ == "__main__": + run() diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/refactored_crawler/models.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/refactored_crawler/models.py" new file mode 100644 index 0000000..b9abdf0 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/refactored_crawler/models.py" @@ -0,0 +1,39 @@ +# -*- coding: utf-8 -*- +# @Desc: 数据模型定义 + +from typing import List, Optional +from pydantic import BaseModel, Field + + +class NoteItem(BaseModel): + """帖子列表项""" + title: str = Field(default="", description="帖子标题") + author: str = Field(default="", description="作者") + publish_date: str = Field(default="", description="发布日期") + detail_link: str = Field(default="", description="详情链接") + + +class PushComment(BaseModel): + """推文/评论""" + user_name: str = Field(default="", description="用户名") + content: str = Field(default="", description="评论内容") + push_time: str = Field(default="", description="评论时间") + + +class NoteDetail(BaseModel): + """帖子详情""" + title: str = Field(default="", description="帖子标题") + author: str = Field(default="", description="作者") + detail_link: str = Field(default="", description="详情链接") + publish_datetime: str = Field(default="", description="发布时间") + content: str = Field(default="", description="正文内容") + comments: List[PushComment] = Field(default_factory=list, description="评论列表") + + +class CrawlResult(BaseModel): + """爬取结果""" + total_pages: int = Field(default=0, description="爬取的页数") + total_notes: int = Field(default=0, description="帖子总数") + success_count: int = Field(default=0, description="成功数") + fail_count: int = Field(default=0, description="失败数") + notes: List[NoteDetail] = Field(default_factory=list, description="帖子详情列表") diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/refactored_crawler/parser.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/refactored_crawler/parser.py" new file mode 100644 index 0000000..b4d9ba2 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/01_\345\267\245\347\250\213\345\214\226\347\210\254\350\231\253\345\274\200\345\217\221\350\247\204\350\214\203/refactored_crawler/parser.py" @@ -0,0 +1,150 @@ +# -*- coding: utf-8 -*- +# @Desc: 页面解析器 + +from typing import List, Optional +from parsel import Selector +from loguru import logger + +from .models import NoteItem, NoteDetail, PushComment +from .exceptions import ParseException + + +class BBSParser: + """BBS 页面解析器""" + + @staticmethod + def parse_previous_page_number(html: str) -> int: + """ + 解析首页获取上一页的分页号 + + Args: + html: 页面 HTML 内容 + + Returns: + 分页号 + """ + try: + selector = Selector(text=html) + css_selector = "#action-bar-container > div > div.btn-group.btn-group-paging > a:nth-child(2)" + pagination_link = selector.css(css_selector).attrib.get("href", "") + + if not pagination_link: + raise ParseException("无法找到分页链接") + + # 从 /bbs/Stock/index7084.html 提取数字 + page_number = int( + pagination_link + .replace("/bbs/Stock/index", "") + .replace(".html", "") + ) + + logger.debug(f"解析到上一页分页号: {page_number}") + return page_number + + except Exception as e: + logger.error(f"解析分页号失败: {e}") + raise ParseException(f"解析分页号失败: {e}") + + @staticmethod + def parse_note_list(html: str) -> List[NoteItem]: + """ + 解析帖子列表页 + + Args: + html: 页面 HTML 内容 + + Returns: + 帖子列表 + """ + notes = [] + selector = Selector(text=html) + + try: + note_elements = selector.css("div.r-ent") + + for element in note_elements: + title_el = element.css("div.title a") + author_el = element.css("div.meta div.author") + date_el = element.css("div.meta div.date") + + note = NoteItem( + title=title_el.css("::text").get("").strip() if title_el else "", + author=author_el.css("::text").get("").strip() if author_el else "", + publish_date=date_el.css("::text").get("").strip() if date_el else "", + detail_link=title_el.attrib.get("href", "") if title_el else "" + ) + + # 跳过无链接的帖子（可能已删除） + if note.detail_link: + notes.append(note) + + logger.debug(f"解析到 {len(notes)} 个帖子") + return notes + + except Exception as e: + logger.error(f"解析帖子列表失败: {e}") + raise ParseException(f"解析帖子列表失败: {e}") + + @staticmethod + def parse_note_detail(html: str, note_item: NoteItem) -> NoteDetail: + """ + 解析帖子详情页 + + Args: + html: 页面 HTML 内容 + note_item: 帖子基本信息 + + Returns: + 帖子详情 + """ + selector = Selector(text=html) + + try: + # 基本信息从列表项复制 + detail = NoteDetail( + title=note_item.title, + author=note_item.author, + detail_link=note_item.detail_link + ) + + # 解析发布时间 + time_el = selector.css( + "#main-content > div:nth-child(4) > span.article-meta-value" + ) + if time_el: + detail.publish_datetime = time_el.css("::text").get("").strip() + + # 解析正文内容 + # 获取 main-content 的全部文本，排除元信息和评论部分 + main_content = selector.css("#main-content") + if main_content: + # 提取正文：获取所有文本节点，过滤掉元信息和评论 + content_parts = [] + for text_node in main_content.xpath("text()"): + text = text_node.get().strip() + if text and not text.startswith("--"): # 排除签名分隔线 + content_parts.append(text) + detail.content = "\n".join(content_parts) + + # 解析评论 + comments = [] + push_elements = selector.css("#main-content > div.push") + + for push_el in push_elements: + spans = push_el.css("span") + if len(spans) >= 4: + comment = PushComment( + user_name=spans[1].css("::text").get("").strip(), + content=spans[2].css("::text").get("").strip().lstrip(": "), + push_time=spans[3].css("::text").get("").strip() + ) + comments.append(comment) + + detail.comments = comments + logger.debug(f"解析帖子详情完成: {detail.title}, 评论数: {len(comments)}") + + return detail + + except Exception as e: + logger.error(f"解析帖子详情失败: {e}") + raise ParseException(f"解析帖子详情失败: {e}") diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/02_\345\217\215\347\210\254\350\231\253\345\257\271\346\212\227\345\237\272\347\241\200_\350\257\267\346\261\202\344\274\252\350\243\205/README.md" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/02_\345\217\215\347\210\254\350\231\253\345\257\271\346\212\227\345\237\272\347\241\200_\350\257\267\346\261\202\344\274\252\350\243\205/README.md" new file mode 100644 index 0000000..98b75ac --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/02_\345\217\215\347\210\254\350\231\253\345\257\271\346\212\227\345\237\272\347\241\200_\350\257\267\346\261\202\344\274\252\350\243\205/README.md" @@ -0,0 +1,37 @@ +# 第02章：反爬虫对抗基础 - 请求伪装 + +展示User-Agent轮换、请求头伪装、速率控制等反爬技术。 + +## 快速开始 + +### 使用 uv 安装依赖 + +```bash +cd 02_反爬虫对抗基础_请求伪装 + +# 安装基础依赖 +uv sync + +# 安装高级功能（TLS指纹伪装，可选） +uv sync --extra advanced + +# 运行示例 +uv run python ua_rotator.py +uv run python headers_builder.py +uv run python rate_limiter.py +uv run python anti_detection_crawler.py +``` + +### 核心依赖 + +- `httpx` - 异步HTTP客户端 +- `loguru` - 日志系统 +- `fake-useragent` - User-Agent生成 +- `curl-cffi`（可选）- TLS指纹伪装 + +## 主要文件 + +- `ua_rotator.py` - User-Agent轮换器 +- `headers_builder.py` - 请求头构建器 +- `rate_limiter.py` - 速率限制器 +- `anti_detection_crawler.py` - 完整反检测爬虫示例 diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/02_\345\217\215\347\210\254\350\231\253\345\257\271\346\212\227\345\237\272\347\241\200_\350\257\267\346\261\202\344\274\252\350\243\205/anti_detection_crawler.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/02_\345\217\215\347\210\254\350\231\253\345\257\271\346\212\227\345\237\272\347\241\200_\350\257\267\346\261\202\344\274\252\350\243\205/anti_detection_crawler.py" new file mode 100644 index 0000000..d3f62c0 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/02_\345\217\215\347\210\254\350\231\253\345\257\271\346\212\227\345\237\272\347\241\200_\350\257\267\346\261\202\344\274\252\350\243\205/anti_detection_crawler.py" @@ -0,0 +1,254 @@ +# -*- coding: utf-8 -*- +# @Desc: 完整的反检测爬虫示例 +# 结合 UA 轮换、请求头伪装、速率控制的实战案例 + +import asyncio +import random +from typing import List, Dict, Optional, Any +from dataclasses import dataclass +from loguru import logger + +# 导入本章的模块 +from ua_rotator import UARotator +from headers_builder import HeadersBuilder +from rate_limiter import CompositeRateLimiter, RateLimitConfig + +# 尝试导入 curl_cffi，如果不可用则使用 httpx +try: + from curl_cffi.requests import AsyncSession + USE_CURL_CFFI = True + logger.info("使用 curl_cffi 作为 HTTP 客户端（支持 TLS 指纹模拟）") +except ImportError: + import httpx + USE_CURL_CFFI = False + logger.info("使用 httpx 作为 HTTP 客户端") + + +@dataclass +class CrawlerConfig: + """爬虫配置""" + # 目标站点 + base_url: str = "https://httpbin.org" + + # 速率限制 + requests_per_second: float = 2.0 + max_concurrent: int = 3 + min_delay: float = 0.5 + max_delay: float = 1.5 + + # 请求配置 + timeout: int = 30 + max_retries: int = 3 + + # curl_cffi 配置 + browser_type: str = "chrome120" + + +class AntiDetectionCrawler: + """ + 反检测爬虫 + + 特性： + - User-Agent 随机轮换 + - 完整的请求头伪装 + - TLS 指纹模拟（使用 curl_cffi） + - 智能速率控制 + - 优雅的重试机制 + """ + + def __init__(self, config: Optional[CrawlerConfig] = None): + """ + 初始化爬虫 + + Args: + config: 爬虫配置 + """ + self.config = config or CrawlerConfig() + + # 初始化组件 + self.ua_rotator = UARotator() + self.headers_builder = HeadersBuilder(self.ua_rotator) + self.rate_limiter = CompositeRateLimiter( + RateLimitConfig( + requests_per_second=self.config.requests_per_second, + max_concurrent=self.config.max_concurrent, + min_delay=self.config.min_delay, + max_delay=self.config.max_delay + ) + ) + + self._session = None + self._stats = { + "total_requests": 0, + "success_requests": 0, + "failed_requests": 0 + } + + async def __aenter__(self): + await self.start() + return self + + async def __aexit__(self, *args): + await self.close() + + async def start(self): + """启动爬虫""" + if USE_CURL_CFFI: + self._session = AsyncSession( + impersonate=self.config.browser_type, + timeout=self.config.timeout + ) + else: + self._session = httpx.AsyncClient(timeout=self.config.timeout) + + logger.info(f"爬虫启动 - 目标: {self.config.base_url}") + + async def close(self): + """关闭爬虫""" + if self._session: + if USE_CURL_CFFI: + await self._session.close() + else: + await self._session.aclose() + self._session = None + + logger.info(f"爬虫关闭 - 统计: {self._stats}") + + async def fetch( + self, + url: str, + referer: Optional[str] = None, + is_api: bool = False + ) -> Optional[Dict[str, Any]]: + """ + 获取页面/API 数据 + + Args: + url: 目标 URL + referer: Referer 地址 + is_api: 是否是 API 请求 + + Returns: + 响应数据或 None + """ + async with self.rate_limiter: + self._stats["total_requests"] += 1 + + # 构建请求头 + if is_api: + headers = self.headers_builder.build_api_headers( + referer=referer or self.config.base_url + ) + else: + headers = self.headers_builder.build_page_headers(referer=referer) + + # 重试逻辑 + for attempt in range(self.config.max_retries): + try: + logger.debug(f"请求: {url} (尝试 {attempt + 1}/{self.config.max_retries})") + + response = await self._session.get(url, headers=headers) + + # 检查状态码 + if USE_CURL_CFFI: + status_code = response.status_code + if status_code >= 400: + raise Exception(f"HTTP {status_code}") + data = response.json() if is_api else {"text": response.text} + else: + response.raise_for_status() + data = response.json() if is_api else {"text": response.text} + + self._stats["success_requests"] += 1 + logger.info(f"成功: {url}") + return data + + except Exception as e: + logger.warning(f"请求失败: {url} - {e}") + if attempt < self.config.max_retries - 1: + # 等待后重试 + await asyncio.sleep(random.uniform(1, 3)) + else: + self._stats["failed_requests"] += 1 + logger.error(f"最终失败: {url}") + return None + + async def fetch_batch( + self, + urls: List[str], + referer: Optional[str] = None + ) -> List[Optional[Dict[str, Any]]]: + """ + 批量获取数据 + + Args: + urls: URL 列表 + referer: Referer 地址 + + Returns: + 结果列表 + """ + tasks = [self.fetch(url, referer) for url in urls] + return await asyncio.gather(*tasks) + + +async def demo(): + """演示反检测爬虫""" + logger.remove() + logger.add( + lambda msg: print(msg, end=""), + format="{time:HH:mm:ss} | {level: <8} | {message}", + level="DEBUG" + ) + + print("=" * 60) + print("反检测爬虫演示") + print("=" * 60) + + config = CrawlerConfig( + base_url="https://httpbin.org", + requests_per_second=1.0, + max_concurrent=2, + min_delay=0.5, + max_delay=1.0 + ) + + async with AntiDetectionCrawler(config) as crawler: + # 测试 1: 检查请求头 + print("\n--- 测试 1: 检查请求头 ---") + result = await crawler.fetch( + "https://httpbin.org/headers", + is_api=True + ) + if result: + headers = result.get("headers", {}) + print(f"User-Agent: {headers.get('User-Agent', 'N/A')[:60]}...") + print(f"Accept: {headers.get('Accept', 'N/A')[:40]}...") + + # 测试 2: 检查 IP + print("\n--- 测试 2: 检查 IP ---") + result = await crawler.fetch( + "https://httpbin.org/ip", + is_api=True + ) + if result: + print(f"Origin IP: {result.get('origin', 'N/A')}") + + # 测试 3: 批量请求（测试速率控制） + print("\n--- 测试 3: 批量请求（速率控制） ---") + urls = [ + "https://httpbin.org/get?id=1", + "https://httpbin.org/get?id=2", + "https://httpbin.org/get?id=3", + "https://httpbin.org/get?id=4", + ] + results = await crawler.fetch_batch(urls) + print(f"批量请求完成: 成功 {sum(1 for r in results if r)}/{len(urls)}") + + print("\n" + "=" * 60) + print("演示完成") + print("=" * 60) + + +if __name__ == "__main__": + asyncio.run(demo()) diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/02_\345\217\215\347\210\254\350\231\253\345\257\271\346\212\227\345\237\272\347\241\200_\350\257\267\346\261\202\344\274\252\350\243\205/headers_builder.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/02_\345\217\215\347\210\254\350\231\253\345\257\271\346\212\227\345\237\272\347\241\200_\350\257\267\346\261\202\344\274\252\350\243\205/headers_builder.py" new file mode 100644 index 0000000..805d18f --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/02_\345\217\215\347\210\254\350\231\253\345\257\271\346\212\227\345\237\272\347\241\200_\350\257\267\346\261\202\344\274\252\350\243\205/headers_builder.py" @@ -0,0 +1,216 @@ +# -*- coding: utf-8 -*- +# @Desc: 请求头构建器 + +from typing import Dict, Optional +from ua_rotator import UARotator + + +class HeadersBuilder: + """ + HTTP 请求头构建器 + + 功能： + - 构建完整的浏览器请求头 + - 支持页面请求和 API 请求 + - 自动设置 Referer 和 Origin + - 集成 UA 轮换 + """ + + # 页面请求头模板（访问 HTML 页面） + PAGE_HEADERS_TEMPLATE = { + "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", + "Accept-Language": "zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6", + "Accept-Encoding": "gzip, deflate, br", + "Connection": "keep-alive", + "Upgrade-Insecure-Requests": "1", + "Sec-Ch-Ua": '"Not_A Brand";v="8", "Chromium";v="131", "Google Chrome";v="131"', + "Sec-Ch-Ua-Mobile": "?0", + "Sec-Ch-Ua-Platform": '"Windows"', + "Sec-Fetch-Dest": "document", + "Sec-Fetch-Mode": "navigate", + "Sec-Fetch-Site": "none", + "Sec-Fetch-User": "?1", + "Cache-Control": "max-age=0", + } + + # API 请求头模板（AJAX/Fetch 请求） + API_HEADERS_TEMPLATE = { + "Accept": "application/json, text/plain, */*", + "Accept-Language": "zh-CN,zh;q=0.9,en;q=0.8", + "Accept-Encoding": "gzip, deflate, br", + "Connection": "keep-alive", + "Sec-Ch-Ua": '"Not_A Brand";v="8", "Chromium";v="131", "Google Chrome";v="131"', + "Sec-Ch-Ua-Mobile": "?0", + "Sec-Ch-Ua-Platform": '"Windows"', + "Sec-Fetch-Dest": "empty", + "Sec-Fetch-Mode": "cors", + "Sec-Fetch-Site": "same-origin", + } + + def __init__(self, ua_rotator: Optional[UARotator] = None): + """ + 初始化请求头构建器 + + Args: + ua_rotator: UA 轮换器实例，如果不提供则创建默认实例 + """ + self.ua_rotator = ua_rotator or UARotator() + + def build_page_headers( + self, + referer: Optional[str] = None, + host: Optional[str] = None, + extra_headers: Optional[Dict[str, str]] = None + ) -> Dict[str, str]: + """ + 构建页面请求头 + + Args: + referer: Referer 地址 + host: Host 地址 + extra_headers: 额外的请求头 + + Returns: + 完整的请求头字典 + """ + headers = self.PAGE_HEADERS_TEMPLATE.copy() + headers["User-Agent"] = self.ua_rotator.get_random() + + if host: + headers["Host"] = host + + if referer: + headers["Referer"] = referer + # 有 Referer 时，Sec-Fetch-Site 应该改为 same-origin 或 cross-site + headers["Sec-Fetch-Site"] = "same-origin" + + if extra_headers: + headers.update(extra_headers) + + return headers + + def build_api_headers( + self, + referer: str, + origin: Optional[str] = None, + content_type: Optional[str] = None, + extra_headers: Optional[Dict[str, str]] = None + ) -> Dict[str, str]: + """ + 构建 API 请求头 + + Args: + referer: Referer 地址（API 请求通常需要） + origin: Origin 地址（POST 请求通常需要） + content_type: Content-Type + extra_headers: 额外的请求头 + + Returns: + 完整的请求头字典 + """ + headers = self.API_HEADERS_TEMPLATE.copy() + headers["User-Agent"] = self.ua_rotator.get_random() + headers["Referer"] = referer + + if origin: + headers["Origin"] = origin + + if content_type: + headers["Content-Type"] = content_type + + if extra_headers: + headers.update(extra_headers) + + return headers + + def build_ajax_headers( + self, + referer: str, + x_requested_with: bool = True + ) -> Dict[str, str]: + """ + 构建传统 AJAX 请求头 + + Args: + referer: Referer 地址 + x_requested_with: 是否添加 X-Requested-With 头 + + Returns: + AJAX 请求头字典 + """ + headers = self.build_api_headers(referer=referer) + + if x_requested_with: + headers["X-Requested-With"] = "XMLHttpRequest" + + return headers + + def build_mobile_headers( + self, + referer: Optional[str] = None, + extra_headers: Optional[Dict[str, str]] = None + ) -> Dict[str, str]: + """ + 构建移动端请求头 + + Args: + referer: Referer 地址 + extra_headers: 额外的请求头 + + Returns: + 移动端请求头字典 + """ + headers = self.PAGE_HEADERS_TEMPLATE.copy() + headers["User-Agent"] = self.ua_rotator.get_mobile() + headers["Sec-Ch-Ua-Mobile"] = "?1" + headers["Sec-Ch-Ua-Platform"] = '"Android"' + + if referer: + headers["Referer"] = referer + headers["Sec-Fetch-Site"] = "same-origin" + + if extra_headers: + headers.update(extra_headers) + + return headers + + +def demo(): + """演示请求头构建器的使用""" + print("=" * 60) + print("请求头构建器演示") + print("=" * 60) + + builder = HeadersBuilder() + + print("\n1. 页面请求头:") + page_headers = builder.build_page_headers( + referer="https://example.com/list", + host="example.com" + ) + for key, value in list(page_headers.items())[:8]: + print(f" {key}: {value[:50]}...") + + print("\n2. API 请求头:") + api_headers = builder.build_api_headers( + referer="https://example.com/page", + origin="https://example.com", + content_type="application/json" + ) + for key, value in list(api_headers.items())[:6]: + print(f" {key}: {value[:50]}...") + + print("\n3. AJAX 请求头:") + ajax_headers = builder.build_ajax_headers( + referer="https://example.com/dashboard" + ) + print(f" X-Requested-With: {ajax_headers.get('X-Requested-With')}") + + print("\n4. 移动端请求头:") + mobile_headers = builder.build_mobile_headers() + print(f" User-Agent: {mobile_headers['User-Agent'][:60]}...") + print(f" Sec-Ch-Ua-Mobile: {mobile_headers['Sec-Ch-Ua-Mobile']}") + + +if __name__ == "__main__": + demo() diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/02_\345\217\215\347\210\254\350\231\253\345\257\271\346\212\227\345\237\272\347\241\200_\350\257\267\346\261\202\344\274\252\350\243\205/pyproject.toml" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/02_\345\217\215\347\210\254\350\231\253\345\257\271\346\212\227\345\237\272\347\241\200_\350\257\267\346\261\202\344\274\252\350\243\205/pyproject.toml" new file mode 100644 index 0000000..72fb8b3 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/02_\345\217\215\347\210\254\350\231\253\345\257\271\346\212\227\345\237\272\347\241\200_\350\257\267\346\261\202\344\274\252\350\243\205/pyproject.toml" @@ -0,0 +1,23 @@ +[project] +name = "chapter02-anti-detection-basics" +version = "0.1.0" +description = "第02章：反爬虫对抗基础 - 请求伪装、User-Agent轮换、速率控制" +readme = "README.md" +requires-python = ">=3.11" +dependencies = [ + "httpx>=0.27.0", + "loguru>=0.7.0", + "fake-useragent>=1.5.0", +] + +[project.optional-dependencies] +advanced = [ + "curl-cffi>=0.7.0", # TLS指纹伪装（可选） +] + +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[tool.uv] +dev-dependencies = [] diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/02_\345\217\215\347\210\254\350\231\253\345\257\271\346\212\227\345\237\272\347\241\200_\350\257\267\346\261\202\344\274\252\350\243\205/rate_limiter.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/02_\345\217\215\347\210\254\350\231\253\345\257\271\346\212\227\345\237\272\347\241\200_\350\257\267\346\261\202\344\274\252\350\243\205/rate_limiter.py" new file mode 100644 index 0000000..7b19e1f --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/02_\345\217\215\347\210\254\350\231\253\345\257\271\346\212\227\345\237\272\347\241\200_\350\257\267\346\261\202\344\274\252\350\243\205/rate_limiter.py" @@ -0,0 +1,380 @@ +# -*- coding: utf-8 -*- +# @Desc: 速率限制器实现 + +import asyncio +import time +import random +from typing import Optional, List, Any, Callable, Coroutine +from dataclasses import dataclass + + +class TokenBucket: + """ + 令牌桶限速器 + + 工作原理： + - 桶有固定容量（最大令牌数） + - 以固定速率向桶中添加令牌 + - 每次请求消耗一个令牌 + - 桶空时请求需要等待 + + 适用场景： + - 需要精确控制平均请求速率 + - 允许一定程度的突发请求 + """ + + def __init__( + self, + rate: float, + capacity: Optional[int] = None + ): + """ + 初始化令牌桶 + + Args: + rate: 每秒添加的令牌数（即每秒最多请求数） + capacity: 桶容量，默认等于 rate（允许的最大突发数） + """ + self.rate = rate + self.capacity = capacity if capacity is not None else int(rate) + self.tokens = float(self.capacity) # 初始令牌数 + self.last_time = time.monotonic() + self._lock = asyncio.Lock() + + async def acquire(self, tokens: int = 1) -> float: + """ + 获取令牌 + + Args: + tokens: 需要的令牌数 + + Returns: + 实际等待的时间（秒） + """ + async with self._lock: + now = time.monotonic() + + # 计算从上次到现在应该添加的令牌数 + elapsed = now - self.last_time + self.tokens = min( + float(self.capacity), + self.tokens + elapsed * self.rate + ) + self.last_time = now + + # 如果令牌不足，计算需要等待的时间 + wait_time = 0.0 + if self.tokens < tokens: + wait_time = (tokens - self.tokens) / self.rate + await asyncio.sleep(wait_time) + self.tokens = 0 + else: + self.tokens -= tokens + + return wait_time + + async def __aenter__(self): + await self.acquire() + return self + + async def __aexit__(self, *args): + pass + + +class RandomDelayLimiter: + """ + 随机延迟限速器 + + 在每次请求后添加随机延迟，模拟人类行为 + """ + + def __init__( + self, + min_delay: float = 1.0, + max_delay: float = 3.0 + ): + """ + 初始化随机延迟限速器 + + Args: + min_delay: 最小延迟（秒） + max_delay: 最大延迟（秒） + """ + self.min_delay = min_delay + self.max_delay = max_delay + + async def wait(self) -> float: + """ + 等待随机时间 + + Returns: + 实际等待的时间（秒） + """ + delay = random.uniform(self.min_delay, self.max_delay) + await asyncio.sleep(delay) + return delay + + async def __aenter__(self): + return self + + async def __aexit__(self, *args): + await self.wait() + + +class AdaptiveRateLimiter: + """ + 自适应速率限制器 + + 根据响应情况自动调整请求速率： + - 正常响应：逐渐提升速率 + - 被限制/错误：降低速率 + """ + + def __init__( + self, + initial_rate: float = 1.0, + min_rate: float = 0.1, + max_rate: float = 5.0, + increase_factor: float = 1.1, + decrease_factor: float = 0.5 + ): + """ + 初始化自适应限速器 + + Args: + initial_rate: 初始速率（请求/秒） + min_rate: 最小速率 + max_rate: 最大速率 + increase_factor: 成功时的速率增长因子 + decrease_factor: 失败时的速率降低因子 + """ + self.current_rate = initial_rate + self.min_rate = min_rate + self.max_rate = max_rate + self.increase_factor = increase_factor + self.decrease_factor = decrease_factor + self._bucket = TokenBucket(rate=initial_rate) + self._lock = asyncio.Lock() + + async def acquire(self) -> float: + """获取令牌""" + return await self._bucket.acquire() + + async def report_success(self): + """报告请求成功，提升速率""" + async with self._lock: + new_rate = min( + self.current_rate * self.increase_factor, + self.max_rate + ) + if new_rate != self.current_rate: + self.current_rate = new_rate + self._bucket = TokenBucket(rate=self.current_rate) + + async def report_failure(self): + """报告请求失败/被限制，降低速率""" + async with self._lock: + new_rate = max( + self.current_rate * self.decrease_factor, + self.min_rate + ) + if new_rate != self.current_rate: + self.current_rate = new_rate + self._bucket = TokenBucket(rate=self.current_rate) + # 失败后额外等待 + await asyncio.sleep(2.0) + + @property + def rate(self) -> float: + """当前速率""" + return self.current_rate + + +class ConcurrencyLimiter: + """ + 并发限制器 + + 限制同时进行的请求数量 + """ + + def __init__(self, max_concurrent: int = 10): + """ + 初始化并发限制器 + + Args: + max_concurrent: 最大并发数 + """ + self.max_concurrent = max_concurrent + self.semaphore = asyncio.Semaphore(max_concurrent) + self._active_count = 0 + self._lock = asyncio.Lock() + + async def acquire(self): + """获取并发许可""" + await self.semaphore.acquire() + async with self._lock: + self._active_count += 1 + + async def release(self): + """释放并发许可""" + self.semaphore.release() + async with self._lock: + self._active_count -= 1 + + @property + def active_count(self) -> int: + """当前活跃请求数""" + return self._active_count + + async def __aenter__(self): + await self.acquire() + return self + + async def __aexit__(self, *args): + await self.release() + + +@dataclass +class RateLimitConfig: + """速率限制配置""" + requests_per_second: float = 2.0 + max_concurrent: int = 5 + min_delay: float = 0.5 + max_delay: float = 2.0 + + +class CompositeRateLimiter: + """ + 组合速率限制器 + + 结合多种限速策略： + - 令牌桶控制平均速率 + - 并发限制控制同时请求数 + - 随机延迟模拟人类行为 + """ + + def __init__(self, config: Optional[RateLimitConfig] = None): + """ + 初始化组合限速器 + + Args: + config: 速率限制配置 + """ + self.config = config or RateLimitConfig() + self.token_bucket = TokenBucket(rate=self.config.requests_per_second) + self.concurrency_limiter = ConcurrencyLimiter(self.config.max_concurrent) + self.delay_limiter = RandomDelayLimiter( + self.config.min_delay, + self.config.max_delay + ) + + async def __aenter__(self): + """进入限速上下文""" + # 先获取并发许可 + await self.concurrency_limiter.acquire() + # 再获取令牌 + await self.token_bucket.acquire() + return self + + async def __aexit__(self, *args): + """退出限速上下文""" + # 随机延迟 + await self.delay_limiter.wait() + # 释放并发许可 + await self.concurrency_limiter.release() + + +# ==================== 演示代码 ==================== + +async def demo_token_bucket(): + """演示令牌桶限速器""" + print("\n" + "=" * 50) + print("1. 令牌桶限速器演示") + print("=" * 50) + + # 每秒 2 个请求 + limiter = TokenBucket(rate=2.0) + + start_time = time.time() + for i in range(6): + async with limiter: + elapsed = time.time() - start_time + print(f" 请求 {i+1}: 时间 {elapsed:.2f}s") + + +async def demo_random_delay(): + """演示随机延迟限速器""" + print("\n" + "=" * 50) + print("2. 随机延迟限速器演示") + print("=" * 50) + + limiter = RandomDelayLimiter(min_delay=0.5, max_delay=1.5) + + for i in range(3): + print(f" 请求 {i+1}...") + async with limiter: + pass # 请求完成后自动延迟 + + +async def demo_concurrency(): + """演示并发限制器""" + print("\n" + "=" * 50) + print("3. 并发限制器演示") + print("=" * 50) + + limiter = ConcurrencyLimiter(max_concurrent=3) + + async def task(task_id: int): + async with limiter: + print(f" 任务 {task_id} 开始, 当前并发: {limiter.active_count}") + await asyncio.sleep(0.5) + print(f" 任务 {task_id} 完成") + + # 同时启动 6 个任务，但最多 3 个并发 + tasks = [task(i) for i in range(6)] + await asyncio.gather(*tasks) + + +async def demo_composite(): + """演示组合限速器""" + print("\n" + "=" * 50) + print("4. 组合限速器演示") + print("=" * 50) + + config = RateLimitConfig( + requests_per_second=2.0, + max_concurrent=2, + min_delay=0.3, + max_delay=0.8 + ) + limiter = CompositeRateLimiter(config) + + start_time = time.time() + + async def request(req_id: int): + async with limiter: + elapsed = time.time() - start_time + print(f" 请求 {req_id}: 时间 {elapsed:.2f}s, 并发: {limiter.concurrency_limiter.active_count}") + + tasks = [request(i) for i in range(5)] + await asyncio.gather(*tasks) + + +async def main(): + """主函数""" + print("=" * 50) + print("速率限制器演示") + print("=" * 50) + + await demo_token_bucket() + await demo_random_delay() + await demo_concurrency() + await demo_composite() + + print("\n" + "=" * 50) + print("演示完成") + print("=" * 50) + + +if __name__ == "__main__": + asyncio.run(main()) diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/02_\345\217\215\347\210\254\350\231\253\345\257\271\346\212\227\345\237\272\347\241\200_\350\257\267\346\261\202\344\274\252\350\243\205/ua_rotator.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/02_\345\217\215\347\210\254\350\231\253\345\257\271\346\212\227\345\237\272\347\241\200_\350\257\267\346\261\202\344\274\252\350\243\205/ua_rotator.py" new file mode 100644 index 0000000..b4e168d --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/02_\345\217\215\347\210\254\350\231\253\345\257\271\346\212\227\345\237\272\347\241\200_\350\257\267\346\261\202\344\274\252\350\243\205/ua_rotator.py" @@ -0,0 +1,183 @@ +# -*- coding: utf-8 -*- +# @Desc: User-Agent 轮换器实现 + +import random +from typing import List, Optional + +# 尝试导入 fake_useragent +try: + from fake_useragent import UserAgent + FAKE_UA_AVAILABLE = True +except ImportError: + FAKE_UA_AVAILABLE = False + + +# 预定义的 User-Agent 列表（2025年主流浏览器版本） +DESKTOP_USER_AGENTS = [ + # Chrome - Windows + "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36", + "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36", + "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36", + # Chrome - Mac + "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36", + "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36", + # Firefox - Windows + "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:133.0) Gecko/20100101 Firefox/133.0", + "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:132.0) Gecko/20100101 Firefox/132.0", + # Firefox - Mac + "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:133.0) Gecko/20100101 Firefox/133.0", + # Safari - Mac + "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.2 Safari/605.1.15", + "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.1 Safari/605.1.15", + # Edge - Windows + "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36 Edg/131.0.0.0", +] + +MOBILE_USER_AGENTS = [ + # iPhone - Safari + "Mozilla/5.0 (iPhone; CPU iPhone OS 18_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.2 Mobile/15E148 Safari/604.1", + "Mozilla/5.0 (iPhone; CPU iPhone OS 18_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.1 Mobile/15E148 Safari/604.1", + # iPhone - Chrome + "Mozilla/5.0 (iPhone; CPU iPhone OS 18_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) CriOS/131.0.6778.103 Mobile/15E148 Safari/604.1", + # Android - Chrome + "Mozilla/5.0 (Linux; Android 15; Pixel 9) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.6778.104 Mobile Safari/537.36", + "Mozilla/5.0 (Linux; Android 14; SM-S928B) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.6778.104 Mobile Safari/537.36", + "Mozilla/5.0 (Linux; Android 14; SM-G998B) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.6778.104 Mobile Safari/537.36", +] + + +class UARotator: + """ + User-Agent 轮换器 + + 支持： + - 随机获取桌面端/移动端 UA + - 自定义 UA 列表 + - 集成 fake-useragent 库（可选） + """ + + def __init__( + self, + use_fake_ua: bool = False, + custom_uas: Optional[List[str]] = None + ): + """ + 初始化 UA 轮换器 + + Args: + use_fake_ua: 是否使用 fake-useragent 库 + custom_uas: 自定义 UA 列表（优先级最高） + """ + self.custom_uas = custom_uas or [] + self.use_fake_ua = use_fake_ua and FAKE_UA_AVAILABLE + self._fake_ua = None + + if self.use_fake_ua: + try: + self._fake_ua = UserAgent() + except Exception as e: + print(f"Warning: fake-useragent 初始化失败: {e}") + self.use_fake_ua = False + + def get_random(self) -> str: + """ + 获取随机 User-Agent（桌面端） + + Returns: + 随机的 User-Agent 字符串 + """ + # 优先使用自定义列表 + if self.custom_uas: + return random.choice(self.custom_uas) + + # 使用 fake-useragent + if self.use_fake_ua and self._fake_ua: + try: + return self._fake_ua.random + except Exception: + pass + + # 使用预定义列表 + return random.choice(DESKTOP_USER_AGENTS) + + def get_chrome(self) -> str: + """获取 Chrome User-Agent""" + if self.use_fake_ua and self._fake_ua: + try: + return self._fake_ua.chrome + except Exception: + pass + + # 从预定义列表中筛选 Chrome UA + chrome_uas = [ua for ua in DESKTOP_USER_AGENTS if "Chrome" in ua and "Edg" not in ua] + return random.choice(chrome_uas) if chrome_uas else self.get_random() + + def get_firefox(self) -> str: + """获取 Firefox User-Agent""" + if self.use_fake_ua and self._fake_ua: + try: + return self._fake_ua.firefox + except Exception: + pass + + firefox_uas = [ua for ua in DESKTOP_USER_AGENTS if "Firefox" in ua] + return random.choice(firefox_uas) if firefox_uas else self.get_random() + + def get_safari(self) -> str: + """获取 Safari User-Agent""" + if self.use_fake_ua and self._fake_ua: + try: + return self._fake_ua.safari + except Exception: + pass + + safari_uas = [ua for ua in DESKTOP_USER_AGENTS if "Safari" in ua and "Chrome" not in ua] + return random.choice(safari_uas) if safari_uas else self.get_random() + + def get_mobile(self) -> str: + """获取移动端 User-Agent""" + return random.choice(MOBILE_USER_AGENTS) + + def get_ios(self) -> str: + """获取 iOS User-Agent""" + ios_uas = [ua for ua in MOBILE_USER_AGENTS if "iPhone" in ua] + return random.choice(ios_uas) if ios_uas else self.get_mobile() + + def get_android(self) -> str: + """获取 Android User-Agent""" + android_uas = [ua for ua in MOBILE_USER_AGENTS if "Android" in ua] + return random.choice(android_uas) if android_uas else self.get_mobile() + + +def demo(): + """演示 UA 轮换器的使用""" + print("=" * 60) + print("User-Agent 轮换器演示") + print("=" * 60) + + rotator = UARotator() + + print("\n1. 随机桌面端 UA:") + for i in range(3): + print(f" {i+1}. {rotator.get_random()[:80]}...") + + print("\n2. Chrome UA:") + print(f" {rotator.get_chrome()[:80]}...") + + print("\n3. Firefox UA:") + print(f" {rotator.get_firefox()[:80]}...") + + print("\n4. 移动端 UA:") + print(f" iOS: {rotator.get_ios()[:60]}...") + print(f" Android: {rotator.get_android()[:60]}...") + + # 测试 fake-useragent（如果可用） + if FAKE_UA_AVAILABLE: + print("\n5. 使用 fake-useragent:") + rotator_fake = UARotator(use_fake_ua=True) + for i in range(3): + print(f" {i+1}. {rotator_fake.get_random()[:80]}...") + + +if __name__ == "__main__": + demo() diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/03_\344\273\243\347\220\206IP\347\232\204\344\275\277\347\224\250\344\270\216\347\256\241\347\220\206/README.md" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/03_\344\273\243\347\220\206IP\347\232\204\344\275\277\347\224\250\344\270\216\347\256\241\347\220\206/README.md" new file mode 100644 index 0000000..db69474 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/03_\344\273\243\347\220\206IP\347\232\204\344\275\277\347\224\250\344\270\216\347\256\241\347\220\206/README.md" @@ -0,0 +1,20 @@ +# 第03章：代理IP的使用与管理 + +展示代理池设计、代理检测、多URL测试等代理管理功能。 + +## 快速开始 + +```bash +cd 03_代理IP的使用与管理 +uv sync +uv run python proxy_demo.py +``` + +### 核心依赖 +- `httpx` - HTTP客户端 +- `loguru` - 日志系统 + +## 测试URL +- `httpbin.org/ip` - IP检测 +- `api.ipify.org` - IP服务 +- `ip-api.com/json/` - IP地理位置 diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/03_\344\273\243\347\220\206IP\347\232\204\344\275\277\347\224\250\344\270\216\347\256\241\347\220\206/proxy_checker.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/03_\344\273\243\347\220\206IP\347\232\204\344\275\277\347\224\250\344\270\216\347\256\241\347\220\206/proxy_checker.py" new file mode 100644 index 0000000..0790c98 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/03_\344\273\243\347\220\206IP\347\232\204\344\275\277\347\224\250\344\270\216\347\256\241\347\220\206/proxy_checker.py" @@ -0,0 +1,169 @@ +# -*- coding: utf-8 -*- +# @Desc: 独立的代理检测脚本 +# 用于快速检测代理可用性 + +import asyncio +import sys +from typing import List +from loguru import logger + +# 添加模块路径 +sys.path.insert(0, ".") + +from proxy_pool.base import ProxyInfo, ProxyProtocol +from proxy_pool.checker import ProxyChecker + + +async def check_single_proxy(proxy_str: str) -> bool: + """ + 检测单个代理 + + Args: + proxy_str: 代理字符串，格式为 host:port 或 protocol://host:port + + Returns: + 代理是否可用 + """ + # 解析代理字符串 + if "://" in proxy_str: + protocol, rest = proxy_str.split("://") + host, port = rest.split(":") + protocol = ProxyProtocol(protocol) + else: + host, port = proxy_str.split(":") + protocol = ProxyProtocol.HTTP + + proxy = ProxyInfo( + host=host, + port=int(port), + protocol=protocol + ) + + checker = ProxyChecker(timeout=10) + is_valid = await checker.check(proxy) + + if is_valid: + print(f"✓ 代理可用: {proxy.url}") + print(f" 响应时间: {proxy.avg_response_time:.2f}s") + else: + print(f"✗ 代理不可用: {proxy.url}") + + return is_valid + + +async def check_proxy_list(proxy_list: List[str]) -> List[ProxyInfo]: + """ + 批量检测代理列表 + + Args: + proxy_list: 代理字符串列表 + + Returns: + 可用的代理列表 + """ + proxies = [] + + for proxy_str in proxy_list: + try: + if "://" in proxy_str: + protocol, rest = proxy_str.split("://") + host, port = rest.split(":") + protocol = ProxyProtocol(protocol) + else: + host, port = proxy_str.split(":") + protocol = ProxyProtocol.HTTP + + proxies.append(ProxyInfo( + host=host, + port=int(port), + protocol=protocol + )) + except Exception as e: + logger.warning(f"解析代理失败: {proxy_str} - {e}") + + if not proxies: + print("没有有效的代理可检测") + return [] + + print(f"开始检测 {len(proxies)} 个代理...") + + checker = ProxyChecker(timeout=10) + valid_proxies = await checker.check_batch(proxies, concurrency=20) + + print(f"\n检测结果: {len(valid_proxies)}/{len(proxies)} 可用") + + for proxy in valid_proxies: + print(f" ✓ {proxy.host}:{proxy.port} (响应: {proxy.avg_response_time:.2f}s)") + + return valid_proxies + + +async def check_from_file(filepath: str) -> List[ProxyInfo]: + """ + 从文件读取代理并检测 + + Args: + filepath: 代理列表文件路径（每行一个代理） + + Returns: + 可用的代理列表 + """ + try: + with open(filepath, "r") as f: + proxy_list = [line.strip() for line in f if line.strip()] + except FileNotFoundError: + print(f"文件不存在: {filepath}") + return [] + + return await check_proxy_list(proxy_list) + + +def main(): + """主函数""" + import argparse + + parser = argparse.ArgumentParser(description="代理检测工具") + parser.add_argument( + "proxy", + nargs="?", + help="要检测的代理 (host:port)" + ) + parser.add_argument( + "-f", "--file", + help="从文件读取代理列表" + ) + parser.add_argument( + "-l", "--list", + nargs="+", + help="检测多个代理" + ) + + args = parser.parse_args() + + # 配置日志 + logger.remove() + logger.add( + lambda msg: print(msg, end=""), + format="{time:HH:mm:ss} | {message}", + level="INFO" + ) + + if args.file: + # 从文件读取 + asyncio.run(check_from_file(args.file)) + elif args.list: + # 检测多个代理 + asyncio.run(check_proxy_list(args.list)) + elif args.proxy: + # 检测单个代理 + asyncio.run(check_single_proxy(args.proxy)) + else: + # 演示模式 + print("代理检测工具使用示例:") + print(" 检测单个代理: python proxy_checker.py 127.0.0.1:8080") + print(" 检测多个代理: python proxy_checker.py -l 1.1.1.1:8080 2.2.2.2:8080") + print(" 从文件读取: python proxy_checker.py -f proxies.txt") + + +if __name__ == "__main__": + main() diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/03_\344\273\243\347\220\206IP\347\232\204\344\275\277\347\224\250\344\270\216\347\256\241\347\220\206/proxy_demo.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/03_\344\273\243\347\220\206IP\347\232\204\344\275\277\347\224\250\344\270\216\347\256\241\347\220\206/proxy_demo.py" new file mode 100644 index 0000000..ac43725 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/03_\344\273\243\347\220\206IP\347\232\204\344\275\277\347\224\250\344\270\216\347\256\241\347\220\206/proxy_demo.py" @@ -0,0 +1,274 @@ +# -*- coding: utf-8 -*- +# @Desc: 代理池使用演示 +# 展示完整的代理池使用流程 + +import asyncio +from typing import List, Optional +from loguru import logger +import httpx + +from proxy_pool.base import ProxyInfo, ProxyProtocol, IProxyFetcher +from proxy_pool.checker import ProxyChecker +from proxy_pool.pool import ProxyPool + + +class MockProxyFetcher(IProxyFetcher): + """ + 模拟代理获取器 + + 用于演示，实际使用时替换为真实的代理获取器 + """ + + # 模拟的代理列表（这些是公开的测试代理，可能不可用） + MOCK_PROXIES = [ + # 这些只是示例，实际运行时可能不可用 + ("103.152.112.157", 80), + ("203.142.78.109", 8080), + ("103.83.232.225", 80), + ("190.61.88.147", 8080), + ("45.167.126.108", 3128), + ] + + async def fetch(self) -> List[ProxyInfo]: + """获取模拟代理""" + proxies = [] + for host, port in self.MOCK_PROXIES: + proxies.append(ProxyInfo( + host=host, + port=port, + protocol=ProxyProtocol.HTTP + )) + logger.info(f"获取到 {len(proxies)} 个模拟代理") + return proxies + + +class ProxiedCrawler: + """ + 使用代理池的爬虫示例 + """ + + def __init__(self, proxy_pool: ProxyPool): + self.proxy_pool = proxy_pool + self.success_count = 0 + self.fail_count = 0 + + async def fetch(self, url: str) -> Optional[str]: + """ + 使用代理获取页面 + + Args: + url: 目标 URL + + Returns: + 页面内容或 None + """ + # 获取代理 + proxy = await self.proxy_pool.get_proxy() + + if not proxy: + logger.warning("无可用代理，直接请求") + proxy_url = None + else: + proxy_url = proxy.url + logger.info(f"使用代理: {proxy.host}:{proxy.port}") + + try: + async with httpx.AsyncClient( + proxies=proxy_url, + timeout=15, + verify=False + ) as client: + response = await client.get(url) + response.raise_for_status() + + # 报告成功 + if proxy: + await self.proxy_pool.return_proxy(proxy, success=True) + self.success_count += 1 + + return response.text + + except Exception as e: + logger.warning(f"请求失败: {e}") + # 报告失败 + if proxy: + await self.proxy_pool.return_proxy(proxy, success=False) + self.fail_count += 1 + return None + + +async def demo_proxy_pool(): + """演示代理池基本功能""" + print("=" * 60) + print("代理池基本功能演示") + print("=" * 60) + + # 创建组件 + fetcher = MockProxyFetcher() + checker = ProxyChecker(timeout=10) + + # 创建代理池 + pool = ProxyPool( + fetcher=fetcher, + checker=checker, + min_proxies=2, + max_proxies=10, + check_interval=60 + ) + + async with pool: + print(f"\n代理池大小: {pool.size}") + print(f"统计信息: {pool.get_stats()}") + + # 获取代理 + print("\n获取代理测试:") + for i in range(3): + proxy = await pool.get_proxy() + if proxy: + print(f" {i+1}. {proxy.host}:{proxy.port} (评分: {proxy.score:.2f})") + # 模拟使用结果 + await pool.return_proxy(proxy, success=(i % 2 == 0)) + else: + print(f" {i+1}. 无可用代理") + + print(f"\n最终统计: {pool.get_stats()}") + + +async def demo_proxied_crawler(): + """演示集成代理池的爬虫""" + print("\n" + "=" * 60) + print("代理爬虫演示") + print("=" * 60) + + # 创建代理池 + pool = ProxyPool( + fetcher=MockProxyFetcher(), + checker=ProxyChecker(timeout=10), + min_proxies=2, + max_proxies=10 + ) + + async with pool: + # 创建爬虫 + crawler = ProxiedCrawler(pool) + + # 测试请求 - 使用多个测试URL + urls = [ + "https://httpbin.org/ip", + "https://httpbin.org/headers", + ] + + print("\n开始爬取测试:") + for url in urls: + content = await crawler.fetch(url) + if content: + print(f" ✓ {url[:40]}... ({len(content)} bytes)") + else: + print(f" ✗ {url[:40]}... 失败") + + print(f"\n爬取统计: 成功 {crawler.success_count}, 失败 {crawler.fail_count}") + + +async def demo_multi_url_test(): + """演示多URL代理测试 - 综合验证代理功能""" + print("\n" + "=" * 60) + print("多URL代理测试演示") + print("=" * 60) + + # 定义测试URL列表 + test_urls = [ + { + "url": "https://httpbin.org/ip", + "name": "httpbin IP检测", + "extract": lambda data: data.get("origin", "N/A") + }, + { + "url": "https://api.ipify.org?format=json", + "name": "ipify IP服务", + "extract": lambda data: data.get("ip", "N/A") + }, + { + "url": "http://ip-api.com/json/", + "name": "ip-api 地理位置", + "extract": lambda data: f"{data.get('query', 'N/A')} ({data.get('country', 'N/A')}, {data.get('city', 'N/A')})" + }, + ] + + print("\n测试不使用代理的情况:") + print("-" * 60) + + async with httpx.AsyncClient(timeout=10) as client: + for test in test_urls: + try: + response = await client.get(test["url"]) + if response.status_code == 200: + try: + data = response.json() + result = test["extract"](data) + print(f" ✓ {test['name']}: {result}") + except Exception: + print(f" ✓ {test['name']}: {response.text[:50]}...") + else: + print(f" ✗ {test['name']}: HTTP {response.status_code}") + except Exception as e: + print(f" ✗ {test['name']}: {str(e)[:50]}") + + print("\n说明:") + print(" - httpbin.org: 通用HTTP测试服务，返回请求IP") + print(" - ipify.org: 专门的IP获取服务，简单直接") + print(" - ip-api.com: 提供IP地理位置信息") + print("\n如果使用代理，这些服务返回的IP应该是代理服务器的IP") + + +async def demo_manual_proxy(): + """演示手动使用代理""" + print("\n" + "=" * 60) + print("手动代理使用演示") + print("=" * 60) + + # 直接使用 httpx 设置代理 + proxy_url = "http://127.0.0.1:7890" # 替换为你的代理地址 + + print(f"\n使用代理: {proxy_url}") + print("(如果代理不可用，请求会失败)") + + try: + async with httpx.AsyncClient( + proxies=proxy_url, + timeout=10 + ) as client: + response = await client.get("https://httpbin.org/ip") + print(f"响应: {response.text}") + except Exception as e: + print(f"请求失败 (代理可能不可用): {e}") + + +async def main(): + """主函数""" + # 配置日志 + logger.remove() + logger.add( + lambda msg: print(msg, end=""), + format="{time:HH:mm:ss} | {level: <8} | {message}", + level="DEBUG" + ) + + print("=" * 60) + print("代理 IP 使用与管理演示") + print("=" * 60) + print("\n注意: 演示使用的是模拟代理，可能不可用") + print("实际使用时请替换为真实的代理服务") + + # 运行演示 + await demo_proxy_pool() + await demo_proxied_crawler() + await demo_multi_url_test() # 新增：多URL测试 + # await demo_manual_proxy() # 需要有可用代理才能测试 + + print("\n" + "=" * 60) + print("演示完成") + print("=" * 60) + + +if __name__ == "__main__": + asyncio.run(main()) diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/03_\344\273\243\347\220\206IP\347\232\204\344\275\277\347\224\250\344\270\216\347\256\241\347\220\206/proxy_pool/__init__.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/03_\344\273\243\347\220\206IP\347\232\204\344\275\277\347\224\250\344\270\216\347\256\241\347\220\206/proxy_pool/__init__.py" new file mode 100644 index 0000000..dfb4c81 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/03_\344\273\243\347\220\206IP\347\232\204\344\275\277\347\224\250\344\270\216\347\256\241\347\220\206/proxy_pool/__init__.py" @@ -0,0 +1,16 @@ +# -*- coding: utf-8 -*- +# @Desc: 代理池模块 + +from .base import ProxyInfo, ProxyProtocol, IProxyFetcher, IProxyChecker, IProxyPool +from .checker import ProxyChecker +from .pool import ProxyPool + +__all__ = [ + "ProxyInfo", + "ProxyProtocol", + "IProxyFetcher", + "IProxyChecker", + "IProxyPool", + "ProxyChecker", + "ProxyPool", +] diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/03_\344\273\243\347\220\206IP\347\232\204\344\275\277\347\224\250\344\270\216\347\256\241\347\220\206/proxy_pool/base.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/03_\344\273\243\347\220\206IP\347\232\204\344\275\277\347\224\250\344\270\216\347\256\241\347\220\206/proxy_pool/base.py" new file mode 100644 index 0000000..6923c12 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/03_\344\273\243\347\220\206IP\347\232\204\344\275\277\347\224\250\344\270\216\347\256\241\347\220\206/proxy_pool/base.py" @@ -0,0 +1,190 @@ +# -*- coding: utf-8 -*- +# @Desc: 代理池基础类和接口定义 + +from abc import ABC, abstractmethod +from dataclasses import dataclass, field +from enum import Enum +from typing import Optional, List +import time + + +class ProxyProtocol(Enum): + """代理协议枚举""" + HTTP = "http" + HTTPS = "https" + SOCKS5 = "socks5" + + +@dataclass +class ProxyInfo: + """ + 代理信息数据类 + + 包含代理的基本信息和质量指标 + """ + host: str + port: int + protocol: ProxyProtocol = ProxyProtocol.HTTP + username: Optional[str] = None + password: Optional[str] = None + + # 质量指标 + success_count: int = 0 + fail_count: int = 0 + avg_response_time: float = 0.0 + last_check_time: float = field(default_factory=time.time) + last_use_time: float = 0.0 + + @property + def url(self) -> str: + """ + 构建代理 URL + + Returns: + 代理 URL 字符串，如 http://user:pass@host:port + """ + auth = "" + if self.username and self.password: + auth = f"{self.username}:{self.password}@" + return f"{self.protocol.value}://{auth}{self.host}:{self.port}" + + @property + def score(self) -> float: + """ + 计算代理评分 + + 综合考虑成功率和响应时间 + + Returns: + 0-1 之间的评分 + """ + total = self.success_count + self.fail_count + + if total == 0: + return 0.5 # 未测试的代理给中等分数 + + # 成功率权重 70% + success_rate = self.success_count / total + + # 响应时间权重 30%，响应时间越短分数越高 + # 假设 10 秒以上响应时间为 0 分 + time_score = max(0, 1 - self.avg_response_time / 10) + + return success_rate * 0.7 + time_score * 0.3 + + @property + def is_stale(self) -> bool: + """ + 检查代理是否过期（超过 10 分钟未检测） + """ + return time.time() - self.last_check_time > 600 + + def __hash__(self): + return hash((self.host, self.port)) + + def __eq__(self, other): + if not isinstance(other, ProxyInfo): + return False + return self.host == other.host and self.port == other.port + + def __str__(self): + return f"Proxy({self.host}:{self.port}, score={self.score:.2f})" + + +class IProxyFetcher(ABC): + """ + 代理获取器接口 + + 负责从各种来源获取代理列表 + """ + + @abstractmethod + async def fetch(self) -> List[ProxyInfo]: + """ + 获取代理列表 + + Returns: + 代理信息列表 + """ + pass + + +class IProxyChecker(ABC): + """ + 代理检测器接口 + + 负责检测代理的可用性 + """ + + @abstractmethod + async def check(self, proxy: ProxyInfo) -> bool: + """ + 检测单个代理是否可用 + + Args: + proxy: 代理信息 + + Returns: + 代理是否可用 + """ + pass + + +class IProxyPool(ABC): + """ + 代理池接口 + + 负责代理的存储和分配 + """ + + @abstractmethod + async def get_proxy(self) -> Optional[ProxyInfo]: + """ + 获取一个可用代理 + + Returns: + 代理信息，如果没有可用代理则返回 None + """ + pass + + @abstractmethod + async def return_proxy(self, proxy: ProxyInfo, success: bool): + """ + 归还代理并报告使用结果 + + Args: + proxy: 代理信息 + success: 使用是否成功 + """ + pass + + @abstractmethod + async def add_proxy(self, proxy: ProxyInfo): + """ + 添加代理到池中 + + Args: + proxy: 代理信息 + """ + pass + + @abstractmethod + async def remove_proxy(self, proxy: ProxyInfo): + """ + 从池中移除代理 + + Args: + proxy: 代理信息 + """ + pass + + @property + @abstractmethod + def size(self) -> int: + """ + 获取代理池大小 + + Returns: + 代理数量 + """ + pass diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/03_\344\273\243\347\220\206IP\347\232\204\344\275\277\347\224\250\344\270\216\347\256\241\347\220\206/proxy_pool/checker.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/03_\344\273\243\347\220\206IP\347\232\204\344\275\277\347\224\250\344\270\216\347\256\241\347\220\206/proxy_pool/checker.py" new file mode 100644 index 0000000..a5511a5 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/03_\344\273\243\347\220\206IP\347\232\204\344\275\277\347\224\250\344\270\216\347\256\241\347\220\206/proxy_pool/checker.py" @@ -0,0 +1,174 @@ +# -*- coding: utf-8 -*- +# @Desc: 代理检测器实现 + +import asyncio +import time +from typing import List +from loguru import logger +import httpx + +from .base import ProxyInfo, IProxyChecker + + +class ProxyChecker(IProxyChecker): + """ + 代理检测器 + + 功能： + - 检测代理的可用性 + - 测量响应时间 + - 支持批量检测 + """ + + # 用于检测的 URL 列表（按优先级排序） + CHECK_URLS = [ + "https://httpbin.org/ip", + "https://api.ipify.org?format=json", + "https://ifconfig.me/ip", + ] + + def __init__( + self, + timeout: int = 10, + check_urls: List[str] = None + ): + """ + 初始化检测器 + + Args: + timeout: 检测超时时间（秒） + check_urls: 自定义检测 URL 列表 + """ + self.timeout = timeout + self.check_urls = check_urls or self.CHECK_URLS + + async def check(self, proxy: ProxyInfo) -> bool: + """ + 检测单个代理是否可用 + + Args: + proxy: 代理信息 + + Returns: + 代理是否可用 + """ + start_time = time.time() + + try: + async with httpx.AsyncClient( + proxies=proxy.url, + timeout=self.timeout, + verify=False # 跳过 SSL 验证（某些代理可能有问题） + ) as client: + for url in self.check_urls: + try: + response = await client.get(url) + + if response.status_code == 200: + # 计算响应时间 + response_time = time.time() - start_time + + # 更新代理信息 + # 使用指数移动平均更新响应时间 + if proxy.avg_response_time > 0: + proxy.avg_response_time = ( + proxy.avg_response_time * 0.7 + + response_time * 0.3 + ) + else: + proxy.avg_response_time = response_time + + proxy.last_check_time = time.time() + + logger.debug( + f"代理可用: {proxy.host}:{proxy.port}, " + f"响应时间: {response_time:.2f}s" + ) + return True + + except httpx.TimeoutException: + continue + except httpx.ConnectError: + continue + except Exception as e: + logger.debug(f"检测请求异常: {url} - {e}") + continue + + except Exception as e: + logger.debug(f"代理检测失败: {proxy.host}:{proxy.port} - {e}") + + return False + + async def check_batch( + self, + proxies: List[ProxyInfo], + concurrency: int = 20 + ) -> List[ProxyInfo]: + """ + 批量检测代理 + + Args: + proxies: 代理列表 + concurrency: 并发数 + + Returns: + 可用的代理列表 + """ + semaphore = asyncio.Semaphore(concurrency) + valid_proxies = [] + checked_count = 0 + total_count = len(proxies) + + async def check_one(proxy: ProxyInfo): + nonlocal checked_count + async with semaphore: + is_valid = await self.check(proxy) + checked_count += 1 + + if is_valid: + valid_proxies.append(proxy) + + # 进度日志 + if checked_count % 10 == 0 or checked_count == total_count: + logger.info( + f"检测进度: {checked_count}/{total_count}, " + f"有效: {len(valid_proxies)}" + ) + + # 创建任务 + tasks = [check_one(p) for p in proxies] + + # 并发执行 + await asyncio.gather(*tasks, return_exceptions=True) + + logger.info( + f"检测完成: {len(valid_proxies)}/{len(proxies)} 可用 " + f"({len(valid_proxies)/len(proxies)*100:.1f}%)" + ) + + return valid_proxies + + +class TargetSiteChecker(ProxyChecker): + """ + 目标站点检测器 + + 使用实际目标站点进行检测，确保代理对目标网站可用 + """ + + def __init__( + self, + target_url: str, + expected_status: int = 200, + timeout: int = 15 + ): + """ + 初始化目标站点检测器 + + Args: + target_url: 目标站点 URL + expected_status: 期望的状态码 + timeout: 超时时间 + """ + super().__init__(timeout=timeout, check_urls=[target_url]) + self.expected_status = expected_status diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/03_\344\273\243\347\220\206IP\347\232\204\344\275\277\347\224\250\344\270\216\347\256\241\347\220\206/proxy_pool/pool.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/03_\344\273\243\347\220\206IP\347\232\204\344\275\277\347\224\250\344\270\216\347\256\241\347\220\206/proxy_pool/pool.py" new file mode 100644 index 0000000..f85f3be --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/03_\344\273\243\347\220\206IP\347\232\204\344\275\277\347\224\250\344\270\216\347\256\241\347\220\206/proxy_pool/pool.py" @@ -0,0 +1,283 @@ +# -*- coding: utf-8 -*- +# @Desc: 代理池实现 + +import asyncio +import random +import time +from typing import Optional, Dict, List +from loguru import logger + +from .base import ProxyInfo, IProxyFetcher, IProxyChecker, IProxyPool + + +class ProxyPool(IProxyPool): + """ + 代理池实现 + + 特性： + - 自动获取和检测代理 + - 基于评分的智能分配 + - 自动淘汰失效代理 + - 后台自动刷新 + """ + + def __init__( + self, + fetcher: IProxyFetcher, + checker: IProxyChecker, + min_proxies: int = 10, + max_proxies: int = 100, + check_interval: int = 300, + max_fail_count: int = 3, + score_threshold: float = 0.3 + ): + """ + 初始化代理池 + + Args: + fetcher: 代理获取器 + checker: 代理检测器 + min_proxies: 最小代理数量（低于此数量时自动补充） + max_proxies: 最大代理数量 + check_interval: 检测间隔（秒） + max_fail_count: 最大连续失败次数（超过后触发淘汰检查） + score_threshold: 淘汰评分阈值 + """ + self.fetcher = fetcher + self.checker = checker + self.min_proxies = min_proxies + self.max_proxies = max_proxies + self.check_interval = check_interval + self.max_fail_count = max_fail_count + self.score_threshold = score_threshold + + # 代理存储 {key: ProxyInfo} + self._proxies: Dict[str, ProxyInfo] = {} + self._lock = asyncio.Lock() + + # 后台任务 + self._refresh_task: Optional[asyncio.Task] = None + self._running = False + + def _proxy_key(self, proxy: ProxyInfo) -> str: + """生成代理唯一标识""" + return f"{proxy.host}:{proxy.port}" + + async def start(self): + """启动代理池""" + if self._running: + return + + self._running = True + logger.info("代理池启动中...") + + # 初始获取代理 + await self._refresh_proxies() + + # 启动后台刷新任务 + self._refresh_task = asyncio.create_task(self._refresh_loop()) + + logger.info(f"代理池已启动，当前代理数: {self.size}") + + async def stop(self): + """停止代理池""" + if not self._running: + return + + self._running = False + + if self._refresh_task: + self._refresh_task.cancel() + try: + await self._refresh_task + except asyncio.CancelledError: + pass + + logger.info("代理池已停止") + + async def __aenter__(self): + await self.start() + return self + + async def __aexit__(self, *args): + await self.stop() + + async def _refresh_loop(self): + """后台刷新循环""" + while self._running: + try: + await asyncio.sleep(self.check_interval) + + # 检查是否需要刷新 + if self.size < self.min_proxies: + logger.info("代理数量不足，触发刷新") + await self._refresh_proxies() + else: + # 清理过期代理 + await self._cleanup_stale_proxies() + + except asyncio.CancelledError: + break + except Exception as e: + logger.error(f"代理刷新异常: {e}") + await asyncio.sleep(60) # 出错后等待一分钟 + + async def _refresh_proxies(self): + """刷新代理""" + logger.info(f"开始刷新代理，当前数量: {self.size}") + + try: + # 获取新代理 + new_proxies = await self.fetcher.fetch() + + if not new_proxies: + logger.warning("未获取到新代理") + return + + logger.info(f"获取到 {len(new_proxies)} 个代理，开始检测...") + + # 检测代理 + valid_proxies = await self.checker.check_batch(new_proxies) + + # 添加到池中 + async with self._lock: + added_count = 0 + for proxy in valid_proxies: + key = self._proxy_key(proxy) + if key not in self._proxies and len(self._proxies) < self.max_proxies: + self._proxies[key] = proxy + added_count += 1 + + logger.info(f"添加了 {added_count} 个新代理，当前总数: {self.size}") + + except Exception as e: + logger.error(f"刷新代理失败: {e}") + + async def _cleanup_stale_proxies(self): + """清理过期代理""" + async with self._lock: + stale_keys = [ + key for key, proxy in self._proxies.items() + if proxy.is_stale + ] + + for key in stale_keys: + del self._proxies[key] + + if stale_keys: + logger.info(f"清理了 {len(stale_keys)} 个过期代理") + + async def get_proxy(self) -> Optional[ProxyInfo]: + """ + 获取一个可用代理 + + 使用加权随机选择，评分高的代理被选中概率更大 + """ + async with self._lock: + if not self._proxies: + logger.warning("代理池为空") + return None + + # 获取所有代理 + proxies = list(self._proxies.values()) + + # 计算权重（评分越高权重越大，最小权重 0.1） + weights = [max(p.score, 0.1) for p in proxies] + + # 加权随机选择 + selected = random.choices(proxies, weights=weights, k=1)[0] + selected.last_use_time = time.time() + + logger.debug( + f"分配代理: {selected.host}:{selected.port} " + f"(评分: {selected.score:.2f})" + ) + + return selected + + async def return_proxy(self, proxy: ProxyInfo, success: bool): + """ + 归还代理并报告使用结果 + + Args: + proxy: 代理信息 + success: 使用是否成功 + """ + async with self._lock: + key = self._proxy_key(proxy) + + if key not in self._proxies: + return + + stored_proxy = self._proxies[key] + + if success: + stored_proxy.success_count += 1 + logger.debug(f"代理使用成功: {proxy.host}:{proxy.port}") + else: + stored_proxy.fail_count += 1 + logger.debug(f"代理使用失败: {proxy.host}:{proxy.port}") + + # 检查是否需要淘汰 + if stored_proxy.fail_count >= self.max_fail_count: + total = stored_proxy.success_count + stored_proxy.fail_count + + # 有足够样本且评分过低时淘汰 + if total >= 5 and stored_proxy.score < self.score_threshold: + del self._proxies[key] + logger.info( + f"淘汰低质量代理: {proxy.host}:{proxy.port} " + f"(评分: {stored_proxy.score:.2f})" + ) + + async def add_proxy(self, proxy: ProxyInfo): + """添加代理""" + async with self._lock: + key = self._proxy_key(proxy) + if key not in self._proxies and len(self._proxies) < self.max_proxies: + self._proxies[key] = proxy + logger.debug(f"添加代理: {proxy.host}:{proxy.port}") + + async def remove_proxy(self, proxy: ProxyInfo): + """移除代理""" + async with self._lock: + key = self._proxy_key(proxy) + if key in self._proxies: + del self._proxies[key] + logger.debug(f"移除代理: {proxy.host}:{proxy.port}") + + @property + def size(self) -> int: + """代理池大小""" + return len(self._proxies) + + def get_stats(self) -> Dict: + """ + 获取统计信息 + + Returns: + 包含各种统计指标的字典 + """ + if not self._proxies: + return { + "total": 0, + "avg_score": 0, + "max_score": 0, + "min_score": 0, + } + + proxies = list(self._proxies.values()) + scores = [p.score for p in proxies] + + return { + "total": len(proxies), + "avg_score": sum(scores) / len(scores), + "max_score": max(scores), + "min_score": min(scores), + "total_success": sum(p.success_count for p in proxies), + "total_fail": sum(p.fail_count for p in proxies), + } + + def get_all_proxies(self) -> List[ProxyInfo]: + """获取所有代理列表（用于调试）""" + return list(self._proxies.values()) diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/03_\344\273\243\347\220\206IP\347\232\204\344\275\277\347\224\250\344\270\216\347\256\241\347\220\206/pyproject.toml" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/03_\344\273\243\347\220\206IP\347\232\204\344\275\277\347\224\250\344\270\216\347\256\241\347\220\206/pyproject.toml" new file mode 100644 index 0000000..e0df9a4 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/03_\344\273\243\347\220\206IP\347\232\204\344\275\277\347\224\250\344\270\216\347\256\241\347\220\206/pyproject.toml" @@ -0,0 +1,17 @@ +[project] +name = "chapter03-proxy-management" +version = "0.1.0" +description = "第03章：代理IP的使用与管理 - 代理池设计、代理检测、多URL测试" +readme = "README.md" +requires-python = ">=3.11" +dependencies = [ + "httpx>=0.27.0", + "loguru>=0.7.0", +] + +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[tool.uv] +dev-dependencies = [] diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/04_Playwright\346\265\217\350\247\210\345\231\250\350\207\252\345\212\250\345\214\226\345\205\245\351\227\250/README.md" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/04_Playwright\346\265\217\350\247\210\345\231\250\350\207\252\345\212\250\345\214\226\345\205\245\351\227\250/README.md" new file mode 100644 index 0000000..f17dbfc --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/04_Playwright\346\265\217\350\247\210\345\231\250\350\207\252\345\212\250\345\214\226\345\205\245\351\227\250/README.md" @@ -0,0 +1,18 @@ +# 第04章：Playwright浏览器自动化入门 + +展示Playwright基本操作、元素定位、等待策略等。 + +## 快速开始 + +```bash +cd 04_Playwright浏览器自动化入门 +uv sync +uv run playwright install chromium # 安装浏览器 +uv run python basic_operations.py +uv run python wait_strategies.py +uv run python spa_crawler.py +``` + +### 目标网站 +- **quotes.toscrape.com** - 静态版本 +- **quotes.toscrape.com/js/** - SPA版本（需要JavaScript渲染） diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/04_Playwright\346\265\217\350\247\210\345\231\250\350\207\252\345\212\250\345\214\226\345\205\245\351\227\250/basic_operations.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/04_Playwright\346\265\217\350\247\210\345\231\250\350\207\252\345\212\250\345\214\226\345\205\245\351\227\250/basic_operations.py" new file mode 100644 index 0000000..37ca39b --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/04_Playwright\346\265\217\350\247\210\345\231\250\350\207\252\345\212\250\345\214\226\345\205\245\351\227\250/basic_operations.py" @@ -0,0 +1,252 @@ +# -*- coding: utf-8 -*- +# @Desc: Playwright 基础操作演示 + +import asyncio +from playwright.async_api import async_playwright +from loguru import logger + + +async def demo_navigation(): + """演示页面导航""" + print("\n" + "=" * 50) + print("1. 页面导航演示") + print("=" * 50) + + async with async_playwright() as p: + browser = await p.chromium.launch(headless=True) + page = await browser.new_page() + + try: + # 基本导航 + logger.info("访问 example.com...") + await page.goto("https://example.com") + print(f"页面标题: {await page.title()}") + print(f"当前 URL: {page.url}") + + # 使用不同的等待策略 + logger.info("使用 networkidle 等待策略...") + await page.goto( + "https://quotes.toscrape.com/", + wait_until="networkidle" + ) + print(f"页面标题: {await page.title()}") + + finally: + await browser.close() + + +async def demo_locators(): + """演示元素定位""" + print("\n" + "=" * 50) + print("2. 元素定位演示") + print("=" * 50) + + async with async_playwright() as p: + browser = await p.chromium.launch(headless=True) + page = await browser.new_page() + + try: + await page.goto("https://quotes.toscrape.com/") + + # 使用 CSS 选择器 + title = await page.locator("h1 a").text_content() + print(f"CSS 选择器 - 标题: {title}") + + # 使用 text 定位 + about_link = page.get_by_text("About") + print(f"Text 定位 - About 链接存在: {await about_link.count() > 0}") + + # 使用 role 定位 + links = page.get_by_role("link") + print(f"Role 定位 - 链接数量: {await links.count()}") + + # 组合定位 + first_quote = page.locator("div.quote").first + author = await first_quote.locator("small.author").text_content() + print(f"组合定位 - 第一条名言作者: {author}") + + # 获取所有元素 + quotes = await page.locator("div.quote").all() + print(f"共找到 {len(quotes)} 条名言") + + finally: + await browser.close() + + +async def demo_interactions(): + """演示交互操作""" + print("\n" + "=" * 50) + print("3. 交互操作演示") + print("=" * 50) + + async with async_playwright() as p: + browser = await p.chromium.launch(headless=True) + page = await browser.new_page() + + try: + # 访问登录页面 + await page.goto("https://quotes.toscrape.com/login") + logger.info("访问登录页面") + + # 输入用户名 + await page.fill("input#username", "testuser") + print("输入用户名: testuser") + + # 输入密码 + await page.fill("input#password", "testpass") + print("输入密码: ******") + + # 点击登录按钮 + await page.click("input[type='submit']") + print("点击登录按钮") + + # 等待页面跳转 + await page.wait_for_load_state("networkidle") + print(f"登录后 URL: {page.url}") + + finally: + await browser.close() + + +async def demo_content_extraction(): + """演示内容提取""" + print("\n" + "=" * 50) + print("4. 内容提取演示") + print("=" * 50) + + async with async_playwright() as p: + browser = await p.chromium.launch(headless=True) + page = await browser.new_page() + + try: + await page.goto("https://quotes.toscrape.com/") + + # 提取文本内容 + quotes_data = [] + quotes = await page.locator("div.quote").all() + + for quote in quotes[:3]: # 只取前 3 条 + text = await quote.locator("span.text").text_content() + author = await quote.locator("small.author").text_content() + tags = await quote.locator("a.tag").all_text_contents() + + quotes_data.append({ + "text": text[:60] + "..." if len(text) > 60 else text, + "author": author, + "tags": tags + }) + + print("\n提取的名言数据:") + for i, item in enumerate(quotes_data, 1): + print(f"\n{i}. {item['author']}") + print(f" {item['text']}") + print(f" 标签: {', '.join(item['tags'])}") + + # 获取属性 + first_link = page.locator("div.quote a.tag").first + href = await first_link.get_attribute("href") + print(f"\n第一个标签链接: {href}") + + # 执行 JavaScript + title = await page.evaluate("document.title") + print(f"通过 JS 获取标题: {title}") + + finally: + await browser.close() + + +async def demo_waiting(): + """演示等待策略""" + print("\n" + "=" * 50) + print("5. 等待策略演示") + print("=" * 50) + + async with async_playwright() as p: + browser = await p.chromium.launch(headless=True) + page = await browser.new_page() + + try: + # 设置默认超时 + page.set_default_timeout(30000) + logger.info("设置默认超时: 30秒") + + # 访问页面 + await page.goto("https://quotes.toscrape.com/") + + # 等待选择器 + await page.wait_for_selector("div.quote") + print("等待 div.quote 出现 - 成功") + + # 等待页面状态 + await page.wait_for_load_state("networkidle") + print("等待 networkidle - 成功") + + # 单次操作超时 + try: + await page.wait_for_selector("div.not-exist", timeout=2000) + except Exception as e: + print(f"等待不存在的元素 - 超时 (预期行为)") + + finally: + await browser.close() + + +async def demo_screenshot(): + """演示截图功能""" + print("\n" + "=" * 50) + print("6. 截图功能演示") + print("=" * 50) + + async with async_playwright() as p: + browser = await p.chromium.launch(headless=True) + page = await browser.new_page() + + try: + await page.goto("https://quotes.toscrape.com/") + + # 页面截图 + await page.screenshot(path="screenshot_viewport.png") + print("视口截图已保存: screenshot_viewport.png") + + # 全页面截图 + await page.screenshot(path="screenshot_fullpage.png", full_page=True) + print("全页面截图已保存: screenshot_fullpage.png") + + # 元素截图 + first_quote = page.locator("div.quote").first + await first_quote.screenshot(path="screenshot_element.png") + print("元素截图已保存: screenshot_element.png") + + finally: + await browser.close() + + +async def main(): + """主函数""" + # 配置日志 + logger.remove() + logger.add( + lambda msg: print(msg, end=""), + format="{time:HH:mm:ss} | {message}", + level="INFO" + ) + + print("=" * 50) + print("Playwright 基础操作演示") + print("=" * 50) + + # 运行所有演示 + await demo_navigation() + await demo_locators() + await demo_interactions() + await demo_content_extraction() + await demo_waiting() + await demo_screenshot() + + print("\n" + "=" * 50) + print("演示完成") + print("=" * 50) + + +if __name__ == "__main__": + asyncio.run(main()) diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/04_Playwright\346\265\217\350\247\210\345\231\250\350\207\252\345\212\250\345\214\226\345\205\245\351\227\250/pyproject.toml" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/04_Playwright\346\265\217\350\247\210\345\231\250\350\207\252\345\212\250\345\214\226\345\205\245\351\227\250/pyproject.toml" new file mode 100644 index 0000000..fb491a0 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/04_Playwright\346\265\217\350\247\210\345\231\250\350\207\252\345\212\250\345\214\226\345\205\245\351\227\250/pyproject.toml" @@ -0,0 +1,19 @@ +[project] +name = "chapter04-playwright-basics" +version = "0.1.0" +description = "第04章：Playwright浏览器自动化入门 - 页面操作、元素定位、等待策略" +readme = "README.md" +requires-python = ">=3.11" +dependencies = [ + "playwright>=1.45.0", + "loguru>=0.7.0", +] + +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[tool.uv] +dev-dependencies = [] + +# 提示：安装后需要运行 playwright install 安装浏览器驱动 diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/04_Playwright\346\265\217\350\247\210\345\231\250\350\207\252\345\212\250\345\214\226\345\205\245\351\227\250/spa_crawler.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/04_Playwright\346\265\217\350\247\210\345\231\250\350\207\252\345\212\250\345\214\226\345\205\245\351\227\250/spa_crawler.py" new file mode 100644 index 0000000..27796c6 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/04_Playwright\346\265\217\350\247\210\345\231\250\350\207\252\345\212\250\345\214\226\345\205\245\351\227\250/spa_crawler.py" @@ -0,0 +1,263 @@ +# -*- coding: utf-8 -*- +# @Desc: 使用 Playwright 爬取 SPA（单页应用）完整示例 + +import asyncio +import json +from typing import List, Dict, Optional +from dataclasses import dataclass, asdict +from playwright.async_api import async_playwright, Page, Browser +from loguru import logger + + +@dataclass +class Quote: + """名言数据模型""" + text: str + author: str + tags: List[str] + + +class SPACrawler: + """ + SPA 爬虫示例 + + 爬取 https://quotes.toscrape.com/js/ - 一个需要 JavaScript 渲染的页面 + """ + + BASE_URL = "https://quotes.toscrape.com/js/" + + def __init__( + self, + headless: bool = True, + timeout: int = 30000 + ): + """ + 初始化爬虫 + + Args: + headless: 是否无头模式 + timeout: 默认超时时间（毫秒） + """ + self.headless = headless + self.timeout = timeout + self._browser: Optional[Browser] = None + self._page: Optional[Page] = None + + async def __aenter__(self): + await self.start() + return self + + async def __aexit__(self, *args): + await self.close() + + async def start(self): + """启动浏览器""" + playwright = await async_playwright().start() + self._browser = await playwright.chromium.launch( + headless=self.headless + ) + context = await self._browser.new_context( + viewport={"width": 1920, "height": 1080} + ) + self._page = await context.new_page() + self._page.set_default_timeout(self.timeout) + logger.info(f"浏览器已启动 (headless={self.headless})") + + async def close(self): + """关闭浏览器""" + if self._browser: + await self._browser.close() + logger.info("浏览器已关闭") + + async def crawl_page(self, page_num: int = 1) -> List[Quote]: + """ + 爬取单页数据 + + Args: + page_num: 页码 + + Returns: + 名言列表 + """ + url = f"{self.BASE_URL}page/{page_num}/" if page_num > 1 else self.BASE_URL + + logger.info(f"正在爬取第 {page_num} 页: {url}") + + # 访问页面 + await self._page.goto(url, wait_until="networkidle") + + # 等待内容加载（SPA 需要等待 JS 渲染） + await self._page.wait_for_selector("div.quote") + + # 提取数据 + quotes = [] + quote_elements = await self._page.locator("div.quote").all() + + for element in quote_elements: + text = await element.locator("span.text").text_content() + author = await element.locator("small.author").text_content() + tags = await element.locator("a.tag").all_text_contents() + + # 清理文本（去除引号符号） + text = text.strip().strip(""").strip(""") + + quotes.append(Quote( + text=text, + author=author.strip(), + tags=tags + )) + + logger.info(f"第 {page_num} 页爬取完成，获取 {len(quotes)} 条名言") + return quotes + + async def crawl_all_pages(self, max_pages: int = 10) -> List[Quote]: + """ + 爬取所有页面 + + Args: + max_pages: 最大页数 + + Returns: + 所有名言列表 + """ + all_quotes = [] + + for page_num in range(1, max_pages + 1): + try: + quotes = await self.crawl_page(page_num) + all_quotes.extend(quotes) + + # 检查是否还有下一页 + next_button = self._page.locator("li.next a") + if await next_button.count() == 0: + logger.info("已到达最后一页") + break + + # 页面间延迟 + await asyncio.sleep(0.5) + + except Exception as e: + logger.error(f"爬取第 {page_num} 页失败: {e}") + break + + logger.info(f"爬取完成，共获取 {len(all_quotes)} 条名言") + return all_quotes + + async def crawl_with_pagination(self) -> List[Quote]: + """ + 使用点击分页的方式爬取 + + Returns: + 所有名言列表 + """ + all_quotes = [] + page_num = 1 + + # 访问首页 + await self._page.goto(self.BASE_URL, wait_until="networkidle") + + while True: + logger.info(f"正在爬取第 {page_num} 页...") + + # 等待内容加载 + await self._page.wait_for_selector("div.quote") + + # 提取当前页数据 + quote_elements = await self._page.locator("div.quote").all() + + for element in quote_elements: + text = await element.locator("span.text").text_content() + author = await element.locator("small.author").text_content() + tags = await element.locator("a.tag").all_text_contents() + + text = text.strip().strip(""").strip(""") + + all_quotes.append(Quote( + text=text, + author=author.strip(), + tags=tags + )) + + logger.info(f"第 {page_num} 页完成，累计 {len(all_quotes)} 条") + + # 检查并点击下一页 + next_button = self._page.locator("li.next a") + if await next_button.count() == 0: + logger.info("已到达最后一页") + break + + # 点击下一页 + await next_button.click() + await self._page.wait_for_load_state("networkidle") + + page_num += 1 + + # 防止无限循环 + if page_num > 20: + break + + return all_quotes + + +async def main(): + """主函数""" + # 配置日志 + logger.remove() + logger.add( + lambda msg: print(msg, end=""), + format="{time:HH:mm:ss} | {level: <8} | {message}", + level="INFO" + ) + + print("=" * 60) + print("Playwright SPA 爬虫示例") + print("=" * 60) + print("\n目标网站: https://quotes.toscrape.com/js/") + print("这是一个需要 JavaScript 渲染的页面\n") + + async with SPACrawler(headless=True) as crawler: + # 方法 1: 直接访问各页面 URL + print("\n--- 方法 1: 直接访问各页面 URL ---") + quotes = await crawler.crawl_all_pages(max_pages=3) + + # 输出部分结果 + print("\n爬取结果示例:") + for i, quote in enumerate(quotes[:5], 1): + print(f"\n{i}. {quote.author}") + print(f" \"{quote.text[:60]}...\"") + print(f" 标签: {', '.join(quote.tags)}") + + # 保存结果 + output_file = "quotes_spa.json" + with open(output_file, "w", encoding="utf-8") as f: + json.dump( + [asdict(q) for q in quotes], + f, + ensure_ascii=False, + indent=2 + ) + print(f"\n结果已保存到: {output_file}") + + # 统计信息 + print("\n" + "=" * 60) + print("爬取统计") + print("=" * 60) + print(f"总名言数: {len(quotes)}") + + # 作者统计 + authors = {} + for quote in quotes: + authors[quote.author] = authors.get(quote.author, 0) + 1 + + print(f"作者数量: {len(authors)}") + print("\n出现最多的作者:") + for author, count in sorted(authors.items(), key=lambda x: -x[1])[:5]: + print(f" {author}: {count} 条") + + print("\n" + "=" * 60) + print("演示完成") + print("=" * 60) + + +if __name__ == "__main__": + asyncio.run(main()) diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/04_Playwright\346\265\217\350\247\210\345\231\250\350\207\252\345\212\250\345\214\226\345\205\245\351\227\250/wait_strategies.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/04_Playwright\346\265\217\350\247\210\345\231\250\350\207\252\345\212\250\345\214\226\345\205\245\351\227\250/wait_strategies.py" new file mode 100644 index 0000000..b7af8c6 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/04_Playwright\346\265\217\350\247\210\345\231\250\350\207\252\345\212\250\345\214\226\345\205\245\351\227\250/wait_strategies.py" @@ -0,0 +1,267 @@ +# -*- coding: utf-8 -*- +# @Desc: Playwright 等待策略详解 + +import asyncio +from playwright.async_api import async_playwright, TimeoutError as PlaywrightTimeout +from loguru import logger + + +async def demo_auto_waiting(): + """演示自动等待""" + print("\n" + "=" * 50) + print("1. 自动等待机制") + print("=" * 50) + + async with async_playwright() as p: + browser = await p.chromium.launch(headless=True) + page = await browser.new_page() + + try: + await page.goto("https://quotes.toscrape.com/login") + + # Playwright 的操作会自动等待元素可操作 + # 以下操作会自动等待: + # - 元素存在于 DOM + # - 元素可见 + # - 元素稳定（不在动画中） + # - 元素可接收事件 + # - 元素没有被其他元素遮挡 + + print("自动等待 - 填充用户名...") + await page.fill("input#username", "test") + print("自动等待 - 成功") + + print("自动等待 - 点击登录...") + await page.click("input[type='submit']") + print("自动等待 - 成功") + + finally: + await browser.close() + + +async def demo_wait_for_selector(): + """演示 wait_for_selector""" + print("\n" + "=" * 50) + print("2. wait_for_selector 用法") + print("=" * 50) + + async with async_playwright() as p: + browser = await p.chromium.launch(headless=True) + page = await browser.new_page() + + try: + await page.goto("https://quotes.toscrape.com/") + + # 等待元素出现（默认 state="visible"） + element = await page.wait_for_selector("div.quote") + print(f"等待 visible - 找到元素") + + # 等待元素附加到 DOM（不管是否可见） + element = await page.wait_for_selector("div.quote", state="attached") + print(f"等待 attached - 找到元素") + + # 等待元素消失 + # await page.wait_for_selector("div.loading", state="hidden") + print(f"等待 hidden - 跳过（页面无 loading 元素）") + + # 等待元素从 DOM 移除 + # await page.wait_for_selector("div.temp", state="detached") + print(f"等待 detached - 跳过") + + finally: + await browser.close() + + +async def demo_wait_for_load_state(): + """演示 wait_for_load_state""" + print("\n" + "=" * 50) + print("3. wait_for_load_state 用法") + print("=" * 50) + + async with async_playwright() as p: + browser = await p.chromium.launch(headless=True) + page = await browser.new_page() + + try: + # 导航并等待 DOM 加载 + print("导航到页面...") + await page.goto("https://quotes.toscrape.com/") + + # 等待 domcontentloaded - DOM 解析完成 + await page.wait_for_load_state("domcontentloaded") + print("domcontentloaded - 完成") + + # 等待 load - 所有资源加载完成 + await page.wait_for_load_state("load") + print("load - 完成") + + # 等待 networkidle - 网络空闲（500ms 无新请求） + await page.wait_for_load_state("networkidle") + print("networkidle - 完成") + + finally: + await browser.close() + + +async def demo_wait_for_url(): + """演示 wait_for_url""" + print("\n" + "=" * 50) + print("4. wait_for_url 用法") + print("=" * 50) + + async with async_playwright() as p: + browser = await p.chromium.launch(headless=True) + page = await browser.new_page() + + try: + await page.goto("https://quotes.toscrape.com/login") + + # 填写登录表单 + await page.fill("input#username", "test") + await page.fill("input#password", "test") + await page.click("input[type='submit']") + + # 等待 URL 变化（使用 glob 模式） + await page.wait_for_url("**/") + print(f"URL 已变化到: {page.url}") + + # 也可以使用函数 + # await page.wait_for_url(lambda url: "quotes" in url) + + finally: + await browser.close() + + +async def demo_wait_for_function(): + """演示 wait_for_function""" + print("\n" + "=" * 50) + print("5. wait_for_function 用法") + print("=" * 50) + + async with async_playwright() as p: + browser = await p.chromium.launch(headless=True) + page = await browser.new_page() + + try: + await page.goto("https://quotes.toscrape.com/") + + # 等待 JavaScript 条件为真 + await page.wait_for_function( + "document.querySelectorAll('.quote').length > 0" + ) + print("等待 JS 条件 - 名言已加载") + + # 等待特定元素数量 + await page.wait_for_function( + "document.querySelectorAll('.quote').length >= 10" + ) + print("等待 JS 条件 - 至少 10 条名言") + + # 等待全局变量 + # await page.wait_for_function("window.dataLoaded === true") + + finally: + await browser.close() + + +async def demo_expect_patterns(): + """演示 expect 模式（等待请求/响应）""" + print("\n" + "=" * 50) + print("6. expect 模式（等待请求/响应）") + print("=" * 50) + + async with async_playwright() as p: + browser = await p.chromium.launch(headless=True) + page = await browser.new_page() + + try: + # 等待响应 + async with page.expect_response("**/api/**") as response_info: + # 这个例子使用的网站没有 API，所以会超时 + # 实际使用时，这里会等待匹配的响应 + pass + # response = await response_info.value + + print("expect_response - 跳过（示例网站无 API）") + + # 正常访问 + await page.goto("https://quotes.toscrape.com/") + + # 等待导航 + async with page.expect_navigation(): + await page.click("a[href='/page/2/']") + print(f"expect_navigation - 导航到: {page.url}") + + except PlaywrightTimeout: + print("超时 - 这是预期行为（示例网站无 API）") + + finally: + await browser.close() + + +async def demo_timeout_handling(): + """演示超时处理""" + print("\n" + "=" * 50) + print("7. 超时处理") + print("=" * 50) + + async with async_playwright() as p: + browser = await p.chromium.launch(headless=True) + page = await browser.new_page() + + try: + await page.goto("https://quotes.toscrape.com/") + + # 设置默认超时 + page.set_default_timeout(5000) # 5 秒 + print("设置默认超时: 5秒") + + # 单次操作超时 + try: + await page.wait_for_selector( + "div.not-exist", + timeout=2000 # 2 秒 + ) + except PlaywrightTimeout: + print("捕获超时异常 - 元素不存在") + + # 使用 try-except 处理可能的超时 + try: + await page.click("button.maybe-exist", timeout=1000) + except PlaywrightTimeout: + print("捕获超时异常 - 按钮不存在") + except Exception as e: + print(f"捕获其他异常: {type(e).__name__}") + + finally: + await browser.close() + + +async def main(): + """主函数""" + logger.remove() + logger.add( + lambda msg: print(msg, end=""), + format="{time:HH:mm:ss} | {message}", + level="INFO" + ) + + print("=" * 50) + print("Playwright 等待策略详解") + print("=" * 50) + + await demo_auto_waiting() + await demo_wait_for_selector() + await demo_wait_for_load_state() + await demo_wait_for_url() + await demo_wait_for_function() + await demo_expect_patterns() + await demo_timeout_handling() + + print("\n" + "=" * 50) + print("演示完成") + print("=" * 50) + + +if __name__ == "__main__": + asyncio.run(main()) diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/05_Playwright\350\277\233\351\230\266_\345\217\215\346\243\200\346\265\213\344\270\216\346\200\247\350\203\275\344\274\230\345\214\226/README.md" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/05_Playwright\350\277\233\351\230\266_\345\217\215\346\243\200\346\265\213\344\270\216\346\200\247\350\203\275\344\274\230\345\214\226/README.md" new file mode 100644 index 0000000..455c346 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/05_Playwright\350\277\233\351\230\266_\345\217\215\346\243\200\346\265\213\344\270\216\346\200\247\350\203\275\344\274\230\345\214\226/README.md" @@ -0,0 +1,23 @@ +# 第05章：Playwright进阶 - 反检测与性能优化 + +展示stealth.js注入、CDP模式、浏览器指纹伪装、性能优化等高级技术。 + +## 快速开始 + +```bash +cd 05_Playwright进阶_反检测与性能优化 +uv sync +uv run playwright install chromium +uv run python stealth_demo.py +uv run python cdp_mode.py +uv run python performance_optimization.py +``` + +### 测试网站 +- **bot.sannysoft.com** - 最佳的反检测测试网站 + +### 核心技术 +- stealth.js注入 - 隐藏自动化特征 +- CDP模式 - 直接调用Chrome DevTools Protocol +- 资源拦截 - 阻止不必要的资源加载 +- 浏览器上下文池 - 复用提升性能 diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/05_Playwright\350\277\233\351\230\266_\345\217\215\346\243\200\346\265\213\344\270\216\346\200\247\350\203\275\344\274\230\345\214\226/cdp_mode.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/05_Playwright\350\277\233\351\230\266_\345\217\215\346\243\200\346\265\213\344\270\216\346\200\247\350\203\275\344\274\230\345\214\226/cdp_mode.py" new file mode 100644 index 0000000..d9e1cf6 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/05_Playwright\350\277\233\351\230\266_\345\217\215\346\243\200\346\265\213\344\270\216\346\200\247\350\203\275\344\274\230\345\214\226/cdp_mode.py" @@ -0,0 +1,246 @@ +# -*- coding: utf-8 -*- +# @Desc: Playwright CDP 模式演示 + +import asyncio +import base64 +from playwright.async_api import async_playwright +from loguru import logger + + +async def demo_cdp_session(): + """演示 CDP Session 基本用法""" + print("\n" + "=" * 50) + print("1. CDP Session 基本用法") + print("=" * 50) + + async with async_playwright() as p: + browser = await p.chromium.launch(headless=True) + context = await browser.new_context() + page = await context.new_page() + + # 创建 CDP Session + client = await context.new_cdp_session(page) + + # 启用网络事件 + await client.send("Network.enable") + + # 获取性能指标 + await page.goto("https://example.com") + metrics = await client.send("Performance.getMetrics") + + print("性能指标:") + for metric in metrics.get("metrics", [])[:5]: + print(f" {metric['name']}: {metric['value']:.2f}") + + await browser.close() + + +async def demo_network_emulation(): + """演示网络条件模拟""" + print("\n" + "=" * 50) + print("2. 网络条件模拟") + print("=" * 50) + + async with async_playwright() as p: + browser = await p.chromium.launch(headless=True) + context = await browser.new_context() + page = await context.new_page() + + client = await context.new_cdp_session(page) + + # 启用网络 + await client.send("Network.enable") + + # 模拟慢速网络（3G） + await client.send("Network.emulateNetworkConditions", { + "offline": False, + "downloadThroughput": 750 * 1024 / 8, # 750 Kbps + "uploadThroughput": 250 * 1024 / 8, # 250 Kbps + "latency": 100 # 100ms + }) + + print("已设置 3G 网络条件") + + # 测试加载时间 + import time + start = time.time() + await page.goto("https://example.com", wait_until="load") + elapsed = time.time() - start + print(f"加载时间 (3G): {elapsed:.2f}s") + + # 恢复正常网络 + await client.send("Network.emulateNetworkConditions", { + "offline": False, + "downloadThroughput": -1, # 不限制 + "uploadThroughput": -1, + "latency": 0 + }) + + start = time.time() + await page.reload(wait_until="load") + elapsed = time.time() - start + print(f"加载时间 (正常): {elapsed:.2f}s") + + await browser.close() + + +async def demo_cdp_screenshot(): + """演示 CDP 截图""" + print("\n" + "=" * 50) + print("3. CDP 截图（高级选项）") + print("=" * 50) + + async with async_playwright() as p: + browser = await p.chromium.launch(headless=True) + context = await browser.new_context() + page = await context.new_page() + + client = await context.new_cdp_session(page) + + await page.goto("https://quotes.toscrape.com/") + + # 使用 CDP 截取完整页面 + result = await client.send("Page.captureScreenshot", { + "format": "png", + "captureBeyondViewport": True, # 捕获视口外内容 + "fromSurface": True + }) + + # 保存截图 + image_data = base64.b64decode(result["data"]) + with open("cdp_screenshot.png", "wb") as f: + f.write(image_data) + + print("CDP 截图已保存: cdp_screenshot.png") + + await browser.close() + + +async def demo_cdp_dom(): + """演示 CDP DOM 操作""" + print("\n" + "=" * 50) + print("4. CDP DOM 操作") + print("=" * 50) + + async with async_playwright() as p: + browser = await p.chromium.launch(headless=True) + context = await browser.new_context() + page = await context.new_page() + + client = await context.new_cdp_session(page) + + await page.goto("https://quotes.toscrape.com/") + + # 启用 DOM + await client.send("DOM.enable") + + # 获取文档 + doc = await client.send("DOM.getDocument") + root_node_id = doc["root"]["nodeId"] + + # 查询选择器 + result = await client.send("DOM.querySelectorAll", { + "nodeId": root_node_id, + "selector": "div.quote" + }) + + print(f"找到 {len(result['nodeIds'])} 个 quote 元素") + + # 获取第一个元素的 HTML + if result["nodeIds"]: + outer_html = await client.send("DOM.getOuterHTML", { + "nodeId": result["nodeIds"][0] + }) + print(f"第一个元素 HTML 长度: {len(outer_html['outerHTML'])} 字符") + + await browser.close() + + +async def demo_cdp_cookies(): + """演示 CDP Cookie 操作""" + print("\n" + "=" * 50) + print("5. CDP Cookie 操作") + print("=" * 50) + + async with async_playwright() as p: + browser = await p.chromium.launch(headless=True) + context = await browser.new_context() + page = await context.new_page() + + client = await context.new_cdp_session(page) + + # 启用网络 + await client.send("Network.enable") + + await page.goto("https://quotes.toscrape.com/login") + + # 获取所有 cookies + cookies = await client.send("Network.getAllCookies") + print(f"当前 Cookies 数量: {len(cookies['cookies'])}") + + # 设置自定义 cookie + await client.send("Network.setCookie", { + "name": "custom_cookie", + "value": "test_value", + "domain": "quotes.toscrape.com", + "path": "/" + }) + + # 再次获取 + cookies = await client.send("Network.getAllCookies") + print(f"添加后 Cookies 数量: {len(cookies['cookies'])}") + + for cookie in cookies['cookies']: + print(f" {cookie['name']}: {cookie['value'][:20]}...") + + await browser.close() + + +async def demo_connect_existing(): + """演示连接已有浏览器（需要手动启动 Chrome）""" + print("\n" + "=" * 50) + print("6. 连接已有浏览器") + print("=" * 50) + + print("此功能需要先手动启动带调试端口的 Chrome:") + print(" chrome --remote-debugging-port=9222") + print("") + print("跳过此演示...") + + # 实际代码: + # async with async_playwright() as p: + # browser = await p.chromium.connect_over_cdp("http://localhost:9222") + # contexts = browser.contexts + # if contexts: + # page = contexts[0].pages[0] + # print(f"已连接到: {page.url}") + # await browser.close() + + +async def main(): + """主函数""" + logger.remove() + logger.add( + lambda msg: print(msg, end=""), + format="{time:HH:mm:ss} | {level: <8} | {message}", + level="INFO" + ) + + print("=" * 50) + print("Playwright CDP 模式演示") + print("=" * 50) + + await demo_cdp_session() + await demo_network_emulation() + await demo_cdp_screenshot() + await demo_cdp_dom() + await demo_cdp_cookies() + await demo_connect_existing() + + print("\n" + "=" * 50) + print("演示完成") + print("=" * 50) + + +if __name__ == "__main__": + asyncio.run(main()) diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/05_Playwright\350\277\233\351\230\266_\345\217\215\346\243\200\346\265\213\344\270\216\346\200\247\350\203\275\344\274\230\345\214\226/performance_optimization.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/05_Playwright\350\277\233\351\230\266_\345\217\215\346\243\200\346\265\213\344\270\216\346\200\247\350\203\275\344\274\230\345\214\226/performance_optimization.py" new file mode 100644 index 0000000..4001b1f --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/05_Playwright\350\277\233\351\230\266_\345\217\215\346\243\200\346\265\213\344\270\216\346\200\247\350\203\275\344\274\230\345\214\226/performance_optimization.py" @@ -0,0 +1,332 @@ +# -*- coding: utf-8 -*- +# @Desc: Playwright 性能优化演示 + +import asyncio +import time +from typing import List, Dict, Optional +from playwright.async_api import async_playwright, Browser, BrowserContext, Page +from loguru import logger + + +class ResourceBlocker: + """资源拦截器 - 阻止不必要的资源加载""" + + # 要阻止的资源类型 + BLOCKED_RESOURCE_TYPES = { + "image", + "media", + "font", + "stylesheet", + } + + # 要阻止的 URL 模式 + BLOCKED_URL_PATTERNS = [ + "**/analytics**", + "**/gtag/**", + "**/google-analytics**", + "**/facebook.com/**", + "**/doubleclick.net/**", + "**/*.woff", + "**/*.woff2", + "**/*.ttf", + ] + + @classmethod + async def setup(cls, context: BrowserContext): + """设置资源拦截""" + # 按资源类型拦截 + async def block_by_type(route): + if route.request.resource_type in cls.BLOCKED_RESOURCE_TYPES: + await route.abort() + else: + await route.continue_() + + await context.route("**/*", block_by_type) + + logger.debug("资源拦截器已设置") + + @classmethod + async def setup_selective(cls, context: BrowserContext, allow_css: bool = False): + """选择性资源拦截""" + blocked_types = cls.BLOCKED_RESOURCE_TYPES.copy() + if allow_css: + blocked_types.discard("stylesheet") + + async def block_selective(route): + if route.request.resource_type in blocked_types: + await route.abort() + else: + await route.continue_() + + await context.route("**/*", block_selective) + + +class ContextPool: + """浏览器上下文池""" + + def __init__(self, browser: Browser, pool_size: int = 3): + self.browser = browser + self.pool_size = pool_size + self._contexts: List[BrowserContext] = [] + self._available: asyncio.Queue = asyncio.Queue() + self._lock = asyncio.Lock() + + async def initialize(self, with_stealth: bool = False, block_resources: bool = True): + """初始化上下文池""" + for i in range(self.pool_size): + context = await self.browser.new_context( + viewport={"width": 1920, "height": 1080} + ) + + if block_resources: + await ResourceBlocker.setup(context) + + if with_stealth: + await context.add_init_script(""" + Object.defineProperty(navigator, 'webdriver', { get: () => undefined }); + """) + + self._contexts.append(context) + await self._available.put(context) + + logger.info(f"上下文池初始化完成，大小: {self.pool_size}") + + async def acquire(self) -> BrowserContext: + """获取上下文""" + return await self._available.get() + + async def release(self, context: BrowserContext): + """释放上下文""" + # 清理 cookies + await context.clear_cookies() + await self._available.put(context) + + async def close_all(self): + """关闭所有上下文""" + for context in self._contexts: + await context.close() + self._contexts.clear() + logger.info("所有上下文已关闭") + + +class OptimizedCrawler: + """优化的爬虫""" + + def __init__( + self, + max_concurrent: int = 5, + block_resources: bool = True, + use_stealth: bool = True + ): + self.max_concurrent = max_concurrent + self.block_resources = block_resources + self.use_stealth = use_stealth + + self._browser: Optional[Browser] = None + self._context_pool: Optional[ContextPool] = None + self._semaphore = asyncio.Semaphore(max_concurrent) + + # 统计 + self.stats = { + "total": 0, + "success": 0, + "failed": 0, + "total_time": 0.0, + } + + async def start(self, playwright): + """启动爬虫""" + self._browser = await playwright.chromium.launch(headless=True) + self._context_pool = ContextPool(self._browser, pool_size=self.max_concurrent) + await self._context_pool.initialize( + with_stealth=self.use_stealth, + block_resources=self.block_resources + ) + logger.info("优化爬虫已启动") + + async def stop(self): + """停止爬虫""" + if self._context_pool: + await self._context_pool.close_all() + if self._browser: + await self._browser.close() + logger.info("优化爬虫已停止") + + async def fetch(self, url: str) -> Dict: + """获取单个页面""" + start_time = time.time() + self.stats["total"] += 1 + + async with self._semaphore: + context = await self._context_pool.acquire() + page = await context.new_page() + + try: + await page.goto(url, wait_until="domcontentloaded", timeout=15000) + title = await page.title() + content_length = len(await page.content()) + + elapsed = time.time() - start_time + self.stats["success"] += 1 + self.stats["total_time"] += elapsed + + return { + "url": url, + "title": title, + "content_length": content_length, + "time": elapsed, + "success": True + } + + except Exception as e: + self.stats["failed"] += 1 + return { + "url": url, + "error": str(e), + "success": False + } + + finally: + await page.close() + await self._context_pool.release(context) + + async def fetch_batch(self, urls: List[str]) -> List[Dict]: + """批量获取页面""" + tasks = [self.fetch(url) for url in urls] + return await asyncio.gather(*tasks) + + +async def demo_resource_blocking(): + """演示资源拦截效果""" + print("\n" + "=" * 50) + print("1. 资源拦截效果对比") + print("=" * 50) + + test_url = "https://quotes.toscrape.com/" + + async with async_playwright() as p: + browser = await p.chromium.launch(headless=True) + + # 不拦截资源 + context1 = await browser.new_context() + page1 = await context1.new_page() + + start = time.time() + await page1.goto(test_url, wait_until="load") + time_without_blocking = time.time() - start + await context1.close() + + # 拦截资源 + context2 = await browser.new_context() + await ResourceBlocker.setup(context2) + page2 = await context2.new_page() + + start = time.time() + await page2.goto(test_url, wait_until="load") + time_with_blocking = time.time() - start + await context2.close() + + print(f"不拦截资源: {time_without_blocking:.2f}s") + print(f"拦截资源: {time_with_blocking:.2f}s") + print(f"提升: {(1 - time_with_blocking/time_without_blocking)*100:.1f}%") + + await browser.close() + + +async def demo_context_pool(): + """演示上下文池""" + print("\n" + "=" * 50) + print("2. 上下文池演示") + print("=" * 50) + + async with async_playwright() as p: + browser = await p.chromium.launch(headless=True) + pool = ContextPool(browser, pool_size=3) + await pool.initialize() + + # 模拟并发使用 + async def use_context(task_id: int): + context = await pool.acquire() + page = await context.new_page() + await page.goto("https://example.com") + print(f"任务 {task_id}: 使用上下文完成") + await page.close() + await pool.release(context) + + # 启动 5 个任务（但池大小只有 3） + tasks = [use_context(i) for i in range(5)] + await asyncio.gather(*tasks) + + await pool.close_all() + await browser.close() + + +async def demo_optimized_crawler(): + """演示优化爬虫""" + print("\n" + "=" * 50) + print("3. 优化爬虫演示") + print("=" * 50) + + urls = [ + "https://quotes.toscrape.com/", + "https://quotes.toscrape.com/page/2/", + "https://quotes.toscrape.com/page/3/", + "https://example.com/", + "https://httpbin.org/html", + ] + + async with async_playwright() as p: + crawler = OptimizedCrawler( + max_concurrent=3, + block_resources=True, + use_stealth=True + ) + + await crawler.start(p) + + try: + results = await crawler.fetch_batch(urls) + + print("\n爬取结果:") + for result in results: + if result["success"]: + print(f" ✓ {result['url'][:40]}... ({result['time']:.2f}s)") + else: + print(f" ✗ {result['url'][:40]}... ({result['error'][:30]})") + + print(f"\n统计:") + print(f" 总请求: {crawler.stats['total']}") + print(f" 成功: {crawler.stats['success']}") + print(f" 失败: {crawler.stats['failed']}") + if crawler.stats['success'] > 0: + avg_time = crawler.stats['total_time'] / crawler.stats['success'] + print(f" 平均耗时: {avg_time:.2f}s") + + finally: + await crawler.stop() + + +async def main(): + """主函数""" + logger.remove() + logger.add( + lambda msg: print(msg, end=""), + format="{time:HH:mm:ss} | {level: <8} | {message}", + level="INFO" + ) + + print("=" * 50) + print("Playwright 性能优化演示") + print("=" * 50) + + await demo_resource_blocking() + await demo_context_pool() + await demo_optimized_crawler() + + print("\n" + "=" * 50) + print("演示完成") + print("=" * 50) + + +if __name__ == "__main__": + asyncio.run(main()) diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/05_Playwright\350\277\233\351\230\266_\345\217\215\346\243\200\346\265\213\344\270\216\346\200\247\350\203\275\344\274\230\345\214\226/stealth_demo.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/05_Playwright\350\277\233\351\230\266_\345\217\215\346\243\200\346\265\213\344\270\216\346\200\247\350\203\275\344\274\230\345\214\226/stealth_demo.py" new file mode 100644 index 0000000..c7b1f10 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/05_Playwright\350\277\233\351\230\266_\345\217\215\346\243\200\346\265\213\344\270\216\346\200\247\350\203\275\344\274\230\345\214\226/stealth_demo.py" @@ -0,0 +1,274 @@ +# -*- coding: utf-8 -*- +# @Desc: Playwright stealth 反检测演示 + +import asyncio +from playwright.async_api import async_playwright +from loguru import logger + + +# stealth.min.js 核心脚本（简化版） +# 实际使用时建议使用完整的 stealth.min.js +STEALTH_JS_MINIMAL = """ +// 1. 隐藏 webdriver 标志 +Object.defineProperty(navigator, 'webdriver', { + get: () => undefined +}); + +// 2. 模拟 Chrome 对象 +window.chrome = { + runtime: {}, + loadTimes: function() {}, + csi: function() {}, + app: {} +}; + +// 3. 模拟 plugins 列表 +Object.defineProperty(navigator, 'plugins', { + get: () => { + const plugins = [ + { name: 'Chrome PDF Plugin', filename: 'internal-pdf-viewer' }, + { name: 'Chrome PDF Viewer', filename: 'mhjfbmdgcfjbbpaeojofohoefgiehjai' }, + { name: 'Native Client', filename: 'internal-nacl-plugin' } + ]; + plugins.length = 3; + return plugins; + } +}); + +// 4. 模拟 languages +Object.defineProperty(navigator, 'languages', { + get: () => ['zh-CN', 'zh', 'en-US', 'en'] +}); + +// 5. 修复 permissions API +const originalQuery = window.navigator.permissions.query; +window.navigator.permissions.query = (parameters) => ( + parameters.name === 'notifications' ? + Promise.resolve({ state: Notification.permission }) : + originalQuery(parameters) +); + +// 6. 隐藏 automation 标志 +if (navigator.userAgentData) { + Object.defineProperty(navigator.userAgentData, 'brands', { + get: () => [ + { brand: 'Google Chrome', version: '131' }, + { brand: 'Chromium', version: '131' }, + { brand: 'Not_A Brand', version: '8' } + ] + }); +} + +// 7. 修复 iframe contentWindow +const originalAttachShadow = Element.prototype.attachShadow; +Element.prototype.attachShadow = function(init) { + if (init.mode === 'closed') { + init.mode = 'open'; + } + return originalAttachShadow.call(this, init); +}; + +console.log('Stealth script injected!'); +""" + + +async def create_stealth_browser(playwright, headless: bool = True): + """ + 创建带反检测的浏览器实例 + + Args: + playwright: playwright 实例 + headless: 是否无头模式 + + Returns: + 配置好的浏览器实例 + """ + browser = await playwright.chromium.launch( + headless=headless, + args=[ + '--disable-blink-features=AutomationControlled', # 禁用自动化控制特征 + '--disable-dev-shm-usage', + '--no-sandbox', + ] + ) + return browser + + +async def create_stealth_context(browser, stealth_js: str = None): + """ + 创建带反检测的浏览器上下文 + + Args: + browser: 浏览器实例 + stealth_js: stealth 脚本内容 + + Returns: + 配置好的上下文 + """ + context = await browser.new_context( + viewport={'width': 1920, 'height': 1080}, + user_agent=( + 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) ' + 'AppleWebKit/537.36 (KHTML, like Gecko) ' + 'Chrome/131.0.0.0 Safari/537.36' + ), + locale='zh-CN', + timezone_id='Asia/Shanghai', + ) + + # 注入 stealth 脚本 + js = stealth_js or STEALTH_JS_MINIMAL + await context.add_init_script(js) + + return context + + +async def test_detection(page, test_url: str = "https://bot.sannysoft.com/"): + """ + 测试反检测效果 + + Args: + page: 页面实例 + test_url: 检测网站 URL + """ + logger.info(f"访问检测网站: {test_url}") + await page.goto(test_url, wait_until="networkidle") + + # 检查关键指标 + checks = { + "webdriver": await page.evaluate("navigator.webdriver"), + "chrome": await page.evaluate("!!window.chrome"), + "plugins_length": await page.evaluate("navigator.plugins.length"), + "languages": await page.evaluate("navigator.languages"), + } + + logger.info("检测结果:") + for key, value in checks.items(): + status = "✓" if _check_passed(key, value) else "✗" + logger.info(f" {status} {key}: {value}") + + # 保存截图 + await page.screenshot(path="stealth_test_result.png", full_page=True) + logger.info("截图已保存: stealth_test_result.png") + + +def _check_passed(key: str, value) -> bool: + """检查是否通过""" + if key == "webdriver": + return value is None or value == "undefined" + elif key == "chrome": + return value is True + elif key == "plugins_length": + return value > 0 + elif key == "languages": + return len(value) > 0 + return True + + +async def demo_without_stealth(): + """演示没有 stealth 的情况""" + print("\n" + "=" * 50) + print("1. 没有 stealth 的浏览器") + print("=" * 50) + + async with async_playwright() as p: + browser = await p.chromium.launch(headless=True) + page = await browser.new_page() + + await page.goto("https://bot.sannysoft.com/", wait_until="networkidle") + + # 检查 webdriver + webdriver = await page.evaluate("navigator.webdriver") + chrome = await page.evaluate("!!window.chrome") + + print(f"navigator.webdriver: {webdriver}") + print(f"window.chrome exists: {chrome}") + + await page.screenshot(path="no_stealth.png", full_page=True) + print("截图已保存: no_stealth.png") + + await browser.close() + + +async def demo_with_stealth(): + """演示使用 stealth 的情况""" + print("\n" + "=" * 50) + print("2. 使用 stealth 的浏览器") + print("=" * 50) + + async with async_playwright() as p: + browser = await create_stealth_browser(p, headless=True) + context = await create_stealth_context(browser) + page = await context.new_page() + + await test_detection(page) + + await browser.close() + + +async def demo_custom_stealth(): + """演示自定义 stealth 配置""" + print("\n" + "=" * 50) + print("3. 自定义 stealth 配置") + print("=" * 50) + + # 可以根据需要添加更多的规避代码 + custom_stealth = STEALTH_JS_MINIMAL + """ + // 额外的自定义规避 + Object.defineProperty(navigator, 'hardwareConcurrency', { + get: () => 8 + }); + + Object.defineProperty(navigator, 'deviceMemory', { + get: () => 8 + }); + + Object.defineProperty(screen, 'colorDepth', { + get: () => 24 + }); + """ + + async with async_playwright() as p: + browser = await create_stealth_browser(p, headless=True) + context = await create_stealth_context(browser, custom_stealth) + page = await context.new_page() + + # 检查自定义属性 + await page.goto("about:blank") + + hardware = await page.evaluate("navigator.hardwareConcurrency") + memory = await page.evaluate("navigator.deviceMemory") + color_depth = await page.evaluate("screen.colorDepth") + + print(f"hardwareConcurrency: {hardware}") + print(f"deviceMemory: {memory}") + print(f"colorDepth: {color_depth}") + + await browser.close() + + +async def main(): + """主函数""" + logger.remove() + logger.add( + lambda msg: print(msg, end=""), + format="{time:HH:mm:ss} | {level: <8} | {message}", + level="INFO" + ) + + print("=" * 50) + print("Playwright Stealth 反检测演示") + print("=" * 50) + + await demo_without_stealth() + await demo_with_stealth() + await demo_custom_stealth() + + print("\n" + "=" * 50) + print("演示完成") + print("=" * 50) + print("\n提示: 查看生成的截图文件对比效果") + + +if __name__ == "__main__": + asyncio.run(main()) diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/06_\347\231\273\345\275\225\350\256\244\350\257\201_Cookie\344\270\216Session\347\256\241\347\220\206/README.md" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/06_\347\231\273\345\275\225\350\256\244\350\257\201_Cookie\344\270\216Session\347\256\241\347\220\206/README.md" new file mode 100644 index 0000000..90180cc --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/06_\347\231\273\345\275\225\350\256\244\350\257\201_Cookie\344\270\216Session\347\256\241\347\220\206/README.md" @@ -0,0 +1,21 @@ +# 第06章：登录认证 - Cookie与Session管理 + +展示Cookie管理、登录状态检测、真实网站登录演示。 + +## 快速开始 + +```bash +cd 06_登录认证_Cookie与Session管理 +uv sync +uv run python cookie_manager.py +uv run python login_state_checker.py +uv run python session_demo.py # 包含 quotes.toscrape.com 真实登录演示 +``` + +### 新增功能 +✨ **真实登录演示**：`session_demo.py` 中的 `demo_real_login()` 函数演示了完整的 quotes.toscrape.com 登录流程。 + +### 核心依赖 +- `httpx` - HTTP客户端 +- `loguru` - 日志系统 +- `cryptography`（可选）- Cookie加密 diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/06_\347\231\273\345\275\225\350\256\244\350\257\201_Cookie\344\270\216Session\347\256\241\347\220\206/bilibili_cookie.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/06_\347\231\273\345\275\225\350\256\244\350\257\201_Cookie\344\270\216Session\347\256\241\347\220\206/bilibili_cookie.py" new file mode 100644 index 0000000..21d5118 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/06_\347\231\273\345\275\225\350\256\244\350\257\201_Cookie\344\270\216Session\347\256\241\347\220\206/bilibili_cookie.py" @@ -0,0 +1,483 @@ +# -*- coding: utf-8 -*- +""" +B站 Cookie 管理实战 + +本模块展示如何管理 B站 Cookie，包括： +- B站核心 Cookie 结构（SESSDATA、DedeUserID、bili_jct） +- Cookie 提取与存储 +- 登录状态验证 +- httpx 和 Playwright 集成 + +这是第06章"登录认证-Cookie与Session管理"的B站实战示例。 + +与第11章综合实战项目的关联： +- login/auth.py: BilibiliCookieManager 登录管理 +- models/cookies.py: Cookie 模型定义 +""" + +import json +import asyncio +from pathlib import Path +from typing import Optional, Dict, List +from dataclasses import dataclass, field +from datetime import datetime + +import httpx +from loguru import logger + + +# ============== B站 Cookie 数据类 ============== + +@dataclass +class BilibiliCookies: + """ + B站 Cookie 数据类 + + 核心 Cookie： + - SESSDATA: 会话凭证（最重要） + - DedeUserID: 用户ID + - bili_jct: CSRF Token（POST请求必需） + + 辅助 Cookie： + - buvid3/buvid4: 设备标识 + - sid: 短会话ID + """ + sessdata: str + dede_user_id: str + bili_jct: str + buvid3: str = "" + buvid4: str = "" + sid: str = "" + raw_cookies: List[dict] = field(default_factory=list) + + @classmethod + def from_playwright_cookies(cls, cookies: List[dict]) -> "BilibiliCookies": + """ + 从 Playwright 格式的 Cookie 创建 + + Args: + cookies: Playwright context.cookies() 返回的列表 + + Returns: + BilibiliCookies 实例 + """ + cookie_dict = {c["name"]: c["value"] for c in cookies} + + return cls( + sessdata=cookie_dict.get("SESSDATA", ""), + dede_user_id=cookie_dict.get("DedeUserID", ""), + bili_jct=cookie_dict.get("bili_jct", ""), + buvid3=cookie_dict.get("buvid3", ""), + buvid4=cookie_dict.get("buvid4", ""), + sid=cookie_dict.get("sid", ""), + raw_cookies=cookies + ) + + @classmethod + def from_browser_string(cls, cookie_string: str) -> "BilibiliCookies": + """ + 从浏览器复制的 Cookie 字符串创建 + + Args: + cookie_string: 格式 "SESSDATA=xxx; DedeUserID=xxx; bili_jct=xxx" + + Returns: + BilibiliCookies 实例 + """ + cookie_dict = {} + for item in cookie_string.split(";"): + item = item.strip() + if "=" in item: + key, value = item.split("=", 1) + cookie_dict[key.strip()] = value.strip() + + return cls( + sessdata=cookie_dict.get("SESSDATA", ""), + dede_user_id=cookie_dict.get("DedeUserID", ""), + bili_jct=cookie_dict.get("bili_jct", ""), + buvid3=cookie_dict.get("buvid3", ""), + buvid4=cookie_dict.get("buvid4", ""), + sid=cookie_dict.get("sid", "") + ) + + def to_httpx_cookies(self) -> Dict[str, str]: + """转换为 httpx 可用的字典格式""" + cookies = { + "SESSDATA": self.sessdata, + "DedeUserID": self.dede_user_id, + "bili_jct": self.bili_jct, + } + if self.buvid3: + cookies["buvid3"] = self.buvid3 + if self.buvid4: + cookies["buvid4"] = self.buvid4 + if self.sid: + cookies["sid"] = self.sid + return cookies + + def to_playwright_cookies(self, domain: str = ".bilibili.com") -> List[dict]: + """ + 转换为 Playwright 可用的格式 + + Args: + domain: Cookie 的域名 + + Returns: + Playwright 格式的 Cookie 列表 + """ + if self.raw_cookies: + return self.raw_cookies + + cookies = [] + for name, value in self.to_httpx_cookies().items(): + cookies.append({ + "name": name, + "value": value, + "domain": domain, + "path": "/" + }) + return cookies + + def is_valid(self) -> bool: + """检查核心 Cookie 是否存在""" + return bool(self.sessdata and self.dede_user_id and self.bili_jct) + + def to_header_string(self) -> str: + """转换为请求头 Cookie 格式""" + return "; ".join(f"{k}={v}" for k, v in self.to_httpx_cookies().items()) + + +# ============== B站 Cookie 管理器 ============== + +class BilibiliCookieManager: + """ + B站 Cookie 管理器 + + 功能： + - Cookie 加载/保存 + - 登录状态检测 + - Cookie 有效性验证 + - 支持多种格式（JSON、字符串） + """ + + # 登录状态检测 API + CHECK_URL = "https://api.bilibili.com/x/web-interface/nav" + + # 请求头 + HEADERS = { + "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) " + "AppleWebKit/537.36 (KHTML, like Gecko) " + "Chrome/131.0.0.0 Safari/537.36", + "Referer": "https://www.bilibili.com/" + } + + def __init__(self, storage_path: str = "bilibili_cookies.json"): + """ + 初始化管理器 + + Args: + storage_path: Cookie 存储文件路径 + """ + self.storage_path = Path(storage_path) + self._cookies: Optional[BilibiliCookies] = None + self._last_check: Optional[datetime] = None + self._user_info: Optional[dict] = None + + async def load(self) -> bool: + """ + 从文件加载 Cookie + + Returns: + 是否加载成功 + """ + if not self.storage_path.exists(): + logger.warning(f"Cookie 文件不存在: {self.storage_path}") + return False + + try: + with open(self.storage_path, "r", encoding="utf-8") as f: + data = json.load(f) + + if isinstance(data, list): + # Playwright 格式 + self._cookies = BilibiliCookies.from_playwright_cookies(data) + elif isinstance(data, dict): + # 自定义格式 + self._cookies = BilibiliCookies( + sessdata=data.get("SESSDATA", ""), + dede_user_id=data.get("DedeUserID", ""), + bili_jct=data.get("bili_jct", ""), + buvid3=data.get("buvid3", ""), + buvid4=data.get("buvid4", ""), + sid=data.get("sid", "") + ) + + logger.info(f"Cookie 加载成功，用户ID: {self._cookies.dede_user_id}") + return True + + except Exception as e: + logger.error(f"加载 Cookie 失败: {e}") + return False + + async def save(self, cookies: Optional[BilibiliCookies] = None): + """ + 保存 Cookie 到文件 + + Args: + cookies: 要保存的 Cookie，如果为 None 则保存当前 Cookie + """ + if cookies: + self._cookies = cookies + + if not self._cookies: + logger.warning("没有 Cookie 可保存") + return + + self.storage_path.parent.mkdir(parents=True, exist_ok=True) + + data = { + "SESSDATA": self._cookies.sessdata, + "DedeUserID": self._cookies.dede_user_id, + "bili_jct": self._cookies.bili_jct, + "buvid3": self._cookies.buvid3, + "buvid4": self._cookies.buvid4, + "sid": self._cookies.sid, + "save_time": datetime.now().isoformat() + } + + with open(self.storage_path, "w", encoding="utf-8") as f: + json.dump(data, f, indent=2, ensure_ascii=False) + + logger.info(f"Cookie 已保存到: {self.storage_path}") + + async def verify(self) -> bool: + """ + 验证 Cookie 是否有效 + + Returns: + Cookie 是否有效 + """ + if not self._cookies or not self._cookies.is_valid(): + return False + + try: + async with httpx.AsyncClient( + cookies=self._cookies.to_httpx_cookies(), + headers=self.HEADERS, + timeout=10 + ) as client: + resp = await client.get(self.CHECK_URL) + data = resp.json() + + if data.get("code") == 0: + user_info = data.get("data", {}) + if user_info.get("isLogin"): + self._user_info = user_info + self._last_check = datetime.now() + logger.info(f"Cookie 有效，用户: {user_info.get('uname')}") + return True + + logger.warning("Cookie 已失效") + return False + + except Exception as e: + logger.error(f"验证 Cookie 失败: {e}") + return False + + async def get_valid_cookies(self) -> Optional[BilibiliCookies]: + """ + 获取有效的 Cookie + + Returns: + 有效的 BilibiliCookies，如果无效则返回 None + """ + if self._cookies is None: + await self.load() + + if self._cookies and await self.verify(): + return self._cookies + + return None + + def set_cookies_from_string(self, cookie_string: str): + """ + 从浏览器复制的字符串设置 Cookie + + Args: + cookie_string: 格式 "SESSDATA=xxx; DedeUserID=xxx; bili_jct=xxx" + """ + self._cookies = BilibiliCookies.from_browser_string(cookie_string) + logger.info(f"Cookie 已设置，用户ID: {self._cookies.dede_user_id}") + + @property + def cookies(self) -> Optional[BilibiliCookies]: + """获取当前 Cookie（不验证）""" + return self._cookies + + @property + def user_info(self) -> Optional[dict]: + """获取用户信息（需先调用 verify）""" + return self._user_info + + +# ============== Cookie 格式转换工具 ============== + +def playwright_cookies_to_httpx(playwright_cookies: List[dict]) -> dict: + """ + 将 Playwright 格式的 Cookie 转换为 httpx 格式 + + Args: + playwright_cookies: Playwright 格式 [{"name": "x", "value": "y", ...}, ...] + + Returns: + httpx 格式 {"name": "value", ...} + """ + return {c["name"]: c["value"] for c in playwright_cookies} + + +def httpx_cookies_to_playwright(cookies_dict: dict, domain: str = ".bilibili.com") -> List[dict]: + """ + 将 httpx 字典格式转换为 Playwright 格式 + + Args: + cookies_dict: 简单字典 {"name": "value", ...} + domain: Cookie 的域名 + + Returns: + Playwright 格式的 Cookie 列表 + """ + return [ + { + "name": name, + "value": value, + "domain": domain, + "path": "/" + } + for name, value in cookies_dict.items() + ] + + +# ============== 演示入口 ============== + +async def demo_bilibili_cookie(): + """演示 B站 Cookie 管理""" + logger.info("=" * 50) + logger.info("B站 Cookie 管理示例") + logger.info("=" * 50) + + # 1. 展示 Cookie 结构 + logger.info("\n--- 1. B站核心 Cookie 结构 ---") + logger.info("SESSDATA: 会话凭证（最重要，有效期约1个月）") + logger.info("DedeUserID: 用户ID") + logger.info("bili_jct: CSRF Token（POST请求必需）") + logger.info("buvid3/buvid4: 设备标识") + + # 2. 演示从字符串创建 Cookie + logger.info("\n--- 2. 从浏览器字符串创建 Cookie ---") + # 示例字符串（实际使用时替换为真实值） + sample_string = "SESSDATA=sample_sessdata; DedeUserID=123456; bili_jct=sample_csrf" + cookies = BilibiliCookies.from_browser_string(sample_string) + + logger.info(f"SESSDATA: {cookies.sessdata}") + logger.info(f"DedeUserID: {cookies.dede_user_id}") + logger.info(f"bili_jct: {cookies.bili_jct}") + logger.info(f"Cookie 是否有效: {cookies.is_valid()}") + + # 3. 演示 Cookie 管理器 + logger.info("\n--- 3. Cookie 管理器使用 ---") + manager = BilibiliCookieManager("data/bilibili_cookies.json") + + # 检查是否有现有 Cookie + if await manager.load(): + logger.info("已加载现有 Cookie") + + # 验证 Cookie + if await manager.verify(): + user_info = manager.user_info + if user_info: + logger.info(f"用户名: {user_info.get('uname')}") + logger.info(f"等级: {user_info.get('level_info', {}).get('current_level')}") + logger.info(f"硬币: {user_info.get('money')}") + else: + logger.warning("Cookie 已失效，需要重新登录") + else: + logger.info("没有找到 Cookie 文件") + + # 4. 展示使用方法 + logger.info("\n--- 4. 使用 Cookie 请求 API ---") + logger.info(""" +使用示例代码: + + # 方式1: 直接使用 httpx + async with httpx.AsyncClient( + cookies=cookies.to_httpx_cookies(), + headers={"Referer": "https://www.bilibili.com/"} + ) as client: + resp = await client.get("https://api.bilibili.com/x/web-interface/nav") + data = resp.json() + + # 方式2: 使用 Playwright + context = await browser.new_context() + await context.add_cookies(cookies.to_playwright_cookies()) + page = await context.new_page() + """) + + # 5. 格式转换演示 + logger.info("\n--- 5. Cookie 格式转换 ---") + playwright_format = cookies.to_playwright_cookies() + httpx_format = cookies.to_httpx_cookies() + header_format = cookies.to_header_string() + + logger.info(f"Playwright 格式: {playwright_format[:1]}...") + logger.info(f"httpx 格式: {list(httpx_format.keys())}") + logger.info(f"Header 格式: {header_format[:50]}...") + + +async def demo_with_real_api(): + """使用真实 API 的演示（需要有效 Cookie）""" + manager = BilibiliCookieManager("data/bilibili_cookies.json") + + if not await manager.load(): + logger.info("请先准备 Cookie 文件") + return + + cookies = await manager.get_valid_cookies() + if not cookies: + logger.error("Cookie 无效") + return + + async with httpx.AsyncClient( + cookies=cookies.to_httpx_cookies(), + headers={ + "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) " + "AppleWebKit/537.36 (KHTML, like Gecko) " + "Chrome/131.0.0.0 Safari/537.36", + "Referer": "https://www.bilibili.com/" + } + ) as client: + # 获取用户信息 + resp = await client.get("https://api.bilibili.com/x/web-interface/nav") + data = resp.json() + + if data.get("code") == 0: + user = data.get("data", {}) + print(f"\n用户名: {user.get('uname')}") + print(f"等级: {user.get('level_info', {}).get('current_level')}") + print(f"硬币: {user.get('money')}") + + # 获取收藏夹 + resp = await client.get( + "https://api.bilibili.com/x/v3/fav/folder/created/list-all", + params={"up_mid": cookies.dede_user_id} + ) + fav_data = resp.json() + + if fav_data.get("code") == 0: + folders = fav_data.get("data", {}).get("list", []) + print(f"\n收藏夹列表 ({len(folders)}个):") + for folder in folders[:5]: + print(f" - {folder.get('title')} ({folder.get('media_count')}个)") + + +if __name__ == "__main__": + asyncio.run(demo_bilibili_cookie()) diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/06_\347\231\273\345\275\225\350\256\244\350\257\201_Cookie\344\270\216Session\347\256\241\347\220\206/cookie_manager.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/06_\347\231\273\345\275\225\350\256\244\350\257\201_Cookie\344\270\216Session\347\256\241\347\220\206/cookie_manager.py" new file mode 100644 index 0000000..a42e1fe --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/06_\347\231\273\345\275\225\350\256\244\350\257\201_Cookie\344\270\216Session\347\256\241\347\220\206/cookie_manager.py" @@ -0,0 +1,433 @@ +# -*- coding: utf-8 -*- +# @Desc: Cookie 管理器 - 完整的 Cookie 存储、加载、验证和刷新功能 + +import json +import asyncio +from pathlib import Path +from typing import Optional, Callable, Awaitable, List, Dict +from datetime import datetime +from dataclasses import dataclass, field +from loguru import logger + +# 可选的加密支持 +try: + from cryptography.fernet import Fernet + HAS_CRYPTO = True +except ImportError: + HAS_CRYPTO = False + + +class CookieSerializer: + """Cookie 序列化工具""" + + @staticmethod + def to_json(cookies: list, filepath: str): + """保存为 JSON 格式""" + with open(filepath, "w", encoding="utf-8") as f: + json.dump(cookies, f, indent=2, ensure_ascii=False) + + @staticmethod + def from_json(filepath: str) -> list: + """从 JSON 加载""" + with open(filepath, "r", encoding="utf-8") as f: + return json.load(f) + + @staticmethod + def to_netscape(cookies: list, filepath: str): + """ + 保存为 Netscape 格式（兼容 curl/wget） + 格式: domain, flag, path, secure, expiry, name, value + """ + lines = ["# Netscape HTTP Cookie File", "# https://curl.se/docs/http-cookies.html"] + + for c in cookies: + domain = c.get("domain", "") + flag = "TRUE" if domain.startswith(".") else "FALSE" + path = c.get("path", "/") + secure = "TRUE" if c.get("secure", False) else "FALSE" + expiry = str(int(c.get("expires", 0))) + name = c.get("name", "") + value = c.get("value", "") + + lines.append(f"{domain}\t{flag}\t{path}\t{secure}\t{expiry}\t{name}\t{value}") + + with open(filepath, "w", encoding="utf-8") as f: + f.write("\n".join(lines)) + + @staticmethod + def to_dict(cookies: list) -> dict: + """转换为简单字典格式（name: value）""" + return {c["name"]: c["value"] for c in cookies} + + @staticmethod + def playwright_to_httpx(playwright_cookies: list) -> dict: + """将 Playwright 格式转换为 httpx 格式""" + return {c["name"]: c["value"] for c in playwright_cookies} + + @staticmethod + def dict_to_playwright(cookies_dict: dict, domain: str) -> list: + """将简单字典转换为 Playwright 格式""" + return [ + { + "name": name, + "value": value, + "domain": domain, + "path": "/" + } + for name, value in cookies_dict.items() + ] + + +class SecureCookieStorage: + """加密 Cookie 存储""" + + def __init__(self, key: bytes = None): + if not HAS_CRYPTO: + raise ImportError("需要安装 cryptography 库: pip install cryptography") + + # 如果没有提供密钥，生成新密钥 + self.key = key or Fernet.generate_key() + self.cipher = Fernet(self.key) + + def save_key(self, filepath: str): + """保存密钥（请妥善保管）""" + with open(filepath, "wb") as f: + f.write(self.key) + logger.info(f"密钥已保存到: {filepath}") + + @classmethod + def load_key(cls, filepath: str) -> "SecureCookieStorage": + """从文件加载密钥""" + with open(filepath, "rb") as f: + return cls(f.read()) + + def encrypt_cookies(self, cookies: list, filepath: str): + """加密并保存 Cookie""" + data = json.dumps(cookies).encode("utf-8") + encrypted = self.cipher.encrypt(data) + with open(filepath, "wb") as f: + f.write(encrypted) + logger.info(f"Cookie 已加密保存到: {filepath}") + + def decrypt_cookies(self, filepath: str) -> list: + """解密并加载 Cookie""" + with open(filepath, "rb") as f: + encrypted = f.read() + decrypted = self.cipher.decrypt(encrypted) + cookies = json.loads(decrypted.decode("utf-8")) + logger.info(f"已解密加载 {len(cookies)} 个 Cookie") + return cookies + + +@dataclass +class AccountCookie: + """账号 Cookie 信息""" + account_id: str + cookies: dict + last_used: Optional[datetime] = None + use_count: int = 0 + is_valid: bool = True + created_at: datetime = field(default_factory=datetime.now) + + +class CookieRotator: + """Cookie 轮换器 - 支持多账号 Cookie 管理""" + + def __init__(self, min_interval: float = 5.0): + """ + Args: + min_interval: 同一账号最小使用间隔（秒） + """ + self._accounts: Dict[str, AccountCookie] = {} + self._min_interval = min_interval + self._lock = asyncio.Lock() + + def add_account(self, account_id: str, cookies: dict): + """添加账号""" + self._accounts[account_id] = AccountCookie( + account_id=account_id, + cookies=cookies + ) + logger.info(f"添加账号: {account_id}") + + def remove_account(self, account_id: str): + """移除账号""" + if account_id in self._accounts: + del self._accounts[account_id] + logger.info(f"移除账号: {account_id}") + + async def get_cookies(self) -> Optional[dict]: + """获取一个可用的 Cookie（负载均衡）""" + async with self._lock: + now = datetime.now() + available = [] + + for acc in self._accounts.values(): + if not acc.is_valid: + continue + + # 检查使用间隔 + if acc.last_used: + elapsed = (now - acc.last_used).total_seconds() + if elapsed < self._min_interval: + continue + + available.append(acc) + + if not available: + logger.warning("没有可用的账号") + return None + + # 选择使用次数最少的账号（负载均衡） + selected = min(available, key=lambda x: x.use_count) + selected.last_used = now + selected.use_count += 1 + + logger.debug(f"使用账号: {selected.account_id} (使用次数: {selected.use_count})") + return selected.cookies + + def mark_invalid(self, account_id: str): + """标记账号失效""" + if account_id in self._accounts: + self._accounts[account_id].is_valid = False + logger.warning(f"账号已标记失效: {account_id}") + + def mark_valid(self, account_id: str): + """标记账号有效""" + if account_id in self._accounts: + self._accounts[account_id].is_valid = True + logger.info(f"账号已标记有效: {account_id}") + + @property + def valid_count(self) -> int: + """有效账号数量""" + return sum(1 for acc in self._accounts.values() if acc.is_valid) + + @property + def total_count(self) -> int: + """总账号数量""" + return len(self._accounts) + + def get_stats(self) -> dict: + """获取统计信息""" + return { + "total": self.total_count, + "valid": self.valid_count, + "invalid": self.total_count - self.valid_count, + "accounts": [ + { + "id": acc.account_id, + "valid": acc.is_valid, + "use_count": acc.use_count, + "last_used": acc.last_used.isoformat() if acc.last_used else None + } + for acc in self._accounts.values() + ] + } + + +class CookieManager: + """完整的 Cookie 管理器""" + + def __init__( + self, + storage_path: str, + login_checker: Callable[[dict], Awaitable[bool]], + auto_refresh_callback: Optional[Callable[[], Awaitable[list]]] = None, + check_interval: int = 300 + ): + """ + Args: + storage_path: Cookie 存储路径 + login_checker: 登录状态检测函数，接收 cookies dict，返回是否有效 + auto_refresh_callback: 自动刷新回调（如重新登录），返回新的 cookies list + check_interval: 检测间隔（秒），默认 5 分钟 + """ + self.storage_path = Path(storage_path) + self.login_checker = login_checker + self.auto_refresh_callback = auto_refresh_callback + self._check_interval = check_interval + + self._cookies: Optional[list] = None + self._last_check: Optional[datetime] = None + + async def load(self) -> bool: + """加载 Cookie""" + if not self.storage_path.exists(): + logger.warning(f"Cookie 文件不存在: {self.storage_path}") + return False + + try: + with open(self.storage_path, "r", encoding="utf-8") as f: + self._cookies = json.load(f) + logger.info(f"加载了 {len(self._cookies)} 个 Cookie") + return True + except Exception as e: + logger.error(f"加载 Cookie 失败: {e}") + return False + + async def save(self): + """保存 Cookie""" + if self._cookies: + self.storage_path.parent.mkdir(parents=True, exist_ok=True) + with open(self.storage_path, "w", encoding="utf-8") as f: + json.dump(self._cookies, f, indent=2, ensure_ascii=False) + logger.info(f"保存了 {len(self._cookies)} 个 Cookie") + + def update(self, cookies: list): + """更新 Cookie""" + self._cookies = cookies + self._last_check = datetime.now() + logger.info(f"更新了 {len(cookies)} 个 Cookie") + + async def get_valid_cookies(self) -> Optional[dict]: + """ + 获取有效的 Cookie（自动检测和刷新） + + Returns: + 简单字典格式的 Cookie，如 {"name": "value", ...} + """ + # 首次加载 + if self._cookies is None: + await self.load() + + if not self._cookies: + if self.auto_refresh_callback: + await self._refresh_cookies() + if not self._cookies: + return None + + # 检查是否需要验证 + if self._need_check(): + is_valid = await self._validate() + if not is_valid: + if self.auto_refresh_callback: + await self._refresh_cookies() + else: + return None + + # 返回简单字典格式 + return CookieSerializer.to_dict(self._cookies) + + async def get_playwright_cookies(self) -> Optional[list]: + """ + 获取 Playwright 格式的 Cookie + + Returns: + Playwright 格式的 Cookie 列表 + """ + if self._cookies is None: + await self.load() + return self._cookies + + def _need_check(self) -> bool: + """是否需要检测""" + if self._last_check is None: + return True + elapsed = (datetime.now() - self._last_check).total_seconds() + return elapsed > self._check_interval + + async def _validate(self) -> bool: + """验证 Cookie 是否有效""" + logger.debug("验证 Cookie 有效性...") + cookies_dict = CookieSerializer.to_dict(self._cookies) + is_valid = await self.login_checker(cookies_dict) + self._last_check = datetime.now() + + if is_valid: + logger.info("Cookie 验证通过") + else: + logger.warning("Cookie 已失效") + + return is_valid + + async def _refresh_cookies(self): + """刷新 Cookie""" + if not self.auto_refresh_callback: + return + + logger.info("开始刷新 Cookie...") + try: + new_cookies = await self.auto_refresh_callback() + if new_cookies: + self._cookies = new_cookies + self._last_check = datetime.now() + await self.save() + logger.info("Cookie 刷新成功") + except Exception as e: + logger.error(f"Cookie 刷新失败: {e}") + + async def force_refresh(self) -> bool: + """强制刷新 Cookie""" + if not self.auto_refresh_callback: + logger.error("未设置刷新回调") + return False + + await self._refresh_cookies() + return self._cookies is not None + + +async def demo(): + """演示 Cookie 管理器的使用""" + import httpx + + # 模拟登录检测函数 + async def check_login(cookies: dict) -> bool: + """检测登录状态""" + try: + async with httpx.AsyncClient(cookies=cookies) as client: + resp = await client.get("https://httpbin.org/cookies", timeout=10) + data = resp.json() + # 检查是否有我们设置的 cookie + return "session" in data.get("cookies", {}) + except Exception as e: + logger.error(f"检测失败: {e}") + return False + + # 创建管理器 + manager = CookieManager( + storage_path="data/demo_cookies.json", + login_checker=check_login + ) + + # 手动设置一些测试 Cookie + test_cookies = [ + {"name": "session", "value": "test123", "domain": "httpbin.org", "path": "/"}, + {"name": "user", "value": "demo", "domain": "httpbin.org", "path": "/"} + ] + manager.update(test_cookies) + await manager.save() + + # 获取有效的 Cookie + cookies = await manager.get_valid_cookies() + if cookies: + print(f"获取到 Cookie: {cookies}") + + # 演示 Cookie 轮换器 + print("\n--- Cookie 轮换器演示 ---") + rotator = CookieRotator(min_interval=2.0) + + # 添加多个账号 + rotator.add_account("account_1", {"session": "sess_1", "token": "tok_1"}) + rotator.add_account("account_2", {"session": "sess_2", "token": "tok_2"}) + rotator.add_account("account_3", {"session": "sess_3", "token": "tok_3"}) + + # 模拟获取 Cookie（负载均衡） + for i in range(5): + cookies = await rotator.get_cookies() + print(f"请求 {i+1}: 使用 Cookie = {cookies}") + await asyncio.sleep(0.5) + + # 查看统计 + print(f"\n统计信息: {rotator.get_stats()}") + + +if __name__ == "__main__": + logger.remove() + logger.add( + lambda msg: print(msg, end=""), + format="{time:HH:mm:ss} | {level: <8} | {message}", + level="DEBUG" + ) + + asyncio.run(demo()) diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/06_\347\231\273\345\275\225\350\256\244\350\257\201_Cookie\344\270\216Session\347\256\241\347\220\206/login_state_checker.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/06_\347\231\273\345\275\225\350\256\244\350\257\201_Cookie\344\270\216Session\347\256\241\347\220\206/login_state_checker.py" new file mode 100644 index 0000000..5198ae1 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/06_\347\231\273\345\275\225\350\256\244\350\257\201_Cookie\344\270\216Session\347\256\241\347\220\206/login_state_checker.py" @@ -0,0 +1,396 @@ +# -*- coding: utf-8 -*- +# @Desc: 登录状态检测器 - 支持多种检测方式 + +import asyncio +import time +from typing import Callable, Optional +from datetime import datetime, timedelta +import httpx +from loguru import logger + + +class LoginStateChecker: + """登录状态检测器""" + + def __init__( + self, + check_url: str, + success_indicator: Callable[[httpx.Response], bool], + timeout: float = 10.0 + ): + """ + Args: + check_url: 用于检测登录状态的 URL + success_indicator: 判断是否登录成功的函数，接收响应对象 + timeout: 请求超时时间 + """ + self.check_url = check_url + self.success_indicator = success_indicator + self.timeout = timeout + + async def is_logged_in(self, cookies: dict) -> bool: + """ + 检查是否已登录 + + Args: + cookies: Cookie 字典 + + Returns: + 是否已登录 + """ + try: + async with httpx.AsyncClient(cookies=cookies, follow_redirects=True) as client: + resp = await client.get(self.check_url, timeout=self.timeout) + return self.success_indicator(resp) + except httpx.TimeoutException: + logger.warning(f"登录状态检测超时: {self.check_url}") + return False + except Exception as e: + logger.warning(f"登录状态检测失败: {e}") + return False + + @classmethod + def create_json_checker( + cls, + check_url: str, + success_field: str, + timeout: float = 10.0 + ) -> "LoginStateChecker": + """ + 创建 JSON 响应检测器 + + 检测响应 JSON 中是否包含指定字段 + + Args: + check_url: API 地址 + success_field: 成功时 JSON 中存在的字段 + timeout: 超时时间 + + Example: + # 检测 /api/user/info 返回的 JSON 是否包含 user_id 字段 + checker = LoginStateChecker.create_json_checker( + "https://example.com/api/user/info", + "user_id" + ) + """ + def indicator(resp: httpx.Response) -> bool: + try: + if resp.status_code != 200: + return False + data = resp.json() + return success_field in data + except Exception: + return False + + return cls(check_url, indicator, timeout) + + @classmethod + def create_json_value_checker( + cls, + check_url: str, + field: str, + expected_value, + timeout: float = 10.0 + ) -> "LoginStateChecker": + """ + 创建 JSON 值检测器 + + 检测响应 JSON 中指定字段是否等于预期值 + + Args: + check_url: API 地址 + field: 要检测的字段 + expected_value: 预期值 + timeout: 超时时间 + + Example: + # 检测 /api/status 返回的 logged_in 字段是否为 True + checker = LoginStateChecker.create_json_value_checker( + "https://example.com/api/status", + "logged_in", + True + ) + """ + def indicator(resp: httpx.Response) -> bool: + try: + if resp.status_code != 200: + return False + data = resp.json() + return data.get(field) == expected_value + except Exception: + return False + + return cls(check_url, indicator, timeout) + + @classmethod + def create_redirect_checker( + cls, + check_url: str, + login_url_pattern: str, + timeout: float = 10.0 + ) -> "LoginStateChecker": + """ + 创建重定向检测器 + + 如果被重定向到登录页，说明未登录 + + Args: + check_url: 需要登录的页面 + login_url_pattern: 登录页 URL 特征（如 "/login"） + timeout: 超时时间 + + Example: + # 访问 /dashboard，如果被重定向到 /login 说明未登录 + checker = LoginStateChecker.create_redirect_checker( + "https://example.com/dashboard", + "/login" + ) + """ + def indicator(resp: httpx.Response) -> bool: + # 如果没有被重定向到登录页，说明已登录 + return login_url_pattern not in str(resp.url) + + return cls(check_url, indicator, timeout) + + @classmethod + def create_status_code_checker( + cls, + check_url: str, + success_codes: list = None, + timeout: float = 10.0 + ) -> "LoginStateChecker": + """ + 创建状态码检测器 + + Args: + check_url: 检测 URL + success_codes: 成功的状态码列表，默认 [200] + timeout: 超时时间 + + Example: + # 检测 API 是否返回 200 + checker = LoginStateChecker.create_status_code_checker( + "https://example.com/api/user", + [200] + ) + """ + success_codes = success_codes or [200] + + def indicator(resp: httpx.Response) -> bool: + return resp.status_code in success_codes + + return cls(check_url, indicator, timeout) + + @classmethod + def create_content_checker( + cls, + check_url: str, + success_text: str, + timeout: float = 10.0 + ) -> "LoginStateChecker": + """ + 创建内容检测器 + + 检测响应内容是否包含指定文本 + + Args: + check_url: 检测 URL + success_text: 登录成功时页面包含的文本 + timeout: 超时时间 + + Example: + # 检测页面是否包含"欢迎回来" + checker = LoginStateChecker.create_content_checker( + "https://example.com/dashboard", + "欢迎回来" + ) + """ + def indicator(resp: httpx.Response) -> bool: + return success_text in resp.text + + return cls(check_url, indicator, timeout) + + +class CookieExpiryMonitor: + """Cookie 过期监控器""" + + @staticmethod + def check_expiry(cookies: list) -> tuple: + """ + 检查 Cookie 是否过期 + + Args: + cookies: Playwright 格式的 Cookie 列表 + + Returns: + (是否全部有效, 已过期的 Cookie 列表) + """ + now = time.time() + expired = [] + + for cookie in cookies: + expires = cookie.get("expires", 0) + # expires > 0 表示设置了过期时间 + # expires == -1 或 0 通常表示会话 Cookie + if expires > 0 and expires < now: + expired.append(cookie) + + return len(expired) == 0, expired + + @staticmethod + def get_earliest_expiry(cookies: list) -> Optional[datetime]: + """ + 获取最早过期的时间 + + Args: + cookies: Cookie 列表 + + Returns: + 最早过期的 datetime，如果没有过期时间则返回 None + """ + expiry_times = [] + for c in cookies: + expires = c.get("expires", 0) + if expires > 0: + expiry_times.append(expires) + + if not expiry_times: + return None + + return datetime.fromtimestamp(min(expiry_times)) + + @staticmethod + def will_expire_soon(cookies: list, threshold_hours: int = 1) -> bool: + """ + 检查 Cookie 是否即将过期 + + Args: + cookies: Cookie 列表 + threshold_hours: 阈值小时数 + + Returns: + 是否在阈值时间内过期 + """ + earliest = CookieExpiryMonitor.get_earliest_expiry(cookies) + if earliest is None: + return False # 没有过期时间的认为不会过期 + return earliest < datetime.now() + timedelta(hours=threshold_hours) + + @staticmethod + def get_expiry_summary(cookies: list) -> dict: + """ + 获取 Cookie 过期时间摘要 + + Args: + cookies: Cookie 列表 + + Returns: + 摘要信息字典 + """ + now = time.time() + summary = { + "total": len(cookies), + "session_cookies": 0, # 会话 Cookie（无过期时间） + "valid": 0, + "expired": 0, + "expiring_soon": 0, # 1小时内过期 + "earliest_expiry": None, + "latest_expiry": None + } + + expiry_times = [] + + for cookie in cookies: + expires = cookie.get("expires", 0) + if expires <= 0: + summary["session_cookies"] += 1 + elif expires < now: + summary["expired"] += 1 + elif expires < now + 3600: # 1小时内 + summary["expiring_soon"] += 1 + summary["valid"] += 1 + expiry_times.append(expires) + else: + summary["valid"] += 1 + expiry_times.append(expires) + + if expiry_times: + summary["earliest_expiry"] = datetime.fromtimestamp(min(expiry_times)).isoformat() + summary["latest_expiry"] = datetime.fromtimestamp(max(expiry_times)).isoformat() + + return summary + + +async def demo(): + """演示登录状态检测""" + + print("=" * 50) + print("登录状态检测器演示") + print("=" * 50) + + # 1. JSON 检测器 + print("\n1. JSON 字段检测器") + checker1 = LoginStateChecker.create_json_checker( + "https://httpbin.org/json", + "slideshow" + ) + # httpbin.org/json 总是返回包含 slideshow 的 JSON + result = await checker1.is_logged_in({}) + print(f" 检测结果: {'已登录' if result else '未登录'}") + + # 2. 状态码检测器 + print("\n2. 状态码检测器") + checker2 = LoginStateChecker.create_status_code_checker( + "https://httpbin.org/status/200", + [200] + ) + result = await checker2.is_logged_in({}) + print(f" 检测结果 (200): {'已登录' if result else '未登录'}") + + checker2_fail = LoginStateChecker.create_status_code_checker( + "https://httpbin.org/status/401", + [200] + ) + result = await checker2_fail.is_logged_in({}) + print(f" 检测结果 (401): {'已登录' if result else '未登录'}") + + # 3. 内容检测器 + print("\n3. 内容检测器") + checker3 = LoginStateChecker.create_content_checker( + "https://httpbin.org/html", + "Herman Melville" # httpbin.org/html 页面包含这个名字 + ) + result = await checker3.is_logged_in({}) + print(f" 检测结果: {'已登录' if result else '未登录'}") + + # 4. Cookie 过期监控 + print("\n4. Cookie 过期监控") + test_cookies = [ + {"name": "session", "value": "abc", "expires": time.time() + 3600}, # 1小时后过期 + {"name": "token", "value": "xyz", "expires": time.time() + 86400}, # 1天后过期 + {"name": "temp", "value": "123", "expires": 0}, # 会话 Cookie + ] + + is_valid, expired = CookieExpiryMonitor.check_expiry(test_cookies) + print(f" 全部有效: {is_valid}, 过期数量: {len(expired)}") + + will_expire = CookieExpiryMonitor.will_expire_soon(test_cookies, threshold_hours=2) + print(f" 2小时内过期: {will_expire}") + + summary = CookieExpiryMonitor.get_expiry_summary(test_cookies) + print(f" 摘要: {summary}") + + print("\n" + "=" * 50) + print("演示完成") + print("=" * 50) + + +if __name__ == "__main__": + logger.remove() + logger.add( + lambda msg: print(msg, end=""), + format="{time:HH:mm:ss} | {level: <8} | {message}", + level="DEBUG" + ) + + asyncio.run(demo()) diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/06_\347\231\273\345\275\225\350\256\244\350\257\201_Cookie\344\270\216Session\347\256\241\347\220\206/pyproject.toml" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/06_\347\231\273\345\275\225\350\256\244\350\257\201_Cookie\344\270\216Session\347\256\241\347\220\206/pyproject.toml" new file mode 100644 index 0000000..7f39f75 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/06_\347\231\273\345\275\225\350\256\244\350\257\201_Cookie\344\270\216Session\347\256\241\347\220\206/pyproject.toml" @@ -0,0 +1,22 @@ +[project] +name = "chapter06-cookie-session" +version = "0.1.0" +description = "第06章：登录认证 - Cookie与Session管理、登录状态检测、真实登录演示" +readme = "README.md" +requires-python = ">=3.11" +dependencies = [ + "httpx>=0.27.0", + "loguru>=0.7.0", +] + +[project.optional-dependencies] +encryption = [ + "cryptography>=42.0.0", # Cookie加密存储（可选） +] + +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[tool.uv] +dev-dependencies = [] diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/06_\347\231\273\345\275\225\350\256\244\350\257\201_Cookie\344\270\216Session\347\256\241\347\220\206/session_demo.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/06_\347\231\273\345\275\225\350\256\244\350\257\201_Cookie\344\270\216Session\347\256\241\347\220\206/session_demo.py" new file mode 100644 index 0000000..86cd8b9 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/06_\347\231\273\345\275\225\350\256\244\350\257\201_Cookie\344\270\216Session\347\256\241\347\220\206/session_demo.py" @@ -0,0 +1,468 @@ +# -*- coding: utf-8 -*- +# @Desc: 完整的会话管理演示 - 展示如何在爬虫中管理登录状态 + +import asyncio +import json +from typing import Optional +from pathlib import Path +import httpx +from loguru import logger + +from cookie_manager import CookieManager, CookieSerializer, CookieRotator +from login_state_checker import LoginStateChecker, CookieExpiryMonitor + + +class SessionCrawler: + """带登录状态管理的爬虫""" + + def __init__( + self, + base_url: str, + cookie_path: str, + check_endpoint: str = "/api/user/info" + ): + """ + Args: + base_url: 基础 URL + cookie_path: Cookie 存储路径 + check_endpoint: 登录状态检测端点 + """ + self.base_url = base_url.rstrip("/") + self.cookie_path = cookie_path + self.check_endpoint = check_endpoint + + # 初始化登录检测器 + self.checker = LoginStateChecker.create_json_checker( + check_url=f"{self.base_url}{check_endpoint}", + success_field="user" + ) + + # 初始化 Cookie 管理器 + self.cookie_manager = CookieManager( + storage_path=cookie_path, + login_checker=self.checker.is_logged_in + ) + + self._client: Optional[httpx.AsyncClient] = None + + async def __aenter__(self): + await self.start() + return self + + async def __aexit__(self, *args): + await self.close() + + async def start(self): + """启动爬虫""" + cookies = await self.cookie_manager.get_valid_cookies() + if not cookies: + logger.warning("无法获取有效的 Cookie，将以未登录状态运行") + cookies = {} + + self._client = httpx.AsyncClient( + base_url=self.base_url, + cookies=cookies, + timeout=30.0, + follow_redirects=True + ) + logger.info(f"爬虫启动成功，基础URL: {self.base_url}") + + async def close(self): + """关闭爬虫""" + if self._client: + await self._client.aclose() + await self.cookie_manager.save() + logger.info("爬虫已关闭") + + async def get(self, endpoint: str, **kwargs) -> httpx.Response: + """GET 请求""" + return await self._client.get(endpoint, **kwargs) + + async def post(self, endpoint: str, **kwargs) -> httpx.Response: + """POST 请求""" + return await self._client.post(endpoint, **kwargs) + + async def fetch_json(self, endpoint: str) -> dict: + """获取 JSON 数据""" + try: + resp = await self._client.get(endpoint) + resp.raise_for_status() + return resp.json() + except httpx.HTTPStatusError as e: + if e.response.status_code == 401: + logger.warning("登录状态失效，需要重新登录") + raise + + +async def demo_basic_session(): + """演示基本的会话管理""" + print("\n" + "=" * 50) + print("1. 基本会话管理演示") + print("=" * 50) + + # 使用 httpbin.org 演示 + async with httpx.AsyncClient() as client: + # 设置 Cookie + resp = await client.get( + "https://httpbin.org/cookies/set", + params={"session": "demo123", "user": "test"} + ) + print(f"设置 Cookie 后的响应: {resp.status_code}") + + # 查看当前 Cookie + resp = await client.get("https://httpbin.org/cookies") + cookies = resp.json().get("cookies", {}) + print(f"当前 Cookie: {cookies}") + + +async def demo_cookie_persistence(): + """演示 Cookie 持久化""" + print("\n" + "=" * 50) + print("2. Cookie 持久化演示") + print("=" * 50) + + # 创建测试 Cookie + test_cookies = [ + { + "name": "session_id", + "value": "abc123xyz", + "domain": "example.com", + "path": "/", + "secure": True, + "httpOnly": True + }, + { + "name": "user_token", + "value": "token_value_here", + "domain": "example.com", + "path": "/" + } + ] + + # 保存为 JSON + json_path = "data/demo_cookies.json" + Path("data").mkdir(exist_ok=True) + CookieSerializer.to_json(test_cookies, json_path) + print(f"已保存为 JSON: {json_path}") + + # 保存为 Netscape 格式 + netscape_path = "data/demo_cookies.txt" + CookieSerializer.to_netscape(test_cookies, netscape_path) + print(f"已保存为 Netscape: {netscape_path}") + + # 从 JSON 加载 + loaded = CookieSerializer.from_json(json_path) + print(f"从 JSON 加载了 {len(loaded)} 个 Cookie") + + # 转换为 httpx 格式 + httpx_cookies = CookieSerializer.to_dict(loaded) + print(f"httpx 格式: {httpx_cookies}") + + +async def demo_cookie_rotation(): + """演示多账号 Cookie 轮换""" + print("\n" + "=" * 50) + print("3. 多账号 Cookie 轮换演示") + print("=" * 50) + + rotator = CookieRotator(min_interval=1.0) + + # 添加多个账号 + rotator.add_account("user_001", {"session": "sess_001", "token": "tok_001"}) + rotator.add_account("user_002", {"session": "sess_002", "token": "tok_002"}) + rotator.add_account("user_003", {"session": "sess_003", "token": "tok_003"}) + + print(f"添加了 {rotator.total_count} 个账号") + + # 模拟多次请求 + print("\n模拟请求（负载均衡）:") + for i in range(6): + cookies = await rotator.get_cookies() + if cookies: + print(f" 请求 {i+1}: session={cookies.get('session')}") + await asyncio.sleep(0.3) + + # 标记一个账号失效 + rotator.mark_invalid("user_002") + print(f"\n标记 user_002 失效后，有效账号: {rotator.valid_count}") + + # 继续请求 + print("\n继续请求（排除失效账号）:") + for i in range(3): + cookies = await rotator.get_cookies() + if cookies: + print(f" 请求 {i+1}: session={cookies.get('session')}") + await asyncio.sleep(0.3) + + # 获取统计 + stats = rotator.get_stats() + print(f"\n统计信息:") + for acc in stats["accounts"]: + print(f" {acc['id']}: 有效={acc['valid']}, 使用次数={acc['use_count']}") + + +async def demo_login_detection(): + """演示登录状态检测""" + print("\n" + "=" * 50) + print("4. 登录状态检测演示") + print("=" * 50) + + # 使用不同的检测方式 + + # 1. JSON 字段检测 + print("\n1) JSON 字段检测:") + checker1 = LoginStateChecker.create_json_checker( + "https://httpbin.org/json", + "slideshow" + ) + result = await checker1.is_logged_in({}) + print(f" 结果: {'通过' if result else '失败'}") + + # 2. 状态码检测 + print("\n2) 状态码检测:") + checker2 = LoginStateChecker.create_status_code_checker( + "https://httpbin.org/status/200" + ) + result = await checker2.is_logged_in({}) + print(f" 200 状态: {'通过' if result else '失败'}") + + checker2_fail = LoginStateChecker.create_status_code_checker( + "https://httpbin.org/status/401" + ) + result = await checker2_fail.is_logged_in({}) + print(f" 401 状态: {'通过' if result else '失败'}") + + # 3. 内容检测 + print("\n3) 内容检测:") + checker3 = LoginStateChecker.create_content_checker( + "https://httpbin.org/html", + "Herman Melville" + ) + result = await checker3.is_logged_in({}) + print(f" 包含指定文本: {'通过' if result else '失败'}") + + +async def demo_expiry_monitoring(): + """演示 Cookie 过期监控""" + print("\n" + "=" * 50) + print("5. Cookie 过期监控演示") + print("=" * 50) + + import time + + # 创建不同过期时间的 Cookie + test_cookies = [ + {"name": "expired", "value": "1", "expires": time.time() - 3600}, # 已过期 + {"name": "expiring", "value": "2", "expires": time.time() + 1800}, # 30分钟后过期 + {"name": "valid", "value": "3", "expires": time.time() + 86400}, # 1天后过期 + {"name": "session", "value": "4", "expires": 0}, # 会话 Cookie + ] + + # 检查过期 + is_valid, expired = CookieExpiryMonitor.check_expiry(test_cookies) + print(f"全部有效: {is_valid}") + print(f"已过期 Cookie: {[c['name'] for c in expired]}") + + # 是否即将过期 + will_expire = CookieExpiryMonitor.will_expire_soon(test_cookies, threshold_hours=1) + print(f"1小时内过期: {will_expire}") + + # 获取摘要 + summary = CookieExpiryMonitor.get_expiry_summary(test_cookies) + print(f"\nCookie 摘要:") + print(f" 总数: {summary['total']}") + print(f" 会话 Cookie: {summary['session_cookies']}") + print(f" 有效: {summary['valid']}") + print(f" 已过期: {summary['expired']}") + print(f" 即将过期: {summary['expiring_soon']}") + + +async def demo_real_login(): + """演示真实网站登录流程 - quotes.toscrape.com""" + print("\n" + "=" * 50) + print("6. 真实网站登录演示 (quotes.toscrape.com)") + print("=" * 50) + + login_url = "https://quotes.toscrape.com/login" + home_url = "https://quotes.toscrape.com/" + + async with httpx.AsyncClient(follow_redirects=True) as client: + # 1. 访问登录页面 + print(f"\n1) 访问登录页面: {login_url}") + resp = await client.get(login_url) + print(f" 状态码: {resp.status_code}") + + # 2. 提交登录表单（quotes.toscrape.com 接受任意用户名密码） + print("\n2) 提交登录表单...") + login_data = { + "username": "admin", + "password": "admin" + } + resp = await client.post(login_url, data=login_data) + print(f" 状态码: {resp.status_code}") + + # 3. 检查登录状态 + print("\n3) 检查登录状态...") + if "Logout" in resp.text: + print(" ✅ 登录成功！") + + # 4. 获取并显示 Cookie + print("\n4) 获取到的 Cookie:") + cookies_dict = dict(client.cookies) + for name, value in cookies_dict.items(): + print(f" {name}: {value}") + + # 5. 保存 Cookie 到文件 + print("\n5) 保存 Cookie 到文件...") + Path("data").mkdir(exist_ok=True) + cookie_path = "data/quotes_cookies.json" + + # 转换为标准格式 + cookies_list = [ + { + "name": name, + "value": value, + "domain": "quotes.toscrape.com", + "path": "/" + } + for name, value in cookies_dict.items() + ] + + with open(cookie_path, "w") as f: + json.dump(cookies_list, f, indent=2) + print(f" Cookie 已保存到: {cookie_path}") + + # 6. 测试 Cookie 复用 + print("\n6) 测试 Cookie 复用...") + async with httpx.AsyncClient() as new_client: + # 加载保存的 Cookie + with open(cookie_path, "r") as f: + loaded_cookies = json.load(f) + + # 转换为 httpx 格式并注入 + cookies_dict = {c["name"]: c["value"] for c in loaded_cookies} + new_client.cookies.update(cookies_dict) + + # 访问主页 + resp = await new_client.get(home_url) + if "Logout" in resp.text: + print(" ✅ Cookie 复用成功！保持登录状态") + else: + print(" ❌ Cookie 复用失败") + + # 7. 使用 CookieManager 管理 + print("\n7) 使用 CookieManager 管理...") + + async def check_quotes_login(cookies: dict) -> bool: + """检查 quotes.toscrape.com 登录状态""" + try: + async with httpx.AsyncClient(cookies=cookies, timeout=10) as c: + resp = await c.get(home_url) + return "Logout" in resp.text + except Exception: + return False + + manager = CookieManager( + storage_path="data/quotes_managed_cookies.json", + login_checker=check_quotes_login + ) + + # 保存 Cookie + manager.update(cookies_list) + await manager.save() + print(" Cookie 已通过 CookieManager 保存") + + # 验证并获取有效 Cookie + valid_cookies = await manager.get_valid_cookies() + if valid_cookies: + print(f" ✅ CookieManager 验证通过，获取到 {len(valid_cookies)} 个有效 Cookie") + else: + print(" ❌ CookieManager 验证失败") + + else: + print(" ❌ 登录失败") + + print("\n真实登录演示完成") + + +async def demo_complete_workflow(): + """演示完整的工作流程""" + print("\n" + "=" * 50) + print("7. 完整工作流程演示") + print("=" * 50) + + # 模拟一个完整的登录和爬取流程 + + # 1. 定义检测函数 + async def check_login(cookies: dict) -> bool: + try: + async with httpx.AsyncClient(cookies=cookies) as client: + resp = await client.get("https://httpbin.org/cookies", timeout=10) + data = resp.json() + return "session" in data.get("cookies", {}) + except Exception: + return False + + # 2. 创建 Cookie 管理器 + manager = CookieManager( + storage_path="data/workflow_cookies.json", + login_checker=check_login, + check_interval=60 + ) + + # 3. 模拟已有 Cookie（实际场景中可能是从浏览器提取的） + existing_cookies = [ + { + "name": "session", + "value": "workflow_session_123", + "domain": "httpbin.org", + "path": "/" + } + ] + manager.update(existing_cookies) + await manager.save() + print("已保存初始 Cookie") + + # 4. 获取有效 Cookie 并使用 + cookies = await manager.get_valid_cookies() + if cookies: + print(f"获取到有效 Cookie: {cookies}") + + # 使用 Cookie 发起请求 + async with httpx.AsyncClient(cookies=cookies) as client: + resp = await client.get("https://httpbin.org/cookies") + print(f"请求结果: {resp.json()}") + else: + print("Cookie 无效或不存在") + + print("\n工作流程演示完成") + + +async def main(): + """主函数""" + logger.remove() + logger.add( + lambda msg: print(msg, end=""), + format="{time:HH:mm:ss} | {level: <8} | {message}", + level="INFO" + ) + + print("=" * 50) + print("Cookie 与 Session 管理演示") + print("=" * 50) + + await demo_basic_session() + await demo_cookie_persistence() + await demo_cookie_rotation() + await demo_login_detection() + await demo_expiry_monitoring() + await demo_real_login() # 新增：真实登录演示 + await demo_complete_workflow() + + print("\n" + "=" * 50) + print("所有演示完成") + print("=" * 50) + + +if __name__ == "__main__": + asyncio.run(main()) diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/07_\347\231\273\345\275\225\350\256\244\350\257\201_\346\211\253\347\240\201\344\270\216\347\237\255\344\277\241\347\231\273\345\275\225\345\256\236\347\216\260/README.md" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/07_\347\231\273\345\275\225\350\256\244\350\257\201_\346\211\253\347\240\201\344\270\216\347\237\255\344\277\241\347\231\273\345\275\225\345\256\236\347\216\260/README.md" new file mode 100644 index 0000000..053c2d0 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/07_\347\231\273\345\275\225\350\256\244\350\257\201_\346\211\253\347\240\201\344\270\216\347\237\255\344\277\241\347\231\273\345\275\225\345\256\236\347\216\260/README.md" @@ -0,0 +1,37 @@ +# 第07章：登录认证 - 扫码与短信登录实现 + +展示扫码登录和短信登录的技术框架和实现思路。 + +## 快速开始 + +```bash +cd 07_登录认证_扫码与短信登录实现 + +# 安装基础依赖 +uv sync + +# 安装二维码功能（可选） +uv sync --extra qrcode + +# 安装浏览器 +uv run playwright install chromium + +# 运行示例 +uv run python qrcode_login.py +uv run python sms_login.py +uv run python login_factory.py +``` + +### 重要说明 + +⚠️ 本章代码提供的是**技术框架**而非特定网站实战，因为没有公开的练习网站提供真实扫码/短信登录功能。 + +代码展示了： +- 扫码登录的技术原理和流程 +- 短信登录的实现模式 +- 登录工厂模式设计 + +实际使用时需要根据目标网站适配具体的： +- 二维码选择器 +- 状态轮询机制 +- Cookie获取方式 diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/07_\347\231\273\345\275\225\350\256\244\350\257\201_\346\211\253\347\240\201\344\270\216\347\237\255\344\277\241\347\231\273\345\275\225\345\256\236\347\216\260/bilibili_qrcode_login.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/07_\347\231\273\345\275\225\350\256\244\350\257\201_\346\211\253\347\240\201\344\270\216\347\237\255\344\277\241\347\231\273\345\275\225\345\256\236\347\216\260/bilibili_qrcode_login.py" new file mode 100644 index 0000000..b1d45fa --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/07_\347\231\273\345\275\225\350\256\244\350\257\201_\346\211\253\347\240\201\344\270\216\347\237\255\344\277\241\347\231\273\345\275\225\345\256\236\347\216\260/bilibili_qrcode_login.py" @@ -0,0 +1,470 @@ +# -*- coding: utf-8 -*- +""" +B站扫码登录实战 + +本模块展示B站扫码登录的完整实现，包括： +- B站扫码API调用（二维码生成、状态轮询） +- 二维码图片生成与终端显示 +- 登录状态监控与回调 +- Cookie提取与保存 + +这是第07章"登录认证-扫码与短信登录实现"的B站实战示例。 + +与第11章综合实战项目的关联： +- login/auth.py: 登录模块实现 +- config/bilibili_config.py: 登录相关常量定义 +""" + +import asyncio +import json +from io import BytesIO +from pathlib import Path +from dataclasses import dataclass, field +from typing import Optional, Callable, Awaitable, Dict +from enum import IntEnum + +import httpx +from loguru import logger + +# 可选依赖：二维码生成 +try: + import qrcode + HAS_QRCODE = True +except ImportError: + HAS_QRCODE = False + logger.warning("未安装 qrcode 库，终端二维码显示功能不可用。安装: pip install qrcode") + + +# ============== B站扫码状态枚举 ============== + +class BilibiliQRStatus(IntEnum): + """ + B站扫码状态码 + + | 状态码 | 含义 | + |-------|------| + | 0 | 登录成功 | + | 86101 | 未扫描 | + | 86090 | 已扫描，待确认 | + | 86038 | 已过期 | + """ + SUCCESS = 0 # 登录成功 + NOT_SCANNED = 86101 # 未扫描 + SCANNED = 86090 # 已扫描，待确认 + EXPIRED = 86038 # 已过期 + + +# ============== B站 Cookie 数据类 ============== + +@dataclass +class BilibiliLoginCookies: + """ + B站登录 Cookie 数据类 + + 扫码登录成功后提取的Cookie + """ + sessdata: str + dede_user_id: str + bili_jct: str + buvid3: str = "" + buvid4: str = "" + sid: str = "" + + def to_dict(self) -> Dict[str, str]: + """转换为 httpx 可用的字典格式""" + cookies = { + "SESSDATA": self.sessdata, + "DedeUserID": self.dede_user_id, + "bili_jct": self.bili_jct, + } + if self.buvid3: + cookies["buvid3"] = self.buvid3 + if self.buvid4: + cookies["buvid4"] = self.buvid4 + if self.sid: + cookies["sid"] = self.sid + return cookies + + def to_header_string(self) -> str: + """转换为请求头 Cookie 格式""" + return "; ".join(f"{k}={v}" for k, v in self.to_dict().items()) + + def is_valid(self) -> bool: + """检查核心 Cookie 是否存在""" + return bool(self.sessdata and self.dede_user_id and self.bili_jct) + + +# ============== B站扫码登录实现 ============== + +class BilibiliQRCodeLogin: + """ + B站扫码登录实现 + + 流程： + 1. 调用 /qrcode/generate 获取二维码URL和qrcode_key + 2. 生成二维码图片（保存到文件或终端显示） + 3. 轮询 /qrcode/poll 检查扫码状态 + 4. 登录成功后提取Cookie + + 使用示例： + async with BilibiliQRCodeLogin() as login: + cookies = await login.login() + if cookies: + print(f"登录成功: {cookies.dede_user_id}") + """ + + # B站登录相关API + QRCODE_GENERATE_URL = "https://passport.bilibili.com/x/passport-login/web/qrcode/generate" + QRCODE_POLL_URL = "https://passport.bilibili.com/x/passport-login/web/qrcode/poll" + + # 请求头 + HEADERS = { + "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) " + "AppleWebKit/537.36 (KHTML, like Gecko) " + "Chrome/131.0.0.0 Safari/537.36", + "Referer": "https://www.bilibili.com/", + "Origin": "https://www.bilibili.com" + } + + def __init__( + self, + timeout: int = 180, + poll_interval: float = 2.0, + on_status_change: Optional[Callable[[int, str], Awaitable[None]]] = None + ): + """ + 初始化扫码登录 + + Args: + timeout: 登录超时时间（秒） + poll_interval: 状态轮询间隔（秒） + on_status_change: 状态变化回调 (status_code, message) + """ + self.timeout = timeout + self.poll_interval = poll_interval + self.on_status_change = on_status_change + + self._client: Optional[httpx.AsyncClient] = None + self._qrcode_key: str = "" + self._qrcode_url: str = "" + self._current_status: int = -1 + + async def __aenter__(self): + """异步上下文管理器入口""" + self._client = httpx.AsyncClient( + headers=self.HEADERS, + timeout=30 + ) + return self + + async def __aexit__(self, exc_type, exc_val, exc_tb): + """异步上下文管理器出口""" + if self._client: + await self._client.aclose() + + async def _notify_status(self, code: int, message: str): + """通知状态变化""" + if code != self._current_status: + self._current_status = code + logger.info(f"B站登录状态: {message} ({code})") + if self.on_status_change: + await self.on_status_change(code, message) + + async def generate_qrcode(self) -> tuple: + """ + 生成登录二维码 + + Returns: + (qrcode_url, qrcode_image_bytes) 或 (qrcode_url, None) 如果无法生成图片 + """ + resp = await self._client.get(self.QRCODE_GENERATE_URL) + data = resp.json() + + if data["code"] != 0: + raise Exception(f"获取二维码失败: {data['message']}") + + self._qrcode_key = data["data"]["qrcode_key"] + self._qrcode_url = data["data"]["url"] + + logger.info(f"二维码生成成功，qrcode_key: {self._qrcode_key[:20]}...") + + # 生成二维码图片 + image_bytes = None + if HAS_QRCODE: + qr = qrcode.QRCode( + version=1, + error_correction=qrcode.constants.ERROR_CORRECT_L, + box_size=10, + border=2 + ) + qr.add_data(self._qrcode_url) + qr.make(fit=True) + + img = qr.make_image(fill_color="black", back_color="white") + buffer = BytesIO() + img.save(buffer, format="PNG") + image_bytes = buffer.getvalue() + + return self._qrcode_url, image_bytes + + def print_qrcode_to_terminal(self, url: Optional[str] = None): + """在终端打印二维码""" + if not HAS_QRCODE: + logger.warning("需要安装 qrcode 库才能在终端显示二维码: pip install qrcode") + return + + url = url or self._qrcode_url + if not url: + logger.warning("没有可用的二维码URL") + return + + qr = qrcode.QRCode( + version=1, + error_correction=qrcode.constants.ERROR_CORRECT_L, + box_size=1, + border=1 + ) + qr.add_data(url) + qr.make(fit=True) + qr.print_ascii(invert=True) + + async def poll_status(self) -> tuple: + """ + 轮询登录状态 + + Returns: + (status_code, cookies_if_success) + """ + if not self._qrcode_key: + raise RuntimeError("请先调用 generate_qrcode() 生成二维码") + + resp = await self._client.get( + self.QRCODE_POLL_URL, + params={"qrcode_key": self._qrcode_key} + ) + data = resp.json() + + code = data["data"]["code"] + message = data["data"]["message"] + + await self._notify_status(code, message) + + if code == BilibiliQRStatus.SUCCESS: + # 登录成功，从响应中提取Cookie + cookies = self._extract_cookies(resp) + return code, cookies + + return code, None + + def _extract_cookies(self, resp: httpx.Response) -> BilibiliLoginCookies: + """从响应中提取B站Cookie""" + cookies = resp.cookies + + return BilibiliLoginCookies( + sessdata=cookies.get("SESSDATA", ""), + dede_user_id=cookies.get("DedeUserID", ""), + bili_jct=cookies.get("bili_jct", ""), + buvid3=cookies.get("buvid3", ""), + buvid4=cookies.get("buvid4", ""), + sid=cookies.get("sid", ""), + ) + + async def login( + self, + save_qrcode_path: str = "bilibili_qrcode.png", + show_in_terminal: bool = True + ) -> Optional[BilibiliLoginCookies]: + """ + 执行完整的扫码登录流程 + + Args: + save_qrcode_path: 二维码图片保存路径 + show_in_terminal: 是否在终端显示二维码 + + Returns: + 登录成功返回Cookie，失败返回None + """ + # 1. 生成二维码 + url, image_bytes = await self.generate_qrcode() + + # 保存二维码图片 + if image_bytes: + with open(save_qrcode_path, "wb") as f: + f.write(image_bytes) + logger.info(f"二维码已保存至: {save_qrcode_path}") + + # 在终端显示 + if show_in_terminal: + print("\n" + "=" * 50) + print("请使用B站APP扫描以下二维码登录") + print("=" * 50 + "\n") + + if HAS_QRCODE: + self.print_qrcode_to_terminal(url) + else: + print(f"二维码URL: {url}") + + if image_bytes: + print(f"\n二维码图片也已保存至: {save_qrcode_path}\n") + + # 2. 轮询登录状态 + start_time = asyncio.get_event_loop().time() + + while True: + elapsed = asyncio.get_event_loop().time() - start_time + if elapsed > self.timeout: + logger.warning("登录超时") + return None + + code, cookies = await self.poll_status() + + if code == BilibiliQRStatus.SUCCESS: + logger.info("B站登录成功！") + return cookies + + if code == BilibiliQRStatus.EXPIRED: + logger.warning("二维码已过期") + return None + + await asyncio.sleep(self.poll_interval) + + +# ============== Cookie 验证工具 ============== + +async def verify_bilibili_cookies(cookies: Dict[str, str]) -> Optional[dict]: + """ + 验证B站Cookie是否有效 + + Args: + cookies: Cookie字典 + + Returns: + 有效返回用户信息，无效返回None + """ + async with httpx.AsyncClient( + cookies=cookies, + headers={ + "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) " + "AppleWebKit/537.36 (KHTML, like Gecko) " + "Chrome/131.0.0.0 Safari/537.36", + "Referer": "https://www.bilibili.com/", + }, + timeout=10 + ) as client: + resp = await client.get("https://api.bilibili.com/x/web-interface/nav") + data = resp.json() + + if data["code"] == 0 and data["data"].get("isLogin"): + return { + "mid": data["data"]["mid"], + "uname": data["data"]["uname"], + "level": data["data"]["level_info"]["current_level"], + "vip_type": data["data"]["vipType"], + "money": data["data"]["money"], + } + + return None + + +# ============== 演示入口 ============== + +async def demo_bilibili_qrcode_login(): + """演示B站扫码登录""" + logger.info("=" * 50) + logger.info("B站扫码登录示例") + logger.info("=" * 50) + + # 定义状态变化回调 + async def on_status(code: int, message: str): + status_emoji = { + BilibiliQRStatus.NOT_SCANNED: "...", + BilibiliQRStatus.SCANNED: "[已扫描]", + BilibiliQRStatus.SUCCESS: "[成功]", + BilibiliQRStatus.EXPIRED: "[过期]", + } + emoji = status_emoji.get(code, "?") + print(f" {emoji} {message}") + + # 执行扫码登录 + async with BilibiliQRCodeLogin( + timeout=180, + poll_interval=2.0, + on_status_change=on_status + ) as login: + cookies = await login.login( + save_qrcode_path="bilibili_qrcode.png", + show_in_terminal=True + ) + + if cookies: + # 保存Cookie + cookie_path = Path("data/bilibili_login_cookies.json") + cookie_path.parent.mkdir(parents=True, exist_ok=True) + + with open(cookie_path, "w", encoding="utf-8") as f: + json.dump(cookies.to_dict(), f, indent=2, ensure_ascii=False) + + print("\n" + "=" * 50) + print("登录成功！") + print("=" * 50) + print(f"Cookie已保存至: {cookie_path}") + print(f"SESSDATA: {cookies.sessdata[:20]}...") + print(f"DedeUserID: {cookies.dede_user_id}") + print(f"bili_jct: {cookies.bili_jct[:20]}...") + + # 验证Cookie + user_info = await verify_bilibili_cookies(cookies.to_dict()) + if user_info: + print(f"\n用户信息:") + print(f" 用户名: {user_info['uname']}") + print(f" 等级: LV{user_info['level']}") + print(f" 硬币: {user_info['money']}") + else: + print("\n登录失败或超时") + + +async def demo_verify_existing_cookies(): + """演示验证已保存的Cookie""" + logger.info("=" * 50) + logger.info("验证已保存的B站Cookie") + logger.info("=" * 50) + + cookie_path = Path("data/bilibili_login_cookies.json") + + if not cookie_path.exists(): + logger.info("没有找到已保存的Cookie，请先执行扫码登录") + return + + with open(cookie_path, "r", encoding="utf-8") as f: + cookies = json.load(f) + + user_info = await verify_bilibili_cookies(cookies) + + if user_info: + print(f"Cookie有效！") + print(f" 用户名: {user_info['uname']}") + print(f" 用户ID: {user_info['mid']}") + print(f" 等级: LV{user_info['level']}") + else: + print("Cookie已失效，请重新登录") + + +async def main(): + """主演示函数""" + logger.info("\n" + "=" * 60) + logger.info("B站扫码登录实战示例") + logger.info("=" * 60) + + # 检查是否有已保存的Cookie + cookie_path = Path("data/bilibili_login_cookies.json") + if cookie_path.exists(): + logger.info("\n检测到已保存的Cookie，正在验证...") + await demo_verify_existing_cookies() + print("\n如需重新登录，请删除 data/bilibili_login_cookies.json 后重试") + else: + logger.info("\n开始扫码登录流程...") + await demo_bilibili_qrcode_login() + + +if __name__ == "__main__": + asyncio.run(main()) diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/07_\347\231\273\345\275\225\350\256\244\350\257\201_\346\211\253\347\240\201\344\270\216\347\237\255\344\277\241\347\231\273\345\275\225\345\256\236\347\216\260/login_factory.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/07_\347\231\273\345\275\225\350\256\244\350\257\201_\346\211\253\347\240\201\344\270\216\347\237\255\344\277\241\347\231\273\345\275\225\345\256\236\347\216\260/login_factory.py" new file mode 100644 index 0000000..5818aa2 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/07_\347\231\273\345\275\225\350\256\244\350\257\201_\346\211\253\347\240\201\344\270\216\347\237\255\344\277\241\347\231\273\345\275\225\345\256\236\347\216\260/login_factory.py" @@ -0,0 +1,476 @@ +# -*- coding: utf-8 -*- +# @Desc: 登录模块工厂 - 统一封装多种登录方式 + +import asyncio +import json +from abc import ABC, abstractmethod +from pathlib import Path +from typing import Optional, Callable, Awaitable, Dict, Any +from enum import Enum +from dataclasses import dataclass +from loguru import logger + + +class LoginMethod(Enum): + """登录方式枚举""" + QRCODE = "qrcode" # 扫码登录 + SMS = "sms" # 短信验证码 + PASSWORD = "password" # 账号密码 + COOKIE = "cookie" # Cookie 注入 + + +@dataclass +class LoginResult: + """登录结果""" + success: bool + cookies: list = None + error: str = None + method: LoginMethod = None + + def __post_init__(self): + if self.cookies is None: + self.cookies = [] + + +class BaseLogin(ABC): + """登录基类""" + + @property + @abstractmethod + def method(self) -> LoginMethod: + """登录方式""" + pass + + @abstractmethod + async def login(self) -> LoginResult: + """ + 执行登录 + + Returns: + 登录结果 + """ + pass + + @abstractmethod + async def close(self): + """清理资源""" + pass + + +class CookieLogin(BaseLogin): + """Cookie 注入登录""" + + def __init__(self, cookies: list, verify_url: str = None): + """ + Args: + cookies: Cookie 列表 + verify_url: 验证 URL（可选） + """ + self._cookies = cookies + self._verify_url = verify_url + + @property + def method(self) -> LoginMethod: + return LoginMethod.COOKIE + + async def login(self) -> LoginResult: + """Cookie 注入直接返回成功""" + logger.info(f"使用 Cookie 注入登录，Cookie 数量: {len(self._cookies)}") + return LoginResult( + success=True, + cookies=self._cookies, + method=self.method + ) + + async def close(self): + pass + + +class QRCodeLoginWrapper(BaseLogin): + """扫码登录包装器""" + + def __init__( + self, + login_url: str, + qrcode_selector: str, + success_url_pattern: str, + timeout: int = 120, + on_qrcode_ready: Callable[[str], Awaitable[None]] = None, + headless: bool = False + ): + self.login_url = login_url + self.qrcode_selector = qrcode_selector + self.success_url_pattern = success_url_pattern + self.timeout = timeout + self.on_qrcode_ready = on_qrcode_ready + self.headless = headless + + self._playwright = None + self._qrcode_login = None + + @property + def method(self) -> LoginMethod: + return LoginMethod.QRCODE + + async def login(self) -> LoginResult: + from playwright.async_api import async_playwright + from qrcode_login import QRCodeLogin + + try: + self._playwright = await async_playwright().start() + self._qrcode_login = QRCodeLogin( + login_url=self.login_url, + qrcode_selector=self.qrcode_selector, + success_url_pattern=self.success_url_pattern, + timeout=self.timeout + ) + + await self._qrcode_login.start(self._playwright, headless=self.headless) + + cookies = await self._qrcode_login.login( + on_qrcode_ready=self.on_qrcode_ready + ) + + if cookies: + return LoginResult( + success=True, + cookies=cookies, + method=self.method + ) + + return LoginResult( + success=False, + error="登录超时或用户取消", + method=self.method + ) + + except Exception as e: + logger.error(f"扫码登录异常: {e}") + return LoginResult( + success=False, + error=str(e), + method=self.method + ) + + async def close(self): + if self._qrcode_login: + await self._qrcode_login.close() + if self._playwright: + await self._playwright.stop() + + +class SMSLoginWrapper(BaseLogin): + """短信登录包装器""" + + def __init__( + self, + login_url: str, + phone_input_selector: str, + send_code_btn_selector: str, + code_input_selector: str, + submit_btn_selector: str, + success_url_pattern: str, + phone: str, + get_code_callback: Callable[[], Awaitable[str]], + headless: bool = False + ): + self.login_url = login_url + self.phone_input_selector = phone_input_selector + self.send_code_btn_selector = send_code_btn_selector + self.code_input_selector = code_input_selector + self.submit_btn_selector = submit_btn_selector + self.success_url_pattern = success_url_pattern + self.phone = phone + self.get_code_callback = get_code_callback + self.headless = headless + + self._playwright = None + self._sms_login = None + + @property + def method(self) -> LoginMethod: + return LoginMethod.SMS + + async def login(self) -> LoginResult: + from playwright.async_api import async_playwright + from sms_login import SMSLogin + + try: + self._playwright = await async_playwright().start() + self._sms_login = SMSLogin( + login_url=self.login_url, + phone_input_selector=self.phone_input_selector, + send_code_btn_selector=self.send_code_btn_selector, + code_input_selector=self.code_input_selector, + submit_btn_selector=self.submit_btn_selector, + success_url_pattern=self.success_url_pattern + ) + + await self._sms_login.start(self._playwright, headless=self.headless) + + cookies = await self._sms_login.login_with_manual_code( + phone=self.phone, + get_code_callback=self.get_code_callback + ) + + if cookies: + return LoginResult( + success=True, + cookies=cookies, + method=self.method + ) + + return LoginResult( + success=False, + error="登录失败", + method=self.method + ) + + except Exception as e: + logger.error(f"短信登录异常: {e}") + return LoginResult( + success=False, + error=str(e), + method=self.method + ) + + async def close(self): + if self._sms_login: + await self._sms_login.close() + if self._playwright: + await self._playwright.stop() + + +class LoginFactory: + """登录工厂""" + + @staticmethod + def create_cookie_login(cookies: list, **kwargs) -> BaseLogin: + """创建 Cookie 登录""" + return CookieLogin(cookies, **kwargs) + + @staticmethod + def create_qrcode_login( + login_url: str, + qrcode_selector: str, + success_url_pattern: str, + **kwargs + ) -> BaseLogin: + """创建扫码登录""" + return QRCodeLoginWrapper( + login_url=login_url, + qrcode_selector=qrcode_selector, + success_url_pattern=success_url_pattern, + **kwargs + ) + + @staticmethod + def create_sms_login( + login_url: str, + phone: str, + get_code_callback: Callable[[], Awaitable[str]], + phone_input_selector: str, + send_code_btn_selector: str, + code_input_selector: str, + submit_btn_selector: str, + success_url_pattern: str, + **kwargs + ) -> BaseLogin: + """创建短信登录""" + return SMSLoginWrapper( + login_url=login_url, + phone=phone, + get_code_callback=get_code_callback, + phone_input_selector=phone_input_selector, + send_code_btn_selector=send_code_btn_selector, + code_input_selector=code_input_selector, + submit_btn_selector=submit_btn_selector, + success_url_pattern=success_url_pattern, + **kwargs + ) + + +class LoginManager: + """统一登录管理器""" + + def __init__( + self, + platform: str, + cookie_path: str, + default_method: LoginMethod = LoginMethod.COOKIE + ): + """ + Args: + platform: 平台名称 + cookie_path: Cookie 存储路径 + default_method: 默认登录方式 + """ + self.platform = platform + self.cookie_path = Path(cookie_path) + self.default_method = default_method + self._cookies: list = [] + + async def ensure_login( + self, + login_config: Dict[str, Any] = None, + force_login: bool = False + ) -> bool: + """ + 确保已登录 + + 优先使用已保存的 Cookie，如果无效则使用指定方式登录 + + Args: + login_config: 登录配置 + force_login: 是否强制重新登录 + + Returns: + 是否登录成功 + """ + # 如果不强制登录，先尝试加载已保存的 Cookie + if not force_login and await self._try_load_cookies(): + logger.info(f"[{self.platform}] 使用已保存的 Cookie") + return True + + # 执行登录 + if not login_config: + logger.error(f"[{self.platform}] 需要登录配置") + return False + + method = login_config.pop('method', self.default_method) + logger.info(f"[{self.platform}] 开始 {method.value} 登录") + + # 创建登录实例 + login = self._create_login(method, login_config) + if not login: + logger.error(f"[{self.platform}] 不支持的登录方式: {method}") + return False + + try: + result = await login.login() + + if result.success: + self._cookies = result.cookies + await self._save_cookies() + logger.info(f"[{self.platform}] 登录成功") + return True + else: + logger.error(f"[{self.platform}] 登录失败: {result.error}") + return False + + finally: + await login.close() + + def _create_login( + self, + method: LoginMethod, + config: Dict[str, Any] + ) -> Optional[BaseLogin]: + """根据方式创建登录实例""" + if method == LoginMethod.COOKIE: + return LoginFactory.create_cookie_login(**config) + elif method == LoginMethod.QRCODE: + return LoginFactory.create_qrcode_login(**config) + elif method == LoginMethod.SMS: + return LoginFactory.create_sms_login(**config) + return None + + async def _try_load_cookies(self) -> bool: + """尝试加载 Cookie""" + if not self.cookie_path.exists(): + return False + + try: + with open(self.cookie_path, "r", encoding="utf-8") as f: + self._cookies = json.load(f) + + # 简单验证：检查是否有 Cookie + return len(self._cookies) > 0 + + except Exception as e: + logger.warning(f"加载 Cookie 失败: {e}") + return False + + async def _save_cookies(self): + """保存 Cookie""" + self.cookie_path.parent.mkdir(parents=True, exist_ok=True) + with open(self.cookie_path, "w", encoding="utf-8") as f: + json.dump(self._cookies, f, indent=2, ensure_ascii=False) + logger.info(f"Cookie 已保存: {self.cookie_path}") + + def get_cookies(self) -> list: + """获取 Cookie 列表""" + return self._cookies + + def get_cookies_dict(self) -> dict: + """获取字典格式的 Cookie""" + return {c["name"]: c["value"] for c in self._cookies} + + async def clear_cookies(self): + """清除 Cookie""" + self._cookies = [] + if self.cookie_path.exists(): + self.cookie_path.unlink() + logger.info(f"[{self.platform}] Cookie 已清除") + + +async def demo_login_factory(): + """登录工厂演示""" + print("=" * 50) + print("登录工厂演示") + print("=" * 50) + + # 1. Cookie 登录演示 + print("\n1. Cookie 登录演示:") + test_cookies = [ + {"name": "session", "value": "test123", "domain": "example.com", "path": "/"} + ] + cookie_login = LoginFactory.create_cookie_login(cookies=test_cookies) + result = await cookie_login.login() + print(f" 登录方式: {result.method.value}") + print(f" 登录结果: {'成功' if result.success else '失败'}") + print(f" Cookie 数量: {len(result.cookies)}") + await cookie_login.close() + + # 2. 登录管理器演示 + print("\n2. 登录管理器演示:") + manager = LoginManager( + platform="demo_platform", + cookie_path="data/demo_login_cookies.json", + default_method=LoginMethod.COOKIE + ) + + # 使用 Cookie 配置 + success = await manager.ensure_login( + login_config={ + 'method': LoginMethod.COOKIE, + 'cookies': test_cookies + } + ) + print(f" 登录结果: {'成功' if success else '失败'}") + print(f" Cookie: {manager.get_cookies_dict()}") + + # 3. 再次调用会使用缓存的 Cookie + print("\n3. 使用缓存的 Cookie:") + success = await manager.ensure_login() + print(f" 登录结果: {'成功' if success else '失败'}") + + print("\n" + "=" * 50) + print("演示完成") + print("=" * 50) + + +async def main(): + """主函数""" + logger.remove() + logger.add( + lambda msg: print(msg, end=""), + format="{time:HH:mm:ss} | {level: <8} | {message}", + level="INFO" + ) + + await demo_login_factory() + + +if __name__ == "__main__": + asyncio.run(main()) diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/07_\347\231\273\345\275\225\350\256\244\350\257\201_\346\211\253\347\240\201\344\270\216\347\237\255\344\277\241\347\231\273\345\275\225\345\256\236\347\216\260/pyproject.toml" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/07_\347\231\273\345\275\225\350\256\244\350\257\201_\346\211\253\347\240\201\344\270\216\347\237\255\344\277\241\347\231\273\345\275\225\345\256\236\347\216\260/pyproject.toml" new file mode 100644 index 0000000..00e8c1f --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/07_\347\231\273\345\275\225\350\256\244\350\257\201_\346\211\253\347\240\201\344\270\216\347\237\255\344\277\241\347\231\273\345\275\225\345\256\236\347\216\260/pyproject.toml" @@ -0,0 +1,25 @@ +[project] +name = "chapter07-qrcode-sms-login" +version = "0.1.0" +description = "第07章：登录认证 - 扫码登录与短信登录技术框架" +readme = "README.md" +requires-python = ">=3.11" +dependencies = [ + "playwright>=1.45.0", + "loguru>=0.7.0", +] + +[project.optional-dependencies] +qrcode = [ + "qrcode[pil]>=7.4.0", # 二维码生成和显示 + "pillow>=10.0.0", +] + +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[tool.uv] +dev-dependencies = [] + +# 提示：安装后需要运行 playwright install 安装浏览器驱动 diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/07_\347\231\273\345\275\225\350\256\244\350\257\201_\346\211\253\347\240\201\344\270\216\347\237\255\344\277\241\347\231\273\345\275\225\345\256\236\347\216\260/qrcode_login.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/07_\347\231\273\345\275\225\350\256\244\350\257\201_\346\211\253\347\240\201\344\270\216\347\237\255\344\277\241\347\231\273\345\275\225\345\256\236\347\216\260/qrcode_login.py" new file mode 100644 index 0000000..4a1bd1d --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/07_\347\231\273\345\275\225\350\256\244\350\257\201_\346\211\253\347\240\201\344\270\216\347\237\255\344\277\241\347\231\273\345\275\225\345\256\236\347\216\260/qrcode_login.py" @@ -0,0 +1,313 @@ +# -*- coding: utf-8 -*- +# @Desc: 扫码登录实现 + +import asyncio +from pathlib import Path +from typing import Optional, Callable, Awaitable +from playwright.async_api import async_playwright, Page, BrowserContext, Playwright +from loguru import logger +from enum import Enum + +# 可选的终端二维码显示支持 +try: + import qrcode + HAS_QRCODE_LIB = True +except ImportError: + HAS_QRCODE_LIB = False + + +class QRCodeStatus(Enum): + """二维码状态枚举""" + WAITING = "waiting" # 等待扫描 + SCANNED = "scanned" # 已扫描，等待确认 + CONFIRMED = "confirmed" # 已确认登录 + EXPIRED = "expired" # 二维码已过期 + CANCELED = "canceled" # 用户取消 + + +def display_qrcode_in_terminal(data: str): + """在终端显示二维码（需要安装 qrcode 库）""" + if not HAS_QRCODE_LIB: + print("提示: 安装 qrcode 库可在终端显示二维码: pip install qrcode") + return + + qr = qrcode.QRCode( + version=1, + error_correction=qrcode.constants.ERROR_CORRECT_L, + box_size=1, + border=1 + ) + qr.add_data(data) + qr.make(fit=True) + qr.print_ascii(invert=True) + + +def display_qrcode_image_in_terminal(image_path: str): + """ + 将图片二维码转换为终端可显示的 ASCII 艺术 + + 需要安装: pip install pillow + """ + try: + from PIL import Image + + img = Image.open(image_path) + img = img.convert('L') # 转为灰度 + + # 缩小图片以适应终端 + width = 50 + ratio = width / img.width + height = int(img.height * ratio * 0.5) # 0.5 补偿终端字符高宽比 + img = img.resize((width, height)) + + # 转换为 ASCII + chars = " .:-=+*#%@" + pixels = img.getdata() + ascii_img = "" + for i, pixel in enumerate(pixels): + if i > 0 and i % width == 0: + ascii_img += "\n" + char_idx = pixel * len(chars) // 256 + ascii_img += chars[char_idx] + + print(ascii_img) + except ImportError: + print(f"二维码已保存到: {image_path}") + print("提示: 安装 pillow 库可在终端显示: pip install pillow") + + +class QRCodeLogin: + """通用扫码登录实现""" + + def __init__( + self, + login_url: str, + qrcode_selector: str, + success_url_pattern: str, + timeout: int = 120, + poll_interval: float = 2.0 + ): + """ + Args: + login_url: 登录页面 URL + qrcode_selector: 二维码元素选择器 (CSS/XPath) + success_url_pattern: 登录成功后 URL 的特征字符串 + timeout: 等待登录的超时时间（秒） + poll_interval: 状态轮询间隔（秒） + """ + self.login_url = login_url + self.qrcode_selector = qrcode_selector + self.success_url_pattern = success_url_pattern + self.timeout = timeout + self.poll_interval = poll_interval + + self._playwright: Optional[Playwright] = None + self._browser = None + self._context: Optional[BrowserContext] = None + self._page: Optional[Page] = None + + async def start(self, playwright: Playwright, headless: bool = False): + """ + 启动浏览器 + + Args: + playwright: Playwright 实例 + headless: 是否无头模式（扫码时通常使用有头模式） + """ + self._playwright = playwright + self._browser = await playwright.chromium.launch( + headless=headless, + args=['--disable-blink-features=AutomationControlled'] + ) + self._context = await self._browser.new_context( + viewport={'width': 1280, 'height': 720} + ) + self._page = await self._context.new_page() + logger.info(f"浏览器已启动 (headless={headless})") + + async def close(self): + """关闭浏览器""" + if self._browser: + await self._browser.close() + self._browser = None + self._context = None + self._page = None + logger.info("浏览器已关闭") + + async def navigate_to_login(self): + """导航到登录页面""" + await self._page.goto(self.login_url, wait_until="networkidle") + logger.info(f"已打开登录页面: {self.login_url}") + + async def wait_for_qrcode(self, timeout: int = 10000) -> bool: + """等待二维码出现""" + try: + await self._page.wait_for_selector( + self.qrcode_selector, + state="visible", + timeout=timeout + ) + logger.info("二维码已出现") + return True + except Exception as e: + logger.error(f"等待二维码超时: {e}") + return False + + async def save_qrcode(self, filepath: str = "qrcode.png") -> Optional[str]: + """ + 保存二维码图片 + + Args: + filepath: 保存路径 + + Returns: + 成功返回文件路径，失败返回 None + """ + if not await self.wait_for_qrcode(): + return None + + try: + qrcode_element = self._page.locator(self.qrcode_selector) + await qrcode_element.screenshot(path=filepath) + logger.info(f"二维码已保存: {filepath}") + return filepath + except Exception as e: + logger.error(f"保存二维码失败: {e}") + return None + + async def wait_for_login( + self, + on_qrcode_ready: Optional[Callable[[str], Awaitable[None]]] = None, + on_status_change: Optional[Callable[[QRCodeStatus], Awaitable[None]]] = None + ) -> bool: + """ + 等待用户扫码登录 + + Args: + on_qrcode_ready: 二维码准备好后的回调 + on_status_change: 状态变化回调 + + Returns: + 是否登录成功 + """ + # 保存二维码 + qrcode_path = await self.save_qrcode() + if not qrcode_path: + return False + + # 通知二维码已准备好 + if on_qrcode_ready: + await on_qrcode_ready(qrcode_path) + + if on_status_change: + await on_status_change(QRCodeStatus.WAITING) + + # 等待登录成功（URL 变化） + try: + await self._page.wait_for_url( + f"**{self.success_url_pattern}**", + timeout=self.timeout * 1000 + ) + + if on_status_change: + await on_status_change(QRCodeStatus.CONFIRMED) + + logger.info("扫码登录成功！") + return True + + except Exception as e: + logger.warning(f"登录超时或失败: {e}") + if on_status_change: + await on_status_change(QRCodeStatus.EXPIRED) + return False + + async def get_cookies(self) -> list: + """获取登录后的 Cookie""" + if self._context: + return await self._context.cookies() + return [] + + async def login( + self, + on_qrcode_ready: Optional[Callable[[str], Awaitable[None]]] = None, + on_status_change: Optional[Callable[[QRCodeStatus], Awaitable[None]]] = None + ) -> Optional[list]: + """ + 执行完整的扫码登录流程 + + Args: + on_qrcode_ready: 二维码准备好后的回调 + on_status_change: 状态变化回调 + + Returns: + 成功返回 Cookie 列表，失败返回 None + """ + await self.navigate_to_login() + + success = await self.wait_for_login( + on_qrcode_ready=on_qrcode_ready, + on_status_change=on_status_change + ) + + if success: + cookies = await self.get_cookies() + logger.info(f"获取到 {len(cookies)} 个 Cookie") + return cookies + + return None + + +async def demo_qrcode_login(): + """扫码登录演示""" + print("=" * 50) + print("扫码登录演示") + print("=" * 50) + print("\n注意: 这是一个演示示例，实际使用需要替换为真实网站的配置") + + # 二维码准备好后的回调 + async def on_qrcode_ready(path: str): + print(f"\n{'='*40}") + print(f"二维码已准备好: {path}") + print("请使用手机扫描登录") + print(f"{'='*40}\n") + # 在终端显示 + display_qrcode_image_in_terminal(path) + + # 状态变化回调 + async def on_status_change(status: QRCodeStatus): + status_text = { + QRCodeStatus.WAITING: "等待扫描...", + QRCodeStatus.SCANNED: "已扫描，请在手机上确认", + QRCodeStatus.CONFIRMED: "登录成功！", + QRCodeStatus.EXPIRED: "二维码已过期", + QRCodeStatus.CANCELED: "用户取消" + } + print(f"状态: {status_text.get(status, status.value)}") + + async with async_playwright() as p: + # 使用示例网站演示（实际使用时替换） + qr_login = QRCodeLogin( + login_url="https://quotes.toscrape.com/login", # 示例网站 + qrcode_selector="form", # 示例选择器 + success_url_pattern="/", + timeout=30 + ) + + await qr_login.start(p, headless=False) + + try: + print("\n这是一个演示，不会真正执行扫码登录") + print("实际使用时，请配置正确的登录 URL 和选择器\n") + + # 演示导航到页面 + await qr_login.navigate_to_login() + await asyncio.sleep(3) + + print("演示完成") + + finally: + await qr_login.close() + + +if __name__ == "__main__": + asyncio.run(demo_qrcode_login()) diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/07_\347\231\273\345\275\225\350\256\244\350\257\201_\346\211\253\347\240\201\344\270\216\347\237\255\344\277\241\347\231\273\345\275\225\345\256\236\347\216\260/sms_login.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/07_\347\231\273\345\275\225\350\256\244\350\257\201_\346\211\253\347\240\201\344\270\216\347\237\255\344\277\241\347\231\273\345\275\225\345\256\236\347\216\260/sms_login.py" new file mode 100644 index 0000000..549a988 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/07_\347\231\273\345\275\225\350\256\244\350\257\201_\346\211\253\347\240\201\344\270\216\347\237\255\344\277\241\347\231\273\345\275\225\345\256\236\347\216\260/sms_login.py" @@ -0,0 +1,371 @@ +# -*- coding: utf-8 -*- +# @Desc: 短信验证码登录实现 + +import asyncio +from typing import Optional, Callable, Awaitable +from playwright.async_api import async_playwright, Page, BrowserContext, Playwright +from loguru import logger + + +class SMSLogin: + """短信验证码登录""" + + def __init__( + self, + login_url: str, + phone_input_selector: str, + send_code_btn_selector: str, + code_input_selector: str, + submit_btn_selector: str, + success_url_pattern: str, + timeout: int = 60 + ): + """ + Args: + login_url: 登录页 URL + phone_input_selector: 手机号输入框选择器 + send_code_btn_selector: 发送验证码按钮选择器 + code_input_selector: 验证码输入框选择器 + submit_btn_selector: 登录按钮选择器 + success_url_pattern: 登录成功 URL 特征 + timeout: 超时时间（秒） + """ + self.login_url = login_url + self.phone_input_selector = phone_input_selector + self.send_code_btn_selector = send_code_btn_selector + self.code_input_selector = code_input_selector + self.submit_btn_selector = submit_btn_selector + self.success_url_pattern = success_url_pattern + self.timeout = timeout + + self._playwright: Optional[Playwright] = None + self._browser = None + self._context: Optional[BrowserContext] = None + self._page: Optional[Page] = None + + async def start(self, playwright: Playwright, headless: bool = False): + """启动浏览器""" + self._playwright = playwright + self._browser = await playwright.chromium.launch( + headless=headless, + args=['--disable-blink-features=AutomationControlled'] + ) + self._context = await self._browser.new_context( + viewport={'width': 1280, 'height': 720} + ) + self._page = await self._context.new_page() + logger.info(f"浏览器已启动 (headless={headless})") + + async def close(self): + """关闭浏览器""" + if self._browser: + await self._browser.close() + self._browser = None + self._context = None + self._page = None + logger.info("浏览器已关闭") + + async def navigate_to_login(self): + """导航到登录页面""" + await self._page.goto(self.login_url, wait_until="networkidle") + logger.info(f"已打开登录页面: {self.login_url}") + + async def input_phone(self, phone: str): + """输入手机号""" + await self._page.wait_for_selector(self.phone_input_selector) + await self._page.fill(self.phone_input_selector, phone) + # 隐藏中间四位 + masked_phone = f"{phone[:3]}****{phone[-4:]}" + logger.info(f"已输入手机号: {masked_phone}") + + async def send_verification_code(self) -> bool: + """ + 发送验证码 + + Returns: + 是否发送成功 + """ + try: + await self._page.wait_for_selector(self.send_code_btn_selector) + await self._page.click(self.send_code_btn_selector) + logger.info("验证码发送请求已提交") + # 等待一小段时间确保请求发出 + await asyncio.sleep(1) + return True + except Exception as e: + logger.error(f"发送验证码失败: {e}") + return False + + async def input_code(self, code: str): + """输入验证码""" + await self._page.wait_for_selector(self.code_input_selector) + await self._page.fill(self.code_input_selector, code) + logger.info(f"已输入验证码: {code}") + + async def submit_login(self) -> bool: + """ + 提交登录 + + Returns: + 是否登录成功 + """ + try: + await self._page.click(self.submit_btn_selector) + + # 等待登录成功（URL 变化） + await self._page.wait_for_url( + f"**{self.success_url_pattern}**", + timeout=self.timeout * 1000 + ) + logger.info("登录成功！") + return True + + except Exception as e: + logger.error(f"登录失败: {e}") + return False + + async def get_cookies(self) -> list: + """获取 Cookie""" + if self._context: + return await self._context.cookies() + return [] + + async def login_with_manual_code( + self, + phone: str, + get_code_callback: Callable[[], Awaitable[str]] + ) -> Optional[list]: + """ + 使用手动输入验证码的方式登录 + + Args: + phone: 手机号 + get_code_callback: 获取验证码的异步回调 + + Returns: + 成功返回 Cookie，失败返回 None + """ + await self.navigate_to_login() + await self.input_phone(phone) + + if not await self.send_verification_code(): + return None + + # 获取验证码 + logger.info("等待获取验证码...") + code = await get_code_callback() + + await self.input_code(code) + + if await self.submit_login(): + return await self.get_cookies() + + return None + + async def login_with_code( + self, + phone: str, + code: str + ) -> Optional[list]: + """ + 使用指定的验证码登录 + + Args: + phone: 手机号 + code: 验证码 + + Returns: + 成功返回 Cookie，失败返回 None + """ + await self.navigate_to_login() + await self.input_phone(phone) + + if not await self.send_verification_code(): + return None + + await self.input_code(code) + + if await self.submit_login(): + return await self.get_cookies() + + return None + + +async def get_code_from_user() -> str: + """从控制台获取用户输入的验证码""" + print("\n请输入收到的验证码: ", end="", flush=True) + loop = asyncio.get_event_loop() + code = await loop.run_in_executor(None, input) + return code.strip() + + +class SMSCodeReceiver: + """ + 短信接码平台接口基类 + + 注意：这是一个示意接口，实际使用需要接入具体的接码平台 API + """ + + def __init__(self, api_key: str, api_url: str): + """ + Args: + api_key: API 密钥 + api_url: API 地址 + """ + self.api_key = api_key + self.api_url = api_url + + async def get_phone_number(self) -> Optional[str]: + """ + 获取手机号 + + Returns: + 手机号或 None + """ + # 实际实现时调用接码平台 API + raise NotImplementedError("需要实现具体的接码平台接口") + + async def wait_for_code( + self, + phone: str, + timeout: int = 60, + poll_interval: float = 2.0 + ) -> Optional[str]: + """ + 等待接收验证码 + + Args: + phone: 手机号 + timeout: 超时时间（秒） + poll_interval: 轮询间隔（秒） + + Returns: + 验证码或 None + """ + # 实际实现时轮询接码平台获取验证码 + raise NotImplementedError("需要实现具体的接码平台接口") + + async def release_phone(self, phone: str): + """ + 释放手机号 + + Args: + phone: 手机号 + """ + # 实际实现时调用接码平台释放手机号 + raise NotImplementedError("需要实现具体的接码平台接口") + + +class MockSMSCodeReceiver(SMSCodeReceiver): + """模拟接码平台（用于测试）""" + + def __init__(self): + super().__init__("mock_key", "mock_url") + self._phones = ["13800138001", "13800138002", "13800138003"] + self._used_phones = set() + + async def get_phone_number(self) -> Optional[str]: + for phone in self._phones: + if phone not in self._used_phones: + self._used_phones.add(phone) + logger.info(f"[Mock] 获取手机号: {phone}") + return phone + return None + + async def wait_for_code( + self, + phone: str, + timeout: int = 60, + poll_interval: float = 2.0 + ) -> Optional[str]: + logger.info(f"[Mock] 等待验证码 (手机号: {phone})") + await asyncio.sleep(2) # 模拟等待 + code = "123456" # 模拟验证码 + logger.info(f"[Mock] 收到验证码: {code}") + return code + + async def release_phone(self, phone: str): + self._used_phones.discard(phone) + logger.info(f"[Mock] 释放手机号: {phone}") + + +async def demo_sms_login(): + """短信登录演示""" + print("=" * 50) + print("短信验证码登录演示") + print("=" * 50) + print("\n注意: 这是一个演示示例，实际使用需要替换为真实网站的配置") + + async with async_playwright() as p: + # 使用示例网站演示 + sms_login = SMSLogin( + login_url="https://quotes.toscrape.com/login", + phone_input_selector="#username", + send_code_btn_selector="input[type='submit']", # 示例 + code_input_selector="#password", + submit_btn_selector="input[type='submit']", + success_url_pattern="/" + ) + + await sms_login.start(p, headless=False) + + try: + print("\n这是一个演示，使用示例网站的登录表单") + print("实际短信登录需要配置正确的选择器\n") + + # 演示导航到页面 + await sms_login.navigate_to_login() + + # 演示输入（使用示例网站的账号密码） + await sms_login._page.fill("#username", "demo_user") + await sms_login._page.fill("#password", "demo_pass") + + print("已填写表单（演示）") + await asyncio.sleep(3) + + print("演示完成") + + finally: + await sms_login.close() + + +async def demo_mock_receiver(): + """模拟接码平台演示""" + print("\n" + "=" * 50) + print("模拟接码平台演示") + print("=" * 50) + + receiver = MockSMSCodeReceiver() + + # 获取手机号 + phone = await receiver.get_phone_number() + print(f"获取到手机号: {phone}") + + # 等待验证码 + code = await receiver.wait_for_code(phone) + print(f"收到验证码: {code}") + + # 释放手机号 + await receiver.release_phone(phone) + print("手机号已释放") + + +async def main(): + """主函数""" + logger.remove() + logger.add( + lambda msg: print(msg, end=""), + format="{time:HH:mm:ss} | {level: <8} | {message}", + level="INFO" + ) + + await demo_sms_login() + await demo_mock_receiver() + + print("\n" + "=" * 50) + print("所有演示完成") + print("=" * 50) + + +if __name__ == "__main__": + asyncio.run(main()) diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/08_\351\252\214\350\257\201\347\240\201\350\257\206\345\210\253\344\270\216\345\244\204\347\220\206/README.md" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/08_\351\252\214\350\257\201\347\240\201\350\257\206\345\210\253\344\270\216\345\244\204\347\220\206/README.md" new file mode 100644 index 0000000..022f22f --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/08_\351\252\214\350\257\201\347\240\201\350\257\206\345\210\253\344\270\216\345\244\204\347\220\206/README.md" @@ -0,0 +1,37 @@ +# 第08章：验证码识别与处理 + +展示图片验证码OCR识别、滑块验证码处理、人类轨迹生成等。 + +## 快速开始 + +```bash +cd 08_验证码识别与处理 + +# 安装基础依赖 +uv sync + +# 安装OCR功能 +uv sync --extra ocr + +# 安装图像处理功能 +uv sync --extra cv + +# 安装验证码生成功能 +uv sync --extra generate + +# 安装轨迹可视化功能 +uv sync --extra viz + +# 或安装所有功能 +uv sync --extra all + +# 运行示例 +uv run python ocr_captcha.py +uv run python slider_captcha.py +``` + +### 可选依赖说明 +- `ddddocr` - OCR识别引擎 +- `opencv-python` - 图像处理（滑块验证码） +- `captcha` - 本地验证码生成 +- `matplotlib` - 轨迹可视化 diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/08_\351\252\214\350\257\201\347\240\201\350\257\206\345\210\253\344\270\216\345\244\204\347\220\206/captcha_service.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/08_\351\252\214\350\257\201\347\240\201\350\257\206\345\210\253\344\270\216\345\244\204\347\220\206/captcha_service.py" new file mode 100644 index 0000000..d2d0637 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/08_\351\252\214\350\257\201\347\240\201\350\257\206\345\210\253\344\270\216\345\244\204\347\220\206/captcha_service.py" @@ -0,0 +1,407 @@ +# -*- coding: utf-8 -*- +# @Desc: 验证码服务 - 整合本地识别和第三方打码平台 + +import asyncio +import base64 +from abc import ABC, abstractmethod +from typing import Optional, Callable, Awaitable +from datetime import date +from loguru import logger + +try: + import httpx + HAS_HTTPX = True +except ImportError: + HAS_HTTPX = False + + +class CaptchaServiceBase(ABC): + """验证码服务基类""" + + @abstractmethod + async def solve_image(self, image_bytes: bytes) -> Optional[str]: + """识别图片验证码""" + pass + + +class LocalOCRService(CaptchaServiceBase): + """本地 OCR 服务""" + + def __init__(self, preprocess: bool = True): + from ocr_captcha import OCRCaptchaSolver + self.solver = OCRCaptchaSolver(preprocess=preprocess) + + async def solve_image(self, image_bytes: bytes) -> Optional[str]: + """使用本地 OCR 识别""" + return self.solver.solve(image_bytes) + + +class RemoteCaptchaService(CaptchaServiceBase): + """ + 远程打码平台服务（示例实现） + + 注意：这是一个通用的示例接口，实际使用需要根据具体平台 API 调整 + """ + + def __init__( + self, + api_key: str, + api_url: str, + timeout: int = 30 + ): + """ + Args: + api_key: API 密钥 + api_url: API 地址 + timeout: 超时时间（秒） + """ + if not HAS_HTTPX: + raise ImportError("需要安装 httpx: pip install httpx") + + self.api_key = api_key + self.api_url = api_url.rstrip("/") + self.timeout = timeout + self._last_task_id: Optional[str] = None + + async def solve_image( + self, + image_bytes: bytes, + captcha_type: str = "default" + ) -> Optional[str]: + """ + 识别图片验证码 + + Args: + image_bytes: 图片字节 + captcha_type: 验证码类型 + + Returns: + 识别结果 + """ + try: + async with httpx.AsyncClient(timeout=self.timeout) as client: + # 编码图片 + image_base64 = base64.b64encode(image_bytes).decode() + + # 提交任务 + resp = await client.post( + f"{self.api_url}/create_task", + json={ + "api_key": self.api_key, + "image": image_base64, + "type": captcha_type + } + ) + + data = resp.json() + task_id = data.get("task_id") + + if not task_id: + logger.error(f"创建任务失败: {data}") + return None + + self._last_task_id = task_id + logger.debug(f"任务已创建: {task_id}") + + # 轮询获取结果 + return await self._poll_result(client, task_id) + + except Exception as e: + logger.error(f"打码平台请求失败: {e}") + return None + + async def _poll_result( + self, + client: httpx.AsyncClient, + task_id: str, + max_attempts: int = 30 + ) -> Optional[str]: + """轮询获取结果""" + for attempt in range(max_attempts): + try: + resp = await client.get( + f"{self.api_url}/get_result", + params={"task_id": task_id} + ) + + data = resp.json() + status = data.get("status") + + if status == "ready": + result = data.get("result") + logger.debug(f"识别成功: {result}") + return result + elif status == "error": + logger.error(f"识别错误: {data.get('error')}") + return None + + await asyncio.sleep(1) + + except Exception as e: + logger.warning(f"轮询异常: {e}") + await asyncio.sleep(1) + + logger.error("识别超时") + return None + + async def report_error(self, task_id: str = None): + """ + 报告识别错误（用于退款） + + Args: + task_id: 任务 ID，不提供则使用最后一个任务 + """ + task_id = task_id or self._last_task_id + if not task_id: + logger.warning("没有可报告的任务") + return + + try: + async with httpx.AsyncClient() as client: + await client.post( + f"{self.api_url}/report_error", + json={ + "api_key": self.api_key, + "task_id": task_id + } + ) + logger.info(f"已报告错误: {task_id}") + except Exception as e: + logger.warning(f"报告错误失败: {e}") + + async def get_balance(self) -> float: + """获取账户余额""" + try: + async with httpx.AsyncClient() as client: + resp = await client.get( + f"{self.api_url}/balance", + params={"api_key": self.api_key} + ) + data = resp.json() + return data.get("balance", 0.0) + except Exception as e: + logger.error(f"获取余额失败: {e}") + return 0.0 + + +class CostController: + """打码成本控制器""" + + def __init__( + self, + daily_budget: float, + cost_per_captcha: float = 0.01 + ): + """ + Args: + daily_budget: 每日预算 + cost_per_captcha: 每次识别成本 + """ + self.daily_budget = daily_budget + self.cost_per_captcha = cost_per_captcha + self._daily_spent = 0.0 + self._last_reset: Optional[date] = None + self._usage_count = 0 + + def can_use_service(self) -> bool: + """是否可以使用打码服务""" + self._check_reset() + return self._daily_spent < self.daily_budget + + def record_usage(self): + """记录一次使用""" + self._check_reset() + self._daily_spent += self.cost_per_captcha + self._usage_count += 1 + logger.debug(f"记录使用: 已花费 {self._daily_spent:.2f}") + + def _check_reset(self): + """检查是否需要重置（跨天）""" + today = date.today() + if self._last_reset != today: + self._daily_spent = 0.0 + self._usage_count = 0 + self._last_reset = today + logger.info("成本计数器已重置") + + @property + def remaining_budget(self) -> float: + """剩余预算""" + self._check_reset() + return max(0, self.daily_budget - self._daily_spent) + + @property + def today_usage(self) -> int: + """今日使用次数""" + self._check_reset() + return self._usage_count + + def get_stats(self) -> dict: + """获取统计信息""" + self._check_reset() + return { + "daily_budget": self.daily_budget, + "daily_spent": self._daily_spent, + "remaining": self.remaining_budget, + "usage_count": self._usage_count, + "cost_per_captcha": self.cost_per_captcha + } + + +class CaptchaSolverWithFallback: + """带降级策略的验证码解决器""" + + def __init__( + self, + local_service: CaptchaServiceBase = None, + remote_service: CaptchaServiceBase = None, + cost_controller: CostController = None + ): + """ + Args: + local_service: 本地识别服务 + remote_service: 远程打码服务 + cost_controller: 成本控制器 + """ + self.local_service = local_service + self.remote_service = remote_service + self.cost_controller = cost_controller + + async def solve( + self, + image_bytes: bytes, + verify_callback: Callable[[str], Awaitable[bool]] = None, + prefer_local: bool = True, + max_local_retries: int = 2 + ) -> Optional[str]: + """ + 解决验证码 + + Args: + image_bytes: 验证码图片 + verify_callback: 验证回调（可选） + prefer_local: 是否优先使用本地识别 + max_local_retries: 本地识别最大重试次数 + + Returns: + 识别结果 + """ + # 优先本地识别 + if prefer_local and self.local_service: + for attempt in range(max_local_retries): + result = await self.local_service.solve_image(image_bytes) + + if result: + # 如果有验证回调 + if verify_callback: + if await verify_callback(result): + logger.info(f"本地识别成功 (尝试 {attempt + 1})") + return result + else: + return result + + logger.debug(f"本地识别失败，重试 {attempt + 1}/{max_local_retries}") + + # 降级到远程服务 + if self.remote_service: + # 检查成本 + if self.cost_controller and not self.cost_controller.can_use_service(): + logger.warning("超出每日预算，无法使用打码服务") + return None + + result = await self.remote_service.solve_image(image_bytes) + + if result: + if self.cost_controller: + self.cost_controller.record_usage() + logger.info("远程识别成功") + return result + + logger.error("验证码识别失败") + return None + + +async def demo(): + """验证码服务演示""" + print("=" * 50) + print("验证码服务演示") + print("=" * 50) + + # 1. 本地 OCR 服务 + print("\n1. 本地 OCR 服务:") + try: + local_service = LocalOCRService(preprocess=True) + print(" 本地 OCR 服务已创建") + print(" 用法: result = await local_service.solve_image(image_bytes)") + except ImportError as e: + print(f" 创建失败: {e}") + + # 2. 成本控制器 + print("\n2. 成本控制器演示:") + controller = CostController( + daily_budget=10.0, + cost_per_captcha=0.01 + ) + print(f" 每日预算: {controller.daily_budget}") + print(f" 每次成本: {controller.cost_per_captcha}") + print(f" 可以使用: {controller.can_use_service()}") + + # 模拟使用 + for i in range(5): + if controller.can_use_service(): + controller.record_usage() + + stats = controller.get_stats() + print(f" 使用后统计: {stats}") + + # 3. 带降级的解决器 + print("\n3. 带降级策略的解决器:") + print(""" + solver = CaptchaSolverWithFallback( + local_service=local_service, + remote_service=remote_service, + cost_controller=cost_controller + ) + + result = await solver.solve( + image_bytes, + verify_callback=verify_func, # 可选 + prefer_local=True, + max_local_retries=2 + ) + """) + + # 4. 远程服务示例 + print("\n4. 远程打码服务（示例配置）:") + print(""" + remote_service = RemoteCaptchaService( + api_key="your_api_key", + api_url="https://api.captcha-service.com", + timeout=30 + ) + + # 识别验证码 + result = await remote_service.solve_image(image_bytes) + + # 报告错误（退款） + if not is_correct: + await remote_service.report_error() + + # 查询余额 + balance = await remote_service.get_balance() + """) + + print("\n" + "=" * 50) + print("演示完成") + print("=" * 50) + + +if __name__ == "__main__": + logger.remove() + logger.add( + lambda msg: print(msg, end=""), + format="{time:HH:mm:ss} | {level: <8} | {message}", + level="DEBUG" + ) + + asyncio.run(demo()) diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/08_\351\252\214\350\257\201\347\240\201\350\257\206\345\210\253\344\270\216\345\244\204\347\220\206/ocr_captcha.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/08_\351\252\214\350\257\201\347\240\201\350\257\206\345\210\253\344\270\216\345\244\204\347\220\206/ocr_captcha.py" new file mode 100644 index 0000000..74c2580 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/08_\351\252\214\350\257\201\347\240\201\350\257\206\345\210\253\344\270\216\345\244\204\347\220\206/ocr_captcha.py" @@ -0,0 +1,316 @@ +# -*- coding: utf-8 -*- +# @Desc: OCR 图片验证码识别 + +import io +from typing import Optional +from loguru import logger + +# 可选依赖 +try: + import ddddocr + HAS_DDDDOCR = True +except ImportError: + HAS_DDDDOCR = False + logger.warning("ddddocr 未安装，部分功能不可用。安装: pip install ddddocr") + +try: + from PIL import Image, ImageFilter, ImageEnhance + HAS_PIL = True +except ImportError: + HAS_PIL = False + logger.warning("pillow 未安装，图片预处理不可用。安装: pip install pillow") + + +class CaptchaPreprocessor: + """验证码图片预处理器""" + + def __init__(self): + if not HAS_PIL: + raise ImportError("需要安装 pillow: pip install pillow") + + def to_grayscale(self, image_bytes: bytes) -> bytes: + """ + 转为灰度图 + + Args: + image_bytes: 图片字节 + + Returns: + 处理后的图片字节 + """ + img = Image.open(io.BytesIO(image_bytes)) + gray = img.convert('L') + + buffer = io.BytesIO() + gray.save(buffer, format='PNG') + return buffer.getvalue() + + def binarize(self, image_bytes: bytes, threshold: int = 127) -> bytes: + """ + 二值化 + + Args: + image_bytes: 图片字节 + threshold: 阈值 (0-255)，小于阈值变黑，大于变白 + + Returns: + 处理后的图片字节 + """ + img = Image.open(io.BytesIO(image_bytes)) + gray = img.convert('L') + + # 二值化 + binary = gray.point(lambda x: 255 if x > threshold else 0) + + buffer = io.BytesIO() + binary.save(buffer, format='PNG') + return buffer.getvalue() + + def remove_noise(self, image_bytes: bytes, size: int = 3) -> bytes: + """ + 去噪点（中值滤波） + + Args: + image_bytes: 图片字节 + size: 滤波器大小 + + Returns: + 处理后的图片字节 + """ + img = Image.open(io.BytesIO(image_bytes)) + denoised = img.filter(ImageFilter.MedianFilter(size=size)) + + buffer = io.BytesIO() + denoised.save(buffer, format='PNG') + return buffer.getvalue() + + def enhance_contrast(self, image_bytes: bytes, factor: float = 2.0) -> bytes: + """ + 增强对比度 + + Args: + image_bytes: 图片字节 + factor: 对比度因子，>1 增强，<1 降低 + + Returns: + 处理后的图片字节 + """ + img = Image.open(io.BytesIO(image_bytes)) + enhancer = ImageEnhance.Contrast(img) + enhanced = enhancer.enhance(factor) + + buffer = io.BytesIO() + enhanced.save(buffer, format='PNG') + return buffer.getvalue() + + def enhance_sharpness(self, image_bytes: bytes, factor: float = 2.0) -> bytes: + """ + 增强锐度 + + Args: + image_bytes: 图片字节 + factor: 锐度因子 + + Returns: + 处理后的图片字节 + """ + img = Image.open(io.BytesIO(image_bytes)) + enhancer = ImageEnhance.Sharpness(img) + enhanced = enhancer.enhance(factor) + + buffer = io.BytesIO() + enhanced.save(buffer, format='PNG') + return buffer.getvalue() + + def resize(self, image_bytes: bytes, scale: float = 2.0) -> bytes: + """ + 缩放图片 + + Args: + image_bytes: 图片字节 + scale: 缩放比例 + + Returns: + 处理后的图片字节 + """ + img = Image.open(io.BytesIO(image_bytes)) + new_size = (int(img.width * scale), int(img.height * scale)) + resized = img.resize(new_size, Image.Resampling.LANCZOS) + + buffer = io.BytesIO() + resized.save(buffer, format='PNG') + return buffer.getvalue() + + def full_preprocess( + self, + image_bytes: bytes, + threshold: int = 150 + ) -> bytes: + """ + 完整预处理流程 + + Args: + image_bytes: 原始图片 + threshold: 二值化阈值 + + Returns: + 处理后的图片 + """ + # 灰度化 + processed = self.to_grayscale(image_bytes) + # 增强对比度 + processed = self.enhance_contrast(processed, factor=1.5) + # 去噪 + processed = self.remove_noise(processed) + # 二值化 + processed = self.binarize(processed, threshold) + + return processed + + +class OCRCaptchaSolver: + """OCR 验证码识别器""" + + def __init__(self, preprocess: bool = True, show_ad: bool = False): + """ + Args: + preprocess: 是否进行预处理 + show_ad: 是否显示 ddddocr 广告 + """ + if not HAS_DDDDOCR: + raise ImportError("需要安装 ddddocr: pip install ddddocr") + + self.ocr = ddddocr.DdddOcr(show_ad=show_ad) + self.preprocess = preprocess + + if preprocess and HAS_PIL: + self.preprocessor = CaptchaPreprocessor() + else: + self.preprocessor = None + + def solve(self, image_bytes: bytes) -> Optional[str]: + """ + 识别验证码 + + Args: + image_bytes: 图片字节 + + Returns: + 识别结果 + """ + try: + # 预处理 + if self.preprocess and self.preprocessor: + image_bytes = self.preprocessor.full_preprocess(image_bytes) + + # 识别 + result = self.ocr.classification(image_bytes) + logger.debug(f"验证码识别结果: {result}") + return result + + except Exception as e: + logger.error(f"验证码识别失败: {e}") + return None + + def solve_raw(self, image_bytes: bytes) -> Optional[str]: + """ + 不预处理直接识别 + + Args: + image_bytes: 图片字节 + + Returns: + 识别结果 + """ + try: + return self.ocr.classification(image_bytes) + except Exception as e: + logger.error(f"验证码识别失败: {e}") + return None + + def solve_with_multiple_thresholds( + self, + image_bytes: bytes, + thresholds: list = None, + min_length: int = 4 + ) -> Optional[str]: + """ + 尝试不同阈值识别 + + Args: + image_bytes: 图片字节 + thresholds: 要尝试的阈值列表 + min_length: 最小结果长度 + + Returns: + 最佳识别结果 + """ + if not self.preprocessor: + return self.solve_raw(image_bytes) + + thresholds = thresholds or [100, 127, 150, 180] + best_result = None + best_confidence = 0 + + for threshold in thresholds: + try: + processed = self.preprocessor.full_preprocess( + image_bytes, + threshold=threshold + ) + result = self.ocr.classification(processed) + + if result and len(result) >= min_length: + # 简单的置信度评估：结果长度和字符类型 + confidence = len(result) + if result.isalnum(): + confidence += 2 + + if confidence > best_confidence: + best_confidence = confidence + best_result = result + + except Exception as e: + logger.debug(f"阈值 {threshold} 识别失败: {e}") + continue + + return best_result + + +def demo(): + """演示 OCR 验证码识别""" + print("=" * 50) + print("OCR 验证码识别演示") + print("=" * 50) + + if not HAS_DDDDOCR: + print("请先安装 ddddocr: pip install ddddocr") + return + + # 创建识别器 + solver = OCRCaptchaSolver(preprocess=True, show_ad=False) + + # 演示：使用测试图片 + # 实际使用时，从网页获取验证码图片 + print("\n1. 基本识别演示:") + print(" 实际使用时，传入验证码图片的字节数据") + print(" 示例: result = solver.solve(image_bytes)") + + # 演示预处理器 + if HAS_PIL: + print("\n2. 图片预处理演示:") + preprocessor = CaptchaPreprocessor() + print(" 支持的预处理方法:") + print(" - to_grayscale(): 转灰度图") + print(" - binarize(): 二值化") + print(" - remove_noise(): 去噪点") + print(" - enhance_contrast(): 增强对比度") + print(" - full_preprocess(): 完整预处理流程") + + print("\n" + "=" * 50) + print("演示完成") + print("=" * 50) + + +if __name__ == "__main__": + demo() diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/08_\351\252\214\350\257\201\347\240\201\350\257\206\345\210\253\344\270\216\345\244\204\347\220\206/pyproject.toml" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/08_\351\252\214\350\257\201\347\240\201\350\257\206\345\210\253\344\270\216\345\244\204\347\220\206/pyproject.toml" new file mode 100644 index 0000000..6a0f9cb --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/08_\351\252\214\350\257\201\347\240\201\350\257\206\345\210\253\344\270\216\345\244\204\347\220\206/pyproject.toml" @@ -0,0 +1,37 @@ +[project] +name = "chapter08-captcha-recognition" +version = "0.1.0" +description = "第08章：验证码识别与处理 - 图片验证码OCR、滑块验证码、轨迹生成" +readme = "README.md" +requires-python = ">=3.11" +dependencies = [ + "loguru>=0.7.0", + "pillow>=10.0.0", +] + +[project.optional-dependencies] +ocr = [ + "ddddocr>=1.4.0", # OCR识别库 +] +cv = [ + "opencv-python>=4.9.0", # 图像处理（滑块验证码） +] +generate = [ + "captcha>=0.5.0", # 本地验证码生成 +] +viz = [ + "matplotlib>=3.8.0", # 轨迹可视化 +] +all = [ + "ddddocr>=1.4.0", + "opencv-python>=4.9.0", + "captcha>=0.5.0", + "matplotlib>=3.8.0", +] + +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[tool.uv] +dev-dependencies = [] diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/08_\351\252\214\350\257\201\347\240\201\350\257\206\345\210\253\344\270\216\345\244\204\347\220\206/slider_captcha.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/08_\351\252\214\350\257\201\347\240\201\350\257\206\345\210\253\344\270\216\345\244\204\347\220\206/slider_captcha.py" new file mode 100644 index 0000000..27d791c --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/08_\351\252\214\350\257\201\347\240\201\350\257\206\345\210\253\344\270\216\345\244\204\347\220\206/slider_captcha.py" @@ -0,0 +1,472 @@ +# -*- coding: utf-8 -*- +# @Desc: 滑块验证码处理 + +import asyncio +import random +import math +from typing import Optional, List, Tuple +from loguru import logger + +# 可选依赖 +try: + import cv2 + import numpy as np + HAS_CV2 = True +except ImportError: + HAS_CV2 = False + logger.warning("opencv-python 未安装，缺口检测不可用。安装: pip install opencv-python") + + +class SliderGapDetector: + """滑块缺口位置检测器""" + + def __init__(self): + if not HAS_CV2: + raise ImportError("需要安装 opencv-python: pip install opencv-python") + + def detect_by_template_match( + self, + background_bytes: bytes, + slider_bytes: bytes + ) -> Optional[int]: + """ + 通过模板匹配找缺口位置 + + Args: + background_bytes: 背景图片字节 + slider_bytes: 滑块图片字节 + + Returns: + 缺口 x 坐标 + """ + # 读取图片 + bg = cv2.imdecode( + np.frombuffer(background_bytes, np.uint8), + cv2.IMREAD_COLOR + ) + slider = cv2.imdecode( + np.frombuffer(slider_bytes, np.uint8), + cv2.IMREAD_COLOR + ) + + # 转灰度 + bg_gray = cv2.cvtColor(bg, cv2.COLOR_BGR2GRAY) + slider_gray = cv2.cvtColor(slider, cv2.COLOR_BGR2GRAY) + + # 边缘检测 + bg_edges = cv2.Canny(bg_gray, 100, 200) + slider_edges = cv2.Canny(slider_gray, 100, 200) + + # 模板匹配 + result = cv2.matchTemplate( + bg_edges, + slider_edges, + cv2.TM_CCOEFF_NORMED + ) + _, _, _, max_loc = cv2.minMaxLoc(result) + + gap_x = max_loc[0] + logger.debug(f"模板匹配检测到缺口位置: x={gap_x}") + return gap_x + + def detect_by_contour( + self, + background_bytes: bytes, + min_area: int = 1000 + ) -> Optional[int]: + """ + 通过轮廓检测找缺口位置 + + 某些滑块验证码的缺口有明显的轮廓 + + Args: + background_bytes: 背景图片 + min_area: 最小轮廓面积 + + Returns: + 缺口 x 坐标 + """ + bg = cv2.imdecode( + np.frombuffer(background_bytes, np.uint8), + cv2.IMREAD_COLOR + ) + + # 转灰度 + gray = cv2.cvtColor(bg, cv2.COLOR_BGR2GRAY) + + # 高斯模糊 + blurred = cv2.GaussianBlur(gray, (5, 5), 0) + + # 边缘检测 + edges = cv2.Canny(blurred, 50, 150) + + # 找轮廓 + contours, _ = cv2.findContours( + edges, + cv2.RETR_EXTERNAL, + cv2.CHAIN_APPROX_SIMPLE + ) + + # 筛选合适的轮廓 + candidates = [] + for contour in contours: + area = cv2.contourArea(contour) + if area > min_area: + x, y, w, h = cv2.boundingRect(contour) + # 缺口通常在图片中间偏右 + if x > bg.shape[1] * 0.2: + candidates.append((x, area)) + + if candidates: + # 选择面积最大的 + candidates.sort(key=lambda c: c[1], reverse=True) + return candidates[0][0] + + return None + + def detect_by_color_diff( + self, + background_bytes: bytes, + threshold: int = 50 + ) -> Optional[int]: + """ + 通过颜色差异找缺口 + + 缺口区域通常颜色较暗 + + Args: + background_bytes: 背景图片 + threshold: 颜色阈值 + + Returns: + 缺口 x 坐标 + """ + bg = cv2.imdecode( + np.frombuffer(background_bytes, np.uint8), + cv2.IMREAD_COLOR + ) + + # 转灰度 + gray = cv2.cvtColor(bg, cv2.COLOR_BGR2GRAY) + + # 二值化（找暗色区域） + _, binary = cv2.threshold(gray, threshold, 255, cv2.THRESH_BINARY_INV) + + # 找轮廓 + contours, _ = cv2.findContours( + binary, + cv2.RETR_EXTERNAL, + cv2.CHAIN_APPROX_SIMPLE + ) + + if contours: + # 找最大轮廓 + largest = max(contours, key=cv2.contourArea) + x, y, w, h = cv2.boundingRect(largest) + return x + + return None + + +class HumanTrajectoryGenerator: + """人类轨迹生成器""" + + @staticmethod + def generate_linear( + distance: int, + duration: float = 0.5 + ) -> List[Tuple[int, int, float]]: + """ + 生成线性轨迹（简单但容易被检测） + + Args: + distance: 移动距离 + duration: 持续时间 + + Returns: + 轨迹点列表 [(x, y, time), ...] + """ + trajectory = [] + steps = 20 + step_time = duration / steps + + for i in range(steps + 1): + progress = i / steps + x = int(distance * progress) + y = 0 + t = step_time * i + trajectory.append((x, y, t)) + + return trajectory + + @staticmethod + def generate_ease_out( + distance: int, + duration: float = 0.5 + ) -> List[Tuple[int, int, float]]: + """ + 生成缓出轨迹（先快后慢） + + Args: + distance: 移动距离 + duration: 持续时间 + + Returns: + 轨迹点列表 + """ + trajectory = [] + steps = random.randint(25, 35) + step_time = duration / steps + + for i in range(steps + 1): + t = i / steps + # 二次缓出 + eased = t * (2 - t) + + x = int(distance * eased) + y = random.randint(-2, 2) # 小幅度 Y 抖动 + time_point = step_time * i + random.uniform(-0.005, 0.005) + + trajectory.append((x, y, max(0, time_point))) + + return trajectory + + @staticmethod + def generate_bezier( + distance: int, + duration: float = 0.5 + ) -> List[Tuple[int, int, float]]: + """ + 使用贝塞尔曲线生成自然轨迹 + + Args: + distance: 移动距离 + duration: 持续时间 + + Returns: + 轨迹点列表 + """ + trajectory = [] + + # 控制点（随机生成更自然） + p0 = (0, 0) + p1 = (distance * random.uniform(0.2, 0.4), random.randint(-15, 15)) + p2 = (distance * random.uniform(0.6, 0.8), random.randint(-8, 8)) + p3 = (distance, 0) + + steps = random.randint(30, 45) + + for i in range(steps + 1): + t = i / steps + + # 三阶贝塞尔曲线 + x = ( + (1-t)**3 * p0[0] + + 3*(1-t)**2*t * p1[0] + + 3*(1-t)*t**2 * p2[0] + + t**3 * p3[0] + ) + y = ( + (1-t)**3 * p0[1] + + 3*(1-t)**2*t * p1[1] + + 3*(1-t)*t**2 * p2[1] + + t**3 * p3[1] + ) + + # 时间加随机偏移 + time_point = duration * t + random.uniform(-0.003, 0.003) + trajectory.append((int(x), int(y), max(0, time_point))) + + return trajectory + + @staticmethod + def generate_human_like( + distance: int, + duration: float = 0.5 + ) -> List[Tuple[int, int, float]]: + """ + 生成模拟人类的轨迹（综合多种特征） + + 包含：加速、微调、抖动等人类特征 + + Args: + distance: 移动距离 + duration: 持续时间 + + Returns: + 轨迹点列表 + """ + trajectory = [] + + # 分阶段：加速 -> 匀速 -> 减速 -> 微调 + phases = [ + (0.3, 0.2), # 加速阶段 + (0.5, 0.4), # 匀速阶段 + (0.15, 0.3), # 减速阶段 + (0.05, 0.1), # 微调阶段 + ] + + current_x = 0 + current_time = 0 + + for phase_distance_ratio, phase_duration_ratio in phases: + phase_distance = distance * phase_distance_ratio + phase_duration = duration * phase_duration_ratio + phase_steps = random.randint(5, 10) + step_time = phase_duration / phase_steps + + for i in range(phase_steps): + progress = (i + 1) / phase_steps + x = current_x + int(phase_distance * progress) + y = random.randint(-3, 3) + t = current_time + step_time * (i + 1) + random.uniform(-0.002, 0.002) + + trajectory.append((x, y, max(0, t))) + + current_x += int(phase_distance) + current_time += phase_duration + + # 确保最后到达目标 + trajectory.append((distance, 0, duration)) + + return trajectory + + +class SliderCaptchaSolver: + """滑块验证码解决器（需要配合 Playwright 使用）""" + + def __init__(self): + if HAS_CV2: + self.gap_detector = SliderGapDetector() + else: + self.gap_detector = None + + self.trajectory_generator = HumanTrajectoryGenerator() + + def detect_gap( + self, + background_bytes: bytes, + slider_bytes: bytes = None + ) -> Optional[int]: + """ + 检测缺口位置 + + Args: + background_bytes: 背景图片 + slider_bytes: 滑块图片（可选） + + Returns: + 缺口 x 坐标 + """ + if not self.gap_detector: + logger.error("缺口检测需要 opencv-python") + return None + + if slider_bytes: + return self.gap_detector.detect_by_template_match( + background_bytes, + slider_bytes + ) + else: + return self.gap_detector.detect_by_contour(background_bytes) + + def generate_trajectory( + self, + distance: int, + style: str = "human" + ) -> List[Tuple[int, int, float]]: + """ + 生成拖拽轨迹 + + Args: + distance: 拖拽距离 + style: 轨迹风格 (linear/ease_out/bezier/human) + + Returns: + 轨迹点列表 + """ + generators = { + "linear": self.trajectory_generator.generate_linear, + "ease_out": self.trajectory_generator.generate_ease_out, + "bezier": self.trajectory_generator.generate_bezier, + "human": self.trajectory_generator.generate_human_like + } + + generator = generators.get(style, self.trajectory_generator.generate_human_like) + return generator(distance) + + +async def demo(): + """滑块验证码处理演示""" + print("=" * 50) + print("滑块验证码处理演示") + print("=" * 50) + + # 演示轨迹生成 + print("\n1. 轨迹生成演示:") + generator = HumanTrajectoryGenerator() + + distance = 200 # 200像素 + + styles = { + "linear": generator.generate_linear, + "ease_out": generator.generate_ease_out, + "bezier": generator.generate_bezier, + "human": generator.generate_human_like + } + + for style_name, style_func in styles.items(): + trajectory = style_func(distance) + print(f"\n {style_name} 轨迹:") + print(f" - 点数: {len(trajectory)}") + print(f" - 起点: {trajectory[0]}") + print(f" - 终点: {trajectory[-1]}") + + # 演示缺口检测 + if HAS_CV2: + print("\n2. 缺口检测演示:") + print(" 支持的检测方法:") + print(" - detect_by_template_match(): 模板匹配") + print(" - detect_by_contour(): 轮廓检测") + print(" - detect_by_color_diff(): 颜色差异检测") + else: + print("\n2. 缺口检测需要安装 opencv-python:") + print(" pip install opencv-python") + + # 演示完整流程 + print("\n3. 完整使用流程:") + print(""" + # 1. 获取背景和滑块图片 + bg_bytes = await page.locator("#bg-image").screenshot() + slider_bytes = await page.locator("#slider-image").screenshot() + + # 2. 检测缺口位置 + solver = SliderCaptchaSolver() + gap_x = solver.detect_gap(bg_bytes, slider_bytes) + + # 3. 生成轨迹 + trajectory = solver.generate_trajectory(gap_x, style="human") + + # 4. 执行拖拽（Playwright） + slider = page.locator("#slider-btn") + box = await slider.bounding_box() + start_x = box['x'] + box['width'] / 2 + start_y = box['y'] + box['height'] / 2 + + await page.mouse.move(start_x, start_y) + await page.mouse.down() + + for x, y, t in trajectory: + await asyncio.sleep(t - last_t) + await page.mouse.move(start_x + x, start_y + y) + + await page.mouse.up() + """) + + print("\n" + "=" * 50) + print("演示完成") + print("=" * 50) + + +if __name__ == "__main__": + asyncio.run(demo()) diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/09_\346\225\260\346\215\256\346\270\205\346\264\227\344\270\216\351\242\204\345\244\204\347\220\206/README.md" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/09_\346\225\260\346\215\256\346\270\205\346\264\227\344\270\216\351\242\204\345\244\204\347\220\206/README.md" new file mode 100644 index 0000000..8e9c4f2 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/09_\346\225\260\346\215\256\346\270\205\346\264\227\344\270\216\351\242\204\345\244\204\347\220\206/README.md" @@ -0,0 +1,42 @@ +# 第09章：数据清洗与预处理 + +展示文本清洗、数据标准化、去重算法等数据处理技术。 + +## 快速开始 + +```bash +cd 09_数据清洗与预处理 + +# 安装依赖（本章主要使用Python标准库） +uv sync + +# 安装SimHash去重功能（可选） +uv sync --extra simhash + +# 运行示例 +uv run python text_cleaner.py +uv run python data_normalizer.py +uv run python deduplication.py +``` + +### 主要功能 + +- **文本清洗** + - HTML标签移除 + - 空白字符处理 + - 特殊字符清理 + - 编码问题修复 + +- **数据标准化** + - 日期时间格式统一 + - 数值单位换算 + - 文本规范化 + +- **去重算法** + - 精确去重 + - 模糊去重（编辑距离、Jaccard相似度） + - SimHash去重（大规模文本） + +### 依赖说明 + +本章主要使用Python标准库，`simhash` 为可选依赖，用于大规模文本去重。 diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/09_\346\225\260\346\215\256\346\270\205\346\264\227\344\270\216\351\242\204\345\244\204\347\220\206/data_normalizer.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/09_\346\225\260\346\215\256\346\270\205\346\264\227\344\270\216\351\242\204\345\244\204\347\220\206/data_normalizer.py" new file mode 100644 index 0000000..8a1e4a0 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/09_\346\225\260\346\215\256\346\270\205\346\264\227\344\270\216\351\242\204\345\244\204\347\220\206/data_normalizer.py" @@ -0,0 +1,480 @@ +# -*- coding: utf-8 -*- +# @Desc: 数据标准化工具 + +import re +from datetime import datetime, timedelta +from typing import Optional, List, Dict, Any +from loguru import logger + + +class DateTimeNormalizer: + """日期时间标准化器""" + + # 常见日期格式 + DATE_FORMATS = [ + '%Y-%m-%d', + '%Y/%m/%d', + '%Y.%m.%d', + '%Y年%m月%d日', + '%d-%m-%Y', + '%d/%m/%Y', + '%m-%d-%Y', + '%m/%d/%Y', + ] + + # 常见日期时间格式 + DATETIME_FORMATS = [ + '%Y-%m-%d %H:%M:%S', + '%Y-%m-%d %H:%M', + '%Y/%m/%d %H:%M:%S', + '%Y/%m/%d %H:%M', + '%Y年%m月%d日 %H:%M:%S', + '%Y年%m月%d日 %H:%M', + '%Y年%m月%d日 %H时%M分', + '%Y年%m月%d日 %H时%M分%S秒', + '%d/%m/%Y %H:%M:%S', + ] + + @classmethod + def parse(cls, text: str) -> Optional[datetime]: + """ + 解析日期时间字符串 + + Args: + text: 日期时间字符串 + + Returns: + datetime 对象，解析失败返回 None + """ + text = text.strip() + + # 先尝试完整日期时间格式 + for fmt in cls.DATETIME_FORMATS: + try: + return datetime.strptime(text, fmt) + except ValueError: + continue + + # 再尝试日期格式 + for fmt in cls.DATE_FORMATS: + try: + return datetime.strptime(text, fmt) + except ValueError: + continue + + # 尝试解析相对时间 + relative = cls.parse_relative(text) + if relative: + return relative + + return None + + @classmethod + def parse_relative(cls, text: str) -> Optional[datetime]: + """ + 解析相对时间（如"3小时前"） + + Args: + text: 相对时间字符串 + + Returns: + datetime 对象 + """ + now = datetime.now() + text = text.strip() + + patterns = [ + (r'(\d+)\s*秒前', lambda m: now - timedelta(seconds=int(m.group(1)))), + (r'(\d+)\s*分钟前', lambda m: now - timedelta(minutes=int(m.group(1)))), + (r'(\d+)\s*小时前', lambda m: now - timedelta(hours=int(m.group(1)))), + (r'(\d+)\s*天前', lambda m: now - timedelta(days=int(m.group(1)))), + (r'(\d+)\s*周前', lambda m: now - timedelta(weeks=int(m.group(1)))), + (r'(\d+)\s*月前', lambda m: now - timedelta(days=int(m.group(1)) * 30)), + (r'(\d+)\s*年前', lambda m: now - timedelta(days=int(m.group(1)) * 365)), + (r'刚刚', lambda m: now), + (r'刚才', lambda m: now - timedelta(minutes=1)), + (r'昨天', lambda m: now - timedelta(days=1)), + (r'前天', lambda m: now - timedelta(days=2)), + (r'上周', lambda m: now - timedelta(weeks=1)), + (r'上个月', lambda m: now - timedelta(days=30)), + (r'去年', lambda m: now - timedelta(days=365)), + ] + + for pattern, handler in patterns: + match = re.search(pattern, text) + if match: + return handler(match) + + return None + + @classmethod + def normalize( + cls, + text: str, + output_format: str = '%Y-%m-%d %H:%M:%S' + ) -> str: + """ + 标准化日期时间格式 + + Args: + text: 日期时间字符串 + output_format: 输出格式 + + Returns: + 标准化后的字符串，解析失败返回原字符串 + """ + dt = cls.parse(text) + if dt: + return dt.strftime(output_format) + return text + + @classmethod + def normalize_date(cls, text: str) -> str: + """标准化为日期格式 (YYYY-MM-DD)""" + return cls.normalize(text, '%Y-%m-%d') + + @classmethod + def normalize_datetime(cls, text: str) -> str: + """标准化为日期时间格式 (YYYY-MM-DD HH:MM:SS)""" + return cls.normalize(text, '%Y-%m-%d %H:%M:%S') + + +class NumberNormalizer: + """数值标准化器""" + + # 中文数字单位 + CHINESE_UNITS = { + '万': 10000, + '亿': 100000000, + '兆': 1000000000000, + } + + # 英文数字单位 + ENGLISH_UNITS = { + 'k': 1000, + 'K': 1000, + 'm': 1000000, + 'M': 1000000, + 'b': 1000000000, + 'B': 1000000000, + } + + @classmethod + def parse(cls, text: str) -> float: + """ + 解析数字字符串 + + 支持： + - 逗号分隔：1,234,567 + - 中文单位：1.5万、3.2亿 + - 英文单位：1.5K、3.2M + + Args: + text: 数字字符串 + + Returns: + 数值 + """ + if not text: + return 0.0 + + text = str(text).strip() + + # 检查单位 + multiplier = 1 + + # 中文单位 + for unit, value in cls.CHINESE_UNITS.items(): + if unit in text: + multiplier = value + text = text.replace(unit, '') + break + + # 英文单位 + for unit, value in cls.ENGLISH_UNITS.items(): + if unit in text: + multiplier = value + text = text.replace(unit, '') + break + + # 移除货币符号 + text = re.sub(r'[¥$￥€£]', '', text) + + # 移除逗号 + text = text.replace(',', '') + + # 移除空格 + text = text.replace(' ', '') + + # 提取数字 + match = re.search(r'-?\d+\.?\d*', text) + if match: + return float(match.group()) * multiplier + + return 0.0 + + @classmethod + def format( + cls, + value: float, + precision: int = 2, + use_units: bool = True, + lang: str = 'zh' + ) -> str: + """ + 格式化数字 + + Args: + value: 数值 + precision: 小数位数 + use_units: 是否使用单位 + lang: 语言 (zh/en) + + Returns: + 格式化后的字符串 + """ + if not use_units: + return f'{value:.{precision}f}' + + if lang == 'zh': + if abs(value) >= 100000000: + return f'{value/100000000:.{precision}f}亿' + elif abs(value) >= 10000: + return f'{value/10000:.{precision}f}万' + else: + if abs(value) >= 1000000000: + return f'{value/1000000000:.{precision}f}B' + elif abs(value) >= 1000000: + return f'{value/1000000:.{precision}f}M' + elif abs(value) >= 1000: + return f'{value/1000:.{precision}f}K' + + return f'{value:.{precision}f}' + + @classmethod + def format_with_comma(cls, value: float, precision: int = 0) -> str: + """ + 使用逗号分隔的格式 + + Args: + value: 数值 + precision: 小数位数 + + Returns: + 逗号分隔的数字字符串 + """ + if precision == 0: + return f'{int(value):,}' + return f'{value:,.{precision}f}' + + +class TextNormalizer: + """文本标准化器""" + + @staticmethod + def normalize_case(text: str, case: str = 'lower') -> str: + """ + 标准化大小写 + + Args: + text: 输入文本 + case: 大小写类型 (lower/upper/title/capitalize) + + Returns: + 标准化后的文本 + """ + if case == 'lower': + return text.lower() + elif case == 'upper': + return text.upper() + elif case == 'title': + return text.title() + elif case == 'capitalize': + return text.capitalize() + return text + + @staticmethod + def normalize_punctuation(text: str, style: str = 'english') -> str: + """ + 标准化标点符号 + + Args: + text: 输入文本 + style: 标点风格 (chinese/english) + + Returns: + 标准化后的文本 + """ + if style == 'english': + # 中文标点转英文 + mapping = { + '，': ', ', + '。': '. ', + '！': '! ', + '？': '? ', + '；': '; ', + '：': ': ', + '"': '"', + '"': '"', + ''': "'", + ''': "'", + '（': '(', + '）': ')', + '【': '[', + '】': ']', + } + else: + # 英文标点转中文 + mapping = { + ',': '，', + '.': '。', + '!': '！', + '?': '？', + ';': '；', + ':': '：', + '"': '"', + "'": "'", + '(': '（', + ')': '）', + '[': '【', + ']': '】', + } + + for old, new in mapping.items(): + text = text.replace(old, new) + + return text + + +class DataNormalizer: + """综合数据标准化器""" + + def __init__(self): + self.date_normalizer = DateTimeNormalizer() + self.number_normalizer = NumberNormalizer() + self.text_normalizer = TextNormalizer() + + def normalize_record( + self, + record: Dict[str, Any], + date_fields: List[str] = None, + number_fields: List[str] = None, + text_fields: List[str] = None + ) -> Dict[str, Any]: + """ + 标准化数据记录 + + Args: + record: 数据记录 + date_fields: 需要标准化的日期字段 + number_fields: 需要标准化的数值字段 + text_fields: 需要标准化的文本字段 + + Returns: + 标准化后的记录 + """ + result = record.copy() + + # 日期标准化 + if date_fields: + for field in date_fields: + if field in result and result[field]: + result[field] = DateTimeNormalizer.normalize_datetime( + str(result[field]) + ) + + # 数值标准化 + if number_fields: + for field in number_fields: + if field in result: + result[f'{field}_normalized'] = NumberNormalizer.parse( + str(result[field]) + ) + + # 文本标准化 + if text_fields: + for field in text_fields: + if field in result and result[field]: + result[field] = TextNormalizer.normalize_case( + str(result[field]).strip() + ) + + return result + + def normalize_batch( + self, + records: List[Dict], + **kwargs + ) -> List[Dict]: + """批量标准化""" + return [self.normalize_record(r, **kwargs) for r in records] + + +def demo(): + """数据标准化演示""" + print("=" * 50) + print("数据标准化工具演示") + print("=" * 50) + + # 1. 日期标准化 + print("\n1. 日期时间标准化:") + date_tests = [ + "2024年1月15日", + "2024/01/15", + "15-01-2024", + "3小时前", + "昨天", + ] + for dt in date_tests: + normalized = DateTimeNormalizer.normalize_datetime(dt) + print(f" '{dt}' -> '{normalized}'") + + # 2. 数值标准化 + print("\n2. 数值标准化:") + number_tests = [ + "1,234,567", + "1.5万", + "3.2亿", + "1.5K", + "￥99.00", + "2.5M" + ] + for num in number_tests: + parsed = NumberNormalizer.parse(num) + formatted = NumberNormalizer.format(parsed) + print(f" '{num}' -> {parsed} -> '{formatted}'") + + # 3. 文本标准化 + print("\n3. 文本标准化:") + text = "HELLO world" + print(f" 原始: '{text}'") + print(f" lower: '{TextNormalizer.normalize_case(text, 'lower')}'") + print(f" upper: '{TextNormalizer.normalize_case(text, 'upper')}'") + print(f" title: '{TextNormalizer.normalize_case(text, 'title')}'") + + # 4. 综合标准化 + print("\n4. 综合数据标准化:") + test_record = { + "title": " Python 教程 ", + "date": "2024年1月15日", + "views": "1.5万", + "price": "￥99.00" + } + print(f" 原始: {test_record}") + + normalizer = DataNormalizer() + normalized = normalizer.normalize_record( + test_record, + date_fields=["date"], + number_fields=["views", "price"], + text_fields=["title"] + ) + print(f" 标准化: {normalized}") + + print("\n" + "=" * 50) + print("演示完成") + print("=" * 50) + + +if __name__ == "__main__": + demo() diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/09_\346\225\260\346\215\256\346\270\205\346\264\227\344\270\216\351\242\204\345\244\204\347\220\206/deduplication.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/09_\346\225\260\346\215\256\346\270\205\346\264\227\344\270\216\351\242\204\345\244\204\347\220\206/deduplication.py" new file mode 100644 index 0000000..c07247b --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/09_\346\225\260\346\215\256\346\270\205\346\264\227\344\270\216\351\242\204\345\244\204\347\220\206/deduplication.py" @@ -0,0 +1,458 @@ +# -*- coding: utf-8 -*- +# @Desc: 数据去重工具 + +import hashlib +from typing import List, Dict, Any, Callable, Set +from loguru import logger + + +class ExactDeduplicator: + """精确去重器""" + + @staticmethod + def dedupe_list(items: List[str]) -> List[str]: + """ + 列表去重（保持顺序） + + Args: + items: 字符串列表 + + Returns: + 去重后的列表 + """ + seen: Set[str] = set() + result = [] + for item in items: + if item not in seen: + seen.add(item) + result.append(item) + return result + + @staticmethod + def dedupe_dicts_by_field( + items: List[Dict], + key_field: str + ) -> List[Dict]: + """ + 根据单个字段去重 + + Args: + items: 字典列表 + key_field: 用于去重的字段名 + + Returns: + 去重后的列表 + """ + seen: Set[Any] = set() + result = [] + for item in items: + key = item.get(key_field) + if key not in seen: + seen.add(key) + result.append(item) + return result + + @staticmethod + def dedupe_by_hash( + items: List[Dict], + fields: List[str] + ) -> List[Dict]: + """ + 根据多个字段计算哈希去重 + + Args: + items: 数据列表 + fields: 用于计算哈希的字段列表 + + Returns: + 去重后的列表 + """ + seen: Set[str] = set() + result = [] + + for item in items: + # 构建哈希键 + key_parts = [str(item.get(f, '')) for f in fields] + key_str = '|'.join(key_parts) + key_hash = hashlib.md5(key_str.encode()).hexdigest() + + if key_hash not in seen: + seen.add(key_hash) + result.append(item) + + return result + + @staticmethod + def dedupe_by_callback( + items: List[Any], + key_func: Callable[[Any], Any] + ) -> List[Any]: + """ + 使用自定义函数生成键进行去重 + + Args: + items: 数据列表 + key_func: 生成去重键的函数 + + Returns: + 去重后的列表 + """ + seen: Set[Any] = set() + result = [] + + for item in items: + key = key_func(item) + if key not in seen: + seen.add(key) + result.append(item) + + return result + + +class FuzzyDeduplicator: + """模糊去重器（基于相似度）""" + + @staticmethod + def levenshtein_distance(s1: str, s2: str) -> int: + """ + 计算编辑距离（Levenshtein Distance） + + 编辑距离是将一个字符串转换为另一个所需的最少操作数 + + Args: + s1: 字符串1 + s2: 字符串2 + + Returns: + 编辑距离 + """ + if len(s1) < len(s2): + return FuzzyDeduplicator.levenshtein_distance(s2, s1) + + if len(s2) == 0: + return len(s1) + + previous_row = list(range(len(s2) + 1)) + + for i, c1 in enumerate(s1): + current_row = [i + 1] + for j, c2 in enumerate(s2): + # 插入、删除、替换的代价 + insertions = previous_row[j + 1] + 1 + deletions = current_row[j] + 1 + substitutions = previous_row[j] + (c1 != c2) + current_row.append(min(insertions, deletions, substitutions)) + previous_row = current_row + + return previous_row[-1] + + @staticmethod + def similarity(s1: str, s2: str) -> float: + """ + 计算两个字符串的相似度 + + 相似度 = 1 - (编辑距离 / 最大长度) + + Args: + s1: 字符串1 + s2: 字符串2 + + Returns: + 相似度 (0.0 - 1.0) + """ + if not s1 and not s2: + return 1.0 + if not s1 or not s2: + return 0.0 + + distance = FuzzyDeduplicator.levenshtein_distance(s1, s2) + max_len = max(len(s1), len(s2)) + return 1 - distance / max_len + + @staticmethod + def jaccard_similarity(s1: str, s2: str) -> float: + """ + 计算 Jaccard 相似度（基于字符集合） + + Jaccard = |A ∩ B| / |A ∪ B| + + Args: + s1: 字符串1 + s2: 字符串2 + + Returns: + Jaccard 相似度 (0.0 - 1.0) + """ + if not s1 and not s2: + return 1.0 + if not s1 or not s2: + return 0.0 + + set1 = set(s1) + set2 = set(s2) + + intersection = len(set1 & set2) + union = len(set1 | set2) + + return intersection / union if union > 0 else 0.0 + + @staticmethod + def dedupe_fuzzy( + items: List[str], + threshold: float = 0.8, + similarity_func: str = "levenshtein" + ) -> List[str]: + """ + 模糊去重 + + Args: + items: 字符串列表 + threshold: 相似度阈值 (0.0 - 1.0) + similarity_func: 相似度算法 (levenshtein/jaccard) + + Returns: + 去重后的列表 + """ + if not items: + return [] + + # 选择相似度函数 + if similarity_func == "jaccard": + sim_func = FuzzyDeduplicator.jaccard_similarity + else: + sim_func = FuzzyDeduplicator.similarity + + result = [items[0]] + + for item in items[1:]: + is_duplicate = False + for existing in result: + if sim_func(item, existing) >= threshold: + is_duplicate = True + break + + if not is_duplicate: + result.append(item) + + return result + + @staticmethod + def dedupe_dicts_fuzzy( + items: List[Dict], + text_field: str, + threshold: float = 0.8 + ) -> List[Dict]: + """ + 根据文本字段进行模糊去重 + + Args: + items: 字典列表 + text_field: 用于比较的文本字段 + threshold: 相似度阈值 + + Returns: + 去重后的列表 + """ + if not items: + return [] + + result = [items[0]] + + for item in items[1:]: + item_text = item.get(text_field, '') + is_duplicate = False + + for existing in result: + existing_text = existing.get(text_field, '') + if FuzzyDeduplicator.similarity(item_text, existing_text) >= threshold: + is_duplicate = True + break + + if not is_duplicate: + result.append(item) + + return result + + +class ContentDeduplicator: + """基于内容特征的去重器""" + + @staticmethod + def get_content_hash(text: str, normalize: bool = True) -> str: + """ + 计算内容哈希 + + Args: + text: 文本内容 + normalize: 是否标准化（去除空白等） + + Returns: + MD5 哈希值 + """ + if normalize: + # 移除空白字符 + text = ''.join(text.split()) + # 转小写 + text = text.lower() + + return hashlib.md5(text.encode()).hexdigest() + + @staticmethod + def get_simhash(text: str, bits: int = 64) -> int: + """ + 计算 SimHash（用于大规模去重） + + SimHash 是一种局部敏感哈希，相似文本的哈希值相近 + + Args: + text: 文本内容 + bits: 哈希位数 + + Returns: + SimHash 值 + """ + # 分词（简单按字符） + features = list(text) + + # 计算每个特征的哈希 + v = [0] * bits + for feature in features: + h = int(hashlib.md5(feature.encode()).hexdigest(), 16) + for i in range(bits): + bitmask = 1 << i + if h & bitmask: + v[i] += 1 + else: + v[i] -= 1 + + # 生成最终哈希 + fingerprint = 0 + for i in range(bits): + if v[i] >= 0: + fingerprint |= (1 << i) + + return fingerprint + + @staticmethod + def hamming_distance(hash1: int, hash2: int) -> int: + """ + 计算汉明距离 + + Args: + hash1: 哈希值1 + hash2: 哈希值2 + + Returns: + 汉明距离（不同位的数量） + """ + x = hash1 ^ hash2 + distance = 0 + while x: + distance += 1 + x &= x - 1 + return distance + + @staticmethod + def dedupe_by_simhash( + items: List[Dict], + text_field: str, + threshold: int = 3 + ) -> List[Dict]: + """ + 使用 SimHash 去重 + + 适用于大规模文本去重 + + Args: + items: 字典列表 + text_field: 文本字段名 + threshold: 汉明距离阈值（小于此值视为重复） + + Returns: + 去重后的列表 + """ + if not items: + return [] + + result = [] + hashes = [] + + for item in items: + text = item.get(text_field, '') + item_hash = ContentDeduplicator.get_simhash(text) + + is_duplicate = False + for existing_hash in hashes: + if ContentDeduplicator.hamming_distance(item_hash, existing_hash) <= threshold: + is_duplicate = True + break + + if not is_duplicate: + result.append(item) + hashes.append(item_hash) + + return result + + +def demo(): + """去重工具演示""" + print("=" * 50) + print("数据去重工具演示") + print("=" * 50) + + # 1. 精确去重 + print("\n1. 精确去重:") + test_list = ["apple", "banana", "apple", "cherry", "banana"] + print(f" 原始: {test_list}") + deduped = ExactDeduplicator.dedupe_list(test_list) + print(f" 去重: {deduped}") + + # 2. 字典列表去重 + print("\n2. 字典列表去重:") + test_dicts = [ + {"id": 1, "name": "Alice"}, + {"id": 2, "name": "Bob"}, + {"id": 1, "name": "Alice Copy"}, # 重复 ID + ] + print(f" 原始: {test_dicts}") + deduped_dicts = ExactDeduplicator.dedupe_dicts_by_field(test_dicts, "id") + print(f" 去重: {deduped_dicts}") + + # 3. 模糊去重 + print("\n3. 模糊去重:") + similar_items = [ + "Python 爬虫教程", + "Python 爬虫入门教程", # 相似 + "Java 编程指南", + "Java 编程入门指南" # 相似 + ] + print(f" 原始: {similar_items}") + fuzzy_deduped = FuzzyDeduplicator.dedupe_fuzzy(similar_items, threshold=0.7) + print(f" 去重(阈值0.7): {fuzzy_deduped}") + + # 4. 相似度计算 + print("\n4. 相似度计算:") + s1 = "Hello World" + s2 = "Hello World!" + s3 = "Goodbye World" + print(f" '{s1}' vs '{s2}': {FuzzyDeduplicator.similarity(s1, s2):.2f}") + print(f" '{s1}' vs '{s3}': {FuzzyDeduplicator.similarity(s1, s3):.2f}") + + # 5. SimHash + print("\n5. SimHash 去重:") + text1 = "这是一篇关于 Python 的文章" + text2 = "这是一篇关于 Python 的教程" # 相似 + text3 = "Java 是一门编程语言" + hash1 = ContentDeduplicator.get_simhash(text1) + hash2 = ContentDeduplicator.get_simhash(text2) + hash3 = ContentDeduplicator.get_simhash(text3) + print(f" text1-text2 汉明距离: {ContentDeduplicator.hamming_distance(hash1, hash2)}") + print(f" text1-text3 汉明距离: {ContentDeduplicator.hamming_distance(hash1, hash3)}") + + print("\n" + "=" * 50) + print("演示完成") + print("=" * 50) + + +if __name__ == "__main__": + demo() diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/09_\346\225\260\346\215\256\346\270\205\346\264\227\344\270\216\351\242\204\345\244\204\347\220\206/pyproject.toml" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/09_\346\225\260\346\215\256\346\270\205\346\264\227\344\270\216\351\242\204\345\244\204\347\220\206/pyproject.toml" new file mode 100644 index 0000000..2d9bfa1 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/09_\346\225\260\346\215\256\346\270\205\346\264\227\344\270\216\351\242\204\345\244\204\347\220\206/pyproject.toml" @@ -0,0 +1,21 @@ +[project] +name = "chapter09-data-cleaning" +version = "0.1.0" +description = "第09章：数据清洗与预处理 - 文本清洗、数据标准化、去重算法" +readme = "README.md" +requires-python = ">=3.11" +dependencies = [ + # 本章主要使用标准库，无特殊依赖 +] + +[project.optional-dependencies] +simhash = [ + "simhash>=2.1.0", # SimHash去重（可选） +] + +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[tool.uv] +dev-dependencies = [] diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/09_\346\225\260\346\215\256\346\270\205\346\264\227\344\270\216\351\242\204\345\244\204\347\220\206/text_cleaner.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/09_\346\225\260\346\215\256\346\270\205\346\264\227\344\270\216\351\242\204\345\244\204\347\220\206/text_cleaner.py" new file mode 100644 index 0000000..64213cc --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/09_\346\225\260\346\215\256\346\270\205\346\264\227\344\270\216\351\242\204\345\244\204\347\220\206/text_cleaner.py" @@ -0,0 +1,458 @@ +# -*- coding: utf-8 -*- +# @Desc: 文本清洗工具 + +import re +import unicodedata +from typing import List +from loguru import logger + +# 可选依赖 +try: + from bs4 import BeautifulSoup + HAS_BS4 = True +except ImportError: + HAS_BS4 = False + +try: + import chardet + HAS_CHARDET = True +except ImportError: + HAS_CHARDET = False + + +class HTMLCleaner: + """HTML 清洗器""" + + # 需要完全移除的标签（包括内容） + REMOVE_TAGS = ['script', 'style', 'head', 'meta', 'link', 'noscript'] + + @staticmethod + def remove_tags(html: str) -> str: + """ + 移除所有 HTML 标签 + + Args: + html: HTML 文本 + + Returns: + 纯文本 + """ + # 先移除特定标签及其内容 + for tag in HTMLCleaner.REMOVE_TAGS: + pattern = f'<{tag}[^>]*>.*?' + html = re.sub(pattern, '', html, flags=re.DOTALL | re.IGNORECASE) + + # 移除所有标签 + html = re.sub(r'<[^>]+>', '', html) + + return html + + @staticmethod + def remove_tags_keep_structure(html: str) -> str: + """ + 移除标签但保留结构（块级元素转换行） + + Args: + html: HTML 文本 + + Returns: + 保留换行结构的纯文本 + """ + # 处理块级元素，添加换行 + block_pattern = r'' + html = re.sub(block_pattern, '\n', html, flags=re.IGNORECASE) + + # 处理
标签 + html = re.sub(r'', '\n', html, flags=re.IGNORECASE) + + # 移除其他标签 + html = re.sub(r'<[^>]+>', '', html) + + return html + + @staticmethod + def decode_entities(text: str) -> str: + """ + 解码 HTML 实体 + + Args: + text: 包含 HTML 实体的文本 + + Returns: + 解码后的文本 + """ + import html + return html.unescape(text) + + @staticmethod + def clean_with_bs4(html: str) -> str: + """ + 使用 BeautifulSoup 清洗 HTML（推荐） + + Args: + html: HTML 文本 + + Returns: + 纯文本 + """ + if not HAS_BS4: + logger.warning("BeautifulSoup 未安装，使用正则清洗") + return HTMLCleaner.remove_tags(html) + + soup = BeautifulSoup(html, 'html.parser') + + # 移除脚本和样式 + for element in soup(['script', 'style', 'head', 'meta', 'link']): + element.decompose() + + # 获取文本，使用换行分隔 + text = soup.get_text(separator='\n') + + return text + + +class WhitespaceCleaner: + """空白字符清洗器""" + + @staticmethod + def normalize(text: str) -> str: + """ + 标准化空白字符 + + - 将制表符、回车等转为空格 + - 合并多个空格 + - 合并多个换行 + - 去除首尾空白 + + Args: + text: 输入文本 + + Returns: + 标准化后的文本 + """ + # 将各种空白字符转为普通空格 + text = re.sub(r'[\t\r\f\v]', ' ', text) + # 合并多个空格 + text = re.sub(r' +', ' ', text) + # 合并多个换行 + text = re.sub(r'\n+', '\n', text) + # 去除首尾空白 + return text.strip() + + @staticmethod + def remove_all(text: str) -> str: + """移除所有空白字符""" + return re.sub(r'\s+', '', text) + + @staticmethod + def trim_lines(text: str) -> str: + """去除每行首尾空白""" + lines = text.split('\n') + return '\n'.join(line.strip() for line in lines) + + @staticmethod + def remove_empty_lines(text: str) -> str: + """移除空行""" + lines = text.split('\n') + return '\n'.join(line for line in lines if line.strip()) + + @staticmethod + def collapse_whitespace(text: str) -> str: + """将所有连续空白合并为单个空格""" + return re.sub(r'\s+', ' ', text).strip() + + +class SpecialCharCleaner: + """特殊字符清洗器""" + + @staticmethod + def remove_control_chars(text: str) -> str: + """ + 移除控制字符 + + 控制字符是 Unicode 类别为 'Cc' 的字符 + """ + return ''.join( + char for char in text + if unicodedata.category(char) != 'Cc' + ) + + @staticmethod + def normalize_unicode(text: str, form: str = 'NFKC') -> str: + """ + Unicode 标准化 + + Args: + text: 输入文本 + form: 标准化形式 + - NFC: 标准分解后标准合成 + - NFD: 标准分解 + - NFKC: 兼容分解后标准合成（推荐） + - NFKD: 兼容分解 + + Returns: + 标准化后的文本 + """ + return unicodedata.normalize(form, text) + + @staticmethod + def remove_emojis(text: str) -> str: + """移除 emoji 表情""" + emoji_pattern = re.compile( + "[" + "\U0001F600-\U0001F64F" # 表情符号 + "\U0001F300-\U0001F5FF" # 符号和象形文字 + "\U0001F680-\U0001F6FF" # 交通和地图符号 + "\U0001F1E0-\U0001F1FF" # 旗帜 + "\U00002702-\U000027B0" # 装饰符号 + "\U000024C2-\U0001F251" # 封闭字符 + "]+", + flags=re.UNICODE + ) + return emoji_pattern.sub('', text) + + @staticmethod + def to_halfwidth(text: str) -> str: + """ + 全角字符转半角 + + 将全角数字、字母、标点转为半角 + """ + result = [] + for char in text: + code = ord(char) + # 全角空格 + if code == 0x3000: + result.append(' ') + # 其他全角字符 (！到～) + elif 0xFF01 <= code <= 0xFF5E: + result.append(chr(code - 0xFEE0)) + else: + result.append(char) + return ''.join(result) + + @staticmethod + def to_fullwidth(text: str) -> str: + """ + 半角字符转全角 + + 将半角数字、字母、标点转为全角 + """ + result = [] + for char in text: + code = ord(char) + # 空格 + if code == 0x20: + result.append('\u3000') + # 其他半角字符 (!到~) + elif 0x21 <= code <= 0x7E: + result.append(chr(code + 0xFEE0)) + else: + result.append(char) + return ''.join(result) + + @staticmethod + def remove_punctuation(text: str, keep_chinese: bool = True) -> str: + """ + 移除标点符号 + + Args: + text: 输入文本 + keep_chinese: 是否保留中文标点 + + Returns: + 移除标点后的文本 + """ + if keep_chinese: + # 只移除英文标点 + return re.sub(r'[!"#$%&\'()*+,-./:;<=>?@\[\]\\^_`{|}~]', '', text) + else: + # 移除所有标点 + return re.sub(r'[^\w\s]', '', text, flags=re.UNICODE) + + +class EncodingFixer: + """编码问题修复器""" + + @staticmethod + def detect_encoding(data: bytes) -> str: + """ + 检测字节数据的编码 + + Args: + data: 字节数据 + + Returns: + 检测到的编码名称 + """ + if not HAS_CHARDET: + logger.warning("chardet 未安装，默认使用 utf-8") + return 'utf-8' + + result = chardet.detect(data) + return result.get('encoding') or 'utf-8' + + @staticmethod + def safe_decode(data: bytes, fallback: str = 'utf-8') -> str: + """ + 安全解码字节数据 + + 自动检测编码，解码失败则使用 fallback + + Args: + data: 字节数据 + fallback: 备用编码 + + Returns: + 解码后的字符串 + """ + detected = EncodingFixer.detect_encoding(data) + try: + return data.decode(detected) + except (UnicodeDecodeError, TypeError, LookupError): + return data.decode(fallback, errors='ignore') + + @staticmethod + def fix_mojibake(text: str) -> str: + """ + 修复乱码（尝试常见编码） + + Args: + text: 可能乱码的文本 + + Returns: + 修复后的文本 + """ + encodings = ['utf-8', 'gbk', 'gb2312', 'gb18030', 'big5'] + + for encoding in encodings: + try: + # 尝试将文本按 latin1 编码，再用目标编码解码 + fixed = text.encode('latin1').decode(encoding) + # 检查是否包含中文（简单验证） + if re.search(r'[\u4e00-\u9fa5]', fixed): + return fixed + except (UnicodeDecodeError, UnicodeEncodeError): + continue + + return text + + +class TextCleaner: + """综合文本清洗器""" + + def __init__( + self, + remove_html: bool = True, + normalize_whitespace: bool = True, + normalize_unicode: bool = True, + to_halfwidth: bool = True, + remove_emojis: bool = False, + remove_control_chars: bool = True + ): + """ + Args: + remove_html: 是否移除 HTML 标签 + normalize_whitespace: 是否标准化空白 + normalize_unicode: 是否 Unicode 标准化 + to_halfwidth: 是否全角转半角 + remove_emojis: 是否移除 emoji + remove_control_chars: 是否移除控制字符 + """ + self.remove_html = remove_html + self.normalize_whitespace = normalize_whitespace + self.normalize_unicode = normalize_unicode + self.to_halfwidth = to_halfwidth + self.remove_emojis = remove_emojis + self.remove_control_chars = remove_control_chars + + def clean(self, text: str) -> str: + """ + 执行完整的文本清洗 + + Args: + text: 原始文本 + + Returns: + 清洗后的文本 + """ + if not text: + return '' + + # 1. HTML 清洗 + if self.remove_html: + text = HTMLCleaner.clean_with_bs4(text) + text = HTMLCleaner.decode_entities(text) + + # 2. Unicode 标准化 + if self.normalize_unicode: + text = SpecialCharCleaner.normalize_unicode(text, 'NFKC') + + # 3. 移除控制字符 + if self.remove_control_chars: + text = SpecialCharCleaner.remove_control_chars(text) + + # 4. 全角转半角 + if self.to_halfwidth: + text = SpecialCharCleaner.to_halfwidth(text) + + # 5. 移除 emoji + if self.remove_emojis: + text = SpecialCharCleaner.remove_emojis(text) + + # 6. 空白标准化 + if self.normalize_whitespace: + text = WhitespaceCleaner.normalize(text) + + return text + + +def demo(): + """演示文本清洗功能""" + print("=" * 50) + print("文本清洗工具演示") + print("=" * 50) + + # 测试数据 + test_html = """ + + 测试 + + +

这是一段　　HTML　　文本。

+

包含&特殊<字符>。

+ + + """ + + print("\n1. HTML 清洗:") + print(f" 原始: {test_html[:50]}...") + cleaned = HTMLCleaner.clean_with_bs4(test_html) + print(f" 清洗后: {cleaned[:50]}...") + + print("\n2. 空白处理:") + test_whitespace = " 多个 空格 \n\n\n多个换行 " + print(f" 原始: '{test_whitespace}'") + normalized = WhitespaceCleaner.normalize(test_whitespace) + print(f" 标准化: '{normalized}'") + + print("\n3. 全角转半角:") + test_fullwidth = "ＡＢＣ１２３　全角字符" + print(f" 原始: '{test_fullwidth}'") + halfwidth = SpecialCharCleaner.to_halfwidth(test_fullwidth) + print(f" 半角: '{halfwidth}'") + + print("\n4. 综合清洗:") + cleaner = TextCleaner() + test_complex = "

　　复杂的　HTML&文本　　

" + print(f" 原始: '{test_complex}'") + result = cleaner.clean(test_complex) + print(f" 清洗: '{result}'") + + print("\n" + "=" * 50) + print("演示完成") + print("=" * 50) + + +if __name__ == "__main__": + demo() diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/10_\346\225\260\346\215\256\345\210\206\346\236\220\344\270\216\345\217\257\350\247\206\345\214\226/README.md" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/10_\346\225\260\346\215\256\345\210\206\346\236\220\344\270\216\345\217\257\350\247\206\345\214\226/README.md" new file mode 100644 index 0000000..af8b599 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/10_\346\225\260\346\215\256\345\210\206\346\236\220\344\270\216\345\217\257\350\247\206\345\214\226/README.md" @@ -0,0 +1,27 @@ +# 第10章：数据分析与可视化 + +展示Pandas数据分析、词云生成、图表制作等数据处理功能。 + +## 快速开始 + +```bash +cd 10_数据分析与可视化 + +# 安装基础依赖 +uv sync + +# 安装交互式图表功能（可选） +uv sync --extra interactive + +# 运行示例 +uv run python pandas_analysis.py +uv run python wordcloud_generator.py +uv run python chart_demo.py +``` + +### 核心依赖 +- `pandas` - 数据分析 +- `jieba` - 中文分词 +- `wordcloud` - 词云生成 +- `matplotlib` - 图表绘制 +- `pyecharts`（可选）- 交互式图表 diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/10_\346\225\260\346\215\256\345\210\206\346\236\220\344\270\216\345\217\257\350\247\206\345\214\226/chart_demo.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/10_\346\225\260\346\215\256\345\210\206\346\236\220\344\270\216\345\217\257\350\247\206\345\214\226/chart_demo.py" new file mode 100644 index 0000000..36617c1 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/10_\346\225\260\346\215\256\345\210\206\346\236\220\344\270\216\345\217\257\350\247\206\345\214\226/chart_demo.py" @@ -0,0 +1,582 @@ +# -*- coding: utf-8 -*- +# @Desc: 数据可视化工具 + +import os +from typing import List, Dict, Any, Optional, Tuple +from loguru import logger + +# 可选依赖 +try: + import matplotlib.pyplot as plt + import matplotlib + matplotlib.rcParams['font.sans-serif'] = ['SimHei', 'Arial Unicode MS', 'DejaVu Sans'] + matplotlib.rcParams['axes.unicode_minus'] = False + HAS_MATPLOTLIB = True +except ImportError: + HAS_MATPLOTLIB = False + logger.warning("matplotlib 未安装") + +try: + from pyecharts.charts import Bar, Line, Pie, Scatter, WordCloud as PyWordCloud + from pyecharts import options as opts + from pyecharts.globals import ThemeType + HAS_PYECHARTS = True +except ImportError: + HAS_PYECHARTS = False + logger.warning("pyecharts 未安装") + + +class MatplotlibCharts: + """Matplotlib 静态图表生成器""" + + def __init__(self, figsize: Tuple[int, int] = (10, 6), dpi: int = 150): + """ + 初始化 + + Args: + figsize: 图表大小 (宽, 高) + dpi: 分辨率 + """ + if not HAS_MATPLOTLIB: + raise ImportError("请安装 matplotlib: pip install matplotlib") + + self.figsize = figsize + self.dpi = dpi + + def line_chart( + self, + x_data: List, + y_data: List, + title: str, + xlabel: str, + ylabel: str, + output_path: str, + color: str = 'steelblue', + marker: str = 'o' + ) -> str: + """绘制折线图""" + plt.figure(figsize=self.figsize) + plt.plot(x_data, y_data, marker=marker, linewidth=2, markersize=6, color=color) + plt.title(title, fontsize=14) + plt.xlabel(xlabel, fontsize=12) + plt.ylabel(ylabel, fontsize=12) + plt.grid(True, alpha=0.3) + plt.tight_layout() + plt.savefig(output_path, dpi=self.dpi) + plt.close() + logger.info(f"折线图已保存: {output_path}") + return output_path + + def multi_line_chart( + self, + x_data: List, + y_data_dict: Dict[str, List], + title: str, + xlabel: str, + ylabel: str, + output_path: str + ) -> str: + """绘制多条折线图""" + plt.figure(figsize=self.figsize) + + for label, y_data in y_data_dict.items(): + plt.plot(x_data, y_data, marker='o', label=label, linewidth=2) + + plt.title(title, fontsize=14) + plt.xlabel(xlabel, fontsize=12) + plt.ylabel(ylabel, fontsize=12) + plt.legend(loc='best') + plt.grid(True, alpha=0.3) + plt.tight_layout() + plt.savefig(output_path, dpi=self.dpi) + plt.close() + logger.info(f"多折线图已保存: {output_path}") + return output_path + + def bar_chart( + self, + categories: List[str], + values: List, + title: str, + xlabel: str, + ylabel: str, + output_path: str, + horizontal: bool = False, + color: str = 'steelblue' + ) -> str: + """绘制柱状图""" + plt.figure(figsize=self.figsize) + + if horizontal: + plt.barh(categories, values, color=color) + plt.xlabel(ylabel) + plt.ylabel(xlabel) + else: + plt.bar(categories, values, color=color) + plt.xlabel(xlabel) + plt.ylabel(ylabel) + + plt.title(title, fontsize=14) + plt.tight_layout() + plt.savefig(output_path, dpi=self.dpi) + plt.close() + logger.info(f"柱状图已保存: {output_path}") + return output_path + + def grouped_bar_chart( + self, + categories: List[str], + data_dict: Dict[str, List], + title: str, + xlabel: str, + ylabel: str, + output_path: str + ) -> str: + """绘制分组柱状图""" + import numpy as np + + plt.figure(figsize=self.figsize) + + x = np.arange(len(categories)) + width = 0.8 / len(data_dict) + + for i, (label, values) in enumerate(data_dict.items()): + offset = (i - len(data_dict) / 2 + 0.5) * width + plt.bar(x + offset, values, width, label=label) + + plt.xlabel(xlabel, fontsize=12) + plt.ylabel(ylabel, fontsize=12) + plt.title(title, fontsize=14) + plt.xticks(x, categories) + plt.legend() + plt.tight_layout() + plt.savefig(output_path, dpi=self.dpi) + plt.close() + logger.info(f"分组柱状图已保存: {output_path}") + return output_path + + def pie_chart( + self, + labels: List[str], + sizes: List, + title: str, + output_path: str, + explode_max: bool = True + ) -> str: + """绘制饼图""" + plt.figure(figsize=(10, 8)) + + explode = None + if explode_max: + max_idx = sizes.index(max(sizes)) + explode = [0.05 if i == max_idx else 0 for i in range(len(sizes))] + + plt.pie( + sizes, + labels=labels, + explode=explode, + autopct='%1.1f%%', + startangle=90, + colors=plt.cm.Set3.colors[:len(labels)] + ) + plt.title(title, fontsize=14) + plt.axis('equal') + plt.tight_layout() + plt.savefig(output_path, dpi=self.dpi) + plt.close() + logger.info(f"饼图已保存: {output_path}") + return output_path + + def scatter_chart( + self, + x_data: List, + y_data: List, + title: str, + xlabel: str, + ylabel: str, + output_path: str, + color: str = 'steelblue', + alpha: float = 0.6 + ) -> str: + """绘制散点图""" + plt.figure(figsize=self.figsize) + plt.scatter(x_data, y_data, c=color, alpha=alpha, s=50) + plt.title(title, fontsize=14) + plt.xlabel(xlabel, fontsize=12) + plt.ylabel(ylabel, fontsize=12) + plt.grid(True, alpha=0.3) + plt.tight_layout() + plt.savefig(output_path, dpi=self.dpi) + plt.close() + logger.info(f"散点图已保存: {output_path}") + return output_path + + def histogram( + self, + data: List, + title: str, + xlabel: str, + ylabel: str, + output_path: str, + bins: int = 20, + color: str = 'steelblue' + ) -> str: + """绘制直方图""" + plt.figure(figsize=self.figsize) + plt.hist(data, bins=bins, color=color, edgecolor='white', alpha=0.7) + plt.title(title, fontsize=14) + plt.xlabel(xlabel, fontsize=12) + plt.ylabel(ylabel, fontsize=12) + plt.grid(axis='y', alpha=0.3) + plt.tight_layout() + plt.savefig(output_path, dpi=self.dpi) + plt.close() + logger.info(f"直方图已保存: {output_path}") + return output_path + + +class PyechartsCharts: + """Pyecharts 交互式图表生成器""" + + def __init__(self, theme: str = 'light'): + """ + 初始化 + + Args: + theme: 主题名称 + """ + if not HAS_PYECHARTS: + raise ImportError("请安装 pyecharts: pip install pyecharts") + + self.theme = getattr(ThemeType, theme.upper(), ThemeType.LIGHT) + + def bar_chart( + self, + categories: List[str], + values: List, + title: str, + output_path: str, + series_name: str = "数量" + ) -> str: + """创建交互式柱状图""" + bar = ( + Bar(init_opts=opts.InitOpts(theme=self.theme)) + .add_xaxis(categories) + .add_yaxis(series_name, values) + .set_global_opts( + title_opts=opts.TitleOpts(title=title), + toolbox_opts=opts.ToolboxOpts(is_show=True), + datazoom_opts=opts.DataZoomOpts(is_show=True) + ) + ) + bar.render(output_path) + logger.info(f"交互式柱状图已保存: {output_path}") + return output_path + + def stacked_bar_chart( + self, + categories: List[str], + data_dict: Dict[str, List], + title: str, + output_path: str + ) -> str: + """创建堆叠柱状图""" + bar = Bar(init_opts=opts.InitOpts(theme=self.theme)) + bar.add_xaxis(categories) + + for name, values in data_dict.items(): + bar.add_yaxis(name, values, stack="stack1") + + bar.set_global_opts( + title_opts=opts.TitleOpts(title=title), + toolbox_opts=opts.ToolboxOpts(is_show=True), + legend_opts=opts.LegendOpts(is_show=True) + ) + bar.render(output_path) + logger.info(f"堆叠柱状图已保存: {output_path}") + return output_path + + def line_chart( + self, + x_data: List, + y_data_dict: Dict[str, List], + title: str, + output_path: str + ) -> str: + """创建交互式折线图""" + line = Line(init_opts=opts.InitOpts(theme=self.theme)) + line.add_xaxis(x_data) + + for name, y_data in y_data_dict.items(): + line.add_yaxis( + name, + y_data, + is_smooth=True, + markpoint_opts=opts.MarkPointOpts( + data=[ + opts.MarkPointItem(type_="max", name="最大值"), + opts.MarkPointItem(type_="min", name="最小值"), + ] + ), + markline_opts=opts.MarkLineOpts( + data=[opts.MarkLineItem(type_="average", name="平均值")] + ) + ) + + line.set_global_opts( + title_opts=opts.TitleOpts(title=title), + tooltip_opts=opts.TooltipOpts(trigger="axis"), + toolbox_opts=opts.ToolboxOpts(is_show=True), + legend_opts=opts.LegendOpts(is_show=True) + ) + line.render(output_path) + logger.info(f"交互式折线图已保存: {output_path}") + return output_path + + def pie_chart( + self, + data: List[Tuple[str, int]], + title: str, + output_path: str, + rose_type: str = None + ) -> str: + """ + 创建交互式饼图 + + Args: + data: [(名称, 数值), ...] 列表 + title: 标题 + output_path: 输出路径 + rose_type: 玫瑰图类型 ('radius' 或 'area') + """ + pie = Pie(init_opts=opts.InitOpts(theme=self.theme)) + + if rose_type: + pie.add( + "", + data, + radius=["30%", "70%"], + rosetype=rose_type + ) + else: + pie.add("", data, radius=["30%", "70%"]) + + pie.set_global_opts( + title_opts=opts.TitleOpts(title=title), + legend_opts=opts.LegendOpts( + orient="vertical", + pos_top="15%", + pos_left="2%" + ) + ) + pie.set_series_opts( + label_opts=opts.LabelOpts(formatter="{b}: {d}%") + ) + pie.render(output_path) + logger.info(f"交互式饼图已保存: {output_path}") + return output_path + + def scatter_chart( + self, + data: List[Tuple[float, float]], + title: str, + output_path: str, + series_name: str = "数据" + ) -> str: + """创建交互式散点图""" + scatter = ( + Scatter(init_opts=opts.InitOpts(theme=self.theme)) + .add_xaxis([d[0] for d in data]) + .add_yaxis(series_name, [d[1] for d in data]) + .set_global_opts( + title_opts=opts.TitleOpts(title=title), + toolbox_opts=opts.ToolboxOpts(is_show=True), + xaxis_opts=opts.AxisOpts(type_="value"), + ) + ) + scatter.render(output_path) + logger.info(f"交互式散点图已保存: {output_path}") + return output_path + + def wordcloud_chart( + self, + words: List[Tuple[str, int]], + title: str, + output_path: str, + shape: str = "circle" + ) -> str: + """ + 创建交互式词云 + + Args: + words: [(词语, 频次), ...] 列表 + title: 标题 + output_path: 输出路径 + shape: 形状 ('circle', 'cardioid', 'diamond', 'triangle-forward', 'triangle', 'pentagon', 'star') + """ + wc = ( + PyWordCloud(init_opts=opts.InitOpts(theme=self.theme)) + .add( + "", + words, + word_size_range=[20, 100], + shape=shape + ) + .set_global_opts( + title_opts=opts.TitleOpts(title=title), + toolbox_opts=opts.ToolboxOpts(is_show=True) + ) + ) + wc.render(output_path) + logger.info(f"交互式词云已保存: {output_path}") + return output_path + + +class ChartFactory: + """图表工厂类""" + + def __init__(self, output_dir: str = "./charts"): + """ + 初始化 + + Args: + output_dir: 输出目录 + """ + self.output_dir = output_dir + os.makedirs(output_dir, exist_ok=True) + + self.matplotlib = MatplotlibCharts() if HAS_MATPLOTLIB else None + self.pyecharts = PyechartsCharts() if HAS_PYECHARTS else None + + def get_output_path(self, filename: str) -> str: + """获取输出路径""" + return os.path.join(self.output_dir, filename) + + def create_bar( + self, + categories: List[str], + values: List, + title: str, + filename: str, + chart_type: str = 'matplotlib' + ) -> str: + """创建柱状图""" + output_path = self.get_output_path(filename) + + if chart_type == 'matplotlib' and self.matplotlib: + return self.matplotlib.bar_chart( + categories, values, title, "分类", "数值", output_path + ) + elif chart_type == 'pyecharts' and self.pyecharts: + return self.pyecharts.bar_chart(categories, values, title, output_path) + else: + raise ValueError(f"Chart type '{chart_type}' not available") + + def create_line( + self, + x_data: List, + y_data_dict: Dict[str, List], + title: str, + filename: str, + chart_type: str = 'matplotlib' + ) -> str: + """创建折线图""" + output_path = self.get_output_path(filename) + + if chart_type == 'matplotlib' and self.matplotlib: + return self.matplotlib.multi_line_chart( + x_data, y_data_dict, title, "X轴", "Y轴", output_path + ) + elif chart_type == 'pyecharts' and self.pyecharts: + return self.pyecharts.line_chart(x_data, y_data_dict, title, output_path) + else: + raise ValueError(f"Chart type '{chart_type}' not available") + + def create_pie( + self, + data: List[Tuple[str, int]], + title: str, + filename: str, + chart_type: str = 'matplotlib' + ) -> str: + """创建饼图""" + output_path = self.get_output_path(filename) + + if chart_type == 'matplotlib' and self.matplotlib: + labels = [d[0] for d in data] + sizes = [d[1] for d in data] + return self.matplotlib.pie_chart(labels, sizes, title, output_path) + elif chart_type == 'pyecharts' and self.pyecharts: + return self.pyecharts.pie_chart(data, title, output_path) + else: + raise ValueError(f"Chart type '{chart_type}' not available") + + +def demo(): + """演示图表生成功能""" + print("=" * 50) + print("数据可视化工具演示") + print("=" * 50) + + # 测试数据 + categories = ["Python", "Java", "JavaScript", "Go", "Rust"] + values = [85, 70, 75, 45, 30] + + x_data = ["1月", "2月", "3月", "4月", "5月"] + y_data = { + "浏览量": [1000, 1200, 1500, 1300, 1800], + "点赞数": [100, 150, 200, 180, 250] + } + + pie_data = [ + ("技术", 45), + ("生活", 30), + ("娱乐", 15), + ("其他", 10) + ] + + print("\n可用的图表类型:") + + if HAS_MATPLOTLIB: + print(" - Matplotlib (静态图表): 已安装") + print(" 示例: charts.matplotlib.bar_chart(...)") + else: + print(" - Matplotlib: 未安装 (pip install matplotlib)") + + if HAS_PYECHARTS: + print(" - Pyecharts (交互式图表): 已安装") + print(" 示例: charts.pyecharts.bar_chart(...)") + else: + print(" - Pyecharts: 未安装 (pip install pyecharts)") + + print("\n示例数据:") + print(f" 柱状图数据: {dict(zip(categories, values))}") + print(f" 折线图数据: {y_data}") + print(f" 饼图数据: {pie_data}") + + print("\n使用示例:") + print(""" + from chart_demo import ChartFactory + + factory = ChartFactory(output_dir="./output") + + # 创建柱状图 + factory.create_bar(categories, values, "编程语言流行度", "bar.png") + + # 创建折线图 + factory.create_line(x_data, y_data, "月度数据趋势", "line.png") + + # 创建饼图 + factory.create_pie(pie_data, "内容分类分布", "pie.png") + + # 创建交互式图表 (HTML) + factory.create_bar(categories, values, "编程语言流行度", "bar.html", chart_type='pyecharts') + """) + + print("\n" + "=" * 50) + print("演示完成") + print("=" * 50) + + +if __name__ == "__main__": + demo() diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/10_\346\225\260\346\215\256\345\210\206\346\236\220\344\270\216\345\217\257\350\247\206\345\214\226/pandas_analysis.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/10_\346\225\260\346\215\256\345\210\206\346\236\220\344\270\216\345\217\257\350\247\206\345\214\226/pandas_analysis.py" new file mode 100644 index 0000000..b4f90e9 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/10_\346\225\260\346\215\256\345\210\206\346\236\220\344\270\216\345\217\257\350\247\206\345\214\226/pandas_analysis.py" @@ -0,0 +1,353 @@ +# -*- coding: utf-8 -*- +# @Desc: pandas 数据分析工具 + +import pandas as pd +from typing import List, Dict, Any, Optional +from datetime import datetime +from loguru import logger + + +class DataFrameAnalyzer: + """DataFrame 数据分析器""" + + def __init__(self, data: List[Dict[str, Any]]): + """ + 初始化分析器 + + Args: + data: 字典列表形式的数据 + """ + self.df = pd.DataFrame(data) + logger.info(f"加载数据: {len(self.df)} 行, {len(self.df.columns)} 列") + + @classmethod + def from_csv(cls, file_path: str, **kwargs) -> 'DataFrameAnalyzer': + """从 CSV 文件创建分析器""" + df = pd.read_csv(file_path, **kwargs) + return cls(df.to_dict('records')) + + @classmethod + def from_json(cls, file_path: str, **kwargs) -> 'DataFrameAnalyzer': + """从 JSON 文件创建分析器""" + df = pd.read_json(file_path, **kwargs) + return cls(df.to_dict('records')) + + def info(self) -> Dict[str, Any]: + """获取数据集基本信息""" + return { + "rows": len(self.df), + "columns": len(self.df.columns), + "column_names": self.df.columns.tolist(), + "dtypes": self.df.dtypes.to_dict(), + "memory_usage": self.df.memory_usage(deep=True).sum(), + "null_counts": self.df.isnull().sum().to_dict(), + } + + def describe(self, include: str = 'all') -> pd.DataFrame: + """获取描述性统计""" + return self.df.describe(include=include) + + def value_counts(self, column: str, top_n: int = 10) -> pd.Series: + """统计某列的值分布""" + return self.df[column].value_counts().head(top_n) + + def group_by_count(self, group_col: str) -> pd.Series: + """按列分组计数""" + return self.df.groupby(group_col).size().sort_values(ascending=False) + + def group_by_sum(self, group_col: str, value_col: str) -> pd.Series: + """按列分组求和""" + return self.df.groupby(group_col)[value_col].sum().sort_values(ascending=False) + + def group_by_mean(self, group_col: str, value_col: str) -> pd.Series: + """按列分组求均值""" + return self.df.groupby(group_col)[value_col].mean().sort_values(ascending=False) + + def group_by_agg( + self, + group_col: str, + agg_dict: Dict[str, List[str]] + ) -> pd.DataFrame: + """ + 按列分组并进行多种聚合 + + Args: + group_col: 分组列 + agg_dict: 聚合字典，如 {'views': ['sum', 'mean'], 'likes': ['max', 'min']} + """ + return self.df.groupby(group_col).agg(agg_dict) + + def pivot_table( + self, + values: str, + index: str, + columns: str, + aggfunc: str = 'sum', + fill_value: Any = 0 + ) -> pd.DataFrame: + """创建数据透视表""" + return pd.pivot_table( + self.df, + values=values, + index=index, + columns=columns, + aggfunc=aggfunc, + fill_value=fill_value + ) + + def correlation(self, columns: List[str] = None) -> pd.DataFrame: + """计算相关系数矩阵""" + if columns: + return self.df[columns].corr() + return self.df.select_dtypes(include=['number']).corr() + + def filter(self, condition: str) -> 'DataFrameAnalyzer': + """ + 根据条件筛选数据 + + Args: + condition: 查询条件，如 "views > 1000 and category == '技术'" + """ + filtered_df = self.df.query(condition) + return DataFrameAnalyzer(filtered_df.to_dict('records')) + + def sort(self, by: str, ascending: bool = False) -> 'DataFrameAnalyzer': + """排序""" + sorted_df = self.df.sort_values(by=by, ascending=ascending) + return DataFrameAnalyzer(sorted_df.to_dict('records')) + + def top_n(self, n: int, by: str) -> pd.DataFrame: + """获取 Top N""" + return self.df.nlargest(n, by) + + def bottom_n(self, n: int, by: str) -> pd.DataFrame: + """获取 Bottom N""" + return self.df.nsmallest(n, by) + + +class TimeSeriesAnalyzer: + """时间序列分析器""" + + def __init__(self, df: pd.DataFrame, date_col: str): + """ + 初始化时间序列分析器 + + Args: + df: 数据框 + date_col: 日期列名 + """ + self.df = df.copy() + self.date_col = date_col + + # 转换日期列 + self.df[date_col] = pd.to_datetime(self.df[date_col], errors='coerce') + self.df = self.df.dropna(subset=[date_col]) + self.df = self.df.set_index(date_col) + + logger.info(f"时间序列分析器初始化: {len(self.df)} 条记录") + + def resample( + self, + freq: str, + value_col: str, + agg_func: str = 'sum' + ) -> pd.DataFrame: + """ + 重采样 + + Args: + freq: 频率，如 'D'(天), 'W'(周), 'M'(月) + value_col: 值列 + agg_func: 聚合函数 + """ + return self.df.resample(freq)[value_col].agg(agg_func) + + def rolling( + self, + window: int, + value_col: str, + agg_func: str = 'mean' + ) -> pd.Series: + """ + 滚动窗口统计 + + Args: + window: 窗口大小 + value_col: 值列 + agg_func: 聚合函数 + """ + return getattr(self.df[value_col].rolling(window=window), agg_func)() + + def growth_rate(self, value_col: str, periods: int = 1) -> pd.Series: + """ + 计算增长率 + + Args: + value_col: 值列 + periods: 周期数 + """ + return self.df[value_col].pct_change(periods=periods) * 100 + + def trend_analysis(self, value_col: str) -> Dict[str, Any]: + """ + 趋势分析 + + Returns: + 包含趋势统计信息的字典 + """ + series = self.df[value_col] + growth = series.pct_change().dropna() + + return { + "start_value": series.iloc[0], + "end_value": series.iloc[-1], + "total_change": series.iloc[-1] - series.iloc[0], + "total_change_pct": (series.iloc[-1] - series.iloc[0]) / series.iloc[0] * 100, + "avg_growth_rate": growth.mean() * 100, + "max_growth_rate": growth.max() * 100, + "min_growth_rate": growth.min() * 100, + "positive_periods": (growth > 0).sum(), + "negative_periods": (growth < 0).sum(), + } + + def seasonal_decompose( + self, + value_col: str, + period: int = 7 + ) -> Dict[str, pd.Series]: + """ + 季节性分解（简化版） + + Args: + value_col: 值列 + period: 周期 + """ + series = self.df[value_col] + + # 移动平均作为趋势 + trend = series.rolling(window=period, center=True).mean() + + # 去趋势 + detrended = series - trend + + # 季节性（按周期位置平均） + seasonal = detrended.groupby( + detrended.index.dayofweek + ).transform('mean') + + # 残差 + residual = series - trend - seasonal + + return { + "original": series, + "trend": trend, + "seasonal": seasonal, + "residual": residual + } + + +class StatisticsCalculator: + """统计计算器""" + + @staticmethod + def percentile(series: pd.Series, q: float) -> float: + """计算分位数""" + return series.quantile(q) + + @staticmethod + def iqr(series: pd.Series) -> float: + """计算四分位距""" + return series.quantile(0.75) - series.quantile(0.25) + + @staticmethod + def outliers(series: pd.Series, method: str = 'iqr') -> pd.Series: + """ + 检测异常值 + + Args: + series: 数据序列 + method: 检测方法 ('iqr' 或 'zscore') + + Returns: + 布尔序列，True 表示异常值 + """ + if method == 'iqr': + q1 = series.quantile(0.25) + q3 = series.quantile(0.75) + iqr = q3 - q1 + lower = q1 - 1.5 * iqr + upper = q3 + 1.5 * iqr + return (series < lower) | (series > upper) + elif method == 'zscore': + z_scores = (series - series.mean()) / series.std() + return abs(z_scores) > 3 + else: + raise ValueError(f"Unknown method: {method}") + + @staticmethod + def distribution_stats(series: pd.Series) -> Dict[str, float]: + """计算分布统计量""" + return { + "count": series.count(), + "mean": series.mean(), + "std": series.std(), + "min": series.min(), + "q25": series.quantile(0.25), + "median": series.median(), + "q75": series.quantile(0.75), + "max": series.max(), + "skewness": series.skew(), + "kurtosis": series.kurtosis(), + } + + +def demo(): + """演示 pandas 数据分析功能""" + print("=" * 50) + print("pandas 数据分析演示") + print("=" * 50) + + # 模拟数据 + data = [ + {"title": "Python入门", "category": "技术", "views": 15000, "likes": 320, "date": "2024-01-15"}, + {"title": "爬虫教程", "category": "技术", "views": 12000, "likes": 280, "date": "2024-01-16"}, + {"title": "美食分享", "category": "生活", "views": 8000, "likes": 450, "date": "2024-01-17"}, + {"title": "数据分析", "category": "技术", "views": 18000, "likes": 520, "date": "2024-01-18"}, + {"title": "旅行日记", "category": "生活", "views": 10000, "likes": 380, "date": "2024-01-19"}, + {"title": "机器学习", "category": "技术", "views": 20000, "likes": 600, "date": "2024-01-20"}, + ] + + # 创建分析器 + analyzer = DataFrameAnalyzer(data) + + # 基本信息 + print("\n1. 数据集信息:") + info = analyzer.info() + print(f" 行数: {info['rows']}, 列数: {info['columns']}") + print(f" 列名: {info['column_names']}") + + # 描述性统计 + print("\n2. 描述性统计:") + print(analyzer.describe()) + + # 分组统计 + print("\n3. 按分类分组统计:") + print(analyzer.group_by_agg('category', {'views': ['sum', 'mean'], 'likes': ['sum', 'mean']})) + + # Top N + print("\n4. 浏览量 Top 3:") + print(analyzer.top_n(3, 'views')) + + # 分布统计 + print("\n5. 浏览量分布统计:") + stats = StatisticsCalculator.distribution_stats(analyzer.df['views']) + for key, value in stats.items(): + print(f" {key}: {value:.2f}") + + print("\n" + "=" * 50) + print("演示完成") + print("=" * 50) + + +if __name__ == "__main__": + demo() diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/10_\346\225\260\346\215\256\345\210\206\346\236\220\344\270\216\345\217\257\350\247\206\345\214\226/pyproject.toml" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/10_\346\225\260\346\215\256\345\210\206\346\236\220\344\270\216\345\217\257\350\247\206\345\214\226/pyproject.toml" new file mode 100644 index 0000000..4ed32e4 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/10_\346\225\260\346\215\256\345\210\206\346\236\220\344\270\216\345\217\257\350\247\206\345\214\226/pyproject.toml" @@ -0,0 +1,24 @@ +[project] +name = "chapter10-data-analysis" +version = "0.1.0" +description = "第10章：数据分析与可视化 - Pandas分析、词云生成、图表制作" +readme = "README.md" +requires-python = ">=3.11" +dependencies = [ + "pandas>=2.2.0", + "jieba>=0.42.0", + "wordcloud>=1.9.0", + "matplotlib>=3.8.0", +] + +[project.optional-dependencies] +interactive = [ + "pyecharts>=2.0.0", # 交互式图表 +] + +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[tool.uv] +dev-dependencies = [] diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/10_\346\225\260\346\215\256\345\210\206\346\236\220\344\270\216\345\217\257\350\247\206\345\214\226/wordcloud_generator.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/10_\346\225\260\346\215\256\345\210\206\346\236\220\344\270\216\345\217\257\350\247\206\345\214\226/wordcloud_generator.py" new file mode 100644 index 0000000..10c1d3c --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/10_\346\225\260\346\215\256\345\210\206\346\236\220\344\270\216\345\217\257\350\247\206\345\214\226/wordcloud_generator.py" @@ -0,0 +1,398 @@ +# -*- coding: utf-8 -*- +# @Desc: 词云生成工具 +from __future__ import annotations + +import os +from typing import List, Dict, Tuple, Optional +from collections import Counter +from loguru import logger + +# 可选依赖 +try: + import jieba + import jieba.analyse + HAS_JIEBA = True +except ImportError: + HAS_JIEBA = False + logger.warning("jieba 未安装，中文分词功能不可用") + +try: + from wordcloud import WordCloud, ImageColorGenerator + HAS_WORDCLOUD = True +except ImportError: + HAS_WORDCLOUD = False + logger.warning("wordcloud 未安装，词云生成功能不可用") + +try: + import numpy as np + from PIL import Image + HAS_PIL = True +except ImportError: + HAS_PIL = False + + +class ChineseTokenizer: + """中文分词器""" + + # 默认停用词 + DEFAULT_STOPWORDS = { + '的', '是', '在', '了', '和', '与', '或', '有', '个', '人', + '这', '那', '就', '都', '也', '为', '对', '到', '从', '把', + '被', '让', '给', '向', '往', '于', '及', '以', '等', '不', + '很', '会', '能', '可', '要', '我', '你', '他', '她', '它', + '啊', '吧', '呢', '呀', '哦', '嗯', '哈', '嘿', '么', '吗', + '什么', '怎么', '这样', '那样', '如何', '为什么', '怎样', + '没有', '已经', '可以', '一个', '一些', '有些', '还是', + '但是', '然后', '如果', '因为', '所以', '虽然', '而且', + '或者', '并且', '只是', '只有', '就是', '还有', '这个', + '那个', '自己', '什么', '这里', '那里', '这些', '那些', + } + + def __init__(self, stopwords: set = None, user_dict: str = None): + """ + 初始化分词器 + + Args: + stopwords: 自定义停用词集合 + user_dict: 用户词典路径 + """ + if not HAS_JIEBA: + raise ImportError("请安装 jieba: pip install jieba") + + self.stopwords = stopwords or self.DEFAULT_STOPWORDS + + if user_dict and os.path.exists(user_dict): + jieba.load_userdict(user_dict) + logger.info(f"加载用户词典: {user_dict}") + + def add_stopwords(self, words: List[str]): + """添加停用词""" + self.stopwords.update(words) + + def tokenize(self, text: str, min_length: int = 2) -> List[str]: + """ + 分词 + + Args: + text: 输入文本 + min_length: 最小词长度 + + Returns: + 词语列表 + """ + words = jieba.lcut(text) + return [ + w for w in words + if w not in self.stopwords + and len(w) >= min_length + and not w.isspace() + ] + + def extract_keywords( + self, + text: str, + top_k: int = 20, + method: str = 'tfidf' + ) -> List[Tuple[str, float]]: + """ + 提取关键词 + + Args: + text: 输入文本 + top_k: 返回关键词数量 + method: 提取方法 ('tfidf' 或 'textrank') + + Returns: + (关键词, 权重) 列表 + """ + if method == 'tfidf': + return jieba.analyse.extract_tags(text, topK=top_k, withWeight=True) + elif method == 'textrank': + return jieba.analyse.textrank(text, topK=top_k, withWeight=True) + else: + raise ValueError(f"Unknown method: {method}") + + def word_frequency( + self, + texts: List[str], + top_n: int = 100 + ) -> List[Tuple[str, int]]: + """ + 统计词频 + + Args: + texts: 文本列表 + top_n: 返回 Top N + + Returns: + (词语, 频次) 列表 + """ + all_words = [] + for text in texts: + words = self.tokenize(text) + all_words.extend(words) + + return Counter(all_words).most_common(top_n) + + +class WordCloudGenerator: + """词云生成器""" + + # 默认配色方案 + COLOR_SCHEMES = { + 'default': None, + 'viridis': 'viridis', + 'plasma': 'plasma', + 'inferno': 'inferno', + 'magma': 'magma', + 'cool': 'cool', + 'hot': 'hot', + } + + def __init__( + self, + font_path: str = None, + width: int = 800, + height: int = 600, + background_color: str = 'white', + max_words: int = 200, + max_font_size: int = 100, + min_font_size: int = 10, + colormap: str = None + ): + """ + 初始化词云生成器 + + Args: + font_path: 字体路径（中文需要指定） + width: 图片宽度 + height: 图片高度 + background_color: 背景颜色 + max_words: 最大词数 + max_font_size: 最大字体大小 + min_font_size: 最小字体大小 + colormap: matplotlib 色彩映射名称 + """ + if not HAS_WORDCLOUD: + raise ImportError("请安装 wordcloud: pip install wordcloud") + + self.font_path = font_path + self.width = width + self.height = height + self.background_color = background_color + self.max_words = max_words + self.max_font_size = max_font_size + self.min_font_size = min_font_size + self.colormap = colormap + + def _create_wordcloud(self, **kwargs) -> WordCloud: + """创建 WordCloud 对象""" + params = { + 'font_path': self.font_path, + 'width': self.width, + 'height': self.height, + 'background_color': self.background_color, + 'max_words': self.max_words, + 'max_font_size': self.max_font_size, + 'min_font_size': self.min_font_size, + 'colormap': self.colormap, + 'random_state': 42, + } + params.update(kwargs) + return WordCloud(**params) + + def generate_from_text(self, text: str, output_path: str) -> str: + """ + 从文本生成词云 + + Args: + text: 空格分隔的词语文本 + output_path: 输出路径 + + Returns: + 输出文件路径 + """ + wc = self._create_wordcloud() + wc.generate(text) + wc.to_file(output_path) + logger.info(f"词云已保存: {output_path}") + return output_path + + def generate_from_frequencies( + self, + frequencies: Dict[str, int], + output_path: str + ) -> str: + """ + 从词频字典生成词云 + + Args: + frequencies: {词语: 频次} 字典 + output_path: 输出路径 + + Returns: + 输出文件路径 + """ + wc = self._create_wordcloud() + wc.generate_from_frequencies(frequencies) + wc.to_file(output_path) + logger.info(f"词云已保存: {output_path}") + return output_path + + def generate_shaped( + self, + text: str, + mask_image_path: str, + output_path: str, + use_mask_colors: bool = True + ) -> str: + """ + 生成自定义形状的词云 + + Args: + text: 词语文本 + mask_image_path: 形状蒙版图片路径 + output_path: 输出路径 + use_mask_colors: 是否使用蒙版图片的颜色 + + Returns: + 输出文件路径 + """ + if not HAS_PIL: + raise ImportError("请安装 pillow 和 numpy") + + # 读取蒙版 + mask = np.array(Image.open(mask_image_path)) + + wc = self._create_wordcloud( + mask=mask, + contour_width=1, + contour_color='steelblue' + ) + + wc.generate(text) + + # 使用蒙版颜色 + if use_mask_colors: + image_colors = ImageColorGenerator(mask) + wc.recolor(color_func=image_colors) + + wc.to_file(output_path) + logger.info(f"形状词云已保存: {output_path}") + return output_path + + +class TextToWordCloud: + """从原始文本到词云的完整流程""" + + def __init__( + self, + font_path: str = None, + stopwords: set = None, + user_dict: str = None + ): + """ + 初始化 + + Args: + font_path: 字体路径 + stopwords: 停用词 + user_dict: 用户词典 + """ + self.tokenizer = ChineseTokenizer(stopwords, user_dict) + self.generator = WordCloudGenerator(font_path) + + def process( + self, + texts: List[str], + output_path: str, + top_words: int = 200 + ) -> str: + """ + 处理文本并生成词云 + + Args: + texts: 文本列表 + output_path: 输出路径 + top_words: 使用的词数量 + + Returns: + 输出文件路径 + """ + # 1. 统计词频 + word_freq = self.tokenizer.word_frequency(texts, top_words) + logger.info(f"统计完成: {len(word_freq)} 个词") + + # 2. 转换为字典 + freq_dict = dict(word_freq) + + # 3. 生成词云 + return self.generator.generate_from_frequencies(freq_dict, output_path) + + def get_word_stats(self, texts: List[str], top_n: int = 20) -> List[Tuple[str, int]]: + """ + 获取词频统计 + + Args: + texts: 文本列表 + top_n: 返回数量 + + Returns: + (词语, 频次) 列表 + """ + return self.tokenizer.word_frequency(texts, top_n) + + +def demo(): + """演示词云生成功能""" + print("=" * 50) + print("词云生成工具演示") + print("=" * 50) + + if not HAS_JIEBA: + print("请安装 jieba: pip install jieba") + return + + if not HAS_WORDCLOUD: + print("请安装 wordcloud: pip install wordcloud") + return + + # 测试文本 + texts = [ + "Python是一门优雅的编程语言，适合数据分析和机器学习", + "爬虫技术可以帮助我们获取互联网上的数据", + "数据分析是数据科学的重要组成部分", + "机器学习和深度学习是人工智能的核心技术", + "Python在数据科学领域有广泛的应用", + "爬虫需要注意遵守网站的规则和法律法规", + "数据可视化可以帮助我们更好地理解数据", + "编程是一项需要不断学习和实践的技能", + ] + + # 分词 + tokenizer = ChineseTokenizer() + print("\n1. 分词示例:") + sample_words = tokenizer.tokenize(texts[0]) + print(f" 原文: {texts[0]}") + print(f" 分词: {sample_words}") + + # 词频统计 + print("\n2. 词频统计 Top 10:") + word_freq = tokenizer.word_frequency(texts, 10) + for word, freq in word_freq: + print(f" {word}: {freq}") + + # 生成词云（仅打印说明，不实际生成文件） + print("\n3. 词云生成:") + print(" 词云生成需要指定中文字体路径") + print(" 示例代码:") + print(" generator = WordCloudGenerator(font_path='/path/to/font.ttf')") + print(" generator.generate_from_frequencies(dict(word_freq), 'wordcloud.png')") + + print("\n" + "=" * 50) + print("演示完成") + print("=" * 50) + + +if __name__ == "__main__": + demo() diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/README.md" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/README.md" new file mode 100644 index 0000000..1e450cf --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/README.md" @@ -0,0 +1,67 @@ +# 第11章：进阶综合实战项目 + +完整的书籍电商数据采集工具，爬取 **books.toscrape.com**（专门的爬虫练习网站）。 + +## 快速开始 + +### 使用 uv 安装依赖 + +```bash +cd 11_进阶综合实战项目 + +# 安装依赖 +uv sync + +# 安装可选功能（配置管理增强） +uv sync --extra all + +# 安装Playwright浏览器驱动 +uv run playwright install chromium + +# 运行项目 +uv run python main.py +``` + +### 目标网站 + +- **网站**：http://books.toscrape.com +- **类型**：专门用于爬虫练习的合法网站 +- **特点**：电商结构完整，50页书籍数据，无需登录 + +### 核心依赖 + +- `playwright` - 浏览器自动化 +- `httpx` - HTTP客户端 +- `pydantic` - 数据验证 +- `loguru` - 日志系统 +- `pandas` - 数据分析 +- `jieba` + `wordcloud` - 词云生成 + +## 项目结构 + +``` +11_进阶综合实战项目/ +├── config/ # 配置模块 +├── core/ # 核心模块（浏览器管理） +├── login/ # 登录模块 +├── crawler/ # 爬虫模块 +├── store/ # 存储模块 +├── proxy/ # 代理池模块 +├── analysis/ # 分析模块 +└── main.py # 主程序入口 +``` + +## 功能特性 + +- ✅ 浏览器自动化采集 +- ✅ 反检测技术（stealth.js） +- ✅ 多格式数据存储（JSON/CSV） +- ✅ 词云和统计报告生成 +- ✅ 代理池支持（可选） + +## 运行结果 + +运行成功后会在 `output/` 目录下生成： +- `data_*.json` 或 `data_*.csv` - 采集的数据 +- `report.md` - 数据分析报告 +- `wordcloud.png` - 词云图片 diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/analysis/__init__.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/analysis/__init__.py" new file mode 100644 index 0000000..047a1e0 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/analysis/__init__.py" @@ -0,0 +1,5 @@ +# -*- coding: utf-8 -*- +"""数据分析模块""" +from .report import DataAnalyzer, BilibiliAnalyzer, ReportGenerator, generate_report + +__all__ = ['DataAnalyzer', 'BilibiliAnalyzer', 'ReportGenerator', 'generate_report'] diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/analysis/report.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/analysis/report.py" new file mode 100644 index 0000000..7d31de5 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/analysis/report.py" @@ -0,0 +1,584 @@ +# -*- coding: utf-8 -*- +""" +数据分析与报告生成模块 + +本模块实现了 B站视频数据的分析功能，包括： +- DataAnalyzer: 数据分析器（基础统计、词频分析、分布统计） +- BilibiliAnalyzer: B站视频专用分析器 +- ReportGenerator: 报告生成器（Markdown 格式） + +参考 MediaCrawler 项目的数据分析实践。 +""" + +import os +from typing import List, Dict, Any, Optional, Union +from datetime import datetime +from collections import Counter +from pathlib import Path +from loguru import logger + +# 尝试导入模型 +try: + from models.bilibili import BilibiliVideo + HAS_MODEL = True +except ImportError: + HAS_MODEL = False + +# 可选依赖 +try: + import jieba + HAS_JIEBA = True +except ImportError: + HAS_JIEBA = False + +try: + from wordcloud import WordCloud + HAS_WORDCLOUD = True +except ImportError: + HAS_WORDCLOUD = False + +try: + import pandas as pd + HAS_PANDAS = True +except ImportError: + HAS_PANDAS = False + + +class DataAnalyzer: + """数据分析器""" + + # 中文停用词 + STOPWORDS = { + '的', '是', '在', '了', '和', '与', '或', '有', '个', '人', + '这', '那', '就', '都', '也', '为', '对', '到', '从', '把', + '被', '让', '给', '向', '往', '于', '及', '以', '等', '不', + '很', '会', '能', '可', '要', '我', '你', '他', '她', '它', + '啊', '吧', '呢', '呀', '哦', '嗯', '哈', '嘿', '么', '吗', + '什么', '怎么', '这样', '那样', '如何', '为什么', '怎样', + '一个', '一些', '一种', '一下', '没有', '还是', '已经', + } + + def __init__(self, data: List[Dict], output_dir: str = "./output"): + """ + 初始化数据分析器 + + Args: + data: 数据列表 + output_dir: 输出目录 + """ + self.data = data + self.output_dir = Path(output_dir) + self.output_dir.mkdir(parents=True, exist_ok=True) + + if HAS_PANDAS and data: + self.df = pd.DataFrame(data) + else: + self.df = None + + def basic_stats(self) -> Dict[str, Any]: + """基础统计""" + stats = { + 'total_records': len(self.data), + 'fields': list(self.data[0].keys()) if self.data else [], + } + + if self.df is not None: + # 数值列统计 + numeric_cols = self.df.select_dtypes(include=['number']).columns.tolist() + if numeric_cols: + stats['numeric_fields'] = numeric_cols + for col in numeric_cols: + stats[f'{col}_sum'] = float(self.df[col].sum()) + stats[f'{col}_mean'] = float(self.df[col].mean()) + stats[f'{col}_max'] = float(self.df[col].max()) + stats[f'{col}_min'] = float(self.df[col].min()) + + return stats + + def word_frequency( + self, + text_field: str, + top_n: int = 50, + min_length: int = 2 + ) -> List[tuple]: + """ + 词频统计 + + Args: + text_field: 文本字段名 + top_n: 返回 Top N + min_length: 最小词长度 + + Returns: + (词语, 频次) 列表 + """ + if not HAS_JIEBA: + logger.warning("jieba 未安装，跳过词频分析") + return [] + + all_words = [] + for item in self.data: + text = item.get(text_field, '') + if text: + words = jieba.lcut(str(text)) + words = [ + w for w in words + if w not in self.STOPWORDS + and len(w) >= min_length + and not w.isspace() + ] + all_words.extend(words) + + return Counter(all_words).most_common(top_n) + + def generate_wordcloud( + self, + text_field: str, + output_file: str = "wordcloud.png", + font_path: str = None, + width: int = 1200, + height: int = 800 + ) -> Optional[str]: + """ + 生成词云 + + Args: + text_field: 文本字段名 + output_file: 输出文件名 + font_path: 字体路径（中文需要指定） + width: 图片宽度 + height: 图片高度 + + Returns: + 输出文件路径，失败返回 None + """ + if not HAS_WORDCLOUD: + logger.warning("wordcloud 未安装，跳过词云生成") + return None + + word_freq = self.word_frequency(text_field, 200) + if not word_freq: + logger.warning("没有词频数据，跳过词云生成") + return None + + freq_dict = dict(word_freq) + + try: + wc = WordCloud( + width=width, + height=height, + background_color='white', + font_path=font_path, + max_words=200, + max_font_size=150, + random_state=42 + ) + + wc.generate_from_frequencies(freq_dict) + + output_path = self.output_dir / output_file + wc.to_file(str(output_path)) + logger.info(f"词云已保存: {output_path}") + return str(output_path) + except Exception as e: + logger.error(f"生成词云失败: {e}") + return None + + def value_distribution(self, field: str) -> Dict[Any, int]: + """ + 值分布统计 + + Args: + field: 字段名 + + Returns: + {值: 计数} 字典 + """ + counter = Counter() + for item in self.data: + value = item.get(field) + if value is not None: + counter[value] += 1 + return dict(counter) + + +class BilibiliAnalyzer(DataAnalyzer): + """ + B站视频数据分析器 + + 提供针对 B站视频数据的专用分析方法。 + """ + + def __init__( + self, + videos: List[Union[Dict, "BilibiliVideo"]], + output_dir: str = "./output" + ): + """ + 初始化 B站分析器 + + Args: + videos: 视频数据列表（BilibiliVideo 或字典） + output_dir: 输出目录 + """ + # 转换为字典列表 + data = [] + for video in videos: + if HAS_MODEL and isinstance(video, BilibiliVideo): + data.append(video.to_dict()) + elif hasattr(video, 'model_dump'): + data.append(video.model_dump()) + elif isinstance(video, dict): + data.append(video) + else: + data.append(dict(video)) + + super().__init__(data, output_dir) + + def video_metrics_stats(self) -> Dict[str, Any]: + """ + 视频指标统计 + + 统计播放量、点赞、收藏、投币等指标。 + + Returns: + 统计结果字典 + """ + metrics = { + 'play_count': [], + 'liked_count': [], + 'coin_count': [], + 'favorite_count': [], + 'share_count': [], + 'danmaku_count': [], + 'comment_count': [], + } + + for item in self.data: + for key in metrics.keys(): + value = item.get(key, 0) + if value is not None: + try: + metrics[key].append(int(value)) + except (ValueError, TypeError): + metrics[key].append(0) + + stats = {} + for key, values in metrics.items(): + if values: + stats[key] = { + 'total': sum(values), + 'avg': sum(values) / len(values), + 'max': max(values), + 'min': min(values), + } + + return stats + + def up_distribution(self, top_n: int = 10) -> List[tuple]: + """ + UP主分布统计 + + Args: + top_n: 返回 Top N + + Returns: + [(UP主昵称, 视频数量)] 列表 + """ + counter = Counter() + for item in self.data: + nickname = item.get('nickname', '未知UP主') + if nickname: + counter[nickname] += 1 + return counter.most_common(top_n) + + def keyword_distribution(self) -> Dict[str, int]: + """ + 搜索关键词分布统计 + + Returns: + {关键词: 视频数量} 字典 + """ + counter = Counter() + for item in self.data: + keyword = item.get('source_keyword', '') + if keyword: + counter[keyword] += 1 + return dict(counter) + + def top_videos( + self, + metric: str = 'play_count', + top_n: int = 10 + ) -> List[Dict]: + """ + 获取指标排名前 N 的视频 + + Args: + metric: 排序指标（play_count, liked_count 等） + top_n: 返回数量 + + Returns: + 视频列表 + """ + sorted_data = sorted( + self.data, + key=lambda x: x.get(metric, 0) or 0, + reverse=True + ) + return sorted_data[:top_n] + + def generate_title_wordcloud( + self, + output_file: str = "title_wordcloud.png", + font_path: str = None + ) -> Optional[str]: + """ + 生成视频标题词云 + + Args: + output_file: 输出文件名 + font_path: 字体路径 + + Returns: + 输出文件路径 + """ + return self.generate_wordcloud( + text_field='title', + output_file=output_file, + font_path=font_path + ) + + +class ReportGenerator: + """ + 报告生成器 + + 生成 B站视频数据的 Markdown 格式分析报告。 + """ + + def __init__( + self, + videos: List[Union[Dict, "BilibiliVideo"]], + output_dir: str = "./output" + ): + """ + 初始化报告生成器 + + Args: + videos: 视频数据列表 + output_dir: 输出目录 + """ + self.videos = videos + self.output_dir = Path(output_dir) + self.output_dir.mkdir(parents=True, exist_ok=True) + + # 创建分析器 + self.analyzer = BilibiliAnalyzer(videos, output_dir) + + def generate( + self, + font_path: str = None, + title: str = "B站视频数据分析报告" + ) -> str: + """ + 生成完整分析报告 + + Args: + font_path: 字体路径（词云使用） + title: 报告标题 + + Returns: + 报告文件路径 + """ + lines = [] + + # 标题 + lines.append(f"# {title}") + lines.append("") + lines.append(f"> 生成时间: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}") + lines.append(f"> 数据来源: B站视频采集") + lines.append(f"> 数据量: {len(self.analyzer.data)} 条") + lines.append("") + lines.append("---") + lines.append("") + + # 1. 视频指标统计 + lines.append("## 1. 视频指标统计") + lines.append("") + metrics_stats = self.analyzer.video_metrics_stats() + if metrics_stats: + lines.append("| 指标 | 总计 | 平均 | 最高 | 最低 |") + lines.append("| --- | ---: | ---: | ---: | ---: |") + + metric_names = { + 'play_count': '播放量', + 'liked_count': '点赞数', + 'coin_count': '投币数', + 'favorite_count': '收藏数', + 'share_count': '分享数', + 'danmaku_count': '弹幕数', + 'comment_count': '评论数', + } + + for key, name in metric_names.items(): + if key in metrics_stats: + stat = metrics_stats[key] + lines.append( + f"| {name} | " + f"{stat['total']:,} | " + f"{stat['avg']:,.0f} | " + f"{stat['max']:,} | " + f"{stat['min']:,} |" + ) + lines.append("") + + # 2. 热门视频 TOP 10 + lines.append("## 2. 热门视频 TOP 10（按播放量）") + lines.append("") + top_videos = self.analyzer.top_videos('play_count', 10) + if top_videos: + lines.append("| 排名 | 标题 | UP主 | 播放量 | 点赞 |") + lines.append("| --- | --- | --- | ---: | ---: |") + for i, video in enumerate(top_videos, 1): + title_short = video.get('title', '')[:30] + if len(video.get('title', '')) > 30: + title_short += '...' + lines.append( + f"| {i} | {title_short} | " + f"{video.get('nickname', '未知')} | " + f"{video.get('play_count', 0):,} | " + f"{video.get('liked_count', 0):,} |" + ) + lines.append("") + + # 3. UP主分布 + lines.append("## 3. UP主分布 TOP 10") + lines.append("") + up_dist = self.analyzer.up_distribution(10) + if up_dist: + lines.append("| 排名 | UP主 | 视频数 |") + lines.append("| --- | --- | ---: |") + for i, (name, count) in enumerate(up_dist, 1): + lines.append(f"| {i} | {name} | {count} |") + lines.append("") + + # 4. 关键词分布 + keyword_dist = self.analyzer.keyword_distribution() + if keyword_dist and len(keyword_dist) > 1: + lines.append("## 4. 搜索关键词分布") + lines.append("") + lines.append("| 关键词 | 视频数 |") + lines.append("| --- | ---: |") + for keyword, count in sorted( + keyword_dist.items(), + key=lambda x: x[1], + reverse=True + ): + lines.append(f"| {keyword} | {count} |") + lines.append("") + + # 5. 标题热词 TOP 20 + if HAS_JIEBA: + lines.append("## 5. 标题热词 TOP 20") + lines.append("") + word_freq = self.analyzer.word_frequency('title', 20) + if word_freq: + lines.append("| 排名 | 词汇 | 频次 |") + lines.append("| --- | --- | ---: |") + for i, (word, count) in enumerate(word_freq, 1): + lines.append(f"| {i} | {word} | {count} |") + lines.append("") + + # 生成词云 + if HAS_WORDCLOUD: + wordcloud_path = self.analyzer.generate_title_wordcloud( + font_path=font_path + ) + if wordcloud_path: + lines.append("### 标题词云") + lines.append("") + lines.append("![标题词云](title_wordcloud.png)") + lines.append("") + + # 6. 数据样本 + lines.append("## 附录: 数据样本 (前5条)") + lines.append("") + sample_fields = ['bvid', 'title', 'nickname', 'play_count', 'liked_count'] + sample_data = self.analyzer.data[:5] + if sample_data: + lines.append("| " + " | ".join(sample_fields) + " |") + lines.append("| " + " | ".join(["---"] * len(sample_fields)) + " |") + for item in sample_data: + row = [] + for f in sample_fields: + val = str(item.get(f, ''))[:40] + val = val.replace('|', '\\|').replace('\n', ' ') + row.append(val) + lines.append("| " + " | ".join(row) + " |") + lines.append("") + + # 保存报告 + report_content = '\n'.join(lines) + report_path = self.output_dir / "report.md" + with open(report_path, 'w', encoding='utf-8') as f: + f.write(report_content) + + logger.info(f"报告已保存: {report_path}") + return str(report_path) + + +# 便捷函数 +def generate_report( + videos: List[Union[Dict, "BilibiliVideo"]], + output_dir: str = "./output", + font_path: str = None +) -> str: + """ + 生成分析报告（便捷函数） + + Args: + videos: 视频数据列表 + output_dir: 输出目录 + font_path: 字体路径 + + Returns: + 报告文件路径 + """ + generator = ReportGenerator(videos, output_dir) + return generator.generate(font_path=font_path) + + +if __name__ == '__main__': + # 测试代码 + test_data = [ + { + 'bvid': 'BV1234567890', + 'title': 'Python 爬虫入门教程', + 'nickname': '技术UP主', + 'play_count': 10000, + 'liked_count': 500, + 'coin_count': 200, + 'favorite_count': 300, + 'share_count': 50, + 'danmaku_count': 100, + 'comment_count': 80, + 'source_keyword': 'Python教程', + }, + { + 'bvid': 'BV0987654321', + 'title': 'Python 数据分析实战', + 'nickname': '数据分析师', + 'play_count': 8000, + 'liked_count': 400, + 'coin_count': 150, + 'favorite_count': 250, + 'share_count': 30, + 'danmaku_count': 80, + 'comment_count': 60, + 'source_keyword': 'Python教程', + }, + ] + + report_path = generate_report(test_data, './test_output') + print(f"报告已生成: {report_path}") diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/client/__init__.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/client/__init__.py" new file mode 100644 index 0000000..79a75e7 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/client/__init__.py" @@ -0,0 +1,7 @@ +# -*- coding: utf-8 -*- +""" +B站 API 客户端模块 +""" +from .bilibili_client import BilibiliClient + +__all__ = ["BilibiliClient"] diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/client/bilibili_client.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/client/bilibili_client.py" new file mode 100644 index 0000000..09839f7 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/client/bilibili_client.py" @@ -0,0 +1,369 @@ +# -*- coding: utf-8 -*- +""" +B站 API 客户端 + +本模块封装了 B站的 API 请求，包括： +- 视频搜索 API +- 视频详情 API +- WBI 签名处理 + +使用 httpx 作为 HTTP 客户端，支持异步请求。 + +参考 MediaCrawler 项目的实现： +- https://github.com/NanmiCoder/MediaCrawler/blob/main/media_platform/bilibili/client.py +""" + +import json +from typing import Dict, Optional, List, Callable, Any +from loguru import logger + +# 可选依赖 +try: + import httpx + HAS_HTTPX = True +except ImportError: + HAS_HTTPX = False + +try: + from playwright.async_api import BrowserContext, Page + HAS_PLAYWRIGHT = True +except ImportError: + HAS_PLAYWRIGHT = False + +from tools.sign import BilibiliSign, extract_wbi_keys_from_urls +from models.bilibili import BilibiliVideo, BilibiliSearchResponse +from config import bilibili_config + + +class BilibiliClient: + """ + B站 API 客户端 + + 封装 B站的 API 请求，支持 WBI 签名。 + + 使用示例： + ```python + client = BilibiliClient() + await client.update_cookies(browser_context) + + # 搜索视频 + videos = await client.search_video_by_keyword("Python教程", page=1) + + # 获取视频详情 + video = await client.get_video_info(bvid="BV1xx411c7mD") + ``` + """ + + def __init__(self): + """初始化客户端""" + self.headers = bilibili_config.DEFAULT_HEADERS.copy() + self.cookie_dict: Dict[str, str] = {} + self._signer: Optional[BilibiliSign] = None + self._timeout = bilibili_config.REQUEST_TIMEOUT + + async def update_cookies(self, browser_context: "BrowserContext"): + """ + 从浏览器上下文更新 Cookie + + 登录成功后调用此方法，将浏览器的 Cookie 同步到客户端。 + + Args: + browser_context: Playwright 浏览器上下文 + """ + cookies = await browser_context.cookies() + cookie_str = "; ".join([f"{c['name']}={c['value']}" for c in cookies]) + self.headers["Cookie"] = cookie_str + self.cookie_dict = {c['name']: c['value'] for c in cookies} + logger.info(f"[BilibiliClient] 更新了 {len(cookies)} 个 Cookie") + + async def init_wbi_sign(self, page: "Page"): + """ + 初始化 WBI 签名器 + + 从浏览器的 localStorage 中获取 WBI 密钥。 + + Args: + page: Playwright 页面对象 + """ + try: + # 从 localStorage 获取 wbi_img_urls + wbi_img_urls = await page.evaluate(""" + () => { + return localStorage.getItem('wbi_img_urls'); + } + """) + + if not wbi_img_urls: + logger.warning("[BilibiliClient] 未找到 wbi_img_urls，尝试从 API 获取") + await self._fetch_wbi_keys() + return + + # 解析 JSON + wbi_data = json.loads(wbi_img_urls) + img_url = wbi_data.get("imgUrl", "") + sub_url = wbi_data.get("subUrl", "") + + if img_url and sub_url: + img_key, sub_key = extract_wbi_keys_from_urls(img_url, sub_url) + self._signer = BilibiliSign(img_key, sub_key) + logger.info("[BilibiliClient] WBI 签名器初始化成功") + else: + logger.warning("[BilibiliClient] wbi_img_urls 数据不完整") + await self._fetch_wbi_keys() + + except Exception as e: + logger.error(f"[BilibiliClient] 初始化 WBI 签名器失败: {e}") + await self._fetch_wbi_keys() + + async def _fetch_wbi_keys(self): + """ + 从 API 获取 WBI 密钥（备用方案） + """ + try: + async with httpx.AsyncClient(timeout=self._timeout) as client: + response = await client.get( + "https://api.bilibili.com/x/web-interface/nav", + headers=self.headers + ) + data = response.json() + + if data.get("code") == 0: + wbi_img = data.get("data", {}).get("wbi_img", {}) + img_url = wbi_img.get("img_url", "") + sub_url = wbi_img.get("sub_url", "") + + if img_url and sub_url: + img_key, sub_key = extract_wbi_keys_from_urls(img_url, sub_url) + self._signer = BilibiliSign(img_key, sub_key) + logger.info("[BilibiliClient] 从 API 获取 WBI 密钥成功") + return + + logger.error("[BilibiliClient] 无法获取 WBI 密钥") + + except Exception as e: + logger.error(f"[BilibiliClient] 获取 WBI 密钥失败: {e}") + + async def _request( + self, + method: str, + url: str, + params: Optional[Dict] = None, + data: Optional[Dict] = None, + enable_sign: bool = False + ) -> Optional[Dict]: + """ + 发送 HTTP 请求 + + Args: + method: 请求方法（GET/POST） + url: 请求 URL + params: URL 参数 + data: POST 数据 + enable_sign: 是否启用 WBI 签名 + + Returns: + Dict: 响应数据 + """ + if not HAS_HTTPX: + logger.error("[BilibiliClient] httpx 未安装") + return None + + try: + # 如果需要签名 + if enable_sign and self._signer and params: + params = self._signer.sign(params) + + async with httpx.AsyncClient(timeout=self._timeout) as client: + if method.upper() == "GET": + response = await client.get(url, params=params, headers=self.headers) + else: + response = await client.post(url, params=params, data=data, headers=self.headers) + + if response.status_code == 200: + return response.json() + else: + logger.error(f"[BilibiliClient] 请求失败: {response.status_code}") + return None + + except Exception as e: + logger.error(f"[BilibiliClient] 请求出错: {e}") + return None + + async def search_video_by_keyword( + self, + keyword: str, + page: int = 1, + page_size: int = 20, + order: str = "", + ) -> List[BilibiliVideo]: + """ + 按关键词搜索视频 + + Args: + keyword: 搜索关键词 + page: 页码（从1开始） + page_size: 每页数量（B站固定为20） + order: 排序方式（空=综合，click=最多点击，pubdate=最新发布） + + Returns: + List[BilibiliVideo]: 视频列表 + """ + logger.info(f"[BilibiliClient] 搜索视频: {keyword}, 第 {page} 页") + + params = { + "keyword": keyword, + "search_type": "video", + "page": page, + "page_size": page_size, + "order": order, + } + + data = await self._request( + "GET", + bilibili_config.SEARCH_URL, + params=params, + enable_sign=True + ) + + if not data: + return [] + + if data.get("code") != 0: + logger.error(f"[BilibiliClient] 搜索失败: {data.get('message')}") + return [] + + # 解析搜索结果 + result = data.get("data", {}) + video_list = result.get("result", []) + + videos = [] + for item in video_list: + try: + video = BilibiliVideo.from_search_result(item, keyword) + videos.append(video) + except Exception as e: + logger.debug(f"[BilibiliClient] 解析视频失败: {e}") + continue + + logger.info(f"[BilibiliClient] 搜索到 {len(videos)} 个视频") + return videos + + async def get_video_info( + self, + aid: Optional[str] = None, + bvid: Optional[str] = None + ) -> Optional[BilibiliVideo]: + """ + 获取视频详情 + + aid 和 bvid 至少提供一个。 + + Args: + aid: 视频 aid + bvid: 视频 BV 号 + + Returns: + BilibiliVideo: 视频信息 + """ + if not aid and not bvid: + logger.error("[BilibiliClient] aid 和 bvid 至少提供一个") + return None + + params = {} + if bvid: + params["bvid"] = bvid + elif aid: + params["aid"] = aid + + logger.info(f"[BilibiliClient] 获取视频详情: {bvid or aid}") + + data = await self._request( + "GET", + bilibili_config.VIDEO_INFO_URL, + params=params, + enable_sign=False # 视频详情 API 不需要签名 + ) + + if not data: + return None + + if data.get("code") != 0: + logger.error(f"[BilibiliClient] 获取视频详情失败: {data.get('message')}") + return None + + video_data = data.get("data", {}) + return BilibiliVideo.from_api_response(video_data) + + async def get_video_info_batch( + self, + bvid_list: List[str], + callback: Optional[Callable[[BilibiliVideo], Any]] = None + ) -> List[BilibiliVideo]: + """ + 批量获取视频详情 + + Args: + bvid_list: BV 号列表 + callback: 每获取一个视频后的回调函数 + + Returns: + List[BilibiliVideo]: 视频列表 + """ + videos = [] + + for bvid in bvid_list: + video = await self.get_video_info(bvid=bvid) + if video: + videos.append(video) + if callback: + await callback(video) if asyncio.iscoroutinefunction(callback) else callback(video) + + return videos + + async def pong(self) -> bool: + """ + 检查登录状态 + + 通过调用 nav API 检查是否已登录。 + + Returns: + bool: 是否已登录 + """ + try: + data = await self._request( + "GET", + "https://api.bilibili.com/x/web-interface/nav", + enable_sign=False + ) + + if data and data.get("code") == 0: + user_data = data.get("data", {}) + if user_data.get("isLogin"): + username = user_data.get("uname", "未知用户") + logger.info(f"[BilibiliClient] 已登录: {username}") + return True + + return False + + except Exception as e: + logger.debug(f"[BilibiliClient] 检查登录状态失败: {e}") + return False + + +# 为了兼容性，添加 asyncio 导入 +import asyncio + + +if __name__ == '__main__': + # 测试代码 + async def test(): + client = BilibiliClient() + + # 测试搜索（无需登录，但可能被限制） + videos = await client.search_video_by_keyword("Python教程", page=1) + print(f"搜索到 {len(videos)} 个视频") + + for video in videos[:3]: + print(f" - {video.title} (播放: {video.play_count})") + + asyncio.run(test()) diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/config/__init__.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/config/__init__.py" new file mode 100644 index 0000000..668c896 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/config/__init__.py" @@ -0,0 +1,15 @@ +# -*- coding: utf-8 -*- +""" +配置模块 +""" +from .settings import settings, Settings, StorageType, LoginType, CrawlerType +from . import bilibili_config + +__all__ = [ + 'settings', + 'Settings', + 'StorageType', + 'LoginType', + 'CrawlerType', + 'bilibili_config', +] diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/config/bilibili_config.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/config/bilibili_config.py" new file mode 100644 index 0000000..d039bc6 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/config/bilibili_config.py" @@ -0,0 +1,100 @@ +# -*- coding: utf-8 -*- +""" +B站特定配置 + +本模块包含 B站 API 相关的常量和配置，包括： +- API 端点地址 +- 请求头配置 +- WBI 签名相关常量 +- 搜索排序类型 +""" + +from enum import Enum + + +class SearchOrderType(str, Enum): + """搜索排序类型""" + DEFAULT = "" # 综合排序 + MOST_CLICK = "click" # 最多点击 + LAST_PUBLISH = "pubdate" # 最新发布 + MOST_DANMU = "dm" # 最多弹幕 + MOST_MARK = "stow" # 最多收藏 + + +# ==================== API 端点 ==================== + +# B站主站 +BILIBILI_URL = "https://www.bilibili.com" + +# 搜索 API +SEARCH_URL = "https://api.bilibili.com/x/web-interface/wbi/search/type" + +# 视频详情 API +VIDEO_INFO_URL = "https://api.bilibili.com/x/web-interface/view" + +# 视频播放地址 API +VIDEO_PLAY_URL = "https://api.bilibili.com/x/player/playurl" + +# 用户信息 API +USER_INFO_URL = "https://api.bilibili.com/x/space/wbi/acc/info" + + +# ==================== 请求头配置 ==================== + +DEFAULT_HEADERS = { + "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36", + "Referer": "https://www.bilibili.com", + "Accept": "application/json, text/plain, */*", + "Accept-Language": "zh-CN,zh;q=0.9,en;q=0.8", + "Origin": "https://www.bilibili.com", +} + + +# ==================== WBI 签名相关 ==================== + +# WBI 签名混淆映射表（固定值） +WBI_MIXIN_KEY_ENC_TAB = [ + 46, 47, 18, 2, 53, 8, 23, 32, 15, 50, 10, 31, 58, 3, 45, 35, + 27, 43, 5, 49, 33, 9, 42, 19, 29, 28, 14, 39, 12, 38, 41, 13, + 37, 48, 7, 16, 24, 55, 40, 61, 26, 17, 0, 1, 60, 51, 30, 4, + 22, 25, 54, 21, 56, 59, 6, 63, 57, 62, 11, 36, 20, 34, 44, 52, +] + + +# ==================== 登录相关 ==================== + +# 登录页面 URL +LOGIN_URL = "https://www.bilibili.com" + +# 登录按钮选择器 +LOGIN_BUTTON_SELECTOR = "xpath=//div[@class='right-entry__outside go-login-btn']//div" + +# 二维码选择器 +QRCODE_SELECTOR = "//div[@class='login-scan-box']//img" + +# 登录成功后的 Cookie 关键字段 +LOGIN_COOKIE_KEYS = ["SESSDATA", "DedeUserID", "bili_jct"] + + +# ==================== 页面选择器 ==================== + +# 搜索结果项选择器 +SEARCH_ITEM_SELECTOR = ".video-list-item" + +# 视频标题选择器 +VIDEO_TITLE_SELECTOR = ".title" + +# 视频播放量选择器 +VIDEO_PLAY_COUNT_SELECTOR = ".play-count" + + +# ==================== 其他配置 ==================== + +# 每页搜索结果数量（B站固定值） +SEARCH_PAGE_SIZE = 20 + +# 最大重试次数 +MAX_RETRY_COUNT = 3 + +# 请求超时时间（秒） +REQUEST_TIMEOUT = 10 diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/config/settings.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/config/settings.py" new file mode 100644 index 0000000..24f7d61 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/config/settings.py" @@ -0,0 +1,121 @@ +# -*- coding: utf-8 -*- +""" +项目配置模块 + +本模块定义了 B站爬虫项目的所有配置项，包括： +- 基础配置（应用名称、调试模式） +- 浏览器配置（无头模式、超时时间） +- 登录配置（登录方式、Cookie文件） +- 代理配置（是否启用、API地址） +- 爬虫配置（最大页数、延迟时间） +- 存储配置（存储类型、输出目录） +""" + +from typing import Optional, List +from enum import Enum + +# 尝试导入 pydantic-settings，如果不存在则使用简单的配置类 +try: + from pydantic_settings import BaseSettings + from pydantic import Field + HAS_PYDANTIC_SETTINGS = True +except ImportError: + HAS_PYDANTIC_SETTINGS = False + + +class StorageType(str, Enum): + """存储类型枚举""" + JSON = "json" + CSV = "csv" + + +class LoginType(str, Enum): + """登录类型枚举""" + COOKIE = "cookie" + QRCODE = "qrcode" + + +class CrawlerType(str, Enum): + """爬虫类型枚举""" + SEARCH = "search" # 关键词搜索 + DETAIL = "detail" # 指定视频详情 + + +if HAS_PYDANTIC_SETTINGS: + class Settings(BaseSettings): + """项目配置（使用 pydantic-settings）""" + + # 基础配置 + app_name: str = "BilibiliCrawler" + debug: bool = False + + # 浏览器配置 + browser_headless: bool = False # B站扫码登录需要显示浏览器 + browser_timeout: int = 30000 + browser_user_data_dir: Optional[str] = "browser_data/bili_user_data" + + # 登录配置 + login_type: LoginType = LoginType.QRCODE + cookie_str: str = "" # Cookie 字符串，当 login_type=cookie 时使用 + save_login_state: bool = True # 是否保存登录状态 + + # 代理配置 + proxy_enabled: bool = False + proxy_api_url: Optional[str] = None + + # B站爬虫配置 + crawler_type: CrawlerType = CrawlerType.SEARCH + keywords: str = "Python教程" # 搜索关键词，多个用逗号分隔 + specified_id_list: List[str] = [] # 指定视频列表（BV号） + max_video_count: int = 20 # 最大爬取视频数量 + crawl_delay_min: float = 1.0 + crawl_delay_max: float = 3.0 + max_concurrency: int = 3 # 最大并发数 + + # 存储配置 + storage_type: StorageType = StorageType.JSON + storage_output_dir: str = "./output" + + class Config: + env_file = ".env" + env_prefix = "BILI_" + +else: + class Settings: + """项目配置（简单实现，无 pydantic-settings 依赖）""" + + def __init__(self): + # 基础配置 + self.app_name = "BilibiliCrawler" + self.debug = False + + # 浏览器配置 + self.browser_headless = False # B站扫码登录需要显示浏览器 + self.browser_timeout = 30000 + self.browser_user_data_dir = "browser_data/bili_user_data" + + # 登录配置 + self.login_type = LoginType.QRCODE + self.cookie_str = "" # Cookie 字符串 + self.save_login_state = True + + # 代理配置 + self.proxy_enabled = False + self.proxy_api_url = None + + # B站爬虫配置 + self.crawler_type = CrawlerType.SEARCH + self.keywords = "Python教程" + self.specified_id_list = [] + self.max_video_count = 20 + self.crawl_delay_min = 1.0 + self.crawl_delay_max = 3.0 + self.max_concurrency = 3 + + # 存储配置 + self.storage_type = StorageType.JSON + self.storage_output_dir = "./output" + + +# 全局配置实例 +settings = Settings() diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/core/__init__.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/core/__init__.py" new file mode 100644 index 0000000..db73a96 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/core/__init__.py" @@ -0,0 +1,4 @@ +# -*- coding: utf-8 -*- +from .browser import BrowserManager + +__all__ = ['BrowserManager'] diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/core/browser.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/core/browser.py" new file mode 100644 index 0000000..00c76e0 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/core/browser.py" @@ -0,0 +1,155 @@ +# -*- coding: utf-8 -*- +# @Desc: 浏览器管理模块 +from __future__ import annotations + +from typing import Optional +from loguru import logger + +# 可选依赖 +try: + from playwright.async_api import async_playwright, Browser, BrowserContext, Page + HAS_PLAYWRIGHT = True +except ImportError: + HAS_PLAYWRIGHT = False + logger.warning("playwright 未安装") + +# stealth.min.js 内容（简化版） +STEALTH_JS = """ +// 隐藏 webdriver 属性 +Object.defineProperty(navigator, 'webdriver', { + get: () => undefined +}); + +// 修改 plugins +Object.defineProperty(navigator, 'plugins', { + get: () => [1, 2, 3, 4, 5] +}); + +// 修改 languages +Object.defineProperty(navigator, 'languages', { + get: () => ['zh-CN', 'zh', 'en'] +}); + +// 修改 platform +Object.defineProperty(navigator, 'platform', { + get: () => 'MacIntel' +}); + +// 隐藏 automation 特征 +window.chrome = { + runtime: {} +}; + +// 修改 permissions +const originalQuery = window.navigator.permissions.query; +window.navigator.permissions.query = (parameters) => ( + parameters.name === 'notifications' ? + Promise.resolve({ state: Notification.permission }) : + originalQuery(parameters) +); +""" + + +class BrowserManager: + """浏览器管理器""" + + def __init__( + self, + headless: bool = True, + timeout: int = 30000, + user_data_dir: str = None, + proxy: str = None + ): + """ + 初始化浏览器管理器 + + Args: + headless: 是否无头模式 + timeout: 默认超时时间（毫秒） + user_data_dir: 用户数据目录 + proxy: 代理服务器地址 + """ + if not HAS_PLAYWRIGHT: + raise ImportError("请安装 playwright: pip install playwright && playwright install") + + self.headless = headless + self.timeout = timeout + self.user_data_dir = user_data_dir + self.proxy = proxy + + self._playwright = None + self._browser: Optional[Browser] = None + self._context: Optional[BrowserContext] = None + + async def start(self) -> BrowserContext: + """启动浏览器""" + self._playwright = await async_playwright().start() + + # 浏览器启动参数 + launch_args = [ + '--disable-blink-features=AutomationControlled', + '--no-sandbox', + '--disable-dev-shm-usage', + ] + + # 启动浏览器 + self._browser = await self._playwright.chromium.launch( + headless=self.headless, + args=launch_args + ) + + # 创建上下文 + context_options = { + 'viewport': {'width': 1920, 'height': 1080}, + 'user_agent': ( + 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) ' + 'AppleWebKit/537.36 (KHTML, like Gecko) ' + 'Chrome/131.0.0.0 Safari/537.36' + ), + 'locale': 'zh-CN', + 'timezone_id': 'Asia/Shanghai', + } + + if self.proxy: + context_options['proxy'] = {'server': self.proxy} + + self._context = await self._browser.new_context(**context_options) + + # 注入反检测脚本 + await self._context.add_init_script(STEALTH_JS) + + logger.info("浏览器启动成功") + return self._context + + async def new_page(self) -> 'Page': + """创建新页面""" + if not self._context: + await self.start() + page = await self._context.new_page() + page.set_default_timeout(self.timeout) + return page + + async def close(self): + """关闭浏览器""" + if self._context: + await self._context.close() + self._context = None + if self._browser: + await self._browser.close() + self._browser = None + if self._playwright: + await self._playwright.stop() + self._playwright = None + logger.info("浏览器已关闭") + + @property + def context(self) -> Optional[BrowserContext]: + """获取浏览器上下文""" + return self._context + + async def __aenter__(self): + await self.start() + return self + + async def __aexit__(self, exc_type, exc_val, exc_tb): + await self.close() diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/crawler/__init__.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/crawler/__init__.py" new file mode 100644 index 0000000..524d609 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/crawler/__init__.py" @@ -0,0 +1,5 @@ +# -*- coding: utf-8 -*- +"""爬虫模块""" +from .spider import BilibiliCrawler, run_crawler + +__all__ = ['BilibiliCrawler', 'run_crawler'] diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/crawler/spider.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/crawler/spider.py" new file mode 100644 index 0000000..bab1756 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/crawler/spider.py" @@ -0,0 +1,328 @@ +# -*- coding: utf-8 -*- +""" +B站爬虫模块 + +本模块实现了 B站视频数据的爬取功能，包括： +- 关键词搜索视频 +- 获取指定视频详情 +- 并发爬取控制 +- 数据存储回调 + +参考 MediaCrawler 项目的实现： +- https://github.com/NanmiCoder/MediaCrawler/blob/main/media_platform/bilibili/core.py +""" + +import asyncio +import random +from typing import List, Dict, Optional, Callable, Any +from loguru import logger + +# 可选依赖 +try: + from playwright.async_api import async_playwright, Browser, BrowserContext, Page + HAS_PLAYWRIGHT = True +except ImportError: + HAS_PLAYWRIGHT = False + +from config import settings, CrawlerType +from config import bilibili_config +from core.browser import BrowserManager +from login.auth import BilibiliLogin +from client.bilibili_client import BilibiliClient +from models.bilibili import BilibiliVideo +from tools.sign import parse_video_info_from_url + + +class BilibiliCrawler: + """ + B站爬虫类 + + 整合浏览器管理、登录认证、API客户端，实现完整的爬取流程。 + + 使用示例： + ```python + crawler = BilibiliCrawler() + videos = await crawler.start() + ``` + """ + + def __init__(self): + """初始化爬虫""" + self.browser_manager: Optional[BrowserManager] = None + self.browser_context: Optional[BrowserContext] = None + self.context_page: Optional[Page] = None + self.bili_client: Optional[BilibiliClient] = None + + # 爬取结果 + self._results: List[BilibiliVideo] = [] + + # 配置 + self.max_video_count = settings.max_video_count + self.max_concurrency = settings.max_concurrency + self.delay_min = settings.crawl_delay_min + self.delay_max = settings.crawl_delay_max + + async def start(self) -> List[BilibiliVideo]: + """ + 启动爬虫 + + 完整流程： + 1. 启动浏览器 + 2. 执行登录 + 3. 初始化 API 客户端 + 4. 根据配置执行爬取 + 5. 关闭浏览器 + + Returns: + List[BilibiliVideo]: 爬取的视频列表 + """ + logger.info(f"[BilibiliCrawler] 启动爬虫，类型: {settings.crawler_type}") + + try: + # 1. 启动浏览器 + await self._init_browser() + + # 2. 执行登录 + login_success = await self._do_login() + if not login_success: + logger.error("[BilibiliCrawler] 登录失败，退出") + return [] + + # 3. 初始化 API 客户端 + await self._init_client() + + # 4. 根据配置执行爬取 + if settings.crawler_type == CrawlerType.SEARCH: + await self.search_by_keywords() + elif settings.crawler_type == CrawlerType.DETAIL: + await self.get_specified_videos() + else: + logger.error(f"[BilibiliCrawler] 不支持的爬取类型: {settings.crawler_type}") + + logger.info(f"[BilibiliCrawler] 爬取完成，共 {len(self._results)} 个视频") + return self._results + + except Exception as e: + logger.exception(f"[BilibiliCrawler] 爬取出错: {e}") + return self._results + + finally: + # 5. 关闭浏览器 + await self.close() + + async def _init_browser(self): + """初始化浏览器""" + logger.info("[BilibiliCrawler] 初始化浏览器...") + + self.browser_manager = BrowserManager( + headless=settings.browser_headless, + timeout=settings.browser_timeout, + user_data_dir=settings.browser_user_data_dir if settings.save_login_state else None + ) + + self.browser_context = await self.browser_manager.start() + self.context_page = await self.browser_manager.new_page() + + logger.info("[BilibiliCrawler] 浏览器初始化完成") + + async def _do_login(self) -> bool: + """ + 执行登录 + + Returns: + bool: 是否登录成功 + """ + # 先检查是否已登录（通过保存的状态） + self.bili_client = BilibiliClient() + await self.bili_client.update_cookies(self.browser_context) + + if await self.bili_client.pong(): + logger.info("[BilibiliCrawler] 已有登录状态，跳过登录") + return True + + # 执行登录 + login = BilibiliLogin( + login_type=settings.login_type.value, + browser_context=self.browser_context, + context_page=self.context_page, + cookie_str=settings.cookie_str + ) + + success = await login.begin() + + if success: + # 更新客户端 Cookie + await self.bili_client.update_cookies(self.browser_context) + + return success + + async def _init_client(self): + """初始化 API 客户端""" + # 初始化 WBI 签名器 + await self.bili_client.init_wbi_sign(self.context_page) + + async def search_by_keywords(self) -> List[BilibiliVideo]: + """ + 按关键词搜索视频 + + 支持多个关键词（逗号分隔）。 + + Returns: + List[BilibiliVideo]: 视频列表 + """ + keywords = [kw.strip() for kw in settings.keywords.split(",") if kw.strip()] + + if not keywords: + logger.warning("[BilibiliCrawler] 未配置搜索关键词") + return [] + + logger.info(f"[BilibiliCrawler] 开始搜索，关键词: {keywords}") + + for keyword in keywords: + await self._search_single_keyword(keyword) + + # 达到最大数量后停止 + if len(self._results) >= self.max_video_count: + break + + return self._results + + async def _search_single_keyword(self, keyword: str): + """ + 搜索单个关键词 + + Args: + keyword: 搜索关键词 + """ + page = 1 + page_size = bilibili_config.SEARCH_PAGE_SIZE + + while len(self._results) < self.max_video_count: + logger.info(f"[BilibiliCrawler] 搜索 '{keyword}'，第 {page} 页") + + # 搜索视频 + videos = await self.bili_client.search_video_by_keyword( + keyword=keyword, + page=page, + page_size=page_size + ) + + if not videos: + logger.info(f"[BilibiliCrawler] '{keyword}' 第 {page} 页无结果，停止搜索") + break + + # 获取视频详情 + for video in videos: + if len(self._results) >= self.max_video_count: + break + + # 获取完整视频详情 + video_detail = await self.bili_client.get_video_info(bvid=video.bvid) + if video_detail: + video_detail.source_keyword = keyword + self._results.append(video_detail) + logger.info(f"[BilibiliCrawler] 获取视频: {video_detail.title[:30]}...") + else: + # 如果获取详情失败，使用搜索结果 + self._results.append(video) + + # 随机延迟 + await self._random_delay() + + page += 1 + + # 防止无限循环 + if page > 50: + break + + async def get_specified_videos(self) -> List[BilibiliVideo]: + """ + 获取指定视频列表的详情 + + 从配置中读取视频列表（BV号或URL）。 + + Returns: + List[BilibiliVideo]: 视频列表 + """ + video_list = settings.specified_id_list + + if not video_list: + logger.warning("[BilibiliCrawler] 未配置指定视频列表") + return [] + + logger.info(f"[BilibiliCrawler] 获取 {len(video_list)} 个指定视频") + + for video_id in video_list: + if len(self._results) >= self.max_video_count: + break + + # 解析 BV 号 + try: + video_info = parse_video_info_from_url(video_id) + bvid = video_info.video_id + except ValueError: + logger.warning(f"[BilibiliCrawler] 无法解析视频 ID: {video_id}") + continue + + # 获取视频详情 + video = await self.bili_client.get_video_info(bvid=bvid) + if video: + self._results.append(video) + logger.info(f"[BilibiliCrawler] 获取视频: {video.title[:30]}...") + + # 随机延迟 + await self._random_delay() + + return self._results + + async def _random_delay(self): + """随机延迟，避免请求过快""" + delay = random.uniform(self.delay_min, self.delay_max) + await asyncio.sleep(delay) + + def get_results(self) -> List[BilibiliVideo]: + """ + 获取爬取结果 + + Returns: + List[BilibiliVideo]: 视频列表 + """ + return self._results + + async def close(self): + """关闭浏览器""" + if self.browser_manager: + await self.browser_manager.close() + logger.info("[BilibiliCrawler] 浏览器已关闭") + + +async def run_crawler( + on_video: Optional[Callable[[BilibiliVideo], Any]] = None +) -> List[BilibiliVideo]: + """ + 运行爬虫（便捷函数） + + Args: + on_video: 每获取一个视频后的回调函数 + + Returns: + List[BilibiliVideo]: 视频列表 + """ + crawler = BilibiliCrawler() + return await crawler.start() + + +if __name__ == '__main__': + # 测试代码 + async def test(): + crawler = BilibiliCrawler() + videos = await crawler.start() + + print(f"\n爬取完成，共 {len(videos)} 个视频：") + for i, video in enumerate(videos[:5], 1): + print(f"{i}. {video.title}") + print(f" UP主: {video.nickname}") + print(f" 播放: {video.play_count}") + print() + + asyncio.run(test()) diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/login/__init__.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/login/__init__.py" new file mode 100644 index 0000000..5923317 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/login/__init__.py" @@ -0,0 +1,19 @@ +# -*- coding: utf-8 -*- +"""登录认证模块""" +from .auth import ( + BilibiliLogin, + AbstractLogin, + convert_cookies_to_str, + convert_cookies_to_dict, + save_cookies_to_file, + load_cookies_from_file, +) + +__all__ = [ + 'BilibiliLogin', + 'AbstractLogin', + 'convert_cookies_to_str', + 'convert_cookies_to_dict', + 'save_cookies_to_file', + 'load_cookies_from_file', +] diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/login/auth.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/login/auth.py" new file mode 100644 index 0000000..41cd240 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/login/auth.py" @@ -0,0 +1,487 @@ +# -*- coding: utf-8 -*- +""" +B站登录认证模块 + +本模块实现了 B站的登录认证功能，支持两种登录方式： +1. 扫码登录：显示二维码，用户使用 B站 APP 扫码登录 +2. Cookie 登录：使用已有的 Cookie 字符串直接登录 + +登录成功后会将 Cookie 保存到浏览器上下文中，后续请求自动携带。 + +参考 MediaCrawler 项目的实现： +- https://github.com/NanmiCoder/MediaCrawler/blob/main/media_platform/bilibili/login.py +""" + +import asyncio +import base64 +import json +from abc import ABC, abstractmethod +from pathlib import Path +from typing import Optional, List, Dict +from loguru import logger + +# 可选依赖 +try: + from playwright.async_api import BrowserContext, Page + HAS_PLAYWRIGHT = True +except ImportError: + HAS_PLAYWRIGHT = False + +# 尝试导入图片显示库 +try: + from PIL import Image + from io import BytesIO + HAS_PIL = True +except ImportError: + HAS_PIL = False + +# 尝试导入 httpx +try: + import httpx + HAS_HTTPX = True +except ImportError: + HAS_HTTPX = False + + +# ==================== B站登录相关常量 ==================== + +# B站主页 +BILIBILI_URL = "https://www.bilibili.com" + +# 登录按钮选择器 +LOGIN_BUTTON_SELECTOR = "xpath=//div[@class='right-entry__outside go-login-btn']//div" + +# 二维码选择器 +QRCODE_SELECTOR = "//div[@class='login-scan-box']//img" + +# 登录成功后的 Cookie 关键字段 +LOGIN_COOKIE_KEYS = ["SESSDATA", "DedeUserID", "bili_jct"] + + +class AbstractLogin(ABC): + """登录抽象基类""" + + @abstractmethod + async def begin(self) -> bool: + """开始登录流程""" + pass + + @abstractmethod + async def login_by_qrcode(self) -> bool: + """扫码登录""" + pass + + @abstractmethod + async def login_by_cookies(self) -> bool: + """Cookie 登录""" + pass + + @abstractmethod + async def check_login_state(self) -> bool: + """检查登录状态""" + pass + + +class BilibiliLogin(AbstractLogin): + """ + B站登录类 + + 支持扫码登录和 Cookie 登录两种方式。 + + 使用示例： + ```python + login = BilibiliLogin( + login_type="qrcode", + browser_context=context, + context_page=page + ) + success = await login.begin() + ``` + """ + + def __init__( + self, + login_type: str, + browser_context: "BrowserContext", + context_page: "Page", + cookie_str: str = "", + ): + """ + 初始化 B站登录 + + Args: + login_type: 登录类型，"qrcode" 或 "cookie" + browser_context: Playwright 浏览器上下文 + context_page: Playwright 页面对象 + cookie_str: Cookie 字符串（当 login_type="cookie" 时使用） + """ + self.login_type = login_type + self.browser_context = browser_context + self.context_page = context_page + self.cookie_str = cookie_str + + async def begin(self) -> bool: + """ + 开始登录流程 + + 根据 login_type 自动选择登录方式。 + + Returns: + bool: 登录是否成功 + """ + logger.info(f"[BilibiliLogin] 开始登录，方式: {self.login_type}") + + if self.login_type == "qrcode": + return await self.login_by_qrcode() + elif self.login_type == "cookie": + return await self.login_by_cookies() + else: + logger.error(f"[BilibiliLogin] 不支持的登录类型: {self.login_type}") + return False + + async def login_by_qrcode(self) -> bool: + """ + 扫码登录 + + 流程： + 1. 访问 B站首页 + 2. 点击登录按钮 + 3. 获取二维码图片并显示 + 4. 等待用户扫码 + 5. 检查登录状态 + + Returns: + bool: 登录是否成功 + """ + logger.info("[BilibiliLogin] 开始扫码登录...") + + try: + # 1. 访问 B站首页 + await self.context_page.goto(BILIBILI_URL) + await asyncio.sleep(2) + + # 2. 点击登录按钮 + try: + login_button = await self.context_page.wait_for_selector( + LOGIN_BUTTON_SELECTOR, + timeout=10000 + ) + if login_button: + await login_button.click() + await asyncio.sleep(1) + except Exception as e: + logger.warning(f"[BilibiliLogin] 点击登录按钮失败: {e}") + # 可能已经有登录弹窗，继续尝试 + + # 3. 获取二维码 + qrcode_img = await self._find_login_qrcode() + if not qrcode_img: + logger.error("[BilibiliLogin] 未找到二维码") + return False + + # 4. 显示二维码 + await self._show_qrcode(qrcode_img) + + # 5. 等待登录成功 + logger.info("[BilibiliLogin] 请使用 B站 APP 扫描二维码登录...") + logger.info("[BilibiliLogin] 等待登录成功（最长等待 120 秒）...") + + # 轮询检查登录状态 + for _ in range(120): # 最多等待 120 秒 + if await self.check_login_state(): + logger.info("[BilibiliLogin] 扫码登录成功！") + await asyncio.sleep(2) # 等待页面跳转 + return True + await asyncio.sleep(1) + + logger.error("[BilibiliLogin] 扫码登录超时") + return False + + except Exception as e: + logger.error(f"[BilibiliLogin] 扫码登录失败: {e}") + return False + + async def login_by_cookies(self) -> bool: + """ + Cookie 登录 + + 将 Cookie 字符串解析后注入到浏览器上下文中。 + + Returns: + bool: 登录是否成功 + """ + logger.info("[BilibiliLogin] 开始 Cookie 登录...") + + if not self.cookie_str: + logger.error("[BilibiliLogin] Cookie 字符串为空") + return False + + try: + # 解析 Cookie 字符串 + cookies = self._parse_cookie_str(self.cookie_str) + if not cookies: + logger.error("[BilibiliLogin] Cookie 解析失败") + return False + + # 注入 Cookie + await self.browser_context.add_cookies(cookies) + logger.info(f"[BilibiliLogin] 成功注入 {len(cookies)} 个 Cookie") + + # 刷新页面验证 + await self.context_page.goto(BILIBILI_URL) + await asyncio.sleep(2) + + # 检查登录状态 + if await self.check_login_state(): + logger.info("[BilibiliLogin] Cookie 登录成功！") + return True + else: + logger.error("[BilibiliLogin] Cookie 登录失败，Cookie 可能已过期") + return False + + except Exception as e: + logger.error(f"[BilibiliLogin] Cookie 登录失败: {e}") + return False + + async def check_login_state(self) -> bool: + """ + 检查登录状态 + + 通过检查 Cookie 中是否包含关键字段来判断是否已登录。 + 关键字段：SESSDATA、DedeUserID + + Returns: + bool: 是否已登录 + """ + try: + # 获取当前 Cookie + cookies = await self.browser_context.cookies() + cookie_dict = {c['name']: c['value'] for c in cookies} + + # 检查关键 Cookie 是否存在 + for key in ["SESSDATA", "DedeUserID"]: + if key in cookie_dict and cookie_dict[key]: + return True + + return False + + except Exception as e: + logger.debug(f"[BilibiliLogin] 检查登录状态出错: {e}") + return False + + async def _find_login_qrcode(self) -> Optional[str]: + """ + 查找登录二维码 + + Returns: + str: Base64 编码的二维码图片，如果未找到返回 None + """ + try: + # 等待二维码出现 + qrcode_element = await self.context_page.wait_for_selector( + QRCODE_SELECTOR, + timeout=10000 + ) + + if not qrcode_element: + return None + + # 获取二维码图片 src + qrcode_src = await qrcode_element.get_attribute("src") + + if not qrcode_src: + return None + + # 如果是 URL，下载图片 + if qrcode_src.startswith("http"): + return await self._download_qrcode(qrcode_src) + + # 如果是 base64，直接返回 + if qrcode_src.startswith("data:image"): + return qrcode_src.split(",")[1] if "," in qrcode_src else qrcode_src + + return qrcode_src + + except Exception as e: + logger.error(f"[BilibiliLogin] 获取二维码失败: {e}") + return None + + async def _download_qrcode(self, url: str) -> Optional[str]: + """ + 下载二维码图片 + + Args: + url: 二维码图片 URL + + Returns: + str: Base64 编码的图片数据 + """ + if not HAS_HTTPX: + logger.warning("[BilibiliLogin] httpx 未安装，无法下载二维码") + return None + + try: + async with httpx.AsyncClient() as client: + response = await client.get(url, timeout=10) + if response.status_code == 200: + return base64.b64encode(response.content).decode('utf-8') + except Exception as e: + logger.error(f"[BilibiliLogin] 下载二维码失败: {e}") + + return None + + async def _show_qrcode(self, qrcode_base64: str): + """ + 显示二维码 + + 优先使用 PIL 显示，如果不可用则保存到文件。 + + Args: + qrcode_base64: Base64 编码的二维码图片 + """ + # 解码 base64 + try: + qrcode_bytes = base64.b64decode(qrcode_base64) + except Exception: + logger.error("[BilibiliLogin] 二维码 Base64 解码失败") + return + + # 保存到文件 + qrcode_path = Path("qrcode.png") + with open(qrcode_path, 'wb') as f: + f.write(qrcode_bytes) + logger.info(f"[BilibiliLogin] 二维码已保存到: {qrcode_path.absolute()}") + + # 尝试显示图片 + if HAS_PIL: + try: + image = Image.open(BytesIO(qrcode_bytes)) + image.show() + logger.info("[BilibiliLogin] 二维码已显示，请扫码登录") + except Exception as e: + logger.warning(f"[BilibiliLogin] 无法显示二维码: {e}") + logger.info(f"[BilibiliLogin] 请手动打开文件: {qrcode_path.absolute()}") + else: + logger.info(f"[BilibiliLogin] 请手动打开文件扫码: {qrcode_path.absolute()}") + + # 打印提示 + print("\n" + "=" * 60) + print(" 请使用 B站 APP 扫描二维码登录") + print(f" 二维码文件: {qrcode_path.absolute()}") + print(" 等待登录中...") + print("=" * 60 + "\n") + + def _parse_cookie_str(self, cookie_str: str) -> List[Dict]: + """ + 解析 Cookie 字符串 + + Args: + cookie_str: Cookie 字符串，格式如 "name1=value1; name2=value2" + + Returns: + List[Dict]: Playwright 格式的 Cookie 列表 + """ + cookies = [] + + for item in cookie_str.split(";"): + item = item.strip() + if not item or "=" not in item: + continue + + parts = item.split("=", 1) + name = parts[0].strip() + value = parts[1].strip() if len(parts) > 1 else "" + + if name: + cookies.append({ + "name": name, + "value": value, + "domain": ".bilibili.com", + "path": "/" + }) + + return cookies + + +# ==================== 工具函数 ==================== + +def convert_cookies_to_str(cookies: List[Dict]) -> str: + """ + 将 Cookie 列表转换为字符串 + + Args: + cookies: Playwright 格式的 Cookie 列表 + + Returns: + str: Cookie 字符串 + """ + return "; ".join([f"{c['name']}={c['value']}" for c in cookies]) + + +def convert_cookies_to_dict(cookies: List[Dict]) -> Dict[str, str]: + """ + 将 Cookie 列表转换为字典 + + Args: + cookies: Playwright 格式的 Cookie 列表 + + Returns: + Dict[str, str]: Cookie 字典 + """ + return {c['name']: c['value'] for c in cookies} + + +async def save_cookies_to_file(context: "BrowserContext", filepath: str): + """ + 保存 Cookie 到文件 + + Args: + context: Playwright 浏览器上下文 + filepath: 保存路径 + """ + cookies = await context.cookies() + with open(filepath, 'w', encoding='utf-8') as f: + json.dump(cookies, f, indent=2, ensure_ascii=False) + logger.info(f"[BilibiliLogin] Cookie 已保存到: {filepath}") + + +async def load_cookies_from_file(context: "BrowserContext", filepath: str) -> bool: + """ + 从文件加载 Cookie + + Args: + context: Playwright 浏览器上下文 + filepath: Cookie 文件路径 + + Returns: + bool: 是否加载成功 + """ + path = Path(filepath) + if not path.exists(): + logger.warning(f"[BilibiliLogin] Cookie 文件不存在: {filepath}") + return False + + try: + with open(filepath, 'r', encoding='utf-8') as f: + cookies = json.load(f) + await context.add_cookies(cookies) + logger.info(f"[BilibiliLogin] 从文件加载了 {len(cookies)} 个 Cookie") + return True + except Exception as e: + logger.error(f"[BilibiliLogin] 加载 Cookie 失败: {e}") + return False + + +if __name__ == '__main__': + # 测试 Cookie 解析 + test_cookie_str = "SESSDATA=abc123; DedeUserID=12345; bili_jct=xyz789" + login = BilibiliLogin( + login_type="cookie", + browser_context=None, + context_page=None, + cookie_str=test_cookie_str + ) + cookies = login._parse_cookie_str(test_cookie_str) + print("解析后的 Cookie:") + for c in cookies: + print(f" {c['name']}: {c['value']}") diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/main.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/main.py" new file mode 100644 index 0000000..3491b7c --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/main.py" @@ -0,0 +1,232 @@ +# -*- coding: utf-8 -*- +""" +B站视频数据采集工具 - 主程序入口 + +综合实战项目：B站视频数据采集与分析工具 + +功能特点： +- 多种登录方式（扫码登录 / Cookie 登录） +- 反检测浏览器自动化（Playwright + stealth.js） +- WBI 签名算法支持（B站 API 签名） +- 视频搜索和详情获取 +- 多格式数据存储（JSON / CSV） +- 词云和统计报告自动生成 + +参考 MediaCrawler 项目的实现： +- https://github.com/NanmiCoder/MediaCrawler + +使用方法： + python main.py + +配置说明： + 修改 config/settings.py 中的配置项： + - crawler_type: 爬取类型（search=搜索, detail=指定视频） + - keywords: 搜索关键词（逗号分隔） + - max_video_count: 最大爬取数量 + - login_type: 登录方式（qrcode=扫码, cookie=Cookie） +""" + +import asyncio +import sys +from pathlib import Path +from typing import List +from loguru import logger + +# 添加项目根目录到路径 +sys.path.insert(0, str(Path(__file__).parent)) + +# 导入各模块 +from config import settings, CrawlerType +from crawler.spider import BilibiliCrawler +from store.backend import StorageManager +from analysis.report import ReportGenerator, generate_report +from models.bilibili import BilibiliVideo + + +# 配置日志 +def setup_logger(): + """配置日志""" + logger.remove() + logger.add( + sys.stderr, + format="{time:YYYY-MM-DD HH:mm:ss} | " + "{level: <8} | " + "{message}", + level="INFO" + ) + logger.add( + "logs/bilibili_{time:YYYY-MM-DD}.log", + rotation="1 day", + retention="7 days", + level="DEBUG", + encoding="utf-8" + ) + + +async def run_crawler() -> List[BilibiliVideo]: + """ + 运行爬虫 + + 完整流程： + 1. 启动浏览器 + 2. 执行登录（扫码或Cookie） + 3. 初始化 API 客户端（获取 WBI 密钥） + 4. 根据配置执行爬取（搜索或指定视频） + 5. 返回爬取结果 + + Returns: + List[BilibiliVideo]: 爬取的视频列表 + """ + crawler = BilibiliCrawler() + return await crawler.start() + + +async def save_data(videos: List[BilibiliVideo]) -> str: + """ + 保存数据 + + Args: + videos: 视频列表 + + Returns: + str: 保存的文件路径 + """ + if not videos: + logger.warning("没有数据需要保存") + return "" + + # 转换为字典列表 + data = [video.to_dict() for video in videos] + + # 创建存储管理器 + storage = StorageManager( + storage_type=settings.storage_type.value, + output_dir=settings.storage_output_dir + ) + + # 保存数据 + success = await storage.save(data) + + if success: + return str(storage.filepath) + return "" + + +def generate_analysis_report(videos: List[BilibiliVideo]) -> str: + """ + 生成分析报告 + + Args: + videos: 视频列表 + + Returns: + str: 报告文件路径 + """ + if not videos: + logger.warning("没有数据，跳过报告生成") + return "" + + report_path = generate_report( + videos=videos, + output_dir=settings.storage_output_dir + ) + + return report_path + + +async def main(): + """主函数""" + + # 打印欢迎信息 + print(""" + ╔══════════════════════════════════════════════════════════╗ + ║ B站视频数据采集与分析工具 v2.0 ║ + ║ ║ + ║ 功能： ║ + ║ - 视频搜索与详情获取 ║ + ║ - 扫码登录 / Cookie 登录 ║ + ║ - JSON / CSV 数据存储 ║ + ║ - 词云和统计分析报告 ║ + ║ ║ + ║ 参考项目：MediaCrawler ║ + ║ 注意：请遵守 B站的使用条款和法律法规 ║ + ╚══════════════════════════════════════════════════════════╝ + """) + + # 显示当前配置 + logger.info(f"启动 {settings.app_name}") + logger.info(f"爬取类型: {settings.crawler_type.value}") + logger.info(f"登录方式: {settings.login_type.value}") + logger.info(f"最大数量: {settings.max_video_count}") + logger.info(f"存储类型: {settings.storage_type.value}") + + if settings.crawler_type == CrawlerType.SEARCH: + logger.info(f"搜索关键词: {settings.keywords}") + else: + logger.info(f"指定视频: {len(settings.specified_id_list)} 个") + + logger.info("=" * 50) + + try: + # 1. 运行爬虫 + logger.info("开始爬取数据...") + videos = await run_crawler() + logger.info(f"爬取完成: {len(videos)} 条视频") + + if not videos: + logger.warning("没有爬取到数据，退出") + return + + # 2. 保存数据 + logger.info("保存数据...") + data_path = await save_data(videos) + if data_path: + logger.info(f"数据已保存: {data_path}") + + # 3. 生成分析报告 + logger.info("生成分析报告...") + report_path = generate_analysis_report(videos) + if report_path: + logger.info(f"报告已生成: {report_path}") + + # 4. 打印结果摘要 + logger.info("=" * 50) + logger.info("任务完成！") + logger.info(f"爬取视频: {len(videos)} 个") + if data_path: + logger.info(f"数据文件: {data_path}") + if report_path: + logger.info(f"分析报告: {report_path}") + logger.info("=" * 50) + + # 打印部分结果预览 + print("\n视频预览（前 5 条）:") + print("-" * 60) + for i, video in enumerate(videos[:5], 1): + print(f"{i}. {video.title[:40]}...") + print(f" UP主: {video.nickname}") + print(f" 播放: {video.play_count:,} 点赞: {video.liked_count:,}") + print() + + except KeyboardInterrupt: + logger.warning("用户中断执行") + except Exception as e: + logger.exception(f"执行出错: {e}") + raise + + +def cli(): + """命令行入口""" + # 设置日志 + setup_logger() + + # 创建必要的目录 + Path("logs").mkdir(exist_ok=True) + Path(settings.storage_output_dir).mkdir(parents=True, exist_ok=True) + + # 运行主程序 + asyncio.run(main()) + + +if __name__ == "__main__": + cli() diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/models/__init__.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/models/__init__.py" new file mode 100644 index 0000000..c423ca7 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/models/__init__.py" @@ -0,0 +1,7 @@ +# -*- coding: utf-8 -*- +""" +数据模型模块 +""" +from .bilibili import BilibiliVideo, BilibiliSearchResponse + +__all__ = ["BilibiliVideo", "BilibiliSearchResponse"] diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/models/bilibili.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/models/bilibili.py" new file mode 100644 index 0000000..baf8994 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/models/bilibili.py" @@ -0,0 +1,373 @@ +# -*- coding: utf-8 -*- +""" +B站数据模型 + +本模块定义了 B站爬虫使用的 Pydantic 数据模型，用于： +- 数据验证：确保爬取的数据符合预期格式 +- 数据序列化：方便存储为 JSON、CSV 等格式 +- 类型提示：提供更好的 IDE 支持和代码可读性 + +数据模型说明： +- BilibiliVideo: 视频信息模型，包含视频的所有元数据 +- BilibiliSearchResult: 搜索结果模型，对应 API 返回的搜索结果 +""" + +from datetime import datetime +from typing import Optional, List +from pydantic import BaseModel, Field + + +class BilibiliVideo(BaseModel): + """ + B站视频信息模型 + + 该模型对应 B站视频详情 API 返回的数据结构， + 包含了视频的基本信息、统计数据和作者信息。 + """ + + # ==================== 视频基本信息 ==================== + + video_id: str = Field( + description="视频 aid（旧版 ID）" + ) + bvid: str = Field( + description="视频 BV 号（新版 ID）" + ) + title: str = Field( + description="视频标题" + ) + desc: str = Field( + default="", + description="视频描述/简介" + ) + cover_url: str = Field( + default="", + description="视频封面图 URL" + ) + duration: int = Field( + default=0, + description="视频时长（秒）" + ) + create_time: int = Field( + default=0, + description="发布时间戳" + ) + pubdate_str: str = Field( + default="", + description="发布时间（格式化字符串）" + ) + + # ==================== UP主信息 ==================== + + user_id: int = Field( + description="UP主 UID" + ) + nickname: str = Field( + default="", + description="UP主昵称" + ) + avatar: str = Field( + default="", + description="UP主头像 URL" + ) + + # ==================== 统计数据 ==================== + + play_count: int = Field( + default=0, + description="播放量" + ) + danmaku_count: int = Field( + default=0, + description="弹幕数" + ) + comment_count: int = Field( + default=0, + description="评论数" + ) + liked_count: int = Field( + default=0, + description="点赞数" + ) + coin_count: int = Field( + default=0, + description="投币数" + ) + favorite_count: int = Field( + default=0, + description="收藏数" + ) + share_count: int = Field( + default=0, + description="分享数" + ) + + # ==================== 其他信息 ==================== + + video_url: str = Field( + default="", + description="视频页面 URL" + ) + tname: str = Field( + default="", + description="视频分区名称" + ) + source_keyword: str = Field( + default="", + description="搜索来源关键词" + ) + crawl_time: str = Field( + default="", + description="爬取时间" + ) + + def __init__(self, **data): + """初始化时自动设置默认值""" + # 如果没有提供 crawl_time，使用当前时间 + if 'crawl_time' not in data or not data['crawl_time']: + data['crawl_time'] = datetime.now().strftime('%Y-%m-%d %H:%M:%S') + + # 如果没有提供 video_url，根据 bvid 生成 + if 'video_url' not in data or not data['video_url']: + if 'bvid' in data: + data['video_url'] = f"https://www.bilibili.com/video/{data['bvid']}" + + # 如果没有提供 pubdate_str，根据 create_time 生成 + if 'pubdate_str' not in data or not data['pubdate_str']: + if 'create_time' in data and data['create_time']: + try: + dt = datetime.fromtimestamp(data['create_time']) + data['pubdate_str'] = dt.strftime('%Y-%m-%d %H:%M:%S') + except (ValueError, OSError): + pass + + super().__init__(**data) + + @classmethod + def from_api_response(cls, data: dict, source_keyword: str = "") -> "BilibiliVideo": + """ + 从 API 响应数据创建 BilibiliVideo 实例 + + Args: + data: API 返回的视频详情数据 + source_keyword: 搜索来源关键词 + + Returns: + BilibiliVideo: 视频信息模型实例 + """ + # 提取作者信息 + owner = data.get("owner", {}) + + # 提取统计信息 + stat = data.get("stat", {}) + + return cls( + video_id=str(data.get("aid", "")), + bvid=data.get("bvid", ""), + title=data.get("title", ""), + desc=data.get("desc", ""), + cover_url=data.get("pic", ""), + duration=data.get("duration", 0), + create_time=data.get("pubdate", 0), + user_id=owner.get("mid", 0), + nickname=owner.get("name", ""), + avatar=owner.get("face", ""), + play_count=stat.get("view", 0), + danmaku_count=stat.get("danmaku", 0), + comment_count=stat.get("reply", 0), + liked_count=stat.get("like", 0), + coin_count=stat.get("coin", 0), + favorite_count=stat.get("favorite", 0), + share_count=stat.get("share", 0), + tname=data.get("tname", ""), + source_keyword=source_keyword, + ) + + @classmethod + def from_search_result(cls, data: dict, source_keyword: str = "") -> "BilibiliVideo": + """ + 从搜索结果数据创建 BilibiliVideo 实例 + + 搜索结果的数据结构与视频详情略有不同，需要单独处理。 + + Args: + data: API 返回的搜索结果数据 + source_keyword: 搜索关键词 + + Returns: + BilibiliVideo: 视频信息模型实例 + """ + return cls( + video_id=str(data.get("aid", "")), + bvid=data.get("bvid", ""), + title=data.get("title", "").replace("", "").replace("", ""), + desc=data.get("description", ""), + cover_url="https:" + data.get("pic", "") if data.get("pic", "").startswith("//") else data.get("pic", ""), + duration=cls._parse_duration(data.get("duration", "0:00")), + create_time=data.get("pubdate", 0), + user_id=data.get("mid", 0), + nickname=data.get("author", ""), + avatar="", # 搜索结果中没有头像 + play_count=data.get("play", 0), + danmaku_count=data.get("danmaku", 0), + comment_count=data.get("review", 0), + liked_count=data.get("like", 0), + favorite_count=data.get("favorites", 0), + tname=data.get("typename", ""), + source_keyword=source_keyword, + ) + + @staticmethod + def _parse_duration(duration_str: str) -> int: + """ + 解析时长字符串为秒数 + + Args: + duration_str: 时长字符串，如 "3:45" 或 "1:23:45" + + Returns: + int: 时长（秒） + """ + if isinstance(duration_str, int): + return duration_str + + try: + parts = str(duration_str).split(":") + if len(parts) == 2: + return int(parts[0]) * 60 + int(parts[1]) + elif len(parts) == 3: + return int(parts[0]) * 3600 + int(parts[1]) * 60 + int(parts[2]) + return 0 + except (ValueError, TypeError): + return 0 + + def to_dict(self) -> dict: + """转换为字典格式""" + return self.model_dump() + + def to_csv_row(self) -> dict: + """ + 转换为适合 CSV 存储的行格式 + + Returns: + dict: CSV 行数据 + """ + return { + "BV号": self.bvid, + "标题": self.title, + "UP主": self.nickname, + "UP主ID": self.user_id, + "播放量": self.play_count, + "点赞数": self.liked_count, + "投币数": self.coin_count, + "收藏数": self.favorite_count, + "分享数": self.share_count, + "弹幕数": self.danmaku_count, + "评论数": self.comment_count, + "发布时间": self.pubdate_str, + "视频时长(秒)": self.duration, + "分区": self.tname, + "描述": self.desc[:100] + "..." if len(self.desc) > 100 else self.desc, + "视频链接": self.video_url, + "搜索关键词": self.source_keyword, + "爬取时间": self.crawl_time, + } + + +class BilibiliSearchResponse(BaseModel): + """ + B站搜索响应模型 + + 对应搜索 API 的完整响应结构。 + """ + + seid: str = Field(default="", description="搜索会话 ID") + page: int = Field(default=1, description="当前页码") + pagesize: int = Field(default=20, description="每页数量") + numResults: int = Field(default=0, description="搜索结果总数") + numPages: int = Field(default=0, description="总页数") + result: List[dict] = Field(default_factory=list, description="搜索结果列表") + + @property + def has_more(self) -> bool: + """是否还有更多结果""" + return self.page < self.numPages + + def get_videos(self, source_keyword: str = "") -> List[BilibiliVideo]: + """ + 获取视频列表 + + Args: + source_keyword: 搜索关键词 + + Returns: + List[BilibiliVideo]: 视频列表 + """ + videos = [] + for item in self.result: + try: + video = BilibiliVideo.from_search_result(item, source_keyword) + videos.append(video) + except Exception: + continue + return videos + + +if __name__ == '__main__': + # 测试代码 + print("=" * 50) + print("数据模型测试") + print("=" * 50) + + # 测试创建视频模型 + video = BilibiliVideo( + video_id="123456", + bvid="BV1xx411c7mD", + title="测试视频标题", + user_id=12345, + nickname="测试UP主", + play_count=10000, + liked_count=500, + ) + + print(f"视频信息: {video}") + print(f"视频 URL: {video.video_url}") + print(f"爬取时间: {video.crawl_time}") + + # 测试 CSV 行格式 + print(f"\nCSV 行格式:") + for key, value in video.to_csv_row().items(): + print(f" {key}: {value}") + + # 测试从 API 响应创建 + mock_api_response = { + "aid": 789012, + "bvid": "BV1yy411a7bC", + "title": "Python 入门教程", + "desc": "这是一个 Python 入门教程视频", + "pic": "https://example.com/cover.jpg", + "duration": 600, + "pubdate": 1640000000, + "owner": { + "mid": 54321, + "name": "Python老师", + "face": "https://example.com/avatar.jpg" + }, + "stat": { + "view": 50000, + "danmaku": 200, + "reply": 100, + "like": 2000, + "coin": 500, + "favorite": 1000, + "share": 300 + }, + "tname": "知识" + } + + video2 = BilibiliVideo.from_api_response(mock_api_response, "Python教程") + print(f"\n从 API 响应创建的视频:") + print(f" 标题: {video2.title}") + print(f" UP主: {video2.nickname}") + print(f" 播放量: {video2.play_count}") + print(f" 发布时间: {video2.pubdate_str}") diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/proxy/__init__.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/proxy/__init__.py" new file mode 100644 index 0000000..c607dbc --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/proxy/__init__.py" @@ -0,0 +1,4 @@ +# -*- coding: utf-8 -*- +from .pool import ProxyPool, ProxyInfo + +__all__ = ['ProxyPool', 'ProxyInfo'] diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/proxy/pool.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/proxy/pool.py" new file mode 100644 index 0000000..2868386 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/proxy/pool.py" @@ -0,0 +1,213 @@ +# -*- coding: utf-8 -*- +# @Desc: 代理池模块 + +import random +import asyncio +from typing import Optional, List +from dataclasses import dataclass, field +from datetime import datetime +from loguru import logger + +# 可选依赖 +try: + import httpx + HAS_HTTPX = True +except ImportError: + HAS_HTTPX = False + + +@dataclass +class ProxyInfo: + """代理信息""" + ip: str + port: int + protocol: str = "http" + username: str = None + password: str = None + expire_time: datetime = None + fail_count: int = 0 + success_count: int = 0 + + @property + def url(self) -> str: + """获取代理 URL""" + if self.username and self.password: + return f"{self.protocol}://{self.username}:{self.password}@{self.ip}:{self.port}" + return f"{self.protocol}://{self.ip}:{self.port}" + + @property + def is_valid(self) -> bool: + """检查是否有效""" + if self.expire_time and datetime.now() > self.expire_time: + return False + return self.fail_count < 3 + + def __str__(self) -> str: + return f"{self.ip}:{self.port}" + + +class ProxyPool: + """代理池""" + + def __init__(self, api_url: str = None): + """ + 初始化代理池 + + Args: + api_url: 代理 API 地址（可选） + """ + self.api_url = api_url + self._proxies: List[ProxyInfo] = [] + self._lock = asyncio.Lock() + + async def fetch_proxies(self, count: int = 10) -> List[ProxyInfo]: + """ + 从 API 获取代理 + + Args: + count: 获取数量 + + Returns: + 代理列表 + """ + if not self.api_url: + logger.warning("未配置代理 API") + return [] + + if not HAS_HTTPX: + logger.warning("httpx 未安装，无法获取代理") + return [] + + try: + async with httpx.AsyncClient() as client: + response = await client.get( + self.api_url, + params={'count': count}, + timeout=10 + ) + + if response.status_code != 200: + logger.error(f"获取代理失败: HTTP {response.status_code}") + return [] + + data = response.json() + + proxies = [] + # 支持多种 API 响应格式 + items = data.get('data', data.get('proxies', data)) + if isinstance(items, list): + for item in items: + if isinstance(item, dict): + proxy = ProxyInfo( + ip=item.get('ip', item.get('host', '')), + port=int(item.get('port', 0)), + protocol=item.get('protocol', item.get('type', 'http')), + expire_time=datetime.fromisoformat(item['expire_time']) + if 'expire_time' in item else None + ) + if proxy.ip and proxy.port: + proxies.append(proxy) + + logger.info(f"获取 {len(proxies)} 个代理") + return proxies + except Exception as e: + logger.error(f"获取代理失败: {e}") + return [] + + async def add_proxy(self, proxy: ProxyInfo): + """添加单个代理""" + async with self._lock: + self._proxies.append(proxy) + + async def add_proxies(self, proxies: List[ProxyInfo]): + """添加多个代理""" + async with self._lock: + self._proxies.extend(proxies) + logger.info(f"添加 {len(proxies)} 个代理到池中") + + async def add_proxy_from_url(self, url: str): + """从 URL 添加代理""" + try: + # 解析 URL，格式：protocol://[user:pass@]ip:port + from urllib.parse import urlparse + parsed = urlparse(url) + + proxy = ProxyInfo( + ip=parsed.hostname, + port=parsed.port, + protocol=parsed.scheme or 'http', + username=parsed.username, + password=parsed.password + ) + await self.add_proxy(proxy) + except Exception as e: + logger.error(f"解析代理 URL 失败: {e}") + + async def get_proxy(self) -> Optional[ProxyInfo]: + """ + 获取一个可用代理 + + Returns: + 代理信息，如果没有可用代理返回 None + """ + async with self._lock: + # 过滤有效代理 + valid_proxies = [p for p in self._proxies if p.is_valid] + + if not valid_proxies: + # 尝试获取新代理 + if self.api_url: + new_proxies = await self.fetch_proxies() + if new_proxies: + self._proxies = new_proxies + valid_proxies = new_proxies + + if not valid_proxies: + return None + + # 随机选择一个 + return random.choice(valid_proxies) + + async def report_success(self, proxy: ProxyInfo): + """报告代理成功""" + proxy.success_count += 1 + proxy.fail_count = 0 + logger.debug(f"代理 {proxy} 成功，总成功次数: {proxy.success_count}") + + async def report_failure(self, proxy: ProxyInfo): + """报告代理失败""" + proxy.fail_count += 1 + logger.debug(f"代理 {proxy} 失败，失败次数: {proxy.fail_count}") + + async def remove_invalid(self): + """移除无效代理""" + async with self._lock: + before = len(self._proxies) + self._proxies = [p for p in self._proxies if p.is_valid] + removed = before - len(self._proxies) + if removed > 0: + logger.info(f"移除 {removed} 个无效代理") + + async def clear(self): + """清空代理池""" + async with self._lock: + self._proxies.clear() + logger.info("代理池已清空") + + @property + def size(self) -> int: + """代理池大小""" + return len(self._proxies) + + @property + def valid_size(self) -> int: + """有效代理数量""" + return len([p for p in self._proxies if p.is_valid]) + + def get_stats(self) -> dict: + """获取代理池统计信息""" + return { + "total": self.size, + "valid": self.valid_size, + "invalid": self.size - self.valid_size + } diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/pyproject.toml" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/pyproject.toml" new file mode 100644 index 0000000..4e98bf3 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/pyproject.toml" @@ -0,0 +1,50 @@ +[project] +name = "bilibili-crawler" +version = "0.1.0" +description = "第11章：进阶综合实战项目 - B站视频数据采集与分析工具" +readme = "README.md" +requires-python = ">=3.11" +dependencies = [ + # 核心依赖 + "playwright>=1.45.0", + "httpx>=0.27.0", + "pydantic>=2.0.0", + "pydantic-settings>=2.0.0", + "loguru>=0.7.0", + + # 数据分析（可选，但默认安装） + "pandas>=2.2.0", + "jieba>=0.42.0", + "wordcloud>=1.9.0", + "pillow>=10.0.0", +] + +[project.scripts] +bilibili-crawler = "main:cli" + +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[tool.hatch.build.targets.wheel] +# 不打包为 wheel，仅作为脚本项目运行 +packages = [ + "config", + "core", + "login", + "client", + "crawler", + "store", + "proxy", + "models", + "tools", + "analysis", +] + +[tool.uv] +dev-dependencies = [] + +# 提示： +# 1. 安装后需要运行 playwright install chromium 安装浏览器驱动 +# 2. 目标网站：https://www.bilibili.com +# 3. 运行方式：python main.py 或 uv run python main.py diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/qrcode.png" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/qrcode.png" new file mode 100644 index 0000000..bf513be Binary files /dev/null and "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/qrcode.png" differ diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/store/__init__.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/store/__init__.py" new file mode 100644 index 0000000..6a8ec73 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/store/__init__.py" @@ -0,0 +1,4 @@ +# -*- coding: utf-8 -*- +from .backend import StorageManager, JSONStorage, CSVStorage + +__all__ = ['StorageManager', 'JSONStorage', 'CSVStorage'] diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/store/backend.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/store/backend.py" new file mode 100644 index 0000000..38849b8 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/store/backend.py" @@ -0,0 +1,222 @@ +# -*- coding: utf-8 -*- +""" +数据存储后端模块 + +本模块实现了多种数据存储后端，包括： +- JSONStorage: JSON 文件存储 +- CSVStorage: CSV 文件存储 +- StorageManager: 统一存储管理器 + +支持保存 BilibiliVideo 模型数据，自动转换为适合存储的格式。 + +参考 MediaCrawler 项目的实现： +- https://github.com/NanmiCoder/MediaCrawler/blob/main/store/bilibili/_store_impl.py +""" + +import json +import csv +from abc import ABC, abstractmethod +from datetime import datetime +from pathlib import Path +from typing import List, Dict, Any, Optional, Union +from loguru import logger + +# 尝试导入模型 +try: + from models.bilibili import BilibiliVideo + HAS_MODEL = True +except ImportError: + HAS_MODEL = False + + +class BaseStorage(ABC): + """存储基类""" + + @abstractmethod + async def save(self, data: List[Dict]) -> bool: + """保存数据""" + pass + + @abstractmethod + async def load(self) -> List[Dict]: + """加载数据""" + pass + + def _convert_to_dict(self, item: Any) -> Dict: + """ + 将对象转换为字典 + + 支持 BilibiliVideo 模型和普通字典。 + """ + if HAS_MODEL and isinstance(item, BilibiliVideo): + return item.to_dict() + elif hasattr(item, 'model_dump'): + return item.model_dump() + elif hasattr(item, 'dict'): + return item.dict() + elif isinstance(item, dict): + return item + else: + return dict(item) + + +class JSONStorage(BaseStorage): + """JSON 存储""" + + def __init__(self, output_dir: str, filename: str = None): + """ + 初始化 JSON 存储 + + Args: + output_dir: 输出目录 + filename: 文件名（可选，默认按时间戳生成） + """ + self.output_dir = Path(output_dir) + self.output_dir.mkdir(parents=True, exist_ok=True) + + if filename: + self.filepath = self.output_dir / filename + else: + timestamp = datetime.now().strftime('%Y%m%d_%H%M%S') + self.filepath = self.output_dir / f"data_{timestamp}.json" + + async def save(self, data: List[Dict]) -> bool: + """保存数据到 JSON 文件""" + try: + with open(self.filepath, 'w', encoding='utf-8') as f: + json.dump(data, f, ensure_ascii=False, indent=2) + logger.info(f"数据已保存到: {self.filepath} ({len(data)} 条)") + return True + except Exception as e: + logger.error(f"保存失败: {e}") + return False + + async def load(self) -> List[Dict]: + """从 JSON 文件加载数据""" + if not self.filepath.exists(): + return [] + try: + with open(self.filepath, 'r', encoding='utf-8') as f: + return json.load(f) + except Exception as e: + logger.error(f"加载失败: {e}") + return [] + + async def append(self, data: Dict) -> bool: + """追加单条数据""" + existing = await self.load() + existing.append(data) + return await self.save(existing) + + async def append_batch(self, data: List[Dict]) -> bool: + """追加多条数据""" + existing = await self.load() + existing.extend(data) + return await self.save(existing) + + +class CSVStorage(BaseStorage): + """CSV 存储""" + + def __init__( + self, + output_dir: str, + filename: str = None, + fields: List[str] = None + ): + """ + 初始化 CSV 存储 + + Args: + output_dir: 输出目录 + filename: 文件名（可选） + fields: 字段列表（可选，默认从数据推断） + """ + self.output_dir = Path(output_dir) + self.output_dir.mkdir(parents=True, exist_ok=True) + + if filename: + self.filepath = self.output_dir / filename + else: + timestamp = datetime.now().strftime('%Y%m%d_%H%M%S') + self.filepath = self.output_dir / f"data_{timestamp}.csv" + + self.fields = fields + + async def save(self, data: List[Dict]) -> bool: + """保存数据到 CSV 文件""" + if not data: + logger.warning("没有数据需要保存") + return True + + try: + # 确定字段列表 + fields = self.fields or list(data[0].keys()) + + with open(self.filepath, 'w', encoding='utf-8-sig', newline='') as f: + writer = csv.DictWriter(f, fieldnames=fields, extrasaction='ignore') + writer.writeheader() + writer.writerows(data) + + logger.info(f"数据已保存到: {self.filepath} ({len(data)} 条)") + return True + except Exception as e: + logger.error(f"保存失败: {e}") + return False + + async def load(self) -> List[Dict]: + """从 CSV 文件加载数据""" + if not self.filepath.exists(): + return [] + try: + with open(self.filepath, 'r', encoding='utf-8-sig') as f: + reader = csv.DictReader(f) + return list(reader) + except Exception as e: + logger.error(f"加载失败: {e}") + return [] + + +class StorageManager: + """存储管理器""" + + def __init__( + self, + storage_type: str, + output_dir: str, + filename: str = None, + **kwargs + ): + """ + 初始化存储管理器 + + Args: + storage_type: 存储类型 ('json' 或 'csv') + output_dir: 输出目录 + filename: 文件名（可选） + **kwargs: 传递给具体存储类的参数 + """ + self.output_dir = output_dir + + if storage_type == 'json': + self._storage = JSONStorage(output_dir, filename) + elif storage_type == 'csv': + self._storage = CSVStorage(output_dir, filename, **kwargs) + else: + raise ValueError(f"不支持的存储类型: {storage_type}") + + self.storage_type = storage_type + logger.info(f"存储管理器初始化: {storage_type} -> {output_dir}") + + async def save(self, data: List[Dict]) -> bool: + """保存数据""" + return await self._storage.save(data) + + async def load(self) -> List[Dict]: + """加载数据""" + return await self._storage.load() + + @property + def filepath(self) -> Path: + """获取文件路径""" + return self._storage.filepath diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/test_quick.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/test_quick.py" new file mode 100644 index 0000000..3328c3e --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/test_quick.py" @@ -0,0 +1,69 @@ +# -*- coding: utf-8 -*- +# 快速测试脚本 - 只爬取1页数据验证功能 + +import asyncio +import sys +from pathlib import Path +from loguru import logger + +# 添加项目根目录到路径 +sys.path.insert(0, str(Path(__file__).parent)) + +from core.browser import BrowserManager +from crawler.spider import ContentCrawler + +# 配置简单日志 +logger.remove() +logger.add(sys.stderr, level="INFO") + + +async def quick_test(): + """快速测试""" + logger.info("开始快速测试 books.toscrape.com") + + browser = BrowserManager(headless=True, timeout=30000) + + try: + async with browser: + context = await browser.start() + page = await browser.new_page() + + # 创建爬虫 - 只爬取1页数据 + crawler = ContentCrawler( + start_url="http://books.toscrape.com/catalogue/page-1.html", + item_selector="article.product_pod", + fields={ + "title": "h3 a|title", + "price": ".price_color", + "rating": ".star-rating|class", + "availability": ".availability", + "link": "h3 a|href", + }, + next_page_selector=".next a", + max_pages=1, # 只爬取1页 + delay_min=0.5, + delay_max=1.0 + ) + + logger.info("开始爬取...") + results = await crawler.crawl(page) + + logger.info(f"爬取完成: {len(results)} 条数据") + + if results: + logger.info("第一条数据示例:") + logger.info(results[0]) + logger.success("✅ 测试成功！") + return True + else: + logger.error("❌ 没有爬取到数据") + return False + + except Exception as e: + logger.exception(f"测试失败: {e}") + return False + + +if __name__ == "__main__": + result = asyncio.run(quick_test()) + sys.exit(0 if result else 1) diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/tools/__init__.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/tools/__init__.py" new file mode 100644 index 0000000..a2f7497 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/tools/__init__.py" @@ -0,0 +1,7 @@ +# -*- coding: utf-8 -*- +""" +工具模块 +""" +from .sign import BilibiliSign + +__all__ = ["BilibiliSign"] diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/tools/sign.py" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/tools/sign.py" new file mode 100644 index 0000000..1756cb4 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/tools/sign.py" @@ -0,0 +1,273 @@ +# -*- coding: utf-8 -*- +""" +B站 WBI 签名工具 + +本模块实现了 B站 API 请求的 WBI 签名算法，用于防止接口被滥用。 +WBI 签名是 B站 API 的一种保护机制，需要在请求参数中添加 wts（时间戳）和 w_rid（签名）。 + +签名算法说明： +1. 从浏览器的 localStorage 中获取 wbi_img_urls，包含 img_url 和 sub_url +2. 从这两个 URL 中提取 img_key 和 sub_key +3. 将两个 key 拼接后，按照混淆映射表重新排列，取前32位作为 salt +4. 将请求参数按 key 排序，URL 编码后与 salt 拼接 +5. 计算 MD5 哈希值作为 w_rid + +参考资料： +- https://socialsisteryi.github.io/bilibili-API-collect/docs/misc/sign/wbi.html +""" + +import re +import time +import urllib.parse +from hashlib import md5 +from typing import Dict, Optional +from dataclasses import dataclass + + +# WBI 签名混淆映射表（固定值，从 B站 JS 代码中逆向得到） +WBI_MIXIN_KEY_ENC_TAB = [ + 46, 47, 18, 2, 53, 8, 23, 32, 15, 50, 10, 31, 58, 3, 45, 35, + 27, 43, 5, 49, 33, 9, 42, 19, 29, 28, 14, 39, 12, 38, 41, 13, + 37, 48, 7, 16, 24, 55, 40, 61, 26, 17, 0, 1, 60, 51, 30, 4, + 22, 25, 54, 21, 56, 59, 6, 63, 57, 62, 11, 36, 20, 34, 44, 52, +] + + +@dataclass +class VideoUrlInfo: + """视频 URL 信息""" + video_id: str # BV 号 + video_type: str = "video" + + +@dataclass +class CreatorUrlInfo: + """创作者 URL 信息""" + creator_id: str # UID + + +class BilibiliSign: + """ + B站 WBI 签名类 + + 使用方法： + ```python + # 创建签名实例（需要先从浏览器获取 img_key 和 sub_key） + signer = BilibiliSign(img_key="xxx", sub_key="yyy") + + # 对请求参数进行签名 + params = {"keyword": "Python教程", "page": 1} + signed_params = signer.sign(params) + # signed_params 会包含 wts 和 w_rid + ``` + """ + + def __init__(self, img_key: str, sub_key: str): + """ + 初始化签名器 + + Args: + img_key: 从 wbi_img_urls 中提取的 img_key + sub_key: 从 wbi_img_urls 中提取的 sub_key + """ + self.img_key = img_key + self.sub_key = sub_key + + def get_salt(self) -> str: + """ + 生成混淆后的 salt + + 算法： + 1. 将 img_key 和 sub_key 拼接 + 2. 按照映射表重新排列字符 + 3. 取前 32 位作为 salt + + Returns: + str: 32位的 salt 字符串 + """ + salt = "" + mixin_key = self.img_key + self.sub_key + for index in WBI_MIXIN_KEY_ENC_TAB: + salt += mixin_key[index] + return salt[:32] + + def sign(self, req_data: Dict) -> Dict: + """ + 对请求参数进行签名 + + 签名流程： + 1. 添加当前时间戳 wts + 2. 按 key 字典序排序 + 3. 过滤特殊字符 + 4. URL 编码后与 salt 拼接 + 5. 计算 MD5 作为 w_rid + + Args: + req_data: 原始请求参数 + + Returns: + Dict: 签名后的请求参数（包含 wts 和 w_rid） + """ + # 复制参数，避免修改原始数据 + params = req_data.copy() + + # 添加当前时间戳（秒级） + current_ts = int(time.time()) + params["wts"] = current_ts + + # 按 key 字典序排序 + params = dict(sorted(params.items())) + + # 过滤特殊字符 !'()* + params = { + k: ''.join(filter(lambda ch: ch not in "!'()*", str(v))) + for k, v in params.items() + } + + # URL 编码 + query = urllib.parse.urlencode(params) + + # 获取 salt 并计算签名 + salt = self.get_salt() + w_rid = md5((query + salt).encode()).hexdigest() + + # 添加签名到参数 + params['w_rid'] = w_rid + + return params + + +def extract_wbi_keys_from_urls(img_url: str, sub_url: str) -> tuple: + """ + 从 wbi_img_urls 中提取 img_key 和 sub_key + + Args: + img_url: img_url 完整地址 + sub_url: sub_url 完整地址 + + Returns: + tuple: (img_key, sub_key) + + Example: + img_url = "https://i0.hdslb.com/bfs/wbi/7cd084941338484aae1ad9425b84077c.png" + sub_url = "https://i0.hdslb.com/bfs/wbi/4932caff0ff746eab6f01bf08b70ac45.png" + # 返回 ("7cd084941338484aae1ad9425b84077c", "4932caff0ff746eab6f01bf08b70ac45") + """ + # 从 URL 中提取文件名（不含扩展名） + img_key = img_url.rsplit('/', 1)[-1].split('.')[0] + sub_key = sub_url.rsplit('/', 1)[-1].split('.')[0] + return img_key, sub_key + + +def parse_video_info_from_url(url: str) -> VideoUrlInfo: + """ + 从 B站视频 URL 中解析视频 ID + + 支持的格式： + - https://www.bilibili.com/video/BV1dwuKzmE26/?spm_id_from=... + - https://www.bilibili.com/video/BV1d54y1g7db + - BV1d54y1g7db（直接传入 BV 号） + + Args: + url: B站视频链接或 BV 号 + + Returns: + VideoUrlInfo: 包含视频 ID 的对象 + + Raises: + ValueError: 无法解析视频 ID + """ + # 如果直接是 BV 号，直接返回 + if url.startswith("BV"): + return VideoUrlInfo(video_id=url) + + # 使用正则提取 BV 号 + bv_pattern = r'/video/(BV[a-zA-Z0-9]+)' + match = re.search(bv_pattern, url) + + if match: + video_id = match.group(1) + return VideoUrlInfo(video_id=video_id) + + raise ValueError(f"无法从 URL 解析视频 ID: {url}") + + +def parse_creator_info_from_url(url: str) -> CreatorUrlInfo: + """ + 从 B站用户空间 URL 中解析用户 ID + + 支持的格式： + - https://space.bilibili.com/434377496?spm_id_from=... + - https://space.bilibili.com/20813884 + - 434377496（直接传入 UID） + + Args: + url: B站用户空间链接或 UID + + Returns: + CreatorUrlInfo: 包含用户 ID 的对象 + + Raises: + ValueError: 无法解析用户 ID + """ + # 如果直接是数字 ID，直接返回 + if url.isdigit(): + return CreatorUrlInfo(creator_id=url) + + # 使用正则提取 UID + uid_pattern = r'space\.bilibili\.com/(\d+)' + match = re.search(uid_pattern, url) + + if match: + creator_id = match.group(1) + return CreatorUrlInfo(creator_id=creator_id) + + raise ValueError(f"无法从 URL 解析用户 ID: {url}") + + +if __name__ == '__main__': + # 测试代码 + print("=" * 50) + print("WBI 签名测试") + print("=" * 50) + + # 模拟从浏览器获取的 key + test_img_key = "7cd084941338484aae1ad9425b84077c" + test_sub_key = "4932caff0ff746eab6f01bf08b70ac45" + + signer = BilibiliSign(test_img_key, test_sub_key) + print(f"Salt: {signer.get_salt()}") + + # 测试签名 + test_params = {"keyword": "Python教程", "page": 1, "search_type": "video"} + signed_params = signer.sign(test_params) + print(f"原始参数: {test_params}") + print(f"签名参数: {signed_params}") + + print("\n" + "=" * 50) + print("URL 解析测试") + print("=" * 50) + + # 测试视频 URL 解析 + video_urls = [ + "https://www.bilibili.com/video/BV1dwuKzmE26/?spm_id_from=333.1387", + "BV1d54y1g7db", + ] + for url in video_urls: + try: + info = parse_video_info_from_url(url) + print(f"视频 URL: {url} -> {info}") + except ValueError as e: + print(f"解析失败: {e}") + + # 测试创作者 URL 解析 + creator_urls = [ + "https://space.bilibili.com/434377496?spm_id_from=333.1007.0.0", + "20813884", + ] + for url in creator_urls: + try: + info = parse_creator_info_from_url(url) + print(f"创作者 URL: {url} -> {info}") + except ValueError as e: + print(f"解析失败: {e}") diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/uv.lock" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/uv.lock" new file mode 100644 index 0000000..ab2e405 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/11_\350\277\233\351\230\266\347\273\274\345\220\210\345\256\236\346\210\230\351\241\271\347\233\256/uv.lock" @@ -0,0 +1,1011 @@ +version = 1 +revision = 1 +requires-python = ">=3.11" +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version < '3.12'", +] + +[[package]] +name = "annotated-types" +version = "0.7.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/ee/67/531ea369ba64dcff5ec9c3402f9f51bf748cec26dde048a2f973a4eea7f5/annotated_types-0.7.0.tar.gz", hash = "sha256:aff07c09a53a08bc8cfccb9c85b05f1aa9a2a6f23728d790723543408344ce89", size = 16081 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/78/b6/6307fbef88d9b5ee7421e68d78a9f162e0da4900bc5f5793f6d3d0e34fb8/annotated_types-0.7.0-py3-none-any.whl", hash = "sha256:1f02e8b43a8fbbc3f3e0d4f0f4bfc8131bcb4eebe8849b8e5c773f3a1c582a53", size = 13643 }, +] + +[[package]] +name = "anyio" +version = "4.12.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "idna" }, + { name = "typing-extensions", marker = "python_full_version < '3.13'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/96/f0/5eb65b2bb0d09ac6776f2eb54adee6abe8228ea05b20a5ad0e4945de8aac/anyio-4.12.1.tar.gz", hash = "sha256:41cfcc3a4c85d3f05c932da7c26d0201ac36f72abd4435ba90d0464a3ffed703", size = 228685 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/38/0e/27be9fdef66e72d64c0cdc3cc2823101b80585f8119b5c112c2e8f5f7dab/anyio-4.12.1-py3-none-any.whl", hash = "sha256:d405828884fc140aa80a3c667b8beed277f1dfedec42ba031bd6ac3db606ab6c", size = 113592 }, +] + +[[package]] +name = "bilibili-crawler" +version = "0.1.0" +source = { editable = "." } +dependencies = [ + { name = "httpx" }, + { name = "jieba" }, + { name = "loguru" }, + { name = "pandas" }, + { name = "pillow" }, + { name = "playwright" }, + { name = "pydantic" }, + { name = "pydantic-settings" }, + { name = "wordcloud" }, +] + +[package.metadata] +requires-dist = [ + { name = "httpx", specifier = ">=0.27.0" }, + { name = "jieba", specifier = ">=0.42.0" }, + { name = "loguru", specifier = ">=0.7.0" }, + { name = "pandas", specifier = ">=2.2.0" }, + { name = "pillow", specifier = ">=10.0.0" }, + { name = "playwright", specifier = ">=1.45.0" }, + { name = "pydantic", specifier = ">=2.0.0" }, + { name = "pydantic-settings", specifier = ">=2.0.0" }, + { name = "wordcloud", specifier = ">=1.9.0" }, +] + +[package.metadata.requires-dev] +dev = [] + +[[package]] +name = "certifi" +version = "2026.1.4" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/e0/2d/a891ca51311197f6ad14a7ef42e2399f36cf2f9bd44752b3dc4eab60fdc5/certifi-2026.1.4.tar.gz", hash = "sha256:ac726dd470482006e014ad384921ed6438c457018f4b3d204aea4281258b2120", size = 154268 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e6/ad/3cc14f097111b4de0040c83a525973216457bbeeb63739ef1ed275c1c021/certifi-2026.1.4-py3-none-any.whl", hash = "sha256:9943707519e4add1115f44c2bc244f782c0249876bf51b6599fee1ffbedd685c", size = 152900 }, +] + +[[package]] +name = "colorama" +version = "0.4.6" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d8/53/6f443c9a4a8358a93a6792e2acffb9d9d5cb0a5cfd8802644b7b1c9a02e4/colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44", size = 27697 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335 }, +] + +[[package]] +name = "contourpy" +version = "1.3.3" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "numpy" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/58/01/1253e6698a07380cd31a736d248a3f2a50a7c88779a1813da27503cadc2a/contourpy-1.3.3.tar.gz", hash = "sha256:083e12155b210502d0bca491432bb04d56dc3432f95a979b429f2848c3dbe880", size = 13466174 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/91/2e/c4390a31919d8a78b90e8ecf87cd4b4c4f05a5b48d05ec17db8e5404c6f4/contourpy-1.3.3-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:709a48ef9a690e1343202916450bc48b9e51c049b089c7f79a267b46cffcdaa1", size = 288773 }, + { url = "https://files.pythonhosted.org/packages/0d/44/c4b0b6095fef4dc9c420e041799591e3b63e9619e3044f7f4f6c21c0ab24/contourpy-1.3.3-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:23416f38bfd74d5d28ab8429cc4d63fa67d5068bd711a85edb1c3fb0c3e2f381", size = 270149 }, + { url = "https://files.pythonhosted.org/packages/30/2e/dd4ced42fefac8470661d7cb7e264808425e6c5d56d175291e93890cce09/contourpy-1.3.3-cp311-cp311-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:929ddf8c4c7f348e4c0a5a3a714b5c8542ffaa8c22954862a46ca1813b667ee7", size = 329222 }, + { url = "https://files.pythonhosted.org/packages/f2/74/cc6ec2548e3d276c71389ea4802a774b7aa3558223b7bade3f25787fafc2/contourpy-1.3.3-cp311-cp311-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:9e999574eddae35f1312c2b4b717b7885d4edd6cb46700e04f7f02db454e67c1", size = 377234 }, + { url = "https://files.pythonhosted.org/packages/03/b3/64ef723029f917410f75c09da54254c5f9ea90ef89b143ccadb09df14c15/contourpy-1.3.3-cp311-cp311-manylinux_2_26_s390x.manylinux_2_28_s390x.whl", hash = "sha256:0bf67e0e3f482cb69779dd3061b534eb35ac9b17f163d851e2a547d56dba0a3a", size = 380555 }, + { url = "https://files.pythonhosted.org/packages/5f/4b/6157f24ca425b89fe2eb7e7be642375711ab671135be21e6faa100f7448c/contourpy-1.3.3-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:51e79c1f7470158e838808d4a996fa9bac72c498e93d8ebe5119bc1e6becb0db", size = 355238 }, + { url = "https://files.pythonhosted.org/packages/98/56/f914f0dd678480708a04cfd2206e7c382533249bc5001eb9f58aa693e200/contourpy-1.3.3-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:598c3aaece21c503615fd59c92a3598b428b2f01bfb4b8ca9c4edeecc2438620", size = 1326218 }, + { url = "https://files.pythonhosted.org/packages/fb/d7/4a972334a0c971acd5172389671113ae82aa7527073980c38d5868ff1161/contourpy-1.3.3-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:322ab1c99b008dad206d406bb61d014cf0174df491ae9d9d0fac6a6fda4f977f", size = 1392867 }, + { url = "https://files.pythonhosted.org/packages/75/3e/f2cc6cd56dc8cff46b1a56232eabc6feea52720083ea71ab15523daab796/contourpy-1.3.3-cp311-cp311-win32.whl", hash = "sha256:fd907ae12cd483cd83e414b12941c632a969171bf90fc937d0c9f268a31cafff", size = 183677 }, + { url = "https://files.pythonhosted.org/packages/98/4b/9bd370b004b5c9d8045c6c33cf65bae018b27aca550a3f657cdc99acdbd8/contourpy-1.3.3-cp311-cp311-win_amd64.whl", hash = "sha256:3519428f6be58431c56581f1694ba8e50626f2dd550af225f82fb5f5814d2a42", size = 225234 }, + { url = "https://files.pythonhosted.org/packages/d9/b6/71771e02c2e004450c12b1120a5f488cad2e4d5b590b1af8bad060360fe4/contourpy-1.3.3-cp311-cp311-win_arm64.whl", hash = "sha256:15ff10bfada4bf92ec8b31c62bf7c1834c244019b4a33095a68000d7075df470", size = 193123 }, + { url = "https://files.pythonhosted.org/packages/be/45/adfee365d9ea3d853550b2e735f9d66366701c65db7855cd07621732ccfc/contourpy-1.3.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:b08a32ea2f8e42cf1d4be3169a98dd4be32bafe4f22b6c4cb4ba810fa9e5d2cb", size = 293419 }, + { url = "https://files.pythonhosted.org/packages/53/3e/405b59cfa13021a56bba395a6b3aca8cec012b45bf177b0eaf7a202cde2c/contourpy-1.3.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:556dba8fb6f5d8742f2923fe9457dbdd51e1049c4a43fd3986a0b14a1d815fc6", size = 273979 }, + { url = "https://files.pythonhosted.org/packages/d4/1c/a12359b9b2ca3a845e8f7f9ac08bdf776114eb931392fcad91743e2ea17b/contourpy-1.3.3-cp312-cp312-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:92d9abc807cf7d0e047b95ca5d957cf4792fcd04e920ca70d48add15c1a90ea7", size = 332653 }, + { url = "https://files.pythonhosted.org/packages/63/12/897aeebfb475b7748ea67b61e045accdfcf0d971f8a588b67108ed7f5512/contourpy-1.3.3-cp312-cp312-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:b2e8faa0ed68cb29af51edd8e24798bb661eac3bd9f65420c1887b6ca89987c8", size = 379536 }, + { url = "https://files.pythonhosted.org/packages/43/8a/a8c584b82deb248930ce069e71576fc09bd7174bbd35183b7943fb1064fd/contourpy-1.3.3-cp312-cp312-manylinux_2_26_s390x.manylinux_2_28_s390x.whl", hash = "sha256:626d60935cf668e70a5ce6ff184fd713e9683fb458898e4249b63be9e28286ea", size = 384397 }, + { url = "https://files.pythonhosted.org/packages/cc/8f/ec6289987824b29529d0dfda0d74a07cec60e54b9c92f3c9da4c0ac732de/contourpy-1.3.3-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4d00e655fcef08aba35ec9610536bfe90267d7ab5ba944f7032549c55a146da1", size = 362601 }, + { url = "https://files.pythonhosted.org/packages/05/0a/a3fe3be3ee2dceb3e615ebb4df97ae6f3828aa915d3e10549ce016302bd1/contourpy-1.3.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:451e71b5a7d597379ef572de31eeb909a87246974d960049a9848c3bc6c41bf7", size = 1331288 }, + { url = "https://files.pythonhosted.org/packages/33/1d/acad9bd4e97f13f3e2b18a3977fe1b4a37ecf3d38d815333980c6c72e963/contourpy-1.3.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:459c1f020cd59fcfe6650180678a9993932d80d44ccde1fa1868977438f0b411", size = 1403386 }, + { url = "https://files.pythonhosted.org/packages/cf/8f/5847f44a7fddf859704217a99a23a4f6417b10e5ab1256a179264561540e/contourpy-1.3.3-cp312-cp312-win32.whl", hash = "sha256:023b44101dfe49d7d53932be418477dba359649246075c996866106da069af69", size = 185018 }, + { url = "https://files.pythonhosted.org/packages/19/e8/6026ed58a64563186a9ee3f29f41261fd1828f527dd93d33b60feca63352/contourpy-1.3.3-cp312-cp312-win_amd64.whl", hash = "sha256:8153b8bfc11e1e4d75bcb0bff1db232f9e10b274e0929de9d608027e0d34ff8b", size = 226567 }, + { url = "https://files.pythonhosted.org/packages/d1/e2/f05240d2c39a1ed228d8328a78b6f44cd695f7ef47beb3e684cf93604f86/contourpy-1.3.3-cp312-cp312-win_arm64.whl", hash = "sha256:07ce5ed73ecdc4a03ffe3e1b3e3c1166db35ae7584be76f65dbbe28a7791b0cc", size = 193655 }, + { url = "https://files.pythonhosted.org/packages/68/35/0167aad910bbdb9599272bd96d01a9ec6852f36b9455cf2ca67bd4cc2d23/contourpy-1.3.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:177fb367556747a686509d6fef71d221a4b198a3905fe824430e5ea0fda54eb5", size = 293257 }, + { url = "https://files.pythonhosted.org/packages/96/e4/7adcd9c8362745b2210728f209bfbcf7d91ba868a2c5f40d8b58f54c509b/contourpy-1.3.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:d002b6f00d73d69333dac9d0b8d5e84d9724ff9ef044fd63c5986e62b7c9e1b1", size = 274034 }, + { url = "https://files.pythonhosted.org/packages/73/23/90e31ceeed1de63058a02cb04b12f2de4b40e3bef5e082a7c18d9c8ae281/contourpy-1.3.3-cp313-cp313-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:348ac1f5d4f1d66d3322420f01d42e43122f43616e0f194fc1c9f5d830c5b286", size = 334672 }, + { url = "https://files.pythonhosted.org/packages/ed/93/b43d8acbe67392e659e1d984700e79eb67e2acb2bd7f62012b583a7f1b55/contourpy-1.3.3-cp313-cp313-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:655456777ff65c2c548b7c454af9c6f33f16c8884f11083244b5819cc214f1b5", size = 381234 }, + { url = "https://files.pythonhosted.org/packages/46/3b/bec82a3ea06f66711520f75a40c8fc0b113b2a75edb36aa633eb11c4f50f/contourpy-1.3.3-cp313-cp313-manylinux_2_26_s390x.manylinux_2_28_s390x.whl", hash = "sha256:644a6853d15b2512d67881586bd03f462c7ab755db95f16f14d7e238f2852c67", size = 385169 }, + { url = "https://files.pythonhosted.org/packages/4b/32/e0f13a1c5b0f8572d0ec6ae2f6c677b7991fafd95da523159c19eff0696a/contourpy-1.3.3-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4debd64f124ca62069f313a9cb86656ff087786016d76927ae2cf37846b006c9", size = 362859 }, + { url = "https://files.pythonhosted.org/packages/33/71/e2a7945b7de4e58af42d708a219f3b2f4cff7386e6b6ab0a0fa0033c49a9/contourpy-1.3.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:a15459b0f4615b00bbd1e91f1b9e19b7e63aea7483d03d804186f278c0af2659", size = 1332062 }, + { url = "https://files.pythonhosted.org/packages/12/fc/4e87ac754220ccc0e807284f88e943d6d43b43843614f0a8afa469801db0/contourpy-1.3.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:ca0fdcd73925568ca027e0b17ab07aad764be4706d0a925b89227e447d9737b7", size = 1403932 }, + { url = "https://files.pythonhosted.org/packages/a6/2e/adc197a37443f934594112222ac1aa7dc9a98faf9c3842884df9a9d8751d/contourpy-1.3.3-cp313-cp313-win32.whl", hash = "sha256:b20c7c9a3bf701366556e1b1984ed2d0cedf999903c51311417cf5f591d8c78d", size = 185024 }, + { url = "https://files.pythonhosted.org/packages/18/0b/0098c214843213759692cc638fce7de5c289200a830e5035d1791d7a2338/contourpy-1.3.3-cp313-cp313-win_amd64.whl", hash = "sha256:1cadd8b8969f060ba45ed7c1b714fe69185812ab43bd6b86a9123fe8f99c3263", size = 226578 }, + { url = "https://files.pythonhosted.org/packages/8a/9a/2f6024a0c5995243cd63afdeb3651c984f0d2bc727fd98066d40e141ad73/contourpy-1.3.3-cp313-cp313-win_arm64.whl", hash = "sha256:fd914713266421b7536de2bfa8181aa8c699432b6763a0ea64195ebe28bff6a9", size = 193524 }, + { url = "https://files.pythonhosted.org/packages/c0/b3/f8a1a86bd3298513f500e5b1f5fd92b69896449f6cab6a146a5d52715479/contourpy-1.3.3-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:88df9880d507169449d434c293467418b9f6cbe82edd19284aa0409e7fdb933d", size = 306730 }, + { url = "https://files.pythonhosted.org/packages/3f/11/4780db94ae62fc0c2053909b65dc3246bd7cecfc4f8a20d957ad43aa4ad8/contourpy-1.3.3-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:d06bb1f751ba5d417047db62bca3c8fde202b8c11fb50742ab3ab962c81e8216", size = 287897 }, + { url = "https://files.pythonhosted.org/packages/ae/15/e59f5f3ffdd6f3d4daa3e47114c53daabcb18574a26c21f03dc9e4e42ff0/contourpy-1.3.3-cp313-cp313t-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e4e6b05a45525357e382909a4c1600444e2a45b4795163d3b22669285591c1ae", size = 326751 }, + { url = "https://files.pythonhosted.org/packages/0f/81/03b45cfad088e4770b1dcf72ea78d3802d04200009fb364d18a493857210/contourpy-1.3.3-cp313-cp313t-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:ab3074b48c4e2cf1a960e6bbeb7f04566bf36b1861d5c9d4d8ac04b82e38ba20", size = 375486 }, + { url = "https://files.pythonhosted.org/packages/0c/ba/49923366492ffbdd4486e970d421b289a670ae8cf539c1ea9a09822b371a/contourpy-1.3.3-cp313-cp313t-manylinux_2_26_s390x.manylinux_2_28_s390x.whl", hash = "sha256:6c3d53c796f8647d6deb1abe867daeb66dcc8a97e8455efa729516b997b8ed99", size = 388106 }, + { url = "https://files.pythonhosted.org/packages/9f/52/5b00ea89525f8f143651f9f03a0df371d3cbd2fccd21ca9b768c7a6500c2/contourpy-1.3.3-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:50ed930df7289ff2a8d7afeb9603f8289e5704755c7e5c3bbd929c90c817164b", size = 352548 }, + { url = "https://files.pythonhosted.org/packages/32/1d/a209ec1a3a3452d490f6b14dd92e72280c99ae3d1e73da74f8277d4ee08f/contourpy-1.3.3-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:4feffb6537d64b84877da813a5c30f1422ea5739566abf0bd18065ac040e120a", size = 1322297 }, + { url = "https://files.pythonhosted.org/packages/bc/9e/46f0e8ebdd884ca0e8877e46a3f4e633f6c9c8c4f3f6e72be3fe075994aa/contourpy-1.3.3-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:2b7e9480ffe2b0cd2e787e4df64270e3a0440d9db8dc823312e2c940c167df7e", size = 1391023 }, + { url = "https://files.pythonhosted.org/packages/b9/70/f308384a3ae9cd2209e0849f33c913f658d3326900d0ff5d378d6a1422d2/contourpy-1.3.3-cp313-cp313t-win32.whl", hash = "sha256:283edd842a01e3dcd435b1c5116798d661378d83d36d337b8dde1d16a5fc9ba3", size = 196157 }, + { url = "https://files.pythonhosted.org/packages/b2/dd/880f890a6663b84d9e34a6f88cded89d78f0091e0045a284427cb6b18521/contourpy-1.3.3-cp313-cp313t-win_amd64.whl", hash = "sha256:87acf5963fc2b34825e5b6b048f40e3635dd547f590b04d2ab317c2619ef7ae8", size = 240570 }, + { url = "https://files.pythonhosted.org/packages/80/99/2adc7d8ffead633234817ef8e9a87115c8a11927a94478f6bb3d3f4d4f7d/contourpy-1.3.3-cp313-cp313t-win_arm64.whl", hash = "sha256:3c30273eb2a55024ff31ba7d052dde990d7d8e5450f4bbb6e913558b3d6c2301", size = 199713 }, + { url = "https://files.pythonhosted.org/packages/72/8b/4546f3ab60f78c514ffb7d01a0bd743f90de36f0019d1be84d0a708a580a/contourpy-1.3.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:fde6c716d51c04b1c25d0b90364d0be954624a0ee9d60e23e850e8d48353d07a", size = 292189 }, + { url = "https://files.pythonhosted.org/packages/fd/e1/3542a9cb596cadd76fcef413f19c79216e002623158befe6daa03dbfa88c/contourpy-1.3.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:cbedb772ed74ff5be440fa8eee9bd49f64f6e3fc09436d9c7d8f1c287b121d77", size = 273251 }, + { url = "https://files.pythonhosted.org/packages/b1/71/f93e1e9471d189f79d0ce2497007731c1e6bf9ef6d1d61b911430c3db4e5/contourpy-1.3.3-cp314-cp314-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:22e9b1bd7a9b1d652cd77388465dc358dafcd2e217d35552424aa4f996f524f5", size = 335810 }, + { url = "https://files.pythonhosted.org/packages/91/f9/e35f4c1c93f9275d4e38681a80506b5510e9327350c51f8d4a5a724d178c/contourpy-1.3.3-cp314-cp314-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:a22738912262aa3e254e4f3cb079a95a67132fc5a063890e224393596902f5a4", size = 382871 }, + { url = "https://files.pythonhosted.org/packages/b5/71/47b512f936f66a0a900d81c396a7e60d73419868fba959c61efed7a8ab46/contourpy-1.3.3-cp314-cp314-manylinux_2_26_s390x.manylinux_2_28_s390x.whl", hash = "sha256:afe5a512f31ee6bd7d0dda52ec9864c984ca3d66664444f2d72e0dc4eb832e36", size = 386264 }, + { url = "https://files.pythonhosted.org/packages/04/5f/9ff93450ba96b09c7c2b3f81c94de31c89f92292f1380261bd7195bea4ea/contourpy-1.3.3-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f64836de09927cba6f79dcd00fdd7d5329f3fccc633468507079c829ca4db4e3", size = 363819 }, + { url = "https://files.pythonhosted.org/packages/3e/a6/0b185d4cc480ee494945cde102cb0149ae830b5fa17bf855b95f2e70ad13/contourpy-1.3.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:1fd43c3be4c8e5fd6e4f2baeae35ae18176cf2e5cced681cca908addf1cdd53b", size = 1333650 }, + { url = "https://files.pythonhosted.org/packages/43/d7/afdc95580ca56f30fbcd3060250f66cedbde69b4547028863abd8aa3b47e/contourpy-1.3.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:6afc576f7b33cf00996e5c1102dc2a8f7cc89e39c0b55df93a0b78c1bd992b36", size = 1404833 }, + { url = "https://files.pythonhosted.org/packages/e2/e2/366af18a6d386f41132a48f033cbd2102e9b0cf6345d35ff0826cd984566/contourpy-1.3.3-cp314-cp314-win32.whl", hash = "sha256:66c8a43a4f7b8df8b71ee1840e4211a3c8d93b214b213f590e18a1beca458f7d", size = 189692 }, + { url = "https://files.pythonhosted.org/packages/7d/c2/57f54b03d0f22d4044b8afb9ca0e184f8b1afd57b4f735c2fa70883dc601/contourpy-1.3.3-cp314-cp314-win_amd64.whl", hash = "sha256:cf9022ef053f2694e31d630feaacb21ea24224be1c3ad0520b13d844274614fd", size = 232424 }, + { url = "https://files.pythonhosted.org/packages/18/79/a9416650df9b525737ab521aa181ccc42d56016d2123ddcb7b58e926a42c/contourpy-1.3.3-cp314-cp314-win_arm64.whl", hash = "sha256:95b181891b4c71de4bb404c6621e7e2390745f887f2a026b2d99e92c17892339", size = 198300 }, + { url = "https://files.pythonhosted.org/packages/1f/42/38c159a7d0f2b7b9c04c64ab317042bb6952b713ba875c1681529a2932fe/contourpy-1.3.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:33c82d0138c0a062380332c861387650c82e4cf1747aaa6938b9b6516762e772", size = 306769 }, + { url = "https://files.pythonhosted.org/packages/c3/6c/26a8205f24bca10974e77460de68d3d7c63e282e23782f1239f226fcae6f/contourpy-1.3.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:ea37e7b45949df430fe649e5de8351c423430046a2af20b1c1961cae3afcda77", size = 287892 }, + { url = "https://files.pythonhosted.org/packages/66/06/8a475c8ab718ebfd7925661747dbb3c3ee9c82ac834ccb3570be49d129f4/contourpy-1.3.3-cp314-cp314t-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d304906ecc71672e9c89e87c4675dc5c2645e1f4269a5063b99b0bb29f232d13", size = 326748 }, + { url = "https://files.pythonhosted.org/packages/b4/a3/c5ca9f010a44c223f098fccd8b158bb1cb287378a31ac141f04730dc49be/contourpy-1.3.3-cp314-cp314t-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:ca658cd1a680a5c9ea96dc61cdbae1e85c8f25849843aa799dfd3cb370ad4fbe", size = 375554 }, + { url = "https://files.pythonhosted.org/packages/80/5b/68bd33ae63fac658a4145088c1e894405e07584a316738710b636c6d0333/contourpy-1.3.3-cp314-cp314t-manylinux_2_26_s390x.manylinux_2_28_s390x.whl", hash = "sha256:ab2fd90904c503739a75b7c8c5c01160130ba67944a7b77bbf36ef8054576e7f", size = 388118 }, + { url = "https://files.pythonhosted.org/packages/40/52/4c285a6435940ae25d7410a6c36bda5145839bc3f0beb20c707cda18b9d2/contourpy-1.3.3-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b7301b89040075c30e5768810bc96a8e8d78085b47d8be6e4c3f5a0b4ed478a0", size = 352555 }, + { url = "https://files.pythonhosted.org/packages/24/ee/3e81e1dd174f5c7fefe50e85d0892de05ca4e26ef1c9a59c2a57e43b865a/contourpy-1.3.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:2a2a8b627d5cc6b7c41a4beff6c5ad5eb848c88255fda4a8745f7e901b32d8e4", size = 1322295 }, + { url = "https://files.pythonhosted.org/packages/3c/b2/6d913d4d04e14379de429057cd169e5e00f6c2af3bb13e1710bcbdb5da12/contourpy-1.3.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:fd6ec6be509c787f1caf6b247f0b1ca598bef13f4ddeaa126b7658215529ba0f", size = 1391027 }, + { url = "https://files.pythonhosted.org/packages/93/8a/68a4ec5c55a2971213d29a9374913f7e9f18581945a7a31d1a39b5d2dfe5/contourpy-1.3.3-cp314-cp314t-win32.whl", hash = "sha256:e74a9a0f5e3fff48fb5a7f2fd2b9b70a3fe014a67522f79b7cca4c0c7e43c9ae", size = 202428 }, + { url = "https://files.pythonhosted.org/packages/fa/96/fd9f641ffedc4fa3ace923af73b9d07e869496c9cc7a459103e6e978992f/contourpy-1.3.3-cp314-cp314t-win_amd64.whl", hash = "sha256:13b68d6a62db8eafaebb8039218921399baf6e47bf85006fd8529f2a08ef33fc", size = 250331 }, + { url = "https://files.pythonhosted.org/packages/ae/8c/469afb6465b853afff216f9528ffda78a915ff880ed58813ba4faf4ba0b6/contourpy-1.3.3-cp314-cp314t-win_arm64.whl", hash = "sha256:b7448cb5a725bb1e35ce88771b86fba35ef418952474492cf7c764059933ff8b", size = 203831 }, + { url = "https://files.pythonhosted.org/packages/a5/29/8dcfe16f0107943fa92388c23f6e05cff0ba58058c4c95b00280d4c75a14/contourpy-1.3.3-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:cd5dfcaeb10f7b7f9dc8941717c6c2ade08f587be2226222c12b25f0483ed497", size = 278809 }, + { url = "https://files.pythonhosted.org/packages/85/a9/8b37ef4f7dafeb335daee3c8254645ef5725be4d9c6aa70b50ec46ef2f7e/contourpy-1.3.3-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:0c1fc238306b35f246d61a1d416a627348b5cf0648648a031e14bb8705fcdfe8", size = 261593 }, + { url = "https://files.pythonhosted.org/packages/0a/59/ebfb8c677c75605cc27f7122c90313fd2f375ff3c8d19a1694bda74aaa63/contourpy-1.3.3-pp311-pypy311_pp73-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:70f9aad7de812d6541d29d2bbf8feb22ff7e1c299523db288004e3157ff4674e", size = 302202 }, + { url = "https://files.pythonhosted.org/packages/3c/37/21972a15834d90bfbfb009b9d004779bd5a07a0ec0234e5ba8f64d5736f4/contourpy-1.3.3-pp311-pypy311_pp73-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5ed3657edf08512fc3fe81b510e35c2012fbd3081d2e26160f27ca28affec989", size = 329207 }, + { url = "https://files.pythonhosted.org/packages/0c/58/bd257695f39d05594ca4ad60df5bcb7e32247f9951fd09a9b8edb82d1daa/contourpy-1.3.3-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:3d1a3799d62d45c18bafd41c5fa05120b96a28079f2393af559b843d1a966a77", size = 225315 }, +] + +[[package]] +name = "cycler" +version = "0.12.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/a9/95/a3dbbb5028f35eafb79008e7522a75244477d2838f38cbb722248dabc2a8/cycler-0.12.1.tar.gz", hash = "sha256:88bb128f02ba341da8ef447245a9e138fae777f6a23943da4540077d3601eb1c", size = 7615 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e7/05/c19819d5e3d95294a6f5947fb9b9629efb316b96de511b418c53d245aae6/cycler-0.12.1-py3-none-any.whl", hash = "sha256:85cef7cff222d8644161529808465972e51340599459b8ac3ccbac5a854e0d30", size = 8321 }, +] + +[[package]] +name = "fonttools" +version = "4.61.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/ec/ca/cf17b88a8df95691275a3d77dc0a5ad9907f328ae53acbe6795da1b2f5ed/fonttools-4.61.1.tar.gz", hash = "sha256:6675329885c44657f826ef01d9e4fb33b9158e9d93c537d84ad8399539bc6f69", size = 3565756 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/69/12/bf9f4eaa2fad039356cc627587e30ed008c03f1cebd3034376b5ee8d1d44/fonttools-4.61.1-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:c6604b735bb12fef8e0efd5578c9fb5d3d8532d5001ea13a19cddf295673ee09", size = 2852213 }, + { url = "https://files.pythonhosted.org/packages/ac/49/4138d1acb6261499bedde1c07f8c2605d1d8f9d77a151e5507fd3ef084b6/fonttools-4.61.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:5ce02f38a754f207f2f06557523cd39a06438ba3aafc0639c477ac409fc64e37", size = 2401689 }, + { url = "https://files.pythonhosted.org/packages/e5/fe/e6ce0fe20a40e03aef906af60aa87668696f9e4802fa283627d0b5ed777f/fonttools-4.61.1-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:77efb033d8d7ff233385f30c62c7c79271c8885d5c9657d967ede124671bbdfb", size = 5058809 }, + { url = "https://files.pythonhosted.org/packages/79/61/1ca198af22f7dd22c17ab86e9024ed3c06299cfdb08170640e9996d501a0/fonttools-4.61.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:75c1a6dfac6abd407634420c93864a1e274ebc1c7531346d9254c0d8f6ca00f9", size = 5036039 }, + { url = "https://files.pythonhosted.org/packages/99/cc/fa1801e408586b5fce4da9f5455af8d770f4fc57391cd5da7256bb364d38/fonttools-4.61.1-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:0de30bfe7745c0d1ffa2b0b7048fb7123ad0d71107e10ee090fa0b16b9452e87", size = 5034714 }, + { url = "https://files.pythonhosted.org/packages/bf/aa/b7aeafe65adb1b0a925f8f25725e09f078c635bc22754f3fecb7456955b0/fonttools-4.61.1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:58b0ee0ab5b1fc9921eccfe11d1435added19d6494dde14e323f25ad2bc30c56", size = 5158648 }, + { url = "https://files.pythonhosted.org/packages/99/f9/08ea7a38663328881384c6e7777bbefc46fd7d282adfd87a7d2b84ec9d50/fonttools-4.61.1-cp311-cp311-win32.whl", hash = "sha256:f79b168428351d11e10c5aeb61a74e1851ec221081299f4cf56036a95431c43a", size = 2280681 }, + { url = "https://files.pythonhosted.org/packages/07/ad/37dd1ae5fa6e01612a1fbb954f0927681f282925a86e86198ccd7b15d515/fonttools-4.61.1-cp311-cp311-win_amd64.whl", hash = "sha256:fe2efccb324948a11dd09d22136fe2ac8a97d6c1347cf0b58a911dcd529f66b7", size = 2331951 }, + { url = "https://files.pythonhosted.org/packages/6f/16/7decaa24a1bd3a70c607b2e29f0adc6159f36a7e40eaba59846414765fd4/fonttools-4.61.1-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:f3cb4a569029b9f291f88aafc927dd53683757e640081ca8c412781ea144565e", size = 2851593 }, + { url = "https://files.pythonhosted.org/packages/94/98/3c4cb97c64713a8cf499b3245c3bf9a2b8fd16a3e375feff2aed78f96259/fonttools-4.61.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:41a7170d042e8c0024703ed13b71893519a1a6d6e18e933e3ec7507a2c26a4b2", size = 2400231 }, + { url = "https://files.pythonhosted.org/packages/b7/37/82dbef0f6342eb01f54bca073ac1498433d6ce71e50c3c3282b655733b31/fonttools-4.61.1-cp312-cp312-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:10d88e55330e092940584774ee5e8a6971b01fc2f4d3466a1d6c158230880796", size = 4954103 }, + { url = "https://files.pythonhosted.org/packages/6c/44/f3aeac0fa98e7ad527f479e161aca6c3a1e47bb6996b053d45226fe37bf2/fonttools-4.61.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:15acc09befd16a0fb8a8f62bc147e1a82817542d72184acca9ce6e0aeda9fa6d", size = 5004295 }, + { url = "https://files.pythonhosted.org/packages/14/e8/7424ced75473983b964d09f6747fa09f054a6d656f60e9ac9324cf40c743/fonttools-4.61.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:e6bcdf33aec38d16508ce61fd81838f24c83c90a1d1b8c68982857038673d6b8", size = 4944109 }, + { url = "https://files.pythonhosted.org/packages/c8/8b/6391b257fa3d0b553d73e778f953a2f0154292a7a7a085e2374b111e5410/fonttools-4.61.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:5fade934607a523614726119164ff621e8c30e8fa1ffffbbd358662056ba69f0", size = 5093598 }, + { url = "https://files.pythonhosted.org/packages/d9/71/fd2ea96cdc512d92da5678a1c98c267ddd4d8c5130b76d0f7a80f9a9fde8/fonttools-4.61.1-cp312-cp312-win32.whl", hash = "sha256:75da8f28eff26defba42c52986de97b22106cb8f26515b7c22443ebc9c2d3261", size = 2269060 }, + { url = "https://files.pythonhosted.org/packages/80/3b/a3e81b71aed5a688e89dfe0e2694b26b78c7d7f39a5ffd8a7d75f54a12a8/fonttools-4.61.1-cp312-cp312-win_amd64.whl", hash = "sha256:497c31ce314219888c0e2fce5ad9178ca83fe5230b01a5006726cdf3ac9f24d9", size = 2319078 }, + { url = "https://files.pythonhosted.org/packages/4b/cf/00ba28b0990982530addb8dc3e9e6f2fa9cb5c20df2abdda7baa755e8fe1/fonttools-4.61.1-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:8c56c488ab471628ff3bfa80964372fc13504ece601e0d97a78ee74126b2045c", size = 2846454 }, + { url = "https://files.pythonhosted.org/packages/5a/ca/468c9a8446a2103ae645d14fee3f610567b7042aba85031c1c65e3ef7471/fonttools-4.61.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:dc492779501fa723b04d0ab1f5be046797fee17d27700476edc7ee9ae535a61e", size = 2398191 }, + { url = "https://files.pythonhosted.org/packages/a3/4b/d67eedaed19def5967fade3297fed8161b25ba94699efc124b14fb68cdbc/fonttools-4.61.1-cp313-cp313-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:64102ca87e84261419c3747a0d20f396eb024bdbeb04c2bfb37e2891f5fadcb5", size = 4928410 }, + { url = "https://files.pythonhosted.org/packages/b0/8d/6fb3494dfe61a46258cd93d979cf4725ded4eb46c2a4ca35e4490d84daea/fonttools-4.61.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4c1b526c8d3f615a7b1867f38a9410849c8f4aef078535742198e942fba0e9bd", size = 4984460 }, + { url = "https://files.pythonhosted.org/packages/f7/f1/a47f1d30b3dc00d75e7af762652d4cbc3dff5c2697a0dbd5203c81afd9c3/fonttools-4.61.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:41ed4b5ec103bd306bb68f81dc166e77409e5209443e5773cb4ed837bcc9b0d3", size = 4925800 }, + { url = "https://files.pythonhosted.org/packages/a7/01/e6ae64a0981076e8a66906fab01539799546181e32a37a0257b77e4aa88b/fonttools-4.61.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:b501c862d4901792adaec7c25b1ecc749e2662543f68bb194c42ba18d6eec98d", size = 5067859 }, + { url = "https://files.pythonhosted.org/packages/73/aa/28e40b8d6809a9b5075350a86779163f074d2b617c15d22343fce81918db/fonttools-4.61.1-cp313-cp313-win32.whl", hash = "sha256:4d7092bb38c53bbc78e9255a59158b150bcdc115a1e3b3ce0b5f267dc35dd63c", size = 2267821 }, + { url = "https://files.pythonhosted.org/packages/1a/59/453c06d1d83dc0951b69ef692d6b9f1846680342927df54e9a1ca91c6f90/fonttools-4.61.1-cp313-cp313-win_amd64.whl", hash = "sha256:21e7c8d76f62ab13c9472ccf74515ca5b9a761d1bde3265152a6dc58700d895b", size = 2318169 }, + { url = "https://files.pythonhosted.org/packages/32/8f/4e7bf82c0cbb738d3c2206c920ca34ca74ef9dabde779030145d28665104/fonttools-4.61.1-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:fff4f534200a04b4a36e7ae3cb74493afe807b517a09e99cb4faa89a34ed6ecd", size = 2846094 }, + { url = "https://files.pythonhosted.org/packages/71/09/d44e45d0a4f3a651f23a1e9d42de43bc643cce2971b19e784cc67d823676/fonttools-4.61.1-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:d9203500f7c63545b4ce3799319fe4d9feb1a1b89b28d3cb5abd11b9dd64147e", size = 2396589 }, + { url = "https://files.pythonhosted.org/packages/89/18/58c64cafcf8eb677a99ef593121f719e6dcbdb7d1c594ae5a10d4997ca8a/fonttools-4.61.1-cp314-cp314-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:fa646ecec9528bef693415c79a86e733c70a4965dd938e9a226b0fc64c9d2e6c", size = 4877892 }, + { url = "https://files.pythonhosted.org/packages/8a/ec/9e6b38c7ba1e09eb51db849d5450f4c05b7e78481f662c3b79dbde6f3d04/fonttools-4.61.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:11f35ad7805edba3aac1a3710d104592df59f4b957e30108ae0ba6c10b11dd75", size = 4972884 }, + { url = "https://files.pythonhosted.org/packages/5e/87/b5339da8e0256734ba0dbbf5b6cdebb1dd79b01dc8c270989b7bcd465541/fonttools-4.61.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:b931ae8f62db78861b0ff1ac017851764602288575d65b8e8ff1963fed419063", size = 4924405 }, + { url = "https://files.pythonhosted.org/packages/0b/47/e3409f1e1e69c073a3a6fd8cb886eb18c0bae0ee13db2c8d5e7f8495e8b7/fonttools-4.61.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:b148b56f5de675ee16d45e769e69f87623a4944f7443850bf9a9376e628a89d2", size = 5035553 }, + { url = "https://files.pythonhosted.org/packages/bf/b6/1f6600161b1073a984294c6c031e1a56ebf95b6164249eecf30012bb2e38/fonttools-4.61.1-cp314-cp314-win32.whl", hash = "sha256:9b666a475a65f4e839d3d10473fad6d47e0a9db14a2f4a224029c5bfde58ad2c", size = 2271915 }, + { url = "https://files.pythonhosted.org/packages/52/7b/91e7b01e37cc8eb0e1f770d08305b3655e4f002fc160fb82b3390eabacf5/fonttools-4.61.1-cp314-cp314-win_amd64.whl", hash = "sha256:4f5686e1fe5fce75d82d93c47a438a25bf0d1319d2843a926f741140b2b16e0c", size = 2323487 }, + { url = "https://files.pythonhosted.org/packages/39/5c/908ad78e46c61c3e3ed70c3b58ff82ab48437faf84ec84f109592cabbd9f/fonttools-4.61.1-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:e76ce097e3c57c4bcb67c5aa24a0ecdbd9f74ea9219997a707a4061fbe2707aa", size = 2929571 }, + { url = "https://files.pythonhosted.org/packages/bd/41/975804132c6dea64cdbfbaa59f3518a21c137a10cccf962805b301ac6ab2/fonttools-4.61.1-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:9cfef3ab326780c04d6646f68d4b4742aae222e8b8ea1d627c74e38afcbc9d91", size = 2435317 }, + { url = "https://files.pythonhosted.org/packages/b0/5a/aef2a0a8daf1ebaae4cfd83f84186d4a72ee08fd6a8451289fcd03ffa8a4/fonttools-4.61.1-cp314-cp314t-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:a75c301f96db737e1c5ed5fd7d77d9c34466de16095a266509e13da09751bd19", size = 4882124 }, + { url = "https://files.pythonhosted.org/packages/80/33/d6db3485b645b81cea538c9d1c9219d5805f0877fda18777add4671c5240/fonttools-4.61.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:91669ccac46bbc1d09e9273546181919064e8df73488ea087dcac3e2968df9ba", size = 5100391 }, + { url = "https://files.pythonhosted.org/packages/6c/d6/675ba631454043c75fcf76f0ca5463eac8eb0666ea1d7badae5fea001155/fonttools-4.61.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:c33ab3ca9d3ccd581d58e989d67554e42d8d4ded94ab3ade3508455fe70e65f7", size = 4978800 }, + { url = "https://files.pythonhosted.org/packages/7f/33/d3ec753d547a8d2bdaedd390d4a814e8d5b45a093d558f025c6b990b554c/fonttools-4.61.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:664c5a68ec406f6b1547946683008576ef8b38275608e1cee6c061828171c118", size = 5006426 }, + { url = "https://files.pythonhosted.org/packages/b4/40/cc11f378b561a67bea850ab50063366a0d1dd3f6d0a30ce0f874b0ad5664/fonttools-4.61.1-cp314-cp314t-win32.whl", hash = "sha256:aed04cabe26f30c1647ef0e8fbb207516fd40fe9472e9439695f5c6998e60ac5", size = 2335377 }, + { url = "https://files.pythonhosted.org/packages/e4/ff/c9a2b66b39f8628531ea58b320d66d951267c98c6a38684daa8f50fb02f8/fonttools-4.61.1-cp314-cp314t-win_amd64.whl", hash = "sha256:2180f14c141d2f0f3da43f3a81bc8aa4684860f6b0e6f9e165a4831f24e6a23b", size = 2400613 }, + { url = "https://files.pythonhosted.org/packages/c7/4e/ce75a57ff3aebf6fc1f4e9d508b8e5810618a33d900ad6c19eb30b290b97/fonttools-4.61.1-py3-none-any.whl", hash = "sha256:17d2bf5d541add43822bcf0c43d7d847b160c9bb01d15d5007d84e2217aaa371", size = 1148996 }, +] + +[[package]] +name = "greenlet" +version = "3.3.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/c7/e5/40dbda2736893e3e53d25838e0f19a2b417dfc122b9989c91918db30b5d3/greenlet-3.3.0.tar.gz", hash = "sha256:a82bb225a4e9e4d653dd2fb7b8b2d36e4fb25bc0165422a11e48b88e9e6f78fb", size = 190651 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/1f/cb/48e964c452ca2b92175a9b2dca037a553036cb053ba69e284650ce755f13/greenlet-3.3.0-cp311-cp311-macosx_11_0_universal2.whl", hash = "sha256:e29f3018580e8412d6aaf5641bb7745d38c85228dacf51a73bd4e26ddf2a6a8e", size = 274908 }, + { url = "https://files.pythonhosted.org/packages/28/da/38d7bff4d0277b594ec557f479d65272a893f1f2a716cad91efeb8680953/greenlet-3.3.0-cp311-cp311-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a687205fb22794e838f947e2194c0566d3812966b41c78709554aa883183fb62", size = 577113 }, + { url = "https://files.pythonhosted.org/packages/3c/f2/89c5eb0faddc3ff014f1c04467d67dee0d1d334ab81fadbf3744847f8a8a/greenlet-3.3.0-cp311-cp311-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:4243050a88ba61842186cb9e63c7dfa677ec146160b0efd73b855a3d9c7fcf32", size = 590338 }, + { url = "https://files.pythonhosted.org/packages/80/d7/db0a5085035d05134f8c089643da2b44cc9b80647c39e93129c5ef170d8f/greenlet-3.3.0-cp311-cp311-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:670d0f94cd302d81796e37299bcd04b95d62403883b24225c6b5271466612f45", size = 601098 }, + { url = "https://files.pythonhosted.org/packages/dc/a6/e959a127b630a58e23529972dbc868c107f9d583b5a9f878fb858c46bc1a/greenlet-3.3.0-cp311-cp311-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6cb3a8ec3db4a3b0eb8a3c25436c2d49e3505821802074969db017b87bc6a948", size = 590206 }, + { url = "https://files.pythonhosted.org/packages/48/60/29035719feb91798693023608447283b266b12efc576ed013dd9442364bb/greenlet-3.3.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:2de5a0b09eab81fc6a382791b995b1ccf2b172a9fec934747a7a23d2ff291794", size = 1550668 }, + { url = "https://files.pythonhosted.org/packages/0a/5f/783a23754b691bfa86bd72c3033aa107490deac9b2ef190837b860996c9f/greenlet-3.3.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:4449a736606bd30f27f8e1ff4678ee193bc47f6ca810d705981cfffd6ce0d8c5", size = 1615483 }, + { url = "https://files.pythonhosted.org/packages/1d/d5/c339b3b4bc8198b7caa4f2bd9fd685ac9f29795816d8db112da3d04175bb/greenlet-3.3.0-cp311-cp311-win_amd64.whl", hash = "sha256:7652ee180d16d447a683c04e4c5f6441bae7ba7b17ffd9f6b3aff4605e9e6f71", size = 301164 }, + { url = "https://files.pythonhosted.org/packages/f8/0a/a3871375c7b9727edaeeea994bfff7c63ff7804c9829c19309ba2e058807/greenlet-3.3.0-cp312-cp312-macosx_11_0_universal2.whl", hash = "sha256:b01548f6e0b9e9784a2c99c5651e5dc89ffcbe870bc5fb2e5ef864e9cc6b5dcb", size = 276379 }, + { url = "https://files.pythonhosted.org/packages/43/ab/7ebfe34dce8b87be0d11dae91acbf76f7b8246bf9d6b319c741f99fa59c6/greenlet-3.3.0-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:349345b770dc88f81506c6861d22a6ccd422207829d2c854ae2af8025af303e3", size = 597294 }, + { url = "https://files.pythonhosted.org/packages/a4/39/f1c8da50024feecd0793dbd5e08f526809b8ab5609224a2da40aad3a7641/greenlet-3.3.0-cp312-cp312-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:e8e18ed6995e9e2c0b4ed264d2cf89260ab3ac7e13555b8032b25a74c6d18655", size = 607742 }, + { url = "https://files.pythonhosted.org/packages/77/cb/43692bcd5f7a0da6ec0ec6d58ee7cddb606d055ce94a62ac9b1aa481e969/greenlet-3.3.0-cp312-cp312-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:c024b1e5696626890038e34f76140ed1daf858e37496d33f2af57f06189e70d7", size = 622297 }, + { url = "https://files.pythonhosted.org/packages/75/b0/6bde0b1011a60782108c01de5913c588cf51a839174538d266de15e4bf4d/greenlet-3.3.0-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:047ab3df20ede6a57c35c14bf5200fcf04039d50f908270d3f9a7a82064f543b", size = 609885 }, + { url = "https://files.pythonhosted.org/packages/49/0e/49b46ac39f931f59f987b7cd9f34bfec8ef81d2a1e6e00682f55be5de9f4/greenlet-3.3.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:2d9ad37fc657b1102ec880e637cccf20191581f75c64087a549e66c57e1ceb53", size = 1567424 }, + { url = "https://files.pythonhosted.org/packages/05/f5/49a9ac2dff7f10091935def9165c90236d8f175afb27cbed38fb1d61ab6b/greenlet-3.3.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:83cd0e36932e0e7f36a64b732a6f60c2fc2df28c351bae79fbaf4f8092fe7614", size = 1636017 }, + { url = "https://files.pythonhosted.org/packages/6c/79/3912a94cf27ec503e51ba493692d6db1e3cd8ac7ac52b0b47c8e33d7f4f9/greenlet-3.3.0-cp312-cp312-win_amd64.whl", hash = "sha256:a7a34b13d43a6b78abf828a6d0e87d3385680eaf830cd60d20d52f249faabf39", size = 301964 }, + { url = "https://files.pythonhosted.org/packages/02/2f/28592176381b9ab2cafa12829ba7b472d177f3acc35d8fbcf3673d966fff/greenlet-3.3.0-cp313-cp313-macosx_11_0_universal2.whl", hash = "sha256:a1e41a81c7e2825822f4e068c48cb2196002362619e2d70b148f20a831c00739", size = 275140 }, + { url = "https://files.pythonhosted.org/packages/2c/80/fbe937bf81e9fca98c981fe499e59a3f45df2a04da0baa5c2be0dca0d329/greenlet-3.3.0-cp313-cp313-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9f515a47d02da4d30caaa85b69474cec77b7929b2e936ff7fb853d42f4bf8808", size = 599219 }, + { url = "https://files.pythonhosted.org/packages/c2/ff/7c985128f0514271b8268476af89aee6866df5eec04ac17dcfbc676213df/greenlet-3.3.0-cp313-cp313-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:7d2d9fd66bfadf230b385fdc90426fcd6eb64db54b40c495b72ac0feb5766c54", size = 610211 }, + { url = "https://files.pythonhosted.org/packages/79/07/c47a82d881319ec18a4510bb30463ed6891f2ad2c1901ed5ec23d3de351f/greenlet-3.3.0-cp313-cp313-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:30a6e28487a790417d036088b3bcb3f3ac7d8babaa7d0139edbaddebf3af9492", size = 624311 }, + { url = "https://files.pythonhosted.org/packages/fd/8e/424b8c6e78bd9837d14ff7df01a9829fc883ba2ab4ea787d4f848435f23f/greenlet-3.3.0-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:087ea5e004437321508a8d6f20efc4cfec5e3c30118e1417ea96ed1d93950527", size = 612833 }, + { url = "https://files.pythonhosted.org/packages/b5/ba/56699ff9b7c76ca12f1cdc27a886d0f81f2189c3455ff9f65246780f713d/greenlet-3.3.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:ab97cf74045343f6c60a39913fa59710e4bd26a536ce7ab2397adf8b27e67c39", size = 1567256 }, + { url = "https://files.pythonhosted.org/packages/1e/37/f31136132967982d698c71a281a8901daf1a8fbab935dce7c0cf15f942cc/greenlet-3.3.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:5375d2e23184629112ca1ea89a53389dddbffcf417dad40125713d88eb5f96e8", size = 1636483 }, + { url = "https://files.pythonhosted.org/packages/7e/71/ba21c3fb8c5dce83b8c01f458a42e99ffdb1963aeec08fff5a18588d8fd7/greenlet-3.3.0-cp313-cp313-win_amd64.whl", hash = "sha256:9ee1942ea19550094033c35d25d20726e4f1c40d59545815e1128ac58d416d38", size = 301833 }, + { url = "https://files.pythonhosted.org/packages/d7/7c/f0a6d0ede2c7bf092d00bc83ad5bafb7e6ec9b4aab2fbdfa6f134dc73327/greenlet-3.3.0-cp314-cp314-macosx_11_0_universal2.whl", hash = "sha256:60c2ef0f578afb3c8d92ea07ad327f9a062547137afe91f38408f08aacab667f", size = 275671 }, + { url = "https://files.pythonhosted.org/packages/44/06/dac639ae1a50f5969d82d2e3dd9767d30d6dbdbab0e1a54010c8fe90263c/greenlet-3.3.0-cp314-cp314-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0a5d554d0712ba1de0a6c94c640f7aeba3f85b3a6e1f2899c11c2c0428da9365", size = 646360 }, + { url = "https://files.pythonhosted.org/packages/e0/94/0fb76fe6c5369fba9bf98529ada6f4c3a1adf19e406a47332245ef0eb357/greenlet-3.3.0-cp314-cp314-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:3a898b1e9c5f7307ebbde4102908e6cbfcb9ea16284a3abe15cab996bee8b9b3", size = 658160 }, + { url = "https://files.pythonhosted.org/packages/93/79/d2c70cae6e823fac36c3bbc9077962105052b7ef81db2f01ec3b9bf17e2b/greenlet-3.3.0-cp314-cp314-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:dcd2bdbd444ff340e8d6bdf54d2f206ccddbb3ccfdcd3c25bf4afaa7b8f0cf45", size = 671388 }, + { url = "https://files.pythonhosted.org/packages/b8/14/bab308fc2c1b5228c3224ec2bf928ce2e4d21d8046c161e44a2012b5203e/greenlet-3.3.0-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5773edda4dc00e173820722711d043799d3adb4f01731f40619e07ea2750b955", size = 660166 }, + { url = "https://files.pythonhosted.org/packages/4b/d2/91465d39164eaa0085177f61983d80ffe746c5a1860f009811d498e7259c/greenlet-3.3.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:ac0549373982b36d5fd5d30beb8a7a33ee541ff98d2b502714a09f1169f31b55", size = 1615193 }, + { url = "https://files.pythonhosted.org/packages/42/1b/83d110a37044b92423084d52d5d5a3b3a73cafb51b547e6d7366ff62eff1/greenlet-3.3.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:d198d2d977460358c3b3a4dc844f875d1adb33817f0613f663a656f463764ccc", size = 1683653 }, + { url = "https://files.pythonhosted.org/packages/7c/9a/9030e6f9aa8fd7808e9c31ba4c38f87c4f8ec324ee67431d181fe396d705/greenlet-3.3.0-cp314-cp314-win_amd64.whl", hash = "sha256:73f51dd0e0bdb596fb0417e475fa3c5e32d4c83638296e560086b8d7da7c4170", size = 305387 }, + { url = "https://files.pythonhosted.org/packages/a0/66/bd6317bc5932accf351fc19f177ffba53712a202f9df10587da8df257c7e/greenlet-3.3.0-cp314-cp314t-macosx_11_0_universal2.whl", hash = "sha256:d6ed6f85fae6cdfdb9ce04c9bf7a08d666cfcfb914e7d006f44f840b46741931", size = 282638 }, + { url = "https://files.pythonhosted.org/packages/30/cf/cc81cb030b40e738d6e69502ccbd0dd1bced0588e958f9e757945de24404/greenlet-3.3.0-cp314-cp314t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d9125050fcf24554e69c4cacb086b87b3b55dc395a8b3ebe6487b045b2614388", size = 651145 }, + { url = "https://files.pythonhosted.org/packages/9c/ea/1020037b5ecfe95ca7df8d8549959baceb8186031da83d5ecceff8b08cd2/greenlet-3.3.0-cp314-cp314t-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:87e63ccfa13c0a0f6234ed0add552af24cc67dd886731f2261e46e241608bee3", size = 654236 }, + { url = "https://files.pythonhosted.org/packages/69/cc/1e4bae2e45ca2fa55299f4e85854606a78ecc37fead20d69322f96000504/greenlet-3.3.0-cp314-cp314t-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:2662433acbca297c9153a4023fe2161c8dcfdcc91f10433171cf7e7d94ba2221", size = 662506 }, + { url = "https://files.pythonhosted.org/packages/57/b9/f8025d71a6085c441a7eaff0fd928bbb275a6633773667023d19179fe815/greenlet-3.3.0-cp314-cp314t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:3c6e9b9c1527a78520357de498b0e709fb9e2f49c3a513afd5a249007261911b", size = 653783 }, + { url = "https://files.pythonhosted.org/packages/f6/c7/876a8c7a7485d5d6b5c6821201d542ef28be645aa024cfe1145b35c120c1/greenlet-3.3.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:286d093f95ec98fdd92fcb955003b8a3d054b4e2cab3e2707a5039e7b50520fd", size = 1614857 }, + { url = "https://files.pythonhosted.org/packages/4f/dc/041be1dff9f23dac5f48a43323cd0789cb798342011c19a248d9c9335536/greenlet-3.3.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:6c10513330af5b8ae16f023e8ddbfb486ab355d04467c4679c5cfe4659975dd9", size = 1676034 }, +] + +[[package]] +name = "h11" +version = "0.16.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/01/ee/02a2c011bdab74c6fb3c75474d40b3052059d95df7e73351460c8588d963/h11-0.16.0.tar.gz", hash = "sha256:4e35b956cf45792e4caa5885e69fba00bdbc6ffafbfa020300e549b208ee5ff1", size = 101250 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/04/4b/29cac41a4d98d144bf5f6d33995617b185d14b22401f75ca86f384e87ff1/h11-0.16.0-py3-none-any.whl", hash = "sha256:63cf8bbe7522de3bf65932fda1d9c2772064ffb3dae62d55932da54b31cb6c86", size = 37515 }, +] + +[[package]] +name = "httpcore" +version = "1.0.9" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "certifi" }, + { name = "h11" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/06/94/82699a10bca87a5556c9c59b5963f2d039dbd239f25bc2a63907a05a14cb/httpcore-1.0.9.tar.gz", hash = "sha256:6e34463af53fd2ab5d807f399a9b45ea31c3dfa2276f15a2c3f00afff6e176e8", size = 85484 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/7e/f5/f66802a942d491edb555dd61e3a9961140fd64c90bce1eafd741609d334d/httpcore-1.0.9-py3-none-any.whl", hash = "sha256:2d400746a40668fc9dec9810239072b40b4484b640a8c38fd654a024c7a1bf55", size = 78784 }, +] + +[[package]] +name = "httpx" +version = "0.28.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "anyio" }, + { name = "certifi" }, + { name = "httpcore" }, + { name = "idna" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/b1/df/48c586a5fe32a0f01324ee087459e112ebb7224f646c0b5023f5e79e9956/httpx-0.28.1.tar.gz", hash = "sha256:75e98c5f16b0f35b567856f597f06ff2270a374470a5c2392242528e3e3e42fc", size = 141406 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/2a/39/e50c7c3a983047577ee07d2a9e53faf5a69493943ec3f6a384bdc792deb2/httpx-0.28.1-py3-none-any.whl", hash = "sha256:d909fcccc110f8c7faf814ca82a9a4d816bc5a6dbfea25d6591d6985b8ba59ad", size = 73517 }, +] + +[[package]] +name = "idna" +version = "3.11" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/6f/6d/0703ccc57f3a7233505399edb88de3cbd678da106337b9fcde432b65ed60/idna-3.11.tar.gz", hash = "sha256:795dafcc9c04ed0c1fb032c2aa73654d8e8c5023a7df64a53f39190ada629902", size = 194582 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/0e/61/66938bbb5fc52dbdf84594873d5b51fb1f7c7794e9c0f5bd885f30bc507b/idna-3.11-py3-none-any.whl", hash = "sha256:771a87f49d9defaf64091e6e6fe9c18d4833f140bd19464795bc32d966ca37ea", size = 71008 }, +] + +[[package]] +name = "jieba" +version = "0.42.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/c6/cb/18eeb235f833b726522d7ebed54f2278ce28ba9438e3135ab0278d9792a2/jieba-0.42.1.tar.gz", hash = "sha256:055ca12f62674fafed09427f176506079bc135638a14e23e25be909131928db2", size = 19214172 } + +[[package]] +name = "kiwisolver" +version = "1.4.9" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/5c/3c/85844f1b0feb11ee581ac23fe5fce65cd049a200c1446708cc1b7f922875/kiwisolver-1.4.9.tar.gz", hash = "sha256:c3b22c26c6fd6811b0ae8363b95ca8ce4ea3c202d3d0975b2914310ceb1bcc4d", size = 97564 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/6f/ab/c80b0d5a9d8a1a65f4f815f2afff9798b12c3b9f31f1d304dd233dd920e2/kiwisolver-1.4.9-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:eb14a5da6dc7642b0f3a18f13654847cd8b7a2550e2645a5bda677862b03ba16", size = 124167 }, + { url = "https://files.pythonhosted.org/packages/a0/c0/27fe1a68a39cf62472a300e2879ffc13c0538546c359b86f149cc19f6ac3/kiwisolver-1.4.9-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:39a219e1c81ae3b103643d2aedb90f1ef22650deb266ff12a19e7773f3e5f089", size = 66579 }, + { url = "https://files.pythonhosted.org/packages/31/a2/a12a503ac1fd4943c50f9822678e8015a790a13b5490354c68afb8489814/kiwisolver-1.4.9-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:2405a7d98604b87f3fc28b1716783534b1b4b8510d8142adca34ee0bc3c87543", size = 65309 }, + { url = "https://files.pythonhosted.org/packages/66/e1/e533435c0be77c3f64040d68d7a657771194a63c279f55573188161e81ca/kiwisolver-1.4.9-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:dc1ae486f9abcef254b5618dfb4113dd49f94c68e3e027d03cf0143f3f772b61", size = 1435596 }, + { url = "https://files.pythonhosted.org/packages/67/1e/51b73c7347f9aabdc7215aa79e8b15299097dc2f8e67dee2b095faca9cb0/kiwisolver-1.4.9-cp311-cp311-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8a1f570ce4d62d718dce3f179ee78dac3b545ac16c0c04bb363b7607a949c0d1", size = 1246548 }, + { url = "https://files.pythonhosted.org/packages/21/aa/72a1c5d1e430294f2d32adb9542719cfb441b5da368d09d268c7757af46c/kiwisolver-1.4.9-cp311-cp311-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:cb27e7b78d716c591e88e0a09a2139c6577865d7f2e152488c2cc6257f460872", size = 1263618 }, + { url = "https://files.pythonhosted.org/packages/a3/af/db1509a9e79dbf4c260ce0cfa3903ea8945f6240e9e59d1e4deb731b1a40/kiwisolver-1.4.9-cp311-cp311-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:15163165efc2f627eb9687ea5f3a28137217d217ac4024893d753f46bce9de26", size = 1317437 }, + { url = "https://files.pythonhosted.org/packages/e0/f2/3ea5ee5d52abacdd12013a94130436e19969fa183faa1e7c7fbc89e9a42f/kiwisolver-1.4.9-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:bdee92c56a71d2b24c33a7d4c2856bd6419d017e08caa7802d2963870e315028", size = 2195742 }, + { url = "https://files.pythonhosted.org/packages/6f/9b/1efdd3013c2d9a2566aa6a337e9923a00590c516add9a1e89a768a3eb2fc/kiwisolver-1.4.9-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:412f287c55a6f54b0650bd9b6dce5aceddb95864a1a90c87af16979d37c89771", size = 2290810 }, + { url = "https://files.pythonhosted.org/packages/fb/e5/cfdc36109ae4e67361f9bc5b41323648cb24a01b9ade18784657e022e65f/kiwisolver-1.4.9-cp311-cp311-musllinux_1_2_s390x.whl", hash = "sha256:2c93f00dcba2eea70af2be5f11a830a742fe6b579a1d4e00f47760ef13be247a", size = 2461579 }, + { url = "https://files.pythonhosted.org/packages/62/86/b589e5e86c7610842213994cdea5add00960076bef4ae290c5fa68589cac/kiwisolver-1.4.9-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:f117e1a089d9411663a3207ba874f31be9ac8eaa5b533787024dc07aeb74f464", size = 2268071 }, + { url = "https://files.pythonhosted.org/packages/3b/c6/f8df8509fd1eee6c622febe54384a96cfaf4d43bf2ccec7a0cc17e4715c9/kiwisolver-1.4.9-cp311-cp311-win_amd64.whl", hash = "sha256:be6a04e6c79819c9a8c2373317d19a96048e5a3f90bec587787e86a1153883c2", size = 73840 }, + { url = "https://files.pythonhosted.org/packages/e2/2d/16e0581daafd147bc11ac53f032a2b45eabac897f42a338d0a13c1e5c436/kiwisolver-1.4.9-cp311-cp311-win_arm64.whl", hash = "sha256:0ae37737256ba2de764ddc12aed4956460277f00c4996d51a197e72f62f5eec7", size = 65159 }, + { url = "https://files.pythonhosted.org/packages/86/c9/13573a747838aeb1c76e3267620daa054f4152444d1f3d1a2324b78255b5/kiwisolver-1.4.9-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:ac5a486ac389dddcc5bef4f365b6ae3ffff2c433324fb38dd35e3fab7c957999", size = 123686 }, + { url = "https://files.pythonhosted.org/packages/51/ea/2ecf727927f103ffd1739271ca19c424d0e65ea473fbaeea1c014aea93f6/kiwisolver-1.4.9-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:f2ba92255faa7309d06fe44c3a4a97efe1c8d640c2a79a5ef728b685762a6fd2", size = 66460 }, + { url = "https://files.pythonhosted.org/packages/5b/5a/51f5464373ce2aeb5194508298a508b6f21d3867f499556263c64c621914/kiwisolver-1.4.9-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:4a2899935e724dd1074cb568ce7ac0dce28b2cd6ab539c8e001a8578eb106d14", size = 64952 }, + { url = "https://files.pythonhosted.org/packages/70/90/6d240beb0f24b74371762873e9b7f499f1e02166a2d9c5801f4dbf8fa12e/kiwisolver-1.4.9-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:f6008a4919fdbc0b0097089f67a1eb55d950ed7e90ce2cc3e640abadd2757a04", size = 1474756 }, + { url = "https://files.pythonhosted.org/packages/12/42/f36816eaf465220f683fb711efdd1bbf7a7005a2473d0e4ed421389bd26c/kiwisolver-1.4.9-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:67bb8b474b4181770f926f7b7d2f8c0248cbcb78b660fdd41a47054b28d2a752", size = 1276404 }, + { url = "https://files.pythonhosted.org/packages/2e/64/bc2de94800adc830c476dce44e9b40fd0809cddeef1fde9fcf0f73da301f/kiwisolver-1.4.9-cp312-cp312-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:2327a4a30d3ee07d2fbe2e7933e8a37c591663b96ce42a00bc67461a87d7df77", size = 1294410 }, + { url = "https://files.pythonhosted.org/packages/5f/42/2dc82330a70aa8e55b6d395b11018045e58d0bb00834502bf11509f79091/kiwisolver-1.4.9-cp312-cp312-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:7a08b491ec91b1d5053ac177afe5290adacf1f0f6307d771ccac5de30592d198", size = 1343631 }, + { url = "https://files.pythonhosted.org/packages/22/fd/f4c67a6ed1aab149ec5a8a401c323cee7a1cbe364381bb6c9c0d564e0e20/kiwisolver-1.4.9-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:d8fc5c867c22b828001b6a38d2eaeb88160bf5783c6cb4a5e440efc981ce286d", size = 2224963 }, + { url = "https://files.pythonhosted.org/packages/45/aa/76720bd4cb3713314677d9ec94dcc21ced3f1baf4830adde5bb9b2430a5f/kiwisolver-1.4.9-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:3b3115b2581ea35bb6d1f24a4c90af37e5d9b49dcff267eeed14c3893c5b86ab", size = 2321295 }, + { url = "https://files.pythonhosted.org/packages/80/19/d3ec0d9ab711242f56ae0dc2fc5d70e298bb4a1f9dfab44c027668c673a1/kiwisolver-1.4.9-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:858e4c22fb075920b96a291928cb7dea5644e94c0ee4fcd5af7e865655e4ccf2", size = 2487987 }, + { url = "https://files.pythonhosted.org/packages/39/e9/61e4813b2c97e86b6fdbd4dd824bf72d28bcd8d4849b8084a357bc0dd64d/kiwisolver-1.4.9-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:ed0fecd28cc62c54b262e3736f8bb2512d8dcfdc2bcf08be5f47f96bf405b145", size = 2291817 }, + { url = "https://files.pythonhosted.org/packages/a0/41/85d82b0291db7504da3c2defe35c9a8a5c9803a730f297bd823d11d5fb77/kiwisolver-1.4.9-cp312-cp312-win_amd64.whl", hash = "sha256:f68208a520c3d86ea51acf688a3e3002615a7f0238002cccc17affecc86a8a54", size = 73895 }, + { url = "https://files.pythonhosted.org/packages/e2/92/5f3068cf15ee5cb624a0c7596e67e2a0bb2adee33f71c379054a491d07da/kiwisolver-1.4.9-cp312-cp312-win_arm64.whl", hash = "sha256:2c1a4f57df73965f3f14df20b80ee29e6a7930a57d2d9e8491a25f676e197c60", size = 64992 }, + { url = "https://files.pythonhosted.org/packages/31/c1/c2686cda909742ab66c7388e9a1a8521a59eb89f8bcfbee28fc980d07e24/kiwisolver-1.4.9-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:a5d0432ccf1c7ab14f9949eec60c5d1f924f17c037e9f8b33352fa05799359b8", size = 123681 }, + { url = "https://files.pythonhosted.org/packages/ca/f0/f44f50c9f5b1a1860261092e3bc91ecdc9acda848a8b8c6abfda4a24dd5c/kiwisolver-1.4.9-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:efb3a45b35622bb6c16dbfab491a8f5a391fe0e9d45ef32f4df85658232ca0e2", size = 66464 }, + { url = "https://files.pythonhosted.org/packages/2d/7a/9d90a151f558e29c3936b8a47ac770235f436f2120aca41a6d5f3d62ae8d/kiwisolver-1.4.9-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:1a12cf6398e8a0a001a059747a1cbf24705e18fe413bc22de7b3d15c67cffe3f", size = 64961 }, + { url = "https://files.pythonhosted.org/packages/e9/e9/f218a2cb3a9ffbe324ca29a9e399fa2d2866d7f348ec3a88df87fc248fc5/kiwisolver-1.4.9-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:b67e6efbf68e077dd71d1a6b37e43e1a99d0bff1a3d51867d45ee8908b931098", size = 1474607 }, + { url = "https://files.pythonhosted.org/packages/d9/28/aac26d4c882f14de59041636292bc838db8961373825df23b8eeb807e198/kiwisolver-1.4.9-cp313-cp313-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5656aa670507437af0207645273ccdfee4f14bacd7f7c67a4306d0dcaeaf6eed", size = 1276546 }, + { url = "https://files.pythonhosted.org/packages/8b/ad/8bfc1c93d4cc565e5069162f610ba2f48ff39b7de4b5b8d93f69f30c4bed/kiwisolver-1.4.9-cp313-cp313-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:bfc08add558155345129c7803b3671cf195e6a56e7a12f3dde7c57d9b417f525", size = 1294482 }, + { url = "https://files.pythonhosted.org/packages/da/f1/6aca55ff798901d8ce403206d00e033191f63d82dd708a186e0ed2067e9c/kiwisolver-1.4.9-cp313-cp313-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:40092754720b174e6ccf9e845d0d8c7d8e12c3d71e7fc35f55f3813e96376f78", size = 1343720 }, + { url = "https://files.pythonhosted.org/packages/d1/91/eed031876c595c81d90d0f6fc681ece250e14bf6998c3d7c419466b523b7/kiwisolver-1.4.9-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:497d05f29a1300d14e02e6441cf0f5ee81c1ff5a304b0d9fb77423974684e08b", size = 2224907 }, + { url = "https://files.pythonhosted.org/packages/e9/ec/4d1925f2e49617b9cca9c34bfa11adefad49d00db038e692a559454dfb2e/kiwisolver-1.4.9-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:bdd1a81a1860476eb41ac4bc1e07b3f07259e6d55bbf739b79c8aaedcf512799", size = 2321334 }, + { url = "https://files.pythonhosted.org/packages/43/cb/450cd4499356f68802750c6ddc18647b8ea01ffa28f50d20598e0befe6e9/kiwisolver-1.4.9-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:e6b93f13371d341afee3be9f7c5964e3fe61d5fa30f6a30eb49856935dfe4fc3", size = 2488313 }, + { url = "https://files.pythonhosted.org/packages/71/67/fc76242bd99f885651128a5d4fa6083e5524694b7c88b489b1b55fdc491d/kiwisolver-1.4.9-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:d75aa530ccfaa593da12834b86a0724f58bff12706659baa9227c2ccaa06264c", size = 2291970 }, + { url = "https://files.pythonhosted.org/packages/75/bd/f1a5d894000941739f2ae1b65a32892349423ad49c2e6d0771d0bad3fae4/kiwisolver-1.4.9-cp313-cp313-win_amd64.whl", hash = "sha256:dd0a578400839256df88c16abddf9ba14813ec5f21362e1fe65022e00c883d4d", size = 73894 }, + { url = "https://files.pythonhosted.org/packages/95/38/dce480814d25b99a391abbddadc78f7c117c6da34be68ca8b02d5848b424/kiwisolver-1.4.9-cp313-cp313-win_arm64.whl", hash = "sha256:d4188e73af84ca82468f09cadc5ac4db578109e52acb4518d8154698d3a87ca2", size = 64995 }, + { url = "https://files.pythonhosted.org/packages/e2/37/7d218ce5d92dadc5ebdd9070d903e0c7cf7edfe03f179433ac4d13ce659c/kiwisolver-1.4.9-cp313-cp313t-macosx_10_13_universal2.whl", hash = "sha256:5a0f2724dfd4e3b3ac5a82436a8e6fd16baa7d507117e4279b660fe8ca38a3a1", size = 126510 }, + { url = "https://files.pythonhosted.org/packages/23/b0/e85a2b48233daef4b648fb657ebbb6f8367696a2d9548a00b4ee0eb67803/kiwisolver-1.4.9-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:1b11d6a633e4ed84fc0ddafd4ebfd8ea49b3f25082c04ad12b8315c11d504dc1", size = 67903 }, + { url = "https://files.pythonhosted.org/packages/44/98/f2425bc0113ad7de24da6bb4dae1343476e95e1d738be7c04d31a5d037fd/kiwisolver-1.4.9-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:61874cdb0a36016354853593cffc38e56fc9ca5aa97d2c05d3dcf6922cd55a11", size = 66402 }, + { url = "https://files.pythonhosted.org/packages/98/d8/594657886df9f34c4177cc353cc28ca7e6e5eb562d37ccc233bff43bbe2a/kiwisolver-1.4.9-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:60c439763a969a6af93b4881db0eed8fadf93ee98e18cbc35bc8da868d0c4f0c", size = 1582135 }, + { url = "https://files.pythonhosted.org/packages/5c/c6/38a115b7170f8b306fc929e166340c24958347308ea3012c2b44e7e295db/kiwisolver-1.4.9-cp313-cp313t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:92a2f997387a1b79a75e7803aa7ded2cfbe2823852ccf1ba3bcf613b62ae3197", size = 1389409 }, + { url = "https://files.pythonhosted.org/packages/bf/3b/e04883dace81f24a568bcee6eb3001da4ba05114afa622ec9b6fafdc1f5e/kiwisolver-1.4.9-cp313-cp313t-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:a31d512c812daea6d8b3be3b2bfcbeb091dbb09177706569bcfc6240dcf8b41c", size = 1401763 }, + { url = "https://files.pythonhosted.org/packages/9f/80/20ace48e33408947af49d7d15c341eaee69e4e0304aab4b7660e234d6288/kiwisolver-1.4.9-cp313-cp313t-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:52a15b0f35dad39862d376df10c5230155243a2c1a436e39eb55623ccbd68185", size = 1453643 }, + { url = "https://files.pythonhosted.org/packages/64/31/6ce4380a4cd1f515bdda976a1e90e547ccd47b67a1546d63884463c92ca9/kiwisolver-1.4.9-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:a30fd6fdef1430fd9e1ba7b3398b5ee4e2887783917a687d86ba69985fb08748", size = 2330818 }, + { url = "https://files.pythonhosted.org/packages/fa/e9/3f3fcba3bcc7432c795b82646306e822f3fd74df0ee81f0fa067a1f95668/kiwisolver-1.4.9-cp313-cp313t-musllinux_1_2_ppc64le.whl", hash = "sha256:cc9617b46837c6468197b5945e196ee9ca43057bb7d9d1ae688101e4e1dddf64", size = 2419963 }, + { url = "https://files.pythonhosted.org/packages/99/43/7320c50e4133575c66e9f7dadead35ab22d7c012a3b09bb35647792b2a6d/kiwisolver-1.4.9-cp313-cp313t-musllinux_1_2_s390x.whl", hash = "sha256:0ab74e19f6a2b027ea4f845a78827969af45ce790e6cb3e1ebab71bdf9f215ff", size = 2594639 }, + { url = "https://files.pythonhosted.org/packages/65/d6/17ae4a270d4a987ef8a385b906d2bdfc9fce502d6dc0d3aea865b47f548c/kiwisolver-1.4.9-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:dba5ee5d3981160c28d5490f0d1b7ed730c22470ff7f6cc26cfcfaacb9896a07", size = 2391741 }, + { url = "https://files.pythonhosted.org/packages/2a/8f/8f6f491d595a9e5912971f3f863d81baddccc8a4d0c3749d6a0dd9ffc9df/kiwisolver-1.4.9-cp313-cp313t-win_arm64.whl", hash = "sha256:0749fd8f4218ad2e851e11cc4dc05c7cbc0cbc4267bdfdb31782e65aace4ee9c", size = 68646 }, + { url = "https://files.pythonhosted.org/packages/6b/32/6cc0fbc9c54d06c2969faa9c1d29f5751a2e51809dd55c69055e62d9b426/kiwisolver-1.4.9-cp314-cp314-macosx_10_13_universal2.whl", hash = "sha256:9928fe1eb816d11ae170885a74d074f57af3a0d65777ca47e9aeb854a1fba386", size = 123806 }, + { url = "https://files.pythonhosted.org/packages/b2/dd/2bfb1d4a4823d92e8cbb420fe024b8d2167f72079b3bb941207c42570bdf/kiwisolver-1.4.9-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:d0005b053977e7b43388ddec89fa567f43d4f6d5c2c0affe57de5ebf290dc552", size = 66605 }, + { url = "https://files.pythonhosted.org/packages/f7/69/00aafdb4e4509c2ca6064646cba9cd4b37933898f426756adb2cb92ebbed/kiwisolver-1.4.9-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:2635d352d67458b66fd0667c14cb1d4145e9560d503219034a18a87e971ce4f3", size = 64925 }, + { url = "https://files.pythonhosted.org/packages/43/dc/51acc6791aa14e5cb6d8a2e28cefb0dc2886d8862795449d021334c0df20/kiwisolver-1.4.9-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:767c23ad1c58c9e827b649a9ab7809fd5fd9db266a9cf02b0e926ddc2c680d58", size = 1472414 }, + { url = "https://files.pythonhosted.org/packages/3d/bb/93fa64a81db304ac8a246f834d5094fae4b13baf53c839d6bb6e81177129/kiwisolver-1.4.9-cp314-cp314-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:72d0eb9fba308b8311685c2268cf7d0a0639a6cd027d8128659f72bdd8a024b4", size = 1281272 }, + { url = "https://files.pythonhosted.org/packages/70/e6/6df102916960fb8d05069d4bd92d6d9a8202d5a3e2444494e7cd50f65b7a/kiwisolver-1.4.9-cp314-cp314-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:f68e4f3eeca8fb22cc3d731f9715a13b652795ef657a13df1ad0c7dc0e9731df", size = 1298578 }, + { url = "https://files.pythonhosted.org/packages/7c/47/e142aaa612f5343736b087864dbaebc53ea8831453fb47e7521fa8658f30/kiwisolver-1.4.9-cp314-cp314-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:d84cd4061ae292d8ac367b2c3fa3aad11cb8625a95d135fe93f286f914f3f5a6", size = 1345607 }, + { url = "https://files.pythonhosted.org/packages/54/89/d641a746194a0f4d1a3670fb900d0dbaa786fb98341056814bc3f058fa52/kiwisolver-1.4.9-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:a60ea74330b91bd22a29638940d115df9dc00af5035a9a2a6ad9399ffb4ceca5", size = 2230150 }, + { url = "https://files.pythonhosted.org/packages/aa/6b/5ee1207198febdf16ac11f78c5ae40861b809cbe0e6d2a8d5b0b3044b199/kiwisolver-1.4.9-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:ce6a3a4e106cf35c2d9c4fa17c05ce0b180db622736845d4315519397a77beaf", size = 2325979 }, + { url = "https://files.pythonhosted.org/packages/fc/ff/b269eefd90f4ae14dcc74973d5a0f6d28d3b9bb1afd8c0340513afe6b39a/kiwisolver-1.4.9-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:77937e5e2a38a7b48eef0585114fe7930346993a88060d0bf886086d2aa49ef5", size = 2491456 }, + { url = "https://files.pythonhosted.org/packages/fc/d4/10303190bd4d30de547534601e259a4fbf014eed94aae3e5521129215086/kiwisolver-1.4.9-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:24c175051354f4a28c5d6a31c93906dc653e2bf234e8a4bbfb964892078898ce", size = 2294621 }, + { url = "https://files.pythonhosted.org/packages/28/e0/a9a90416fce5c0be25742729c2ea52105d62eda6c4be4d803c2a7be1fa50/kiwisolver-1.4.9-cp314-cp314-win_amd64.whl", hash = "sha256:0763515d4df10edf6d06a3c19734e2566368980d21ebec439f33f9eb936c07b7", size = 75417 }, + { url = "https://files.pythonhosted.org/packages/1f/10/6949958215b7a9a264299a7db195564e87900f709db9245e4ebdd3c70779/kiwisolver-1.4.9-cp314-cp314-win_arm64.whl", hash = "sha256:0e4e2bf29574a6a7b7f6cb5fa69293b9f96c928949ac4a53ba3f525dffb87f9c", size = 66582 }, + { url = "https://files.pythonhosted.org/packages/ec/79/60e53067903d3bc5469b369fe0dfc6b3482e2133e85dae9daa9527535991/kiwisolver-1.4.9-cp314-cp314t-macosx_10_13_universal2.whl", hash = "sha256:d976bbb382b202f71c67f77b0ac11244021cfa3f7dfd9e562eefcea2df711548", size = 126514 }, + { url = "https://files.pythonhosted.org/packages/25/d1/4843d3e8d46b072c12a38c97c57fab4608d36e13fe47d47ee96b4d61ba6f/kiwisolver-1.4.9-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:2489e4e5d7ef9a1c300a5e0196e43d9c739f066ef23270607d45aba368b91f2d", size = 67905 }, + { url = "https://files.pythonhosted.org/packages/8c/ae/29ffcbd239aea8b93108de1278271ae764dfc0d803a5693914975f200596/kiwisolver-1.4.9-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:e2ea9f7ab7fbf18fffb1b5434ce7c69a07582f7acc7717720f1d69f3e806f90c", size = 66399 }, + { url = "https://files.pythonhosted.org/packages/a1/ae/d7ba902aa604152c2ceba5d352d7b62106bedbccc8e95c3934d94472bfa3/kiwisolver-1.4.9-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:b34e51affded8faee0dfdb705416153819d8ea9250bbbf7ea1b249bdeb5f1122", size = 1582197 }, + { url = "https://files.pythonhosted.org/packages/f2/41/27c70d427eddb8bc7e4f16420a20fefc6f480312122a59a959fdfe0445ad/kiwisolver-1.4.9-cp314-cp314t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d8aacd3d4b33b772542b2e01beb50187536967b514b00003bdda7589722d2a64", size = 1390125 }, + { url = "https://files.pythonhosted.org/packages/41/42/b3799a12bafc76d962ad69083f8b43b12bf4fe78b097b12e105d75c9b8f1/kiwisolver-1.4.9-cp314-cp314t-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:7cf974dd4e35fa315563ac99d6287a1024e4dc2077b8a7d7cd3d2fb65d283134", size = 1402612 }, + { url = "https://files.pythonhosted.org/packages/d2/b5/a210ea073ea1cfaca1bb5c55a62307d8252f531beb364e18aa1e0888b5a0/kiwisolver-1.4.9-cp314-cp314t-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:85bd218b5ecfbee8c8a82e121802dcb519a86044c9c3b2e4aef02fa05c6da370", size = 1453990 }, + { url = "https://files.pythonhosted.org/packages/5f/ce/a829eb8c033e977d7ea03ed32fb3c1781b4fa0433fbadfff29e39c676f32/kiwisolver-1.4.9-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:0856e241c2d3df4efef7c04a1e46b1936b6120c9bcf36dd216e3acd84bc4fb21", size = 2331601 }, + { url = "https://files.pythonhosted.org/packages/e0/4b/b5e97eb142eb9cd0072dacfcdcd31b1c66dc7352b0f7c7255d339c0edf00/kiwisolver-1.4.9-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:9af39d6551f97d31a4deebeac6f45b156f9755ddc59c07b402c148f5dbb6482a", size = 2422041 }, + { url = "https://files.pythonhosted.org/packages/40/be/8eb4cd53e1b85ba4edc3a9321666f12b83113a178845593307a3e7891f44/kiwisolver-1.4.9-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:bb4ae2b57fc1d8cbd1cf7b1d9913803681ffa903e7488012be5b76dedf49297f", size = 2594897 }, + { url = "https://files.pythonhosted.org/packages/99/dd/841e9a66c4715477ea0abc78da039832fbb09dac5c35c58dc4c41a407b8a/kiwisolver-1.4.9-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:aedff62918805fb62d43a4aa2ecd4482c380dc76cd31bd7c8878588a61bd0369", size = 2391835 }, + { url = "https://files.pythonhosted.org/packages/0c/28/4b2e5c47a0da96896fdfdb006340ade064afa1e63675d01ea5ac222b6d52/kiwisolver-1.4.9-cp314-cp314t-win_amd64.whl", hash = "sha256:1fa333e8b2ce4d9660f2cda9c0e1b6bafcfb2457a9d259faa82289e73ec24891", size = 79988 }, + { url = "https://files.pythonhosted.org/packages/80/be/3578e8afd18c88cdf9cb4cffde75a96d2be38c5a903f1ed0ceec061bd09e/kiwisolver-1.4.9-cp314-cp314t-win_arm64.whl", hash = "sha256:4a48a2ce79d65d363597ef7b567ce3d14d68783d2b2263d98db3d9477805ba32", size = 70260 }, + { url = "https://files.pythonhosted.org/packages/a3/0f/36d89194b5a32c054ce93e586d4049b6c2c22887b0eb229c61c68afd3078/kiwisolver-1.4.9-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:720e05574713db64c356e86732c0f3c5252818d05f9df320f0ad8380641acea5", size = 60104 }, + { url = "https://files.pythonhosted.org/packages/52/ba/4ed75f59e4658fd21fe7dde1fee0ac397c678ec3befba3fe6482d987af87/kiwisolver-1.4.9-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:17680d737d5335b552994a2008fab4c851bcd7de33094a82067ef3a576ff02fa", size = 58592 }, + { url = "https://files.pythonhosted.org/packages/33/01/a8ea7c5ea32a9b45ceeaee051a04c8ed4320f5add3c51bfa20879b765b70/kiwisolver-1.4.9-pp311-pypy311_pp73-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:85b5352f94e490c028926ea567fc569c52ec79ce131dadb968d3853e809518c2", size = 80281 }, + { url = "https://files.pythonhosted.org/packages/da/e3/dbd2ecdce306f1d07a1aaf324817ee993aab7aee9db47ceac757deabafbe/kiwisolver-1.4.9-pp311-pypy311_pp73-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:464415881e4801295659462c49461a24fb107c140de781d55518c4b80cb6790f", size = 78009 }, + { url = "https://files.pythonhosted.org/packages/da/e9/0d4add7873a73e462aeb45c036a2dead2562b825aa46ba326727b3f31016/kiwisolver-1.4.9-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:fb940820c63a9590d31d88b815e7a3aa5915cad3ce735ab45f0c730b39547de1", size = 73929 }, +] + +[[package]] +name = "loguru" +version = "0.7.3" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "colorama", marker = "sys_platform == 'win32'" }, + { name = "win32-setctime", marker = "sys_platform == 'win32'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/3a/05/a1dae3dffd1116099471c643b8924f5aa6524411dc6c63fdae648c4f1aca/loguru-0.7.3.tar.gz", hash = "sha256:19480589e77d47b8d85b2c827ad95d49bf31b0dcde16593892eb51dd18706eb6", size = 63559 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/0c/29/0348de65b8cc732daa3e33e67806420b2ae89bdce2b04af740289c5c6c8c/loguru-0.7.3-py3-none-any.whl", hash = "sha256:31a33c10c8e1e10422bfd431aeb5d351c7cf7fa671e3c4df004162264b28220c", size = 61595 }, +] + +[[package]] +name = "matplotlib" +version = "3.10.8" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "contourpy" }, + { name = "cycler" }, + { name = "fonttools" }, + { name = "kiwisolver" }, + { name = "numpy" }, + { name = "packaging" }, + { name = "pillow" }, + { name = "pyparsing" }, + { name = "python-dateutil" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/8a/76/d3c6e3a13fe484ebe7718d14e269c9569c4eb0020a968a327acb3b9a8fe6/matplotlib-3.10.8.tar.gz", hash = "sha256:2299372c19d56bcd35cf05a2738308758d32b9eaed2371898d8f5bd33f084aa3", size = 34806269 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f8/86/de7e3a1cdcfc941483af70609edc06b83e7c8a0e0dc9ac325200a3f4d220/matplotlib-3.10.8-cp311-cp311-macosx_10_12_x86_64.whl", hash = "sha256:6be43b667360fef5c754dda5d25a32e6307a03c204f3c0fc5468b78fa87b4160", size = 8251215 }, + { url = "https://files.pythonhosted.org/packages/fd/14/baad3222f424b19ce6ad243c71de1ad9ec6b2e4eb1e458a48fdc6d120401/matplotlib-3.10.8-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:a2b336e2d91a3d7006864e0990c83b216fcdca64b5a6484912902cef87313d78", size = 8139625 }, + { url = "https://files.pythonhosted.org/packages/8f/a0/7024215e95d456de5883e6732e708d8187d9753a21d32f8ddb3befc0c445/matplotlib-3.10.8-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:efb30e3baaea72ce5928e32bab719ab4770099079d66726a62b11b1ef7273be4", size = 8712614 }, + { url = "https://files.pythonhosted.org/packages/5a/f4/b8347351da9a5b3f41e26cf547252d861f685c6867d179a7c9d60ad50189/matplotlib-3.10.8-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d56a1efd5bfd61486c8bc968fa18734464556f0fb8e51690f4ac25d85cbbbbc2", size = 9540997 }, + { url = "https://files.pythonhosted.org/packages/9e/c0/c7b914e297efe0bc36917bf216b2acb91044b91e930e878ae12981e461e5/matplotlib-3.10.8-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:238b7ce5717600615c895050239ec955d91f321c209dd110db988500558e70d6", size = 9596825 }, + { url = "https://files.pythonhosted.org/packages/6f/d3/a4bbc01c237ab710a1f22b4da72f4ff6d77eb4c7735ea9811a94ae239067/matplotlib-3.10.8-cp311-cp311-win_amd64.whl", hash = "sha256:18821ace09c763ec93aef5eeff087ee493a24051936d7b9ebcad9662f66501f9", size = 8135090 }, + { url = "https://files.pythonhosted.org/packages/89/dd/a0b6588f102beab33ca6f5218b31725216577b2a24172f327eaf6417d5c9/matplotlib-3.10.8-cp311-cp311-win_arm64.whl", hash = "sha256:bab485bcf8b1c7d2060b4fcb6fc368a9e6f4cd754c9c2fea281f4be21df394a2", size = 8012377 }, + { url = "https://files.pythonhosted.org/packages/9e/67/f997cdcbb514012eb0d10cd2b4b332667997fb5ebe26b8d41d04962fa0e6/matplotlib-3.10.8-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:64fcc24778ca0404ce0cb7b6b77ae1f4c7231cdd60e6778f999ee05cbd581b9a", size = 8260453 }, + { url = "https://files.pythonhosted.org/packages/7e/65/07d5f5c7f7c994f12c768708bd2e17a4f01a2b0f44a1c9eccad872433e2e/matplotlib-3.10.8-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:b9a5ca4ac220a0cdd1ba6bcba3608547117d30468fefce49bb26f55c1a3d5c58", size = 8148321 }, + { url = "https://files.pythonhosted.org/packages/3e/f3/c5195b1ae57ef85339fd7285dfb603b22c8b4e79114bae5f4f0fcf688677/matplotlib-3.10.8-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:3ab4aabc72de4ff77b3ec33a6d78a68227bf1123465887f9905ba79184a1cc04", size = 8716944 }, + { url = "https://files.pythonhosted.org/packages/00/f9/7638f5cc82ec8a7aa005de48622eecc3ed7c9854b96ba15bd76b7fd27574/matplotlib-3.10.8-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:24d50994d8c5816ddc35411e50a86ab05f575e2530c02752e02538122613371f", size = 9550099 }, + { url = "https://files.pythonhosted.org/packages/57/61/78cd5920d35b29fd2a0fe894de8adf672ff52939d2e9b43cb83cd5ce1bc7/matplotlib-3.10.8-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:99eefd13c0dc3b3c1b4d561c1169e65fe47aab7b8158754d7c084088e2329466", size = 9613040 }, + { url = "https://files.pythonhosted.org/packages/30/4e/c10f171b6e2f44d9e3a2b96efa38b1677439d79c99357600a62cc1e9594e/matplotlib-3.10.8-cp312-cp312-win_amd64.whl", hash = "sha256:dd80ecb295460a5d9d260df63c43f4afbdd832d725a531f008dad1664f458adf", size = 8142717 }, + { url = "https://files.pythonhosted.org/packages/f1/76/934db220026b5fef85f45d51a738b91dea7d70207581063cd9bd8fafcf74/matplotlib-3.10.8-cp312-cp312-win_arm64.whl", hash = "sha256:3c624e43ed56313651bc18a47f838b60d7b8032ed348911c54906b130b20071b", size = 8012751 }, + { url = "https://files.pythonhosted.org/packages/3d/b9/15fd5541ef4f5b9a17eefd379356cf12175fe577424e7b1d80676516031a/matplotlib-3.10.8-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:3f2e409836d7f5ac2f1c013110a4d50b9f7edc26328c108915f9075d7d7a91b6", size = 8261076 }, + { url = "https://files.pythonhosted.org/packages/8d/a0/2ba3473c1b66b9c74dc7107c67e9008cb1782edbe896d4c899d39ae9cf78/matplotlib-3.10.8-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:56271f3dac49a88d7fca5060f004d9d22b865f743a12a23b1e937a0be4818ee1", size = 8148794 }, + { url = "https://files.pythonhosted.org/packages/75/97/a471f1c3eb1fd6f6c24a31a5858f443891d5127e63a7788678d14e249aea/matplotlib-3.10.8-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:a0a7f52498f72f13d4a25ea70f35f4cb60642b466cbb0a9be951b5bc3f45a486", size = 8718474 }, + { url = "https://files.pythonhosted.org/packages/01/be/cd478f4b66f48256f42927d0acbcd63a26a893136456cd079c0cc24fbabf/matplotlib-3.10.8-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:646d95230efb9ca614a7a594d4fcacde0ac61d25e37dd51710b36477594963ce", size = 9549637 }, + { url = "https://files.pythonhosted.org/packages/5d/7c/8dc289776eae5109e268c4fb92baf870678dc048a25d4ac903683b86d5bf/matplotlib-3.10.8-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:f89c151aab2e2e23cb3fe0acad1e8b82841fd265379c4cecd0f3fcb34c15e0f6", size = 9613678 }, + { url = "https://files.pythonhosted.org/packages/64/40/37612487cc8a437d4dd261b32ca21fe2d79510fe74af74e1f42becb1bdb8/matplotlib-3.10.8-cp313-cp313-win_amd64.whl", hash = "sha256:e8ea3e2d4066083e264e75c829078f9e149fa119d27e19acd503de65e0b13149", size = 8142686 }, + { url = "https://files.pythonhosted.org/packages/66/52/8d8a8730e968185514680c2a6625943f70269509c3dcfc0dcf7d75928cb8/matplotlib-3.10.8-cp313-cp313-win_arm64.whl", hash = "sha256:c108a1d6fa78a50646029cb6d49808ff0fc1330fda87fa6f6250c6b5369b6645", size = 8012917 }, + { url = "https://files.pythonhosted.org/packages/b5/27/51fe26e1062f298af5ef66343d8ef460e090a27fea73036c76c35821df04/matplotlib-3.10.8-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:ad3d9833a64cf48cc4300f2b406c3d0f4f4724a91c0bd5640678a6ba7c102077", size = 8305679 }, + { url = "https://files.pythonhosted.org/packages/2c/1e/4de865bc591ac8e3062e835f42dd7fe7a93168d519557837f0e37513f629/matplotlib-3.10.8-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:eb3823f11823deade26ce3b9f40dcb4a213da7a670013929f31d5f5ed1055b22", size = 8198336 }, + { url = "https://files.pythonhosted.org/packages/c6/cb/2f7b6e75fb4dce87ef91f60cac4f6e34f4c145ab036a22318ec837971300/matplotlib-3.10.8-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:d9050fee89a89ed57b4fb2c1bfac9a3d0c57a0d55aed95949eedbc42070fea39", size = 8731653 }, + { url = "https://files.pythonhosted.org/packages/46/b3/bd9c57d6ba670a37ab31fb87ec3e8691b947134b201f881665b28cc039ff/matplotlib-3.10.8-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b44d07310e404ba95f8c25aa5536f154c0a8ec473303535949e52eb71d0a1565", size = 9561356 }, + { url = "https://files.pythonhosted.org/packages/c0/3d/8b94a481456dfc9dfe6e39e93b5ab376e50998cddfd23f4ae3b431708f16/matplotlib-3.10.8-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:0a33deb84c15ede243aead39f77e990469fff93ad1521163305095b77b72ce4a", size = 9614000 }, + { url = "https://files.pythonhosted.org/packages/bd/cd/bc06149fe5585ba800b189a6a654a75f1f127e8aab02fd2be10df7fa500c/matplotlib-3.10.8-cp313-cp313t-win_amd64.whl", hash = "sha256:3a48a78d2786784cc2413e57397981fb45c79e968d99656706018d6e62e57958", size = 8220043 }, + { url = "https://files.pythonhosted.org/packages/e3/de/b22cf255abec916562cc04eef457c13e58a1990048de0c0c3604d082355e/matplotlib-3.10.8-cp313-cp313t-win_arm64.whl", hash = "sha256:15d30132718972c2c074cd14638c7f4592bd98719e2308bccea40e0538bc0cb5", size = 8062075 }, + { url = "https://files.pythonhosted.org/packages/3c/43/9c0ff7a2f11615e516c3b058e1e6e8f9614ddeca53faca06da267c48345d/matplotlib-3.10.8-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:b53285e65d4fa4c86399979e956235deb900be5baa7fc1218ea67fbfaeaadd6f", size = 8262481 }, + { url = "https://files.pythonhosted.org/packages/6f/ca/e8ae28649fcdf039fda5ef554b40a95f50592a3c47e6f7270c9561c12b07/matplotlib-3.10.8-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:32f8dce744be5569bebe789e46727946041199030db8aeb2954d26013a0eb26b", size = 8151473 }, + { url = "https://files.pythonhosted.org/packages/f1/6f/009d129ae70b75e88cbe7e503a12a4c0670e08ed748a902c2568909e9eb5/matplotlib-3.10.8-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4cf267add95b1c88300d96ca837833d4112756045364f5c734a2276038dae27d", size = 9553896 }, + { url = "https://files.pythonhosted.org/packages/f5/26/4221a741eb97967bc1fd5e4c52b9aa5a91b2f4ec05b59f6def4d820f9df9/matplotlib-3.10.8-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:2cf5bd12cecf46908f286d7838b2abc6c91cda506c0445b8223a7c19a00df008", size = 9824193 }, + { url = "https://files.pythonhosted.org/packages/1f/f3/3abf75f38605772cf48a9daf5821cd4f563472f38b4b828c6fba6fa6d06e/matplotlib-3.10.8-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:41703cc95688f2516b480f7f339d8851a6035f18e100ee6a32bc0b8536a12a9c", size = 9615444 }, + { url = "https://files.pythonhosted.org/packages/93/a5/de89ac80f10b8dc615807ee1133cd99ac74082581196d4d9590bea10690d/matplotlib-3.10.8-cp314-cp314-win_amd64.whl", hash = "sha256:83d282364ea9f3e52363da262ce32a09dfe241e4080dcedda3c0db059d3c1f11", size = 8272719 }, + { url = "https://files.pythonhosted.org/packages/69/ce/b006495c19ccc0a137b48083168a37bd056392dee02f87dba0472f2797fe/matplotlib-3.10.8-cp314-cp314-win_arm64.whl", hash = "sha256:2c1998e92cd5999e295a731bcb2911c75f597d937341f3030cc24ef2733d78a8", size = 8144205 }, + { url = "https://files.pythonhosted.org/packages/68/d9/b31116a3a855bd313c6fcdb7226926d59b041f26061c6c5b1be66a08c826/matplotlib-3.10.8-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:b5a2b97dbdc7d4f353ebf343744f1d1f1cca8aa8bfddb4262fcf4306c3761d50", size = 8305785 }, + { url = "https://files.pythonhosted.org/packages/1e/90/6effe8103f0272685767ba5f094f453784057072f49b393e3ea178fe70a5/matplotlib-3.10.8-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:3f5c3e4da343bba819f0234186b9004faba952cc420fbc522dc4e103c1985908", size = 8198361 }, + { url = "https://files.pythonhosted.org/packages/d7/65/a73188711bea603615fc0baecca1061429ac16940e2385433cc778a9d8e7/matplotlib-3.10.8-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5f62550b9a30afde8c1c3ae450e5eb547d579dd69b25c2fc7a1c67f934c1717a", size = 9561357 }, + { url = "https://files.pythonhosted.org/packages/f4/3d/b5c5d5d5be8ce63292567f0e2c43dde9953d3ed86ac2de0a72e93c8f07a1/matplotlib-3.10.8-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:495672de149445ec1b772ff2c9ede9b769e3cb4f0d0aa7fa730d7f59e2d4e1c1", size = 9823610 }, + { url = "https://files.pythonhosted.org/packages/4d/4b/e7beb6bbd49f6bae727a12b270a2654d13c397576d25bd6786e47033300f/matplotlib-3.10.8-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:595ba4d8fe983b88f0eec8c26a241e16d6376fe1979086232f481f8f3f67494c", size = 9614011 }, + { url = "https://files.pythonhosted.org/packages/7c/e6/76f2813d31f032e65f6f797e3f2f6e4aab95b65015924b1c51370395c28a/matplotlib-3.10.8-cp314-cp314t-win_amd64.whl", hash = "sha256:25d380fe8b1dc32cf8f0b1b448470a77afb195438bafdf1d858bfb876f3edf7b", size = 8362801 }, + { url = "https://files.pythonhosted.org/packages/5d/49/d651878698a0b67f23aa28e17f45a6d6dd3d3f933fa29087fa4ce5947b5a/matplotlib-3.10.8-cp314-cp314t-win_arm64.whl", hash = "sha256:113bb52413ea508ce954a02c10ffd0d565f9c3bc7f2eddc27dfe1731e71c7b5f", size = 8192560 }, + { url = "https://files.pythonhosted.org/packages/04/30/3afaa31c757f34b7725ab9d2ba8b48b5e89c2019c003e7d0ead143aabc5a/matplotlib-3.10.8-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:6da7c2ce169267d0d066adcf63758f0604aa6c3eebf67458930f9d9b79ad1db1", size = 8249198 }, + { url = "https://files.pythonhosted.org/packages/48/2f/6334aec331f57485a642a7c8be03cb286f29111ae71c46c38b363230063c/matplotlib-3.10.8-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:9153c3292705be9f9c64498a8872118540c3f4123d1a1c840172edf262c8be4a", size = 8136817 }, + { url = "https://files.pythonhosted.org/packages/73/e4/6d6f14b2a759c622f191b2d67e9075a3f56aaccb3be4bb9bb6890030d0a0/matplotlib-3.10.8-pp311-pypy311_pp73-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:1ae029229a57cd1e8fe542485f27e7ca7b23aa9e8944ddb4985d0bc444f1eca2", size = 8713867 }, +] + +[[package]] +name = "numpy" +version = "2.4.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/24/62/ae72ff66c0f1fd959925b4c11f8c2dea61f47f6acaea75a08512cdfe3fed/numpy-2.4.1.tar.gz", hash = "sha256:a1ceafc5042451a858231588a104093474c6a5c57dcc724841f5c888d237d690", size = 20721320 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a5/34/2b1bc18424f3ad9af577f6ce23600319968a70575bd7db31ce66731bbef9/numpy-2.4.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:0cce2a669e3c8ba02ee563c7835f92c153cf02edff1ae05e1823f1dde21b16a5", size = 16944563 }, + { url = "https://files.pythonhosted.org/packages/2c/57/26e5f97d075aef3794045a6ca9eada6a4ed70eb9a40e7a4a93f9ac80d704/numpy-2.4.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:899d2c18024984814ac7e83f8f49d8e8180e2fbe1b2e252f2e7f1d06bea92425", size = 12645658 }, + { url = "https://files.pythonhosted.org/packages/8e/ba/80fc0b1e3cb2fd5c6143f00f42eb67762aa043eaa05ca924ecc3222a7849/numpy-2.4.1-cp311-cp311-macosx_14_0_arm64.whl", hash = "sha256:09aa8a87e45b55a1c2c205d42e2808849ece5c484b2aab11fecabec3841cafba", size = 5474132 }, + { url = "https://files.pythonhosted.org/packages/40/ae/0a5b9a397f0e865ec171187c78d9b57e5588afc439a04ba9cab1ebb2c945/numpy-2.4.1-cp311-cp311-macosx_14_0_x86_64.whl", hash = "sha256:edee228f76ee2dab4579fad6f51f6a305de09d444280109e0f75df247ff21501", size = 6804159 }, + { url = "https://files.pythonhosted.org/packages/86/9c/841c15e691c7085caa6fd162f063eff494099c8327aeccd509d1ab1e36ab/numpy-2.4.1-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a92f227dbcdc9e4c3e193add1a189a9909947d4f8504c576f4a732fd0b54240a", size = 14708058 }, + { url = "https://files.pythonhosted.org/packages/5d/9d/7862db06743f489e6a502a3b93136d73aea27d97b2cf91504f70a27501d6/numpy-2.4.1-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:538bf4ec353709c765ff75ae616c34d3c3dca1a68312727e8f2676ea644f8509", size = 16651501 }, + { url = "https://files.pythonhosted.org/packages/a6/9c/6fc34ebcbd4015c6e5f0c0ce38264010ce8a546cb6beacb457b84a75dfc8/numpy-2.4.1-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:ac08c63cb7779b85e9d5318e6c3518b424bc1f364ac4cb2c6136f12e5ff2dccc", size = 16492627 }, + { url = "https://files.pythonhosted.org/packages/aa/63/2494a8597502dacda439f61b3c0db4da59928150e62be0e99395c3ad23c5/numpy-2.4.1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:4f9c360ecef085e5841c539a9a12b883dff005fbd7ce46722f5e9cef52634d82", size = 18585052 }, + { url = "https://files.pythonhosted.org/packages/6a/93/098e1162ae7522fc9b618d6272b77404c4656c72432ecee3abc029aa3de0/numpy-2.4.1-cp311-cp311-win32.whl", hash = "sha256:0f118ce6b972080ba0758c6087c3617b5ba243d806268623dc34216d69099ba0", size = 6236575 }, + { url = "https://files.pythonhosted.org/packages/8c/de/f5e79650d23d9e12f38a7bc6b03ea0835b9575494f8ec94c11c6e773b1b1/numpy-2.4.1-cp311-cp311-win_amd64.whl", hash = "sha256:18e14c4d09d55eef39a6ab5b08406e84bc6869c1e34eef45564804f90b7e0574", size = 12604479 }, + { url = "https://files.pythonhosted.org/packages/dd/65/e1097a7047cff12ce3369bd003811516b20ba1078dbdec135e1cd7c16c56/numpy-2.4.1-cp311-cp311-win_arm64.whl", hash = "sha256:6461de5113088b399d655d45c3897fa188766415d0f568f175ab071c8873bd73", size = 10578325 }, + { url = "https://files.pythonhosted.org/packages/78/7f/ec53e32bf10c813604edf07a3682616bd931d026fcde7b6d13195dfb684a/numpy-2.4.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:d3703409aac693fa82c0aee023a1ae06a6e9d065dba10f5e8e80f642f1e9d0a2", size = 16656888 }, + { url = "https://files.pythonhosted.org/packages/b8/e0/1f9585d7dae8f14864e948fd7fa86c6cb72dee2676ca2748e63b1c5acfe0/numpy-2.4.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:7211b95ca365519d3596a1d8688a95874cc94219d417504d9ecb2df99fa7bfa8", size = 12373956 }, + { url = "https://files.pythonhosted.org/packages/8e/43/9762e88909ff2326f5e7536fa8cb3c49fb03a7d92705f23e6e7f553d9cb3/numpy-2.4.1-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:5adf01965456a664fc727ed69cc71848f28d063217c63e1a0e200a118d5eec9a", size = 5202567 }, + { url = "https://files.pythonhosted.org/packages/4b/ee/34b7930eb61e79feb4478800a4b95b46566969d837546aa7c034c742ef98/numpy-2.4.1-cp312-cp312-macosx_14_0_x86_64.whl", hash = "sha256:26f0bcd9c79a00e339565b303badc74d3ea2bd6d52191eeca5f95936cad107d0", size = 6549459 }, + { url = "https://files.pythonhosted.org/packages/79/e3/5f115fae982565771be994867c89bcd8d7208dbfe9469185497d70de5ddf/numpy-2.4.1-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0093e85df2960d7e4049664b26afc58b03236e967fb942354deef3208857a04c", size = 14404859 }, + { url = "https://files.pythonhosted.org/packages/d9/7d/9c8a781c88933725445a859cac5d01b5871588a15969ee6aeb618ba99eee/numpy-2.4.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:7ad270f438cbdd402c364980317fb6b117d9ec5e226fff5b4148dd9aa9fc6e02", size = 16371419 }, + { url = "https://files.pythonhosted.org/packages/a6/d2/8aa084818554543f17cf4162c42f162acbd3bb42688aefdba6628a859f77/numpy-2.4.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:297c72b1b98100c2e8f873d5d35fb551fce7040ade83d67dd51d38c8d42a2162", size = 16182131 }, + { url = "https://files.pythonhosted.org/packages/60/db/0425216684297c58a8df35f3284ef56ec4a043e6d283f8a59c53562caf1b/numpy-2.4.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:cf6470d91d34bf669f61d515499859fa7a4c2f7c36434afb70e82df7217933f9", size = 18295342 }, + { url = "https://files.pythonhosted.org/packages/31/4c/14cb9d86240bd8c386c881bafbe43f001284b7cce3bc01623ac9475da163/numpy-2.4.1-cp312-cp312-win32.whl", hash = "sha256:b6bcf39112e956594b3331316d90c90c90fb961e39696bda97b89462f5f3943f", size = 5959015 }, + { url = "https://files.pythonhosted.org/packages/51/cf/52a703dbeb0c65807540d29699fef5fda073434ff61846a564d5c296420f/numpy-2.4.1-cp312-cp312-win_amd64.whl", hash = "sha256:e1a27bb1b2dee45a2a53f5ca6ff2d1a7f135287883a1689e930d44d1ff296c87", size = 12310730 }, + { url = "https://files.pythonhosted.org/packages/69/80/a828b2d0ade5e74a9fe0f4e0a17c30fdc26232ad2bc8c9f8b3197cf7cf18/numpy-2.4.1-cp312-cp312-win_arm64.whl", hash = "sha256:0e6e8f9d9ecf95399982019c01223dc130542960a12edfa8edd1122dfa66a8a8", size = 10312166 }, + { url = "https://files.pythonhosted.org/packages/04/68/732d4b7811c00775f3bd522a21e8dd5a23f77eb11acdeb663e4a4ebf0ef4/numpy-2.4.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:d797454e37570cfd61143b73b8debd623c3c0952959adb817dd310a483d58a1b", size = 16652495 }, + { url = "https://files.pythonhosted.org/packages/20/ca/857722353421a27f1465652b2c66813eeeccea9d76d5f7b74b99f298e60e/numpy-2.4.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:82c55962006156aeef1629b953fd359064aa47e4d82cfc8e67f0918f7da3344f", size = 12368657 }, + { url = "https://files.pythonhosted.org/packages/81/0d/2377c917513449cc6240031a79d30eb9a163d32a91e79e0da47c43f2c0c8/numpy-2.4.1-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:71abbea030f2cfc3092a0ff9f8c8fdefdc5e0bf7d9d9c99663538bb0ecdac0b9", size = 5197256 }, + { url = "https://files.pythonhosted.org/packages/17/39/569452228de3f5de9064ac75137082c6214be1f5c532016549a7923ab4b5/numpy-2.4.1-cp313-cp313-macosx_14_0_x86_64.whl", hash = "sha256:5b55aa56165b17aaf15520beb9cbd33c9039810e0d9643dd4379e44294c7303e", size = 6545212 }, + { url = "https://files.pythonhosted.org/packages/8c/a4/77333f4d1e4dac4395385482557aeecf4826e6ff517e32ca48e1dafbe42a/numpy-2.4.1-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c0faba4a331195bfa96f93dd9dfaa10b2c7aa8cda3a02b7fd635e588fe821bf5", size = 14402871 }, + { url = "https://files.pythonhosted.org/packages/ba/87/d341e519956273b39d8d47969dd1eaa1af740615394fe67d06f1efa68773/numpy-2.4.1-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d3e3087f53e2b4428766b54932644d148613c5a595150533ae7f00dab2f319a8", size = 16359305 }, + { url = "https://files.pythonhosted.org/packages/32/91/789132c6666288eaa20ae8066bb99eba1939362e8f1a534949a215246e97/numpy-2.4.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:49e792ec351315e16da54b543db06ca8a86985ab682602d90c60ef4ff4db2a9c", size = 16181909 }, + { url = "https://files.pythonhosted.org/packages/cf/b8/090b8bd27b82a844bb22ff8fdf7935cb1980b48d6e439ae116f53cdc2143/numpy-2.4.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:79e9e06c4c2379db47f3f6fc7a8652e7498251789bf8ff5bd43bf478ef314ca2", size = 18284380 }, + { url = "https://files.pythonhosted.org/packages/67/78/722b62bd31842ff029412271556a1a27a98f45359dea78b1548a3a9996aa/numpy-2.4.1-cp313-cp313-win32.whl", hash = "sha256:3d1a100e48cb266090a031397863ff8a30050ceefd798f686ff92c67a486753d", size = 5957089 }, + { url = "https://files.pythonhosted.org/packages/da/a6/cf32198b0b6e18d4fbfa9a21a992a7fca535b9bb2b0cdd217d4a3445b5ca/numpy-2.4.1-cp313-cp313-win_amd64.whl", hash = "sha256:92a0e65272fd60bfa0d9278e0484c2f52fe03b97aedc02b357f33fe752c52ffb", size = 12307230 }, + { url = "https://files.pythonhosted.org/packages/44/6c/534d692bfb7d0afe30611320c5fb713659dcb5104d7cc182aff2aea092f5/numpy-2.4.1-cp313-cp313-win_arm64.whl", hash = "sha256:20d4649c773f66cc2fc36f663e091f57c3b7655f936a4c681b4250855d1da8f5", size = 10313125 }, + { url = "https://files.pythonhosted.org/packages/da/a1/354583ac5c4caa566de6ddfbc42744409b515039e085fab6e0ff942e0df5/numpy-2.4.1-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:f93bc6892fe7b0663e5ffa83b61aab510aacffd58c16e012bb9352d489d90cb7", size = 12496156 }, + { url = "https://files.pythonhosted.org/packages/51/b0/42807c6e8cce58c00127b1dc24d365305189991f2a7917aa694a109c8d7d/numpy-2.4.1-cp313-cp313t-macosx_14_0_arm64.whl", hash = "sha256:178de8f87948163d98a4c9ab5bee4ce6519ca918926ec8df195af582de28544d", size = 5324663 }, + { url = "https://files.pythonhosted.org/packages/fe/55/7a621694010d92375ed82f312b2f28017694ed784775269115323e37f5e2/numpy-2.4.1-cp313-cp313t-macosx_14_0_x86_64.whl", hash = "sha256:98b35775e03ab7f868908b524fc0a84d38932d8daf7b7e1c3c3a1b6c7a2c9f15", size = 6645224 }, + { url = "https://files.pythonhosted.org/packages/50/96/9fa8635ed9d7c847d87e30c834f7109fac5e88549d79ef3324ab5c20919f/numpy-2.4.1-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:941c2a93313d030f219f3a71fd3d91a728b82979a5e8034eb2e60d394a2b83f9", size = 14462352 }, + { url = "https://files.pythonhosted.org/packages/03/d1/8cf62d8bb2062da4fb82dd5d49e47c923f9c0738032f054e0a75342faba7/numpy-2.4.1-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:529050522e983e00a6c1c6b67411083630de8b57f65e853d7b03d9281b8694d2", size = 16407279 }, + { url = "https://files.pythonhosted.org/packages/86/1c/95c86e17c6b0b31ce6ef219da00f71113b220bcb14938c8d9a05cee0ff53/numpy-2.4.1-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:2302dc0224c1cbc49bb94f7064f3f923a971bfae45c33870dcbff63a2a550505", size = 16248316 }, + { url = "https://files.pythonhosted.org/packages/30/b4/e7f5ff8697274c9d0fa82398b6a372a27e5cef069b37df6355ccb1f1db1a/numpy-2.4.1-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:9171a42fcad32dcf3fa86f0a4faa5e9f8facefdb276f54b8b390d90447cff4e2", size = 18329884 }, + { url = "https://files.pythonhosted.org/packages/37/a4/b073f3e9d77f9aec8debe8ca7f9f6a09e888ad1ba7488f0c3b36a94c03ac/numpy-2.4.1-cp313-cp313t-win32.whl", hash = "sha256:382ad67d99ef49024f11d1ce5dcb5ad8432446e4246a4b014418ba3a1175a1f4", size = 6081138 }, + { url = "https://files.pythonhosted.org/packages/16/16/af42337b53844e67752a092481ab869c0523bc95c4e5c98e4dac4e9581ac/numpy-2.4.1-cp313-cp313t-win_amd64.whl", hash = "sha256:62fea415f83ad8fdb6c20840578e5fbaf5ddd65e0ec6c3c47eda0f69da172510", size = 12447478 }, + { url = "https://files.pythonhosted.org/packages/6c/f8/fa85b2eac68ec631d0b631abc448552cb17d39afd17ec53dcbcc3537681a/numpy-2.4.1-cp313-cp313t-win_arm64.whl", hash = "sha256:a7870e8c5fc11aef57d6fea4b4085e537a3a60ad2cdd14322ed531fdca68d261", size = 10382981 }, + { url = "https://files.pythonhosted.org/packages/1b/a7/ef08d25698e0e4b4efbad8d55251d20fe2a15f6d9aa7c9b30cd03c165e6f/numpy-2.4.1-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:3869ea1ee1a1edc16c29bbe3a2f2a4e515cc3a44d43903ad41e0cacdbaf733dc", size = 16652046 }, + { url = "https://files.pythonhosted.org/packages/8f/39/e378b3e3ca13477e5ac70293ec027c438d1927f18637e396fe90b1addd72/numpy-2.4.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:e867df947d427cdd7a60e3e271729090b0f0df80f5f10ab7dd436f40811699c3", size = 12378858 }, + { url = "https://files.pythonhosted.org/packages/c3/74/7ec6154f0006910ed1fdbb7591cf4432307033102b8a22041599935f8969/numpy-2.4.1-cp314-cp314-macosx_14_0_arm64.whl", hash = "sha256:e3bd2cb07841166420d2fa7146c96ce00cb3410664cbc1a6be028e456c4ee220", size = 5207417 }, + { url = "https://files.pythonhosted.org/packages/f7/b7/053ac11820d84e42f8feea5cb81cc4fcd1091499b45b1ed8c7415b1bf831/numpy-2.4.1-cp314-cp314-macosx_14_0_x86_64.whl", hash = "sha256:f0a90aba7d521e6954670550e561a4cb925713bd944445dbe9e729b71f6cabee", size = 6542643 }, + { url = "https://files.pythonhosted.org/packages/c0/c4/2e7908915c0e32ca636b92e4e4a3bdec4cb1e7eb0f8aedf1ed3c68a0d8cd/numpy-2.4.1-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5d558123217a83b2d1ba316b986e9248a1ed1971ad495963d555ccd75dcb1556", size = 14418963 }, + { url = "https://files.pythonhosted.org/packages/eb/c0/3ed5083d94e7ffd7c404e54619c088e11f2e1939a9544f5397f4adb1b8ba/numpy-2.4.1-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:2f44de05659b67d20499cbc96d49f2650769afcb398b79b324bb6e297bfe3844", size = 16363811 }, + { url = "https://files.pythonhosted.org/packages/0e/68/42b66f1852bf525050a67315a4fb94586ab7e9eaa541b1bef530fab0c5dd/numpy-2.4.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:69e7419c9012c4aaf695109564e3387f1259f001b4326dfa55907b098af082d3", size = 16197643 }, + { url = "https://files.pythonhosted.org/packages/d2/40/e8714fc933d85f82c6bfc7b998a0649ad9769a32f3494ba86598aaf18a48/numpy-2.4.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:2ffd257026eb1b34352e749d7cc1678b5eeec3e329ad8c9965a797e08ccba205", size = 18289601 }, + { url = "https://files.pythonhosted.org/packages/80/9a/0d44b468cad50315127e884802351723daca7cf1c98d102929468c81d439/numpy-2.4.1-cp314-cp314-win32.whl", hash = "sha256:727c6c3275ddefa0dc078524a85e064c057b4f4e71ca5ca29a19163c607be745", size = 6005722 }, + { url = "https://files.pythonhosted.org/packages/7e/bb/c6513edcce5a831810e2dddc0d3452ce84d208af92405a0c2e58fd8e7881/numpy-2.4.1-cp314-cp314-win_amd64.whl", hash = "sha256:7d5d7999df434a038d75a748275cd6c0094b0ecdb0837342b332a82defc4dc4d", size = 12438590 }, + { url = "https://files.pythonhosted.org/packages/e9/da/a598d5cb260780cf4d255102deba35c1d072dc028c4547832f45dd3323a8/numpy-2.4.1-cp314-cp314-win_arm64.whl", hash = "sha256:ce9ce141a505053b3c7bce3216071f3bf5c182b8b28930f14cd24d43932cd2df", size = 10596180 }, + { url = "https://files.pythonhosted.org/packages/de/bc/ea3f2c96fcb382311827231f911723aeff596364eb6e1b6d1d91128aa29b/numpy-2.4.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:4e53170557d37ae404bf8d542ca5b7c629d6efa1117dac6a83e394142ea0a43f", size = 12498774 }, + { url = "https://files.pythonhosted.org/packages/aa/ab/ef9d939fe4a812648c7a712610b2ca6140b0853c5efea361301006c02ae5/numpy-2.4.1-cp314-cp314t-macosx_14_0_arm64.whl", hash = "sha256:a73044b752f5d34d4232f25f18160a1cc418ea4507f5f11e299d8ac36875f8a0", size = 5327274 }, + { url = "https://files.pythonhosted.org/packages/bd/31/d381368e2a95c3b08b8cf7faac6004849e960f4a042d920337f71cef0cae/numpy-2.4.1-cp314-cp314t-macosx_14_0_x86_64.whl", hash = "sha256:fb1461c99de4d040666ca0444057b06541e5642f800b71c56e6ea92d6a853a0c", size = 6648306 }, + { url = "https://files.pythonhosted.org/packages/c8/e5/0989b44ade47430be6323d05c23207636d67d7362a1796ccbccac6773dd2/numpy-2.4.1-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:423797bdab2eeefbe608d7c1ec7b2b4fd3c58d51460f1ee26c7500a1d9c9ee93", size = 14464653 }, + { url = "https://files.pythonhosted.org/packages/10/a7/cfbe475c35371cae1358e61f20c5f075badc18c4797ab4354140e1d283cf/numpy-2.4.1-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:52b5f61bdb323b566b528899cc7db2ba5d1015bda7ea811a8bcf3c89c331fa42", size = 16405144 }, + { url = "https://files.pythonhosted.org/packages/f8/a3/0c63fe66b534888fa5177cc7cef061541064dbe2b4b60dcc60ffaf0d2157/numpy-2.4.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:42d7dd5fa36d16d52a84f821eb96031836fd405ee6955dd732f2023724d0aa01", size = 16247425 }, + { url = "https://files.pythonhosted.org/packages/6b/2b/55d980cfa2c93bd40ff4c290bf824d792bd41d2fe3487b07707559071760/numpy-2.4.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:e7b6b5e28bbd47b7532698e5db2fe1db693d84b58c254e4389d99a27bb9b8f6b", size = 18330053 }, + { url = "https://files.pythonhosted.org/packages/23/12/8b5fc6b9c487a09a7957188e0943c9ff08432c65e34567cabc1623b03a51/numpy-2.4.1-cp314-cp314t-win32.whl", hash = "sha256:5de60946f14ebe15e713a6f22850c2372fa72f4ff9a432ab44aa90edcadaa65a", size = 6152482 }, + { url = "https://files.pythonhosted.org/packages/00/a5/9f8ca5856b8940492fc24fbe13c1bc34d65ddf4079097cf9e53164d094e1/numpy-2.4.1-cp314-cp314t-win_amd64.whl", hash = "sha256:8f085da926c0d491ffff3096f91078cc97ea67e7e6b65e490bc8dcda65663be2", size = 12627117 }, + { url = "https://files.pythonhosted.org/packages/ad/0d/eca3d962f9eef265f01a8e0d20085c6dd1f443cbffc11b6dede81fd82356/numpy-2.4.1-cp314-cp314t-win_arm64.whl", hash = "sha256:6436cffb4f2bf26c974344439439c95e152c9a527013f26b3577be6c2ca64295", size = 10667121 }, + { url = "https://files.pythonhosted.org/packages/1e/48/d86f97919e79314a1cdee4c832178763e6e98e623e123d0bada19e92c15a/numpy-2.4.1-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:8ad35f20be147a204e28b6a0575fbf3540c5e5f802634d4258d55b1ff5facce1", size = 16822202 }, + { url = "https://files.pythonhosted.org/packages/51/e9/1e62a7f77e0f37dcfb0ad6a9744e65df00242b6ea37dfafb55debcbf5b55/numpy-2.4.1-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:8097529164c0f3e32bb89412a0905d9100bf434d9692d9fc275e18dcf53c9344", size = 12569985 }, + { url = "https://files.pythonhosted.org/packages/c7/7e/914d54f0c801342306fdcdce3e994a56476f1b818c46c47fc21ae968088c/numpy-2.4.1-pp311-pypy311_pp73-macosx_14_0_arm64.whl", hash = "sha256:ea66d2b41ca4a1630aae5507ee0a71647d3124d1741980138aa8f28f44dac36e", size = 5398484 }, + { url = "https://files.pythonhosted.org/packages/1c/d8/9570b68584e293a33474e7b5a77ca404f1dcc655e40050a600dee81d27fb/numpy-2.4.1-pp311-pypy311_pp73-macosx_14_0_x86_64.whl", hash = "sha256:d3f8f0df9f4b8be57b3bf74a1d087fec68f927a2fab68231fdb442bf2c12e426", size = 6713216 }, + { url = "https://files.pythonhosted.org/packages/33/9b/9dd6e2db8d49eb24f86acaaa5258e5f4c8ed38209a4ee9de2d1a0ca25045/numpy-2.4.1-pp311-pypy311_pp73-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:2023ef86243690c2791fd6353e5b4848eedaa88ca8a2d129f462049f6d484696", size = 14538937 }, + { url = "https://files.pythonhosted.org/packages/53/87/d5bd995b0f798a37105b876350d346eea5838bd8f77ea3d7a48392f3812b/numpy-2.4.1-pp311-pypy311_pp73-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8361ea4220d763e54cff2fbe7d8c93526b744f7cd9ddab47afeff7e14e8503be", size = 16479830 }, + { url = "https://files.pythonhosted.org/packages/5b/c7/b801bf98514b6ae6475e941ac05c58e6411dd863ea92916bfd6d510b08c1/numpy-2.4.1-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:4f1b68ff47680c2925f8063402a693ede215f0257f02596b1318ecdfb1d79e33", size = 12492579 }, +] + +[[package]] +name = "packaging" +version = "25.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/a1/d4/1fc4078c65507b51b96ca8f8c3ba19e6a61c8253c72794544580a7b6c24d/packaging-25.0.tar.gz", hash = "sha256:d443872c98d677bf60f6a1f2f8c1cb748e8fe762d2bf9d3148b5599295b0fc4f", size = 165727 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/20/12/38679034af332785aac8774540895e234f4d07f7545804097de4b666afd8/packaging-25.0-py3-none-any.whl", hash = "sha256:29572ef2b1f17581046b3a2227d5c611fb25ec70ca1ba8554b24b0e69331a484", size = 66469 }, +] + +[[package]] +name = "pandas" +version = "2.3.3" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "numpy" }, + { name = "python-dateutil" }, + { name = "pytz" }, + { name = "tzdata" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/33/01/d40b85317f86cf08d853a4f495195c73815fdf205eef3993821720274518/pandas-2.3.3.tar.gz", hash = "sha256:e05e1af93b977f7eafa636d043f9f94c7ee3ac81af99c13508215942e64c993b", size = 4495223 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c1/fa/7ac648108144a095b4fb6aa3de1954689f7af60a14cf25583f4960ecb878/pandas-2.3.3-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:602b8615ebcc4a0c1751e71840428ddebeb142ec02c786e8ad6b1ce3c8dec523", size = 11578790 }, + { url = "https://files.pythonhosted.org/packages/9b/35/74442388c6cf008882d4d4bdfc4109be87e9b8b7ccd097ad1e7f006e2e95/pandas-2.3.3-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:8fe25fc7b623b0ef6b5009149627e34d2a4657e880948ec3c840e9402e5c1b45", size = 10833831 }, + { url = "https://files.pythonhosted.org/packages/fe/e4/de154cbfeee13383ad58d23017da99390b91d73f8c11856f2095e813201b/pandas-2.3.3-cp311-cp311-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b468d3dad6ff947df92dcb32ede5b7bd41a9b3cceef0a30ed925f6d01fb8fa66", size = 12199267 }, + { url = "https://files.pythonhosted.org/packages/bf/c9/63f8d545568d9ab91476b1818b4741f521646cbdd151c6efebf40d6de6f7/pandas-2.3.3-cp311-cp311-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b98560e98cb334799c0b07ca7967ac361a47326e9b4e5a7dfb5ab2b1c9d35a1b", size = 12789281 }, + { url = "https://files.pythonhosted.org/packages/f2/00/a5ac8c7a0e67fd1a6059e40aa08fa1c52cc00709077d2300e210c3ce0322/pandas-2.3.3-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:1d37b5848ba49824e5c30bedb9c830ab9b7751fd049bc7914533e01c65f79791", size = 13240453 }, + { url = "https://files.pythonhosted.org/packages/27/4d/5c23a5bc7bd209231618dd9e606ce076272c9bc4f12023a70e03a86b4067/pandas-2.3.3-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:db4301b2d1f926ae677a751eb2bd0e8c5f5319c9cb3f88b0becbbb0b07b34151", size = 13890361 }, + { url = "https://files.pythonhosted.org/packages/8e/59/712db1d7040520de7a4965df15b774348980e6df45c129b8c64d0dbe74ef/pandas-2.3.3-cp311-cp311-win_amd64.whl", hash = "sha256:f086f6fe114e19d92014a1966f43a3e62285109afe874f067f5abbdcbb10e59c", size = 11348702 }, + { url = "https://files.pythonhosted.org/packages/9c/fb/231d89e8637c808b997d172b18e9d4a4bc7bf31296196c260526055d1ea0/pandas-2.3.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:6d21f6d74eb1725c2efaa71a2bfc661a0689579b58e9c0ca58a739ff0b002b53", size = 11597846 }, + { url = "https://files.pythonhosted.org/packages/5c/bd/bf8064d9cfa214294356c2d6702b716d3cf3bb24be59287a6a21e24cae6b/pandas-2.3.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:3fd2f887589c7aa868e02632612ba39acb0b8948faf5cc58f0850e165bd46f35", size = 10729618 }, + { url = "https://files.pythonhosted.org/packages/57/56/cf2dbe1a3f5271370669475ead12ce77c61726ffd19a35546e31aa8edf4e/pandas-2.3.3-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ecaf1e12bdc03c86ad4a7ea848d66c685cb6851d807a26aa245ca3d2017a1908", size = 11737212 }, + { url = "https://files.pythonhosted.org/packages/e5/63/cd7d615331b328e287d8233ba9fdf191a9c2d11b6af0c7a59cfcec23de68/pandas-2.3.3-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b3d11d2fda7eb164ef27ffc14b4fcab16a80e1ce67e9f57e19ec0afaf715ba89", size = 12362693 }, + { url = "https://files.pythonhosted.org/packages/a6/de/8b1895b107277d52f2b42d3a6806e69cfef0d5cf1d0ba343470b9d8e0a04/pandas-2.3.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:a68e15f780eddf2b07d242e17a04aa187a7ee12b40b930bfdd78070556550e98", size = 12771002 }, + { url = "https://files.pythonhosted.org/packages/87/21/84072af3187a677c5893b170ba2c8fbe450a6ff911234916da889b698220/pandas-2.3.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:371a4ab48e950033bcf52b6527eccb564f52dc826c02afd9a1bc0ab731bba084", size = 13450971 }, + { url = "https://files.pythonhosted.org/packages/86/41/585a168330ff063014880a80d744219dbf1dd7a1c706e75ab3425a987384/pandas-2.3.3-cp312-cp312-win_amd64.whl", hash = "sha256:a16dcec078a01eeef8ee61bf64074b4e524a2a3f4b3be9326420cabe59c4778b", size = 10992722 }, + { url = "https://files.pythonhosted.org/packages/cd/4b/18b035ee18f97c1040d94debd8f2e737000ad70ccc8f5513f4eefad75f4b/pandas-2.3.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:56851a737e3470de7fa88e6131f41281ed440d29a9268dcbf0002da5ac366713", size = 11544671 }, + { url = "https://files.pythonhosted.org/packages/31/94/72fac03573102779920099bcac1c3b05975c2cb5f01eac609faf34bed1ca/pandas-2.3.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:bdcd9d1167f4885211e401b3036c0c8d9e274eee67ea8d0758a256d60704cfe8", size = 10680807 }, + { url = "https://files.pythonhosted.org/packages/16/87/9472cf4a487d848476865321de18cc8c920b8cab98453ab79dbbc98db63a/pandas-2.3.3-cp313-cp313-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e32e7cc9af0f1cc15548288a51a3b681cc2a219faa838e995f7dc53dbab1062d", size = 11709872 }, + { url = "https://files.pythonhosted.org/packages/15/07/284f757f63f8a8d69ed4472bfd85122bd086e637bf4ed09de572d575a693/pandas-2.3.3-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:318d77e0e42a628c04dc56bcef4b40de67918f7041c2b061af1da41dcff670ac", size = 12306371 }, + { url = "https://files.pythonhosted.org/packages/33/81/a3afc88fca4aa925804a27d2676d22dcd2031c2ebe08aabd0ae55b9ff282/pandas-2.3.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:4e0a175408804d566144e170d0476b15d78458795bb18f1304fb94160cabf40c", size = 12765333 }, + { url = "https://files.pythonhosted.org/packages/8d/0f/b4d4ae743a83742f1153464cf1a8ecfafc3ac59722a0b5c8602310cb7158/pandas-2.3.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:93c2d9ab0fc11822b5eece72ec9587e172f63cff87c00b062f6e37448ced4493", size = 13418120 }, + { url = "https://files.pythonhosted.org/packages/4f/c7/e54682c96a895d0c808453269e0b5928a07a127a15704fedb643e9b0a4c8/pandas-2.3.3-cp313-cp313-win_amd64.whl", hash = "sha256:f8bfc0e12dc78f777f323f55c58649591b2cd0c43534e8355c51d3fede5f4dee", size = 10993991 }, + { url = "https://files.pythonhosted.org/packages/f9/ca/3f8d4f49740799189e1395812f3bf23b5e8fc7c190827d55a610da72ce55/pandas-2.3.3-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:75ea25f9529fdec2d2e93a42c523962261e567d250b0013b16210e1d40d7c2e5", size = 12048227 }, + { url = "https://files.pythonhosted.org/packages/0e/5a/f43efec3e8c0cc92c4663ccad372dbdff72b60bdb56b2749f04aa1d07d7e/pandas-2.3.3-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:74ecdf1d301e812db96a465a525952f4dde225fdb6d8e5a521d47e1f42041e21", size = 11411056 }, + { url = "https://files.pythonhosted.org/packages/46/b1/85331edfc591208c9d1a63a06baa67b21d332e63b7a591a5ba42a10bb507/pandas-2.3.3-cp313-cp313t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6435cb949cb34ec11cc9860246ccb2fdc9ecd742c12d3304989017d53f039a78", size = 11645189 }, + { url = "https://files.pythonhosted.org/packages/44/23/78d645adc35d94d1ac4f2a3c4112ab6f5b8999f4898b8cdf01252f8df4a9/pandas-2.3.3-cp313-cp313t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:900f47d8f20860de523a1ac881c4c36d65efcb2eb850e6948140fa781736e110", size = 12121912 }, + { url = "https://files.pythonhosted.org/packages/53/da/d10013df5e6aaef6b425aa0c32e1fc1f3e431e4bcabd420517dceadce354/pandas-2.3.3-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:a45c765238e2ed7d7c608fc5bc4a6f88b642f2f01e70c0c23d2224dd21829d86", size = 12712160 }, + { url = "https://files.pythonhosted.org/packages/bd/17/e756653095a083d8a37cbd816cb87148debcfcd920129b25f99dd8d04271/pandas-2.3.3-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:c4fc4c21971a1a9f4bdb4c73978c7f7256caa3e62b323f70d6cb80db583350bc", size = 13199233 }, + { url = "https://files.pythonhosted.org/packages/04/fd/74903979833db8390b73b3a8a7d30d146d710bd32703724dd9083950386f/pandas-2.3.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:ee15f284898e7b246df8087fc82b87b01686f98ee67d85a17b7ab44143a3a9a0", size = 11540635 }, + { url = "https://files.pythonhosted.org/packages/21/00/266d6b357ad5e6d3ad55093a7e8efc7dd245f5a842b584db9f30b0f0a287/pandas-2.3.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:1611aedd912e1ff81ff41c745822980c49ce4a7907537be8692c8dbc31924593", size = 10759079 }, + { url = "https://files.pythonhosted.org/packages/ca/05/d01ef80a7a3a12b2f8bbf16daba1e17c98a2f039cbc8e2f77a2c5a63d382/pandas-2.3.3-cp314-cp314-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6d2cefc361461662ac48810cb14365a365ce864afe85ef1f447ff5a1e99ea81c", size = 11814049 }, + { url = "https://files.pythonhosted.org/packages/15/b2/0e62f78c0c5ba7e3d2c5945a82456f4fac76c480940f805e0b97fcbc2f65/pandas-2.3.3-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ee67acbbf05014ea6c763beb097e03cd629961c8a632075eeb34247120abcb4b", size = 12332638 }, + { url = "https://files.pythonhosted.org/packages/c5/33/dd70400631b62b9b29c3c93d2feee1d0964dc2bae2e5ad7a6c73a7f25325/pandas-2.3.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:c46467899aaa4da076d5abc11084634e2d197e9460643dd455ac3db5856b24d6", size = 12886834 }, + { url = "https://files.pythonhosted.org/packages/d3/18/b5d48f55821228d0d2692b34fd5034bb185e854bdb592e9c640f6290e012/pandas-2.3.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:6253c72c6a1d990a410bc7de641d34053364ef8bcd3126f7e7450125887dffe3", size = 13409925 }, + { url = "https://files.pythonhosted.org/packages/a6/3d/124ac75fcd0ecc09b8fdccb0246ef65e35b012030defb0e0eba2cbbbe948/pandas-2.3.3-cp314-cp314-win_amd64.whl", hash = "sha256:1b07204a219b3b7350abaae088f451860223a52cfb8a6c53358e7948735158e5", size = 11109071 }, + { url = "https://files.pythonhosted.org/packages/89/9c/0e21c895c38a157e0faa1fb64587a9226d6dd46452cac4532d80c3c4a244/pandas-2.3.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:2462b1a365b6109d275250baaae7b760fd25c726aaca0054649286bcfbb3e8ec", size = 12048504 }, + { url = "https://files.pythonhosted.org/packages/d7/82/b69a1c95df796858777b68fbe6a81d37443a33319761d7c652ce77797475/pandas-2.3.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:0242fe9a49aa8b4d78a4fa03acb397a58833ef6199e9aa40a95f027bb3a1b6e7", size = 11410702 }, + { url = "https://files.pythonhosted.org/packages/f9/88/702bde3ba0a94b8c73a0181e05144b10f13f29ebfc2150c3a79062a8195d/pandas-2.3.3-cp314-cp314t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a21d830e78df0a515db2b3d2f5570610f5e6bd2e27749770e8bb7b524b89b450", size = 11634535 }, + { url = "https://files.pythonhosted.org/packages/a4/1e/1bac1a839d12e6a82ec6cb40cda2edde64a2013a66963293696bbf31fbbb/pandas-2.3.3-cp314-cp314t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:2e3ebdb170b5ef78f19bfb71b0dc5dc58775032361fa188e814959b74d726dd5", size = 12121582 }, + { url = "https://files.pythonhosted.org/packages/44/91/483de934193e12a3b1d6ae7c8645d083ff88dec75f46e827562f1e4b4da6/pandas-2.3.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:d051c0e065b94b7a3cea50eb1ec32e912cd96dba41647eb24104b6c6c14c5788", size = 12699963 }, + { url = "https://files.pythonhosted.org/packages/70/44/5191d2e4026f86a2a109053e194d3ba7a31a2d10a9c2348368c63ed4e85a/pandas-2.3.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:3869faf4bd07b3b66a9f462417d0ca3a9df29a9f6abd5d0d0dbab15dac7abe87", size = 13202175 }, +] + +[[package]] +name = "pillow" +version = "12.1.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d0/02/d52c733a2452ef1ffcc123b68e6606d07276b0e358db70eabad7e40042b7/pillow-12.1.0.tar.gz", hash = "sha256:5c5ae0a06e9ea030ab786b0251b32c7e4ce10e58d983c0d5c56029455180b5b9", size = 46977283 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/43/c4/bf8328039de6cc22182c3ef007a2abfbbdab153661c0a9aa78af8d706391/pillow-12.1.0-cp311-cp311-macosx_10_10_x86_64.whl", hash = "sha256:a83e0850cb8f5ac975291ebfc4170ba481f41a28065277f7f735c202cd8e0af3", size = 5304057 }, + { url = "https://files.pythonhosted.org/packages/43/06/7264c0597e676104cc22ca73ee48f752767cd4b1fe084662620b17e10120/pillow-12.1.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:b6e53e82ec2db0717eabb276aa56cf4e500c9a7cec2c2e189b55c24f65a3e8c0", size = 4657811 }, + { url = "https://files.pythonhosted.org/packages/72/64/f9189e44474610daf83da31145fa56710b627b5c4c0b9c235e34058f6b31/pillow-12.1.0-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:40a8e3b9e8773876d6e30daed22f016509e3987bab61b3b7fe309d7019a87451", size = 6232243 }, + { url = "https://files.pythonhosted.org/packages/ef/30/0df458009be6a4caca4ca2c52975e6275c387d4e5c95544e34138b41dc86/pillow-12.1.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:800429ac32c9b72909c671aaf17ecd13110f823ddb7db4dfef412a5587c2c24e", size = 8037872 }, + { url = "https://files.pythonhosted.org/packages/e4/86/95845d4eda4f4f9557e25381d70876aa213560243ac1a6d619c46caaedd9/pillow-12.1.0-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0b022eaaf709541b391ee069f0022ee5b36c709df71986e3f7be312e46f42c84", size = 6345398 }, + { url = "https://files.pythonhosted.org/packages/5c/1f/8e66ab9be3aaf1435bc03edd1ebdf58ffcd17f7349c1d970cafe87af27d9/pillow-12.1.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1f345e7bc9d7f368887c712aa5054558bad44d2a301ddf9248599f4161abc7c0", size = 7034667 }, + { url = "https://files.pythonhosted.org/packages/f9/f6/683b83cb9b1db1fb52b87951b1c0b99bdcfceaa75febf11406c19f82cb5e/pillow-12.1.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:d70347c8a5b7ccd803ec0c85c8709f036e6348f1e6a5bf048ecd9c64d3550b8b", size = 6458743 }, + { url = "https://files.pythonhosted.org/packages/9a/7d/de833d63622538c1d58ce5395e7c6cb7e7dce80decdd8bde4a484e095d9f/pillow-12.1.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:1fcc52d86ce7a34fd17cb04e87cfdb164648a3662a6f20565910a99653d66c18", size = 7159342 }, + { url = "https://files.pythonhosted.org/packages/8c/40/50d86571c9e5868c42b81fe7da0c76ca26373f3b95a8dd675425f4a92ec1/pillow-12.1.0-cp311-cp311-win32.whl", hash = "sha256:3ffaa2f0659e2f740473bcf03c702c39a8d4b2b7ffc629052028764324842c64", size = 6328655 }, + { url = "https://files.pythonhosted.org/packages/6c/af/b1d7e301c4cd26cd45d4af884d9ee9b6fab893b0ad2450d4746d74a6968c/pillow-12.1.0-cp311-cp311-win_amd64.whl", hash = "sha256:806f3987ffe10e867bab0ddad45df1148a2b98221798457fa097ad85d6e8bc75", size = 7031469 }, + { url = "https://files.pythonhosted.org/packages/48/36/d5716586d887fb2a810a4a61518a327a1e21c8b7134c89283af272efe84b/pillow-12.1.0-cp311-cp311-win_arm64.whl", hash = "sha256:9f5fefaca968e700ad1a4a9de98bf0869a94e397fe3524c4c9450c1445252304", size = 2452515 }, + { url = "https://files.pythonhosted.org/packages/20/31/dc53fe21a2f2996e1b7d92bf671cdb157079385183ef7c1ae08b485db510/pillow-12.1.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:a332ac4ccb84b6dde65dbace8431f3af08874bf9770719d32a635c4ef411b18b", size = 5262642 }, + { url = "https://files.pythonhosted.org/packages/ab/c1/10e45ac9cc79419cedf5121b42dcca5a50ad2b601fa080f58c22fb27626e/pillow-12.1.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:907bfa8a9cb790748a9aa4513e37c88c59660da3bcfffbd24a7d9e6abf224551", size = 4657464 }, + { url = "https://files.pythonhosted.org/packages/ad/26/7b82c0ab7ef40ebede7a97c72d473bda5950f609f8e0c77b04af574a0ddb/pillow-12.1.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:efdc140e7b63b8f739d09a99033aa430accce485ff78e6d311973a67b6bf3208", size = 6234878 }, + { url = "https://files.pythonhosted.org/packages/76/25/27abc9792615b5e886ca9411ba6637b675f1b77af3104710ac7353fe5605/pillow-12.1.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:bef9768cab184e7ae6e559c032e95ba8d07b3023c289f79a2bd36e8bf85605a5", size = 8044868 }, + { url = "https://files.pythonhosted.org/packages/0a/ea/f200a4c36d836100e7bc738fc48cd963d3ba6372ebc8298a889e0cfc3359/pillow-12.1.0-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:742aea052cf5ab5034a53c3846165bc3ce88d7c38e954120db0ab867ca242661", size = 6349468 }, + { url = "https://files.pythonhosted.org/packages/11/8f/48d0b77ab2200374c66d344459b8958c86693be99526450e7aee714e03e4/pillow-12.1.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a6dfc2af5b082b635af6e08e0d1f9f1c4e04d17d4e2ca0ef96131e85eda6eb17", size = 7041518 }, + { url = "https://files.pythonhosted.org/packages/1d/23/c281182eb986b5d31f0a76d2a2c8cd41722d6fb8ed07521e802f9bba52de/pillow-12.1.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:609e89d9f90b581c8d16358c9087df76024cf058fa693dd3e1e1620823f39670", size = 6462829 }, + { url = "https://files.pythonhosted.org/packages/25/ef/7018273e0faac099d7b00982abdcc39142ae6f3bd9ceb06de09779c4a9d6/pillow-12.1.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:43b4899cfd091a9693a1278c4982f3e50f7fb7cff5153b05174b4afc9593b616", size = 7166756 }, + { url = "https://files.pythonhosted.org/packages/8f/c8/993d4b7ab2e341fe02ceef9576afcf5830cdec640be2ac5bee1820d693d4/pillow-12.1.0-cp312-cp312-win32.whl", hash = "sha256:aa0c9cc0b82b14766a99fbe6084409972266e82f459821cd26997a488a7261a7", size = 6328770 }, + { url = "https://files.pythonhosted.org/packages/a7/87/90b358775a3f02765d87655237229ba64a997b87efa8ccaca7dd3e36e7a7/pillow-12.1.0-cp312-cp312-win_amd64.whl", hash = "sha256:d70534cea9e7966169ad29a903b99fc507e932069a881d0965a1a84bb57f6c6d", size = 7033406 }, + { url = "https://files.pythonhosted.org/packages/5d/cf/881b457eccacac9e5b2ddd97d5071fb6d668307c57cbf4e3b5278e06e536/pillow-12.1.0-cp312-cp312-win_arm64.whl", hash = "sha256:65b80c1ee7e14a87d6a068dd3b0aea268ffcabfe0498d38661b00c5b4b22e74c", size = 2452612 }, + { url = "https://files.pythonhosted.org/packages/dd/c7/2530a4aa28248623e9d7f27316b42e27c32ec410f695929696f2e0e4a778/pillow-12.1.0-cp313-cp313-ios_13_0_arm64_iphoneos.whl", hash = "sha256:7b5dd7cbae20285cdb597b10eb5a2c13aa9de6cde9bb64a3c1317427b1db1ae1", size = 4062543 }, + { url = "https://files.pythonhosted.org/packages/8f/1f/40b8eae823dc1519b87d53c30ed9ef085506b05281d313031755c1705f73/pillow-12.1.0-cp313-cp313-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:29a4cef9cb672363926f0470afc516dbf7305a14d8c54f7abbb5c199cd8f8179", size = 4138373 }, + { url = "https://files.pythonhosted.org/packages/d4/77/6fa60634cf06e52139fd0e89e5bbf055e8166c691c42fb162818b7fda31d/pillow-12.1.0-cp313-cp313-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:681088909d7e8fa9e31b9799aaa59ba5234c58e5e4f1951b4c4d1082a2e980e0", size = 3601241 }, + { url = "https://files.pythonhosted.org/packages/4f/bf/28ab865de622e14b747f0cd7877510848252d950e43002e224fb1c9ababf/pillow-12.1.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:983976c2ab753166dc66d36af6e8ec15bb511e4a25856e2227e5f7e00a160587", size = 5262410 }, + { url = "https://files.pythonhosted.org/packages/1c/34/583420a1b55e715937a85bd48c5c0991598247a1fd2eb5423188e765ea02/pillow-12.1.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:db44d5c160a90df2d24a24760bbd37607d53da0b34fb546c4c232af7192298ac", size = 4657312 }, + { url = "https://files.pythonhosted.org/packages/1d/fd/f5a0896839762885b3376ff04878f86ab2b097c2f9a9cdccf4eda8ba8dc0/pillow-12.1.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:6b7a9d1db5dad90e2991645874f708e87d9a3c370c243c2d7684d28f7e133e6b", size = 6232605 }, + { url = "https://files.pythonhosted.org/packages/98/aa/938a09d127ac1e70e6ed467bd03834350b33ef646b31edb7452d5de43792/pillow-12.1.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:6258f3260986990ba2fa8a874f8b6e808cf5abb51a94015ca3dc3c68aa4f30ea", size = 8041617 }, + { url = "https://files.pythonhosted.org/packages/17/e8/538b24cb426ac0186e03f80f78bc8dc7246c667f58b540bdd57c71c9f79d/pillow-12.1.0-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e115c15e3bc727b1ca3e641a909f77f8ca72a64fff150f666fcc85e57701c26c", size = 6346509 }, + { url = "https://files.pythonhosted.org/packages/01/9a/632e58ec89a32738cabfd9ec418f0e9898a2b4719afc581f07c04a05e3c9/pillow-12.1.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6741e6f3074a35e47c77b23a4e4f2d90db3ed905cb1c5e6e0d49bff2045632bc", size = 7038117 }, + { url = "https://files.pythonhosted.org/packages/c7/a2/d40308cf86eada842ca1f3ffa45d0ca0df7e4ab33c83f81e73f5eaed136d/pillow-12.1.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:935b9d1aed48fcfb3f838caac506f38e29621b44ccc4f8a64d575cb1b2a88644", size = 6460151 }, + { url = "https://files.pythonhosted.org/packages/f1/88/f5b058ad6453a085c5266660a1417bdad590199da1b32fb4efcff9d33b05/pillow-12.1.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:5fee4c04aad8932da9f8f710af2c1a15a83582cfb884152a9caa79d4efcdbf9c", size = 7164534 }, + { url = "https://files.pythonhosted.org/packages/19/ce/c17334caea1db789163b5d855a5735e47995b0b5dc8745e9a3605d5f24c0/pillow-12.1.0-cp313-cp313-win32.whl", hash = "sha256:a786bf667724d84aa29b5db1c61b7bfdde380202aaca12c3461afd6b71743171", size = 6332551 }, + { url = "https://files.pythonhosted.org/packages/e5/07/74a9d941fa45c90a0d9465098fe1ec85de3e2afbdc15cc4766622d516056/pillow-12.1.0-cp313-cp313-win_amd64.whl", hash = "sha256:461f9dfdafa394c59cd6d818bdfdbab4028b83b02caadaff0ffd433faf4c9a7a", size = 7040087 }, + { url = "https://files.pythonhosted.org/packages/88/09/c99950c075a0e9053d8e880595926302575bc742b1b47fe1bbcc8d388d50/pillow-12.1.0-cp313-cp313-win_arm64.whl", hash = "sha256:9212d6b86917a2300669511ed094a9406888362e085f2431a7da985a6b124f45", size = 2452470 }, + { url = "https://files.pythonhosted.org/packages/b5/ba/970b7d85ba01f348dee4d65412476321d40ee04dcb51cd3735b9dc94eb58/pillow-12.1.0-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:00162e9ca6d22b7c3ee8e61faa3c3253cd19b6a37f126cad04f2f88b306f557d", size = 5264816 }, + { url = "https://files.pythonhosted.org/packages/10/60/650f2fb55fdba7a510d836202aa52f0baac633e50ab1cf18415d332188fb/pillow-12.1.0-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:7d6daa89a00b58c37cb1747ec9fb7ac3bc5ffd5949f5888657dfddde6d1312e0", size = 4660472 }, + { url = "https://files.pythonhosted.org/packages/2b/c0/5273a99478956a099d533c4f46cbaa19fd69d606624f4334b85e50987a08/pillow-12.1.0-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:e2479c7f02f9d505682dc47df8c0ea1fc5e264c4d1629a5d63fe3e2334b89554", size = 6268974 }, + { url = "https://files.pythonhosted.org/packages/b4/26/0bf714bc2e73d5267887d47931d53c4ceeceea6978148ed2ab2a4e6463c4/pillow-12.1.0-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:f188d580bd870cda1e15183790d1cc2fa78f666e76077d103edf048eed9c356e", size = 8073070 }, + { url = "https://files.pythonhosted.org/packages/43/cf/1ea826200de111a9d65724c54f927f3111dc5ae297f294b370a670c17786/pillow-12.1.0-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0fde7ec5538ab5095cc02df38ee99b0443ff0e1c847a045554cf5f9af1f4aa82", size = 6380176 }, + { url = "https://files.pythonhosted.org/packages/03/e0/7938dd2b2013373fd85d96e0f38d62b7a5a262af21ac274250c7ca7847c9/pillow-12.1.0-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0ed07dca4a8464bada6139ab38f5382f83e5f111698caf3191cb8dbf27d908b4", size = 7067061 }, + { url = "https://files.pythonhosted.org/packages/86/ad/a2aa97d37272a929a98437a8c0ac37b3cf012f4f8721e1bd5154699b2518/pillow-12.1.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:f45bd71d1fa5e5749587613037b172e0b3b23159d1c00ef2fc920da6f470e6f0", size = 6491824 }, + { url = "https://files.pythonhosted.org/packages/a4/44/80e46611b288d51b115826f136fb3465653c28f491068a72d3da49b54cd4/pillow-12.1.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:277518bf4fe74aa91489e1b20577473b19ee70fb97c374aa50830b279f25841b", size = 7190911 }, + { url = "https://files.pythonhosted.org/packages/86/77/eacc62356b4cf81abe99ff9dbc7402750044aed02cfd6a503f7c6fc11f3e/pillow-12.1.0-cp313-cp313t-win32.whl", hash = "sha256:7315f9137087c4e0ee73a761b163fc9aa3b19f5f606a7fc08d83fd3e4379af65", size = 6336445 }, + { url = "https://files.pythonhosted.org/packages/e7/3c/57d81d0b74d218706dafccb87a87ea44262c43eef98eb3b164fd000e0491/pillow-12.1.0-cp313-cp313t-win_amd64.whl", hash = "sha256:0ddedfaa8b5f0b4ffbc2fa87b556dc59f6bb4ecb14a53b33f9189713ae8053c0", size = 7045354 }, + { url = "https://files.pythonhosted.org/packages/ac/82/8b9b97bba2e3576a340f93b044a3a3a09841170ab4c1eb0d5c93469fd32f/pillow-12.1.0-cp313-cp313t-win_arm64.whl", hash = "sha256:80941e6d573197a0c28f394753de529bb436b1ca990ed6e765cf42426abc39f8", size = 2454547 }, + { url = "https://files.pythonhosted.org/packages/8c/87/bdf971d8bbcf80a348cc3bacfcb239f5882100fe80534b0ce67a784181d8/pillow-12.1.0-cp314-cp314-ios_13_0_arm64_iphoneos.whl", hash = "sha256:5cb7bc1966d031aec37ddb9dcf15c2da5b2e9f7cc3ca7c54473a20a927e1eb91", size = 4062533 }, + { url = "https://files.pythonhosted.org/packages/ff/4f/5eb37a681c68d605eb7034c004875c81f86ec9ef51f5be4a63eadd58859a/pillow-12.1.0-cp314-cp314-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:97e9993d5ed946aba26baf9c1e8cf18adbab584b99f452ee72f7ee8acb882796", size = 4138546 }, + { url = "https://files.pythonhosted.org/packages/11/6d/19a95acb2edbace40dcd582d077b991646b7083c41b98da4ed7555b59733/pillow-12.1.0-cp314-cp314-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:414b9a78e14ffeb98128863314e62c3f24b8a86081066625700b7985b3f529bd", size = 3601163 }, + { url = "https://files.pythonhosted.org/packages/fc/36/2b8138e51cb42e4cc39c3297713455548be855a50558c3ac2beebdc251dd/pillow-12.1.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:e6bdb408f7c9dd2a5ff2b14a3b0bb6d4deb29fb9961e6eb3ae2031ae9a5cec13", size = 5266086 }, + { url = "https://files.pythonhosted.org/packages/53/4b/649056e4d22e1caa90816bf99cef0884aed607ed38075bd75f091a607a38/pillow-12.1.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:3413c2ae377550f5487991d444428f1a8ae92784aac79caa8b1e3b89b175f77e", size = 4657344 }, + { url = "https://files.pythonhosted.org/packages/6c/6b/c5742cea0f1ade0cd61485dc3d81f05261fc2276f537fbdc00802de56779/pillow-12.1.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:e5dcbe95016e88437ecf33544ba5db21ef1b8dd6e1b434a2cb2a3d605299e643", size = 6232114 }, + { url = "https://files.pythonhosted.org/packages/bf/8f/9f521268ce22d63991601aafd3d48d5ff7280a246a1ef62d626d67b44064/pillow-12.1.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:d0a7735df32ccbcc98b98a1ac785cc4b19b580be1bdf0aeb5c03223220ea09d5", size = 8042708 }, + { url = "https://files.pythonhosted.org/packages/1a/eb/257f38542893f021502a1bbe0c2e883c90b5cff26cc33b1584a841a06d30/pillow-12.1.0-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0c27407a2d1b96774cbc4a7594129cc027339fd800cd081e44497722ea1179de", size = 6347762 }, + { url = "https://files.pythonhosted.org/packages/c4/5a/8ba375025701c09b309e8d5163c5a4ce0102fa86bbf8800eb0d7ac87bc51/pillow-12.1.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:15c794d74303828eaa957ff8070846d0efe8c630901a1c753fdc63850e19ecd9", size = 7039265 }, + { url = "https://files.pythonhosted.org/packages/cf/dc/cf5e4cdb3db533f539e88a7bbf9f190c64ab8a08a9bc7a4ccf55067872e4/pillow-12.1.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:c990547452ee2800d8506c4150280757f88532f3de2a58e3022e9b179107862a", size = 6462341 }, + { url = "https://files.pythonhosted.org/packages/d0/47/0291a25ac9550677e22eda48510cfc4fa4b2ef0396448b7fbdc0a6946309/pillow-12.1.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:b63e13dd27da389ed9475b3d28510f0f954bca0041e8e551b2a4eb1eab56a39a", size = 7165395 }, + { url = "https://files.pythonhosted.org/packages/4f/4c/e005a59393ec4d9416be06e6b45820403bb946a778e39ecec62f5b2b991e/pillow-12.1.0-cp314-cp314-win32.whl", hash = "sha256:1a949604f73eb07a8adab38c4fe50791f9919344398bdc8ac6b307f755fc7030", size = 6431413 }, + { url = "https://files.pythonhosted.org/packages/1c/af/f23697f587ac5f9095d67e31b81c95c0249cd461a9798a061ed6709b09b5/pillow-12.1.0-cp314-cp314-win_amd64.whl", hash = "sha256:4f9f6a650743f0ddee5593ac9e954ba1bdbc5e150bc066586d4f26127853ab94", size = 7176779 }, + { url = "https://files.pythonhosted.org/packages/b3/36/6a51abf8599232f3e9afbd16d52829376a68909fe14efe29084445db4b73/pillow-12.1.0-cp314-cp314-win_arm64.whl", hash = "sha256:808b99604f7873c800c4840f55ff389936ef1948e4e87645eaf3fccbc8477ac4", size = 2543105 }, + { url = "https://files.pythonhosted.org/packages/82/54/2e1dd20c8749ff225080d6ba465a0cab4387f5db0d1c5fb1439e2d99923f/pillow-12.1.0-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:bc11908616c8a283cf7d664f77411a5ed2a02009b0097ff8abbba5e79128ccf2", size = 5268571 }, + { url = "https://files.pythonhosted.org/packages/57/61/571163a5ef86ec0cf30d265ac2a70ae6fc9e28413d1dc94fa37fae6bda89/pillow-12.1.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:896866d2d436563fa2a43a9d72f417874f16b5545955c54a64941e87c1376c61", size = 4660426 }, + { url = "https://files.pythonhosted.org/packages/5e/e1/53ee5163f794aef1bf84243f755ee6897a92c708505350dd1923f4afec48/pillow-12.1.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:8e178e3e99d3c0ea8fc64b88447f7cac8ccf058af422a6cedc690d0eadd98c51", size = 6269908 }, + { url = "https://files.pythonhosted.org/packages/bc/0b/b4b4106ff0ee1afa1dc599fde6ab230417f800279745124f6c50bcffed8e/pillow-12.1.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:079af2fb0c599c2ec144ba2c02766d1b55498e373b3ac64687e43849fbbef5bc", size = 8074733 }, + { url = "https://files.pythonhosted.org/packages/19/9f/80b411cbac4a732439e629a26ad3ef11907a8c7fc5377b7602f04f6fe4e7/pillow-12.1.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:bdec5e43377761c5dbca620efb69a77f6855c5a379e32ac5b158f54c84212b14", size = 6381431 }, + { url = "https://files.pythonhosted.org/packages/8f/b7/d65c45db463b66ecb6abc17c6ba6917a911202a07662247e1355ce1789e7/pillow-12.1.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:565c986f4b45c020f5421a4cea13ef294dde9509a8577f29b2fc5edc7587fff8", size = 7068529 }, + { url = "https://files.pythonhosted.org/packages/50/96/dfd4cd726b4a45ae6e3c669fc9e49deb2241312605d33aba50499e9d9bd1/pillow-12.1.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:43aca0a55ce1eefc0aefa6253661cb54571857b1a7b2964bd8a1e3ef4b729924", size = 6492981 }, + { url = "https://files.pythonhosted.org/packages/4d/1c/b5dc52cf713ae46033359c5ca920444f18a6359ce1020dd3e9c553ea5bc6/pillow-12.1.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:0deedf2ea233722476b3a81e8cdfbad786f7adbed5d848469fa59fe52396e4ef", size = 7191878 }, + { url = "https://files.pythonhosted.org/packages/53/26/c4188248bd5edaf543864fe4834aebe9c9cb4968b6f573ce014cc42d0720/pillow-12.1.0-cp314-cp314t-win32.whl", hash = "sha256:b17fbdbe01c196e7e159aacb889e091f28e61020a8abeac07b68079b6e626988", size = 6438703 }, + { url = "https://files.pythonhosted.org/packages/b8/0e/69ed296de8ea05cb03ee139cee600f424ca166e632567b2d66727f08c7ed/pillow-12.1.0-cp314-cp314t-win_amd64.whl", hash = "sha256:27b9baecb428899db6c0de572d6d305cfaf38ca1596b5c0542a5182e3e74e8c6", size = 7182927 }, + { url = "https://files.pythonhosted.org/packages/fc/f5/68334c015eed9b5cff77814258717dec591ded209ab5b6fb70e2ae873d1d/pillow-12.1.0-cp314-cp314t-win_arm64.whl", hash = "sha256:f61333d817698bdcdd0f9d7793e365ac3d2a21c1f1eb02b32ad6aefb8d8ea831", size = 2545104 }, + { url = "https://files.pythonhosted.org/packages/8b/bc/224b1d98cffd7164b14707c91aac83c07b047fbd8f58eba4066a3e53746a/pillow-12.1.0-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:ca94b6aac0d7af2a10ba08c0f888b3d5114439b6b3ef39968378723622fed377", size = 5228605 }, + { url = "https://files.pythonhosted.org/packages/0c/ca/49ca7769c4550107de049ed85208240ba0f330b3f2e316f24534795702ce/pillow-12.1.0-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:351889afef0f485b84078ea40fe33727a0492b9af3904661b0abbafee0355b72", size = 4622245 }, + { url = "https://files.pythonhosted.org/packages/73/48/fac807ce82e5955bcc2718642b94b1bd22a82a6d452aea31cbb678cddf12/pillow-12.1.0-pp311-pypy311_pp73-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:bb0984b30e973f7e2884362b7d23d0a348c7143ee559f38ef3eaab640144204c", size = 5247593 }, + { url = "https://files.pythonhosted.org/packages/d2/95/3e0742fe358c4664aed4fd05d5f5373dcdad0b27af52aa0972568541e3f4/pillow-12.1.0-pp311-pypy311_pp73-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:84cabc7095dd535ca934d57e9ce2a72ffd216e435a84acb06b2277b1de2689bd", size = 6989008 }, + { url = "https://files.pythonhosted.org/packages/5a/74/fe2ac378e4e202e56d50540d92e1ef4ff34ed687f3c60f6a121bcf99437e/pillow-12.1.0-pp311-pypy311_pp73-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:53d8b764726d3af1a138dd353116f774e3862ec7e3794e0c8781e30db0f35dfc", size = 5313824 }, + { url = "https://files.pythonhosted.org/packages/f3/77/2a60dee1adee4e2655ac328dd05c02a955c1cd683b9f1b82ec3feb44727c/pillow-12.1.0-pp311-pypy311_pp73-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5da841d81b1a05ef940a8567da92decaa15bc4d7dedb540a8c219ad83d91808a", size = 5963278 }, + { url = "https://files.pythonhosted.org/packages/2d/71/64e9b1c7f04ae0027f788a248e6297d7fcc29571371fe7d45495a78172c0/pillow-12.1.0-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:75af0b4c229ac519b155028fa1be632d812a519abba9b46b20e50c6caa184f19", size = 7029809 }, +] + +[[package]] +name = "playwright" +version = "1.57.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "greenlet" }, + { name = "pyee" }, +] +wheels = [ + { url = "https://files.pythonhosted.org/packages/ed/b6/e17543cea8290ae4dced10be21d5a43c360096aa2cce0aa7039e60c50df3/playwright-1.57.0-py3-none-macosx_10_13_x86_64.whl", hash = "sha256:9351c1ac3dfd9b3820fe7fc4340d96c0d3736bb68097b9b7a69bd45d25e9370c", size = 41985039 }, + { url = "https://files.pythonhosted.org/packages/8b/04/ef95b67e1ff59c080b2effd1a9a96984d6953f667c91dfe9d77c838fc956/playwright-1.57.0-py3-none-macosx_11_0_arm64.whl", hash = "sha256:a4a9d65027bce48eeba842408bcc1421502dfd7e41e28d207e94260fa93ca67e", size = 40775575 }, + { url = "https://files.pythonhosted.org/packages/60/bd/5563850322a663956c927eefcf1457d12917e8f118c214410e815f2147d1/playwright-1.57.0-py3-none-macosx_11_0_universal2.whl", hash = "sha256:99104771abc4eafee48f47dac2369e0015516dc1ce8c409807d2dd440828b9a4", size = 41985042 }, + { url = "https://files.pythonhosted.org/packages/56/61/3a803cb5ae0321715bfd5247ea871d25b32c8f372aeb70550a90c5f586df/playwright-1.57.0-py3-none-manylinux1_x86_64.whl", hash = "sha256:284ed5a706b7c389a06caa431b2f0ba9ac4130113c3a779767dda758c2497bb1", size = 45975252 }, + { url = "https://files.pythonhosted.org/packages/83/d7/b72eb59dfbea0013a7f9731878df8c670f5f35318cedb010c8a30292c118/playwright-1.57.0-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:38a1bae6c0a07839cdeaddbc0756b3b2b85e476c07945f64ece08f1f956a86f1", size = 45706917 }, + { url = "https://files.pythonhosted.org/packages/e4/09/3fc9ebd7c95ee54ba6a68d5c0bc23e449f7235f4603fc60534a364934c16/playwright-1.57.0-py3-none-win32.whl", hash = "sha256:1dd93b265688da46e91ecb0606d36f777f8eadcf7fbef12f6426b20bf0c9137c", size = 36553860 }, + { url = "https://files.pythonhosted.org/packages/58/d4/dcdfd2a33096aeda6ca0d15584800443dd2be64becca8f315634044b135b/playwright-1.57.0-py3-none-win_amd64.whl", hash = "sha256:6caefb08ed2c6f29d33b8088d05d09376946e49a73be19271c8cd5384b82b14c", size = 36553864 }, + { url = "https://files.pythonhosted.org/packages/6a/60/fe31d7e6b8907789dcb0584f88be741ba388413e4fbce35f1eba4e3073de/playwright-1.57.0-py3-none-win_arm64.whl", hash = "sha256:5f065f5a133dbc15e6e7c71e7bc04f258195755b1c32a432b792e28338c8335e", size = 32837940 }, +] + +[[package]] +name = "pydantic" +version = "2.12.5" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "annotated-types" }, + { name = "pydantic-core" }, + { name = "typing-extensions" }, + { name = "typing-inspection" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/69/44/36f1a6e523abc58ae5f928898e4aca2e0ea509b5aa6f6f392a5d882be928/pydantic-2.12.5.tar.gz", hash = "sha256:4d351024c75c0f085a9febbb665ce8c0c6ec5d30e903bdb6394b7ede26aebb49", size = 821591 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/5a/87/b70ad306ebb6f9b585f114d0ac2137d792b48be34d732d60e597c2f8465a/pydantic-2.12.5-py3-none-any.whl", hash = "sha256:e561593fccf61e8a20fc46dfc2dfe075b8be7d0188df33f221ad1f0139180f9d", size = 463580 }, +] + +[[package]] +name = "pydantic-core" +version = "2.41.5" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/71/70/23b021c950c2addd24ec408e9ab05d59b035b39d97cdc1130e1bce647bb6/pydantic_core-2.41.5.tar.gz", hash = "sha256:08daa51ea16ad373ffd5e7606252cc32f07bc72b28284b6bc9c6df804816476e", size = 460952 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e8/72/74a989dd9f2084b3d9530b0915fdda64ac48831c30dbf7c72a41a5232db8/pydantic_core-2.41.5-cp311-cp311-macosx_10_12_x86_64.whl", hash = "sha256:a3a52f6156e73e7ccb0f8cced536adccb7042be67cb45f9562e12b319c119da6", size = 2105873 }, + { url = "https://files.pythonhosted.org/packages/12/44/37e403fd9455708b3b942949e1d7febc02167662bf1a7da5b78ee1ea2842/pydantic_core-2.41.5-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:7f3bf998340c6d4b0c9a2f02d6a400e51f123b59565d74dc60d252ce888c260b", size = 1899826 }, + { url = "https://files.pythonhosted.org/packages/33/7f/1d5cab3ccf44c1935a359d51a8a2a9e1a654b744b5e7f80d41b88d501eec/pydantic_core-2.41.5-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:378bec5c66998815d224c9ca994f1e14c0c21cb95d2f52b6021cc0b2a58f2a5a", size = 1917869 }, + { url = "https://files.pythonhosted.org/packages/6e/6a/30d94a9674a7fe4f4744052ed6c5e083424510be1e93da5bc47569d11810/pydantic_core-2.41.5-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:e7b576130c69225432866fe2f4a469a85a54ade141d96fd396dffcf607b558f8", size = 2063890 }, + { url = "https://files.pythonhosted.org/packages/50/be/76e5d46203fcb2750e542f32e6c371ffa9b8ad17364cf94bb0818dbfb50c/pydantic_core-2.41.5-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:6cb58b9c66f7e4179a2d5e0f849c48eff5c1fca560994d6eb6543abf955a149e", size = 2229740 }, + { url = "https://files.pythonhosted.org/packages/d3/ee/fed784df0144793489f87db310a6bbf8118d7b630ed07aa180d6067e653a/pydantic_core-2.41.5-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:88942d3a3dff3afc8288c21e565e476fc278902ae4d6d134f1eeda118cc830b1", size = 2350021 }, + { url = "https://files.pythonhosted.org/packages/c8/be/8fed28dd0a180dca19e72c233cbf58efa36df055e5b9d90d64fd1740b828/pydantic_core-2.41.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f31d95a179f8d64d90f6831d71fa93290893a33148d890ba15de25642c5d075b", size = 2066378 }, + { url = "https://files.pythonhosted.org/packages/b0/3b/698cf8ae1d536a010e05121b4958b1257f0b5522085e335360e53a6b1c8b/pydantic_core-2.41.5-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:c1df3d34aced70add6f867a8cf413e299177e0c22660cc767218373d0779487b", size = 2175761 }, + { url = "https://files.pythonhosted.org/packages/b8/ba/15d537423939553116dea94ce02f9c31be0fa9d0b806d427e0308ec17145/pydantic_core-2.41.5-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:4009935984bd36bd2c774e13f9a09563ce8de4abaa7226f5108262fa3e637284", size = 2146303 }, + { url = "https://files.pythonhosted.org/packages/58/7f/0de669bf37d206723795f9c90c82966726a2ab06c336deba4735b55af431/pydantic_core-2.41.5-cp311-cp311-musllinux_1_1_armv7l.whl", hash = "sha256:34a64bc3441dc1213096a20fe27e8e128bd3ff89921706e83c0b1ac971276594", size = 2340355 }, + { url = "https://files.pythonhosted.org/packages/e5/de/e7482c435b83d7e3c3ee5ee4451f6e8973cff0eb6007d2872ce6383f6398/pydantic_core-2.41.5-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:c9e19dd6e28fdcaa5a1de679aec4141f691023916427ef9bae8584f9c2fb3b0e", size = 2319875 }, + { url = "https://files.pythonhosted.org/packages/fe/e6/8c9e81bb6dd7560e33b9053351c29f30c8194b72f2d6932888581f503482/pydantic_core-2.41.5-cp311-cp311-win32.whl", hash = "sha256:2c010c6ded393148374c0f6f0bf89d206bf3217f201faa0635dcd56bd1520f6b", size = 1987549 }, + { url = "https://files.pythonhosted.org/packages/11/66/f14d1d978ea94d1bc21fc98fcf570f9542fe55bfcc40269d4e1a21c19bf7/pydantic_core-2.41.5-cp311-cp311-win_amd64.whl", hash = "sha256:76ee27c6e9c7f16f47db7a94157112a2f3a00e958bc626e2f4ee8bec5c328fbe", size = 2011305 }, + { url = "https://files.pythonhosted.org/packages/56/d8/0e271434e8efd03186c5386671328154ee349ff0354d83c74f5caaf096ed/pydantic_core-2.41.5-cp311-cp311-win_arm64.whl", hash = "sha256:4bc36bbc0b7584de96561184ad7f012478987882ebf9f9c389b23f432ea3d90f", size = 1972902 }, + { url = "https://files.pythonhosted.org/packages/5f/5d/5f6c63eebb5afee93bcaae4ce9a898f3373ca23df3ccaef086d0233a35a7/pydantic_core-2.41.5-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:f41a7489d32336dbf2199c8c0a215390a751c5b014c2c1c5366e817202e9cdf7", size = 2110990 }, + { url = "https://files.pythonhosted.org/packages/aa/32/9c2e8ccb57c01111e0fd091f236c7b371c1bccea0fa85247ac55b1e2b6b6/pydantic_core-2.41.5-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:070259a8818988b9a84a449a2a7337c7f430a22acc0859c6b110aa7212a6d9c0", size = 1896003 }, + { url = "https://files.pythonhosted.org/packages/68/b8/a01b53cb0e59139fbc9e4fda3e9724ede8de279097179be4ff31f1abb65a/pydantic_core-2.41.5-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e96cea19e34778f8d59fe40775a7a574d95816eb150850a85a7a4c8f4b94ac69", size = 1919200 }, + { url = "https://files.pythonhosted.org/packages/38/de/8c36b5198a29bdaade07b5985e80a233a5ac27137846f3bc2d3b40a47360/pydantic_core-2.41.5-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:ed2e99c456e3fadd05c991f8f437ef902e00eedf34320ba2b0842bd1c3ca3a75", size = 2052578 }, + { url = "https://files.pythonhosted.org/packages/00/b5/0e8e4b5b081eac6cb3dbb7e60a65907549a1ce035a724368c330112adfdd/pydantic_core-2.41.5-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:65840751b72fbfd82c3c640cff9284545342a4f1eb1586ad0636955b261b0b05", size = 2208504 }, + { url = "https://files.pythonhosted.org/packages/77/56/87a61aad59c7c5b9dc8caad5a41a5545cba3810c3e828708b3d7404f6cef/pydantic_core-2.41.5-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:e536c98a7626a98feb2d3eaf75944ef6f3dbee447e1f841eae16f2f0a72d8ddc", size = 2335816 }, + { url = "https://files.pythonhosted.org/packages/0d/76/941cc9f73529988688a665a5c0ecff1112b3d95ab48f81db5f7606f522d3/pydantic_core-2.41.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:eceb81a8d74f9267ef4081e246ffd6d129da5d87e37a77c9bde550cb04870c1c", size = 2075366 }, + { url = "https://files.pythonhosted.org/packages/d3/43/ebef01f69baa07a482844faaa0a591bad1ef129253ffd0cdaa9d8a7f72d3/pydantic_core-2.41.5-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:d38548150c39b74aeeb0ce8ee1d8e82696f4a4e16ddc6de7b1d8823f7de4b9b5", size = 2171698 }, + { url = "https://files.pythonhosted.org/packages/b1/87/41f3202e4193e3bacfc2c065fab7706ebe81af46a83d3e27605029c1f5a6/pydantic_core-2.41.5-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:c23e27686783f60290e36827f9c626e63154b82b116d7fe9adba1fda36da706c", size = 2132603 }, + { url = "https://files.pythonhosted.org/packages/49/7d/4c00df99cb12070b6bccdef4a195255e6020a550d572768d92cc54dba91a/pydantic_core-2.41.5-cp312-cp312-musllinux_1_1_armv7l.whl", hash = "sha256:482c982f814460eabe1d3bb0adfdc583387bd4691ef00b90575ca0d2b6fe2294", size = 2329591 }, + { url = "https://files.pythonhosted.org/packages/cc/6a/ebf4b1d65d458f3cda6a7335d141305dfa19bdc61140a884d165a8a1bbc7/pydantic_core-2.41.5-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:bfea2a5f0b4d8d43adf9d7b8bf019fb46fdd10a2e5cde477fbcb9d1fa08c68e1", size = 2319068 }, + { url = "https://files.pythonhosted.org/packages/49/3b/774f2b5cd4192d5ab75870ce4381fd89cf218af999515baf07e7206753f0/pydantic_core-2.41.5-cp312-cp312-win32.whl", hash = "sha256:b74557b16e390ec12dca509bce9264c3bbd128f8a2c376eaa68003d7f327276d", size = 1985908 }, + { url = "https://files.pythonhosted.org/packages/86/45/00173a033c801cacf67c190fef088789394feaf88a98a7035b0e40d53dc9/pydantic_core-2.41.5-cp312-cp312-win_amd64.whl", hash = "sha256:1962293292865bca8e54702b08a4f26da73adc83dd1fcf26fbc875b35d81c815", size = 2020145 }, + { url = "https://files.pythonhosted.org/packages/f9/22/91fbc821fa6d261b376a3f73809f907cec5ca6025642c463d3488aad22fb/pydantic_core-2.41.5-cp312-cp312-win_arm64.whl", hash = "sha256:1746d4a3d9a794cacae06a5eaaccb4b8643a131d45fbc9af23e353dc0a5ba5c3", size = 1976179 }, + { url = "https://files.pythonhosted.org/packages/87/06/8806241ff1f70d9939f9af039c6c35f2360cf16e93c2ca76f184e76b1564/pydantic_core-2.41.5-cp313-cp313-macosx_10_12_x86_64.whl", hash = "sha256:941103c9be18ac8daf7b7adca8228f8ed6bb7a1849020f643b3a14d15b1924d9", size = 2120403 }, + { url = "https://files.pythonhosted.org/packages/94/02/abfa0e0bda67faa65fef1c84971c7e45928e108fe24333c81f3bfe35d5f5/pydantic_core-2.41.5-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:112e305c3314f40c93998e567879e887a3160bb8689ef3d2c04b6cc62c33ac34", size = 1896206 }, + { url = "https://files.pythonhosted.org/packages/15/df/a4c740c0943e93e6500f9eb23f4ca7ec9bf71b19e608ae5b579678c8d02f/pydantic_core-2.41.5-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0cbaad15cb0c90aa221d43c00e77bb33c93e8d36e0bf74760cd00e732d10a6a0", size = 1919307 }, + { url = "https://files.pythonhosted.org/packages/9a/e3/6324802931ae1d123528988e0e86587c2072ac2e5394b4bc2bc34b61ff6e/pydantic_core-2.41.5-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:03ca43e12fab6023fc79d28ca6b39b05f794ad08ec2feccc59a339b02f2b3d33", size = 2063258 }, + { url = "https://files.pythonhosted.org/packages/c9/d4/2230d7151d4957dd79c3044ea26346c148c98fbf0ee6ebd41056f2d62ab5/pydantic_core-2.41.5-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:dc799088c08fa04e43144b164feb0c13f9a0bc40503f8df3e9fde58a3c0c101e", size = 2214917 }, + { url = "https://files.pythonhosted.org/packages/e6/9f/eaac5df17a3672fef0081b6c1bb0b82b33ee89aa5cec0d7b05f52fd4a1fa/pydantic_core-2.41.5-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:97aeba56665b4c3235a0e52b2c2f5ae9cd071b8a8310ad27bddb3f7fb30e9aa2", size = 2332186 }, + { url = "https://files.pythonhosted.org/packages/cf/4e/35a80cae583a37cf15604b44240e45c05e04e86f9cfd766623149297e971/pydantic_core-2.41.5-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:406bf18d345822d6c21366031003612b9c77b3e29ffdb0f612367352aab7d586", size = 2073164 }, + { url = "https://files.pythonhosted.org/packages/bf/e3/f6e262673c6140dd3305d144d032f7bd5f7497d3871c1428521f19f9efa2/pydantic_core-2.41.5-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:b93590ae81f7010dbe380cdeab6f515902ebcbefe0b9327cc4804d74e93ae69d", size = 2179146 }, + { url = "https://files.pythonhosted.org/packages/75/c7/20bd7fc05f0c6ea2056a4565c6f36f8968c0924f19b7d97bbfea55780e73/pydantic_core-2.41.5-cp313-cp313-musllinux_1_1_aarch64.whl", hash = "sha256:01a3d0ab748ee531f4ea6c3e48ad9dac84ddba4b0d82291f87248f2f9de8d740", size = 2137788 }, + { url = "https://files.pythonhosted.org/packages/3a/8d/34318ef985c45196e004bc46c6eab2eda437e744c124ef0dbe1ff2c9d06b/pydantic_core-2.41.5-cp313-cp313-musllinux_1_1_armv7l.whl", hash = "sha256:6561e94ba9dacc9c61bce40e2d6bdc3bfaa0259d3ff36ace3b1e6901936d2e3e", size = 2340133 }, + { url = "https://files.pythonhosted.org/packages/9c/59/013626bf8c78a5a5d9350d12e7697d3d4de951a75565496abd40ccd46bee/pydantic_core-2.41.5-cp313-cp313-musllinux_1_1_x86_64.whl", hash = "sha256:915c3d10f81bec3a74fbd4faebe8391013ba61e5a1a8d48c4455b923bdda7858", size = 2324852 }, + { url = "https://files.pythonhosted.org/packages/1a/d9/c248c103856f807ef70c18a4f986693a46a8ffe1602e5d361485da502d20/pydantic_core-2.41.5-cp313-cp313-win32.whl", hash = "sha256:650ae77860b45cfa6e2cdafc42618ceafab3a2d9a3811fcfbd3bbf8ac3c40d36", size = 1994679 }, + { url = "https://files.pythonhosted.org/packages/9e/8b/341991b158ddab181cff136acd2552c9f35bd30380422a639c0671e99a91/pydantic_core-2.41.5-cp313-cp313-win_amd64.whl", hash = "sha256:79ec52ec461e99e13791ec6508c722742ad745571f234ea6255bed38c6480f11", size = 2019766 }, + { url = "https://files.pythonhosted.org/packages/73/7d/f2f9db34af103bea3e09735bb40b021788a5e834c81eedb541991badf8f5/pydantic_core-2.41.5-cp313-cp313-win_arm64.whl", hash = "sha256:3f84d5c1b4ab906093bdc1ff10484838aca54ef08de4afa9de0f5f14d69639cd", size = 1981005 }, + { url = "https://files.pythonhosted.org/packages/ea/28/46b7c5c9635ae96ea0fbb779e271a38129df2550f763937659ee6c5dbc65/pydantic_core-2.41.5-cp314-cp314-macosx_10_12_x86_64.whl", hash = "sha256:3f37a19d7ebcdd20b96485056ba9e8b304e27d9904d233d7b1015db320e51f0a", size = 2119622 }, + { url = "https://files.pythonhosted.org/packages/74/1a/145646e5687e8d9a1e8d09acb278c8535ebe9e972e1f162ed338a622f193/pydantic_core-2.41.5-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:1d1d9764366c73f996edd17abb6d9d7649a7eb690006ab6adbda117717099b14", size = 1891725 }, + { url = "https://files.pythonhosted.org/packages/23/04/e89c29e267b8060b40dca97bfc64a19b2a3cf99018167ea1677d96368273/pydantic_core-2.41.5-cp314-cp314-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:25e1c2af0fce638d5f1988b686f3b3ea8cd7de5f244ca147c777769e798a9cd1", size = 1915040 }, + { url = "https://files.pythonhosted.org/packages/84/a3/15a82ac7bd97992a82257f777b3583d3e84bdb06ba6858f745daa2ec8a85/pydantic_core-2.41.5-cp314-cp314-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:506d766a8727beef16b7adaeb8ee6217c64fc813646b424d0804d67c16eddb66", size = 2063691 }, + { url = "https://files.pythonhosted.org/packages/74/9b/0046701313c6ef08c0c1cf0e028c67c770a4e1275ca73131563c5f2a310a/pydantic_core-2.41.5-cp314-cp314-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:4819fa52133c9aa3c387b3328f25c1facc356491e6135b459f1de698ff64d869", size = 2213897 }, + { url = "https://files.pythonhosted.org/packages/8a/cd/6bac76ecd1b27e75a95ca3a9a559c643b3afcd2dd62086d4b7a32a18b169/pydantic_core-2.41.5-cp314-cp314-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:2b761d210c9ea91feda40d25b4efe82a1707da2ef62901466a42492c028553a2", size = 2333302 }, + { url = "https://files.pythonhosted.org/packages/4c/d2/ef2074dc020dd6e109611a8be4449b98cd25e1b9b8a303c2f0fca2f2bcf7/pydantic_core-2.41.5-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:22f0fb8c1c583a3b6f24df2470833b40207e907b90c928cc8d3594b76f874375", size = 2064877 }, + { url = "https://files.pythonhosted.org/packages/18/66/e9db17a9a763d72f03de903883c057b2592c09509ccfe468187f2a2eef29/pydantic_core-2.41.5-cp314-cp314-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:2782c870e99878c634505236d81e5443092fba820f0373997ff75f90f68cd553", size = 2180680 }, + { url = "https://files.pythonhosted.org/packages/d3/9e/3ce66cebb929f3ced22be85d4c2399b8e85b622db77dad36b73c5387f8f8/pydantic_core-2.41.5-cp314-cp314-musllinux_1_1_aarch64.whl", hash = "sha256:0177272f88ab8312479336e1d777f6b124537d47f2123f89cb37e0accea97f90", size = 2138960 }, + { url = "https://files.pythonhosted.org/packages/a6/62/205a998f4327d2079326b01abee48e502ea739d174f0a89295c481a2272e/pydantic_core-2.41.5-cp314-cp314-musllinux_1_1_armv7l.whl", hash = "sha256:63510af5e38f8955b8ee5687740d6ebf7c2a0886d15a6d65c32814613681bc07", size = 2339102 }, + { url = "https://files.pythonhosted.org/packages/3c/0d/f05e79471e889d74d3d88f5bd20d0ed189ad94c2423d81ff8d0000aab4ff/pydantic_core-2.41.5-cp314-cp314-musllinux_1_1_x86_64.whl", hash = "sha256:e56ba91f47764cc14f1daacd723e3e82d1a89d783f0f5afe9c364b8bb491ccdb", size = 2326039 }, + { url = "https://files.pythonhosted.org/packages/ec/e1/e08a6208bb100da7e0c4b288eed624a703f4d129bde2da475721a80cab32/pydantic_core-2.41.5-cp314-cp314-win32.whl", hash = "sha256:aec5cf2fd867b4ff45b9959f8b20ea3993fc93e63c7363fe6851424c8a7e7c23", size = 1995126 }, + { url = "https://files.pythonhosted.org/packages/48/5d/56ba7b24e9557f99c9237e29f5c09913c81eeb2f3217e40e922353668092/pydantic_core-2.41.5-cp314-cp314-win_amd64.whl", hash = "sha256:8e7c86f27c585ef37c35e56a96363ab8de4e549a95512445b85c96d3e2f7c1bf", size = 2015489 }, + { url = "https://files.pythonhosted.org/packages/4e/bb/f7a190991ec9e3e0ba22e4993d8755bbc4a32925c0b5b42775c03e8148f9/pydantic_core-2.41.5-cp314-cp314-win_arm64.whl", hash = "sha256:e672ba74fbc2dc8eea59fb6d4aed6845e6905fc2a8afe93175d94a83ba2a01a0", size = 1977288 }, + { url = "https://files.pythonhosted.org/packages/92/ed/77542d0c51538e32e15afe7899d79efce4b81eee631d99850edc2f5e9349/pydantic_core-2.41.5-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:8566def80554c3faa0e65ac30ab0932b9e3a5cd7f8323764303d468e5c37595a", size = 2120255 }, + { url = "https://files.pythonhosted.org/packages/bb/3d/6913dde84d5be21e284439676168b28d8bbba5600d838b9dca99de0fad71/pydantic_core-2.41.5-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:b80aa5095cd3109962a298ce14110ae16b8c1aece8b72f9dafe81cf597ad80b3", size = 1863760 }, + { url = "https://files.pythonhosted.org/packages/5a/f0/e5e6b99d4191da102f2b0eb9687aaa7f5bea5d9964071a84effc3e40f997/pydantic_core-2.41.5-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:3006c3dd9ba34b0c094c544c6006cc79e87d8612999f1a5d43b769b89181f23c", size = 1878092 }, + { url = "https://files.pythonhosted.org/packages/71/48/36fb760642d568925953bcc8116455513d6e34c4beaa37544118c36aba6d/pydantic_core-2.41.5-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:72f6c8b11857a856bcfa48c86f5368439f74453563f951e473514579d44aa612", size = 2053385 }, + { url = "https://files.pythonhosted.org/packages/20/25/92dc684dd8eb75a234bc1c764b4210cf2646479d54b47bf46061657292a8/pydantic_core-2.41.5-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:5cb1b2f9742240e4bb26b652a5aeb840aa4b417c7748b6f8387927bc6e45e40d", size = 2218832 }, + { url = "https://files.pythonhosted.org/packages/e2/09/f53e0b05023d3e30357d82eb35835d0f6340ca344720a4599cd663dca599/pydantic_core-2.41.5-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:bd3d54f38609ff308209bd43acea66061494157703364ae40c951f83ba99a1a9", size = 2327585 }, + { url = "https://files.pythonhosted.org/packages/aa/4e/2ae1aa85d6af35a39b236b1b1641de73f5a6ac4d5a7509f77b814885760c/pydantic_core-2.41.5-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:2ff4321e56e879ee8d2a879501c8e469414d948f4aba74a2d4593184eb326660", size = 2041078 }, + { url = "https://files.pythonhosted.org/packages/cd/13/2e215f17f0ef326fc72afe94776edb77525142c693767fc347ed6288728d/pydantic_core-2.41.5-cp314-cp314t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:d0d2568a8c11bf8225044aa94409e21da0cb09dcdafe9ecd10250b2baad531a9", size = 2173914 }, + { url = "https://files.pythonhosted.org/packages/02/7a/f999a6dcbcd0e5660bc348a3991c8915ce6599f4f2c6ac22f01d7a10816c/pydantic_core-2.41.5-cp314-cp314t-musllinux_1_1_aarch64.whl", hash = "sha256:a39455728aabd58ceabb03c90e12f71fd30fa69615760a075b9fec596456ccc3", size = 2129560 }, + { url = "https://files.pythonhosted.org/packages/3a/b1/6c990ac65e3b4c079a4fb9f5b05f5b013afa0f4ed6780a3dd236d2cbdc64/pydantic_core-2.41.5-cp314-cp314t-musllinux_1_1_armv7l.whl", hash = "sha256:239edca560d05757817c13dc17c50766136d21f7cd0fac50295499ae24f90fdf", size = 2329244 }, + { url = "https://files.pythonhosted.org/packages/d9/02/3c562f3a51afd4d88fff8dffb1771b30cfdfd79befd9883ee094f5b6c0d8/pydantic_core-2.41.5-cp314-cp314t-musllinux_1_1_x86_64.whl", hash = "sha256:2a5e06546e19f24c6a96a129142a75cee553cc018ffee48a460059b1185f4470", size = 2331955 }, + { url = "https://files.pythonhosted.org/packages/5c/96/5fb7d8c3c17bc8c62fdb031c47d77a1af698f1d7a406b0f79aaa1338f9ad/pydantic_core-2.41.5-cp314-cp314t-win32.whl", hash = "sha256:b4ececa40ac28afa90871c2cc2b9ffd2ff0bf749380fbdf57d165fd23da353aa", size = 1988906 }, + { url = "https://files.pythonhosted.org/packages/22/ed/182129d83032702912c2e2d8bbe33c036f342cc735737064668585dac28f/pydantic_core-2.41.5-cp314-cp314t-win_amd64.whl", hash = "sha256:80aa89cad80b32a912a65332f64a4450ed00966111b6615ca6816153d3585a8c", size = 1981607 }, + { url = "https://files.pythonhosted.org/packages/9f/ed/068e41660b832bb0b1aa5b58011dea2a3fe0ba7861ff38c4d4904c1c1a99/pydantic_core-2.41.5-cp314-cp314t-win_arm64.whl", hash = "sha256:35b44f37a3199f771c3eaa53051bc8a70cd7b54f333531c59e29fd4db5d15008", size = 1974769 }, + { url = "https://files.pythonhosted.org/packages/11/72/90fda5ee3b97e51c494938a4a44c3a35a9c96c19bba12372fb9c634d6f57/pydantic_core-2.41.5-graalpy311-graalpy242_311_native-macosx_10_12_x86_64.whl", hash = "sha256:b96d5f26b05d03cc60f11a7761a5ded1741da411e7fe0909e27a5e6a0cb7b034", size = 2115441 }, + { url = "https://files.pythonhosted.org/packages/1f/53/8942f884fa33f50794f119012dc6a1a02ac43a56407adaac20463df8e98f/pydantic_core-2.41.5-graalpy311-graalpy242_311_native-macosx_11_0_arm64.whl", hash = "sha256:634e8609e89ceecea15e2d61bc9ac3718caaaa71963717bf3c8f38bfde64242c", size = 1930291 }, + { url = "https://files.pythonhosted.org/packages/79/c8/ecb9ed9cd942bce09fc888ee960b52654fbdbede4ba6c2d6e0d3b1d8b49c/pydantic_core-2.41.5-graalpy311-graalpy242_311_native-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:93e8740d7503eb008aa2df04d3b9735f845d43ae845e6dcd2be0b55a2da43cd2", size = 1948632 }, + { url = "https://files.pythonhosted.org/packages/2e/1b/687711069de7efa6af934e74f601e2a4307365e8fdc404703afc453eab26/pydantic_core-2.41.5-graalpy311-graalpy242_311_native-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f15489ba13d61f670dcc96772e733aad1a6f9c429cc27574c6cdaed82d0146ad", size = 2138905 }, + { url = "https://files.pythonhosted.org/packages/09/32/59b0c7e63e277fa7911c2fc70ccfb45ce4b98991e7ef37110663437005af/pydantic_core-2.41.5-graalpy312-graalpy250_312_native-macosx_10_12_x86_64.whl", hash = "sha256:7da7087d756b19037bc2c06edc6c170eeef3c3bafcb8f532ff17d64dc427adfd", size = 2110495 }, + { url = "https://files.pythonhosted.org/packages/aa/81/05e400037eaf55ad400bcd318c05bb345b57e708887f07ddb2d20e3f0e98/pydantic_core-2.41.5-graalpy312-graalpy250_312_native-macosx_11_0_arm64.whl", hash = "sha256:aabf5777b5c8ca26f7824cb4a120a740c9588ed58df9b2d196ce92fba42ff8dc", size = 1915388 }, + { url = "https://files.pythonhosted.org/packages/6e/0d/e3549b2399f71d56476b77dbf3cf8937cec5cd70536bdc0e374a421d0599/pydantic_core-2.41.5-graalpy312-graalpy250_312_native-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c007fe8a43d43b3969e8469004e9845944f1a80e6acd47c150856bb87f230c56", size = 1942879 }, + { url = "https://files.pythonhosted.org/packages/f7/07/34573da085946b6a313d7c42f82f16e8920bfd730665de2d11c0c37a74b5/pydantic_core-2.41.5-graalpy312-graalpy250_312_native-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:76d0819de158cd855d1cbb8fcafdf6f5cf1eb8e470abe056d5d161106e38062b", size = 2139017 }, + { url = "https://files.pythonhosted.org/packages/5f/9b/1b3f0e9f9305839d7e84912f9e8bfbd191ed1b1ef48083609f0dabde978c/pydantic_core-2.41.5-pp311-pypy311_pp73-macosx_10_12_x86_64.whl", hash = "sha256:b2379fa7ed44ddecb5bfe4e48577d752db9fc10be00a6b7446e9663ba143de26", size = 2101980 }, + { url = "https://files.pythonhosted.org/packages/a4/ed/d71fefcb4263df0da6a85b5d8a7508360f2f2e9b3bf5814be9c8bccdccc1/pydantic_core-2.41.5-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:266fb4cbf5e3cbd0b53669a6d1b039c45e3ce651fd5442eff4d07c2cc8d66808", size = 1923865 }, + { url = "https://files.pythonhosted.org/packages/ce/3a/626b38db460d675f873e4444b4bb030453bbe7b4ba55df821d026a0493c4/pydantic_core-2.41.5-pp311-pypy311_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:58133647260ea01e4d0500089a8c4f07bd7aa6ce109682b1426394988d8aaacc", size = 2134256 }, + { url = "https://files.pythonhosted.org/packages/83/d9/8412d7f06f616bbc053d30cb4e5f76786af3221462ad5eee1f202021eb4e/pydantic_core-2.41.5-pp311-pypy311_pp73-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:287dad91cfb551c363dc62899a80e9e14da1f0e2b6ebde82c806612ca2a13ef1", size = 2174762 }, + { url = "https://files.pythonhosted.org/packages/55/4c/162d906b8e3ba3a99354e20faa1b49a85206c47de97a639510a0e673f5da/pydantic_core-2.41.5-pp311-pypy311_pp73-musllinux_1_1_aarch64.whl", hash = "sha256:03b77d184b9eb40240ae9fd676ca364ce1085f203e1b1256f8ab9984dca80a84", size = 2143141 }, + { url = "https://files.pythonhosted.org/packages/1f/f2/f11dd73284122713f5f89fc940f370d035fa8e1e078d446b3313955157fe/pydantic_core-2.41.5-pp311-pypy311_pp73-musllinux_1_1_armv7l.whl", hash = "sha256:a668ce24de96165bb239160b3d854943128f4334822900534f2fe947930e5770", size = 2330317 }, + { url = "https://files.pythonhosted.org/packages/88/9d/b06ca6acfe4abb296110fb1273a4d848a0bfb2ff65f3ee92127b3244e16b/pydantic_core-2.41.5-pp311-pypy311_pp73-musllinux_1_1_x86_64.whl", hash = "sha256:f14f8f046c14563f8eb3f45f499cc658ab8d10072961e07225e507adb700e93f", size = 2316992 }, + { url = "https://files.pythonhosted.org/packages/36/c7/cfc8e811f061c841d7990b0201912c3556bfeb99cdcb7ed24adc8d6f8704/pydantic_core-2.41.5-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:56121965f7a4dc965bff783d70b907ddf3d57f6eba29b6d2e5dabfaf07799c51", size = 2145302 }, +] + +[[package]] +name = "pydantic-settings" +version = "2.12.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "pydantic" }, + { name = "python-dotenv" }, + { name = "typing-inspection" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/43/4b/ac7e0aae12027748076d72a8764ff1c9d82ca75a7a52622e67ed3f765c54/pydantic_settings-2.12.0.tar.gz", hash = "sha256:005538ef951e3c2a68e1c08b292b5f2e71490def8589d4221b95dab00dafcfd0", size = 194184 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c1/60/5d4751ba3f4a40a6891f24eec885f51afd78d208498268c734e256fb13c4/pydantic_settings-2.12.0-py3-none-any.whl", hash = "sha256:fddb9fd99a5b18da837b29710391e945b1e30c135477f484084ee513adb93809", size = 51880 }, +] + +[[package]] +name = "pyee" +version = "13.0.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/95/03/1fd98d5841cd7964a27d729ccf2199602fe05eb7a405c1462eb7277945ed/pyee-13.0.0.tar.gz", hash = "sha256:b391e3c5a434d1f5118a25615001dbc8f669cf410ab67d04c4d4e07c55481c37", size = 31250 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/9b/4d/b9add7c84060d4c1906abe9a7e5359f2a60f7a9a4f67268b2766673427d8/pyee-13.0.0-py3-none-any.whl", hash = "sha256:48195a3cddb3b1515ce0695ed76036b5ccc2ef3a9f963ff9f77aec0139845498", size = 15730 }, +] + +[[package]] +name = "pyparsing" +version = "3.3.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/33/c1/1d9de9aeaa1b89b0186e5fe23294ff6517fce1bc69149185577cd31016b2/pyparsing-3.3.1.tar.gz", hash = "sha256:47fad0f17ac1e2cad3de3b458570fbc9b03560aa029ed5e16ee5554da9a2251c", size = 1550512 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/8b/40/2614036cdd416452f5bf98ec037f38a1afb17f327cb8e6b652d4729e0af8/pyparsing-3.3.1-py3-none-any.whl", hash = "sha256:023b5e7e5520ad96642e2c6db4cb683d3970bd640cdf7115049a6e9c3682df82", size = 121793 }, +] + +[[package]] +name = "python-dateutil" +version = "2.9.0.post0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "six" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/66/c0/0c8b6ad9f17a802ee498c46e004a0eb49bc148f2fd230864601a86dcf6db/python-dateutil-2.9.0.post0.tar.gz", hash = "sha256:37dd54208da7e1cd875388217d5e00ebd4179249f90fb72437e91a35459a0ad3", size = 342432 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/ec/57/56b9bcc3c9c6a792fcbaf139543cee77261f3651ca9da0c93f5c1221264b/python_dateutil-2.9.0.post0-py2.py3-none-any.whl", hash = "sha256:a8b2bc7bffae282281c8140a97d3aa9c14da0b136dfe83f850eea9a5f7470427", size = 229892 }, +] + +[[package]] +name = "python-dotenv" +version = "1.2.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/f0/26/19cadc79a718c5edbec86fd4919a6b6d3f681039a2f6d66d14be94e75fb9/python_dotenv-1.2.1.tar.gz", hash = "sha256:42667e897e16ab0d66954af0e60a9caa94f0fd4ecf3aaf6d2d260eec1aa36ad6", size = 44221 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/14/1b/a298b06749107c305e1fe0f814c6c74aea7b2f1e10989cb30f544a1b3253/python_dotenv-1.2.1-py3-none-any.whl", hash = "sha256:b81ee9561e9ca4004139c6cbba3a238c32b03e4894671e181b671e8cb8425d61", size = 21230 }, +] + +[[package]] +name = "pytz" +version = "2025.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/f8/bf/abbd3cdfb8fbc7fb3d4d38d320f2441b1e7cbe29be4f23797b4a2b5d8aac/pytz-2025.2.tar.gz", hash = "sha256:360b9e3dbb49a209c21ad61809c7fb453643e048b38924c765813546746e81c3", size = 320884 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/81/c4/34e93fe5f5429d7570ec1fa436f1986fb1f00c3e0f43a589fe2bbcd22c3f/pytz-2025.2-py2.py3-none-any.whl", hash = "sha256:5ddf76296dd8c44c26eb8f4b6f35488f3ccbf6fbbd7adee0b7262d43f0ec2f00", size = 509225 }, +] + +[[package]] +name = "six" +version = "1.17.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/94/e7/b2c673351809dca68a0e064b6af791aa332cf192da575fd474ed7d6f16a2/six-1.17.0.tar.gz", hash = "sha256:ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81", size = 34031 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b7/ce/149a00dd41f10bc29e5921b496af8b574d8413afcd5e30dfa0ed46c2cc5e/six-1.17.0-py2.py3-none-any.whl", hash = "sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274", size = 11050 }, +] + +[[package]] +name = "typing-extensions" +version = "4.15.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/72/94/1a15dd82efb362ac84269196e94cf00f187f7ed21c242792a923cdb1c61f/typing_extensions-4.15.0.tar.gz", hash = "sha256:0cea48d173cc12fa28ecabc3b837ea3cf6f38c6d1136f85cbaaf598984861466", size = 109391 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl", hash = "sha256:f0fa19c6845758ab08074a0cfa8b7aecb71c999ca73d62883bc25cc018c4e548", size = 44614 }, +] + +[[package]] +name = "typing-inspection" +version = "0.4.2" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/55/e3/70399cb7dd41c10ac53367ae42139cf4b1ca5f36bb3dc6c9d33acdb43655/typing_inspection-0.4.2.tar.gz", hash = "sha256:ba561c48a67c5958007083d386c3295464928b01faa735ab8547c5692e87f464", size = 75949 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/dc/9b/47798a6c91d8bdb567fe2698fe81e0c6b7cb7ef4d13da4114b41d239f65d/typing_inspection-0.4.2-py3-none-any.whl", hash = "sha256:4ed1cacbdc298c220f1bd249ed5287caa16f34d44ef4e9c3d0cbad5b521545e7", size = 14611 }, +] + +[[package]] +name = "tzdata" +version = "2025.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/5e/a7/c202b344c5ca7daf398f3b8a477eeb205cf3b6f32e7ec3a6bac0629ca975/tzdata-2025.3.tar.gz", hash = "sha256:de39c2ca5dc7b0344f2eba86f49d614019d29f060fc4ebc8a417896a620b56a7", size = 196772 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c7/b0/003792df09decd6849a5e39c28b513c06e84436a54440380862b5aeff25d/tzdata-2025.3-py2.py3-none-any.whl", hash = "sha256:06a47e5700f3081aab02b2e513160914ff0694bce9947d6b76ebd6bf57cfc5d1", size = 348521 }, +] + +[[package]] +name = "win32-setctime" +version = "1.2.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/b3/8f/705086c9d734d3b663af0e9bb3d4de6578d08f46b1b101c2442fd9aecaa2/win32_setctime-1.2.0.tar.gz", hash = "sha256:ae1fdf948f5640aae05c511ade119313fb6a30d7eabe25fef9764dca5873c4c0", size = 4867 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e1/07/c6fe3ad3e685340704d314d765b7912993bcb8dc198f0e7a89382d37974b/win32_setctime-1.2.0-py3-none-any.whl", hash = "sha256:95d644c4e708aba81dc3704a116d8cbc974d70b3bdb8be1d150e36be6e9d1390", size = 4083 }, +] + +[[package]] +name = "wordcloud" +version = "1.9.5" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "matplotlib" }, + { name = "numpy" }, + { name = "pillow" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/1f/a2/108cd319f6315931708a7c03d0824cd8684eb56e0af56e375e61785e4b3c/wordcloud-1.9.5.tar.gz", hash = "sha256:6ac7c1378f2886d7e849600a306febd41d0d46b15ce876d665a3e549f5403b0b", size = 27563652 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/7d/a2/319d4fac92cc9a943d86fd1feb39077e6ca74dfeca8b0bc5a5be409d235f/wordcloud-1.9.5-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:c4156064e1d20a553125fef32f7bb5b2333a1966661f71688e714380a88ca4ea", size = 168771 }, + { url = "https://files.pythonhosted.org/packages/b7/0c/d4a1510749489b1ac1390ec15f8f814923ae65014367533ebbc167222cc5/wordcloud-1.9.5-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:6ab1913657a2189b84ae9ab7eacb50516e37586f420ccdc6abdfaa23512e4424", size = 168402 }, + { url = "https://files.pythonhosted.org/packages/e9/03/1ff71d1ba850aa15f2c1959ead7142db781e4a767bb95045358fd0927290/wordcloud-1.9.5-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ed6dbd945c3e4a18b39823a7ffea839499e684c85f10d25cd7693d1b9892c52a", size = 547684 }, + { url = "https://files.pythonhosted.org/packages/bf/0c/606dde0beb4abd952a8d1631e4630fac70c51daedd94a204aa0086f3da6e/wordcloud-1.9.5-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:55c0a6d38ba4fef29738e4725142b4c0c3769f301047758d480eb110e6796060", size = 551670 }, + { url = "https://files.pythonhosted.org/packages/4f/74/2ebe53a215e7e88b89983f16b14fb85ab75ba94817fa11e466f51808ac12/wordcloud-1.9.5-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:b31c2568af3201806193e7ca5c05e7d66b60ba2bcd854b08501b2496d77ca2ff", size = 544097 }, + { url = "https://files.pythonhosted.org/packages/06/65/83525a140ed7b26b367e56010846a3e655f26053eb336112d8b6559e87a2/wordcloud-1.9.5-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:1a630bfff5614b90e3cbb4b6d8a74bb296dc247126868fe40c1e5678c3f2ca58", size = 555467 }, + { url = "https://files.pythonhosted.org/packages/41/6a/8d22301bc03756652cc7081f2f196614c1cd505e8cd22d771e6bc7530d6e/wordcloud-1.9.5-cp311-cp311-win32.whl", hash = "sha256:69ff2f262ca349ae59482b647aef63222d19aeba54f38a705ffeca558847826c", size = 295608 }, + { url = "https://files.pythonhosted.org/packages/14/9e/6d5357fe58af3a1c6f7e58eaa88e77fb86556ff4027e8ea90032a6185ba0/wordcloud-1.9.5-cp311-cp311-win_amd64.whl", hash = "sha256:ded40e3ebaaa96eaa7ae86df0bbe89da7ebf5301efcfe9e429145a2a04dc72fb", size = 306082 }, + { url = "https://files.pythonhosted.org/packages/f0/af/8ca23d9a29c7e646e9e21ca0c3f798b08dd3e58ea61b32f4431a13d27d41/wordcloud-1.9.5-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:ac56a1bdd961253f528d48044650ede52534a2fc47427c16729e8386a2beba29", size = 170100 }, + { url = "https://files.pythonhosted.org/packages/3a/18/4239c7a209a55a1dbb58d2bfca215d9a53500eab5b4386d1b5c44d47a073/wordcloud-1.9.5-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:daf0a1a80fdd8bb60619fa3e6855ecf862efaa6149e1c86fcbefe5db02354cd4", size = 168920 }, + { url = "https://files.pythonhosted.org/packages/bf/4f/dc24ca5c366a7f5ff2d7ea510cf50ed5b3773825ceba56af1cf9b803437e/wordcloud-1.9.5-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d36983701f194828fa99c1162cd16610f43959a5fe09f9f8b7b6619ab7390051", size = 548944 }, + { url = "https://files.pythonhosted.org/packages/90/99/a4bc45e087f7f3f11893b0a4feea5d9d72ecd75d9c615341e04de069023b/wordcloud-1.9.5-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b86143325c12479f02c7f994cf19afcc5bc194a2d74456adf03d71c91215bb99", size = 555208 }, + { url = "https://files.pythonhosted.org/packages/10/c0/021e86f11fe660adb88d58e7b3f66658ae9a93b02f1d75c5fb036d4a7359/wordcloud-1.9.5-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:599ff3e40121ad6b2714daf3fb19c094d3248de69429792a674321079dc93bfe", size = 539344 }, + { url = "https://files.pythonhosted.org/packages/7d/ef/a08ea52eb7649d9296abbc6319634b1ece7bc14b1080ac92a460d725a410/wordcloud-1.9.5-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:f32fbe231518534b20703dadc7e397cf6edee82c0ff572ce9193447e4418cf94", size = 554955 }, + { url = "https://files.pythonhosted.org/packages/a0/42/116c6f3365ad2f0d882ad68c33122e3a82c7503a6eadfff1286a59121efa/wordcloud-1.9.5-cp312-cp312-win32.whl", hash = "sha256:5a8954b28d5c9d515944343adecd9dfde8dbe723815768fdc5ae5eb541426f1e", size = 296178 }, + { url = "https://files.pythonhosted.org/packages/ba/66/04e0f33135d7b8d76bd1721c1c7a42a0cbe748ff48588ab6ea01316e1ab0/wordcloud-1.9.5-cp312-cp312-win_amd64.whl", hash = "sha256:790cf92513a1f5e4d65c801d9fe35c607a4219079075f342bb2fe32d427d64ce", size = 307255 }, + { url = "https://files.pythonhosted.org/packages/04/14/261b76055dda37c4adda27d81b4c4917c0c8c0beeb82bc17cc929112fd19/wordcloud-1.9.5-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:a8468c5943ca2c95a5785314dd8b9c2aff622a16459ec16f44fbabfdb47ed68d", size = 169342 }, + { url = "https://files.pythonhosted.org/packages/50/14/3d60c08364ae1a8c54ab7b1f326f69c681e7c59dff32081bf75adb2b2b26/wordcloud-1.9.5-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:cc68fc27f2fc613a903271c03ff9c47801bce9e4663cf940831d434cb8e7aae8", size = 168295 }, + { url = "https://files.pythonhosted.org/packages/08/55/275ded21f0b815c93d12d66b0425b1ee549b19bb5c5e8d60924e43f05b7e/wordcloud-1.9.5-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:000c8ee667a572334763de28d49838456a12642f272b97c5883550ba143f2d93", size = 543738 }, + { url = "https://files.pythonhosted.org/packages/e8/56/72d77bc4416a6aa97ffbba633ac6d8f75156cd593e5f559cad8d84553be4/wordcloud-1.9.5-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f5ae9697c4a6674fd19b16178f1c40b8f3445eb6324fa06cbe0ff053c03c3d61", size = 551843 }, + { url = "https://files.pythonhosted.org/packages/cb/6f/937f53365cc67f98325057490d63a27749c8526fd8bea8e4a1fbe74045e3/wordcloud-1.9.5-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:a16c7bc45b1625374620d0a859c0c6dea0b86a7161b054a9949deaabac5da3dc", size = 536626 }, + { url = "https://files.pythonhosted.org/packages/63/cc/d566ab24da637787a381a1bc9999a166932b53d2b73b12b989c8fb1fa595/wordcloud-1.9.5-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:ecd0400835ee92d9d335aac3d73b3a50cb44f3ba408f03132d5d0bd915b8feef", size = 552904 }, + { url = "https://files.pythonhosted.org/packages/fb/96/47bfce33702be594bad4c1f652e7e16c1d8efd5b8163f9389f38dfa4b3fa/wordcloud-1.9.5-cp313-cp313-win32.whl", hash = "sha256:dc2027675d9c8a72e6565844e442736ff55ce670421ddb02c77d5d9178a9a798", size = 296069 }, + { url = "https://files.pythonhosted.org/packages/c1/b9/440cf09c98680f15ccd83aa31a71d8789ad70ee65a731ead23a6ba8b169c/wordcloud-1.9.5-cp313-cp313-win_amd64.whl", hash = "sha256:b02205ff66f81bc6be1c418c98da08c353847196f2ccf945bd3ea5a52f22aec7", size = 307047 }, + { url = "https://files.pythonhosted.org/packages/29/44/59372ad37df4c93a662f405689ad35039a40c6a0e0dc72e420a6b9bf8aab/wordcloud-1.9.5-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:8cc6baa2f2a64cf6c65eb0034997b54e2a7538e14833582ad91f3686826ebb14", size = 169693 }, + { url = "https://files.pythonhosted.org/packages/72/b3/60481e917ce07f3139961eb1dc32f45746dda132fc919d27f2aefa6ca2a2/wordcloud-1.9.5-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:3e744e70d99b2083f7a544c77fea59e0df92053c3d310669c43235bbdaa9a6b9", size = 168940 }, + { url = "https://files.pythonhosted.org/packages/6a/c5/dd5e409a9ecb2e1fa026b061703b16282257b350dd5a15c5b8de682ecc04/wordcloud-1.9.5-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:508a569826751f52db1225c91e01f3ea4c725421431338a6cb17768aa29b3d15", size = 543226 }, + { url = "https://files.pythonhosted.org/packages/01/ab/9c24089ea2883403a3a1586745f827cb6866bd43b41276a0d00d12a4b978/wordcloud-1.9.5-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8e09563dd90aeb42d2ca034b39ba691298f783be373dc7d7c2d7430e729c1f41", size = 547357 }, + { url = "https://files.pythonhosted.org/packages/03/c2/7d24d1fdddf329da69a2c72a6204e40250e42478037c593545d70691539e/wordcloud-1.9.5-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:97d0ac557dc80710e56609760f733ae4f382fc93e4c807c4abcfacbfb1a7a4c3", size = 535710 }, + { url = "https://files.pythonhosted.org/packages/ba/b9/c474a1d651fe2d941a6f92ad65e8debaebc8da7df9773182002ccef9787a/wordcloud-1.9.5-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:7addfa9623317ff7b59de1afe308bfb42db575ef804b271b11777797c0699e02", size = 549386 }, + { url = "https://files.pythonhosted.org/packages/12/9c/026fee4942a6e21074d19929b4833b6d4c1019e7e225d73370e85cba3e91/wordcloud-1.9.5-cp314-cp314-win32.whl", hash = "sha256:0e8c8aab33d9b495656b1315715a31c222f1ec8c9ca40d719372e200fee0204c", size = 297175 }, + { url = "https://files.pythonhosted.org/packages/6b/4e/938315f85438df0e225cb613d783301585bf1adf8d5fe869dca18b029e71/wordcloud-1.9.5-cp314-cp314-win_amd64.whl", hash = "sha256:cda8de69df5fac5a90aea3646993b03b4a920d8aa6454b6f6e58c341397b9ca6", size = 308691 }, + { url = "https://files.pythonhosted.org/packages/55/a2/d04ca5669acddefe29faeb3d7103b6f735b23ebaad82cf73a067561c906b/wordcloud-1.9.5-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:6cee7e8668b1905844a4901597f2dc12ddb97f58586dc520eaf1016a6949cd6e", size = 174155 }, + { url = "https://files.pythonhosted.org/packages/68/05/f77b6ceb7eead741a3b2abbdbad5cb404f1d1297a9708766f452a115341b/wordcloud-1.9.5-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:4c3d3c2477613a61ea671ace84b0acde3b00d0e9afd49636d4bf3e504a3a8a05", size = 174214 }, + { url = "https://files.pythonhosted.org/packages/a6/2c/265936f5efc0edcd9204107f3f5ecf224514370fe4886fe1b7ae35018b63/wordcloud-1.9.5-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4f81f7978cf247a981764b15e90c46adc90a1af8c8983e376dbddd6a94137862", size = 559691 }, + { url = "https://files.pythonhosted.org/packages/31/74/ee7ea5117554e36fcb2ed878d4a271ef8c0af0c3cd4727694d67814c131a/wordcloud-1.9.5-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5c71a569259983733a496df6918c72fbecb480929f8b6514fbe754030b41ab7a", size = 552045 }, + { url = "https://files.pythonhosted.org/packages/55/71/674f39d3a766b1d89c56a9671746653e169fd84251617a848258167a4936/wordcloud-1.9.5-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:c3c1c9b865b8ddf22a9eb529d83395f73151117d06e3e9aeaa1ef0f6db1979af", size = 542971 }, + { url = "https://files.pythonhosted.org/packages/27/8c/613b2f63ed3231ac536e1efb45c0ab73037e53b5f00449ece9664df6b31f/wordcloud-1.9.5-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:c9bd4cbd662774e4d83ffb70cfb4a5de4107ae468247e8b68c4cc87a81dd4efc", size = 546190 }, + { url = "https://files.pythonhosted.org/packages/92/ca/0041d90657e2422c76d75a1f32cced0152251bf9d2e9005975cc69b3c953/wordcloud-1.9.5-cp314-cp314t-win32.whl", hash = "sha256:de4749944686c5cfc10143f718d24c965bfbff48d920273cd5b15e889b89a3ae", size = 306425 }, + { url = "https://files.pythonhosted.org/packages/ea/47/5f27d088000e301d174d33a6dd852f7ea6bae6e914e9971d24a9460fa35e/wordcloud-1.9.5-cp314-cp314t-win_amd64.whl", hash = "sha256:e19c3883165967ad4e0cb7baa9208fdca758cfd0f75d68743a9390269180d47a", size = 320479 }, +] diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/README.md" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/README.md" new file mode 100644 index 0000000..347e13e --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/README.md" @@ -0,0 +1,176 @@ +# Python 爬虫进阶教程 - 源代码 + +本目录包含11个进阶章节的完整示例代码，每个章节都是独立的 uv 项目。 + +## 🚀 快速开始 + +每个章节目录都配置了独立的 `pyproject.toml`，可以使用 uv 快速安装依赖： + +```bash +# 进入任意章节目录 +cd 01_工程化爬虫开发规范 + +# 安装依赖 +uv sync + +# 运行示例代码 +uv run python main.py +``` + +## 📚 章节目录 + +### [01_工程化爬虫开发规范](./01_工程化爬虫开发规范/) +**技术栈**：httpx, pydantic, parsel, loguru +**内容**：日志系统、配置管理、异常处理、项目结构规范 + +### [02_反爬虫对抗基础_请求伪装](./02_反爬虫对抗基础_请求伪装/) +**技术栈**：httpx, loguru, fake-useragent +**内容**：User-Agent轮换、请求头伪装、速率控制、TLS指纹 + +### [03_代理IP的使用与管理](./03_代理IP的使用与管理/) +**技术栈**：httpx, loguru +**内容**：代理池设计、代理检测、多URL测试、代理轮换 +**新增**：✨ 添加了 ipify.org 和 ip-api.com 多URL测试 + +### [04_Playwright浏览器自动化入门](./04_Playwright浏览器自动化入门/) +**技术栈**：playwright, loguru +**内容**：页面操作、元素定位、等待策略、SPA爬取 +**目标网站**：quotes.toscrape.com + +### [05_Playwright进阶_反检测与性能优化](./05_Playwright进阶_反检测与性能优化/) +**技术栈**：playwright, loguru +**内容**：stealth.js注入、CDP模式、资源拦截、上下文池 +**测试网站**：bot.sannysoft.com + +### [06_登录认证_Cookie与Session管理](./06_登录认证_Cookie与Session管理/) +**技术栈**：httpx, loguru +**内容**：Cookie管理、登录状态检测、Cookie轮换 +**新增**：✨ 添加了 quotes.toscrape.com 真实登录演示 + +### [07_登录认证_扫码与短信登录实现](./07_登录认证_扫码与短信登录实现/) +**技术栈**：playwright, loguru +**内容**：扫码登录框架、短信登录框架、登录工厂模式 +**说明**：⚠️ 技术框架示例，需根据实际网站适配 + +### [08_验证码识别与处理](./08_验证码识别与处理/) +**技术栈**：loguru, pillow, ddddocr(可选), opencv(可选) +**内容**：图片验证码OCR、滑块验证码、人类轨迹生成 + +### [09_数据清洗与预处理](./09_数据清洗与预处理/) +**技术栈**：Python标准库 +**内容**：文本清洗、数据标准化、精确/模糊去重 + +### [10_数据分析与可视化](./10_数据分析与可视化/) +**技术栈**：pandas, jieba, wordcloud, matplotlib +**内容**：数据统计分析、词云生成、图表绘制 + +### [11_进阶综合实战项目](./11_进阶综合实战项目/) ⭐ +**技术栈**：playwright, httpx, pydantic, pandas, wordcloud +**内容**：完整的电商数据采集工具 +**目标网站**：✅ books.toscrape.com（专门的爬虫练习网站） +**更新**：✨ 已替换为真实可用的练习网站 + +## 📦 依赖安装 + +### 基础用法 + +```bash +# 1. 进入章节目录 +cd 01_工程化爬虫开发规范 + +# 2. 安装依赖 +uv sync + +# 3. 运行代码 +uv run python main.py +``` + +### 可选依赖 + +某些章节提供了可选功能，可以按需安装： + +```bash +# 章节02 - 安装TLS指纹伪装 +uv sync --extra advanced + +# 章节08 - 安装所有验证码识别功能 +uv sync --extra all + +# 章节10 - 安装交互式图表功能 +uv sync --extra interactive +``` + +### Playwright 特别说明 + +使用 Playwright 的章节（04, 05, 07, 11）需要额外安装浏览器驱动： + +```bash +uv sync +uv run playwright install chromium +``` + +## 🎯 重点更新说明 + +### ✨ 章节03 - 代理IP管理 +新增了多URL测试功能（`demo_multi_url_test()`）： +- httpbin.org/ip - 通用HTTP测试 +- api.ipify.org - 专门IP服务 +- ip-api.com/json/ - IP地理位置信息 + +### ✨ 章节06 - Cookie管理 +新增了真实登录演示（`demo_real_login()`）： +- 演示 quotes.toscrape.com 完整登录流程 +- Cookie保存和复用 +- CookieManager集成 + +### ✨ 章节11 - 综合实战项目 +**重大更新**：将 example.com 替换为真实可用的练习网站： +- **新目标**：http://books.toscrape.com +- **特点**：50页书籍数据，1000+条记录 +- **字段**：标题、价格、评分、库存、链接 +- **状态**：✅ 代码可直接运行 + +## 📖 使用建议 + +1. **按顺序学习**：章节之间有递进关系，建议按顺序学习 +2. **动手实践**：每章都提供了可运行的示例代码 +3. **查看文档**：每个章节的 README.md 提供了详细说明 +4. **遵守规范**：爬虫技术仅用于学习研究，请遵守法律法规 + +## 🛠️ 系统要求 + +- **Python**：>= 3.11 +- **uv**：最新版本（用于依赖管理） +- **操作系统**：支持 macOS / Linux / Windows + +## 📝 常见问题 + +### Q: 如何安装 uv？ +```bash +# macOS/Linux +curl -LsSf https://astral.sh/uv/install.sh | sh + +# Windows +powershell -c "irm https://astral.sh/uv/install.ps1 | iex" +``` + +### Q: Playwright 浏览器安装失败？ +```bash +# 单独安装 Chromium +uv run playwright install chromium + +# 如果网络问题，可以设置镜像 +export PLAYWRIGHT_DOWNLOAD_HOST=https://npmmirror.com/mirrors/playwright/ +uv run playwright install chromium +``` + +### Q: 某些库无法安装？ +部分库可能需要系统级依赖，请查看各章节 README 的说明。 + +## 🤝 贡献 + +欢迎提交 Issue 和 Pull Request 改进教程代码！ + +## 📄 许可证 + +本教程代码仅供学习参考使用。 diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/pyproject.toml" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/pyproject.toml" new file mode 100644 index 0000000..39f07a5 --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/pyproject.toml" @@ -0,0 +1,43 @@ +[project] +name = "crawler-advanced-tutorial" +version = "0.1.0" +description = "爬虫进阶教程 - 包含11个章节的完整学习代码" +readme = "README.md" +requires-python = ">=3.11" +dependencies = [ + # 核心依赖（所有章节共用） + "loguru>=0.7.0", + "httpx>=0.27.0", + "pydantic>=2.0.0", + "pydantic-settings>=2.0.0", + + # 数据解析 + "parsel>=1.9.0", + + # 反爬虫对抗 + "fake-useragent>=1.5.0", + "curl-cffi>=0.7.0", + + # 浏览器自动化 + "playwright>=1.45.0", + + # 验证码处理 + "pillow>=10.0.0", + "ddddocr>=1.4.0", + "opencv-python>=4.9.0", + + # 数据分析与可视化 + "pandas>=2.2.0", + "jieba>=0.42.0", + "wordcloud>=1.9.0", + "matplotlib>=3.8.0", + "pyecharts>=2.0.0", + + # 其他工具 + "cryptography>=42.0.0", + "qrcode[pil]>=7.4.0", + "simhash>=2.1.0", +] + +[tool.uv] +dev-dependencies = [] diff --git "a/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/uv.lock" "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/uv.lock" new file mode 100644 index 0000000..83f21ed --- /dev/null +++ "b/\346\272\220\344\273\243\347\240\201/\347\210\254\350\231\253\350\277\233\351\230\266/uv.lock" @@ -0,0 +1,1674 @@ +version = 1 +revision = 1 +requires-python = ">=3.11" +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version < '3.12'", +] + +[[package]] +name = "annotated-types" +version = "0.7.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/ee/67/531ea369ba64dcff5ec9c3402f9f51bf748cec26dde048a2f973a4eea7f5/annotated_types-0.7.0.tar.gz", hash = "sha256:aff07c09a53a08bc8cfccb9c85b05f1aa9a2a6f23728d790723543408344ce89", size = 16081 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/78/b6/6307fbef88d9b5ee7421e68d78a9f162e0da4900bc5f5793f6d3d0e34fb8/annotated_types-0.7.0-py3-none-any.whl", hash = "sha256:1f02e8b43a8fbbc3f3e0d4f0f4bfc8131bcb4eebe8849b8e5c773f3a1c582a53", size = 13643 }, +] + +[[package]] +name = "anyio" +version = "4.12.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "idna" }, + { name = "typing-extensions", marker = "python_full_version < '3.13'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/96/f0/5eb65b2bb0d09ac6776f2eb54adee6abe8228ea05b20a5ad0e4945de8aac/anyio-4.12.1.tar.gz", hash = "sha256:41cfcc3a4c85d3f05c932da7c26d0201ac36f72abd4435ba90d0464a3ffed703", size = 228685 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/38/0e/27be9fdef66e72d64c0cdc3cc2823101b80585f8119b5c112c2e8f5f7dab/anyio-4.12.1-py3-none-any.whl", hash = "sha256:d405828884fc140aa80a3c667b8beed277f1dfedec42ba031bd6ac3db606ab6c", size = 113592 }, +] + +[[package]] +name = "certifi" +version = "2026.1.4" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/e0/2d/a891ca51311197f6ad14a7ef42e2399f36cf2f9bd44752b3dc4eab60fdc5/certifi-2026.1.4.tar.gz", hash = "sha256:ac726dd470482006e014ad384921ed6438c457018f4b3d204aea4281258b2120", size = 154268 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e6/ad/3cc14f097111b4de0040c83a525973216457bbeeb63739ef1ed275c1c021/certifi-2026.1.4-py3-none-any.whl", hash = "sha256:9943707519e4add1115f44c2bc244f782c0249876bf51b6599fee1ffbedd685c", size = 152900 }, +] + +[[package]] +name = "cffi" +version = "2.0.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "pycparser", marker = "implementation_name != 'PyPy'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/eb/56/b1ba7935a17738ae8453301356628e8147c79dbb825bcbc73dc7401f9846/cffi-2.0.0.tar.gz", hash = "sha256:44d1b5909021139fe36001ae048dbdde8214afa20200eda0f64c068cac5d5529", size = 523588 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/12/4a/3dfd5f7850cbf0d06dc84ba9aa00db766b52ca38d8b86e3a38314d52498c/cffi-2.0.0-cp311-cp311-macosx_10_13_x86_64.whl", hash = "sha256:b4c854ef3adc177950a8dfc81a86f5115d2abd545751a304c5bcf2c2c7283cfe", size = 184344 }, + { url = "https://files.pythonhosted.org/packages/4f/8b/f0e4c441227ba756aafbe78f117485b25bb26b1c059d01f137fa6d14896b/cffi-2.0.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:2de9a304e27f7596cd03d16f1b7c72219bd944e99cc52b84d0145aefb07cbd3c", size = 180560 }, + { url = "https://files.pythonhosted.org/packages/b1/b7/1200d354378ef52ec227395d95c2576330fd22a869f7a70e88e1447eb234/cffi-2.0.0-cp311-cp311-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl", hash = "sha256:baf5215e0ab74c16e2dd324e8ec067ef59e41125d3eade2b863d294fd5035c92", size = 209613 }, + { url = "https://files.pythonhosted.org/packages/b8/56/6033f5e86e8cc9bb629f0077ba71679508bdf54a9a5e112a3c0b91870332/cffi-2.0.0-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:730cacb21e1bdff3ce90babf007d0a0917cc3e6492f336c2f0134101e0944f93", size = 216476 }, + { url = "https://files.pythonhosted.org/packages/dc/7f/55fecd70f7ece178db2f26128ec41430d8720f2d12ca97bf8f0a628207d5/cffi-2.0.0-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:6824f87845e3396029f3820c206e459ccc91760e8fa24422f8b0c3d1731cbec5", size = 203374 }, + { url = "https://files.pythonhosted.org/packages/84/ef/a7b77c8bdc0f77adc3b46888f1ad54be8f3b7821697a7b89126e829e676a/cffi-2.0.0-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:9de40a7b0323d889cf8d23d1ef214f565ab154443c42737dfe52ff82cf857664", size = 202597 }, + { url = "https://files.pythonhosted.org/packages/d7/91/500d892b2bf36529a75b77958edfcd5ad8e2ce4064ce2ecfeab2125d72d1/cffi-2.0.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:8941aaadaf67246224cee8c3803777eed332a19d909b47e29c9842ef1e79ac26", size = 215574 }, + { url = "https://files.pythonhosted.org/packages/44/64/58f6255b62b101093d5df22dcb752596066c7e89dd725e0afaed242a61be/cffi-2.0.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:a05d0c237b3349096d3981b727493e22147f934b20f6f125a3eba8f994bec4a9", size = 218971 }, + { url = "https://files.pythonhosted.org/packages/ab/49/fa72cebe2fd8a55fbe14956f9970fe8eb1ac59e5df042f603ef7c8ba0adc/cffi-2.0.0-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:94698a9c5f91f9d138526b48fe26a199609544591f859c870d477351dc7b2414", size = 211972 }, + { url = "https://files.pythonhosted.org/packages/0b/28/dd0967a76aab36731b6ebfe64dec4e981aff7e0608f60c2d46b46982607d/cffi-2.0.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:5fed36fccc0612a53f1d4d9a816b50a36702c28a2aa880cb8a122b3466638743", size = 217078 }, + { url = "https://files.pythonhosted.org/packages/2b/c0/015b25184413d7ab0a410775fdb4a50fca20f5589b5dab1dbbfa3baad8ce/cffi-2.0.0-cp311-cp311-win32.whl", hash = "sha256:c649e3a33450ec82378822b3dad03cc228b8f5963c0c12fc3b1e0ab940f768a5", size = 172076 }, + { url = "https://files.pythonhosted.org/packages/ae/8f/dc5531155e7070361eb1b7e4c1a9d896d0cb21c49f807a6c03fd63fc877e/cffi-2.0.0-cp311-cp311-win_amd64.whl", hash = "sha256:66f011380d0e49ed280c789fbd08ff0d40968ee7b665575489afa95c98196ab5", size = 182820 }, + { url = "https://files.pythonhosted.org/packages/95/5c/1b493356429f9aecfd56bc171285a4c4ac8697f76e9bbbbb105e537853a1/cffi-2.0.0-cp311-cp311-win_arm64.whl", hash = "sha256:c6638687455baf640e37344fe26d37c404db8b80d037c3d29f58fe8d1c3b194d", size = 177635 }, + { url = "https://files.pythonhosted.org/packages/ea/47/4f61023ea636104d4f16ab488e268b93008c3d0bb76893b1b31db1f96802/cffi-2.0.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:6d02d6655b0e54f54c4ef0b94eb6be0607b70853c45ce98bd278dc7de718be5d", size = 185271 }, + { url = "https://files.pythonhosted.org/packages/df/a2/781b623f57358e360d62cdd7a8c681f074a71d445418a776eef0aadb4ab4/cffi-2.0.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:8eca2a813c1cb7ad4fb74d368c2ffbbb4789d377ee5bb8df98373c2cc0dee76c", size = 181048 }, + { url = "https://files.pythonhosted.org/packages/ff/df/a4f0fbd47331ceeba3d37c2e51e9dfc9722498becbeec2bd8bc856c9538a/cffi-2.0.0-cp312-cp312-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl", hash = "sha256:21d1152871b019407d8ac3985f6775c079416c282e431a4da6afe7aefd2bccbe", size = 212529 }, + { url = "https://files.pythonhosted.org/packages/d5/72/12b5f8d3865bf0f87cf1404d8c374e7487dcf097a1c91c436e72e6badd83/cffi-2.0.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:b21e08af67b8a103c71a250401c78d5e0893beff75e28c53c98f4de42f774062", size = 220097 }, + { url = "https://files.pythonhosted.org/packages/c2/95/7a135d52a50dfa7c882ab0ac17e8dc11cec9d55d2c18dda414c051c5e69e/cffi-2.0.0-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:1e3a615586f05fc4065a8b22b8152f0c1b00cdbc60596d187c2a74f9e3036e4e", size = 207983 }, + { url = "https://files.pythonhosted.org/packages/3a/c8/15cb9ada8895957ea171c62dc78ff3e99159ee7adb13c0123c001a2546c1/cffi-2.0.0-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:81afed14892743bbe14dacb9e36d9e0e504cd204e0b165062c488942b9718037", size = 206519 }, + { url = "https://files.pythonhosted.org/packages/78/2d/7fa73dfa841b5ac06c7b8855cfc18622132e365f5b81d02230333ff26e9e/cffi-2.0.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:3e17ed538242334bf70832644a32a7aae3d83b57567f9fd60a26257e992b79ba", size = 219572 }, + { url = "https://files.pythonhosted.org/packages/07/e0/267e57e387b4ca276b90f0434ff88b2c2241ad72b16d31836adddfd6031b/cffi-2.0.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:3925dd22fa2b7699ed2617149842d2e6adde22b262fcbfada50e3d195e4b3a94", size = 222963 }, + { url = "https://files.pythonhosted.org/packages/b6/75/1f2747525e06f53efbd878f4d03bac5b859cbc11c633d0fb81432d98a795/cffi-2.0.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:2c8f814d84194c9ea681642fd164267891702542f028a15fc97d4674b6206187", size = 221361 }, + { url = "https://files.pythonhosted.org/packages/7b/2b/2b6435f76bfeb6bbf055596976da087377ede68df465419d192acf00c437/cffi-2.0.0-cp312-cp312-win32.whl", hash = "sha256:da902562c3e9c550df360bfa53c035b2f241fed6d9aef119048073680ace4a18", size = 172932 }, + { url = "https://files.pythonhosted.org/packages/f8/ed/13bd4418627013bec4ed6e54283b1959cf6db888048c7cf4b4c3b5b36002/cffi-2.0.0-cp312-cp312-win_amd64.whl", hash = "sha256:da68248800ad6320861f129cd9c1bf96ca849a2771a59e0344e88681905916f5", size = 183557 }, + { url = "https://files.pythonhosted.org/packages/95/31/9f7f93ad2f8eff1dbc1c3656d7ca5bfd8fb52c9d786b4dcf19b2d02217fa/cffi-2.0.0-cp312-cp312-win_arm64.whl", hash = "sha256:4671d9dd5ec934cb9a73e7ee9676f9362aba54f7f34910956b84d727b0d73fb6", size = 177762 }, + { url = "https://files.pythonhosted.org/packages/4b/8d/a0a47a0c9e413a658623d014e91e74a50cdd2c423f7ccfd44086ef767f90/cffi-2.0.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:00bdf7acc5f795150faa6957054fbbca2439db2f775ce831222b66f192f03beb", size = 185230 }, + { url = "https://files.pythonhosted.org/packages/4a/d2/a6c0296814556c68ee32009d9c2ad4f85f2707cdecfd7727951ec228005d/cffi-2.0.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:45d5e886156860dc35862657e1494b9bae8dfa63bf56796f2fb56e1679fc0bca", size = 181043 }, + { url = "https://files.pythonhosted.org/packages/b0/1e/d22cc63332bd59b06481ceaac49d6c507598642e2230f201649058a7e704/cffi-2.0.0-cp313-cp313-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl", hash = "sha256:07b271772c100085dd28b74fa0cd81c8fb1a3ba18b21e03d7c27f3436a10606b", size = 212446 }, + { url = "https://files.pythonhosted.org/packages/a9/f5/a2c23eb03b61a0b8747f211eb716446c826ad66818ddc7810cc2cc19b3f2/cffi-2.0.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:d48a880098c96020b02d5a1f7d9251308510ce8858940e6fa99ece33f610838b", size = 220101 }, + { url = "https://files.pythonhosted.org/packages/f2/7f/e6647792fc5850d634695bc0e6ab4111ae88e89981d35ac269956605feba/cffi-2.0.0-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:f93fd8e5c8c0a4aa1f424d6173f14a892044054871c771f8566e4008eaa359d2", size = 207948 }, + { url = "https://files.pythonhosted.org/packages/cb/1e/a5a1bd6f1fb30f22573f76533de12a00bf274abcdc55c8edab639078abb6/cffi-2.0.0-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:dd4f05f54a52fb558f1ba9f528228066954fee3ebe629fc1660d874d040ae5a3", size = 206422 }, + { url = "https://files.pythonhosted.org/packages/98/df/0a1755e750013a2081e863e7cd37e0cdd02664372c754e5560099eb7aa44/cffi-2.0.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:c8d3b5532fc71b7a77c09192b4a5a200ea992702734a2e9279a37f2478236f26", size = 219499 }, + { url = "https://files.pythonhosted.org/packages/50/e1/a969e687fcf9ea58e6e2a928ad5e2dd88cc12f6f0ab477e9971f2309b57c/cffi-2.0.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:d9b29c1f0ae438d5ee9acb31cadee00a58c46cc9c0b2f9038c6b0b3470877a8c", size = 222928 }, + { url = "https://files.pythonhosted.org/packages/36/54/0362578dd2c9e557a28ac77698ed67323ed5b9775ca9d3fe73fe191bb5d8/cffi-2.0.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:6d50360be4546678fc1b79ffe7a66265e28667840010348dd69a314145807a1b", size = 221302 }, + { url = "https://files.pythonhosted.org/packages/eb/6d/bf9bda840d5f1dfdbf0feca87fbdb64a918a69bca42cfa0ba7b137c48cb8/cffi-2.0.0-cp313-cp313-win32.whl", hash = "sha256:74a03b9698e198d47562765773b4a8309919089150a0bb17d829ad7b44b60d27", size = 172909 }, + { url = "https://files.pythonhosted.org/packages/37/18/6519e1ee6f5a1e579e04b9ddb6f1676c17368a7aba48299c3759bbc3c8b3/cffi-2.0.0-cp313-cp313-win_amd64.whl", hash = "sha256:19f705ada2530c1167abacb171925dd886168931e0a7b78f5bffcae5c6b5be75", size = 183402 }, + { url = "https://files.pythonhosted.org/packages/cb/0e/02ceeec9a7d6ee63bb596121c2c8e9b3a9e150936f4fbef6ca1943e6137c/cffi-2.0.0-cp313-cp313-win_arm64.whl", hash = "sha256:256f80b80ca3853f90c21b23ee78cd008713787b1b1e93eae9f3d6a7134abd91", size = 177780 }, + { url = "https://files.pythonhosted.org/packages/92/c4/3ce07396253a83250ee98564f8d7e9789fab8e58858f35d07a9a2c78de9f/cffi-2.0.0-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:fc33c5141b55ed366cfaad382df24fe7dcbc686de5be719b207bb248e3053dc5", size = 185320 }, + { url = "https://files.pythonhosted.org/packages/59/dd/27e9fa567a23931c838c6b02d0764611c62290062a6d4e8ff7863daf9730/cffi-2.0.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:c654de545946e0db659b3400168c9ad31b5d29593291482c43e3564effbcee13", size = 181487 }, + { url = "https://files.pythonhosted.org/packages/d6/43/0e822876f87ea8a4ef95442c3d766a06a51fc5298823f884ef87aaad168c/cffi-2.0.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:24b6f81f1983e6df8db3adc38562c83f7d4a0c36162885ec7f7b77c7dcbec97b", size = 220049 }, + { url = "https://files.pythonhosted.org/packages/b4/89/76799151d9c2d2d1ead63c2429da9ea9d7aac304603de0c6e8764e6e8e70/cffi-2.0.0-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:12873ca6cb9b0f0d3a0da705d6086fe911591737a59f28b7936bdfed27c0d47c", size = 207793 }, + { url = "https://files.pythonhosted.org/packages/bb/dd/3465b14bb9e24ee24cb88c9e3730f6de63111fffe513492bf8c808a3547e/cffi-2.0.0-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:d9b97165e8aed9272a6bb17c01e3cc5871a594a446ebedc996e2397a1c1ea8ef", size = 206300 }, + { url = "https://files.pythonhosted.org/packages/47/d9/d83e293854571c877a92da46fdec39158f8d7e68da75bf73581225d28e90/cffi-2.0.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:afb8db5439b81cf9c9d0c80404b60c3cc9c3add93e114dcae767f1477cb53775", size = 219244 }, + { url = "https://files.pythonhosted.org/packages/2b/0f/1f177e3683aead2bb00f7679a16451d302c436b5cbf2505f0ea8146ef59e/cffi-2.0.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:737fe7d37e1a1bffe70bd5754ea763a62a066dc5913ca57e957824b72a85e205", size = 222828 }, + { url = "https://files.pythonhosted.org/packages/c6/0f/cafacebd4b040e3119dcb32fed8bdef8dfe94da653155f9d0b9dc660166e/cffi-2.0.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:38100abb9d1b1435bc4cc340bb4489635dc2f0da7456590877030c9b3d40b0c1", size = 220926 }, + { url = "https://files.pythonhosted.org/packages/3e/aa/df335faa45b395396fcbc03de2dfcab242cd61a9900e914fe682a59170b1/cffi-2.0.0-cp314-cp314-win32.whl", hash = "sha256:087067fa8953339c723661eda6b54bc98c5625757ea62e95eb4898ad5e776e9f", size = 175328 }, + { url = "https://files.pythonhosted.org/packages/bb/92/882c2d30831744296ce713f0feb4c1cd30f346ef747b530b5318715cc367/cffi-2.0.0-cp314-cp314-win_amd64.whl", hash = "sha256:203a48d1fb583fc7d78a4c6655692963b860a417c0528492a6bc21f1aaefab25", size = 185650 }, + { url = "https://files.pythonhosted.org/packages/9f/2c/98ece204b9d35a7366b5b2c6539c350313ca13932143e79dc133ba757104/cffi-2.0.0-cp314-cp314-win_arm64.whl", hash = "sha256:dbd5c7a25a7cb98f5ca55d258b103a2054f859a46ae11aaf23134f9cc0d356ad", size = 180687 }, + { url = "https://files.pythonhosted.org/packages/3e/61/c768e4d548bfa607abcda77423448df8c471f25dbe64fb2ef6d555eae006/cffi-2.0.0-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:9a67fc9e8eb39039280526379fb3a70023d77caec1852002b4da7e8b270c4dd9", size = 188773 }, + { url = "https://files.pythonhosted.org/packages/2c/ea/5f76bce7cf6fcd0ab1a1058b5af899bfbef198bea4d5686da88471ea0336/cffi-2.0.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:7a66c7204d8869299919db4d5069a82f1561581af12b11b3c9f48c584eb8743d", size = 185013 }, + { url = "https://files.pythonhosted.org/packages/be/b4/c56878d0d1755cf9caa54ba71e5d049479c52f9e4afc230f06822162ab2f/cffi-2.0.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:7cc09976e8b56f8cebd752f7113ad07752461f48a58cbba644139015ac24954c", size = 221593 }, + { url = "https://files.pythonhosted.org/packages/e0/0d/eb704606dfe8033e7128df5e90fee946bbcb64a04fcdaa97321309004000/cffi-2.0.0-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl", hash = "sha256:92b68146a71df78564e4ef48af17551a5ddd142e5190cdf2c5624d0c3ff5b2e8", size = 209354 }, + { url = "https://files.pythonhosted.org/packages/d8/19/3c435d727b368ca475fb8742ab97c9cb13a0de600ce86f62eab7fa3eea60/cffi-2.0.0-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.whl", hash = "sha256:b1e74d11748e7e98e2f426ab176d4ed720a64412b6a15054378afdb71e0f37dc", size = 208480 }, + { url = "https://files.pythonhosted.org/packages/d0/44/681604464ed9541673e486521497406fadcc15b5217c3e326b061696899a/cffi-2.0.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:28a3a209b96630bca57cce802da70c266eb08c6e97e5afd61a75611ee6c64592", size = 221584 }, + { url = "https://files.pythonhosted.org/packages/25/8e/342a504ff018a2825d395d44d63a767dd8ebc927ebda557fecdaca3ac33a/cffi-2.0.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:7553fb2090d71822f02c629afe6042c299edf91ba1bf94951165613553984512", size = 224443 }, + { url = "https://files.pythonhosted.org/packages/e1/5e/b666bacbbc60fbf415ba9988324a132c9a7a0448a9a8f125074671c0f2c3/cffi-2.0.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:6c6c373cfc5c83a975506110d17457138c8c63016b563cc9ed6e056a82f13ce4", size = 223437 }, + { url = "https://files.pythonhosted.org/packages/a0/1d/ec1a60bd1a10daa292d3cd6bb0b359a81607154fb8165f3ec95fe003b85c/cffi-2.0.0-cp314-cp314t-win32.whl", hash = "sha256:1fc9ea04857caf665289b7a75923f2c6ed559b8298a1b8c49e59f7dd95c8481e", size = 180487 }, + { url = "https://files.pythonhosted.org/packages/bf/41/4c1168c74fac325c0c8156f04b6749c8b6a8f405bbf91413ba088359f60d/cffi-2.0.0-cp314-cp314t-win_amd64.whl", hash = "sha256:d68b6cef7827e8641e8ef16f4494edda8b36104d79773a334beaa1e3521430f6", size = 191726 }, + { url = "https://files.pythonhosted.org/packages/ae/3a/dbeec9d1ee0844c679f6bb5d6ad4e9f198b1224f4e7a32825f47f6192b0c/cffi-2.0.0-cp314-cp314t-win_arm64.whl", hash = "sha256:0a1527a803f0a659de1af2e1fd700213caba79377e27e4693648c2923da066f9", size = 184195 }, +] + +[[package]] +name = "colorama" +version = "0.4.6" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d8/53/6f443c9a4a8358a93a6792e2acffb9d9d5cb0a5cfd8802644b7b1c9a02e4/colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44", size = 27697 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335 }, +] + +[[package]] +name = "coloredlogs" +version = "15.0.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "humanfriendly" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/cc/c7/eed8f27100517e8c0e6b923d5f0845d0cb99763da6fdee00478f91db7325/coloredlogs-15.0.1.tar.gz", hash = "sha256:7c991aa71a4577af2f82600d8f8f3a89f936baeaf9b50a9c197da014e5bf16b0", size = 278520 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a7/06/3d6badcf13db419e25b07041d9c7b4a2c331d3f4e7134445ec5df57714cd/coloredlogs-15.0.1-py2.py3-none-any.whl", hash = "sha256:612ee75c546f53e92e70049c9dbfcc18c935a2b9a53b66085ce9ef6a6e5c0934", size = 46018 }, +] + +[[package]] +name = "contourpy" +version = "1.3.3" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "numpy" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/58/01/1253e6698a07380cd31a736d248a3f2a50a7c88779a1813da27503cadc2a/contourpy-1.3.3.tar.gz", hash = "sha256:083e12155b210502d0bca491432bb04d56dc3432f95a979b429f2848c3dbe880", size = 13466174 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/91/2e/c4390a31919d8a78b90e8ecf87cd4b4c4f05a5b48d05ec17db8e5404c6f4/contourpy-1.3.3-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:709a48ef9a690e1343202916450bc48b9e51c049b089c7f79a267b46cffcdaa1", size = 288773 }, + { url = "https://files.pythonhosted.org/packages/0d/44/c4b0b6095fef4dc9c420e041799591e3b63e9619e3044f7f4f6c21c0ab24/contourpy-1.3.3-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:23416f38bfd74d5d28ab8429cc4d63fa67d5068bd711a85edb1c3fb0c3e2f381", size = 270149 }, + { url = "https://files.pythonhosted.org/packages/30/2e/dd4ced42fefac8470661d7cb7e264808425e6c5d56d175291e93890cce09/contourpy-1.3.3-cp311-cp311-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:929ddf8c4c7f348e4c0a5a3a714b5c8542ffaa8c22954862a46ca1813b667ee7", size = 329222 }, + { url = "https://files.pythonhosted.org/packages/f2/74/cc6ec2548e3d276c71389ea4802a774b7aa3558223b7bade3f25787fafc2/contourpy-1.3.3-cp311-cp311-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:9e999574eddae35f1312c2b4b717b7885d4edd6cb46700e04f7f02db454e67c1", size = 377234 }, + { url = "https://files.pythonhosted.org/packages/03/b3/64ef723029f917410f75c09da54254c5f9ea90ef89b143ccadb09df14c15/contourpy-1.3.3-cp311-cp311-manylinux_2_26_s390x.manylinux_2_28_s390x.whl", hash = "sha256:0bf67e0e3f482cb69779dd3061b534eb35ac9b17f163d851e2a547d56dba0a3a", size = 380555 }, + { url = "https://files.pythonhosted.org/packages/5f/4b/6157f24ca425b89fe2eb7e7be642375711ab671135be21e6faa100f7448c/contourpy-1.3.3-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:51e79c1f7470158e838808d4a996fa9bac72c498e93d8ebe5119bc1e6becb0db", size = 355238 }, + { url = "https://files.pythonhosted.org/packages/98/56/f914f0dd678480708a04cfd2206e7c382533249bc5001eb9f58aa693e200/contourpy-1.3.3-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:598c3aaece21c503615fd59c92a3598b428b2f01bfb4b8ca9c4edeecc2438620", size = 1326218 }, + { url = "https://files.pythonhosted.org/packages/fb/d7/4a972334a0c971acd5172389671113ae82aa7527073980c38d5868ff1161/contourpy-1.3.3-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:322ab1c99b008dad206d406bb61d014cf0174df491ae9d9d0fac6a6fda4f977f", size = 1392867 }, + { url = "https://files.pythonhosted.org/packages/75/3e/f2cc6cd56dc8cff46b1a56232eabc6feea52720083ea71ab15523daab796/contourpy-1.3.3-cp311-cp311-win32.whl", hash = "sha256:fd907ae12cd483cd83e414b12941c632a969171bf90fc937d0c9f268a31cafff", size = 183677 }, + { url = "https://files.pythonhosted.org/packages/98/4b/9bd370b004b5c9d8045c6c33cf65bae018b27aca550a3f657cdc99acdbd8/contourpy-1.3.3-cp311-cp311-win_amd64.whl", hash = "sha256:3519428f6be58431c56581f1694ba8e50626f2dd550af225f82fb5f5814d2a42", size = 225234 }, + { url = "https://files.pythonhosted.org/packages/d9/b6/71771e02c2e004450c12b1120a5f488cad2e4d5b590b1af8bad060360fe4/contourpy-1.3.3-cp311-cp311-win_arm64.whl", hash = "sha256:15ff10bfada4bf92ec8b31c62bf7c1834c244019b4a33095a68000d7075df470", size = 193123 }, + { url = "https://files.pythonhosted.org/packages/be/45/adfee365d9ea3d853550b2e735f9d66366701c65db7855cd07621732ccfc/contourpy-1.3.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:b08a32ea2f8e42cf1d4be3169a98dd4be32bafe4f22b6c4cb4ba810fa9e5d2cb", size = 293419 }, + { url = "https://files.pythonhosted.org/packages/53/3e/405b59cfa13021a56bba395a6b3aca8cec012b45bf177b0eaf7a202cde2c/contourpy-1.3.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:556dba8fb6f5d8742f2923fe9457dbdd51e1049c4a43fd3986a0b14a1d815fc6", size = 273979 }, + { url = "https://files.pythonhosted.org/packages/d4/1c/a12359b9b2ca3a845e8f7f9ac08bdf776114eb931392fcad91743e2ea17b/contourpy-1.3.3-cp312-cp312-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:92d9abc807cf7d0e047b95ca5d957cf4792fcd04e920ca70d48add15c1a90ea7", size = 332653 }, + { url = "https://files.pythonhosted.org/packages/63/12/897aeebfb475b7748ea67b61e045accdfcf0d971f8a588b67108ed7f5512/contourpy-1.3.3-cp312-cp312-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:b2e8faa0ed68cb29af51edd8e24798bb661eac3bd9f65420c1887b6ca89987c8", size = 379536 }, + { url = "https://files.pythonhosted.org/packages/43/8a/a8c584b82deb248930ce069e71576fc09bd7174bbd35183b7943fb1064fd/contourpy-1.3.3-cp312-cp312-manylinux_2_26_s390x.manylinux_2_28_s390x.whl", hash = "sha256:626d60935cf668e70a5ce6ff184fd713e9683fb458898e4249b63be9e28286ea", size = 384397 }, + { url = "https://files.pythonhosted.org/packages/cc/8f/ec6289987824b29529d0dfda0d74a07cec60e54b9c92f3c9da4c0ac732de/contourpy-1.3.3-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4d00e655fcef08aba35ec9610536bfe90267d7ab5ba944f7032549c55a146da1", size = 362601 }, + { url = "https://files.pythonhosted.org/packages/05/0a/a3fe3be3ee2dceb3e615ebb4df97ae6f3828aa915d3e10549ce016302bd1/contourpy-1.3.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:451e71b5a7d597379ef572de31eeb909a87246974d960049a9848c3bc6c41bf7", size = 1331288 }, + { url = "https://files.pythonhosted.org/packages/33/1d/acad9bd4e97f13f3e2b18a3977fe1b4a37ecf3d38d815333980c6c72e963/contourpy-1.3.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:459c1f020cd59fcfe6650180678a9993932d80d44ccde1fa1868977438f0b411", size = 1403386 }, + { url = "https://files.pythonhosted.org/packages/cf/8f/5847f44a7fddf859704217a99a23a4f6417b10e5ab1256a179264561540e/contourpy-1.3.3-cp312-cp312-win32.whl", hash = "sha256:023b44101dfe49d7d53932be418477dba359649246075c996866106da069af69", size = 185018 }, + { url = "https://files.pythonhosted.org/packages/19/e8/6026ed58a64563186a9ee3f29f41261fd1828f527dd93d33b60feca63352/contourpy-1.3.3-cp312-cp312-win_amd64.whl", hash = "sha256:8153b8bfc11e1e4d75bcb0bff1db232f9e10b274e0929de9d608027e0d34ff8b", size = 226567 }, + { url = "https://files.pythonhosted.org/packages/d1/e2/f05240d2c39a1ed228d8328a78b6f44cd695f7ef47beb3e684cf93604f86/contourpy-1.3.3-cp312-cp312-win_arm64.whl", hash = "sha256:07ce5ed73ecdc4a03ffe3e1b3e3c1166db35ae7584be76f65dbbe28a7791b0cc", size = 193655 }, + { url = "https://files.pythonhosted.org/packages/68/35/0167aad910bbdb9599272bd96d01a9ec6852f36b9455cf2ca67bd4cc2d23/contourpy-1.3.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:177fb367556747a686509d6fef71d221a4b198a3905fe824430e5ea0fda54eb5", size = 293257 }, + { url = "https://files.pythonhosted.org/packages/96/e4/7adcd9c8362745b2210728f209bfbcf7d91ba868a2c5f40d8b58f54c509b/contourpy-1.3.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:d002b6f00d73d69333dac9d0b8d5e84d9724ff9ef044fd63c5986e62b7c9e1b1", size = 274034 }, + { url = "https://files.pythonhosted.org/packages/73/23/90e31ceeed1de63058a02cb04b12f2de4b40e3bef5e082a7c18d9c8ae281/contourpy-1.3.3-cp313-cp313-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:348ac1f5d4f1d66d3322420f01d42e43122f43616e0f194fc1c9f5d830c5b286", size = 334672 }, + { url = "https://files.pythonhosted.org/packages/ed/93/b43d8acbe67392e659e1d984700e79eb67e2acb2bd7f62012b583a7f1b55/contourpy-1.3.3-cp313-cp313-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:655456777ff65c2c548b7c454af9c6f33f16c8884f11083244b5819cc214f1b5", size = 381234 }, + { url = "https://files.pythonhosted.org/packages/46/3b/bec82a3ea06f66711520f75a40c8fc0b113b2a75edb36aa633eb11c4f50f/contourpy-1.3.3-cp313-cp313-manylinux_2_26_s390x.manylinux_2_28_s390x.whl", hash = "sha256:644a6853d15b2512d67881586bd03f462c7ab755db95f16f14d7e238f2852c67", size = 385169 }, + { url = "https://files.pythonhosted.org/packages/4b/32/e0f13a1c5b0f8572d0ec6ae2f6c677b7991fafd95da523159c19eff0696a/contourpy-1.3.3-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4debd64f124ca62069f313a9cb86656ff087786016d76927ae2cf37846b006c9", size = 362859 }, + { url = "https://files.pythonhosted.org/packages/33/71/e2a7945b7de4e58af42d708a219f3b2f4cff7386e6b6ab0a0fa0033c49a9/contourpy-1.3.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:a15459b0f4615b00bbd1e91f1b9e19b7e63aea7483d03d804186f278c0af2659", size = 1332062 }, + { url = "https://files.pythonhosted.org/packages/12/fc/4e87ac754220ccc0e807284f88e943d6d43b43843614f0a8afa469801db0/contourpy-1.3.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:ca0fdcd73925568ca027e0b17ab07aad764be4706d0a925b89227e447d9737b7", size = 1403932 }, + { url = "https://files.pythonhosted.org/packages/a6/2e/adc197a37443f934594112222ac1aa7dc9a98faf9c3842884df9a9d8751d/contourpy-1.3.3-cp313-cp313-win32.whl", hash = "sha256:b20c7c9a3bf701366556e1b1984ed2d0cedf999903c51311417cf5f591d8c78d", size = 185024 }, + { url = "https://files.pythonhosted.org/packages/18/0b/0098c214843213759692cc638fce7de5c289200a830e5035d1791d7a2338/contourpy-1.3.3-cp313-cp313-win_amd64.whl", hash = "sha256:1cadd8b8969f060ba45ed7c1b714fe69185812ab43bd6b86a9123fe8f99c3263", size = 226578 }, + { url = "https://files.pythonhosted.org/packages/8a/9a/2f6024a0c5995243cd63afdeb3651c984f0d2bc727fd98066d40e141ad73/contourpy-1.3.3-cp313-cp313-win_arm64.whl", hash = "sha256:fd914713266421b7536de2bfa8181aa8c699432b6763a0ea64195ebe28bff6a9", size = 193524 }, + { url = "https://files.pythonhosted.org/packages/c0/b3/f8a1a86bd3298513f500e5b1f5fd92b69896449f6cab6a146a5d52715479/contourpy-1.3.3-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:88df9880d507169449d434c293467418b9f6cbe82edd19284aa0409e7fdb933d", size = 306730 }, + { url = "https://files.pythonhosted.org/packages/3f/11/4780db94ae62fc0c2053909b65dc3246bd7cecfc4f8a20d957ad43aa4ad8/contourpy-1.3.3-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:d06bb1f751ba5d417047db62bca3c8fde202b8c11fb50742ab3ab962c81e8216", size = 287897 }, + { url = "https://files.pythonhosted.org/packages/ae/15/e59f5f3ffdd6f3d4daa3e47114c53daabcb18574a26c21f03dc9e4e42ff0/contourpy-1.3.3-cp313-cp313t-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e4e6b05a45525357e382909a4c1600444e2a45b4795163d3b22669285591c1ae", size = 326751 }, + { url = "https://files.pythonhosted.org/packages/0f/81/03b45cfad088e4770b1dcf72ea78d3802d04200009fb364d18a493857210/contourpy-1.3.3-cp313-cp313t-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:ab3074b48c4e2cf1a960e6bbeb7f04566bf36b1861d5c9d4d8ac04b82e38ba20", size = 375486 }, + { url = "https://files.pythonhosted.org/packages/0c/ba/49923366492ffbdd4486e970d421b289a670ae8cf539c1ea9a09822b371a/contourpy-1.3.3-cp313-cp313t-manylinux_2_26_s390x.manylinux_2_28_s390x.whl", hash = "sha256:6c3d53c796f8647d6deb1abe867daeb66dcc8a97e8455efa729516b997b8ed99", size = 388106 }, + { url = "https://files.pythonhosted.org/packages/9f/52/5b00ea89525f8f143651f9f03a0df371d3cbd2fccd21ca9b768c7a6500c2/contourpy-1.3.3-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:50ed930df7289ff2a8d7afeb9603f8289e5704755c7e5c3bbd929c90c817164b", size = 352548 }, + { url = "https://files.pythonhosted.org/packages/32/1d/a209ec1a3a3452d490f6b14dd92e72280c99ae3d1e73da74f8277d4ee08f/contourpy-1.3.3-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:4feffb6537d64b84877da813a5c30f1422ea5739566abf0bd18065ac040e120a", size = 1322297 }, + { url = "https://files.pythonhosted.org/packages/bc/9e/46f0e8ebdd884ca0e8877e46a3f4e633f6c9c8c4f3f6e72be3fe075994aa/contourpy-1.3.3-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:2b7e9480ffe2b0cd2e787e4df64270e3a0440d9db8dc823312e2c940c167df7e", size = 1391023 }, + { url = "https://files.pythonhosted.org/packages/b9/70/f308384a3ae9cd2209e0849f33c913f658d3326900d0ff5d378d6a1422d2/contourpy-1.3.3-cp313-cp313t-win32.whl", hash = "sha256:283edd842a01e3dcd435b1c5116798d661378d83d36d337b8dde1d16a5fc9ba3", size = 196157 }, + { url = "https://files.pythonhosted.org/packages/b2/dd/880f890a6663b84d9e34a6f88cded89d78f0091e0045a284427cb6b18521/contourpy-1.3.3-cp313-cp313t-win_amd64.whl", hash = "sha256:87acf5963fc2b34825e5b6b048f40e3635dd547f590b04d2ab317c2619ef7ae8", size = 240570 }, + { url = "https://files.pythonhosted.org/packages/80/99/2adc7d8ffead633234817ef8e9a87115c8a11927a94478f6bb3d3f4d4f7d/contourpy-1.3.3-cp313-cp313t-win_arm64.whl", hash = "sha256:3c30273eb2a55024ff31ba7d052dde990d7d8e5450f4bbb6e913558b3d6c2301", size = 199713 }, + { url = "https://files.pythonhosted.org/packages/72/8b/4546f3ab60f78c514ffb7d01a0bd743f90de36f0019d1be84d0a708a580a/contourpy-1.3.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:fde6c716d51c04b1c25d0b90364d0be954624a0ee9d60e23e850e8d48353d07a", size = 292189 }, + { url = "https://files.pythonhosted.org/packages/fd/e1/3542a9cb596cadd76fcef413f19c79216e002623158befe6daa03dbfa88c/contourpy-1.3.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:cbedb772ed74ff5be440fa8eee9bd49f64f6e3fc09436d9c7d8f1c287b121d77", size = 273251 }, + { url = "https://files.pythonhosted.org/packages/b1/71/f93e1e9471d189f79d0ce2497007731c1e6bf9ef6d1d61b911430c3db4e5/contourpy-1.3.3-cp314-cp314-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:22e9b1bd7a9b1d652cd77388465dc358dafcd2e217d35552424aa4f996f524f5", size = 335810 }, + { url = "https://files.pythonhosted.org/packages/91/f9/e35f4c1c93f9275d4e38681a80506b5510e9327350c51f8d4a5a724d178c/contourpy-1.3.3-cp314-cp314-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:a22738912262aa3e254e4f3cb079a95a67132fc5a063890e224393596902f5a4", size = 382871 }, + { url = "https://files.pythonhosted.org/packages/b5/71/47b512f936f66a0a900d81c396a7e60d73419868fba959c61efed7a8ab46/contourpy-1.3.3-cp314-cp314-manylinux_2_26_s390x.manylinux_2_28_s390x.whl", hash = "sha256:afe5a512f31ee6bd7d0dda52ec9864c984ca3d66664444f2d72e0dc4eb832e36", size = 386264 }, + { url = "https://files.pythonhosted.org/packages/04/5f/9ff93450ba96b09c7c2b3f81c94de31c89f92292f1380261bd7195bea4ea/contourpy-1.3.3-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f64836de09927cba6f79dcd00fdd7d5329f3fccc633468507079c829ca4db4e3", size = 363819 }, + { url = "https://files.pythonhosted.org/packages/3e/a6/0b185d4cc480ee494945cde102cb0149ae830b5fa17bf855b95f2e70ad13/contourpy-1.3.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:1fd43c3be4c8e5fd6e4f2baeae35ae18176cf2e5cced681cca908addf1cdd53b", size = 1333650 }, + { url = "https://files.pythonhosted.org/packages/43/d7/afdc95580ca56f30fbcd3060250f66cedbde69b4547028863abd8aa3b47e/contourpy-1.3.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:6afc576f7b33cf00996e5c1102dc2a8f7cc89e39c0b55df93a0b78c1bd992b36", size = 1404833 }, + { url = "https://files.pythonhosted.org/packages/e2/e2/366af18a6d386f41132a48f033cbd2102e9b0cf6345d35ff0826cd984566/contourpy-1.3.3-cp314-cp314-win32.whl", hash = "sha256:66c8a43a4f7b8df8b71ee1840e4211a3c8d93b214b213f590e18a1beca458f7d", size = 189692 }, + { url = "https://files.pythonhosted.org/packages/7d/c2/57f54b03d0f22d4044b8afb9ca0e184f8b1afd57b4f735c2fa70883dc601/contourpy-1.3.3-cp314-cp314-win_amd64.whl", hash = "sha256:cf9022ef053f2694e31d630feaacb21ea24224be1c3ad0520b13d844274614fd", size = 232424 }, + { url = "https://files.pythonhosted.org/packages/18/79/a9416650df9b525737ab521aa181ccc42d56016d2123ddcb7b58e926a42c/contourpy-1.3.3-cp314-cp314-win_arm64.whl", hash = "sha256:95b181891b4c71de4bb404c6621e7e2390745f887f2a026b2d99e92c17892339", size = 198300 }, + { url = "https://files.pythonhosted.org/packages/1f/42/38c159a7d0f2b7b9c04c64ab317042bb6952b713ba875c1681529a2932fe/contourpy-1.3.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:33c82d0138c0a062380332c861387650c82e4cf1747aaa6938b9b6516762e772", size = 306769 }, + { url = "https://files.pythonhosted.org/packages/c3/6c/26a8205f24bca10974e77460de68d3d7c63e282e23782f1239f226fcae6f/contourpy-1.3.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:ea37e7b45949df430fe649e5de8351c423430046a2af20b1c1961cae3afcda77", size = 287892 }, + { url = "https://files.pythonhosted.org/packages/66/06/8a475c8ab718ebfd7925661747dbb3c3ee9c82ac834ccb3570be49d129f4/contourpy-1.3.3-cp314-cp314t-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d304906ecc71672e9c89e87c4675dc5c2645e1f4269a5063b99b0bb29f232d13", size = 326748 }, + { url = "https://files.pythonhosted.org/packages/b4/a3/c5ca9f010a44c223f098fccd8b158bb1cb287378a31ac141f04730dc49be/contourpy-1.3.3-cp314-cp314t-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:ca658cd1a680a5c9ea96dc61cdbae1e85c8f25849843aa799dfd3cb370ad4fbe", size = 375554 }, + { url = "https://files.pythonhosted.org/packages/80/5b/68bd33ae63fac658a4145088c1e894405e07584a316738710b636c6d0333/contourpy-1.3.3-cp314-cp314t-manylinux_2_26_s390x.manylinux_2_28_s390x.whl", hash = "sha256:ab2fd90904c503739a75b7c8c5c01160130ba67944a7b77bbf36ef8054576e7f", size = 388118 }, + { url = "https://files.pythonhosted.org/packages/40/52/4c285a6435940ae25d7410a6c36bda5145839bc3f0beb20c707cda18b9d2/contourpy-1.3.3-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b7301b89040075c30e5768810bc96a8e8d78085b47d8be6e4c3f5a0b4ed478a0", size = 352555 }, + { url = "https://files.pythonhosted.org/packages/24/ee/3e81e1dd174f5c7fefe50e85d0892de05ca4e26ef1c9a59c2a57e43b865a/contourpy-1.3.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:2a2a8b627d5cc6b7c41a4beff6c5ad5eb848c88255fda4a8745f7e901b32d8e4", size = 1322295 }, + { url = "https://files.pythonhosted.org/packages/3c/b2/6d913d4d04e14379de429057cd169e5e00f6c2af3bb13e1710bcbdb5da12/contourpy-1.3.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:fd6ec6be509c787f1caf6b247f0b1ca598bef13f4ddeaa126b7658215529ba0f", size = 1391027 }, + { url = "https://files.pythonhosted.org/packages/93/8a/68a4ec5c55a2971213d29a9374913f7e9f18581945a7a31d1a39b5d2dfe5/contourpy-1.3.3-cp314-cp314t-win32.whl", hash = "sha256:e74a9a0f5e3fff48fb5a7f2fd2b9b70a3fe014a67522f79b7cca4c0c7e43c9ae", size = 202428 }, + { url = "https://files.pythonhosted.org/packages/fa/96/fd9f641ffedc4fa3ace923af73b9d07e869496c9cc7a459103e6e978992f/contourpy-1.3.3-cp314-cp314t-win_amd64.whl", hash = "sha256:13b68d6a62db8eafaebb8039218921399baf6e47bf85006fd8529f2a08ef33fc", size = 250331 }, + { url = "https://files.pythonhosted.org/packages/ae/8c/469afb6465b853afff216f9528ffda78a915ff880ed58813ba4faf4ba0b6/contourpy-1.3.3-cp314-cp314t-win_arm64.whl", hash = "sha256:b7448cb5a725bb1e35ce88771b86fba35ef418952474492cf7c764059933ff8b", size = 203831 }, + { url = "https://files.pythonhosted.org/packages/a5/29/8dcfe16f0107943fa92388c23f6e05cff0ba58058c4c95b00280d4c75a14/contourpy-1.3.3-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:cd5dfcaeb10f7b7f9dc8941717c6c2ade08f587be2226222c12b25f0483ed497", size = 278809 }, + { url = "https://files.pythonhosted.org/packages/85/a9/8b37ef4f7dafeb335daee3c8254645ef5725be4d9c6aa70b50ec46ef2f7e/contourpy-1.3.3-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:0c1fc238306b35f246d61a1d416a627348b5cf0648648a031e14bb8705fcdfe8", size = 261593 }, + { url = "https://files.pythonhosted.org/packages/0a/59/ebfb8c677c75605cc27f7122c90313fd2f375ff3c8d19a1694bda74aaa63/contourpy-1.3.3-pp311-pypy311_pp73-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:70f9aad7de812d6541d29d2bbf8feb22ff7e1c299523db288004e3157ff4674e", size = 302202 }, + { url = "https://files.pythonhosted.org/packages/3c/37/21972a15834d90bfbfb009b9d004779bd5a07a0ec0234e5ba8f64d5736f4/contourpy-1.3.3-pp311-pypy311_pp73-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5ed3657edf08512fc3fe81b510e35c2012fbd3081d2e26160f27ca28affec989", size = 329207 }, + { url = "https://files.pythonhosted.org/packages/0c/58/bd257695f39d05594ca4ad60df5bcb7e32247f9951fd09a9b8edb82d1daa/contourpy-1.3.3-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:3d1a3799d62d45c18bafd41c5fa05120b96a28079f2393af559b843d1a966a77", size = 225315 }, +] + +[[package]] +name = "crawler-advanced-tutorial" +version = "0.1.0" +source = { virtual = "." } +dependencies = [ + { name = "cryptography" }, + { name = "curl-cffi" }, + { name = "ddddocr" }, + { name = "fake-useragent" }, + { name = "httpx" }, + { name = "jieba" }, + { name = "loguru" }, + { name = "matplotlib" }, + { name = "opencv-python" }, + { name = "pandas" }, + { name = "parsel" }, + { name = "pillow" }, + { name = "playwright" }, + { name = "pydantic" }, + { name = "pydantic-settings" }, + { name = "pyecharts" }, + { name = "qrcode", extra = ["pil"] }, + { name = "simhash" }, + { name = "wordcloud" }, +] + +[package.metadata] +requires-dist = [ + { name = "cryptography", specifier = ">=42.0.0" }, + { name = "curl-cffi", specifier = ">=0.7.0" }, + { name = "ddddocr", specifier = ">=1.4.0" }, + { name = "fake-useragent", specifier = ">=1.5.0" }, + { name = "httpx", specifier = ">=0.27.0" }, + { name = "jieba", specifier = ">=0.42.0" }, + { name = "loguru", specifier = ">=0.7.0" }, + { name = "matplotlib", specifier = ">=3.8.0" }, + { name = "opencv-python", specifier = ">=4.9.0" }, + { name = "pandas", specifier = ">=2.2.0" }, + { name = "parsel", specifier = ">=1.9.0" }, + { name = "pillow", specifier = ">=10.0.0" }, + { name = "playwright", specifier = ">=1.45.0" }, + { name = "pydantic", specifier = ">=2.0.0" }, + { name = "pydantic-settings", specifier = ">=2.0.0" }, + { name = "pyecharts", specifier = ">=2.0.0" }, + { name = "qrcode", extras = ["pil"], specifier = ">=7.4.0" }, + { name = "simhash", specifier = ">=2.1.0" }, + { name = "wordcloud", specifier = ">=1.9.0" }, +] + +[package.metadata.requires-dev] +dev = [] + +[[package]] +name = "cryptography" +version = "46.0.3" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "cffi", marker = "platform_python_implementation != 'PyPy'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/9f/33/c00162f49c0e2fe8064a62cb92b93e50c74a72bc370ab92f86112b33ff62/cryptography-46.0.3.tar.gz", hash = "sha256:a8b17438104fed022ce745b362294d9ce35b4c2e45c1d958ad4a4b019285f4a1", size = 749258 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/1d/42/9c391dd801d6cf0d561b5890549d4b27bafcc53b39c31a817e69d87c625b/cryptography-46.0.3-cp311-abi3-macosx_10_9_universal2.whl", hash = "sha256:109d4ddfadf17e8e7779c39f9b18111a09efb969a301a31e987416a0191ed93a", size = 7225004 }, + { url = "https://files.pythonhosted.org/packages/1c/67/38769ca6b65f07461eb200e85fc1639b438bdc667be02cf7f2cd6a64601c/cryptography-46.0.3-cp311-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:09859af8466b69bc3c27bdf4f5d84a665e0f7ab5088412e9e2ec49758eca5cbc", size = 4296667 }, + { url = "https://files.pythonhosted.org/packages/5c/49/498c86566a1d80e978b42f0d702795f69887005548c041636df6ae1ca64c/cryptography-46.0.3-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:01ca9ff2885f3acc98c29f1860552e37f6d7c7d013d7334ff2a9de43a449315d", size = 4450807 }, + { url = "https://files.pythonhosted.org/packages/4b/0a/863a3604112174c8624a2ac3c038662d9e59970c7f926acdcfaed8d61142/cryptography-46.0.3-cp311-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:6eae65d4c3d33da080cff9c4ab1f711b15c1d9760809dad6ea763f3812d254cb", size = 4299615 }, + { url = "https://files.pythonhosted.org/packages/64/02/b73a533f6b64a69f3cd3872acb6ebc12aef924d8d103133bb3ea750dc703/cryptography-46.0.3-cp311-abi3-manylinux_2_28_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:e5bf0ed4490068a2e72ac03d786693adeb909981cc596425d09032d372bcc849", size = 4016800 }, + { url = "https://files.pythonhosted.org/packages/25/d5/16e41afbfa450cde85a3b7ec599bebefaef16b5c6ba4ec49a3532336ed72/cryptography-46.0.3-cp311-abi3-manylinux_2_28_ppc64le.whl", hash = "sha256:5ecfccd2329e37e9b7112a888e76d9feca2347f12f37918facbb893d7bb88ee8", size = 4984707 }, + { url = "https://files.pythonhosted.org/packages/c9/56/e7e69b427c3878352c2fb9b450bd0e19ed552753491d39d7d0a2f5226d41/cryptography-46.0.3-cp311-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:a2c0cd47381a3229c403062f764160d57d4d175e022c1df84e168c6251a22eec", size = 4482541 }, + { url = "https://files.pythonhosted.org/packages/78/f6/50736d40d97e8483172f1bb6e698895b92a223dba513b0ca6f06b2365339/cryptography-46.0.3-cp311-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:549e234ff32571b1f4076ac269fcce7a808d3bf98b76c8dd560e42dbc66d7d91", size = 4299464 }, + { url = "https://files.pythonhosted.org/packages/00/de/d8e26b1a855f19d9994a19c702fa2e93b0456beccbcfe437eda00e0701f2/cryptography-46.0.3-cp311-abi3-manylinux_2_34_ppc64le.whl", hash = "sha256:c0a7bb1a68a5d3471880e264621346c48665b3bf1c3759d682fc0864c540bd9e", size = 4950838 }, + { url = "https://files.pythonhosted.org/packages/8f/29/798fc4ec461a1c9e9f735f2fc58741b0daae30688f41b2497dcbc9ed1355/cryptography-46.0.3-cp311-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:10b01676fc208c3e6feeb25a8b83d81767e8059e1fe86e1dc62d10a3018fa926", size = 4481596 }, + { url = "https://files.pythonhosted.org/packages/15/8d/03cd48b20a573adfff7652b76271078e3045b9f49387920e7f1f631d125e/cryptography-46.0.3-cp311-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:0abf1ffd6e57c67e92af68330d05760b7b7efb243aab8377e583284dbab72c71", size = 4426782 }, + { url = "https://files.pythonhosted.org/packages/fa/b1/ebacbfe53317d55cf33165bda24c86523497a6881f339f9aae5c2e13e57b/cryptography-46.0.3-cp311-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:a04bee9ab6a4da801eb9b51f1b708a1b5b5c9eb48c03f74198464c66f0d344ac", size = 4698381 }, + { url = "https://files.pythonhosted.org/packages/96/92/8a6a9525893325fc057a01f654d7efc2c64b9de90413adcf605a85744ff4/cryptography-46.0.3-cp311-abi3-win32.whl", hash = "sha256:f260d0d41e9b4da1ed1e0f1ce571f97fe370b152ab18778e9e8f67d6af432018", size = 3055988 }, + { url = "https://files.pythonhosted.org/packages/7e/bf/80fbf45253ea585a1e492a6a17efcb93467701fa79e71550a430c5e60df0/cryptography-46.0.3-cp311-abi3-win_amd64.whl", hash = "sha256:a9a3008438615669153eb86b26b61e09993921ebdd75385ddd748702c5adfddb", size = 3514451 }, + { url = "https://files.pythonhosted.org/packages/2e/af/9b302da4c87b0beb9db4e756386a7c6c5b8003cd0e742277888d352ae91d/cryptography-46.0.3-cp311-abi3-win_arm64.whl", hash = "sha256:5d7f93296ee28f68447397bf5198428c9aeeab45705a55d53a6343455dcb2c3c", size = 2928007 }, + { url = "https://files.pythonhosted.org/packages/f5/e2/a510aa736755bffa9d2f75029c229111a1d02f8ecd5de03078f4c18d91a3/cryptography-46.0.3-cp314-cp314t-macosx_10_9_universal2.whl", hash = "sha256:00a5e7e87938e5ff9ff5447ab086a5706a957137e6e433841e9d24f38a065217", size = 7158012 }, + { url = "https://files.pythonhosted.org/packages/73/dc/9aa866fbdbb95b02e7f9d086f1fccfeebf8953509b87e3f28fff927ff8a0/cryptography-46.0.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:c8daeb2d2174beb4575b77482320303f3d39b8e81153da4f0fb08eb5fe86a6c5", size = 4288728 }, + { url = "https://files.pythonhosted.org/packages/c5/fd/bc1daf8230eaa075184cbbf5f8cd00ba9db4fd32d63fb83da4671b72ed8a/cryptography-46.0.3-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:39b6755623145ad5eff1dab323f4eae2a32a77a7abef2c5089a04a3d04366715", size = 4435078 }, + { url = "https://files.pythonhosted.org/packages/82/98/d3bd5407ce4c60017f8ff9e63ffee4200ab3e23fe05b765cab805a7db008/cryptography-46.0.3-cp314-cp314t-manylinux_2_28_aarch64.whl", hash = "sha256:db391fa7c66df6762ee3f00c95a89e6d428f4d60e7abc8328f4fe155b5ac6e54", size = 4293460 }, + { url = "https://files.pythonhosted.org/packages/26/e9/e23e7900983c2b8af7a08098db406cf989d7f09caea7897e347598d4cd5b/cryptography-46.0.3-cp314-cp314t-manylinux_2_28_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:78a97cf6a8839a48c49271cdcbd5cf37ca2c1d6b7fdd86cc864f302b5e9bf459", size = 3995237 }, + { url = "https://files.pythonhosted.org/packages/91/15/af68c509d4a138cfe299d0d7ddb14afba15233223ebd933b4bbdbc7155d3/cryptography-46.0.3-cp314-cp314t-manylinux_2_28_ppc64le.whl", hash = "sha256:dfb781ff7eaa91a6f7fd41776ec37c5853c795d3b358d4896fdbb5df168af422", size = 4967344 }, + { url = "https://files.pythonhosted.org/packages/ca/e3/8643d077c53868b681af077edf6b3cb58288b5423610f21c62aadcbe99f4/cryptography-46.0.3-cp314-cp314t-manylinux_2_28_x86_64.whl", hash = "sha256:6f61efb26e76c45c4a227835ddeae96d83624fb0d29eb5df5b96e14ed1a0afb7", size = 4466564 }, + { url = "https://files.pythonhosted.org/packages/0e/43/c1e8726fa59c236ff477ff2b5dc071e54b21e5a1e51aa2cee1676f1c986f/cryptography-46.0.3-cp314-cp314t-manylinux_2_34_aarch64.whl", hash = "sha256:23b1a8f26e43f47ceb6d6a43115f33a5a37d57df4ea0ca295b780ae8546e8044", size = 4292415 }, + { url = "https://files.pythonhosted.org/packages/42/f9/2f8fefdb1aee8a8e3256a0568cffc4e6d517b256a2fe97a029b3f1b9fe7e/cryptography-46.0.3-cp314-cp314t-manylinux_2_34_ppc64le.whl", hash = "sha256:b419ae593c86b87014b9be7396b385491ad7f320bde96826d0dd174459e54665", size = 4931457 }, + { url = "https://files.pythonhosted.org/packages/79/30/9b54127a9a778ccd6d27c3da7563e9f2d341826075ceab89ae3b41bf5be2/cryptography-46.0.3-cp314-cp314t-manylinux_2_34_x86_64.whl", hash = "sha256:50fc3343ac490c6b08c0cf0d704e881d0d660be923fd3076db3e932007e726e3", size = 4466074 }, + { url = "https://files.pythonhosted.org/packages/ac/68/b4f4a10928e26c941b1b6a179143af9f4d27d88fe84a6a3c53592d2e76bf/cryptography-46.0.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:22d7e97932f511d6b0b04f2bfd818d73dcd5928db509460aaf48384778eb6d20", size = 4420569 }, + { url = "https://files.pythonhosted.org/packages/a3/49/3746dab4c0d1979888f125226357d3262a6dd40e114ac29e3d2abdf1ec55/cryptography-46.0.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:d55f3dffadd674514ad19451161118fd010988540cee43d8bc20675e775925de", size = 4681941 }, + { url = "https://files.pythonhosted.org/packages/fd/30/27654c1dbaf7e4a3531fa1fc77986d04aefa4d6d78259a62c9dc13d7ad36/cryptography-46.0.3-cp314-cp314t-win32.whl", hash = "sha256:8a6e050cb6164d3f830453754094c086ff2d0b2f3a897a1d9820f6139a1f0914", size = 3022339 }, + { url = "https://files.pythonhosted.org/packages/f6/30/640f34ccd4d2a1bc88367b54b926b781b5a018d65f404d409aba76a84b1c/cryptography-46.0.3-cp314-cp314t-win_amd64.whl", hash = "sha256:760f83faa07f8b64e9c33fc963d790a2edb24efb479e3520c14a45741cd9b2db", size = 3494315 }, + { url = "https://files.pythonhosted.org/packages/ba/8b/88cc7e3bd0a8e7b861f26981f7b820e1f46aa9d26cc482d0feba0ecb4919/cryptography-46.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:516ea134e703e9fe26bcd1277a4b59ad30586ea90c365a87781d7887a646fe21", size = 2919331 }, + { url = "https://files.pythonhosted.org/packages/fd/23/45fe7f376a7df8daf6da3556603b36f53475a99ce4faacb6ba2cf3d82021/cryptography-46.0.3-cp38-abi3-macosx_10_9_universal2.whl", hash = "sha256:cb3d760a6117f621261d662bccc8ef5bc32ca673e037c83fbe565324f5c46936", size = 7218248 }, + { url = "https://files.pythonhosted.org/packages/27/32/b68d27471372737054cbd34c84981f9edbc24fe67ca225d389799614e27f/cryptography-46.0.3-cp38-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:4b7387121ac7d15e550f5cb4a43aef2559ed759c35df7336c402bb8275ac9683", size = 4294089 }, + { url = "https://files.pythonhosted.org/packages/26/42/fa8389d4478368743e24e61eea78846a0006caffaf72ea24a15159215a14/cryptography-46.0.3-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:15ab9b093e8f09daab0f2159bb7e47532596075139dd74365da52ecc9cb46c5d", size = 4440029 }, + { url = "https://files.pythonhosted.org/packages/5f/eb/f483db0ec5ac040824f269e93dd2bd8a21ecd1027e77ad7bdf6914f2fd80/cryptography-46.0.3-cp38-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:46acf53b40ea38f9c6c229599a4a13f0d46a6c3fa9ef19fc1a124d62e338dfa0", size = 4297222 }, + { url = "https://files.pythonhosted.org/packages/fd/cf/da9502c4e1912cb1da3807ea3618a6829bee8207456fbbeebc361ec38ba3/cryptography-46.0.3-cp38-abi3-manylinux_2_28_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:10ca84c4668d066a9878890047f03546f3ae0a6b8b39b697457b7757aaf18dbc", size = 4012280 }, + { url = "https://files.pythonhosted.org/packages/6b/8f/9adb86b93330e0df8b3dcf03eae67c33ba89958fc2e03862ef1ac2b42465/cryptography-46.0.3-cp38-abi3-manylinux_2_28_ppc64le.whl", hash = "sha256:36e627112085bb3b81b19fed209c05ce2a52ee8b15d161b7c643a7d5a88491f3", size = 4978958 }, + { url = "https://files.pythonhosted.org/packages/d1/a0/5fa77988289c34bdb9f913f5606ecc9ada1adb5ae870bd0d1054a7021cc4/cryptography-46.0.3-cp38-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:1000713389b75c449a6e979ffc7dcc8ac90b437048766cef052d4d30b8220971", size = 4473714 }, + { url = "https://files.pythonhosted.org/packages/14/e5/fc82d72a58d41c393697aa18c9abe5ae1214ff6f2a5c18ac470f92777895/cryptography-46.0.3-cp38-abi3-manylinux_2_34_aarch64.whl", hash = "sha256:b02cf04496f6576afffef5ddd04a0cb7d49cf6be16a9059d793a30b035f6b6ac", size = 4296970 }, + { url = "https://files.pythonhosted.org/packages/78/06/5663ed35438d0b09056973994f1aec467492b33bd31da36e468b01ec1097/cryptography-46.0.3-cp38-abi3-manylinux_2_34_ppc64le.whl", hash = "sha256:71e842ec9bc7abf543b47cf86b9a743baa95f4677d22baa4c7d5c69e49e9bc04", size = 4940236 }, + { url = "https://files.pythonhosted.org/packages/fc/59/873633f3f2dcd8a053b8dd1d38f783043b5fce589c0f6988bf55ef57e43e/cryptography-46.0.3-cp38-abi3-manylinux_2_34_x86_64.whl", hash = "sha256:402b58fc32614f00980b66d6e56a5b4118e6cb362ae8f3fda141ba4689bd4506", size = 4472642 }, + { url = "https://files.pythonhosted.org/packages/3d/39/8e71f3930e40f6877737d6f69248cf74d4e34b886a3967d32f919cc50d3b/cryptography-46.0.3-cp38-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:ef639cb3372f69ec44915fafcd6698b6cc78fbe0c2ea41be867f6ed612811963", size = 4423126 }, + { url = "https://files.pythonhosted.org/packages/cd/c7/f65027c2810e14c3e7268353b1681932b87e5a48e65505d8cc17c99e36ae/cryptography-46.0.3-cp38-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:3b51b8ca4f1c6453d8829e1eb7299499ca7f313900dd4d89a24b8b87c0a780d4", size = 4686573 }, + { url = "https://files.pythonhosted.org/packages/0a/6e/1c8331ddf91ca4730ab3086a0f1be19c65510a33b5a441cb334e7a2d2560/cryptography-46.0.3-cp38-abi3-win32.whl", hash = "sha256:6276eb85ef938dc035d59b87c8a7dc559a232f954962520137529d77b18ff1df", size = 3036695 }, + { url = "https://files.pythonhosted.org/packages/90/45/b0d691df20633eff80955a0fc7695ff9051ffce8b69741444bd9ed7bd0db/cryptography-46.0.3-cp38-abi3-win_amd64.whl", hash = "sha256:416260257577718c05135c55958b674000baef9a1c7d9e8f306ec60d71db850f", size = 3501720 }, + { url = "https://files.pythonhosted.org/packages/e8/cb/2da4cc83f5edb9c3257d09e1e7ab7b23f049c7962cae8d842bbef0a9cec9/cryptography-46.0.3-cp38-abi3-win_arm64.whl", hash = "sha256:d89c3468de4cdc4f08a57e214384d0471911a3830fcdaf7a8cc587e42a866372", size = 2918740 }, + { url = "https://files.pythonhosted.org/packages/06/8a/e60e46adab4362a682cf142c7dcb5bf79b782ab2199b0dcb81f55970807f/cryptography-46.0.3-pp311-pypy311_pp73-macosx_10_9_x86_64.whl", hash = "sha256:7ce938a99998ed3c8aa7e7272dca1a610401ede816d36d0693907d863b10d9ea", size = 3698132 }, + { url = "https://files.pythonhosted.org/packages/da/38/f59940ec4ee91e93d3311f7532671a5cef5570eb04a144bf203b58552d11/cryptography-46.0.3-pp311-pypy311_pp73-manylinux_2_28_aarch64.whl", hash = "sha256:191bb60a7be5e6f54e30ba16fdfae78ad3a342a0599eb4193ba88e3f3d6e185b", size = 4243992 }, + { url = "https://files.pythonhosted.org/packages/b0/0c/35b3d92ddebfdfda76bb485738306545817253d0a3ded0bfe80ef8e67aa5/cryptography-46.0.3-pp311-pypy311_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:c70cc23f12726be8f8bc72e41d5065d77e4515efae3690326764ea1b07845cfb", size = 4409944 }, + { url = "https://files.pythonhosted.org/packages/99/55/181022996c4063fc0e7666a47049a1ca705abb9c8a13830f074edb347495/cryptography-46.0.3-pp311-pypy311_pp73-manylinux_2_34_aarch64.whl", hash = "sha256:9394673a9f4de09e28b5356e7fff97d778f8abad85c9d5ac4a4b7e25a0de7717", size = 4242957 }, + { url = "https://files.pythonhosted.org/packages/ba/af/72cd6ef29f9c5f731251acadaeb821559fe25f10852f44a63374c9ca08c1/cryptography-46.0.3-pp311-pypy311_pp73-manylinux_2_34_x86_64.whl", hash = "sha256:94cd0549accc38d1494e1f8de71eca837d0509d0d44bf11d158524b0e12cebf9", size = 4409447 }, + { url = "https://files.pythonhosted.org/packages/0d/c3/e90f4a4feae6410f914f8ebac129b9ae7a8c92eb60a638012dde42030a9d/cryptography-46.0.3-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:6b5063083824e5509fdba180721d55909ffacccc8adbec85268b48439423d78c", size = 3438528 }, +] + +[[package]] +name = "cssselect" +version = "1.3.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/72/0a/c3ea9573b1dc2e151abfe88c7fe0c26d1892fe6ed02d0cdb30f0d57029d5/cssselect-1.3.0.tar.gz", hash = "sha256:57f8a99424cfab289a1b6a816a43075a4b00948c86b4dcf3ef4ee7e15f7ab0c7", size = 42870 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/ee/58/257350f7db99b4ae12b614a36256d9cc870d71d9e451e79c2dc3b23d7c3c/cssselect-1.3.0-py3-none-any.whl", hash = "sha256:56d1bf3e198080cc1667e137bc51de9cadfca259f03c2d4e09037b3e01e30f0d", size = 18786 }, +] + +[[package]] +name = "curl-cffi" +version = "0.14.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "certifi" }, + { name = "cffi" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/9b/c9/0067d9a25ed4592b022d4558157fcdb6e123516083700786d38091688767/curl_cffi-0.14.0.tar.gz", hash = "sha256:5ffbc82e59f05008ec08ea432f0e535418823cda44178ee518906a54f27a5f0f", size = 162633 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/aa/f0/0f21e9688eaac85e705537b3a87a5588d0cefb2f09d83e83e0e8be93aa99/curl_cffi-0.14.0-cp39-abi3-macosx_14_0_arm64.whl", hash = "sha256:e35e89c6a69872f9749d6d5fda642ed4fc159619329e99d577d0104c9aad5893", size = 3087277 }, + { url = "https://files.pythonhosted.org/packages/ba/a3/0419bd48fce5b145cb6a2344c6ac17efa588f5b0061f212c88e0723da026/curl_cffi-0.14.0-cp39-abi3-macosx_15_0_x86_64.whl", hash = "sha256:5945478cd28ad7dfb5c54473bcfb6743ee1d66554d57951fdf8fc0e7d8cf4e45", size = 5804650 }, + { url = "https://files.pythonhosted.org/packages/e2/07/a238dd062b7841b8caa2fa8a359eb997147ff3161288f0dd46654d898b4d/curl_cffi-0.14.0-cp39-abi3-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c42e8fa3c667db9ccd2e696ee47adcd3cd5b0838d7282f3fc45f6c0ef3cfdfa7", size = 8231918 }, + { url = "https://files.pythonhosted.org/packages/7c/d2/ce907c9b37b5caf76ac08db40cc4ce3d9f94c5500db68a195af3513eacbc/curl_cffi-0.14.0-cp39-abi3-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:060fe2c99c41d3cb7f894de318ddf4b0301b08dca70453d769bd4e74b36b8483", size = 8654624 }, + { url = "https://files.pythonhosted.org/packages/f2/ae/6256995b18c75e6ef76b30753a5109e786813aa79088b27c8eabb1ef85c9/curl_cffi-0.14.0-cp39-abi3-manylinux_2_28_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:b158c41a25388690dd0d40b5bc38d1e0f512135f17fdb8029868cbc1993d2e5b", size = 8010654 }, + { url = "https://files.pythonhosted.org/packages/fb/10/ff64249e516b103cb762e0a9dca3ee0f04cf25e2a1d5d9838e0f1273d071/curl_cffi-0.14.0-cp39-abi3-manylinux_2_28_i686.whl", hash = "sha256:1439fbef3500fb723333c826adf0efb0e2e5065a703fb5eccce637a2250db34a", size = 7781969 }, + { url = "https://files.pythonhosted.org/packages/51/76/d6f7bb76c2d12811aa7ff16f5e17b678abdd1b357b9a8ac56310ceccabd5/curl_cffi-0.14.0-cp39-abi3-manylinux_2_34_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:e7176f2c2d22b542e3cf261072a81deb018cfa7688930f95dddef215caddb469", size = 7969133 }, + { url = "https://files.pythonhosted.org/packages/23/7c/cca39c0ed4e1772613d3cba13091c0e9d3b89365e84b9bf9838259a3cd8f/curl_cffi-0.14.0-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:03f21ade2d72978c2bb8670e9b6de5260e2755092b02d94b70b906813662998d", size = 9080167 }, + { url = "https://files.pythonhosted.org/packages/75/03/a942d7119d3e8911094d157598ae0169b1c6ca1bd3f27d7991b279bcc45b/curl_cffi-0.14.0-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:58ebf02de64ee5c95613209ddacb014c2d2f86298d7080c0a1c12ed876ee0690", size = 9520464 }, + { url = "https://files.pythonhosted.org/packages/a2/77/78900e9b0833066d2274bda75cba426fdb4cef7fbf6a4f6a6ca447607bec/curl_cffi-0.14.0-cp39-abi3-win_amd64.whl", hash = "sha256:6e503f9a103f6ae7acfb3890c843b53ec030785a22ae7682a22cc43afb94123e", size = 1677416 }, + { url = "https://files.pythonhosted.org/packages/5c/7c/d2ba86b0b3e1e2830bd94163d047de122c69a8df03c5c7c36326c456ad82/curl_cffi-0.14.0-cp39-abi3-win_arm64.whl", hash = "sha256:2eed50a969201605c863c4c31269dfc3e0da52916086ac54553cfa353022425c", size = 1425067 }, +] + +[[package]] +name = "cycler" +version = "0.12.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/a9/95/a3dbbb5028f35eafb79008e7522a75244477d2838f38cbb722248dabc2a8/cycler-0.12.1.tar.gz", hash = "sha256:88bb128f02ba341da8ef447245a9e138fae777f6a23943da4540077d3601eb1c", size = 7615 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e7/05/c19819d5e3d95294a6f5947fb9b9629efb316b96de511b418c53d245aae6/cycler-0.12.1-py3-none-any.whl", hash = "sha256:85cef7cff222d8644161529808465972e51340599459b8ac3ccbac5a854e0d30", size = 8321 }, +] + +[[package]] +name = "ddddocr" +version = "1.5.6" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "numpy" }, + { name = "onnxruntime" }, + { name = "opencv-python-headless" }, + { name = "pillow" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/0e/cf/1243d5f0d03763a287375366f68eadb5c14418f5b3df00c09eb971e526a7/ddddocr-1.5.6.tar.gz", hash = "sha256:2839a940bfabe02e3284ef3f9d2a037292aa9f641f355b43a9b70bece9e1b73d", size = 75825027 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/54/74/418c1c0be49463799f9eeb307a8aa4013ff5fca5e0387f0ef2762fcdb4e2/ddddocr-1.5.6-py3-none-any.whl", hash = "sha256:f13865b00e42de5c2507c1889ba73c2bacd218a49d15b928c2a5c82667062ac5", size = 75868010 }, +] + +[[package]] +name = "fake-useragent" +version = "2.2.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/41/43/948d10bf42735709edb5ae51e23297d034086f17fc7279fef385a7acb473/fake_useragent-2.2.0.tar.gz", hash = "sha256:4e6ab6571e40cc086d788523cf9e018f618d07f9050f822ff409a4dfe17c16b2", size = 158898 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/51/37/b3ea9cd5558ff4cb51957caca2193981c6b0ff30bd0d2630ac62505d99d0/fake_useragent-2.2.0-py3-none-any.whl", hash = "sha256:67f35ca4d847b0d298187443aaf020413746e56acd985a611908c73dba2daa24", size = 161695 }, +] + +[[package]] +name = "flatbuffers" +version = "25.12.19" +source = { registry = "https://pypi.org/simple" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e8/2d/d2a548598be01649e2d46231d151a6c56d10b964d94043a335ae56ea2d92/flatbuffers-25.12.19-py2.py3-none-any.whl", hash = "sha256:7634f50c427838bb021c2d66a3d1168e9d199b0607e6329399f04846d42e20b4", size = 26661 }, +] + +[[package]] +name = "fonttools" +version = "4.61.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/ec/ca/cf17b88a8df95691275a3d77dc0a5ad9907f328ae53acbe6795da1b2f5ed/fonttools-4.61.1.tar.gz", hash = "sha256:6675329885c44657f826ef01d9e4fb33b9158e9d93c537d84ad8399539bc6f69", size = 3565756 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/69/12/bf9f4eaa2fad039356cc627587e30ed008c03f1cebd3034376b5ee8d1d44/fonttools-4.61.1-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:c6604b735bb12fef8e0efd5578c9fb5d3d8532d5001ea13a19cddf295673ee09", size = 2852213 }, + { url = "https://files.pythonhosted.org/packages/ac/49/4138d1acb6261499bedde1c07f8c2605d1d8f9d77a151e5507fd3ef084b6/fonttools-4.61.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:5ce02f38a754f207f2f06557523cd39a06438ba3aafc0639c477ac409fc64e37", size = 2401689 }, + { url = "https://files.pythonhosted.org/packages/e5/fe/e6ce0fe20a40e03aef906af60aa87668696f9e4802fa283627d0b5ed777f/fonttools-4.61.1-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:77efb033d8d7ff233385f30c62c7c79271c8885d5c9657d967ede124671bbdfb", size = 5058809 }, + { url = "https://files.pythonhosted.org/packages/79/61/1ca198af22f7dd22c17ab86e9024ed3c06299cfdb08170640e9996d501a0/fonttools-4.61.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:75c1a6dfac6abd407634420c93864a1e274ebc1c7531346d9254c0d8f6ca00f9", size = 5036039 }, + { url = "https://files.pythonhosted.org/packages/99/cc/fa1801e408586b5fce4da9f5455af8d770f4fc57391cd5da7256bb364d38/fonttools-4.61.1-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:0de30bfe7745c0d1ffa2b0b7048fb7123ad0d71107e10ee090fa0b16b9452e87", size = 5034714 }, + { url = "https://files.pythonhosted.org/packages/bf/aa/b7aeafe65adb1b0a925f8f25725e09f078c635bc22754f3fecb7456955b0/fonttools-4.61.1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:58b0ee0ab5b1fc9921eccfe11d1435added19d6494dde14e323f25ad2bc30c56", size = 5158648 }, + { url = "https://files.pythonhosted.org/packages/99/f9/08ea7a38663328881384c6e7777bbefc46fd7d282adfd87a7d2b84ec9d50/fonttools-4.61.1-cp311-cp311-win32.whl", hash = "sha256:f79b168428351d11e10c5aeb61a74e1851ec221081299f4cf56036a95431c43a", size = 2280681 }, + { url = "https://files.pythonhosted.org/packages/07/ad/37dd1ae5fa6e01612a1fbb954f0927681f282925a86e86198ccd7b15d515/fonttools-4.61.1-cp311-cp311-win_amd64.whl", hash = "sha256:fe2efccb324948a11dd09d22136fe2ac8a97d6c1347cf0b58a911dcd529f66b7", size = 2331951 }, + { url = "https://files.pythonhosted.org/packages/6f/16/7decaa24a1bd3a70c607b2e29f0adc6159f36a7e40eaba59846414765fd4/fonttools-4.61.1-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:f3cb4a569029b9f291f88aafc927dd53683757e640081ca8c412781ea144565e", size = 2851593 }, + { url = "https://files.pythonhosted.org/packages/94/98/3c4cb97c64713a8cf499b3245c3bf9a2b8fd16a3e375feff2aed78f96259/fonttools-4.61.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:41a7170d042e8c0024703ed13b71893519a1a6d6e18e933e3ec7507a2c26a4b2", size = 2400231 }, + { url = "https://files.pythonhosted.org/packages/b7/37/82dbef0f6342eb01f54bca073ac1498433d6ce71e50c3c3282b655733b31/fonttools-4.61.1-cp312-cp312-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:10d88e55330e092940584774ee5e8a6971b01fc2f4d3466a1d6c158230880796", size = 4954103 }, + { url = "https://files.pythonhosted.org/packages/6c/44/f3aeac0fa98e7ad527f479e161aca6c3a1e47bb6996b053d45226fe37bf2/fonttools-4.61.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:15acc09befd16a0fb8a8f62bc147e1a82817542d72184acca9ce6e0aeda9fa6d", size = 5004295 }, + { url = "https://files.pythonhosted.org/packages/14/e8/7424ced75473983b964d09f6747fa09f054a6d656f60e9ac9324cf40c743/fonttools-4.61.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:e6bcdf33aec38d16508ce61fd81838f24c83c90a1d1b8c68982857038673d6b8", size = 4944109 }, + { url = "https://files.pythonhosted.org/packages/c8/8b/6391b257fa3d0b553d73e778f953a2f0154292a7a7a085e2374b111e5410/fonttools-4.61.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:5fade934607a523614726119164ff621e8c30e8fa1ffffbbd358662056ba69f0", size = 5093598 }, + { url = "https://files.pythonhosted.org/packages/d9/71/fd2ea96cdc512d92da5678a1c98c267ddd4d8c5130b76d0f7a80f9a9fde8/fonttools-4.61.1-cp312-cp312-win32.whl", hash = "sha256:75da8f28eff26defba42c52986de97b22106cb8f26515b7c22443ebc9c2d3261", size = 2269060 }, + { url = "https://files.pythonhosted.org/packages/80/3b/a3e81b71aed5a688e89dfe0e2694b26b78c7d7f39a5ffd8a7d75f54a12a8/fonttools-4.61.1-cp312-cp312-win_amd64.whl", hash = "sha256:497c31ce314219888c0e2fce5ad9178ca83fe5230b01a5006726cdf3ac9f24d9", size = 2319078 }, + { url = "https://files.pythonhosted.org/packages/4b/cf/00ba28b0990982530addb8dc3e9e6f2fa9cb5c20df2abdda7baa755e8fe1/fonttools-4.61.1-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:8c56c488ab471628ff3bfa80964372fc13504ece601e0d97a78ee74126b2045c", size = 2846454 }, + { url = "https://files.pythonhosted.org/packages/5a/ca/468c9a8446a2103ae645d14fee3f610567b7042aba85031c1c65e3ef7471/fonttools-4.61.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:dc492779501fa723b04d0ab1f5be046797fee17d27700476edc7ee9ae535a61e", size = 2398191 }, + { url = "https://files.pythonhosted.org/packages/a3/4b/d67eedaed19def5967fade3297fed8161b25ba94699efc124b14fb68cdbc/fonttools-4.61.1-cp313-cp313-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:64102ca87e84261419c3747a0d20f396eb024bdbeb04c2bfb37e2891f5fadcb5", size = 4928410 }, + { url = "https://files.pythonhosted.org/packages/b0/8d/6fb3494dfe61a46258cd93d979cf4725ded4eb46c2a4ca35e4490d84daea/fonttools-4.61.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4c1b526c8d3f615a7b1867f38a9410849c8f4aef078535742198e942fba0e9bd", size = 4984460 }, + { url = "https://files.pythonhosted.org/packages/f7/f1/a47f1d30b3dc00d75e7af762652d4cbc3dff5c2697a0dbd5203c81afd9c3/fonttools-4.61.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:41ed4b5ec103bd306bb68f81dc166e77409e5209443e5773cb4ed837bcc9b0d3", size = 4925800 }, + { url = "https://files.pythonhosted.org/packages/a7/01/e6ae64a0981076e8a66906fab01539799546181e32a37a0257b77e4aa88b/fonttools-4.61.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:b501c862d4901792adaec7c25b1ecc749e2662543f68bb194c42ba18d6eec98d", size = 5067859 }, + { url = "https://files.pythonhosted.org/packages/73/aa/28e40b8d6809a9b5075350a86779163f074d2b617c15d22343fce81918db/fonttools-4.61.1-cp313-cp313-win32.whl", hash = "sha256:4d7092bb38c53bbc78e9255a59158b150bcdc115a1e3b3ce0b5f267dc35dd63c", size = 2267821 }, + { url = "https://files.pythonhosted.org/packages/1a/59/453c06d1d83dc0951b69ef692d6b9f1846680342927df54e9a1ca91c6f90/fonttools-4.61.1-cp313-cp313-win_amd64.whl", hash = "sha256:21e7c8d76f62ab13c9472ccf74515ca5b9a761d1bde3265152a6dc58700d895b", size = 2318169 }, + { url = "https://files.pythonhosted.org/packages/32/8f/4e7bf82c0cbb738d3c2206c920ca34ca74ef9dabde779030145d28665104/fonttools-4.61.1-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:fff4f534200a04b4a36e7ae3cb74493afe807b517a09e99cb4faa89a34ed6ecd", size = 2846094 }, + { url = "https://files.pythonhosted.org/packages/71/09/d44e45d0a4f3a651f23a1e9d42de43bc643cce2971b19e784cc67d823676/fonttools-4.61.1-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:d9203500f7c63545b4ce3799319fe4d9feb1a1b89b28d3cb5abd11b9dd64147e", size = 2396589 }, + { url = "https://files.pythonhosted.org/packages/89/18/58c64cafcf8eb677a99ef593121f719e6dcbdb7d1c594ae5a10d4997ca8a/fonttools-4.61.1-cp314-cp314-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:fa646ecec9528bef693415c79a86e733c70a4965dd938e9a226b0fc64c9d2e6c", size = 4877892 }, + { url = "https://files.pythonhosted.org/packages/8a/ec/9e6b38c7ba1e09eb51db849d5450f4c05b7e78481f662c3b79dbde6f3d04/fonttools-4.61.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:11f35ad7805edba3aac1a3710d104592df59f4b957e30108ae0ba6c10b11dd75", size = 4972884 }, + { url = "https://files.pythonhosted.org/packages/5e/87/b5339da8e0256734ba0dbbf5b6cdebb1dd79b01dc8c270989b7bcd465541/fonttools-4.61.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:b931ae8f62db78861b0ff1ac017851764602288575d65b8e8ff1963fed419063", size = 4924405 }, + { url = "https://files.pythonhosted.org/packages/0b/47/e3409f1e1e69c073a3a6fd8cb886eb18c0bae0ee13db2c8d5e7f8495e8b7/fonttools-4.61.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:b148b56f5de675ee16d45e769e69f87623a4944f7443850bf9a9376e628a89d2", size = 5035553 }, + { url = "https://files.pythonhosted.org/packages/bf/b6/1f6600161b1073a984294c6c031e1a56ebf95b6164249eecf30012bb2e38/fonttools-4.61.1-cp314-cp314-win32.whl", hash = "sha256:9b666a475a65f4e839d3d10473fad6d47e0a9db14a2f4a224029c5bfde58ad2c", size = 2271915 }, + { url = "https://files.pythonhosted.org/packages/52/7b/91e7b01e37cc8eb0e1f770d08305b3655e4f002fc160fb82b3390eabacf5/fonttools-4.61.1-cp314-cp314-win_amd64.whl", hash = "sha256:4f5686e1fe5fce75d82d93c47a438a25bf0d1319d2843a926f741140b2b16e0c", size = 2323487 }, + { url = "https://files.pythonhosted.org/packages/39/5c/908ad78e46c61c3e3ed70c3b58ff82ab48437faf84ec84f109592cabbd9f/fonttools-4.61.1-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:e76ce097e3c57c4bcb67c5aa24a0ecdbd9f74ea9219997a707a4061fbe2707aa", size = 2929571 }, + { url = "https://files.pythonhosted.org/packages/bd/41/975804132c6dea64cdbfbaa59f3518a21c137a10cccf962805b301ac6ab2/fonttools-4.61.1-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:9cfef3ab326780c04d6646f68d4b4742aae222e8b8ea1d627c74e38afcbc9d91", size = 2435317 }, + { url = "https://files.pythonhosted.org/packages/b0/5a/aef2a0a8daf1ebaae4cfd83f84186d4a72ee08fd6a8451289fcd03ffa8a4/fonttools-4.61.1-cp314-cp314t-manylinux1_x86_64.manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:a75c301f96db737e1c5ed5fd7d77d9c34466de16095a266509e13da09751bd19", size = 4882124 }, + { url = "https://files.pythonhosted.org/packages/80/33/d6db3485b645b81cea538c9d1c9219d5805f0877fda18777add4671c5240/fonttools-4.61.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:91669ccac46bbc1d09e9273546181919064e8df73488ea087dcac3e2968df9ba", size = 5100391 }, + { url = "https://files.pythonhosted.org/packages/6c/d6/675ba631454043c75fcf76f0ca5463eac8eb0666ea1d7badae5fea001155/fonttools-4.61.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:c33ab3ca9d3ccd581d58e989d67554e42d8d4ded94ab3ade3508455fe70e65f7", size = 4978800 }, + { url = "https://files.pythonhosted.org/packages/7f/33/d3ec753d547a8d2bdaedd390d4a814e8d5b45a093d558f025c6b990b554c/fonttools-4.61.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:664c5a68ec406f6b1547946683008576ef8b38275608e1cee6c061828171c118", size = 5006426 }, + { url = "https://files.pythonhosted.org/packages/b4/40/cc11f378b561a67bea850ab50063366a0d1dd3f6d0a30ce0f874b0ad5664/fonttools-4.61.1-cp314-cp314t-win32.whl", hash = "sha256:aed04cabe26f30c1647ef0e8fbb207516fd40fe9472e9439695f5c6998e60ac5", size = 2335377 }, + { url = "https://files.pythonhosted.org/packages/e4/ff/c9a2b66b39f8628531ea58b320d66d951267c98c6a38684daa8f50fb02f8/fonttools-4.61.1-cp314-cp314t-win_amd64.whl", hash = "sha256:2180f14c141d2f0f3da43f3a81bc8aa4684860f6b0e6f9e165a4831f24e6a23b", size = 2400613 }, + { url = "https://files.pythonhosted.org/packages/c7/4e/ce75a57ff3aebf6fc1f4e9d508b8e5810618a33d900ad6c19eb30b290b97/fonttools-4.61.1-py3-none-any.whl", hash = "sha256:17d2bf5d541add43822bcf0c43d7d847b160c9bb01d15d5007d84e2217aaa371", size = 1148996 }, +] + +[[package]] +name = "greenlet" +version = "3.3.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/c7/e5/40dbda2736893e3e53d25838e0f19a2b417dfc122b9989c91918db30b5d3/greenlet-3.3.0.tar.gz", hash = "sha256:a82bb225a4e9e4d653dd2fb7b8b2d36e4fb25bc0165422a11e48b88e9e6f78fb", size = 190651 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/1f/cb/48e964c452ca2b92175a9b2dca037a553036cb053ba69e284650ce755f13/greenlet-3.3.0-cp311-cp311-macosx_11_0_universal2.whl", hash = "sha256:e29f3018580e8412d6aaf5641bb7745d38c85228dacf51a73bd4e26ddf2a6a8e", size = 274908 }, + { url = "https://files.pythonhosted.org/packages/28/da/38d7bff4d0277b594ec557f479d65272a893f1f2a716cad91efeb8680953/greenlet-3.3.0-cp311-cp311-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a687205fb22794e838f947e2194c0566d3812966b41c78709554aa883183fb62", size = 577113 }, + { url = "https://files.pythonhosted.org/packages/3c/f2/89c5eb0faddc3ff014f1c04467d67dee0d1d334ab81fadbf3744847f8a8a/greenlet-3.3.0-cp311-cp311-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:4243050a88ba61842186cb9e63c7dfa677ec146160b0efd73b855a3d9c7fcf32", size = 590338 }, + { url = "https://files.pythonhosted.org/packages/80/d7/db0a5085035d05134f8c089643da2b44cc9b80647c39e93129c5ef170d8f/greenlet-3.3.0-cp311-cp311-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:670d0f94cd302d81796e37299bcd04b95d62403883b24225c6b5271466612f45", size = 601098 }, + { url = "https://files.pythonhosted.org/packages/dc/a6/e959a127b630a58e23529972dbc868c107f9d583b5a9f878fb858c46bc1a/greenlet-3.3.0-cp311-cp311-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6cb3a8ec3db4a3b0eb8a3c25436c2d49e3505821802074969db017b87bc6a948", size = 590206 }, + { url = "https://files.pythonhosted.org/packages/48/60/29035719feb91798693023608447283b266b12efc576ed013dd9442364bb/greenlet-3.3.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:2de5a0b09eab81fc6a382791b995b1ccf2b172a9fec934747a7a23d2ff291794", size = 1550668 }, + { url = "https://files.pythonhosted.org/packages/0a/5f/783a23754b691bfa86bd72c3033aa107490deac9b2ef190837b860996c9f/greenlet-3.3.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:4449a736606bd30f27f8e1ff4678ee193bc47f6ca810d705981cfffd6ce0d8c5", size = 1615483 }, + { url = "https://files.pythonhosted.org/packages/1d/d5/c339b3b4bc8198b7caa4f2bd9fd685ac9f29795816d8db112da3d04175bb/greenlet-3.3.0-cp311-cp311-win_amd64.whl", hash = "sha256:7652ee180d16d447a683c04e4c5f6441bae7ba7b17ffd9f6b3aff4605e9e6f71", size = 301164 }, + { url = "https://files.pythonhosted.org/packages/f8/0a/a3871375c7b9727edaeeea994bfff7c63ff7804c9829c19309ba2e058807/greenlet-3.3.0-cp312-cp312-macosx_11_0_universal2.whl", hash = "sha256:b01548f6e0b9e9784a2c99c5651e5dc89ffcbe870bc5fb2e5ef864e9cc6b5dcb", size = 276379 }, + { url = "https://files.pythonhosted.org/packages/43/ab/7ebfe34dce8b87be0d11dae91acbf76f7b8246bf9d6b319c741f99fa59c6/greenlet-3.3.0-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:349345b770dc88f81506c6861d22a6ccd422207829d2c854ae2af8025af303e3", size = 597294 }, + { url = "https://files.pythonhosted.org/packages/a4/39/f1c8da50024feecd0793dbd5e08f526809b8ab5609224a2da40aad3a7641/greenlet-3.3.0-cp312-cp312-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:e8e18ed6995e9e2c0b4ed264d2cf89260ab3ac7e13555b8032b25a74c6d18655", size = 607742 }, + { url = "https://files.pythonhosted.org/packages/77/cb/43692bcd5f7a0da6ec0ec6d58ee7cddb606d055ce94a62ac9b1aa481e969/greenlet-3.3.0-cp312-cp312-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:c024b1e5696626890038e34f76140ed1daf858e37496d33f2af57f06189e70d7", size = 622297 }, + { url = "https://files.pythonhosted.org/packages/75/b0/6bde0b1011a60782108c01de5913c588cf51a839174538d266de15e4bf4d/greenlet-3.3.0-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:047ab3df20ede6a57c35c14bf5200fcf04039d50f908270d3f9a7a82064f543b", size = 609885 }, + { url = "https://files.pythonhosted.org/packages/49/0e/49b46ac39f931f59f987b7cd9f34bfec8ef81d2a1e6e00682f55be5de9f4/greenlet-3.3.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:2d9ad37fc657b1102ec880e637cccf20191581f75c64087a549e66c57e1ceb53", size = 1567424 }, + { url = "https://files.pythonhosted.org/packages/05/f5/49a9ac2dff7f10091935def9165c90236d8f175afb27cbed38fb1d61ab6b/greenlet-3.3.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:83cd0e36932e0e7f36a64b732a6f60c2fc2df28c351bae79fbaf4f8092fe7614", size = 1636017 }, + { url = "https://files.pythonhosted.org/packages/6c/79/3912a94cf27ec503e51ba493692d6db1e3cd8ac7ac52b0b47c8e33d7f4f9/greenlet-3.3.0-cp312-cp312-win_amd64.whl", hash = "sha256:a7a34b13d43a6b78abf828a6d0e87d3385680eaf830cd60d20d52f249faabf39", size = 301964 }, + { url = "https://files.pythonhosted.org/packages/02/2f/28592176381b9ab2cafa12829ba7b472d177f3acc35d8fbcf3673d966fff/greenlet-3.3.0-cp313-cp313-macosx_11_0_universal2.whl", hash = "sha256:a1e41a81c7e2825822f4e068c48cb2196002362619e2d70b148f20a831c00739", size = 275140 }, + { url = "https://files.pythonhosted.org/packages/2c/80/fbe937bf81e9fca98c981fe499e59a3f45df2a04da0baa5c2be0dca0d329/greenlet-3.3.0-cp313-cp313-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9f515a47d02da4d30caaa85b69474cec77b7929b2e936ff7fb853d42f4bf8808", size = 599219 }, + { url = "https://files.pythonhosted.org/packages/c2/ff/7c985128f0514271b8268476af89aee6866df5eec04ac17dcfbc676213df/greenlet-3.3.0-cp313-cp313-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:7d2d9fd66bfadf230b385fdc90426fcd6eb64db54b40c495b72ac0feb5766c54", size = 610211 }, + { url = "https://files.pythonhosted.org/packages/79/07/c47a82d881319ec18a4510bb30463ed6891f2ad2c1901ed5ec23d3de351f/greenlet-3.3.0-cp313-cp313-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:30a6e28487a790417d036088b3bcb3f3ac7d8babaa7d0139edbaddebf3af9492", size = 624311 }, + { url = "https://files.pythonhosted.org/packages/fd/8e/424b8c6e78bd9837d14ff7df01a9829fc883ba2ab4ea787d4f848435f23f/greenlet-3.3.0-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:087ea5e004437321508a8d6f20efc4cfec5e3c30118e1417ea96ed1d93950527", size = 612833 }, + { url = "https://files.pythonhosted.org/packages/b5/ba/56699ff9b7c76ca12f1cdc27a886d0f81f2189c3455ff9f65246780f713d/greenlet-3.3.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:ab97cf74045343f6c60a39913fa59710e4bd26a536ce7ab2397adf8b27e67c39", size = 1567256 }, + { url = "https://files.pythonhosted.org/packages/1e/37/f31136132967982d698c71a281a8901daf1a8fbab935dce7c0cf15f942cc/greenlet-3.3.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:5375d2e23184629112ca1ea89a53389dddbffcf417dad40125713d88eb5f96e8", size = 1636483 }, + { url = "https://files.pythonhosted.org/packages/7e/71/ba21c3fb8c5dce83b8c01f458a42e99ffdb1963aeec08fff5a18588d8fd7/greenlet-3.3.0-cp313-cp313-win_amd64.whl", hash = "sha256:9ee1942ea19550094033c35d25d20726e4f1c40d59545815e1128ac58d416d38", size = 301833 }, + { url = "https://files.pythonhosted.org/packages/d7/7c/f0a6d0ede2c7bf092d00bc83ad5bafb7e6ec9b4aab2fbdfa6f134dc73327/greenlet-3.3.0-cp314-cp314-macosx_11_0_universal2.whl", hash = "sha256:60c2ef0f578afb3c8d92ea07ad327f9a062547137afe91f38408f08aacab667f", size = 275671 }, + { url = "https://files.pythonhosted.org/packages/44/06/dac639ae1a50f5969d82d2e3dd9767d30d6dbdbab0e1a54010c8fe90263c/greenlet-3.3.0-cp314-cp314-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0a5d554d0712ba1de0a6c94c640f7aeba3f85b3a6e1f2899c11c2c0428da9365", size = 646360 }, + { url = "https://files.pythonhosted.org/packages/e0/94/0fb76fe6c5369fba9bf98529ada6f4c3a1adf19e406a47332245ef0eb357/greenlet-3.3.0-cp314-cp314-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:3a898b1e9c5f7307ebbde4102908e6cbfcb9ea16284a3abe15cab996bee8b9b3", size = 658160 }, + { url = "https://files.pythonhosted.org/packages/93/79/d2c70cae6e823fac36c3bbc9077962105052b7ef81db2f01ec3b9bf17e2b/greenlet-3.3.0-cp314-cp314-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:dcd2bdbd444ff340e8d6bdf54d2f206ccddbb3ccfdcd3c25bf4afaa7b8f0cf45", size = 671388 }, + { url = "https://files.pythonhosted.org/packages/b8/14/bab308fc2c1b5228c3224ec2bf928ce2e4d21d8046c161e44a2012b5203e/greenlet-3.3.0-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5773edda4dc00e173820722711d043799d3adb4f01731f40619e07ea2750b955", size = 660166 }, + { url = "https://files.pythonhosted.org/packages/4b/d2/91465d39164eaa0085177f61983d80ffe746c5a1860f009811d498e7259c/greenlet-3.3.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:ac0549373982b36d5fd5d30beb8a7a33ee541ff98d2b502714a09f1169f31b55", size = 1615193 }, + { url = "https://files.pythonhosted.org/packages/42/1b/83d110a37044b92423084d52d5d5a3b3a73cafb51b547e6d7366ff62eff1/greenlet-3.3.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:d198d2d977460358c3b3a4dc844f875d1adb33817f0613f663a656f463764ccc", size = 1683653 }, + { url = "https://files.pythonhosted.org/packages/7c/9a/9030e6f9aa8fd7808e9c31ba4c38f87c4f8ec324ee67431d181fe396d705/greenlet-3.3.0-cp314-cp314-win_amd64.whl", hash = "sha256:73f51dd0e0bdb596fb0417e475fa3c5e32d4c83638296e560086b8d7da7c4170", size = 305387 }, + { url = "https://files.pythonhosted.org/packages/a0/66/bd6317bc5932accf351fc19f177ffba53712a202f9df10587da8df257c7e/greenlet-3.3.0-cp314-cp314t-macosx_11_0_universal2.whl", hash = "sha256:d6ed6f85fae6cdfdb9ce04c9bf7a08d666cfcfb914e7d006f44f840b46741931", size = 282638 }, + { url = "https://files.pythonhosted.org/packages/30/cf/cc81cb030b40e738d6e69502ccbd0dd1bced0588e958f9e757945de24404/greenlet-3.3.0-cp314-cp314t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d9125050fcf24554e69c4cacb086b87b3b55dc395a8b3ebe6487b045b2614388", size = 651145 }, + { url = "https://files.pythonhosted.org/packages/9c/ea/1020037b5ecfe95ca7df8d8549959baceb8186031da83d5ecceff8b08cd2/greenlet-3.3.0-cp314-cp314t-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:87e63ccfa13c0a0f6234ed0add552af24cc67dd886731f2261e46e241608bee3", size = 654236 }, + { url = "https://files.pythonhosted.org/packages/69/cc/1e4bae2e45ca2fa55299f4e85854606a78ecc37fead20d69322f96000504/greenlet-3.3.0-cp314-cp314t-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:2662433acbca297c9153a4023fe2161c8dcfdcc91f10433171cf7e7d94ba2221", size = 662506 }, + { url = "https://files.pythonhosted.org/packages/57/b9/f8025d71a6085c441a7eaff0fd928bbb275a6633773667023d19179fe815/greenlet-3.3.0-cp314-cp314t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:3c6e9b9c1527a78520357de498b0e709fb9e2f49c3a513afd5a249007261911b", size = 653783 }, + { url = "https://files.pythonhosted.org/packages/f6/c7/876a8c7a7485d5d6b5c6821201d542ef28be645aa024cfe1145b35c120c1/greenlet-3.3.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:286d093f95ec98fdd92fcb955003b8a3d054b4e2cab3e2707a5039e7b50520fd", size = 1614857 }, + { url = "https://files.pythonhosted.org/packages/4f/dc/041be1dff9f23dac5f48a43323cd0789cb798342011c19a248d9c9335536/greenlet-3.3.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:6c10513330af5b8ae16f023e8ddbfb486ab355d04467c4679c5cfe4659975dd9", size = 1676034 }, +] + +[[package]] +name = "h11" +version = "0.16.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/01/ee/02a2c011bdab74c6fb3c75474d40b3052059d95df7e73351460c8588d963/h11-0.16.0.tar.gz", hash = "sha256:4e35b956cf45792e4caa5885e69fba00bdbc6ffafbfa020300e549b208ee5ff1", size = 101250 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/04/4b/29cac41a4d98d144bf5f6d33995617b185d14b22401f75ca86f384e87ff1/h11-0.16.0-py3-none-any.whl", hash = "sha256:63cf8bbe7522de3bf65932fda1d9c2772064ffb3dae62d55932da54b31cb6c86", size = 37515 }, +] + +[[package]] +name = "httpcore" +version = "1.0.9" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "certifi" }, + { name = "h11" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/06/94/82699a10bca87a5556c9c59b5963f2d039dbd239f25bc2a63907a05a14cb/httpcore-1.0.9.tar.gz", hash = "sha256:6e34463af53fd2ab5d807f399a9b45ea31c3dfa2276f15a2c3f00afff6e176e8", size = 85484 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/7e/f5/f66802a942d491edb555dd61e3a9961140fd64c90bce1eafd741609d334d/httpcore-1.0.9-py3-none-any.whl", hash = "sha256:2d400746a40668fc9dec9810239072b40b4484b640a8c38fd654a024c7a1bf55", size = 78784 }, +] + +[[package]] +name = "httpx" +version = "0.28.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "anyio" }, + { name = "certifi" }, + { name = "httpcore" }, + { name = "idna" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/b1/df/48c586a5fe32a0f01324ee087459e112ebb7224f646c0b5023f5e79e9956/httpx-0.28.1.tar.gz", hash = "sha256:75e98c5f16b0f35b567856f597f06ff2270a374470a5c2392242528e3e3e42fc", size = 141406 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/2a/39/e50c7c3a983047577ee07d2a9e53faf5a69493943ec3f6a384bdc792deb2/httpx-0.28.1-py3-none-any.whl", hash = "sha256:d909fcccc110f8c7faf814ca82a9a4d816bc5a6dbfea25d6591d6985b8ba59ad", size = 73517 }, +] + +[[package]] +name = "humanfriendly" +version = "10.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "pyreadline3", marker = "sys_platform == 'win32'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/cc/3f/2c29224acb2e2df4d2046e4c73ee2662023c58ff5b113c4c1adac0886c43/humanfriendly-10.0.tar.gz", hash = "sha256:6b0b831ce8f15f7300721aa49829fc4e83921a9a301cc7f606be6686a2288ddc", size = 360702 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f0/0f/310fb31e39e2d734ccaa2c0fb981ee41f7bd5056ce9bc29b2248bd569169/humanfriendly-10.0-py2.py3-none-any.whl", hash = "sha256:1697e1a8a8f550fd43c2865cd84542fc175a61dcb779b6fee18cf6b6ccba1477", size = 86794 }, +] + +[[package]] +name = "idna" +version = "3.11" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/6f/6d/0703ccc57f3a7233505399edb88de3cbd678da106337b9fcde432b65ed60/idna-3.11.tar.gz", hash = "sha256:795dafcc9c04ed0c1fb032c2aa73654d8e8c5023a7df64a53f39190ada629902", size = 194582 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/0e/61/66938bbb5fc52dbdf84594873d5b51fb1f7c7794e9c0f5bd885f30bc507b/idna-3.11-py3-none-any.whl", hash = "sha256:771a87f49d9defaf64091e6e6fe9c18d4833f140bd19464795bc32d966ca37ea", size = 71008 }, +] + +[[package]] +name = "jieba" +version = "0.42.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/c6/cb/18eeb235f833b726522d7ebed54f2278ce28ba9438e3135ab0278d9792a2/jieba-0.42.1.tar.gz", hash = "sha256:055ca12f62674fafed09427f176506079bc135638a14e23e25be909131928db2", size = 19214172 } + +[[package]] +name = "jinja2" +version = "3.1.6" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "markupsafe" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/df/bf/f7da0350254c0ed7c72f3e33cef02e048281fec7ecec5f032d4aac52226b/jinja2-3.1.6.tar.gz", hash = "sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d", size = 245115 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/62/a1/3d680cbfd5f4b8f15abc1d571870c5fc3e594bb582bc3b64ea099db13e56/jinja2-3.1.6-py3-none-any.whl", hash = "sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67", size = 134899 }, +] + +[[package]] +name = "jmespath" +version = "1.0.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/00/2a/e867e8531cf3e36b41201936b7fa7ba7b5702dbef42922193f05c8976cd6/jmespath-1.0.1.tar.gz", hash = "sha256:90261b206d6defd58fdd5e85f478bf633a2901798906be2ad389150c5c60edbe", size = 25843 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/31/b4/b9b800c45527aadd64d5b442f9b932b00648617eb5d63d2c7a6587b7cafc/jmespath-1.0.1-py3-none-any.whl", hash = "sha256:02e2e4cc71b5bcab88332eebf907519190dd9e6e82107fa7f83b1003a6252980", size = 20256 }, +] + +[[package]] +name = "kiwisolver" +version = "1.4.9" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/5c/3c/85844f1b0feb11ee581ac23fe5fce65cd049a200c1446708cc1b7f922875/kiwisolver-1.4.9.tar.gz", hash = "sha256:c3b22c26c6fd6811b0ae8363b95ca8ce4ea3c202d3d0975b2914310ceb1bcc4d", size = 97564 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/6f/ab/c80b0d5a9d8a1a65f4f815f2afff9798b12c3b9f31f1d304dd233dd920e2/kiwisolver-1.4.9-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:eb14a5da6dc7642b0f3a18f13654847cd8b7a2550e2645a5bda677862b03ba16", size = 124167 }, + { url = "https://files.pythonhosted.org/packages/a0/c0/27fe1a68a39cf62472a300e2879ffc13c0538546c359b86f149cc19f6ac3/kiwisolver-1.4.9-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:39a219e1c81ae3b103643d2aedb90f1ef22650deb266ff12a19e7773f3e5f089", size = 66579 }, + { url = "https://files.pythonhosted.org/packages/31/a2/a12a503ac1fd4943c50f9822678e8015a790a13b5490354c68afb8489814/kiwisolver-1.4.9-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:2405a7d98604b87f3fc28b1716783534b1b4b8510d8142adca34ee0bc3c87543", size = 65309 }, + { url = "https://files.pythonhosted.org/packages/66/e1/e533435c0be77c3f64040d68d7a657771194a63c279f55573188161e81ca/kiwisolver-1.4.9-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:dc1ae486f9abcef254b5618dfb4113dd49f94c68e3e027d03cf0143f3f772b61", size = 1435596 }, + { url = "https://files.pythonhosted.org/packages/67/1e/51b73c7347f9aabdc7215aa79e8b15299097dc2f8e67dee2b095faca9cb0/kiwisolver-1.4.9-cp311-cp311-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8a1f570ce4d62d718dce3f179ee78dac3b545ac16c0c04bb363b7607a949c0d1", size = 1246548 }, + { url = "https://files.pythonhosted.org/packages/21/aa/72a1c5d1e430294f2d32adb9542719cfb441b5da368d09d268c7757af46c/kiwisolver-1.4.9-cp311-cp311-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:cb27e7b78d716c591e88e0a09a2139c6577865d7f2e152488c2cc6257f460872", size = 1263618 }, + { url = "https://files.pythonhosted.org/packages/a3/af/db1509a9e79dbf4c260ce0cfa3903ea8945f6240e9e59d1e4deb731b1a40/kiwisolver-1.4.9-cp311-cp311-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:15163165efc2f627eb9687ea5f3a28137217d217ac4024893d753f46bce9de26", size = 1317437 }, + { url = "https://files.pythonhosted.org/packages/e0/f2/3ea5ee5d52abacdd12013a94130436e19969fa183faa1e7c7fbc89e9a42f/kiwisolver-1.4.9-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:bdee92c56a71d2b24c33a7d4c2856bd6419d017e08caa7802d2963870e315028", size = 2195742 }, + { url = "https://files.pythonhosted.org/packages/6f/9b/1efdd3013c2d9a2566aa6a337e9923a00590c516add9a1e89a768a3eb2fc/kiwisolver-1.4.9-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:412f287c55a6f54b0650bd9b6dce5aceddb95864a1a90c87af16979d37c89771", size = 2290810 }, + { url = "https://files.pythonhosted.org/packages/fb/e5/cfdc36109ae4e67361f9bc5b41323648cb24a01b9ade18784657e022e65f/kiwisolver-1.4.9-cp311-cp311-musllinux_1_2_s390x.whl", hash = "sha256:2c93f00dcba2eea70af2be5f11a830a742fe6b579a1d4e00f47760ef13be247a", size = 2461579 }, + { url = "https://files.pythonhosted.org/packages/62/86/b589e5e86c7610842213994cdea5add00960076bef4ae290c5fa68589cac/kiwisolver-1.4.9-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:f117e1a089d9411663a3207ba874f31be9ac8eaa5b533787024dc07aeb74f464", size = 2268071 }, + { url = "https://files.pythonhosted.org/packages/3b/c6/f8df8509fd1eee6c622febe54384a96cfaf4d43bf2ccec7a0cc17e4715c9/kiwisolver-1.4.9-cp311-cp311-win_amd64.whl", hash = "sha256:be6a04e6c79819c9a8c2373317d19a96048e5a3f90bec587787e86a1153883c2", size = 73840 }, + { url = "https://files.pythonhosted.org/packages/e2/2d/16e0581daafd147bc11ac53f032a2b45eabac897f42a338d0a13c1e5c436/kiwisolver-1.4.9-cp311-cp311-win_arm64.whl", hash = "sha256:0ae37737256ba2de764ddc12aed4956460277f00c4996d51a197e72f62f5eec7", size = 65159 }, + { url = "https://files.pythonhosted.org/packages/86/c9/13573a747838aeb1c76e3267620daa054f4152444d1f3d1a2324b78255b5/kiwisolver-1.4.9-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:ac5a486ac389dddcc5bef4f365b6ae3ffff2c433324fb38dd35e3fab7c957999", size = 123686 }, + { url = "https://files.pythonhosted.org/packages/51/ea/2ecf727927f103ffd1739271ca19c424d0e65ea473fbaeea1c014aea93f6/kiwisolver-1.4.9-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:f2ba92255faa7309d06fe44c3a4a97efe1c8d640c2a79a5ef728b685762a6fd2", size = 66460 }, + { url = "https://files.pythonhosted.org/packages/5b/5a/51f5464373ce2aeb5194508298a508b6f21d3867f499556263c64c621914/kiwisolver-1.4.9-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:4a2899935e724dd1074cb568ce7ac0dce28b2cd6ab539c8e001a8578eb106d14", size = 64952 }, + { url = "https://files.pythonhosted.org/packages/70/90/6d240beb0f24b74371762873e9b7f499f1e02166a2d9c5801f4dbf8fa12e/kiwisolver-1.4.9-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:f6008a4919fdbc0b0097089f67a1eb55d950ed7e90ce2cc3e640abadd2757a04", size = 1474756 }, + { url = "https://files.pythonhosted.org/packages/12/42/f36816eaf465220f683fb711efdd1bbf7a7005a2473d0e4ed421389bd26c/kiwisolver-1.4.9-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:67bb8b474b4181770f926f7b7d2f8c0248cbcb78b660fdd41a47054b28d2a752", size = 1276404 }, + { url = "https://files.pythonhosted.org/packages/2e/64/bc2de94800adc830c476dce44e9b40fd0809cddeef1fde9fcf0f73da301f/kiwisolver-1.4.9-cp312-cp312-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:2327a4a30d3ee07d2fbe2e7933e8a37c591663b96ce42a00bc67461a87d7df77", size = 1294410 }, + { url = "https://files.pythonhosted.org/packages/5f/42/2dc82330a70aa8e55b6d395b11018045e58d0bb00834502bf11509f79091/kiwisolver-1.4.9-cp312-cp312-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:7a08b491ec91b1d5053ac177afe5290adacf1f0f6307d771ccac5de30592d198", size = 1343631 }, + { url = "https://files.pythonhosted.org/packages/22/fd/f4c67a6ed1aab149ec5a8a401c323cee7a1cbe364381bb6c9c0d564e0e20/kiwisolver-1.4.9-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:d8fc5c867c22b828001b6a38d2eaeb88160bf5783c6cb4a5e440efc981ce286d", size = 2224963 }, + { url = "https://files.pythonhosted.org/packages/45/aa/76720bd4cb3713314677d9ec94dcc21ced3f1baf4830adde5bb9b2430a5f/kiwisolver-1.4.9-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:3b3115b2581ea35bb6d1f24a4c90af37e5d9b49dcff267eeed14c3893c5b86ab", size = 2321295 }, + { url = "https://files.pythonhosted.org/packages/80/19/d3ec0d9ab711242f56ae0dc2fc5d70e298bb4a1f9dfab44c027668c673a1/kiwisolver-1.4.9-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:858e4c22fb075920b96a291928cb7dea5644e94c0ee4fcd5af7e865655e4ccf2", size = 2487987 }, + { url = "https://files.pythonhosted.org/packages/39/e9/61e4813b2c97e86b6fdbd4dd824bf72d28bcd8d4849b8084a357bc0dd64d/kiwisolver-1.4.9-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:ed0fecd28cc62c54b262e3736f8bb2512d8dcfdc2bcf08be5f47f96bf405b145", size = 2291817 }, + { url = "https://files.pythonhosted.org/packages/a0/41/85d82b0291db7504da3c2defe35c9a8a5c9803a730f297bd823d11d5fb77/kiwisolver-1.4.9-cp312-cp312-win_amd64.whl", hash = "sha256:f68208a520c3d86ea51acf688a3e3002615a7f0238002cccc17affecc86a8a54", size = 73895 }, + { url = "https://files.pythonhosted.org/packages/e2/92/5f3068cf15ee5cb624a0c7596e67e2a0bb2adee33f71c379054a491d07da/kiwisolver-1.4.9-cp312-cp312-win_arm64.whl", hash = "sha256:2c1a4f57df73965f3f14df20b80ee29e6a7930a57d2d9e8491a25f676e197c60", size = 64992 }, + { url = "https://files.pythonhosted.org/packages/31/c1/c2686cda909742ab66c7388e9a1a8521a59eb89f8bcfbee28fc980d07e24/kiwisolver-1.4.9-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:a5d0432ccf1c7ab14f9949eec60c5d1f924f17c037e9f8b33352fa05799359b8", size = 123681 }, + { url = "https://files.pythonhosted.org/packages/ca/f0/f44f50c9f5b1a1860261092e3bc91ecdc9acda848a8b8c6abfda4a24dd5c/kiwisolver-1.4.9-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:efb3a45b35622bb6c16dbfab491a8f5a391fe0e9d45ef32f4df85658232ca0e2", size = 66464 }, + { url = "https://files.pythonhosted.org/packages/2d/7a/9d90a151f558e29c3936b8a47ac770235f436f2120aca41a6d5f3d62ae8d/kiwisolver-1.4.9-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:1a12cf6398e8a0a001a059747a1cbf24705e18fe413bc22de7b3d15c67cffe3f", size = 64961 }, + { url = "https://files.pythonhosted.org/packages/e9/e9/f218a2cb3a9ffbe324ca29a9e399fa2d2866d7f348ec3a88df87fc248fc5/kiwisolver-1.4.9-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:b67e6efbf68e077dd71d1a6b37e43e1a99d0bff1a3d51867d45ee8908b931098", size = 1474607 }, + { url = "https://files.pythonhosted.org/packages/d9/28/aac26d4c882f14de59041636292bc838db8961373825df23b8eeb807e198/kiwisolver-1.4.9-cp313-cp313-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5656aa670507437af0207645273ccdfee4f14bacd7f7c67a4306d0dcaeaf6eed", size = 1276546 }, + { url = "https://files.pythonhosted.org/packages/8b/ad/8bfc1c93d4cc565e5069162f610ba2f48ff39b7de4b5b8d93f69f30c4bed/kiwisolver-1.4.9-cp313-cp313-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:bfc08add558155345129c7803b3671cf195e6a56e7a12f3dde7c57d9b417f525", size = 1294482 }, + { url = "https://files.pythonhosted.org/packages/da/f1/6aca55ff798901d8ce403206d00e033191f63d82dd708a186e0ed2067e9c/kiwisolver-1.4.9-cp313-cp313-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:40092754720b174e6ccf9e845d0d8c7d8e12c3d71e7fc35f55f3813e96376f78", size = 1343720 }, + { url = "https://files.pythonhosted.org/packages/d1/91/eed031876c595c81d90d0f6fc681ece250e14bf6998c3d7c419466b523b7/kiwisolver-1.4.9-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:497d05f29a1300d14e02e6441cf0f5ee81c1ff5a304b0d9fb77423974684e08b", size = 2224907 }, + { url = "https://files.pythonhosted.org/packages/e9/ec/4d1925f2e49617b9cca9c34bfa11adefad49d00db038e692a559454dfb2e/kiwisolver-1.4.9-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:bdd1a81a1860476eb41ac4bc1e07b3f07259e6d55bbf739b79c8aaedcf512799", size = 2321334 }, + { url = "https://files.pythonhosted.org/packages/43/cb/450cd4499356f68802750c6ddc18647b8ea01ffa28f50d20598e0befe6e9/kiwisolver-1.4.9-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:e6b93f13371d341afee3be9f7c5964e3fe61d5fa30f6a30eb49856935dfe4fc3", size = 2488313 }, + { url = "https://files.pythonhosted.org/packages/71/67/fc76242bd99f885651128a5d4fa6083e5524694b7c88b489b1b55fdc491d/kiwisolver-1.4.9-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:d75aa530ccfaa593da12834b86a0724f58bff12706659baa9227c2ccaa06264c", size = 2291970 }, + { url = "https://files.pythonhosted.org/packages/75/bd/f1a5d894000941739f2ae1b65a32892349423ad49c2e6d0771d0bad3fae4/kiwisolver-1.4.9-cp313-cp313-win_amd64.whl", hash = "sha256:dd0a578400839256df88c16abddf9ba14813ec5f21362e1fe65022e00c883d4d", size = 73894 }, + { url = "https://files.pythonhosted.org/packages/95/38/dce480814d25b99a391abbddadc78f7c117c6da34be68ca8b02d5848b424/kiwisolver-1.4.9-cp313-cp313-win_arm64.whl", hash = "sha256:d4188e73af84ca82468f09cadc5ac4db578109e52acb4518d8154698d3a87ca2", size = 64995 }, + { url = "https://files.pythonhosted.org/packages/e2/37/7d218ce5d92dadc5ebdd9070d903e0c7cf7edfe03f179433ac4d13ce659c/kiwisolver-1.4.9-cp313-cp313t-macosx_10_13_universal2.whl", hash = "sha256:5a0f2724dfd4e3b3ac5a82436a8e6fd16baa7d507117e4279b660fe8ca38a3a1", size = 126510 }, + { url = "https://files.pythonhosted.org/packages/23/b0/e85a2b48233daef4b648fb657ebbb6f8367696a2d9548a00b4ee0eb67803/kiwisolver-1.4.9-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:1b11d6a633e4ed84fc0ddafd4ebfd8ea49b3f25082c04ad12b8315c11d504dc1", size = 67903 }, + { url = "https://files.pythonhosted.org/packages/44/98/f2425bc0113ad7de24da6bb4dae1343476e95e1d738be7c04d31a5d037fd/kiwisolver-1.4.9-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:61874cdb0a36016354853593cffc38e56fc9ca5aa97d2c05d3dcf6922cd55a11", size = 66402 }, + { url = "https://files.pythonhosted.org/packages/98/d8/594657886df9f34c4177cc353cc28ca7e6e5eb562d37ccc233bff43bbe2a/kiwisolver-1.4.9-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:60c439763a969a6af93b4881db0eed8fadf93ee98e18cbc35bc8da868d0c4f0c", size = 1582135 }, + { url = "https://files.pythonhosted.org/packages/5c/c6/38a115b7170f8b306fc929e166340c24958347308ea3012c2b44e7e295db/kiwisolver-1.4.9-cp313-cp313t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:92a2f997387a1b79a75e7803aa7ded2cfbe2823852ccf1ba3bcf613b62ae3197", size = 1389409 }, + { url = "https://files.pythonhosted.org/packages/bf/3b/e04883dace81f24a568bcee6eb3001da4ba05114afa622ec9b6fafdc1f5e/kiwisolver-1.4.9-cp313-cp313t-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:a31d512c812daea6d8b3be3b2bfcbeb091dbb09177706569bcfc6240dcf8b41c", size = 1401763 }, + { url = "https://files.pythonhosted.org/packages/9f/80/20ace48e33408947af49d7d15c341eaee69e4e0304aab4b7660e234d6288/kiwisolver-1.4.9-cp313-cp313t-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:52a15b0f35dad39862d376df10c5230155243a2c1a436e39eb55623ccbd68185", size = 1453643 }, + { url = "https://files.pythonhosted.org/packages/64/31/6ce4380a4cd1f515bdda976a1e90e547ccd47b67a1546d63884463c92ca9/kiwisolver-1.4.9-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:a30fd6fdef1430fd9e1ba7b3398b5ee4e2887783917a687d86ba69985fb08748", size = 2330818 }, + { url = "https://files.pythonhosted.org/packages/fa/e9/3f3fcba3bcc7432c795b82646306e822f3fd74df0ee81f0fa067a1f95668/kiwisolver-1.4.9-cp313-cp313t-musllinux_1_2_ppc64le.whl", hash = "sha256:cc9617b46837c6468197b5945e196ee9ca43057bb7d9d1ae688101e4e1dddf64", size = 2419963 }, + { url = "https://files.pythonhosted.org/packages/99/43/7320c50e4133575c66e9f7dadead35ab22d7c012a3b09bb35647792b2a6d/kiwisolver-1.4.9-cp313-cp313t-musllinux_1_2_s390x.whl", hash = "sha256:0ab74e19f6a2b027ea4f845a78827969af45ce790e6cb3e1ebab71bdf9f215ff", size = 2594639 }, + { url = "https://files.pythonhosted.org/packages/65/d6/17ae4a270d4a987ef8a385b906d2bdfc9fce502d6dc0d3aea865b47f548c/kiwisolver-1.4.9-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:dba5ee5d3981160c28d5490f0d1b7ed730c22470ff7f6cc26cfcfaacb9896a07", size = 2391741 }, + { url = "https://files.pythonhosted.org/packages/2a/8f/8f6f491d595a9e5912971f3f863d81baddccc8a4d0c3749d6a0dd9ffc9df/kiwisolver-1.4.9-cp313-cp313t-win_arm64.whl", hash = "sha256:0749fd8f4218ad2e851e11cc4dc05c7cbc0cbc4267bdfdb31782e65aace4ee9c", size = 68646 }, + { url = "https://files.pythonhosted.org/packages/6b/32/6cc0fbc9c54d06c2969faa9c1d29f5751a2e51809dd55c69055e62d9b426/kiwisolver-1.4.9-cp314-cp314-macosx_10_13_universal2.whl", hash = "sha256:9928fe1eb816d11ae170885a74d074f57af3a0d65777ca47e9aeb854a1fba386", size = 123806 }, + { url = "https://files.pythonhosted.org/packages/b2/dd/2bfb1d4a4823d92e8cbb420fe024b8d2167f72079b3bb941207c42570bdf/kiwisolver-1.4.9-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:d0005b053977e7b43388ddec89fa567f43d4f6d5c2c0affe57de5ebf290dc552", size = 66605 }, + { url = "https://files.pythonhosted.org/packages/f7/69/00aafdb4e4509c2ca6064646cba9cd4b37933898f426756adb2cb92ebbed/kiwisolver-1.4.9-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:2635d352d67458b66fd0667c14cb1d4145e9560d503219034a18a87e971ce4f3", size = 64925 }, + { url = "https://files.pythonhosted.org/packages/43/dc/51acc6791aa14e5cb6d8a2e28cefb0dc2886d8862795449d021334c0df20/kiwisolver-1.4.9-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:767c23ad1c58c9e827b649a9ab7809fd5fd9db266a9cf02b0e926ddc2c680d58", size = 1472414 }, + { url = "https://files.pythonhosted.org/packages/3d/bb/93fa64a81db304ac8a246f834d5094fae4b13baf53c839d6bb6e81177129/kiwisolver-1.4.9-cp314-cp314-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:72d0eb9fba308b8311685c2268cf7d0a0639a6cd027d8128659f72bdd8a024b4", size = 1281272 }, + { url = "https://files.pythonhosted.org/packages/70/e6/6df102916960fb8d05069d4bd92d6d9a8202d5a3e2444494e7cd50f65b7a/kiwisolver-1.4.9-cp314-cp314-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:f68e4f3eeca8fb22cc3d731f9715a13b652795ef657a13df1ad0c7dc0e9731df", size = 1298578 }, + { url = "https://files.pythonhosted.org/packages/7c/47/e142aaa612f5343736b087864dbaebc53ea8831453fb47e7521fa8658f30/kiwisolver-1.4.9-cp314-cp314-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:d84cd4061ae292d8ac367b2c3fa3aad11cb8625a95d135fe93f286f914f3f5a6", size = 1345607 }, + { url = "https://files.pythonhosted.org/packages/54/89/d641a746194a0f4d1a3670fb900d0dbaa786fb98341056814bc3f058fa52/kiwisolver-1.4.9-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:a60ea74330b91bd22a29638940d115df9dc00af5035a9a2a6ad9399ffb4ceca5", size = 2230150 }, + { url = "https://files.pythonhosted.org/packages/aa/6b/5ee1207198febdf16ac11f78c5ae40861b809cbe0e6d2a8d5b0b3044b199/kiwisolver-1.4.9-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:ce6a3a4e106cf35c2d9c4fa17c05ce0b180db622736845d4315519397a77beaf", size = 2325979 }, + { url = "https://files.pythonhosted.org/packages/fc/ff/b269eefd90f4ae14dcc74973d5a0f6d28d3b9bb1afd8c0340513afe6b39a/kiwisolver-1.4.9-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:77937e5e2a38a7b48eef0585114fe7930346993a88060d0bf886086d2aa49ef5", size = 2491456 }, + { url = "https://files.pythonhosted.org/packages/fc/d4/10303190bd4d30de547534601e259a4fbf014eed94aae3e5521129215086/kiwisolver-1.4.9-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:24c175051354f4a28c5d6a31c93906dc653e2bf234e8a4bbfb964892078898ce", size = 2294621 }, + { url = "https://files.pythonhosted.org/packages/28/e0/a9a90416fce5c0be25742729c2ea52105d62eda6c4be4d803c2a7be1fa50/kiwisolver-1.4.9-cp314-cp314-win_amd64.whl", hash = "sha256:0763515d4df10edf6d06a3c19734e2566368980d21ebec439f33f9eb936c07b7", size = 75417 }, + { url = "https://files.pythonhosted.org/packages/1f/10/6949958215b7a9a264299a7db195564e87900f709db9245e4ebdd3c70779/kiwisolver-1.4.9-cp314-cp314-win_arm64.whl", hash = "sha256:0e4e2bf29574a6a7b7f6cb5fa69293b9f96c928949ac4a53ba3f525dffb87f9c", size = 66582 }, + { url = "https://files.pythonhosted.org/packages/ec/79/60e53067903d3bc5469b369fe0dfc6b3482e2133e85dae9daa9527535991/kiwisolver-1.4.9-cp314-cp314t-macosx_10_13_universal2.whl", hash = "sha256:d976bbb382b202f71c67f77b0ac11244021cfa3f7dfd9e562eefcea2df711548", size = 126514 }, + { url = "https://files.pythonhosted.org/packages/25/d1/4843d3e8d46b072c12a38c97c57fab4608d36e13fe47d47ee96b4d61ba6f/kiwisolver-1.4.9-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:2489e4e5d7ef9a1c300a5e0196e43d9c739f066ef23270607d45aba368b91f2d", size = 67905 }, + { url = "https://files.pythonhosted.org/packages/8c/ae/29ffcbd239aea8b93108de1278271ae764dfc0d803a5693914975f200596/kiwisolver-1.4.9-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:e2ea9f7ab7fbf18fffb1b5434ce7c69a07582f7acc7717720f1d69f3e806f90c", size = 66399 }, + { url = "https://files.pythonhosted.org/packages/a1/ae/d7ba902aa604152c2ceba5d352d7b62106bedbccc8e95c3934d94472bfa3/kiwisolver-1.4.9-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:b34e51affded8faee0dfdb705416153819d8ea9250bbbf7ea1b249bdeb5f1122", size = 1582197 }, + { url = "https://files.pythonhosted.org/packages/f2/41/27c70d427eddb8bc7e4f16420a20fefc6f480312122a59a959fdfe0445ad/kiwisolver-1.4.9-cp314-cp314t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d8aacd3d4b33b772542b2e01beb50187536967b514b00003bdda7589722d2a64", size = 1390125 }, + { url = "https://files.pythonhosted.org/packages/41/42/b3799a12bafc76d962ad69083f8b43b12bf4fe78b097b12e105d75c9b8f1/kiwisolver-1.4.9-cp314-cp314t-manylinux_2_24_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:7cf974dd4e35fa315563ac99d6287a1024e4dc2077b8a7d7cd3d2fb65d283134", size = 1402612 }, + { url = "https://files.pythonhosted.org/packages/d2/b5/a210ea073ea1cfaca1bb5c55a62307d8252f531beb364e18aa1e0888b5a0/kiwisolver-1.4.9-cp314-cp314t-manylinux_2_24_s390x.manylinux_2_28_s390x.whl", hash = "sha256:85bd218b5ecfbee8c8a82e121802dcb519a86044c9c3b2e4aef02fa05c6da370", size = 1453990 }, + { url = "https://files.pythonhosted.org/packages/5f/ce/a829eb8c033e977d7ea03ed32fb3c1781b4fa0433fbadfff29e39c676f32/kiwisolver-1.4.9-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:0856e241c2d3df4efef7c04a1e46b1936b6120c9bcf36dd216e3acd84bc4fb21", size = 2331601 }, + { url = "https://files.pythonhosted.org/packages/e0/4b/b5e97eb142eb9cd0072dacfcdcd31b1c66dc7352b0f7c7255d339c0edf00/kiwisolver-1.4.9-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:9af39d6551f97d31a4deebeac6f45b156f9755ddc59c07b402c148f5dbb6482a", size = 2422041 }, + { url = "https://files.pythonhosted.org/packages/40/be/8eb4cd53e1b85ba4edc3a9321666f12b83113a178845593307a3e7891f44/kiwisolver-1.4.9-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:bb4ae2b57fc1d8cbd1cf7b1d9913803681ffa903e7488012be5b76dedf49297f", size = 2594897 }, + { url = "https://files.pythonhosted.org/packages/99/dd/841e9a66c4715477ea0abc78da039832fbb09dac5c35c58dc4c41a407b8a/kiwisolver-1.4.9-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:aedff62918805fb62d43a4aa2ecd4482c380dc76cd31bd7c8878588a61bd0369", size = 2391835 }, + { url = "https://files.pythonhosted.org/packages/0c/28/4b2e5c47a0da96896fdfdb006340ade064afa1e63675d01ea5ac222b6d52/kiwisolver-1.4.9-cp314-cp314t-win_amd64.whl", hash = "sha256:1fa333e8b2ce4d9660f2cda9c0e1b6bafcfb2457a9d259faa82289e73ec24891", size = 79988 }, + { url = "https://files.pythonhosted.org/packages/80/be/3578e8afd18c88cdf9cb4cffde75a96d2be38c5a903f1ed0ceec061bd09e/kiwisolver-1.4.9-cp314-cp314t-win_arm64.whl", hash = "sha256:4a48a2ce79d65d363597ef7b567ce3d14d68783d2b2263d98db3d9477805ba32", size = 70260 }, + { url = "https://files.pythonhosted.org/packages/a3/0f/36d89194b5a32c054ce93e586d4049b6c2c22887b0eb229c61c68afd3078/kiwisolver-1.4.9-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:720e05574713db64c356e86732c0f3c5252818d05f9df320f0ad8380641acea5", size = 60104 }, + { url = "https://files.pythonhosted.org/packages/52/ba/4ed75f59e4658fd21fe7dde1fee0ac397c678ec3befba3fe6482d987af87/kiwisolver-1.4.9-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:17680d737d5335b552994a2008fab4c851bcd7de33094a82067ef3a576ff02fa", size = 58592 }, + { url = "https://files.pythonhosted.org/packages/33/01/a8ea7c5ea32a9b45ceeaee051a04c8ed4320f5add3c51bfa20879b765b70/kiwisolver-1.4.9-pp311-pypy311_pp73-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:85b5352f94e490c028926ea567fc569c52ec79ce131dadb968d3853e809518c2", size = 80281 }, + { url = "https://files.pythonhosted.org/packages/da/e3/dbd2ecdce306f1d07a1aaf324817ee993aab7aee9db47ceac757deabafbe/kiwisolver-1.4.9-pp311-pypy311_pp73-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:464415881e4801295659462c49461a24fb107c140de781d55518c4b80cb6790f", size = 78009 }, + { url = "https://files.pythonhosted.org/packages/da/e9/0d4add7873a73e462aeb45c036a2dead2562b825aa46ba326727b3f31016/kiwisolver-1.4.9-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:fb940820c63a9590d31d88b815e7a3aa5915cad3ce735ab45f0c730b39547de1", size = 73929 }, +] + +[[package]] +name = "loguru" +version = "0.7.3" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "colorama", marker = "sys_platform == 'win32'" }, + { name = "win32-setctime", marker = "sys_platform == 'win32'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/3a/05/a1dae3dffd1116099471c643b8924f5aa6524411dc6c63fdae648c4f1aca/loguru-0.7.3.tar.gz", hash = "sha256:19480589e77d47b8d85b2c827ad95d49bf31b0dcde16593892eb51dd18706eb6", size = 63559 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/0c/29/0348de65b8cc732daa3e33e67806420b2ae89bdce2b04af740289c5c6c8c/loguru-0.7.3-py3-none-any.whl", hash = "sha256:31a33c10c8e1e10422bfd431aeb5d351c7cf7fa671e3c4df004162264b28220c", size = 61595 }, +] + +[[package]] +name = "lxml" +version = "6.0.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/aa/88/262177de60548e5a2bfc46ad28232c9e9cbde697bd94132aeb80364675cb/lxml-6.0.2.tar.gz", hash = "sha256:cd79f3367bd74b317dda655dc8fcfa304d9eb6e4fb06b7168c5cf27f96e0cd62", size = 4073426 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/77/d5/becbe1e2569b474a23f0c672ead8a29ac50b2dc1d5b9de184831bda8d14c/lxml-6.0.2-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:13e35cbc684aadf05d8711a5d1b5857c92e5e580efa9a0d2be197199c8def607", size = 8634365 }, + { url = "https://files.pythonhosted.org/packages/28/66/1ced58f12e804644426b85d0bb8a4478ca77bc1761455da310505f1a3526/lxml-6.0.2-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:3b1675e096e17c6fe9c0e8c81434f5736c0739ff9ac6123c87c2d452f48fc938", size = 4650793 }, + { url = "https://files.pythonhosted.org/packages/11/84/549098ffea39dfd167e3f174b4ce983d0eed61f9d8d25b7bf2a57c3247fc/lxml-6.0.2-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:8ac6e5811ae2870953390452e3476694196f98d447573234592d30488147404d", size = 4944362 }, + { url = "https://files.pythonhosted.org/packages/ac/bd/f207f16abf9749d2037453d56b643a7471d8fde855a231a12d1e095c4f01/lxml-6.0.2-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:5aa0fc67ae19d7a64c3fe725dc9a1bb11f80e01f78289d05c6f62545affec438", size = 5083152 }, + { url = "https://files.pythonhosted.org/packages/15/ae/bd813e87d8941d52ad5b65071b1affb48da01c4ed3c9c99e40abb266fbff/lxml-6.0.2-cp311-cp311-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:de496365750cc472b4e7902a485d3f152ecf57bd3ba03ddd5578ed8ceb4c5964", size = 5023539 }, + { url = "https://files.pythonhosted.org/packages/02/cd/9bfef16bd1d874fbe0cb51afb00329540f30a3283beb9f0780adbb7eec03/lxml-6.0.2-cp311-cp311-manylinux_2_26_i686.manylinux_2_28_i686.whl", hash = "sha256:200069a593c5e40b8f6fc0d84d86d970ba43138c3e68619ffa234bc9bb806a4d", size = 5344853 }, + { url = "https://files.pythonhosted.org/packages/b8/89/ea8f91594bc5dbb879734d35a6f2b0ad50605d7fb419de2b63d4211765cc/lxml-6.0.2-cp311-cp311-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:7d2de809c2ee3b888b59f995625385f74629707c9355e0ff856445cdcae682b7", size = 5225133 }, + { url = "https://files.pythonhosted.org/packages/b9/37/9c735274f5dbec726b2db99b98a43950395ba3d4a1043083dba2ad814170/lxml-6.0.2-cp311-cp311-manylinux_2_31_armv7l.whl", hash = "sha256:b2c3da8d93cf5db60e8858c17684c47d01fee6405e554fb55018dd85fc23b178", size = 4677944 }, + { url = "https://files.pythonhosted.org/packages/20/28/7dfe1ba3475d8bfca3878365075abe002e05d40dfaaeb7ec01b4c587d533/lxml-6.0.2-cp311-cp311-manylinux_2_38_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:442de7530296ef5e188373a1ea5789a46ce90c4847e597856570439621d9c553", size = 5284535 }, + { url = "https://files.pythonhosted.org/packages/e7/cf/5f14bc0de763498fc29510e3532bf2b4b3a1c1d5d0dff2e900c16ba021ef/lxml-6.0.2-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:2593c77efde7bfea7f6389f1ab249b15ed4aa5bc5cb5131faa3b843c429fbedb", size = 5067343 }, + { url = "https://files.pythonhosted.org/packages/1c/b0/bb8275ab5472f32b28cfbbcc6db7c9d092482d3439ca279d8d6fa02f7025/lxml-6.0.2-cp311-cp311-musllinux_1_2_armv7l.whl", hash = "sha256:3e3cb08855967a20f553ff32d147e14329b3ae70ced6edc2f282b94afbc74b2a", size = 4725419 }, + { url = "https://files.pythonhosted.org/packages/25/4c/7c222753bc72edca3b99dbadba1b064209bc8ed4ad448af990e60dcce462/lxml-6.0.2-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:2ed6c667fcbb8c19c6791bbf40b7268ef8ddf5a96940ba9404b9f9a304832f6c", size = 5275008 }, + { url = "https://files.pythonhosted.org/packages/6c/8c/478a0dc6b6ed661451379447cdbec77c05741a75736d97e5b2b729687828/lxml-6.0.2-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:b8f18914faec94132e5b91e69d76a5c1d7b0c73e2489ea8929c4aaa10b76bbf7", size = 5248906 }, + { url = "https://files.pythonhosted.org/packages/2d/d9/5be3a6ab2784cdf9accb0703b65e1b64fcdd9311c9f007630c7db0cfcce1/lxml-6.0.2-cp311-cp311-win32.whl", hash = "sha256:6605c604e6daa9e0d7f0a2137bdc47a2e93b59c60a65466353e37f8272f47c46", size = 3610357 }, + { url = "https://files.pythonhosted.org/packages/e2/7d/ca6fb13349b473d5732fb0ee3eec8f6c80fc0688e76b7d79c1008481bf1f/lxml-6.0.2-cp311-cp311-win_amd64.whl", hash = "sha256:e5867f2651016a3afd8dd2c8238baa66f1e2802f44bc17e236f547ace6647078", size = 4036583 }, + { url = "https://files.pythonhosted.org/packages/ab/a2/51363b5ecd3eab46563645f3a2c3836a2fc67d01a1b87c5017040f39f567/lxml-6.0.2-cp311-cp311-win_arm64.whl", hash = "sha256:4197fb2534ee05fd3e7afaab5d8bfd6c2e186f65ea7f9cd6a82809c887bd1285", size = 3680591 }, + { url = "https://files.pythonhosted.org/packages/f3/c8/8ff2bc6b920c84355146cd1ab7d181bc543b89241cfb1ebee824a7c81457/lxml-6.0.2-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:a59f5448ba2ceccd06995c95ea59a7674a10de0810f2ce90c9006f3cbc044456", size = 8661887 }, + { url = "https://files.pythonhosted.org/packages/37/6f/9aae1008083bb501ef63284220ce81638332f9ccbfa53765b2b7502203cf/lxml-6.0.2-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:e8113639f3296706fbac34a30813929e29247718e88173ad849f57ca59754924", size = 4667818 }, + { url = "https://files.pythonhosted.org/packages/f1/ca/31fb37f99f37f1536c133476674c10b577e409c0a624384147653e38baf2/lxml-6.0.2-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:a8bef9b9825fa8bc816a6e641bb67219489229ebc648be422af695f6e7a4fa7f", size = 4950807 }, + { url = "https://files.pythonhosted.org/packages/da/87/f6cb9442e4bada8aab5ae7e1046264f62fdbeaa6e3f6211b93f4c0dd97f1/lxml-6.0.2-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:65ea18d710fd14e0186c2f973dc60bb52039a275f82d3c44a0e42b43440ea534", size = 5109179 }, + { url = "https://files.pythonhosted.org/packages/c8/20/a7760713e65888db79bbae4f6146a6ae5c04e4a204a3c48896c408cd6ed2/lxml-6.0.2-cp312-cp312-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c371aa98126a0d4c739ca93ceffa0fd7a5d732e3ac66a46e74339acd4d334564", size = 5023044 }, + { url = "https://files.pythonhosted.org/packages/a2/b0/7e64e0460fcb36471899f75831509098f3fd7cd02a3833ac517433cb4f8f/lxml-6.0.2-cp312-cp312-manylinux_2_26_i686.manylinux_2_28_i686.whl", hash = "sha256:700efd30c0fa1a3581d80a748157397559396090a51d306ea59a70020223d16f", size = 5359685 }, + { url = "https://files.pythonhosted.org/packages/b9/e1/e5df362e9ca4e2f48ed6411bd4b3a0ae737cc842e96877f5bf9428055ab4/lxml-6.0.2-cp312-cp312-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:c33e66d44fe60e72397b487ee92e01da0d09ba2d66df8eae42d77b6d06e5eba0", size = 5654127 }, + { url = "https://files.pythonhosted.org/packages/c6/d1/232b3309a02d60f11e71857778bfcd4acbdb86c07db8260caf7d008b08f8/lxml-6.0.2-cp312-cp312-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:90a345bbeaf9d0587a3aaffb7006aa39ccb6ff0e96a57286c0cb2fd1520ea192", size = 5253958 }, + { url = "https://files.pythonhosted.org/packages/35/35/d955a070994725c4f7d80583a96cab9c107c57a125b20bb5f708fe941011/lxml-6.0.2-cp312-cp312-manylinux_2_31_armv7l.whl", hash = "sha256:064fdadaf7a21af3ed1dcaa106b854077fbeada827c18f72aec9346847cd65d0", size = 4711541 }, + { url = "https://files.pythonhosted.org/packages/1e/be/667d17363b38a78c4bd63cfd4b4632029fd68d2c2dc81f25ce9eb5224dd5/lxml-6.0.2-cp312-cp312-manylinux_2_38_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:fbc74f42c3525ac4ffa4b89cbdd00057b6196bcefe8bce794abd42d33a018092", size = 5267426 }, + { url = "https://files.pythonhosted.org/packages/ea/47/62c70aa4a1c26569bc958c9ca86af2bb4e1f614e8c04fb2989833874f7ae/lxml-6.0.2-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:6ddff43f702905a4e32bc24f3f2e2edfe0f8fde3277d481bffb709a4cced7a1f", size = 5064917 }, + { url = "https://files.pythonhosted.org/packages/bd/55/6ceddaca353ebd0f1908ef712c597f8570cc9c58130dbb89903198e441fd/lxml-6.0.2-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:6da5185951d72e6f5352166e3da7b0dc27aa70bd1090b0eb3f7f7212b53f1bb8", size = 4788795 }, + { url = "https://files.pythonhosted.org/packages/cf/e8/fd63e15da5e3fd4c2146f8bbb3c14e94ab850589beab88e547b2dbce22e1/lxml-6.0.2-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:57a86e1ebb4020a38d295c04fc79603c7899e0df71588043eb218722dabc087f", size = 5676759 }, + { url = "https://files.pythonhosted.org/packages/76/47/b3ec58dc5c374697f5ba37412cd2728f427d056315d124dd4b61da381877/lxml-6.0.2-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:2047d8234fe735ab77802ce5f2297e410ff40f5238aec569ad7c8e163d7b19a6", size = 5255666 }, + { url = "https://files.pythonhosted.org/packages/19/93/03ba725df4c3d72afd9596eef4a37a837ce8e4806010569bedfcd2cb68fd/lxml-6.0.2-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:6f91fd2b2ea15a6800c8e24418c0775a1694eefc011392da73bc6cef2623b322", size = 5277989 }, + { url = "https://files.pythonhosted.org/packages/c6/80/c06de80bfce881d0ad738576f243911fccf992687ae09fd80b734712b39c/lxml-6.0.2-cp312-cp312-win32.whl", hash = "sha256:3ae2ce7d6fedfb3414a2b6c5e20b249c4c607f72cb8d2bb7cc9c6ec7c6f4e849", size = 3611456 }, + { url = "https://files.pythonhosted.org/packages/f7/d7/0cdfb6c3e30893463fb3d1e52bc5f5f99684a03c29a0b6b605cfae879cd5/lxml-6.0.2-cp312-cp312-win_amd64.whl", hash = "sha256:72c87e5ee4e58a8354fb9c7c84cbf95a1c8236c127a5d1b7683f04bed8361e1f", size = 4011793 }, + { url = "https://files.pythonhosted.org/packages/ea/7b/93c73c67db235931527301ed3785f849c78991e2e34f3fd9a6663ffda4c5/lxml-6.0.2-cp312-cp312-win_arm64.whl", hash = "sha256:61cb10eeb95570153e0c0e554f58df92ecf5109f75eacad4a95baa709e26c3d6", size = 3672836 }, + { url = "https://files.pythonhosted.org/packages/53/fd/4e8f0540608977aea078bf6d79f128e0e2c2bba8af1acf775c30baa70460/lxml-6.0.2-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:9b33d21594afab46f37ae58dfadd06636f154923c4e8a4d754b0127554eb2e77", size = 8648494 }, + { url = "https://files.pythonhosted.org/packages/5d/f4/2a94a3d3dfd6c6b433501b8d470a1960a20ecce93245cf2db1706adf6c19/lxml-6.0.2-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:6c8963287d7a4c5c9a432ff487c52e9c5618667179c18a204bdedb27310f022f", size = 4661146 }, + { url = "https://files.pythonhosted.org/packages/25/2e/4efa677fa6b322013035d38016f6ae859d06cac67437ca7dc708a6af7028/lxml-6.0.2-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:1941354d92699fb5ffe6ed7b32f9649e43c2feb4b97205f75866f7d21aa91452", size = 4946932 }, + { url = "https://files.pythonhosted.org/packages/ce/0f/526e78a6d38d109fdbaa5049c62e1d32fdd70c75fb61c4eadf3045d3d124/lxml-6.0.2-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:bb2f6ca0ae2d983ded09357b84af659c954722bbf04dea98030064996d156048", size = 5100060 }, + { url = "https://files.pythonhosted.org/packages/81/76/99de58d81fa702cc0ea7edae4f4640416c2062813a00ff24bd70ac1d9c9b/lxml-6.0.2-cp313-cp313-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:eb2a12d704f180a902d7fa778c6d71f36ceb7b0d317f34cdc76a5d05aa1dd1df", size = 5019000 }, + { url = "https://files.pythonhosted.org/packages/b5/35/9e57d25482bc9a9882cb0037fdb9cc18f4b79d85df94fa9d2a89562f1d25/lxml-6.0.2-cp313-cp313-manylinux_2_26_i686.manylinux_2_28_i686.whl", hash = "sha256:6ec0e3f745021bfed19c456647f0298d60a24c9ff86d9d051f52b509663feeb1", size = 5348496 }, + { url = "https://files.pythonhosted.org/packages/a6/8e/cb99bd0b83ccc3e8f0f528e9aa1f7a9965dfec08c617070c5db8d63a87ce/lxml-6.0.2-cp313-cp313-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:846ae9a12d54e368933b9759052d6206a9e8b250291109c48e350c1f1f49d916", size = 5643779 }, + { url = "https://files.pythonhosted.org/packages/d0/34/9e591954939276bb679b73773836c6684c22e56d05980e31d52a9a8deb18/lxml-6.0.2-cp313-cp313-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ef9266d2aa545d7374938fb5c484531ef5a2ec7f2d573e62f8ce722c735685fd", size = 5244072 }, + { url = "https://files.pythonhosted.org/packages/8d/27/b29ff065f9aaca443ee377aff699714fcbffb371b4fce5ac4ca759e436d5/lxml-6.0.2-cp313-cp313-manylinux_2_31_armv7l.whl", hash = "sha256:4077b7c79f31755df33b795dc12119cb557a0106bfdab0d2c2d97bd3cf3dffa6", size = 4718675 }, + { url = "https://files.pythonhosted.org/packages/2b/9f/f756f9c2cd27caa1a6ef8c32ae47aadea697f5c2c6d07b0dae133c244fbe/lxml-6.0.2-cp313-cp313-manylinux_2_38_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:a7c5d5e5f1081955358533be077166ee97ed2571d6a66bdba6ec2f609a715d1a", size = 5255171 }, + { url = "https://files.pythonhosted.org/packages/61/46/bb85ea42d2cb1bd8395484fd72f38e3389611aa496ac7772da9205bbda0e/lxml-6.0.2-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:8f8d0cbd0674ee89863a523e6994ac25fd5be9c8486acfc3e5ccea679bad2679", size = 5057175 }, + { url = "https://files.pythonhosted.org/packages/95/0c/443fc476dcc8e41577f0af70458c50fe299a97bb6b7505bb1ae09aa7f9ac/lxml-6.0.2-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:2cbcbf6d6e924c28f04a43f3b6f6e272312a090f269eff68a2982e13e5d57659", size = 4785688 }, + { url = "https://files.pythonhosted.org/packages/48/78/6ef0b359d45bb9697bc5a626e1992fa5d27aa3f8004b137b2314793b50a0/lxml-6.0.2-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:dfb874cfa53340009af6bdd7e54ebc0d21012a60a4e65d927c2e477112e63484", size = 5660655 }, + { url = "https://files.pythonhosted.org/packages/ff/ea/e1d33808f386bc1339d08c0dcada6e4712d4ed8e93fcad5f057070b7988a/lxml-6.0.2-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:fb8dae0b6b8b7f9e96c26fdd8121522ce5de9bb5538010870bd538683d30e9a2", size = 5247695 }, + { url = "https://files.pythonhosted.org/packages/4f/47/eba75dfd8183673725255247a603b4ad606f4ae657b60c6c145b381697da/lxml-6.0.2-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:358d9adae670b63e95bc59747c72f4dc97c9ec58881d4627fe0120da0f90d314", size = 5269841 }, + { url = "https://files.pythonhosted.org/packages/76/04/5c5e2b8577bc936e219becb2e98cdb1aca14a4921a12995b9d0c523502ae/lxml-6.0.2-cp313-cp313-win32.whl", hash = "sha256:e8cd2415f372e7e5a789d743d133ae474290a90b9023197fd78f32e2dc6873e2", size = 3610700 }, + { url = "https://files.pythonhosted.org/packages/fe/0a/4643ccc6bb8b143e9f9640aa54e38255f9d3b45feb2cbe7ae2ca47e8782e/lxml-6.0.2-cp313-cp313-win_amd64.whl", hash = "sha256:b30d46379644fbfc3ab81f8f82ae4de55179414651f110a1514f0b1f8f6cb2d7", size = 4010347 }, + { url = "https://files.pythonhosted.org/packages/31/ef/dcf1d29c3f530577f61e5fe2f1bd72929acf779953668a8a47a479ae6f26/lxml-6.0.2-cp313-cp313-win_arm64.whl", hash = "sha256:13dcecc9946dca97b11b7c40d29fba63b55ab4170d3c0cf8c0c164343b9bfdcf", size = 3671248 }, + { url = "https://files.pythonhosted.org/packages/03/15/d4a377b385ab693ce97b472fe0c77c2b16ec79590e688b3ccc71fba19884/lxml-6.0.2-cp314-cp314-macosx_10_13_universal2.whl", hash = "sha256:b0c732aa23de8f8aec23f4b580d1e52905ef468afb4abeafd3fec77042abb6fe", size = 8659801 }, + { url = "https://files.pythonhosted.org/packages/c8/e8/c128e37589463668794d503afaeb003987373c5f94d667124ffd8078bbd9/lxml-6.0.2-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:4468e3b83e10e0317a89a33d28f7aeba1caa4d1a6fd457d115dd4ffe90c5931d", size = 4659403 }, + { url = "https://files.pythonhosted.org/packages/00/ce/74903904339decdf7da7847bb5741fc98a5451b42fc419a86c0c13d26fe2/lxml-6.0.2-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:abd44571493973bad4598a3be7e1d807ed45aa2adaf7ab92ab7c62609569b17d", size = 4966974 }, + { url = "https://files.pythonhosted.org/packages/1f/d3/131dec79ce61c5567fecf82515bd9bc36395df42501b50f7f7f3bd065df0/lxml-6.0.2-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:370cd78d5855cfbffd57c422851f7d3864e6ae72d0da615fca4dad8c45d375a5", size = 5102953 }, + { url = "https://files.pythonhosted.org/packages/3a/ea/a43ba9bb750d4ffdd885f2cd333572f5bb900cd2408b67fdda07e85978a0/lxml-6.0.2-cp314-cp314-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:901e3b4219fa04ef766885fb40fa516a71662a4c61b80c94d25336b4934b71c0", size = 5055054 }, + { url = "https://files.pythonhosted.org/packages/60/23/6885b451636ae286c34628f70a7ed1fcc759f8d9ad382d132e1c8d3d9bfd/lxml-6.0.2-cp314-cp314-manylinux_2_26_i686.manylinux_2_28_i686.whl", hash = "sha256:a4bf42d2e4cf52c28cc1812d62426b9503cdb0c87a6de81442626aa7d69707ba", size = 5352421 }, + { url = "https://files.pythonhosted.org/packages/48/5b/fc2ddfc94ddbe3eebb8e9af6e3fd65e2feba4967f6a4e9683875c394c2d8/lxml-6.0.2-cp314-cp314-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:b2c7fdaa4d7c3d886a42534adec7cfac73860b89b4e5298752f60aa5984641a0", size = 5673684 }, + { url = "https://files.pythonhosted.org/packages/29/9c/47293c58cc91769130fbf85531280e8cc7868f7fbb6d92f4670071b9cb3e/lxml-6.0.2-cp314-cp314-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:98a5e1660dc7de2200b00d53fa00bcd3c35a3608c305d45a7bbcaf29fa16e83d", size = 5252463 }, + { url = "https://files.pythonhosted.org/packages/9b/da/ba6eceb830c762b48e711ded880d7e3e89fc6c7323e587c36540b6b23c6b/lxml-6.0.2-cp314-cp314-manylinux_2_31_armv7l.whl", hash = "sha256:dc051506c30b609238d79eda75ee9cab3e520570ec8219844a72a46020901e37", size = 4698437 }, + { url = "https://files.pythonhosted.org/packages/a5/24/7be3f82cb7990b89118d944b619e53c656c97dc89c28cfb143fdb7cd6f4d/lxml-6.0.2-cp314-cp314-manylinux_2_38_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:8799481bbdd212470d17513a54d568f44416db01250f49449647b5ab5b5dccb9", size = 5269890 }, + { url = "https://files.pythonhosted.org/packages/1b/bd/dcfb9ea1e16c665efd7538fc5d5c34071276ce9220e234217682e7d2c4a5/lxml-6.0.2-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:9261bb77c2dab42f3ecd9103951aeca2c40277701eb7e912c545c1b16e0e4917", size = 5097185 }, + { url = "https://files.pythonhosted.org/packages/21/04/a60b0ff9314736316f28316b694bccbbabe100f8483ad83852d77fc7468e/lxml-6.0.2-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:65ac4a01aba353cfa6d5725b95d7aed6356ddc0a3cd734de00124d285b04b64f", size = 4745895 }, + { url = "https://files.pythonhosted.org/packages/d6/bd/7d54bd1846e5a310d9c715921c5faa71cf5c0853372adf78aee70c8d7aa2/lxml-6.0.2-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:b22a07cbb82fea98f8a2fd814f3d1811ff9ed76d0fc6abc84eb21527596e7cc8", size = 5695246 }, + { url = "https://files.pythonhosted.org/packages/fd/32/5643d6ab947bc371da21323acb2a6e603cedbe71cb4c99c8254289ab6f4e/lxml-6.0.2-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:d759cdd7f3e055d6bc8d9bec3ad905227b2e4c785dc16c372eb5b5e83123f48a", size = 5260797 }, + { url = "https://files.pythonhosted.org/packages/33/da/34c1ec4cff1eea7d0b4cd44af8411806ed943141804ac9c5d565302afb78/lxml-6.0.2-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:945da35a48d193d27c188037a05fec5492937f66fb1958c24fc761fb9d40d43c", size = 5277404 }, + { url = "https://files.pythonhosted.org/packages/82/57/4eca3e31e54dc89e2c3507e1cd411074a17565fa5ffc437c4ae0a00d439e/lxml-6.0.2-cp314-cp314-win32.whl", hash = "sha256:be3aaa60da67e6153eb15715cc2e19091af5dc75faef8b8a585aea372507384b", size = 3670072 }, + { url = "https://files.pythonhosted.org/packages/e3/e0/c96cf13eccd20c9421ba910304dae0f619724dcf1702864fd59dd386404d/lxml-6.0.2-cp314-cp314-win_amd64.whl", hash = "sha256:fa25afbadead523f7001caf0c2382afd272c315a033a7b06336da2637d92d6ed", size = 4080617 }, + { url = "https://files.pythonhosted.org/packages/d5/5d/b3f03e22b3d38d6f188ef044900a9b29b2fe0aebb94625ce9fe244011d34/lxml-6.0.2-cp314-cp314-win_arm64.whl", hash = "sha256:063eccf89df5b24e361b123e257e437f9e9878f425ee9aae3144c77faf6da6d8", size = 3754930 }, + { url = "https://files.pythonhosted.org/packages/5e/5c/42c2c4c03554580708fc738d13414801f340c04c3eff90d8d2d227145275/lxml-6.0.2-cp314-cp314t-macosx_10_13_universal2.whl", hash = "sha256:6162a86d86893d63084faaf4ff937b3daea233e3682fb4474db07395794fa80d", size = 8910380 }, + { url = "https://files.pythonhosted.org/packages/bf/4f/12df843e3e10d18d468a7557058f8d3733e8b6e12401f30b1ef29360740f/lxml-6.0.2-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:414aaa94e974e23a3e92e7ca5b97d10c0cf37b6481f50911032c69eeb3991bba", size = 4775632 }, + { url = "https://files.pythonhosted.org/packages/e4/0c/9dc31e6c2d0d418483cbcb469d1f5a582a1cd00a1f4081953d44051f3c50/lxml-6.0.2-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:48461bd21625458dd01e14e2c38dd0aea69addc3c4f960c30d9f59d7f93be601", size = 4975171 }, + { url = "https://files.pythonhosted.org/packages/e7/2b/9b870c6ca24c841bdd887504808f0417aa9d8d564114689266f19ddf29c8/lxml-6.0.2-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:25fcc59afc57d527cfc78a58f40ab4c9b8fd096a9a3f964d2781ffb6eb33f4ed", size = 5110109 }, + { url = "https://files.pythonhosted.org/packages/bf/0c/4f5f2a4dd319a178912751564471355d9019e220c20d7db3fb8307ed8582/lxml-6.0.2-cp314-cp314t-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5179c60288204e6ddde3f774a93350177e08876eaf3ab78aa3a3649d43eb7d37", size = 5041061 }, + { url = "https://files.pythonhosted.org/packages/12/64/554eed290365267671fe001a20d72d14f468ae4e6acef1e179b039436967/lxml-6.0.2-cp314-cp314t-manylinux_2_26_i686.manylinux_2_28_i686.whl", hash = "sha256:967aab75434de148ec80597b75062d8123cadf2943fb4281f385141e18b21338", size = 5306233 }, + { url = "https://files.pythonhosted.org/packages/7a/31/1d748aa275e71802ad9722df32a7a35034246b42c0ecdd8235412c3396ef/lxml-6.0.2-cp314-cp314t-manylinux_2_26_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:d100fcc8930d697c6561156c6810ab4a508fb264c8b6779e6e61e2ed5e7558f9", size = 5604739 }, + { url = "https://files.pythonhosted.org/packages/8f/41/2c11916bcac09ed561adccacceaedd2bf0e0b25b297ea92aab99fd03d0fa/lxml-6.0.2-cp314-cp314t-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:2ca59e7e13e5981175b8b3e4ab84d7da57993eeff53c07764dcebda0d0e64ecd", size = 5225119 }, + { url = "https://files.pythonhosted.org/packages/99/05/4e5c2873d8f17aa018e6afde417c80cc5d0c33be4854cce3ef5670c49367/lxml-6.0.2-cp314-cp314t-manylinux_2_31_armv7l.whl", hash = "sha256:957448ac63a42e2e49531b9d6c0fa449a1970dbc32467aaad46f11545be9af1d", size = 4633665 }, + { url = "https://files.pythonhosted.org/packages/0f/c9/dcc2da1bebd6275cdc723b515f93edf548b82f36a5458cca3578bc899332/lxml-6.0.2-cp314-cp314t-manylinux_2_38_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:b7fc49c37f1786284b12af63152fe1d0990722497e2d5817acfe7a877522f9a9", size = 5234997 }, + { url = "https://files.pythonhosted.org/packages/9c/e2/5172e4e7468afca64a37b81dba152fc5d90e30f9c83c7c3213d6a02a5ce4/lxml-6.0.2-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e19e0643cc936a22e837f79d01a550678da8377d7d801a14487c10c34ee49c7e", size = 5090957 }, + { url = "https://files.pythonhosted.org/packages/a5/b3/15461fd3e5cd4ddcb7938b87fc20b14ab113b92312fc97afe65cd7c85de1/lxml-6.0.2-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:1db01e5cf14345628e0cbe71067204db658e2fb8e51e7f33631f5f4735fefd8d", size = 4764372 }, + { url = "https://files.pythonhosted.org/packages/05/33/f310b987c8bf9e61c4dd8e8035c416bd3230098f5e3cfa69fc4232de7059/lxml-6.0.2-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:875c6b5ab39ad5291588aed6925fac99d0097af0dd62f33c7b43736043d4a2ec", size = 5634653 }, + { url = "https://files.pythonhosted.org/packages/70/ff/51c80e75e0bc9382158133bdcf4e339b5886c6ee2418b5199b3f1a61ed6d/lxml-6.0.2-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:cdcbed9ad19da81c480dfd6dd161886db6096083c9938ead313d94b30aadf272", size = 5233795 }, + { url = "https://files.pythonhosted.org/packages/56/4d/4856e897df0d588789dd844dbed9d91782c4ef0b327f96ce53c807e13128/lxml-6.0.2-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:80dadc234ebc532e09be1975ff538d154a7fa61ea5031c03d25178855544728f", size = 5257023 }, + { url = "https://files.pythonhosted.org/packages/0f/85/86766dfebfa87bea0ab78e9ff7a4b4b45225df4b4d3b8cc3c03c5cd68464/lxml-6.0.2-cp314-cp314t-win32.whl", hash = "sha256:da08e7bb297b04e893d91087df19638dc7a6bb858a954b0cc2b9f5053c922312", size = 3911420 }, + { url = "https://files.pythonhosted.org/packages/fe/1a/b248b355834c8e32614650b8008c69ffeb0ceb149c793961dd8c0b991bb3/lxml-6.0.2-cp314-cp314t-win_amd64.whl", hash = "sha256:252a22982dca42f6155125ac76d3432e548a7625d56f5a273ee78a5057216eca", size = 4406837 }, + { url = "https://files.pythonhosted.org/packages/92/aa/df863bcc39c5e0946263454aba394de8a9084dbaff8ad143846b0d844739/lxml-6.0.2-cp314-cp314t-win_arm64.whl", hash = "sha256:bb4c1847b303835d89d785a18801a883436cdfd5dc3d62947f9c49e24f0f5a2c", size = 3822205 }, + { url = "https://files.pythonhosted.org/packages/0b/11/29d08bc103a62c0eba8016e7ed5aeebbf1e4312e83b0b1648dd203b0e87d/lxml-6.0.2-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:1c06035eafa8404b5cf475bb37a9f6088b0aca288d4ccc9d69389750d5543700", size = 3949829 }, + { url = "https://files.pythonhosted.org/packages/12/b3/52ab9a3b31e5ab8238da241baa19eec44d2ab426532441ee607165aebb52/lxml-6.0.2-pp311-pypy311_pp73-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:c7d13103045de1bdd6fe5d61802565f1a3537d70cd3abf596aa0af62761921ee", size = 4226277 }, + { url = "https://files.pythonhosted.org/packages/a0/33/1eaf780c1baad88224611df13b1c2a9dfa460b526cacfe769103ff50d845/lxml-6.0.2-pp311-pypy311_pp73-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:0a3c150a95fbe5ac91de323aa756219ef9cf7fde5a3f00e2281e30f33fa5fa4f", size = 4330433 }, + { url = "https://files.pythonhosted.org/packages/7a/c1/27428a2ff348e994ab4f8777d3a0ad510b6b92d37718e5887d2da99952a2/lxml-6.0.2-pp311-pypy311_pp73-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:60fa43be34f78bebb27812ed90f1925ec99560b0fa1decdb7d12b84d857d31e9", size = 4272119 }, + { url = "https://files.pythonhosted.org/packages/f0/d0/3020fa12bcec4ab62f97aab026d57c2f0cfd480a558758d9ca233bb6a79d/lxml-6.0.2-pp311-pypy311_pp73-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:21c73b476d3cfe836be731225ec3421fa2f048d84f6df6a8e70433dff1376d5a", size = 4417314 }, + { url = "https://files.pythonhosted.org/packages/6c/77/d7f491cbc05303ac6801651aabeb262d43f319288c1ea96c66b1d2692ff3/lxml-6.0.2-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:27220da5be049e936c3aca06f174e8827ca6445a4353a1995584311487fc4e3e", size = 3518768 }, +] + +[[package]] +name = "markupsafe" +version = "3.0.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/7e/99/7690b6d4034fffd95959cbe0c02de8deb3098cc577c67bb6a24fe5d7caa7/markupsafe-3.0.3.tar.gz", hash = "sha256:722695808f4b6457b320fdc131280796bdceb04ab50fe1795cd540799ebe1698", size = 80313 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/08/db/fefacb2136439fc8dd20e797950e749aa1f4997ed584c62cfb8ef7c2be0e/markupsafe-3.0.3-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:1cc7ea17a6824959616c525620e387f6dd30fec8cb44f649e31712db02123dad", size = 11631 }, + { url = "https://files.pythonhosted.org/packages/e1/2e/5898933336b61975ce9dc04decbc0a7f2fee78c30353c5efba7f2d6ff27a/markupsafe-3.0.3-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:4bd4cd07944443f5a265608cc6aab442e4f74dff8088b0dfc8238647b8f6ae9a", size = 12058 }, + { url = "https://files.pythonhosted.org/packages/1d/09/adf2df3699d87d1d8184038df46a9c80d78c0148492323f4693df54e17bb/markupsafe-3.0.3-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6b5420a1d9450023228968e7e6a9ce57f65d148ab56d2313fcd589eee96a7a50", size = 24287 }, + { url = "https://files.pythonhosted.org/packages/30/ac/0273f6fcb5f42e314c6d8cd99effae6a5354604d461b8d392b5ec9530a54/markupsafe-3.0.3-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0bf2a864d67e76e5c9a34dc26ec616a66b9888e25e7b9460e1c76d3293bd9dbf", size = 22940 }, + { url = "https://files.pythonhosted.org/packages/19/ae/31c1be199ef767124c042c6c3e904da327a2f7f0cd63a0337e1eca2967a8/markupsafe-3.0.3-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:bc51efed119bc9cfdf792cdeaa4d67e8f6fcccab66ed4bfdd6bde3e59bfcbb2f", size = 21887 }, + { url = "https://files.pythonhosted.org/packages/b2/76/7edcab99d5349a4532a459e1fe64f0b0467a3365056ae550d3bcf3f79e1e/markupsafe-3.0.3-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:068f375c472b3e7acbe2d5318dea141359e6900156b5b2ba06a30b169086b91a", size = 23692 }, + { url = "https://files.pythonhosted.org/packages/a4/28/6e74cdd26d7514849143d69f0bf2399f929c37dc2b31e6829fd2045b2765/markupsafe-3.0.3-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:7be7b61bb172e1ed687f1754f8e7484f1c8019780f6f6b0786e76bb01c2ae115", size = 21471 }, + { url = "https://files.pythonhosted.org/packages/62/7e/a145f36a5c2945673e590850a6f8014318d5577ed7e5920a4b3448e0865d/markupsafe-3.0.3-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:f9e130248f4462aaa8e2552d547f36ddadbeaa573879158d721bbd33dfe4743a", size = 22923 }, + { url = "https://files.pythonhosted.org/packages/0f/62/d9c46a7f5c9adbeeeda52f5b8d802e1094e9717705a645efc71b0913a0a8/markupsafe-3.0.3-cp311-cp311-win32.whl", hash = "sha256:0db14f5dafddbb6d9208827849fad01f1a2609380add406671a26386cdf15a19", size = 14572 }, + { url = "https://files.pythonhosted.org/packages/83/8a/4414c03d3f891739326e1783338e48fb49781cc915b2e0ee052aa490d586/markupsafe-3.0.3-cp311-cp311-win_amd64.whl", hash = "sha256:de8a88e63464af587c950061a5e6a67d3632e36df62b986892331d4620a35c01", size = 15077 }, + { url = "https://files.pythonhosted.org/packages/35/73/893072b42e6862f319b5207adc9ae06070f095b358655f077f69a35601f0/markupsafe-3.0.3-cp311-cp311-win_arm64.whl", hash = "sha256:3b562dd9e9ea93f13d53989d23a7e775fdfd1066c33494ff43f5418bc8c58a5c", size = 13876 }, + { url = "https://files.pythonhosted.org/packages/5a/72/147da192e38635ada20e0a2e1a51cf8823d2119ce8883f7053879c2199b5/markupsafe-3.0.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:d53197da72cc091b024dd97249dfc7794d6a56530370992a5e1a08983ad9230e", size = 11615 }, + { url = "https://files.pythonhosted.org/packages/9a/81/7e4e08678a1f98521201c3079f77db69fb552acd56067661f8c2f534a718/markupsafe-3.0.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:1872df69a4de6aead3491198eaf13810b565bdbeec3ae2dc8780f14458ec73ce", size = 12020 }, + { url = "https://files.pythonhosted.org/packages/1e/2c/799f4742efc39633a1b54a92eec4082e4f815314869865d876824c257c1e/markupsafe-3.0.3-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3a7e8ae81ae39e62a41ec302f972ba6ae23a5c5396c8e60113e9066ef893da0d", size = 24332 }, + { url = "https://files.pythonhosted.org/packages/3c/2e/8d0c2ab90a8c1d9a24f0399058ab8519a3279d1bd4289511d74e909f060e/markupsafe-3.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d6dd0be5b5b189d31db7cda48b91d7e0a9795f31430b7f271219ab30f1d3ac9d", size = 22947 }, + { url = "https://files.pythonhosted.org/packages/2c/54/887f3092a85238093a0b2154bd629c89444f395618842e8b0c41783898ea/markupsafe-3.0.3-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:94c6f0bb423f739146aec64595853541634bde58b2135f27f61c1ffd1cd4d16a", size = 21962 }, + { url = "https://files.pythonhosted.org/packages/c9/2f/336b8c7b6f4a4d95e91119dc8521402461b74a485558d8f238a68312f11c/markupsafe-3.0.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:be8813b57049a7dc738189df53d69395eba14fb99345e0a5994914a3864c8a4b", size = 23760 }, + { url = "https://files.pythonhosted.org/packages/32/43/67935f2b7e4982ffb50a4d169b724d74b62a3964bc1a9a527f5ac4f1ee2b/markupsafe-3.0.3-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:83891d0e9fb81a825d9a6d61e3f07550ca70a076484292a70fde82c4b807286f", size = 21529 }, + { url = "https://files.pythonhosted.org/packages/89/e0/4486f11e51bbba8b0c041098859e869e304d1c261e59244baa3d295d47b7/markupsafe-3.0.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:77f0643abe7495da77fb436f50f8dab76dbc6e5fd25d39589a0f1fe6548bfa2b", size = 23015 }, + { url = "https://files.pythonhosted.org/packages/2f/e1/78ee7a023dac597a5825441ebd17170785a9dab23de95d2c7508ade94e0e/markupsafe-3.0.3-cp312-cp312-win32.whl", hash = "sha256:d88b440e37a16e651bda4c7c2b930eb586fd15ca7406cb39e211fcff3bf3017d", size = 14540 }, + { url = "https://files.pythonhosted.org/packages/aa/5b/bec5aa9bbbb2c946ca2733ef9c4ca91c91b6a24580193e891b5f7dbe8e1e/markupsafe-3.0.3-cp312-cp312-win_amd64.whl", hash = "sha256:26a5784ded40c9e318cfc2bdb30fe164bdb8665ded9cd64d500a34fb42067b1c", size = 15105 }, + { url = "https://files.pythonhosted.org/packages/e5/f1/216fc1bbfd74011693a4fd837e7026152e89c4bcf3e77b6692fba9923123/markupsafe-3.0.3-cp312-cp312-win_arm64.whl", hash = "sha256:35add3b638a5d900e807944a078b51922212fb3dedb01633a8defc4b01a3c85f", size = 13906 }, + { url = "https://files.pythonhosted.org/packages/38/2f/907b9c7bbba283e68f20259574b13d005c121a0fa4c175f9bed27c4597ff/markupsafe-3.0.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:e1cf1972137e83c5d4c136c43ced9ac51d0e124706ee1c8aa8532c1287fa8795", size = 11622 }, + { url = "https://files.pythonhosted.org/packages/9c/d9/5f7756922cdd676869eca1c4e3c0cd0df60ed30199ffd775e319089cb3ed/markupsafe-3.0.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:116bb52f642a37c115f517494ea5feb03889e04df47eeff5b130b1808ce7c219", size = 12029 }, + { url = "https://files.pythonhosted.org/packages/00/07/575a68c754943058c78f30db02ee03a64b3c638586fba6a6dd56830b30a3/markupsafe-3.0.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:133a43e73a802c5562be9bbcd03d090aa5a1fe899db609c29e8c8d815c5f6de6", size = 24374 }, + { url = "https://files.pythonhosted.org/packages/a9/21/9b05698b46f218fc0e118e1f8168395c65c8a2c750ae2bab54fc4bd4e0e8/markupsafe-3.0.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ccfcd093f13f0f0b7fdd0f198b90053bf7b2f02a3927a30e63f3ccc9df56b676", size = 22980 }, + { url = "https://files.pythonhosted.org/packages/7f/71/544260864f893f18b6827315b988c146b559391e6e7e8f7252839b1b846a/markupsafe-3.0.3-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:509fa21c6deb7a7a273d629cf5ec029bc209d1a51178615ddf718f5918992ab9", size = 21990 }, + { url = "https://files.pythonhosted.org/packages/c2/28/b50fc2f74d1ad761af2f5dcce7492648b983d00a65b8c0e0cb457c82ebbe/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:a4afe79fb3de0b7097d81da19090f4df4f8d3a2b3adaa8764138aac2e44f3af1", size = 23784 }, + { url = "https://files.pythonhosted.org/packages/ed/76/104b2aa106a208da8b17a2fb72e033a5a9d7073c68f7e508b94916ed47a9/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:795e7751525cae078558e679d646ae45574b47ed6e7771863fcc079a6171a0fc", size = 21588 }, + { url = "https://files.pythonhosted.org/packages/b5/99/16a5eb2d140087ebd97180d95249b00a03aa87e29cc224056274f2e45fd6/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:8485f406a96febb5140bfeca44a73e3ce5116b2501ac54fe953e488fb1d03b12", size = 23041 }, + { url = "https://files.pythonhosted.org/packages/19/bc/e7140ed90c5d61d77cea142eed9f9c303f4c4806f60a1044c13e3f1471d0/markupsafe-3.0.3-cp313-cp313-win32.whl", hash = "sha256:bdd37121970bfd8be76c5fb069c7751683bdf373db1ed6c010162b2a130248ed", size = 14543 }, + { url = "https://files.pythonhosted.org/packages/05/73/c4abe620b841b6b791f2edc248f556900667a5a1cf023a6646967ae98335/markupsafe-3.0.3-cp313-cp313-win_amd64.whl", hash = "sha256:9a1abfdc021a164803f4d485104931fb8f8c1efd55bc6b748d2f5774e78b62c5", size = 15113 }, + { url = "https://files.pythonhosted.org/packages/f0/3a/fa34a0f7cfef23cf9500d68cb7c32dd64ffd58a12b09225fb03dd37d5b80/markupsafe-3.0.3-cp313-cp313-win_arm64.whl", hash = "sha256:7e68f88e5b8799aa49c85cd116c932a1ac15caaa3f5db09087854d218359e485", size = 13911 }, + { url = "https://files.pythonhosted.org/packages/e4/d7/e05cd7efe43a88a17a37b3ae96e79a19e846f3f456fe79c57ca61356ef01/markupsafe-3.0.3-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:218551f6df4868a8d527e3062d0fb968682fe92054e89978594c28e642c43a73", size = 11658 }, + { url = "https://files.pythonhosted.org/packages/99/9e/e412117548182ce2148bdeacdda3bb494260c0b0184360fe0d56389b523b/markupsafe-3.0.3-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:3524b778fe5cfb3452a09d31e7b5adefeea8c5be1d43c4f810ba09f2ceb29d37", size = 12066 }, + { url = "https://files.pythonhosted.org/packages/bc/e6/fa0ffcda717ef64a5108eaa7b4f5ed28d56122c9a6d70ab8b72f9f715c80/markupsafe-3.0.3-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4e885a3d1efa2eadc93c894a21770e4bc67899e3543680313b09f139e149ab19", size = 25639 }, + { url = "https://files.pythonhosted.org/packages/96/ec/2102e881fe9d25fc16cb4b25d5f5cde50970967ffa5dddafdb771237062d/markupsafe-3.0.3-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8709b08f4a89aa7586de0aadc8da56180242ee0ada3999749b183aa23df95025", size = 23569 }, + { url = "https://files.pythonhosted.org/packages/4b/30/6f2fce1f1f205fc9323255b216ca8a235b15860c34b6798f810f05828e32/markupsafe-3.0.3-cp313-cp313t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:b8512a91625c9b3da6f127803b166b629725e68af71f8184ae7e7d54686a56d6", size = 23284 }, + { url = "https://files.pythonhosted.org/packages/58/47/4a0ccea4ab9f5dcb6f79c0236d954acb382202721e704223a8aafa38b5c8/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:9b79b7a16f7fedff2495d684f2b59b0457c3b493778c9eed31111be64d58279f", size = 24801 }, + { url = "https://files.pythonhosted.org/packages/6a/70/3780e9b72180b6fecb83a4814d84c3bf4b4ae4bf0b19c27196104149734c/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_riscv64.whl", hash = "sha256:12c63dfb4a98206f045aa9563db46507995f7ef6d83b2f68eda65c307c6829eb", size = 22769 }, + { url = "https://files.pythonhosted.org/packages/98/c5/c03c7f4125180fc215220c035beac6b9cb684bc7a067c84fc69414d315f5/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:8f71bc33915be5186016f675cd83a1e08523649b0e33efdb898db577ef5bb009", size = 23642 }, + { url = "https://files.pythonhosted.org/packages/80/d6/2d1b89f6ca4bff1036499b1e29a1d02d282259f3681540e16563f27ebc23/markupsafe-3.0.3-cp313-cp313t-win32.whl", hash = "sha256:69c0b73548bc525c8cb9a251cddf1931d1db4d2258e9599c28c07ef3580ef354", size = 14612 }, + { url = "https://files.pythonhosted.org/packages/2b/98/e48a4bfba0a0ffcf9925fe2d69240bfaa19c6f7507b8cd09c70684a53c1e/markupsafe-3.0.3-cp313-cp313t-win_amd64.whl", hash = "sha256:1b4b79e8ebf6b55351f0d91fe80f893b4743f104bff22e90697db1590e47a218", size = 15200 }, + { url = "https://files.pythonhosted.org/packages/0e/72/e3cc540f351f316e9ed0f092757459afbc595824ca724cbc5a5d4263713f/markupsafe-3.0.3-cp313-cp313t-win_arm64.whl", hash = "sha256:ad2cf8aa28b8c020ab2fc8287b0f823d0a7d8630784c31e9ee5edea20f406287", size = 13973 }, + { url = "https://files.pythonhosted.org/packages/33/8a/8e42d4838cd89b7dde187011e97fe6c3af66d8c044997d2183fbd6d31352/markupsafe-3.0.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:eaa9599de571d72e2daf60164784109f19978b327a3910d3e9de8c97b5b70cfe", size = 11619 }, + { url = "https://files.pythonhosted.org/packages/b5/64/7660f8a4a8e53c924d0fa05dc3a55c9cee10bbd82b11c5afb27d44b096ce/markupsafe-3.0.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:c47a551199eb8eb2121d4f0f15ae0f923d31350ab9280078d1e5f12b249e0026", size = 12029 }, + { url = "https://files.pythonhosted.org/packages/da/ef/e648bfd021127bef5fa12e1720ffed0c6cbb8310c8d9bea7266337ff06de/markupsafe-3.0.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f34c41761022dd093b4b6896d4810782ffbabe30f2d443ff5f083e0cbbb8c737", size = 24408 }, + { url = "https://files.pythonhosted.org/packages/41/3c/a36c2450754618e62008bf7435ccb0f88053e07592e6028a34776213d877/markupsafe-3.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:457a69a9577064c05a97c41f4e65148652db078a3a509039e64d3467b9e7ef97", size = 23005 }, + { url = "https://files.pythonhosted.org/packages/bc/20/b7fdf89a8456b099837cd1dc21974632a02a999ec9bf7ca3e490aacd98e7/markupsafe-3.0.3-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:e8afc3f2ccfa24215f8cb28dcf43f0113ac3c37c2f0f0806d8c70e4228c5cf4d", size = 22048 }, + { url = "https://files.pythonhosted.org/packages/9a/a7/591f592afdc734f47db08a75793a55d7fbcc6902a723ae4cfbab61010cc5/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:ec15a59cf5af7be74194f7ab02d0f59a62bdcf1a537677ce67a2537c9b87fcda", size = 23821 }, + { url = "https://files.pythonhosted.org/packages/7d/33/45b24e4f44195b26521bc6f1a82197118f74df348556594bd2262bda1038/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:0eb9ff8191e8498cca014656ae6b8d61f39da5f95b488805da4bb029cccbfbaf", size = 21606 }, + { url = "https://files.pythonhosted.org/packages/ff/0e/53dfaca23a69fbfbbf17a4b64072090e70717344c52eaaaa9c5ddff1e5f0/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:2713baf880df847f2bece4230d4d094280f4e67b1e813eec43b4c0e144a34ffe", size = 23043 }, + { url = "https://files.pythonhosted.org/packages/46/11/f333a06fc16236d5238bfe74daccbca41459dcd8d1fa952e8fbd5dccfb70/markupsafe-3.0.3-cp314-cp314-win32.whl", hash = "sha256:729586769a26dbceff69f7a7dbbf59ab6572b99d94576a5592625d5b411576b9", size = 14747 }, + { url = "https://files.pythonhosted.org/packages/28/52/182836104b33b444e400b14f797212f720cbc9ed6ba34c800639d154e821/markupsafe-3.0.3-cp314-cp314-win_amd64.whl", hash = "sha256:bdc919ead48f234740ad807933cdf545180bfbe9342c2bb451556db2ed958581", size = 15341 }, + { url = "https://files.pythonhosted.org/packages/6f/18/acf23e91bd94fd7b3031558b1f013adfa21a8e407a3fdb32745538730382/markupsafe-3.0.3-cp314-cp314-win_arm64.whl", hash = "sha256:5a7d5dc5140555cf21a6fefbdbf8723f06fcd2f63ef108f2854de715e4422cb4", size = 14073 }, + { url = "https://files.pythonhosted.org/packages/3c/f0/57689aa4076e1b43b15fdfa646b04653969d50cf30c32a102762be2485da/markupsafe-3.0.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:1353ef0c1b138e1907ae78e2f6c63ff67501122006b0f9abad68fda5f4ffc6ab", size = 11661 }, + { url = "https://files.pythonhosted.org/packages/89/c3/2e67a7ca217c6912985ec766c6393b636fb0c2344443ff9d91404dc4c79f/markupsafe-3.0.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:1085e7fbddd3be5f89cc898938f42c0b3c711fdcb37d75221de2666af647c175", size = 12069 }, + { url = "https://files.pythonhosted.org/packages/f0/00/be561dce4e6ca66b15276e184ce4b8aec61fe83662cce2f7d72bd3249d28/markupsafe-3.0.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1b52b4fb9df4eb9ae465f8d0c228a00624de2334f216f178a995ccdcf82c4634", size = 25670 }, + { url = "https://files.pythonhosted.org/packages/50/09/c419f6f5a92e5fadde27efd190eca90f05e1261b10dbd8cbcb39cd8ea1dc/markupsafe-3.0.3-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:fed51ac40f757d41b7c48425901843666a6677e3e8eb0abcff09e4ba6e664f50", size = 23598 }, + { url = "https://files.pythonhosted.org/packages/22/44/a0681611106e0b2921b3033fc19bc53323e0b50bc70cffdd19f7d679bb66/markupsafe-3.0.3-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:f190daf01f13c72eac4efd5c430a8de82489d9cff23c364c3ea822545032993e", size = 23261 }, + { url = "https://files.pythonhosted.org/packages/5f/57/1b0b3f100259dc9fffe780cfb60d4be71375510e435efec3d116b6436d43/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e56b7d45a839a697b5eb268c82a71bd8c7f6c94d6fd50c3d577fa39a9f1409f5", size = 24835 }, + { url = "https://files.pythonhosted.org/packages/26/6a/4bf6d0c97c4920f1597cc14dd720705eca0bf7c787aebc6bb4d1bead5388/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:f3e98bb3798ead92273dc0e5fd0f31ade220f59a266ffd8a4f6065e0a3ce0523", size = 22733 }, + { url = "https://files.pythonhosted.org/packages/14/c7/ca723101509b518797fedc2fdf79ba57f886b4aca8a7d31857ba3ee8281f/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:5678211cb9333a6468fb8d8be0305520aa073f50d17f089b5b4b477ea6e67fdc", size = 23672 }, + { url = "https://files.pythonhosted.org/packages/fb/df/5bd7a48c256faecd1d36edc13133e51397e41b73bb77e1a69deab746ebac/markupsafe-3.0.3-cp314-cp314t-win32.whl", hash = "sha256:915c04ba3851909ce68ccc2b8e2cd691618c4dc4c4232fb7982bca3f41fd8c3d", size = 14819 }, + { url = "https://files.pythonhosted.org/packages/1a/8a/0402ba61a2f16038b48b39bccca271134be00c5c9f0f623208399333c448/markupsafe-3.0.3-cp314-cp314t-win_amd64.whl", hash = "sha256:4faffd047e07c38848ce017e8725090413cd80cbc23d86e55c587bf979e579c9", size = 15426 }, + { url = "https://files.pythonhosted.org/packages/70/bc/6f1c2f612465f5fa89b95bead1f44dcb607670fd42891d8fdcd5d039f4f4/markupsafe-3.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:32001d6a8fc98c8cb5c947787c5d08b0a50663d139f1305bac5885d98d9b40fa", size = 14146 }, +] + +[[package]] +name = "matplotlib" +version = "3.10.8" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "contourpy" }, + { name = "cycler" }, + { name = "fonttools" }, + { name = "kiwisolver" }, + { name = "numpy" }, + { name = "packaging" }, + { name = "pillow" }, + { name = "pyparsing" }, + { name = "python-dateutil" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/8a/76/d3c6e3a13fe484ebe7718d14e269c9569c4eb0020a968a327acb3b9a8fe6/matplotlib-3.10.8.tar.gz", hash = "sha256:2299372c19d56bcd35cf05a2738308758d32b9eaed2371898d8f5bd33f084aa3", size = 34806269 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f8/86/de7e3a1cdcfc941483af70609edc06b83e7c8a0e0dc9ac325200a3f4d220/matplotlib-3.10.8-cp311-cp311-macosx_10_12_x86_64.whl", hash = "sha256:6be43b667360fef5c754dda5d25a32e6307a03c204f3c0fc5468b78fa87b4160", size = 8251215 }, + { url = "https://files.pythonhosted.org/packages/fd/14/baad3222f424b19ce6ad243c71de1ad9ec6b2e4eb1e458a48fdc6d120401/matplotlib-3.10.8-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:a2b336e2d91a3d7006864e0990c83b216fcdca64b5a6484912902cef87313d78", size = 8139625 }, + { url = "https://files.pythonhosted.org/packages/8f/a0/7024215e95d456de5883e6732e708d8187d9753a21d32f8ddb3befc0c445/matplotlib-3.10.8-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:efb30e3baaea72ce5928e32bab719ab4770099079d66726a62b11b1ef7273be4", size = 8712614 }, + { url = "https://files.pythonhosted.org/packages/5a/f4/b8347351da9a5b3f41e26cf547252d861f685c6867d179a7c9d60ad50189/matplotlib-3.10.8-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d56a1efd5bfd61486c8bc968fa18734464556f0fb8e51690f4ac25d85cbbbbc2", size = 9540997 }, + { url = "https://files.pythonhosted.org/packages/9e/c0/c7b914e297efe0bc36917bf216b2acb91044b91e930e878ae12981e461e5/matplotlib-3.10.8-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:238b7ce5717600615c895050239ec955d91f321c209dd110db988500558e70d6", size = 9596825 }, + { url = "https://files.pythonhosted.org/packages/6f/d3/a4bbc01c237ab710a1f22b4da72f4ff6d77eb4c7735ea9811a94ae239067/matplotlib-3.10.8-cp311-cp311-win_amd64.whl", hash = "sha256:18821ace09c763ec93aef5eeff087ee493a24051936d7b9ebcad9662f66501f9", size = 8135090 }, + { url = "https://files.pythonhosted.org/packages/89/dd/a0b6588f102beab33ca6f5218b31725216577b2a24172f327eaf6417d5c9/matplotlib-3.10.8-cp311-cp311-win_arm64.whl", hash = "sha256:bab485bcf8b1c7d2060b4fcb6fc368a9e6f4cd754c9c2fea281f4be21df394a2", size = 8012377 }, + { url = "https://files.pythonhosted.org/packages/9e/67/f997cdcbb514012eb0d10cd2b4b332667997fb5ebe26b8d41d04962fa0e6/matplotlib-3.10.8-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:64fcc24778ca0404ce0cb7b6b77ae1f4c7231cdd60e6778f999ee05cbd581b9a", size = 8260453 }, + { url = "https://files.pythonhosted.org/packages/7e/65/07d5f5c7f7c994f12c768708bd2e17a4f01a2b0f44a1c9eccad872433e2e/matplotlib-3.10.8-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:b9a5ca4ac220a0cdd1ba6bcba3608547117d30468fefce49bb26f55c1a3d5c58", size = 8148321 }, + { url = "https://files.pythonhosted.org/packages/3e/f3/c5195b1ae57ef85339fd7285dfb603b22c8b4e79114bae5f4f0fcf688677/matplotlib-3.10.8-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:3ab4aabc72de4ff77b3ec33a6d78a68227bf1123465887f9905ba79184a1cc04", size = 8716944 }, + { url = "https://files.pythonhosted.org/packages/00/f9/7638f5cc82ec8a7aa005de48622eecc3ed7c9854b96ba15bd76b7fd27574/matplotlib-3.10.8-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:24d50994d8c5816ddc35411e50a86ab05f575e2530c02752e02538122613371f", size = 9550099 }, + { url = "https://files.pythonhosted.org/packages/57/61/78cd5920d35b29fd2a0fe894de8adf672ff52939d2e9b43cb83cd5ce1bc7/matplotlib-3.10.8-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:99eefd13c0dc3b3c1b4d561c1169e65fe47aab7b8158754d7c084088e2329466", size = 9613040 }, + { url = "https://files.pythonhosted.org/packages/30/4e/c10f171b6e2f44d9e3a2b96efa38b1677439d79c99357600a62cc1e9594e/matplotlib-3.10.8-cp312-cp312-win_amd64.whl", hash = "sha256:dd80ecb295460a5d9d260df63c43f4afbdd832d725a531f008dad1664f458adf", size = 8142717 }, + { url = "https://files.pythonhosted.org/packages/f1/76/934db220026b5fef85f45d51a738b91dea7d70207581063cd9bd8fafcf74/matplotlib-3.10.8-cp312-cp312-win_arm64.whl", hash = "sha256:3c624e43ed56313651bc18a47f838b60d7b8032ed348911c54906b130b20071b", size = 8012751 }, + { url = "https://files.pythonhosted.org/packages/3d/b9/15fd5541ef4f5b9a17eefd379356cf12175fe577424e7b1d80676516031a/matplotlib-3.10.8-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:3f2e409836d7f5ac2f1c013110a4d50b9f7edc26328c108915f9075d7d7a91b6", size = 8261076 }, + { url = "https://files.pythonhosted.org/packages/8d/a0/2ba3473c1b66b9c74dc7107c67e9008cb1782edbe896d4c899d39ae9cf78/matplotlib-3.10.8-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:56271f3dac49a88d7fca5060f004d9d22b865f743a12a23b1e937a0be4818ee1", size = 8148794 }, + { url = "https://files.pythonhosted.org/packages/75/97/a471f1c3eb1fd6f6c24a31a5858f443891d5127e63a7788678d14e249aea/matplotlib-3.10.8-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:a0a7f52498f72f13d4a25ea70f35f4cb60642b466cbb0a9be951b5bc3f45a486", size = 8718474 }, + { url = "https://files.pythonhosted.org/packages/01/be/cd478f4b66f48256f42927d0acbcd63a26a893136456cd079c0cc24fbabf/matplotlib-3.10.8-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:646d95230efb9ca614a7a594d4fcacde0ac61d25e37dd51710b36477594963ce", size = 9549637 }, + { url = "https://files.pythonhosted.org/packages/5d/7c/8dc289776eae5109e268c4fb92baf870678dc048a25d4ac903683b86d5bf/matplotlib-3.10.8-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:f89c151aab2e2e23cb3fe0acad1e8b82841fd265379c4cecd0f3fcb34c15e0f6", size = 9613678 }, + { url = "https://files.pythonhosted.org/packages/64/40/37612487cc8a437d4dd261b32ca21fe2d79510fe74af74e1f42becb1bdb8/matplotlib-3.10.8-cp313-cp313-win_amd64.whl", hash = "sha256:e8ea3e2d4066083e264e75c829078f9e149fa119d27e19acd503de65e0b13149", size = 8142686 }, + { url = "https://files.pythonhosted.org/packages/66/52/8d8a8730e968185514680c2a6625943f70269509c3dcfc0dcf7d75928cb8/matplotlib-3.10.8-cp313-cp313-win_arm64.whl", hash = "sha256:c108a1d6fa78a50646029cb6d49808ff0fc1330fda87fa6f6250c6b5369b6645", size = 8012917 }, + { url = "https://files.pythonhosted.org/packages/b5/27/51fe26e1062f298af5ef66343d8ef460e090a27fea73036c76c35821df04/matplotlib-3.10.8-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:ad3d9833a64cf48cc4300f2b406c3d0f4f4724a91c0bd5640678a6ba7c102077", size = 8305679 }, + { url = "https://files.pythonhosted.org/packages/2c/1e/4de865bc591ac8e3062e835f42dd7fe7a93168d519557837f0e37513f629/matplotlib-3.10.8-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:eb3823f11823deade26ce3b9f40dcb4a213da7a670013929f31d5f5ed1055b22", size = 8198336 }, + { url = "https://files.pythonhosted.org/packages/c6/cb/2f7b6e75fb4dce87ef91f60cac4f6e34f4c145ab036a22318ec837971300/matplotlib-3.10.8-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:d9050fee89a89ed57b4fb2c1bfac9a3d0c57a0d55aed95949eedbc42070fea39", size = 8731653 }, + { url = "https://files.pythonhosted.org/packages/46/b3/bd9c57d6ba670a37ab31fb87ec3e8691b947134b201f881665b28cc039ff/matplotlib-3.10.8-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b44d07310e404ba95f8c25aa5536f154c0a8ec473303535949e52eb71d0a1565", size = 9561356 }, + { url = "https://files.pythonhosted.org/packages/c0/3d/8b94a481456dfc9dfe6e39e93b5ab376e50998cddfd23f4ae3b431708f16/matplotlib-3.10.8-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:0a33deb84c15ede243aead39f77e990469fff93ad1521163305095b77b72ce4a", size = 9614000 }, + { url = "https://files.pythonhosted.org/packages/bd/cd/bc06149fe5585ba800b189a6a654a75f1f127e8aab02fd2be10df7fa500c/matplotlib-3.10.8-cp313-cp313t-win_amd64.whl", hash = "sha256:3a48a78d2786784cc2413e57397981fb45c79e968d99656706018d6e62e57958", size = 8220043 }, + { url = "https://files.pythonhosted.org/packages/e3/de/b22cf255abec916562cc04eef457c13e58a1990048de0c0c3604d082355e/matplotlib-3.10.8-cp313-cp313t-win_arm64.whl", hash = "sha256:15d30132718972c2c074cd14638c7f4592bd98719e2308bccea40e0538bc0cb5", size = 8062075 }, + { url = "https://files.pythonhosted.org/packages/3c/43/9c0ff7a2f11615e516c3b058e1e6e8f9614ddeca53faca06da267c48345d/matplotlib-3.10.8-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:b53285e65d4fa4c86399979e956235deb900be5baa7fc1218ea67fbfaeaadd6f", size = 8262481 }, + { url = "https://files.pythonhosted.org/packages/6f/ca/e8ae28649fcdf039fda5ef554b40a95f50592a3c47e6f7270c9561c12b07/matplotlib-3.10.8-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:32f8dce744be5569bebe789e46727946041199030db8aeb2954d26013a0eb26b", size = 8151473 }, + { url = "https://files.pythonhosted.org/packages/f1/6f/009d129ae70b75e88cbe7e503a12a4c0670e08ed748a902c2568909e9eb5/matplotlib-3.10.8-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4cf267add95b1c88300d96ca837833d4112756045364f5c734a2276038dae27d", size = 9553896 }, + { url = "https://files.pythonhosted.org/packages/f5/26/4221a741eb97967bc1fd5e4c52b9aa5a91b2f4ec05b59f6def4d820f9df9/matplotlib-3.10.8-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:2cf5bd12cecf46908f286d7838b2abc6c91cda506c0445b8223a7c19a00df008", size = 9824193 }, + { url = "https://files.pythonhosted.org/packages/1f/f3/3abf75f38605772cf48a9daf5821cd4f563472f38b4b828c6fba6fa6d06e/matplotlib-3.10.8-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:41703cc95688f2516b480f7f339d8851a6035f18e100ee6a32bc0b8536a12a9c", size = 9615444 }, + { url = "https://files.pythonhosted.org/packages/93/a5/de89ac80f10b8dc615807ee1133cd99ac74082581196d4d9590bea10690d/matplotlib-3.10.8-cp314-cp314-win_amd64.whl", hash = "sha256:83d282364ea9f3e52363da262ce32a09dfe241e4080dcedda3c0db059d3c1f11", size = 8272719 }, + { url = "https://files.pythonhosted.org/packages/69/ce/b006495c19ccc0a137b48083168a37bd056392dee02f87dba0472f2797fe/matplotlib-3.10.8-cp314-cp314-win_arm64.whl", hash = "sha256:2c1998e92cd5999e295a731bcb2911c75f597d937341f3030cc24ef2733d78a8", size = 8144205 }, + { url = "https://files.pythonhosted.org/packages/68/d9/b31116a3a855bd313c6fcdb7226926d59b041f26061c6c5b1be66a08c826/matplotlib-3.10.8-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:b5a2b97dbdc7d4f353ebf343744f1d1f1cca8aa8bfddb4262fcf4306c3761d50", size = 8305785 }, + { url = "https://files.pythonhosted.org/packages/1e/90/6effe8103f0272685767ba5f094f453784057072f49b393e3ea178fe70a5/matplotlib-3.10.8-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:3f5c3e4da343bba819f0234186b9004faba952cc420fbc522dc4e103c1985908", size = 8198361 }, + { url = "https://files.pythonhosted.org/packages/d7/65/a73188711bea603615fc0baecca1061429ac16940e2385433cc778a9d8e7/matplotlib-3.10.8-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5f62550b9a30afde8c1c3ae450e5eb547d579dd69b25c2fc7a1c67f934c1717a", size = 9561357 }, + { url = "https://files.pythonhosted.org/packages/f4/3d/b5c5d5d5be8ce63292567f0e2c43dde9953d3ed86ac2de0a72e93c8f07a1/matplotlib-3.10.8-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:495672de149445ec1b772ff2c9ede9b769e3cb4f0d0aa7fa730d7f59e2d4e1c1", size = 9823610 }, + { url = "https://files.pythonhosted.org/packages/4d/4b/e7beb6bbd49f6bae727a12b270a2654d13c397576d25bd6786e47033300f/matplotlib-3.10.8-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:595ba4d8fe983b88f0eec8c26a241e16d6376fe1979086232f481f8f3f67494c", size = 9614011 }, + { url = "https://files.pythonhosted.org/packages/7c/e6/76f2813d31f032e65f6f797e3f2f6e4aab95b65015924b1c51370395c28a/matplotlib-3.10.8-cp314-cp314t-win_amd64.whl", hash = "sha256:25d380fe8b1dc32cf8f0b1b448470a77afb195438bafdf1d858bfb876f3edf7b", size = 8362801 }, + { url = "https://files.pythonhosted.org/packages/5d/49/d651878698a0b67f23aa28e17f45a6d6dd3d3f933fa29087fa4ce5947b5a/matplotlib-3.10.8-cp314-cp314t-win_arm64.whl", hash = "sha256:113bb52413ea508ce954a02c10ffd0d565f9c3bc7f2eddc27dfe1731e71c7b5f", size = 8192560 }, + { url = "https://files.pythonhosted.org/packages/04/30/3afaa31c757f34b7725ab9d2ba8b48b5e89c2019c003e7d0ead143aabc5a/matplotlib-3.10.8-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:6da7c2ce169267d0d066adcf63758f0604aa6c3eebf67458930f9d9b79ad1db1", size = 8249198 }, + { url = "https://files.pythonhosted.org/packages/48/2f/6334aec331f57485a642a7c8be03cb286f29111ae71c46c38b363230063c/matplotlib-3.10.8-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:9153c3292705be9f9c64498a8872118540c3f4123d1a1c840172edf262c8be4a", size = 8136817 }, + { url = "https://files.pythonhosted.org/packages/73/e4/6d6f14b2a759c622f191b2d67e9075a3f56aaccb3be4bb9bb6890030d0a0/matplotlib-3.10.8-pp311-pypy311_pp73-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:1ae029229a57cd1e8fe542485f27e7ca7b23aa9e8944ddb4985d0bc444f1eca2", size = 8713867 }, +] + +[[package]] +name = "mpmath" +version = "1.3.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/e0/47/dd32fa426cc72114383ac549964eecb20ecfd886d1e5ccf5340b55b02f57/mpmath-1.3.0.tar.gz", hash = "sha256:7a28eb2a9774d00c7bc92411c19a89209d5da7c4c9a9e227be8330a23a25b91f", size = 508106 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/43/e3/7d92a15f894aa0c9c4b49b8ee9ac9850d6e63b03c9c32c0367a13ae62209/mpmath-1.3.0-py3-none-any.whl", hash = "sha256:a0b2b9fe80bbcd81a6647ff13108738cfb482d481d826cc0e02f5b35e5c88d2c", size = 536198 }, +] + +[[package]] +name = "numpy" +version = "2.2.6" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/76/21/7d2a95e4bba9dc13d043ee156a356c0a8f0c6309dff6b21b4d71a073b8a8/numpy-2.2.6.tar.gz", hash = "sha256:e29554e2bef54a90aa5cc07da6ce955accb83f21ab5de01a62c8478897b264fd", size = 20276440 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/da/a8/4f83e2aa666a9fbf56d6118faaaf5f1974d456b1823fda0a176eff722839/numpy-2.2.6-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:f9f1adb22318e121c5c69a09142811a201ef17ab257a1e66ca3025065b7f53ae", size = 21176963 }, + { url = "https://files.pythonhosted.org/packages/b3/2b/64e1affc7972decb74c9e29e5649fac940514910960ba25cd9af4488b66c/numpy-2.2.6-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:c820a93b0255bc360f53eca31a0e676fd1101f673dda8da93454a12e23fc5f7a", size = 14406743 }, + { url = "https://files.pythonhosted.org/packages/4a/9f/0121e375000b5e50ffdd8b25bf78d8e1a5aa4cca3f185d41265198c7b834/numpy-2.2.6-cp311-cp311-macosx_14_0_arm64.whl", hash = "sha256:3d70692235e759f260c3d837193090014aebdf026dfd167834bcba43e30c2a42", size = 5352616 }, + { url = "https://files.pythonhosted.org/packages/31/0d/b48c405c91693635fbe2dcd7bc84a33a602add5f63286e024d3b6741411c/numpy-2.2.6-cp311-cp311-macosx_14_0_x86_64.whl", hash = "sha256:481b49095335f8eed42e39e8041327c05b0f6f4780488f61286ed3c01368d491", size = 6889579 }, + { url = "https://files.pythonhosted.org/packages/52/b8/7f0554d49b565d0171eab6e99001846882000883998e7b7d9f0d98b1f934/numpy-2.2.6-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:b64d8d4d17135e00c8e346e0a738deb17e754230d7e0810ac5012750bbd85a5a", size = 14312005 }, + { url = "https://files.pythonhosted.org/packages/b3/dd/2238b898e51bd6d389b7389ffb20d7f4c10066d80351187ec8e303a5a475/numpy-2.2.6-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ba10f8411898fc418a521833e014a77d3ca01c15b0c6cdcce6a0d2897e6dbbdf", size = 16821570 }, + { url = "https://files.pythonhosted.org/packages/83/6c/44d0325722cf644f191042bf47eedad61c1e6df2432ed65cbe28509d404e/numpy-2.2.6-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:bd48227a919f1bafbdda0583705e547892342c26fb127219d60a5c36882609d1", size = 15818548 }, + { url = "https://files.pythonhosted.org/packages/ae/9d/81e8216030ce66be25279098789b665d49ff19eef08bfa8cb96d4957f422/numpy-2.2.6-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:9551a499bf125c1d4f9e250377c1ee2eddd02e01eac6644c080162c0c51778ab", size = 18620521 }, + { url = "https://files.pythonhosted.org/packages/6a/fd/e19617b9530b031db51b0926eed5345ce8ddc669bb3bc0044b23e275ebe8/numpy-2.2.6-cp311-cp311-win32.whl", hash = "sha256:0678000bb9ac1475cd454c6b8c799206af8107e310843532b04d49649c717a47", size = 6525866 }, + { url = "https://files.pythonhosted.org/packages/31/0a/f354fb7176b81747d870f7991dc763e157a934c717b67b58456bc63da3df/numpy-2.2.6-cp311-cp311-win_amd64.whl", hash = "sha256:e8213002e427c69c45a52bbd94163084025f533a55a59d6f9c5b820774ef3303", size = 12907455 }, + { url = "https://files.pythonhosted.org/packages/82/5d/c00588b6cf18e1da539b45d3598d3557084990dcc4331960c15ee776ee41/numpy-2.2.6-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:41c5a21f4a04fa86436124d388f6ed60a9343a6f767fced1a8a71c3fbca038ff", size = 20875348 }, + { url = "https://files.pythonhosted.org/packages/66/ee/560deadcdde6c2f90200450d5938f63a34b37e27ebff162810f716f6a230/numpy-2.2.6-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:de749064336d37e340f640b05f24e9e3dd678c57318c7289d222a8a2f543e90c", size = 14119362 }, + { url = "https://files.pythonhosted.org/packages/3c/65/4baa99f1c53b30adf0acd9a5519078871ddde8d2339dc5a7fde80d9d87da/numpy-2.2.6-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:894b3a42502226a1cac872f840030665f33326fc3dac8e57c607905773cdcde3", size = 5084103 }, + { url = "https://files.pythonhosted.org/packages/cc/89/e5a34c071a0570cc40c9a54eb472d113eea6d002e9ae12bb3a8407fb912e/numpy-2.2.6-cp312-cp312-macosx_14_0_x86_64.whl", hash = "sha256:71594f7c51a18e728451bb50cc60a3ce4e6538822731b2933209a1f3614e9282", size = 6625382 }, + { url = "https://files.pythonhosted.org/packages/f8/35/8c80729f1ff76b3921d5c9487c7ac3de9b2a103b1cd05e905b3090513510/numpy-2.2.6-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f2618db89be1b4e05f7a1a847a9c1c0abd63e63a1607d892dd54668dd92faf87", size = 14018462 }, + { url = "https://files.pythonhosted.org/packages/8c/3d/1e1db36cfd41f895d266b103df00ca5b3cbe965184df824dec5c08c6b803/numpy-2.2.6-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:fd83c01228a688733f1ded5201c678f0c53ecc1006ffbc404db9f7a899ac6249", size = 16527618 }, + { url = "https://files.pythonhosted.org/packages/61/c6/03ed30992602c85aa3cd95b9070a514f8b3c33e31124694438d88809ae36/numpy-2.2.6-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:37c0ca431f82cd5fa716eca9506aefcabc247fb27ba69c5062a6d3ade8cf8f49", size = 15505511 }, + { url = "https://files.pythonhosted.org/packages/b7/25/5761d832a81df431e260719ec45de696414266613c9ee268394dd5ad8236/numpy-2.2.6-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:fe27749d33bb772c80dcd84ae7e8df2adc920ae8297400dabec45f0dedb3f6de", size = 18313783 }, + { url = "https://files.pythonhosted.org/packages/57/0a/72d5a3527c5ebffcd47bde9162c39fae1f90138c961e5296491ce778e682/numpy-2.2.6-cp312-cp312-win32.whl", hash = "sha256:4eeaae00d789f66c7a25ac5f34b71a7035bb474e679f410e5e1a94deb24cf2d4", size = 6246506 }, + { url = "https://files.pythonhosted.org/packages/36/fa/8c9210162ca1b88529ab76b41ba02d433fd54fecaf6feb70ef9f124683f1/numpy-2.2.6-cp312-cp312-win_amd64.whl", hash = "sha256:c1f9540be57940698ed329904db803cf7a402f3fc200bfe599334c9bd84a40b2", size = 12614190 }, + { url = "https://files.pythonhosted.org/packages/f9/5c/6657823f4f594f72b5471f1db1ab12e26e890bb2e41897522d134d2a3e81/numpy-2.2.6-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:0811bb762109d9708cca4d0b13c4f67146e3c3b7cf8d34018c722adb2d957c84", size = 20867828 }, + { url = "https://files.pythonhosted.org/packages/dc/9e/14520dc3dadf3c803473bd07e9b2bd1b69bc583cb2497b47000fed2fa92f/numpy-2.2.6-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:287cc3162b6f01463ccd86be154f284d0893d2b3ed7292439ea97eafa8170e0b", size = 14143006 }, + { url = "https://files.pythonhosted.org/packages/4f/06/7e96c57d90bebdce9918412087fc22ca9851cceaf5567a45c1f404480e9e/numpy-2.2.6-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:f1372f041402e37e5e633e586f62aa53de2eac8d98cbfb822806ce4bbefcb74d", size = 5076765 }, + { url = "https://files.pythonhosted.org/packages/73/ed/63d920c23b4289fdac96ddbdd6132e9427790977d5457cd132f18e76eae0/numpy-2.2.6-cp313-cp313-macosx_14_0_x86_64.whl", hash = "sha256:55a4d33fa519660d69614a9fad433be87e5252f4b03850642f88993f7b2ca566", size = 6617736 }, + { url = "https://files.pythonhosted.org/packages/85/c5/e19c8f99d83fd377ec8c7e0cf627a8049746da54afc24ef0a0cb73d5dfb5/numpy-2.2.6-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f92729c95468a2f4f15e9bb94c432a9229d0d50de67304399627a943201baa2f", size = 14010719 }, + { url = "https://files.pythonhosted.org/packages/19/49/4df9123aafa7b539317bf6d342cb6d227e49f7a35b99c287a6109b13dd93/numpy-2.2.6-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1bc23a79bfabc5d056d106f9befb8d50c31ced2fbc70eedb8155aec74a45798f", size = 16526072 }, + { url = "https://files.pythonhosted.org/packages/b2/6c/04b5f47f4f32f7c2b0e7260442a8cbcf8168b0e1a41ff1495da42f42a14f/numpy-2.2.6-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:e3143e4451880bed956e706a3220b4e5cf6172ef05fcc397f6f36a550b1dd868", size = 15503213 }, + { url = "https://files.pythonhosted.org/packages/17/0a/5cd92e352c1307640d5b6fec1b2ffb06cd0dabe7d7b8227f97933d378422/numpy-2.2.6-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:b4f13750ce79751586ae2eb824ba7e1e8dba64784086c98cdbbcc6a42112ce0d", size = 18316632 }, + { url = "https://files.pythonhosted.org/packages/f0/3b/5cba2b1d88760ef86596ad0f3d484b1cbff7c115ae2429678465057c5155/numpy-2.2.6-cp313-cp313-win32.whl", hash = "sha256:5beb72339d9d4fa36522fc63802f469b13cdbe4fdab4a288f0c441b74272ebfd", size = 6244532 }, + { url = "https://files.pythonhosted.org/packages/cb/3b/d58c12eafcb298d4e6d0d40216866ab15f59e55d148a5658bb3132311fcf/numpy-2.2.6-cp313-cp313-win_amd64.whl", hash = "sha256:b0544343a702fa80c95ad5d3d608ea3599dd54d4632df855e4c8d24eb6ecfa1c", size = 12610885 }, + { url = "https://files.pythonhosted.org/packages/6b/9e/4bf918b818e516322db999ac25d00c75788ddfd2d2ade4fa66f1f38097e1/numpy-2.2.6-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:0bca768cd85ae743b2affdc762d617eddf3bcf8724435498a1e80132d04879e6", size = 20963467 }, + { url = "https://files.pythonhosted.org/packages/61/66/d2de6b291507517ff2e438e13ff7b1e2cdbdb7cb40b3ed475377aece69f9/numpy-2.2.6-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:fc0c5673685c508a142ca65209b4e79ed6740a4ed6b2267dbba90f34b0b3cfda", size = 14225144 }, + { url = "https://files.pythonhosted.org/packages/e4/25/480387655407ead912e28ba3a820bc69af9adf13bcbe40b299d454ec011f/numpy-2.2.6-cp313-cp313t-macosx_14_0_arm64.whl", hash = "sha256:5bd4fc3ac8926b3819797a7c0e2631eb889b4118a9898c84f585a54d475b7e40", size = 5200217 }, + { url = "https://files.pythonhosted.org/packages/aa/4a/6e313b5108f53dcbf3aca0c0f3e9c92f4c10ce57a0a721851f9785872895/numpy-2.2.6-cp313-cp313t-macosx_14_0_x86_64.whl", hash = "sha256:fee4236c876c4e8369388054d02d0e9bb84821feb1a64dd59e137e6511a551f8", size = 6712014 }, + { url = "https://files.pythonhosted.org/packages/b7/30/172c2d5c4be71fdf476e9de553443cf8e25feddbe185e0bd88b096915bcc/numpy-2.2.6-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e1dda9c7e08dc141e0247a5b8f49cf05984955246a327d4c48bda16821947b2f", size = 14077935 }, + { url = "https://files.pythonhosted.org/packages/12/fb/9e743f8d4e4d3c710902cf87af3512082ae3d43b945d5d16563f26ec251d/numpy-2.2.6-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f447e6acb680fd307f40d3da4852208af94afdfab89cf850986c3ca00562f4fa", size = 16600122 }, + { url = "https://files.pythonhosted.org/packages/12/75/ee20da0e58d3a66f204f38916757e01e33a9737d0b22373b3eb5a27358f9/numpy-2.2.6-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:389d771b1623ec92636b0786bc4ae56abafad4a4c513d36a55dce14bd9ce8571", size = 15586143 }, + { url = "https://files.pythonhosted.org/packages/76/95/bef5b37f29fc5e739947e9ce5179ad402875633308504a52d188302319c8/numpy-2.2.6-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:8e9ace4a37db23421249ed236fdcdd457d671e25146786dfc96835cd951aa7c1", size = 18385260 }, + { url = "https://files.pythonhosted.org/packages/09/04/f2f83279d287407cf36a7a8053a5abe7be3622a4363337338f2585e4afda/numpy-2.2.6-cp313-cp313t-win32.whl", hash = "sha256:038613e9fb8c72b0a41f025a7e4c3f0b7a1b5d768ece4796b674c8f3fe13efff", size = 6377225 }, + { url = "https://files.pythonhosted.org/packages/67/0e/35082d13c09c02c011cf21570543d202ad929d961c02a147493cb0c2bdf5/numpy-2.2.6-cp313-cp313t-win_amd64.whl", hash = "sha256:6031dd6dfecc0cf9f668681a37648373bddd6421fff6c66ec1624eed0180ee06", size = 12771374 }, +] + +[[package]] +name = "onnxruntime" +version = "1.23.2" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "coloredlogs" }, + { name = "flatbuffers" }, + { name = "numpy" }, + { name = "packaging" }, + { name = "protobuf" }, + { name = "sympy" }, +] +wheels = [ + { url = "https://files.pythonhosted.org/packages/44/be/467b00f09061572f022ffd17e49e49e5a7a789056bad95b54dfd3bee73ff/onnxruntime-1.23.2-cp311-cp311-macosx_13_0_arm64.whl", hash = "sha256:6f91d2c9b0965e86827a5ba01531d5b669770b01775b23199565d6c1f136616c", size = 17196113 }, + { url = "https://files.pythonhosted.org/packages/9f/a8/3c23a8f75f93122d2b3410bfb74d06d0f8da4ac663185f91866b03f7da1b/onnxruntime-1.23.2-cp311-cp311-macosx_13_0_x86_64.whl", hash = "sha256:87d8b6eaf0fbeb6835a60a4265fde7a3b60157cf1b2764773ac47237b4d48612", size = 19153857 }, + { url = "https://files.pythonhosted.org/packages/3f/d8/506eed9af03d86f8db4880a4c47cd0dffee973ef7e4f4cff9f1d4bcf7d22/onnxruntime-1.23.2-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:bbfd2fca76c855317568c1b36a885ddea2272c13cb0e395002c402f2360429a6", size = 15220095 }, + { url = "https://files.pythonhosted.org/packages/e9/80/113381ba832d5e777accedc6cb41d10f9eca82321ae31ebb6bcede530cea/onnxruntime-1.23.2-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:da44b99206e77734c5819aa2142c69e64f3b46edc3bd314f6a45a932defc0b3e", size = 17372080 }, + { url = "https://files.pythonhosted.org/packages/3a/db/1b4a62e23183a0c3fe441782462c0ede9a2a65c6bbffb9582fab7c7a0d38/onnxruntime-1.23.2-cp311-cp311-win_amd64.whl", hash = "sha256:902c756d8b633ce0dedd889b7c08459433fbcf35e9c38d1c03ddc020f0648c6e", size = 13468349 }, + { url = "https://files.pythonhosted.org/packages/1b/9e/f748cd64161213adeef83d0cb16cb8ace1e62fa501033acdd9f9341fff57/onnxruntime-1.23.2-cp312-cp312-macosx_13_0_arm64.whl", hash = "sha256:b8f029a6b98d3cf5be564d52802bb50a8489ab73409fa9db0bf583eabb7c2321", size = 17195929 }, + { url = "https://files.pythonhosted.org/packages/91/9d/a81aafd899b900101988ead7fb14974c8a58695338ab6a0f3d6b0100f30b/onnxruntime-1.23.2-cp312-cp312-macosx_13_0_x86_64.whl", hash = "sha256:218295a8acae83905f6f1aed8cacb8e3eb3bd7513a13fe4ba3b2664a19fc4a6b", size = 19157705 }, + { url = "https://files.pythonhosted.org/packages/3c/35/4e40f2fba272a6698d62be2cd21ddc3675edfc1a4b9ddefcc4648f115315/onnxruntime-1.23.2-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:76ff670550dc23e58ea9bc53b5149b99a44e63b34b524f7b8547469aaa0dcb8c", size = 15226915 }, + { url = "https://files.pythonhosted.org/packages/ef/88/9cc25d2bafe6bc0d4d3c1db3ade98196d5b355c0b273e6a5dc09c5d5d0d5/onnxruntime-1.23.2-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0f9b4ae77f8e3c9bee50c27bc1beede83f786fe1d52e99ac85aa8d65a01e9b77", size = 17382649 }, + { url = "https://files.pythonhosted.org/packages/c0/b4/569d298f9fc4d286c11c45e85d9ffa9e877af12ace98af8cab52396e8f46/onnxruntime-1.23.2-cp312-cp312-win_amd64.whl", hash = "sha256:25de5214923ce941a3523739d34a520aac30f21e631de53bba9174dc9c004435", size = 13470528 }, + { url = "https://files.pythonhosted.org/packages/3d/41/fba0cabccecefe4a1b5fc8020c44febb334637f133acefc7ec492029dd2c/onnxruntime-1.23.2-cp313-cp313-macosx_13_0_arm64.whl", hash = "sha256:2ff531ad8496281b4297f32b83b01cdd719617e2351ffe0dba5684fb283afa1f", size = 17196337 }, + { url = "https://files.pythonhosted.org/packages/fe/f9/2d49ca491c6a986acce9f1d1d5fc2099108958cc1710c28e89a032c9cfe9/onnxruntime-1.23.2-cp313-cp313-macosx_13_0_x86_64.whl", hash = "sha256:162f4ca894ec3de1a6fd53589e511e06ecdc3ff646849b62a9da7489dee9ce95", size = 19157691 }, + { url = "https://files.pythonhosted.org/packages/1c/a1/428ee29c6eaf09a6f6be56f836213f104618fb35ac6cc586ff0f477263eb/onnxruntime-1.23.2-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:45d127d6e1e9b99d1ebeae9bcd8f98617a812f53f46699eafeb976275744826b", size = 15226898 }, + { url = "https://files.pythonhosted.org/packages/f2/2b/b57c8a2466a3126dbe0a792f56ad7290949b02f47b86216cd47d857e4b77/onnxruntime-1.23.2-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8bace4e0d46480fbeeb7bbe1ffe1f080e6663a42d1086ff95c1551f2d39e7872", size = 17382518 }, + { url = "https://files.pythonhosted.org/packages/4a/93/aba75358133b3a941d736816dd392f687e7eab77215a6e429879080b76b6/onnxruntime-1.23.2-cp313-cp313-win_amd64.whl", hash = "sha256:1f9cc0a55349c584f083c1c076e611a7c35d5b867d5d6e6d6c823bf821978088", size = 13470276 }, + { url = "https://files.pythonhosted.org/packages/7c/3d/6830fa61c69ca8e905f237001dbfc01689a4e4ab06147020a4518318881f/onnxruntime-1.23.2-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9d2385e774f46ac38f02b3a91a91e30263d41b2f1f4f26ae34805b2a9ddef466", size = 15229610 }, + { url = "https://files.pythonhosted.org/packages/b6/ca/862b1e7a639460f0ca25fd5b6135fb42cf9deea86d398a92e44dfda2279d/onnxruntime-1.23.2-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e2b9233c4947907fd1818d0e581c049c41ccc39b2856cc942ff6d26317cee145", size = 17394184 }, +] + +[[package]] +name = "opencv-python" +version = "4.12.0.88" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "numpy" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/ac/71/25c98e634b6bdeca4727c7f6d6927b056080668c5008ad3c8fc9e7f8f6ec/opencv-python-4.12.0.88.tar.gz", hash = "sha256:8b738389cede219405f6f3880b851efa3415ccd674752219377353f017d2994d", size = 95373294 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/85/68/3da40142e7c21e9b1d4e7ddd6c58738feb013203e6e4b803d62cdd9eb96b/opencv_python-4.12.0.88-cp37-abi3-macosx_13_0_arm64.whl", hash = "sha256:f9a1f08883257b95a5764bf517a32d75aec325319c8ed0f89739a57fae9e92a5", size = 37877727 }, + { url = "https://files.pythonhosted.org/packages/33/7c/042abe49f58d6ee7e1028eefc3334d98ca69b030e3b567fe245a2b28ea6f/opencv_python-4.12.0.88-cp37-abi3-macosx_13_0_x86_64.whl", hash = "sha256:812eb116ad2b4de43ee116fcd8991c3a687f099ada0b04e68f64899c09448e81", size = 57326471 }, + { url = "https://files.pythonhosted.org/packages/62/3a/440bd64736cf8116f01f3b7f9f2e111afb2e02beb2ccc08a6458114a6b5d/opencv_python-4.12.0.88-cp37-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:51fd981c7df6af3e8f70b1556696b05224c4e6b6777bdd2a46b3d4fb09de1a92", size = 45887139 }, + { url = "https://files.pythonhosted.org/packages/68/1f/795e7f4aa2eacc59afa4fb61a2e35e510d06414dd5a802b51a012d691b37/opencv_python-4.12.0.88-cp37-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:092c16da4c5a163a818f120c22c5e4a2f96e0db4f24e659c701f1fe629a690f9", size = 67041680 }, + { url = "https://files.pythonhosted.org/packages/02/96/213fea371d3cb2f1d537612a105792aa0a6659fb2665b22cad709a75bd94/opencv_python-4.12.0.88-cp37-abi3-win32.whl", hash = "sha256:ff554d3f725b39878ac6a2e1fa232ec509c36130927afc18a1719ebf4fbf4357", size = 30284131 }, + { url = "https://files.pythonhosted.org/packages/fa/80/eb88edc2e2b11cd2dd2e56f1c80b5784d11d6e6b7f04a1145df64df40065/opencv_python-4.12.0.88-cp37-abi3-win_amd64.whl", hash = "sha256:d98edb20aa932fd8ebd276a72627dad9dc097695b3d435a4257557bbb49a79d2", size = 39000307 }, +] + +[[package]] +name = "opencv-python-headless" +version = "4.12.0.88" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "numpy" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/a4/63/6861102ec149c3cd298f4d1ea7ce9d6adbc7529221606ff1dab991a19adb/opencv-python-headless-4.12.0.88.tar.gz", hash = "sha256:cfdc017ddf2e59b6c2f53bc12d74b6b0be7ded4ec59083ea70763921af2b6c09", size = 95379675 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f7/7d/414e243c5c8216a5277afd104a319cc1291c5e23f5eeef512db5629ee7f4/opencv_python_headless-4.12.0.88-cp37-abi3-macosx_13_0_arm64.whl", hash = "sha256:1e58d664809b3350c1123484dd441e1667cd7bed3086db1b9ea1b6f6cb20b50e", size = 37877864 }, + { url = "https://files.pythonhosted.org/packages/05/14/7e162714beed1cd5e7b5eb66fcbcba2f065c51b1d9da2463024c84d2f7c0/opencv_python_headless-4.12.0.88-cp37-abi3-macosx_13_0_x86_64.whl", hash = "sha256:365bb2e486b50feffc2d07a405b953a8f3e8eaa63865bc650034e5c71e7a5154", size = 57326608 }, + { url = "https://files.pythonhosted.org/packages/69/4e/116720df7f1f7f3b59abc608ca30fbec9d2b3ae810afe4e4d26483d9dfa0/opencv_python_headless-4.12.0.88-cp37-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:aeb4b13ecb8b4a0beb2668ea07928160ea7c2cd2d9b5ef571bbee6bafe9cc8d0", size = 33145800 }, + { url = "https://files.pythonhosted.org/packages/89/53/e19c21e0c4eb1275c3e2c97b081103b6dfb3938172264d283a519bf728b9/opencv_python_headless-4.12.0.88-cp37-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:236c8df54a90f4d02076e6f9c1cc763d794542e886c576a6fee46ec8ff75a7a9", size = 54023419 }, + { url = "https://files.pythonhosted.org/packages/bf/9c/a76fd5414de6ec9f21f763a600058a0c3e290053cea87e0275692b1375c0/opencv_python_headless-4.12.0.88-cp37-abi3-win32.whl", hash = "sha256:fde2cf5c51e4def5f2132d78e0c08f9c14783cd67356922182c6845b9af87dbd", size = 30225230 }, + { url = "https://files.pythonhosted.org/packages/f2/35/0858e9e71b36948eafbc5e835874b63e515179dc3b742cbe3d76bc683439/opencv_python_headless-4.12.0.88-cp37-abi3-win_amd64.whl", hash = "sha256:86b413bdd6c6bf497832e346cd5371995de148e579b9774f8eba686dee3f5528", size = 38923559 }, +] + +[[package]] +name = "packaging" +version = "25.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/a1/d4/1fc4078c65507b51b96ca8f8c3ba19e6a61c8253c72794544580a7b6c24d/packaging-25.0.tar.gz", hash = "sha256:d443872c98d677bf60f6a1f2f8c1cb748e8fe762d2bf9d3148b5599295b0fc4f", size = 165727 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/20/12/38679034af332785aac8774540895e234f4d07f7545804097de4b666afd8/packaging-25.0-py3-none-any.whl", hash = "sha256:29572ef2b1f17581046b3a2227d5c611fb25ec70ca1ba8554b24b0e69331a484", size = 66469 }, +] + +[[package]] +name = "pandas" +version = "2.3.3" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "numpy" }, + { name = "python-dateutil" }, + { name = "pytz" }, + { name = "tzdata" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/33/01/d40b85317f86cf08d853a4f495195c73815fdf205eef3993821720274518/pandas-2.3.3.tar.gz", hash = "sha256:e05e1af93b977f7eafa636d043f9f94c7ee3ac81af99c13508215942e64c993b", size = 4495223 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c1/fa/7ac648108144a095b4fb6aa3de1954689f7af60a14cf25583f4960ecb878/pandas-2.3.3-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:602b8615ebcc4a0c1751e71840428ddebeb142ec02c786e8ad6b1ce3c8dec523", size = 11578790 }, + { url = "https://files.pythonhosted.org/packages/9b/35/74442388c6cf008882d4d4bdfc4109be87e9b8b7ccd097ad1e7f006e2e95/pandas-2.3.3-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:8fe25fc7b623b0ef6b5009149627e34d2a4657e880948ec3c840e9402e5c1b45", size = 10833831 }, + { url = "https://files.pythonhosted.org/packages/fe/e4/de154cbfeee13383ad58d23017da99390b91d73f8c11856f2095e813201b/pandas-2.3.3-cp311-cp311-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b468d3dad6ff947df92dcb32ede5b7bd41a9b3cceef0a30ed925f6d01fb8fa66", size = 12199267 }, + { url = "https://files.pythonhosted.org/packages/bf/c9/63f8d545568d9ab91476b1818b4741f521646cbdd151c6efebf40d6de6f7/pandas-2.3.3-cp311-cp311-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b98560e98cb334799c0b07ca7967ac361a47326e9b4e5a7dfb5ab2b1c9d35a1b", size = 12789281 }, + { url = "https://files.pythonhosted.org/packages/f2/00/a5ac8c7a0e67fd1a6059e40aa08fa1c52cc00709077d2300e210c3ce0322/pandas-2.3.3-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:1d37b5848ba49824e5c30bedb9c830ab9b7751fd049bc7914533e01c65f79791", size = 13240453 }, + { url = "https://files.pythonhosted.org/packages/27/4d/5c23a5bc7bd209231618dd9e606ce076272c9bc4f12023a70e03a86b4067/pandas-2.3.3-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:db4301b2d1f926ae677a751eb2bd0e8c5f5319c9cb3f88b0becbbb0b07b34151", size = 13890361 }, + { url = "https://files.pythonhosted.org/packages/8e/59/712db1d7040520de7a4965df15b774348980e6df45c129b8c64d0dbe74ef/pandas-2.3.3-cp311-cp311-win_amd64.whl", hash = "sha256:f086f6fe114e19d92014a1966f43a3e62285109afe874f067f5abbdcbb10e59c", size = 11348702 }, + { url = "https://files.pythonhosted.org/packages/9c/fb/231d89e8637c808b997d172b18e9d4a4bc7bf31296196c260526055d1ea0/pandas-2.3.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:6d21f6d74eb1725c2efaa71a2bfc661a0689579b58e9c0ca58a739ff0b002b53", size = 11597846 }, + { url = "https://files.pythonhosted.org/packages/5c/bd/bf8064d9cfa214294356c2d6702b716d3cf3bb24be59287a6a21e24cae6b/pandas-2.3.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:3fd2f887589c7aa868e02632612ba39acb0b8948faf5cc58f0850e165bd46f35", size = 10729618 }, + { url = "https://files.pythonhosted.org/packages/57/56/cf2dbe1a3f5271370669475ead12ce77c61726ffd19a35546e31aa8edf4e/pandas-2.3.3-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ecaf1e12bdc03c86ad4a7ea848d66c685cb6851d807a26aa245ca3d2017a1908", size = 11737212 }, + { url = "https://files.pythonhosted.org/packages/e5/63/cd7d615331b328e287d8233ba9fdf191a9c2d11b6af0c7a59cfcec23de68/pandas-2.3.3-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b3d11d2fda7eb164ef27ffc14b4fcab16a80e1ce67e9f57e19ec0afaf715ba89", size = 12362693 }, + { url = "https://files.pythonhosted.org/packages/a6/de/8b1895b107277d52f2b42d3a6806e69cfef0d5cf1d0ba343470b9d8e0a04/pandas-2.3.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:a68e15f780eddf2b07d242e17a04aa187a7ee12b40b930bfdd78070556550e98", size = 12771002 }, + { url = "https://files.pythonhosted.org/packages/87/21/84072af3187a677c5893b170ba2c8fbe450a6ff911234916da889b698220/pandas-2.3.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:371a4ab48e950033bcf52b6527eccb564f52dc826c02afd9a1bc0ab731bba084", size = 13450971 }, + { url = "https://files.pythonhosted.org/packages/86/41/585a168330ff063014880a80d744219dbf1dd7a1c706e75ab3425a987384/pandas-2.3.3-cp312-cp312-win_amd64.whl", hash = "sha256:a16dcec078a01eeef8ee61bf64074b4e524a2a3f4b3be9326420cabe59c4778b", size = 10992722 }, + { url = "https://files.pythonhosted.org/packages/cd/4b/18b035ee18f97c1040d94debd8f2e737000ad70ccc8f5513f4eefad75f4b/pandas-2.3.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:56851a737e3470de7fa88e6131f41281ed440d29a9268dcbf0002da5ac366713", size = 11544671 }, + { url = "https://files.pythonhosted.org/packages/31/94/72fac03573102779920099bcac1c3b05975c2cb5f01eac609faf34bed1ca/pandas-2.3.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:bdcd9d1167f4885211e401b3036c0c8d9e274eee67ea8d0758a256d60704cfe8", size = 10680807 }, + { url = "https://files.pythonhosted.org/packages/16/87/9472cf4a487d848476865321de18cc8c920b8cab98453ab79dbbc98db63a/pandas-2.3.3-cp313-cp313-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e32e7cc9af0f1cc15548288a51a3b681cc2a219faa838e995f7dc53dbab1062d", size = 11709872 }, + { url = "https://files.pythonhosted.org/packages/15/07/284f757f63f8a8d69ed4472bfd85122bd086e637bf4ed09de572d575a693/pandas-2.3.3-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:318d77e0e42a628c04dc56bcef4b40de67918f7041c2b061af1da41dcff670ac", size = 12306371 }, + { url = "https://files.pythonhosted.org/packages/33/81/a3afc88fca4aa925804a27d2676d22dcd2031c2ebe08aabd0ae55b9ff282/pandas-2.3.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:4e0a175408804d566144e170d0476b15d78458795bb18f1304fb94160cabf40c", size = 12765333 }, + { url = "https://files.pythonhosted.org/packages/8d/0f/b4d4ae743a83742f1153464cf1a8ecfafc3ac59722a0b5c8602310cb7158/pandas-2.3.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:93c2d9ab0fc11822b5eece72ec9587e172f63cff87c00b062f6e37448ced4493", size = 13418120 }, + { url = "https://files.pythonhosted.org/packages/4f/c7/e54682c96a895d0c808453269e0b5928a07a127a15704fedb643e9b0a4c8/pandas-2.3.3-cp313-cp313-win_amd64.whl", hash = "sha256:f8bfc0e12dc78f777f323f55c58649591b2cd0c43534e8355c51d3fede5f4dee", size = 10993991 }, + { url = "https://files.pythonhosted.org/packages/f9/ca/3f8d4f49740799189e1395812f3bf23b5e8fc7c190827d55a610da72ce55/pandas-2.3.3-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:75ea25f9529fdec2d2e93a42c523962261e567d250b0013b16210e1d40d7c2e5", size = 12048227 }, + { url = "https://files.pythonhosted.org/packages/0e/5a/f43efec3e8c0cc92c4663ccad372dbdff72b60bdb56b2749f04aa1d07d7e/pandas-2.3.3-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:74ecdf1d301e812db96a465a525952f4dde225fdb6d8e5a521d47e1f42041e21", size = 11411056 }, + { url = "https://files.pythonhosted.org/packages/46/b1/85331edfc591208c9d1a63a06baa67b21d332e63b7a591a5ba42a10bb507/pandas-2.3.3-cp313-cp313t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6435cb949cb34ec11cc9860246ccb2fdc9ecd742c12d3304989017d53f039a78", size = 11645189 }, + { url = "https://files.pythonhosted.org/packages/44/23/78d645adc35d94d1ac4f2a3c4112ab6f5b8999f4898b8cdf01252f8df4a9/pandas-2.3.3-cp313-cp313t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:900f47d8f20860de523a1ac881c4c36d65efcb2eb850e6948140fa781736e110", size = 12121912 }, + { url = "https://files.pythonhosted.org/packages/53/da/d10013df5e6aaef6b425aa0c32e1fc1f3e431e4bcabd420517dceadce354/pandas-2.3.3-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:a45c765238e2ed7d7c608fc5bc4a6f88b642f2f01e70c0c23d2224dd21829d86", size = 12712160 }, + { url = "https://files.pythonhosted.org/packages/bd/17/e756653095a083d8a37cbd816cb87148debcfcd920129b25f99dd8d04271/pandas-2.3.3-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:c4fc4c21971a1a9f4bdb4c73978c7f7256caa3e62b323f70d6cb80db583350bc", size = 13199233 }, + { url = "https://files.pythonhosted.org/packages/04/fd/74903979833db8390b73b3a8a7d30d146d710bd32703724dd9083950386f/pandas-2.3.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:ee15f284898e7b246df8087fc82b87b01686f98ee67d85a17b7ab44143a3a9a0", size = 11540635 }, + { url = "https://files.pythonhosted.org/packages/21/00/266d6b357ad5e6d3ad55093a7e8efc7dd245f5a842b584db9f30b0f0a287/pandas-2.3.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:1611aedd912e1ff81ff41c745822980c49ce4a7907537be8692c8dbc31924593", size = 10759079 }, + { url = "https://files.pythonhosted.org/packages/ca/05/d01ef80a7a3a12b2f8bbf16daba1e17c98a2f039cbc8e2f77a2c5a63d382/pandas-2.3.3-cp314-cp314-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6d2cefc361461662ac48810cb14365a365ce864afe85ef1f447ff5a1e99ea81c", size = 11814049 }, + { url = "https://files.pythonhosted.org/packages/15/b2/0e62f78c0c5ba7e3d2c5945a82456f4fac76c480940f805e0b97fcbc2f65/pandas-2.3.3-cp314-cp314-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ee67acbbf05014ea6c763beb097e03cd629961c8a632075eeb34247120abcb4b", size = 12332638 }, + { url = "https://files.pythonhosted.org/packages/c5/33/dd70400631b62b9b29c3c93d2feee1d0964dc2bae2e5ad7a6c73a7f25325/pandas-2.3.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:c46467899aaa4da076d5abc11084634e2d197e9460643dd455ac3db5856b24d6", size = 12886834 }, + { url = "https://files.pythonhosted.org/packages/d3/18/b5d48f55821228d0d2692b34fd5034bb185e854bdb592e9c640f6290e012/pandas-2.3.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:6253c72c6a1d990a410bc7de641d34053364ef8bcd3126f7e7450125887dffe3", size = 13409925 }, + { url = "https://files.pythonhosted.org/packages/a6/3d/124ac75fcd0ecc09b8fdccb0246ef65e35b012030defb0e0eba2cbbbe948/pandas-2.3.3-cp314-cp314-win_amd64.whl", hash = "sha256:1b07204a219b3b7350abaae088f451860223a52cfb8a6c53358e7948735158e5", size = 11109071 }, + { url = "https://files.pythonhosted.org/packages/89/9c/0e21c895c38a157e0faa1fb64587a9226d6dd46452cac4532d80c3c4a244/pandas-2.3.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:2462b1a365b6109d275250baaae7b760fd25c726aaca0054649286bcfbb3e8ec", size = 12048504 }, + { url = "https://files.pythonhosted.org/packages/d7/82/b69a1c95df796858777b68fbe6a81d37443a33319761d7c652ce77797475/pandas-2.3.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:0242fe9a49aa8b4d78a4fa03acb397a58833ef6199e9aa40a95f027bb3a1b6e7", size = 11410702 }, + { url = "https://files.pythonhosted.org/packages/f9/88/702bde3ba0a94b8c73a0181e05144b10f13f29ebfc2150c3a79062a8195d/pandas-2.3.3-cp314-cp314t-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a21d830e78df0a515db2b3d2f5570610f5e6bd2e27749770e8bb7b524b89b450", size = 11634535 }, + { url = "https://files.pythonhosted.org/packages/a4/1e/1bac1a839d12e6a82ec6cb40cda2edde64a2013a66963293696bbf31fbbb/pandas-2.3.3-cp314-cp314t-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:2e3ebdb170b5ef78f19bfb71b0dc5dc58775032361fa188e814959b74d726dd5", size = 12121582 }, + { url = "https://files.pythonhosted.org/packages/44/91/483de934193e12a3b1d6ae7c8645d083ff88dec75f46e827562f1e4b4da6/pandas-2.3.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:d051c0e065b94b7a3cea50eb1ec32e912cd96dba41647eb24104b6c6c14c5788", size = 12699963 }, + { url = "https://files.pythonhosted.org/packages/70/44/5191d2e4026f86a2a109053e194d3ba7a31a2d10a9c2348368c63ed4e85a/pandas-2.3.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:3869faf4bd07b3b66a9f462417d0ca3a9df29a9f6abd5d0d0dbab15dac7abe87", size = 13202175 }, +] + +[[package]] +name = "parsel" +version = "1.10.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "cssselect" }, + { name = "jmespath" }, + { name = "lxml" }, + { name = "packaging" }, + { name = "w3lib" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/f6/df/acd504c154c0b9028b0d8491a77fdd5f86e9c06ee04f986abf85e36d9a5f/parsel-1.10.0.tar.gz", hash = "sha256:14f17db9559f51b43357b9dfe43cec870a8efb5ea4857abb624ec6ff80d8a080", size = 51421 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/12/18/35d1d947553d24909dca37e2ff11720eecb601360d1bac8d7a9a1bc7eb08/parsel-1.10.0-py2.py3-none-any.whl", hash = "sha256:6a0c28bd81f9df34ba665884c88efa0b18b8d2c44c81f64e27f2f0cb37d46169", size = 17266 }, +] + +[[package]] +name = "pillow" +version = "12.1.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d0/02/d52c733a2452ef1ffcc123b68e6606d07276b0e358db70eabad7e40042b7/pillow-12.1.0.tar.gz", hash = "sha256:5c5ae0a06e9ea030ab786b0251b32c7e4ce10e58d983c0d5c56029455180b5b9", size = 46977283 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/43/c4/bf8328039de6cc22182c3ef007a2abfbbdab153661c0a9aa78af8d706391/pillow-12.1.0-cp311-cp311-macosx_10_10_x86_64.whl", hash = "sha256:a83e0850cb8f5ac975291ebfc4170ba481f41a28065277f7f735c202cd8e0af3", size = 5304057 }, + { url = "https://files.pythonhosted.org/packages/43/06/7264c0597e676104cc22ca73ee48f752767cd4b1fe084662620b17e10120/pillow-12.1.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:b6e53e82ec2db0717eabb276aa56cf4e500c9a7cec2c2e189b55c24f65a3e8c0", size = 4657811 }, + { url = "https://files.pythonhosted.org/packages/72/64/f9189e44474610daf83da31145fa56710b627b5c4c0b9c235e34058f6b31/pillow-12.1.0-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:40a8e3b9e8773876d6e30daed22f016509e3987bab61b3b7fe309d7019a87451", size = 6232243 }, + { url = "https://files.pythonhosted.org/packages/ef/30/0df458009be6a4caca4ca2c52975e6275c387d4e5c95544e34138b41dc86/pillow-12.1.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:800429ac32c9b72909c671aaf17ecd13110f823ddb7db4dfef412a5587c2c24e", size = 8037872 }, + { url = "https://files.pythonhosted.org/packages/e4/86/95845d4eda4f4f9557e25381d70876aa213560243ac1a6d619c46caaedd9/pillow-12.1.0-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0b022eaaf709541b391ee069f0022ee5b36c709df71986e3f7be312e46f42c84", size = 6345398 }, + { url = "https://files.pythonhosted.org/packages/5c/1f/8e66ab9be3aaf1435bc03edd1ebdf58ffcd17f7349c1d970cafe87af27d9/pillow-12.1.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1f345e7bc9d7f368887c712aa5054558bad44d2a301ddf9248599f4161abc7c0", size = 7034667 }, + { url = "https://files.pythonhosted.org/packages/f9/f6/683b83cb9b1db1fb52b87951b1c0b99bdcfceaa75febf11406c19f82cb5e/pillow-12.1.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:d70347c8a5b7ccd803ec0c85c8709f036e6348f1e6a5bf048ecd9c64d3550b8b", size = 6458743 }, + { url = "https://files.pythonhosted.org/packages/9a/7d/de833d63622538c1d58ce5395e7c6cb7e7dce80decdd8bde4a484e095d9f/pillow-12.1.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:1fcc52d86ce7a34fd17cb04e87cfdb164648a3662a6f20565910a99653d66c18", size = 7159342 }, + { url = "https://files.pythonhosted.org/packages/8c/40/50d86571c9e5868c42b81fe7da0c76ca26373f3b95a8dd675425f4a92ec1/pillow-12.1.0-cp311-cp311-win32.whl", hash = "sha256:3ffaa2f0659e2f740473bcf03c702c39a8d4b2b7ffc629052028764324842c64", size = 6328655 }, + { url = "https://files.pythonhosted.org/packages/6c/af/b1d7e301c4cd26cd45d4af884d9ee9b6fab893b0ad2450d4746d74a6968c/pillow-12.1.0-cp311-cp311-win_amd64.whl", hash = "sha256:806f3987ffe10e867bab0ddad45df1148a2b98221798457fa097ad85d6e8bc75", size = 7031469 }, + { url = "https://files.pythonhosted.org/packages/48/36/d5716586d887fb2a810a4a61518a327a1e21c8b7134c89283af272efe84b/pillow-12.1.0-cp311-cp311-win_arm64.whl", hash = "sha256:9f5fefaca968e700ad1a4a9de98bf0869a94e397fe3524c4c9450c1445252304", size = 2452515 }, + { url = "https://files.pythonhosted.org/packages/20/31/dc53fe21a2f2996e1b7d92bf671cdb157079385183ef7c1ae08b485db510/pillow-12.1.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:a332ac4ccb84b6dde65dbace8431f3af08874bf9770719d32a635c4ef411b18b", size = 5262642 }, + { url = "https://files.pythonhosted.org/packages/ab/c1/10e45ac9cc79419cedf5121b42dcca5a50ad2b601fa080f58c22fb27626e/pillow-12.1.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:907bfa8a9cb790748a9aa4513e37c88c59660da3bcfffbd24a7d9e6abf224551", size = 4657464 }, + { url = "https://files.pythonhosted.org/packages/ad/26/7b82c0ab7ef40ebede7a97c72d473bda5950f609f8e0c77b04af574a0ddb/pillow-12.1.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:efdc140e7b63b8f739d09a99033aa430accce485ff78e6d311973a67b6bf3208", size = 6234878 }, + { url = "https://files.pythonhosted.org/packages/76/25/27abc9792615b5e886ca9411ba6637b675f1b77af3104710ac7353fe5605/pillow-12.1.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:bef9768cab184e7ae6e559c032e95ba8d07b3023c289f79a2bd36e8bf85605a5", size = 8044868 }, + { url = "https://files.pythonhosted.org/packages/0a/ea/f200a4c36d836100e7bc738fc48cd963d3ba6372ebc8298a889e0cfc3359/pillow-12.1.0-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:742aea052cf5ab5034a53c3846165bc3ce88d7c38e954120db0ab867ca242661", size = 6349468 }, + { url = "https://files.pythonhosted.org/packages/11/8f/48d0b77ab2200374c66d344459b8958c86693be99526450e7aee714e03e4/pillow-12.1.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a6dfc2af5b082b635af6e08e0d1f9f1c4e04d17d4e2ca0ef96131e85eda6eb17", size = 7041518 }, + { url = "https://files.pythonhosted.org/packages/1d/23/c281182eb986b5d31f0a76d2a2c8cd41722d6fb8ed07521e802f9bba52de/pillow-12.1.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:609e89d9f90b581c8d16358c9087df76024cf058fa693dd3e1e1620823f39670", size = 6462829 }, + { url = "https://files.pythonhosted.org/packages/25/ef/7018273e0faac099d7b00982abdcc39142ae6f3bd9ceb06de09779c4a9d6/pillow-12.1.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:43b4899cfd091a9693a1278c4982f3e50f7fb7cff5153b05174b4afc9593b616", size = 7166756 }, + { url = "https://files.pythonhosted.org/packages/8f/c8/993d4b7ab2e341fe02ceef9576afcf5830cdec640be2ac5bee1820d693d4/pillow-12.1.0-cp312-cp312-win32.whl", hash = "sha256:aa0c9cc0b82b14766a99fbe6084409972266e82f459821cd26997a488a7261a7", size = 6328770 }, + { url = "https://files.pythonhosted.org/packages/a7/87/90b358775a3f02765d87655237229ba64a997b87efa8ccaca7dd3e36e7a7/pillow-12.1.0-cp312-cp312-win_amd64.whl", hash = "sha256:d70534cea9e7966169ad29a903b99fc507e932069a881d0965a1a84bb57f6c6d", size = 7033406 }, + { url = "https://files.pythonhosted.org/packages/5d/cf/881b457eccacac9e5b2ddd97d5071fb6d668307c57cbf4e3b5278e06e536/pillow-12.1.0-cp312-cp312-win_arm64.whl", hash = "sha256:65b80c1ee7e14a87d6a068dd3b0aea268ffcabfe0498d38661b00c5b4b22e74c", size = 2452612 }, + { url = "https://files.pythonhosted.org/packages/dd/c7/2530a4aa28248623e9d7f27316b42e27c32ec410f695929696f2e0e4a778/pillow-12.1.0-cp313-cp313-ios_13_0_arm64_iphoneos.whl", hash = "sha256:7b5dd7cbae20285cdb597b10eb5a2c13aa9de6cde9bb64a3c1317427b1db1ae1", size = 4062543 }, + { url = "https://files.pythonhosted.org/packages/8f/1f/40b8eae823dc1519b87d53c30ed9ef085506b05281d313031755c1705f73/pillow-12.1.0-cp313-cp313-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:29a4cef9cb672363926f0470afc516dbf7305a14d8c54f7abbb5c199cd8f8179", size = 4138373 }, + { url = "https://files.pythonhosted.org/packages/d4/77/6fa60634cf06e52139fd0e89e5bbf055e8166c691c42fb162818b7fda31d/pillow-12.1.0-cp313-cp313-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:681088909d7e8fa9e31b9799aaa59ba5234c58e5e4f1951b4c4d1082a2e980e0", size = 3601241 }, + { url = "https://files.pythonhosted.org/packages/4f/bf/28ab865de622e14b747f0cd7877510848252d950e43002e224fb1c9ababf/pillow-12.1.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:983976c2ab753166dc66d36af6e8ec15bb511e4a25856e2227e5f7e00a160587", size = 5262410 }, + { url = "https://files.pythonhosted.org/packages/1c/34/583420a1b55e715937a85bd48c5c0991598247a1fd2eb5423188e765ea02/pillow-12.1.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:db44d5c160a90df2d24a24760bbd37607d53da0b34fb546c4c232af7192298ac", size = 4657312 }, + { url = "https://files.pythonhosted.org/packages/1d/fd/f5a0896839762885b3376ff04878f86ab2b097c2f9a9cdccf4eda8ba8dc0/pillow-12.1.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:6b7a9d1db5dad90e2991645874f708e87d9a3c370c243c2d7684d28f7e133e6b", size = 6232605 }, + { url = "https://files.pythonhosted.org/packages/98/aa/938a09d127ac1e70e6ed467bd03834350b33ef646b31edb7452d5de43792/pillow-12.1.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:6258f3260986990ba2fa8a874f8b6e808cf5abb51a94015ca3dc3c68aa4f30ea", size = 8041617 }, + { url = "https://files.pythonhosted.org/packages/17/e8/538b24cb426ac0186e03f80f78bc8dc7246c667f58b540bdd57c71c9f79d/pillow-12.1.0-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e115c15e3bc727b1ca3e641a909f77f8ca72a64fff150f666fcc85e57701c26c", size = 6346509 }, + { url = "https://files.pythonhosted.org/packages/01/9a/632e58ec89a32738cabfd9ec418f0e9898a2b4719afc581f07c04a05e3c9/pillow-12.1.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6741e6f3074a35e47c77b23a4e4f2d90db3ed905cb1c5e6e0d49bff2045632bc", size = 7038117 }, + { url = "https://files.pythonhosted.org/packages/c7/a2/d40308cf86eada842ca1f3ffa45d0ca0df7e4ab33c83f81e73f5eaed136d/pillow-12.1.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:935b9d1aed48fcfb3f838caac506f38e29621b44ccc4f8a64d575cb1b2a88644", size = 6460151 }, + { url = "https://files.pythonhosted.org/packages/f1/88/f5b058ad6453a085c5266660a1417bdad590199da1b32fb4efcff9d33b05/pillow-12.1.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:5fee4c04aad8932da9f8f710af2c1a15a83582cfb884152a9caa79d4efcdbf9c", size = 7164534 }, + { url = "https://files.pythonhosted.org/packages/19/ce/c17334caea1db789163b5d855a5735e47995b0b5dc8745e9a3605d5f24c0/pillow-12.1.0-cp313-cp313-win32.whl", hash = "sha256:a786bf667724d84aa29b5db1c61b7bfdde380202aaca12c3461afd6b71743171", size = 6332551 }, + { url = "https://files.pythonhosted.org/packages/e5/07/74a9d941fa45c90a0d9465098fe1ec85de3e2afbdc15cc4766622d516056/pillow-12.1.0-cp313-cp313-win_amd64.whl", hash = "sha256:461f9dfdafa394c59cd6d818bdfdbab4028b83b02caadaff0ffd433faf4c9a7a", size = 7040087 }, + { url = "https://files.pythonhosted.org/packages/88/09/c99950c075a0e9053d8e880595926302575bc742b1b47fe1bbcc8d388d50/pillow-12.1.0-cp313-cp313-win_arm64.whl", hash = "sha256:9212d6b86917a2300669511ed094a9406888362e085f2431a7da985a6b124f45", size = 2452470 }, + { url = "https://files.pythonhosted.org/packages/b5/ba/970b7d85ba01f348dee4d65412476321d40ee04dcb51cd3735b9dc94eb58/pillow-12.1.0-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:00162e9ca6d22b7c3ee8e61faa3c3253cd19b6a37f126cad04f2f88b306f557d", size = 5264816 }, + { url = "https://files.pythonhosted.org/packages/10/60/650f2fb55fdba7a510d836202aa52f0baac633e50ab1cf18415d332188fb/pillow-12.1.0-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:7d6daa89a00b58c37cb1747ec9fb7ac3bc5ffd5949f5888657dfddde6d1312e0", size = 4660472 }, + { url = "https://files.pythonhosted.org/packages/2b/c0/5273a99478956a099d533c4f46cbaa19fd69d606624f4334b85e50987a08/pillow-12.1.0-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:e2479c7f02f9d505682dc47df8c0ea1fc5e264c4d1629a5d63fe3e2334b89554", size = 6268974 }, + { url = "https://files.pythonhosted.org/packages/b4/26/0bf714bc2e73d5267887d47931d53c4ceeceea6978148ed2ab2a4e6463c4/pillow-12.1.0-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:f188d580bd870cda1e15183790d1cc2fa78f666e76077d103edf048eed9c356e", size = 8073070 }, + { url = "https://files.pythonhosted.org/packages/43/cf/1ea826200de111a9d65724c54f927f3111dc5ae297f294b370a670c17786/pillow-12.1.0-cp313-cp313t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0fde7ec5538ab5095cc02df38ee99b0443ff0e1c847a045554cf5f9af1f4aa82", size = 6380176 }, + { url = "https://files.pythonhosted.org/packages/03/e0/7938dd2b2013373fd85d96e0f38d62b7a5a262af21ac274250c7ca7847c9/pillow-12.1.0-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0ed07dca4a8464bada6139ab38f5382f83e5f111698caf3191cb8dbf27d908b4", size = 7067061 }, + { url = "https://files.pythonhosted.org/packages/86/ad/a2aa97d37272a929a98437a8c0ac37b3cf012f4f8721e1bd5154699b2518/pillow-12.1.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:f45bd71d1fa5e5749587613037b172e0b3b23159d1c00ef2fc920da6f470e6f0", size = 6491824 }, + { url = "https://files.pythonhosted.org/packages/a4/44/80e46611b288d51b115826f136fb3465653c28f491068a72d3da49b54cd4/pillow-12.1.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:277518bf4fe74aa91489e1b20577473b19ee70fb97c374aa50830b279f25841b", size = 7190911 }, + { url = "https://files.pythonhosted.org/packages/86/77/eacc62356b4cf81abe99ff9dbc7402750044aed02cfd6a503f7c6fc11f3e/pillow-12.1.0-cp313-cp313t-win32.whl", hash = "sha256:7315f9137087c4e0ee73a761b163fc9aa3b19f5f606a7fc08d83fd3e4379af65", size = 6336445 }, + { url = "https://files.pythonhosted.org/packages/e7/3c/57d81d0b74d218706dafccb87a87ea44262c43eef98eb3b164fd000e0491/pillow-12.1.0-cp313-cp313t-win_amd64.whl", hash = "sha256:0ddedfaa8b5f0b4ffbc2fa87b556dc59f6bb4ecb14a53b33f9189713ae8053c0", size = 7045354 }, + { url = "https://files.pythonhosted.org/packages/ac/82/8b9b97bba2e3576a340f93b044a3a3a09841170ab4c1eb0d5c93469fd32f/pillow-12.1.0-cp313-cp313t-win_arm64.whl", hash = "sha256:80941e6d573197a0c28f394753de529bb436b1ca990ed6e765cf42426abc39f8", size = 2454547 }, + { url = "https://files.pythonhosted.org/packages/8c/87/bdf971d8bbcf80a348cc3bacfcb239f5882100fe80534b0ce67a784181d8/pillow-12.1.0-cp314-cp314-ios_13_0_arm64_iphoneos.whl", hash = "sha256:5cb7bc1966d031aec37ddb9dcf15c2da5b2e9f7cc3ca7c54473a20a927e1eb91", size = 4062533 }, + { url = "https://files.pythonhosted.org/packages/ff/4f/5eb37a681c68d605eb7034c004875c81f86ec9ef51f5be4a63eadd58859a/pillow-12.1.0-cp314-cp314-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:97e9993d5ed946aba26baf9c1e8cf18adbab584b99f452ee72f7ee8acb882796", size = 4138546 }, + { url = "https://files.pythonhosted.org/packages/11/6d/19a95acb2edbace40dcd582d077b991646b7083c41b98da4ed7555b59733/pillow-12.1.0-cp314-cp314-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:414b9a78e14ffeb98128863314e62c3f24b8a86081066625700b7985b3f529bd", size = 3601163 }, + { url = "https://files.pythonhosted.org/packages/fc/36/2b8138e51cb42e4cc39c3297713455548be855a50558c3ac2beebdc251dd/pillow-12.1.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:e6bdb408f7c9dd2a5ff2b14a3b0bb6d4deb29fb9961e6eb3ae2031ae9a5cec13", size = 5266086 }, + { url = "https://files.pythonhosted.org/packages/53/4b/649056e4d22e1caa90816bf99cef0884aed607ed38075bd75f091a607a38/pillow-12.1.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:3413c2ae377550f5487991d444428f1a8ae92784aac79caa8b1e3b89b175f77e", size = 4657344 }, + { url = "https://files.pythonhosted.org/packages/6c/6b/c5742cea0f1ade0cd61485dc3d81f05261fc2276f537fbdc00802de56779/pillow-12.1.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:e5dcbe95016e88437ecf33544ba5db21ef1b8dd6e1b434a2cb2a3d605299e643", size = 6232114 }, + { url = "https://files.pythonhosted.org/packages/bf/8f/9f521268ce22d63991601aafd3d48d5ff7280a246a1ef62d626d67b44064/pillow-12.1.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:d0a7735df32ccbcc98b98a1ac785cc4b19b580be1bdf0aeb5c03223220ea09d5", size = 8042708 }, + { url = "https://files.pythonhosted.org/packages/1a/eb/257f38542893f021502a1bbe0c2e883c90b5cff26cc33b1584a841a06d30/pillow-12.1.0-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0c27407a2d1b96774cbc4a7594129cc027339fd800cd081e44497722ea1179de", size = 6347762 }, + { url = "https://files.pythonhosted.org/packages/c4/5a/8ba375025701c09b309e8d5163c5a4ce0102fa86bbf8800eb0d7ac87bc51/pillow-12.1.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:15c794d74303828eaa957ff8070846d0efe8c630901a1c753fdc63850e19ecd9", size = 7039265 }, + { url = "https://files.pythonhosted.org/packages/cf/dc/cf5e4cdb3db533f539e88a7bbf9f190c64ab8a08a9bc7a4ccf55067872e4/pillow-12.1.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:c990547452ee2800d8506c4150280757f88532f3de2a58e3022e9b179107862a", size = 6462341 }, + { url = "https://files.pythonhosted.org/packages/d0/47/0291a25ac9550677e22eda48510cfc4fa4b2ef0396448b7fbdc0a6946309/pillow-12.1.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:b63e13dd27da389ed9475b3d28510f0f954bca0041e8e551b2a4eb1eab56a39a", size = 7165395 }, + { url = "https://files.pythonhosted.org/packages/4f/4c/e005a59393ec4d9416be06e6b45820403bb946a778e39ecec62f5b2b991e/pillow-12.1.0-cp314-cp314-win32.whl", hash = "sha256:1a949604f73eb07a8adab38c4fe50791f9919344398bdc8ac6b307f755fc7030", size = 6431413 }, + { url = "https://files.pythonhosted.org/packages/1c/af/f23697f587ac5f9095d67e31b81c95c0249cd461a9798a061ed6709b09b5/pillow-12.1.0-cp314-cp314-win_amd64.whl", hash = "sha256:4f9f6a650743f0ddee5593ac9e954ba1bdbc5e150bc066586d4f26127853ab94", size = 7176779 }, + { url = "https://files.pythonhosted.org/packages/b3/36/6a51abf8599232f3e9afbd16d52829376a68909fe14efe29084445db4b73/pillow-12.1.0-cp314-cp314-win_arm64.whl", hash = "sha256:808b99604f7873c800c4840f55ff389936ef1948e4e87645eaf3fccbc8477ac4", size = 2543105 }, + { url = "https://files.pythonhosted.org/packages/82/54/2e1dd20c8749ff225080d6ba465a0cab4387f5db0d1c5fb1439e2d99923f/pillow-12.1.0-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:bc11908616c8a283cf7d664f77411a5ed2a02009b0097ff8abbba5e79128ccf2", size = 5268571 }, + { url = "https://files.pythonhosted.org/packages/57/61/571163a5ef86ec0cf30d265ac2a70ae6fc9e28413d1dc94fa37fae6bda89/pillow-12.1.0-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:896866d2d436563fa2a43a9d72f417874f16b5545955c54a64941e87c1376c61", size = 4660426 }, + { url = "https://files.pythonhosted.org/packages/5e/e1/53ee5163f794aef1bf84243f755ee6897a92c708505350dd1923f4afec48/pillow-12.1.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:8e178e3e99d3c0ea8fc64b88447f7cac8ccf058af422a6cedc690d0eadd98c51", size = 6269908 }, + { url = "https://files.pythonhosted.org/packages/bc/0b/b4b4106ff0ee1afa1dc599fde6ab230417f800279745124f6c50bcffed8e/pillow-12.1.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:079af2fb0c599c2ec144ba2c02766d1b55498e373b3ac64687e43849fbbef5bc", size = 8074733 }, + { url = "https://files.pythonhosted.org/packages/19/9f/80b411cbac4a732439e629a26ad3ef11907a8c7fc5377b7602f04f6fe4e7/pillow-12.1.0-cp314-cp314t-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:bdec5e43377761c5dbca620efb69a77f6855c5a379e32ac5b158f54c84212b14", size = 6381431 }, + { url = "https://files.pythonhosted.org/packages/8f/b7/d65c45db463b66ecb6abc17c6ba6917a911202a07662247e1355ce1789e7/pillow-12.1.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:565c986f4b45c020f5421a4cea13ef294dde9509a8577f29b2fc5edc7587fff8", size = 7068529 }, + { url = "https://files.pythonhosted.org/packages/50/96/dfd4cd726b4a45ae6e3c669fc9e49deb2241312605d33aba50499e9d9bd1/pillow-12.1.0-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:43aca0a55ce1eefc0aefa6253661cb54571857b1a7b2964bd8a1e3ef4b729924", size = 6492981 }, + { url = "https://files.pythonhosted.org/packages/4d/1c/b5dc52cf713ae46033359c5ca920444f18a6359ce1020dd3e9c553ea5bc6/pillow-12.1.0-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:0deedf2ea233722476b3a81e8cdfbad786f7adbed5d848469fa59fe52396e4ef", size = 7191878 }, + { url = "https://files.pythonhosted.org/packages/53/26/c4188248bd5edaf543864fe4834aebe9c9cb4968b6f573ce014cc42d0720/pillow-12.1.0-cp314-cp314t-win32.whl", hash = "sha256:b17fbdbe01c196e7e159aacb889e091f28e61020a8abeac07b68079b6e626988", size = 6438703 }, + { url = "https://files.pythonhosted.org/packages/b8/0e/69ed296de8ea05cb03ee139cee600f424ca166e632567b2d66727f08c7ed/pillow-12.1.0-cp314-cp314t-win_amd64.whl", hash = "sha256:27b9baecb428899db6c0de572d6d305cfaf38ca1596b5c0542a5182e3e74e8c6", size = 7182927 }, + { url = "https://files.pythonhosted.org/packages/fc/f5/68334c015eed9b5cff77814258717dec591ded209ab5b6fb70e2ae873d1d/pillow-12.1.0-cp314-cp314t-win_arm64.whl", hash = "sha256:f61333d817698bdcdd0f9d7793e365ac3d2a21c1f1eb02b32ad6aefb8d8ea831", size = 2545104 }, + { url = "https://files.pythonhosted.org/packages/8b/bc/224b1d98cffd7164b14707c91aac83c07b047fbd8f58eba4066a3e53746a/pillow-12.1.0-pp311-pypy311_pp73-macosx_10_15_x86_64.whl", hash = "sha256:ca94b6aac0d7af2a10ba08c0f888b3d5114439b6b3ef39968378723622fed377", size = 5228605 }, + { url = "https://files.pythonhosted.org/packages/0c/ca/49ca7769c4550107de049ed85208240ba0f330b3f2e316f24534795702ce/pillow-12.1.0-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:351889afef0f485b84078ea40fe33727a0492b9af3904661b0abbafee0355b72", size = 4622245 }, + { url = "https://files.pythonhosted.org/packages/73/48/fac807ce82e5955bcc2718642b94b1bd22a82a6d452aea31cbb678cddf12/pillow-12.1.0-pp311-pypy311_pp73-manylinux2014_aarch64.manylinux_2_17_aarch64.whl", hash = "sha256:bb0984b30e973f7e2884362b7d23d0a348c7143ee559f38ef3eaab640144204c", size = 5247593 }, + { url = "https://files.pythonhosted.org/packages/d2/95/3e0742fe358c4664aed4fd05d5f5373dcdad0b27af52aa0972568541e3f4/pillow-12.1.0-pp311-pypy311_pp73-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:84cabc7095dd535ca934d57e9ce2a72ffd216e435a84acb06b2277b1de2689bd", size = 6989008 }, + { url = "https://files.pythonhosted.org/packages/5a/74/fe2ac378e4e202e56d50540d92e1ef4ff34ed687f3c60f6a121bcf99437e/pillow-12.1.0-pp311-pypy311_pp73-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:53d8b764726d3af1a138dd353116f774e3862ec7e3794e0c8781e30db0f35dfc", size = 5313824 }, + { url = "https://files.pythonhosted.org/packages/f3/77/2a60dee1adee4e2655ac328dd05c02a955c1cd683b9f1b82ec3feb44727c/pillow-12.1.0-pp311-pypy311_pp73-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5da841d81b1a05ef940a8567da92decaa15bc4d7dedb540a8c219ad83d91808a", size = 5963278 }, + { url = "https://files.pythonhosted.org/packages/2d/71/64e9b1c7f04ae0027f788a248e6297d7fcc29571371fe7d45495a78172c0/pillow-12.1.0-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:75af0b4c229ac519b155028fa1be632d812a519abba9b46b20e50c6caa184f19", size = 7029809 }, +] + +[[package]] +name = "playwright" +version = "1.57.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "greenlet" }, + { name = "pyee" }, +] +wheels = [ + { url = "https://files.pythonhosted.org/packages/ed/b6/e17543cea8290ae4dced10be21d5a43c360096aa2cce0aa7039e60c50df3/playwright-1.57.0-py3-none-macosx_10_13_x86_64.whl", hash = "sha256:9351c1ac3dfd9b3820fe7fc4340d96c0d3736bb68097b9b7a69bd45d25e9370c", size = 41985039 }, + { url = "https://files.pythonhosted.org/packages/8b/04/ef95b67e1ff59c080b2effd1a9a96984d6953f667c91dfe9d77c838fc956/playwright-1.57.0-py3-none-macosx_11_0_arm64.whl", hash = "sha256:a4a9d65027bce48eeba842408bcc1421502dfd7e41e28d207e94260fa93ca67e", size = 40775575 }, + { url = "https://files.pythonhosted.org/packages/60/bd/5563850322a663956c927eefcf1457d12917e8f118c214410e815f2147d1/playwright-1.57.0-py3-none-macosx_11_0_universal2.whl", hash = "sha256:99104771abc4eafee48f47dac2369e0015516dc1ce8c409807d2dd440828b9a4", size = 41985042 }, + { url = "https://files.pythonhosted.org/packages/56/61/3a803cb5ae0321715bfd5247ea871d25b32c8f372aeb70550a90c5f586df/playwright-1.57.0-py3-none-manylinux1_x86_64.whl", hash = "sha256:284ed5a706b7c389a06caa431b2f0ba9ac4130113c3a779767dda758c2497bb1", size = 45975252 }, + { url = "https://files.pythonhosted.org/packages/83/d7/b72eb59dfbea0013a7f9731878df8c670f5f35318cedb010c8a30292c118/playwright-1.57.0-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:38a1bae6c0a07839cdeaddbc0756b3b2b85e476c07945f64ece08f1f956a86f1", size = 45706917 }, + { url = "https://files.pythonhosted.org/packages/e4/09/3fc9ebd7c95ee54ba6a68d5c0bc23e449f7235f4603fc60534a364934c16/playwright-1.57.0-py3-none-win32.whl", hash = "sha256:1dd93b265688da46e91ecb0606d36f777f8eadcf7fbef12f6426b20bf0c9137c", size = 36553860 }, + { url = "https://files.pythonhosted.org/packages/58/d4/dcdfd2a33096aeda6ca0d15584800443dd2be64becca8f315634044b135b/playwright-1.57.0-py3-none-win_amd64.whl", hash = "sha256:6caefb08ed2c6f29d33b8088d05d09376946e49a73be19271c8cd5384b82b14c", size = 36553864 }, + { url = "https://files.pythonhosted.org/packages/6a/60/fe31d7e6b8907789dcb0584f88be741ba388413e4fbce35f1eba4e3073de/playwright-1.57.0-py3-none-win_arm64.whl", hash = "sha256:5f065f5a133dbc15e6e7c71e7bc04f258195755b1c32a432b792e28338c8335e", size = 32837940 }, +] + +[[package]] +name = "prettytable" +version = "3.17.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "wcwidth" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/79/45/b0847d88d6cfeb4413566738c8bbf1e1995fad3d42515327ff32cc1eb578/prettytable-3.17.0.tar.gz", hash = "sha256:59f2590776527f3c9e8cf9fe7b66dd215837cca96a9c39567414cbc632e8ddb0", size = 67892 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/ee/8c/83087ebc47ab0396ce092363001fa37c17153119ee282700c0713a195853/prettytable-3.17.0-py3-none-any.whl", hash = "sha256:aad69b294ddbe3e1f95ef8886a060ed1666a0b83018bbf56295f6f226c43d287", size = 34433 }, +] + +[[package]] +name = "protobuf" +version = "6.33.4" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/53/b8/cda15d9d46d03d4aa3a67cb6bffe05173440ccf86a9541afaf7ac59a1b6b/protobuf-6.33.4.tar.gz", hash = "sha256:dc2e61bca3b10470c1912d166fe0af67bfc20eb55971dcef8dfa48ce14f0ed91", size = 444346 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e0/be/24ef9f3095bacdf95b458543334d0c4908ccdaee5130420bf064492c325f/protobuf-6.33.4-cp310-abi3-win32.whl", hash = "sha256:918966612c8232fc6c24c78e1cd89784307f5814ad7506c308ee3cf86662850d", size = 425612 }, + { url = "https://files.pythonhosted.org/packages/31/ad/e5693e1974a28869e7cd244302911955c1cebc0161eb32dfa2b25b6e96f0/protobuf-6.33.4-cp310-abi3-win_amd64.whl", hash = "sha256:8f11ffae31ec67fc2554c2ef891dcb561dae9a2a3ed941f9e134c2db06657dbc", size = 436962 }, + { url = "https://files.pythonhosted.org/packages/66/15/6ee23553b6bfd82670207ead921f4d8ef14c107e5e11443b04caeb5ab5ec/protobuf-6.33.4-cp39-abi3-macosx_10_9_universal2.whl", hash = "sha256:2fe67f6c014c84f655ee06f6f66213f9254b3a8b6bda6cda0ccd4232c73c06f0", size = 427612 }, + { url = "https://files.pythonhosted.org/packages/2b/48/d301907ce6d0db75f959ca74f44b475a9caa8fcba102d098d3c3dd0f2d3f/protobuf-6.33.4-cp39-abi3-manylinux2014_aarch64.whl", hash = "sha256:757c978f82e74d75cba88eddec479df9b99a42b31193313b75e492c06a51764e", size = 324484 }, + { url = "https://files.pythonhosted.org/packages/92/1c/e53078d3f7fe710572ab2dcffd993e1e3b438ae71cfc031b71bae44fcb2d/protobuf-6.33.4-cp39-abi3-manylinux2014_s390x.whl", hash = "sha256:c7c64f259c618f0bef7bee042075e390debbf9682334be2b67408ec7c1c09ee6", size = 339256 }, + { url = "https://files.pythonhosted.org/packages/e8/8e/971c0edd084914f7ee7c23aa70ba89e8903918adca179319ee94403701d5/protobuf-6.33.4-cp39-abi3-manylinux2014_x86_64.whl", hash = "sha256:3df850c2f8db9934de4cf8f9152f8dc2558f49f298f37f90c517e8e5c84c30e9", size = 323311 }, + { url = "https://files.pythonhosted.org/packages/75/b1/1dc83c2c661b4c62d56cc081706ee33a4fc2835bd90f965baa2663ef7676/protobuf-6.33.4-py3-none-any.whl", hash = "sha256:1fe3730068fcf2e595816a6c34fe66eeedd37d51d0400b72fabc848811fdc1bc", size = 170532 }, +] + +[[package]] +name = "pycparser" +version = "2.23" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/fe/cf/d2d3b9f5699fb1e4615c8e32ff220203e43b248e1dfcc6736ad9057731ca/pycparser-2.23.tar.gz", hash = "sha256:78816d4f24add8f10a06d6f05b4d424ad9e96cfebf68a4ddc99c65c0720d00c2", size = 173734 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a0/e3/59cd50310fc9b59512193629e1984c1f95e5c8ae6e5d8c69532ccc65a7fe/pycparser-2.23-py3-none-any.whl", hash = "sha256:e5c6e8d3fbad53479cab09ac03729e0a9faf2bee3db8208a550daf5af81a5934", size = 118140 }, +] + +[[package]] +name = "pydantic" +version = "2.12.5" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "annotated-types" }, + { name = "pydantic-core" }, + { name = "typing-extensions" }, + { name = "typing-inspection" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/69/44/36f1a6e523abc58ae5f928898e4aca2e0ea509b5aa6f6f392a5d882be928/pydantic-2.12.5.tar.gz", hash = "sha256:4d351024c75c0f085a9febbb665ce8c0c6ec5d30e903bdb6394b7ede26aebb49", size = 821591 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/5a/87/b70ad306ebb6f9b585f114d0ac2137d792b48be34d732d60e597c2f8465a/pydantic-2.12.5-py3-none-any.whl", hash = "sha256:e561593fccf61e8a20fc46dfc2dfe075b8be7d0188df33f221ad1f0139180f9d", size = 463580 }, +] + +[[package]] +name = "pydantic-core" +version = "2.41.5" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/71/70/23b021c950c2addd24ec408e9ab05d59b035b39d97cdc1130e1bce647bb6/pydantic_core-2.41.5.tar.gz", hash = "sha256:08daa51ea16ad373ffd5e7606252cc32f07bc72b28284b6bc9c6df804816476e", size = 460952 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e8/72/74a989dd9f2084b3d9530b0915fdda64ac48831c30dbf7c72a41a5232db8/pydantic_core-2.41.5-cp311-cp311-macosx_10_12_x86_64.whl", hash = "sha256:a3a52f6156e73e7ccb0f8cced536adccb7042be67cb45f9562e12b319c119da6", size = 2105873 }, + { url = "https://files.pythonhosted.org/packages/12/44/37e403fd9455708b3b942949e1d7febc02167662bf1a7da5b78ee1ea2842/pydantic_core-2.41.5-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:7f3bf998340c6d4b0c9a2f02d6a400e51f123b59565d74dc60d252ce888c260b", size = 1899826 }, + { url = "https://files.pythonhosted.org/packages/33/7f/1d5cab3ccf44c1935a359d51a8a2a9e1a654b744b5e7f80d41b88d501eec/pydantic_core-2.41.5-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:378bec5c66998815d224c9ca994f1e14c0c21cb95d2f52b6021cc0b2a58f2a5a", size = 1917869 }, + { url = "https://files.pythonhosted.org/packages/6e/6a/30d94a9674a7fe4f4744052ed6c5e083424510be1e93da5bc47569d11810/pydantic_core-2.41.5-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:e7b576130c69225432866fe2f4a469a85a54ade141d96fd396dffcf607b558f8", size = 2063890 }, + { url = "https://files.pythonhosted.org/packages/50/be/76e5d46203fcb2750e542f32e6c371ffa9b8ad17364cf94bb0818dbfb50c/pydantic_core-2.41.5-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:6cb58b9c66f7e4179a2d5e0f849c48eff5c1fca560994d6eb6543abf955a149e", size = 2229740 }, + { url = "https://files.pythonhosted.org/packages/d3/ee/fed784df0144793489f87db310a6bbf8118d7b630ed07aa180d6067e653a/pydantic_core-2.41.5-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:88942d3a3dff3afc8288c21e565e476fc278902ae4d6d134f1eeda118cc830b1", size = 2350021 }, + { url = "https://files.pythonhosted.org/packages/c8/be/8fed28dd0a180dca19e72c233cbf58efa36df055e5b9d90d64fd1740b828/pydantic_core-2.41.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f31d95a179f8d64d90f6831d71fa93290893a33148d890ba15de25642c5d075b", size = 2066378 }, + { url = "https://files.pythonhosted.org/packages/b0/3b/698cf8ae1d536a010e05121b4958b1257f0b5522085e335360e53a6b1c8b/pydantic_core-2.41.5-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:c1df3d34aced70add6f867a8cf413e299177e0c22660cc767218373d0779487b", size = 2175761 }, + { url = "https://files.pythonhosted.org/packages/b8/ba/15d537423939553116dea94ce02f9c31be0fa9d0b806d427e0308ec17145/pydantic_core-2.41.5-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:4009935984bd36bd2c774e13f9a09563ce8de4abaa7226f5108262fa3e637284", size = 2146303 }, + { url = "https://files.pythonhosted.org/packages/58/7f/0de669bf37d206723795f9c90c82966726a2ab06c336deba4735b55af431/pydantic_core-2.41.5-cp311-cp311-musllinux_1_1_armv7l.whl", hash = "sha256:34a64bc3441dc1213096a20fe27e8e128bd3ff89921706e83c0b1ac971276594", size = 2340355 }, + { url = "https://files.pythonhosted.org/packages/e5/de/e7482c435b83d7e3c3ee5ee4451f6e8973cff0eb6007d2872ce6383f6398/pydantic_core-2.41.5-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:c9e19dd6e28fdcaa5a1de679aec4141f691023916427ef9bae8584f9c2fb3b0e", size = 2319875 }, + { url = "https://files.pythonhosted.org/packages/fe/e6/8c9e81bb6dd7560e33b9053351c29f30c8194b72f2d6932888581f503482/pydantic_core-2.41.5-cp311-cp311-win32.whl", hash = "sha256:2c010c6ded393148374c0f6f0bf89d206bf3217f201faa0635dcd56bd1520f6b", size = 1987549 }, + { url = "https://files.pythonhosted.org/packages/11/66/f14d1d978ea94d1bc21fc98fcf570f9542fe55bfcc40269d4e1a21c19bf7/pydantic_core-2.41.5-cp311-cp311-win_amd64.whl", hash = "sha256:76ee27c6e9c7f16f47db7a94157112a2f3a00e958bc626e2f4ee8bec5c328fbe", size = 2011305 }, + { url = "https://files.pythonhosted.org/packages/56/d8/0e271434e8efd03186c5386671328154ee349ff0354d83c74f5caaf096ed/pydantic_core-2.41.5-cp311-cp311-win_arm64.whl", hash = "sha256:4bc36bbc0b7584de96561184ad7f012478987882ebf9f9c389b23f432ea3d90f", size = 1972902 }, + { url = "https://files.pythonhosted.org/packages/5f/5d/5f6c63eebb5afee93bcaae4ce9a898f3373ca23df3ccaef086d0233a35a7/pydantic_core-2.41.5-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:f41a7489d32336dbf2199c8c0a215390a751c5b014c2c1c5366e817202e9cdf7", size = 2110990 }, + { url = "https://files.pythonhosted.org/packages/aa/32/9c2e8ccb57c01111e0fd091f236c7b371c1bccea0fa85247ac55b1e2b6b6/pydantic_core-2.41.5-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:070259a8818988b9a84a449a2a7337c7f430a22acc0859c6b110aa7212a6d9c0", size = 1896003 }, + { url = "https://files.pythonhosted.org/packages/68/b8/a01b53cb0e59139fbc9e4fda3e9724ede8de279097179be4ff31f1abb65a/pydantic_core-2.41.5-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e96cea19e34778f8d59fe40775a7a574d95816eb150850a85a7a4c8f4b94ac69", size = 1919200 }, + { url = "https://files.pythonhosted.org/packages/38/de/8c36b5198a29bdaade07b5985e80a233a5ac27137846f3bc2d3b40a47360/pydantic_core-2.41.5-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:ed2e99c456e3fadd05c991f8f437ef902e00eedf34320ba2b0842bd1c3ca3a75", size = 2052578 }, + { url = "https://files.pythonhosted.org/packages/00/b5/0e8e4b5b081eac6cb3dbb7e60a65907549a1ce035a724368c330112adfdd/pydantic_core-2.41.5-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:65840751b72fbfd82c3c640cff9284545342a4f1eb1586ad0636955b261b0b05", size = 2208504 }, + { url = "https://files.pythonhosted.org/packages/77/56/87a61aad59c7c5b9dc8caad5a41a5545cba3810c3e828708b3d7404f6cef/pydantic_core-2.41.5-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:e536c98a7626a98feb2d3eaf75944ef6f3dbee447e1f841eae16f2f0a72d8ddc", size = 2335816 }, + { url = "https://files.pythonhosted.org/packages/0d/76/941cc9f73529988688a665a5c0ecff1112b3d95ab48f81db5f7606f522d3/pydantic_core-2.41.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:eceb81a8d74f9267ef4081e246ffd6d129da5d87e37a77c9bde550cb04870c1c", size = 2075366 }, + { url = "https://files.pythonhosted.org/packages/d3/43/ebef01f69baa07a482844faaa0a591bad1ef129253ffd0cdaa9d8a7f72d3/pydantic_core-2.41.5-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:d38548150c39b74aeeb0ce8ee1d8e82696f4a4e16ddc6de7b1d8823f7de4b9b5", size = 2171698 }, + { url = "https://files.pythonhosted.org/packages/b1/87/41f3202e4193e3bacfc2c065fab7706ebe81af46a83d3e27605029c1f5a6/pydantic_core-2.41.5-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:c23e27686783f60290e36827f9c626e63154b82b116d7fe9adba1fda36da706c", size = 2132603 }, + { url = "https://files.pythonhosted.org/packages/49/7d/4c00df99cb12070b6bccdef4a195255e6020a550d572768d92cc54dba91a/pydantic_core-2.41.5-cp312-cp312-musllinux_1_1_armv7l.whl", hash = "sha256:482c982f814460eabe1d3bb0adfdc583387bd4691ef00b90575ca0d2b6fe2294", size = 2329591 }, + { url = "https://files.pythonhosted.org/packages/cc/6a/ebf4b1d65d458f3cda6a7335d141305dfa19bdc61140a884d165a8a1bbc7/pydantic_core-2.41.5-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:bfea2a5f0b4d8d43adf9d7b8bf019fb46fdd10a2e5cde477fbcb9d1fa08c68e1", size = 2319068 }, + { url = "https://files.pythonhosted.org/packages/49/3b/774f2b5cd4192d5ab75870ce4381fd89cf218af999515baf07e7206753f0/pydantic_core-2.41.5-cp312-cp312-win32.whl", hash = "sha256:b74557b16e390ec12dca509bce9264c3bbd128f8a2c376eaa68003d7f327276d", size = 1985908 }, + { url = "https://files.pythonhosted.org/packages/86/45/00173a033c801cacf67c190fef088789394feaf88a98a7035b0e40d53dc9/pydantic_core-2.41.5-cp312-cp312-win_amd64.whl", hash = "sha256:1962293292865bca8e54702b08a4f26da73adc83dd1fcf26fbc875b35d81c815", size = 2020145 }, + { url = "https://files.pythonhosted.org/packages/f9/22/91fbc821fa6d261b376a3f73809f907cec5ca6025642c463d3488aad22fb/pydantic_core-2.41.5-cp312-cp312-win_arm64.whl", hash = "sha256:1746d4a3d9a794cacae06a5eaaccb4b8643a131d45fbc9af23e353dc0a5ba5c3", size = 1976179 }, + { url = "https://files.pythonhosted.org/packages/87/06/8806241ff1f70d9939f9af039c6c35f2360cf16e93c2ca76f184e76b1564/pydantic_core-2.41.5-cp313-cp313-macosx_10_12_x86_64.whl", hash = "sha256:941103c9be18ac8daf7b7adca8228f8ed6bb7a1849020f643b3a14d15b1924d9", size = 2120403 }, + { url = "https://files.pythonhosted.org/packages/94/02/abfa0e0bda67faa65fef1c84971c7e45928e108fe24333c81f3bfe35d5f5/pydantic_core-2.41.5-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:112e305c3314f40c93998e567879e887a3160bb8689ef3d2c04b6cc62c33ac34", size = 1896206 }, + { url = "https://files.pythonhosted.org/packages/15/df/a4c740c0943e93e6500f9eb23f4ca7ec9bf71b19e608ae5b579678c8d02f/pydantic_core-2.41.5-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:0cbaad15cb0c90aa221d43c00e77bb33c93e8d36e0bf74760cd00e732d10a6a0", size = 1919307 }, + { url = "https://files.pythonhosted.org/packages/9a/e3/6324802931ae1d123528988e0e86587c2072ac2e5394b4bc2bc34b61ff6e/pydantic_core-2.41.5-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:03ca43e12fab6023fc79d28ca6b39b05f794ad08ec2feccc59a339b02f2b3d33", size = 2063258 }, + { url = "https://files.pythonhosted.org/packages/c9/d4/2230d7151d4957dd79c3044ea26346c148c98fbf0ee6ebd41056f2d62ab5/pydantic_core-2.41.5-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:dc799088c08fa04e43144b164feb0c13f9a0bc40503f8df3e9fde58a3c0c101e", size = 2214917 }, + { url = "https://files.pythonhosted.org/packages/e6/9f/eaac5df17a3672fef0081b6c1bb0b82b33ee89aa5cec0d7b05f52fd4a1fa/pydantic_core-2.41.5-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:97aeba56665b4c3235a0e52b2c2f5ae9cd071b8a8310ad27bddb3f7fb30e9aa2", size = 2332186 }, + { url = "https://files.pythonhosted.org/packages/cf/4e/35a80cae583a37cf15604b44240e45c05e04e86f9cfd766623149297e971/pydantic_core-2.41.5-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:406bf18d345822d6c21366031003612b9c77b3e29ffdb0f612367352aab7d586", size = 2073164 }, + { url = "https://files.pythonhosted.org/packages/bf/e3/f6e262673c6140dd3305d144d032f7bd5f7497d3871c1428521f19f9efa2/pydantic_core-2.41.5-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:b93590ae81f7010dbe380cdeab6f515902ebcbefe0b9327cc4804d74e93ae69d", size = 2179146 }, + { url = "https://files.pythonhosted.org/packages/75/c7/20bd7fc05f0c6ea2056a4565c6f36f8968c0924f19b7d97bbfea55780e73/pydantic_core-2.41.5-cp313-cp313-musllinux_1_1_aarch64.whl", hash = "sha256:01a3d0ab748ee531f4ea6c3e48ad9dac84ddba4b0d82291f87248f2f9de8d740", size = 2137788 }, + { url = "https://files.pythonhosted.org/packages/3a/8d/34318ef985c45196e004bc46c6eab2eda437e744c124ef0dbe1ff2c9d06b/pydantic_core-2.41.5-cp313-cp313-musllinux_1_1_armv7l.whl", hash = "sha256:6561e94ba9dacc9c61bce40e2d6bdc3bfaa0259d3ff36ace3b1e6901936d2e3e", size = 2340133 }, + { url = "https://files.pythonhosted.org/packages/9c/59/013626bf8c78a5a5d9350d12e7697d3d4de951a75565496abd40ccd46bee/pydantic_core-2.41.5-cp313-cp313-musllinux_1_1_x86_64.whl", hash = "sha256:915c3d10f81bec3a74fbd4faebe8391013ba61e5a1a8d48c4455b923bdda7858", size = 2324852 }, + { url = "https://files.pythonhosted.org/packages/1a/d9/c248c103856f807ef70c18a4f986693a46a8ffe1602e5d361485da502d20/pydantic_core-2.41.5-cp313-cp313-win32.whl", hash = "sha256:650ae77860b45cfa6e2cdafc42618ceafab3a2d9a3811fcfbd3bbf8ac3c40d36", size = 1994679 }, + { url = "https://files.pythonhosted.org/packages/9e/8b/341991b158ddab181cff136acd2552c9f35bd30380422a639c0671e99a91/pydantic_core-2.41.5-cp313-cp313-win_amd64.whl", hash = "sha256:79ec52ec461e99e13791ec6508c722742ad745571f234ea6255bed38c6480f11", size = 2019766 }, + { url = "https://files.pythonhosted.org/packages/73/7d/f2f9db34af103bea3e09735bb40b021788a5e834c81eedb541991badf8f5/pydantic_core-2.41.5-cp313-cp313-win_arm64.whl", hash = "sha256:3f84d5c1b4ab906093bdc1ff10484838aca54ef08de4afa9de0f5f14d69639cd", size = 1981005 }, + { url = "https://files.pythonhosted.org/packages/ea/28/46b7c5c9635ae96ea0fbb779e271a38129df2550f763937659ee6c5dbc65/pydantic_core-2.41.5-cp314-cp314-macosx_10_12_x86_64.whl", hash = "sha256:3f37a19d7ebcdd20b96485056ba9e8b304e27d9904d233d7b1015db320e51f0a", size = 2119622 }, + { url = "https://files.pythonhosted.org/packages/74/1a/145646e5687e8d9a1e8d09acb278c8535ebe9e972e1f162ed338a622f193/pydantic_core-2.41.5-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:1d1d9764366c73f996edd17abb6d9d7649a7eb690006ab6adbda117717099b14", size = 1891725 }, + { url = "https://files.pythonhosted.org/packages/23/04/e89c29e267b8060b40dca97bfc64a19b2a3cf99018167ea1677d96368273/pydantic_core-2.41.5-cp314-cp314-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:25e1c2af0fce638d5f1988b686f3b3ea8cd7de5f244ca147c777769e798a9cd1", size = 1915040 }, + { url = "https://files.pythonhosted.org/packages/84/a3/15a82ac7bd97992a82257f777b3583d3e84bdb06ba6858f745daa2ec8a85/pydantic_core-2.41.5-cp314-cp314-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:506d766a8727beef16b7adaeb8ee6217c64fc813646b424d0804d67c16eddb66", size = 2063691 }, + { url = "https://files.pythonhosted.org/packages/74/9b/0046701313c6ef08c0c1cf0e028c67c770a4e1275ca73131563c5f2a310a/pydantic_core-2.41.5-cp314-cp314-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:4819fa52133c9aa3c387b3328f25c1facc356491e6135b459f1de698ff64d869", size = 2213897 }, + { url = "https://files.pythonhosted.org/packages/8a/cd/6bac76ecd1b27e75a95ca3a9a559c643b3afcd2dd62086d4b7a32a18b169/pydantic_core-2.41.5-cp314-cp314-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:2b761d210c9ea91feda40d25b4efe82a1707da2ef62901466a42492c028553a2", size = 2333302 }, + { url = "https://files.pythonhosted.org/packages/4c/d2/ef2074dc020dd6e109611a8be4449b98cd25e1b9b8a303c2f0fca2f2bcf7/pydantic_core-2.41.5-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:22f0fb8c1c583a3b6f24df2470833b40207e907b90c928cc8d3594b76f874375", size = 2064877 }, + { url = "https://files.pythonhosted.org/packages/18/66/e9db17a9a763d72f03de903883c057b2592c09509ccfe468187f2a2eef29/pydantic_core-2.41.5-cp314-cp314-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:2782c870e99878c634505236d81e5443092fba820f0373997ff75f90f68cd553", size = 2180680 }, + { url = "https://files.pythonhosted.org/packages/d3/9e/3ce66cebb929f3ced22be85d4c2399b8e85b622db77dad36b73c5387f8f8/pydantic_core-2.41.5-cp314-cp314-musllinux_1_1_aarch64.whl", hash = "sha256:0177272f88ab8312479336e1d777f6b124537d47f2123f89cb37e0accea97f90", size = 2138960 }, + { url = "https://files.pythonhosted.org/packages/a6/62/205a998f4327d2079326b01abee48e502ea739d174f0a89295c481a2272e/pydantic_core-2.41.5-cp314-cp314-musllinux_1_1_armv7l.whl", hash = "sha256:63510af5e38f8955b8ee5687740d6ebf7c2a0886d15a6d65c32814613681bc07", size = 2339102 }, + { url = "https://files.pythonhosted.org/packages/3c/0d/f05e79471e889d74d3d88f5bd20d0ed189ad94c2423d81ff8d0000aab4ff/pydantic_core-2.41.5-cp314-cp314-musllinux_1_1_x86_64.whl", hash = "sha256:e56ba91f47764cc14f1daacd723e3e82d1a89d783f0f5afe9c364b8bb491ccdb", size = 2326039 }, + { url = "https://files.pythonhosted.org/packages/ec/e1/e08a6208bb100da7e0c4b288eed624a703f4d129bde2da475721a80cab32/pydantic_core-2.41.5-cp314-cp314-win32.whl", hash = "sha256:aec5cf2fd867b4ff45b9959f8b20ea3993fc93e63c7363fe6851424c8a7e7c23", size = 1995126 }, + { url = "https://files.pythonhosted.org/packages/48/5d/56ba7b24e9557f99c9237e29f5c09913c81eeb2f3217e40e922353668092/pydantic_core-2.41.5-cp314-cp314-win_amd64.whl", hash = "sha256:8e7c86f27c585ef37c35e56a96363ab8de4e549a95512445b85c96d3e2f7c1bf", size = 2015489 }, + { url = "https://files.pythonhosted.org/packages/4e/bb/f7a190991ec9e3e0ba22e4993d8755bbc4a32925c0b5b42775c03e8148f9/pydantic_core-2.41.5-cp314-cp314-win_arm64.whl", hash = "sha256:e672ba74fbc2dc8eea59fb6d4aed6845e6905fc2a8afe93175d94a83ba2a01a0", size = 1977288 }, + { url = "https://files.pythonhosted.org/packages/92/ed/77542d0c51538e32e15afe7899d79efce4b81eee631d99850edc2f5e9349/pydantic_core-2.41.5-cp314-cp314t-macosx_10_12_x86_64.whl", hash = "sha256:8566def80554c3faa0e65ac30ab0932b9e3a5cd7f8323764303d468e5c37595a", size = 2120255 }, + { url = "https://files.pythonhosted.org/packages/bb/3d/6913dde84d5be21e284439676168b28d8bbba5600d838b9dca99de0fad71/pydantic_core-2.41.5-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:b80aa5095cd3109962a298ce14110ae16b8c1aece8b72f9dafe81cf597ad80b3", size = 1863760 }, + { url = "https://files.pythonhosted.org/packages/5a/f0/e5e6b99d4191da102f2b0eb9687aaa7f5bea5d9964071a84effc3e40f997/pydantic_core-2.41.5-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:3006c3dd9ba34b0c094c544c6006cc79e87d8612999f1a5d43b769b89181f23c", size = 1878092 }, + { url = "https://files.pythonhosted.org/packages/71/48/36fb760642d568925953bcc8116455513d6e34c4beaa37544118c36aba6d/pydantic_core-2.41.5-cp314-cp314t-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:72f6c8b11857a856bcfa48c86f5368439f74453563f951e473514579d44aa612", size = 2053385 }, + { url = "https://files.pythonhosted.org/packages/20/25/92dc684dd8eb75a234bc1c764b4210cf2646479d54b47bf46061657292a8/pydantic_core-2.41.5-cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:5cb1b2f9742240e4bb26b652a5aeb840aa4b417c7748b6f8387927bc6e45e40d", size = 2218832 }, + { url = "https://files.pythonhosted.org/packages/e2/09/f53e0b05023d3e30357d82eb35835d0f6340ca344720a4599cd663dca599/pydantic_core-2.41.5-cp314-cp314t-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:bd3d54f38609ff308209bd43acea66061494157703364ae40c951f83ba99a1a9", size = 2327585 }, + { url = "https://files.pythonhosted.org/packages/aa/4e/2ae1aa85d6af35a39b236b1b1641de73f5a6ac4d5a7509f77b814885760c/pydantic_core-2.41.5-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:2ff4321e56e879ee8d2a879501c8e469414d948f4aba74a2d4593184eb326660", size = 2041078 }, + { url = "https://files.pythonhosted.org/packages/cd/13/2e215f17f0ef326fc72afe94776edb77525142c693767fc347ed6288728d/pydantic_core-2.41.5-cp314-cp314t-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:d0d2568a8c11bf8225044aa94409e21da0cb09dcdafe9ecd10250b2baad531a9", size = 2173914 }, + { url = "https://files.pythonhosted.org/packages/02/7a/f999a6dcbcd0e5660bc348a3991c8915ce6599f4f2c6ac22f01d7a10816c/pydantic_core-2.41.5-cp314-cp314t-musllinux_1_1_aarch64.whl", hash = "sha256:a39455728aabd58ceabb03c90e12f71fd30fa69615760a075b9fec596456ccc3", size = 2129560 }, + { url = "https://files.pythonhosted.org/packages/3a/b1/6c990ac65e3b4c079a4fb9f5b05f5b013afa0f4ed6780a3dd236d2cbdc64/pydantic_core-2.41.5-cp314-cp314t-musllinux_1_1_armv7l.whl", hash = "sha256:239edca560d05757817c13dc17c50766136d21f7cd0fac50295499ae24f90fdf", size = 2329244 }, + { url = "https://files.pythonhosted.org/packages/d9/02/3c562f3a51afd4d88fff8dffb1771b30cfdfd79befd9883ee094f5b6c0d8/pydantic_core-2.41.5-cp314-cp314t-musllinux_1_1_x86_64.whl", hash = "sha256:2a5e06546e19f24c6a96a129142a75cee553cc018ffee48a460059b1185f4470", size = 2331955 }, + { url = "https://files.pythonhosted.org/packages/5c/96/5fb7d8c3c17bc8c62fdb031c47d77a1af698f1d7a406b0f79aaa1338f9ad/pydantic_core-2.41.5-cp314-cp314t-win32.whl", hash = "sha256:b4ececa40ac28afa90871c2cc2b9ffd2ff0bf749380fbdf57d165fd23da353aa", size = 1988906 }, + { url = "https://files.pythonhosted.org/packages/22/ed/182129d83032702912c2e2d8bbe33c036f342cc735737064668585dac28f/pydantic_core-2.41.5-cp314-cp314t-win_amd64.whl", hash = "sha256:80aa89cad80b32a912a65332f64a4450ed00966111b6615ca6816153d3585a8c", size = 1981607 }, + { url = "https://files.pythonhosted.org/packages/9f/ed/068e41660b832bb0b1aa5b58011dea2a3fe0ba7861ff38c4d4904c1c1a99/pydantic_core-2.41.5-cp314-cp314t-win_arm64.whl", hash = "sha256:35b44f37a3199f771c3eaa53051bc8a70cd7b54f333531c59e29fd4db5d15008", size = 1974769 }, + { url = "https://files.pythonhosted.org/packages/11/72/90fda5ee3b97e51c494938a4a44c3a35a9c96c19bba12372fb9c634d6f57/pydantic_core-2.41.5-graalpy311-graalpy242_311_native-macosx_10_12_x86_64.whl", hash = "sha256:b96d5f26b05d03cc60f11a7761a5ded1741da411e7fe0909e27a5e6a0cb7b034", size = 2115441 }, + { url = "https://files.pythonhosted.org/packages/1f/53/8942f884fa33f50794f119012dc6a1a02ac43a56407adaac20463df8e98f/pydantic_core-2.41.5-graalpy311-graalpy242_311_native-macosx_11_0_arm64.whl", hash = "sha256:634e8609e89ceecea15e2d61bc9ac3718caaaa71963717bf3c8f38bfde64242c", size = 1930291 }, + { url = "https://files.pythonhosted.org/packages/79/c8/ecb9ed9cd942bce09fc888ee960b52654fbdbede4ba6c2d6e0d3b1d8b49c/pydantic_core-2.41.5-graalpy311-graalpy242_311_native-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:93e8740d7503eb008aa2df04d3b9735f845d43ae845e6dcd2be0b55a2da43cd2", size = 1948632 }, + { url = "https://files.pythonhosted.org/packages/2e/1b/687711069de7efa6af934e74f601e2a4307365e8fdc404703afc453eab26/pydantic_core-2.41.5-graalpy311-graalpy242_311_native-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f15489ba13d61f670dcc96772e733aad1a6f9c429cc27574c6cdaed82d0146ad", size = 2138905 }, + { url = "https://files.pythonhosted.org/packages/09/32/59b0c7e63e277fa7911c2fc70ccfb45ce4b98991e7ef37110663437005af/pydantic_core-2.41.5-graalpy312-graalpy250_312_native-macosx_10_12_x86_64.whl", hash = "sha256:7da7087d756b19037bc2c06edc6c170eeef3c3bafcb8f532ff17d64dc427adfd", size = 2110495 }, + { url = "https://files.pythonhosted.org/packages/aa/81/05e400037eaf55ad400bcd318c05bb345b57e708887f07ddb2d20e3f0e98/pydantic_core-2.41.5-graalpy312-graalpy250_312_native-macosx_11_0_arm64.whl", hash = "sha256:aabf5777b5c8ca26f7824cb4a120a740c9588ed58df9b2d196ce92fba42ff8dc", size = 1915388 }, + { url = "https://files.pythonhosted.org/packages/6e/0d/e3549b2399f71d56476b77dbf3cf8937cec5cd70536bdc0e374a421d0599/pydantic_core-2.41.5-graalpy312-graalpy250_312_native-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c007fe8a43d43b3969e8469004e9845944f1a80e6acd47c150856bb87f230c56", size = 1942879 }, + { url = "https://files.pythonhosted.org/packages/f7/07/34573da085946b6a313d7c42f82f16e8920bfd730665de2d11c0c37a74b5/pydantic_core-2.41.5-graalpy312-graalpy250_312_native-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:76d0819de158cd855d1cbb8fcafdf6f5cf1eb8e470abe056d5d161106e38062b", size = 2139017 }, + { url = "https://files.pythonhosted.org/packages/5f/9b/1b3f0e9f9305839d7e84912f9e8bfbd191ed1b1ef48083609f0dabde978c/pydantic_core-2.41.5-pp311-pypy311_pp73-macosx_10_12_x86_64.whl", hash = "sha256:b2379fa7ed44ddecb5bfe4e48577d752db9fc10be00a6b7446e9663ba143de26", size = 2101980 }, + { url = "https://files.pythonhosted.org/packages/a4/ed/d71fefcb4263df0da6a85b5d8a7508360f2f2e9b3bf5814be9c8bccdccc1/pydantic_core-2.41.5-pp311-pypy311_pp73-macosx_11_0_arm64.whl", hash = "sha256:266fb4cbf5e3cbd0b53669a6d1b039c45e3ce651fd5442eff4d07c2cc8d66808", size = 1923865 }, + { url = "https://files.pythonhosted.org/packages/ce/3a/626b38db460d675f873e4444b4bb030453bbe7b4ba55df821d026a0493c4/pydantic_core-2.41.5-pp311-pypy311_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:58133647260ea01e4d0500089a8c4f07bd7aa6ce109682b1426394988d8aaacc", size = 2134256 }, + { url = "https://files.pythonhosted.org/packages/83/d9/8412d7f06f616bbc053d30cb4e5f76786af3221462ad5eee1f202021eb4e/pydantic_core-2.41.5-pp311-pypy311_pp73-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:287dad91cfb551c363dc62899a80e9e14da1f0e2b6ebde82c806612ca2a13ef1", size = 2174762 }, + { url = "https://files.pythonhosted.org/packages/55/4c/162d906b8e3ba3a99354e20faa1b49a85206c47de97a639510a0e673f5da/pydantic_core-2.41.5-pp311-pypy311_pp73-musllinux_1_1_aarch64.whl", hash = "sha256:03b77d184b9eb40240ae9fd676ca364ce1085f203e1b1256f8ab9984dca80a84", size = 2143141 }, + { url = "https://files.pythonhosted.org/packages/1f/f2/f11dd73284122713f5f89fc940f370d035fa8e1e078d446b3313955157fe/pydantic_core-2.41.5-pp311-pypy311_pp73-musllinux_1_1_armv7l.whl", hash = "sha256:a668ce24de96165bb239160b3d854943128f4334822900534f2fe947930e5770", size = 2330317 }, + { url = "https://files.pythonhosted.org/packages/88/9d/b06ca6acfe4abb296110fb1273a4d848a0bfb2ff65f3ee92127b3244e16b/pydantic_core-2.41.5-pp311-pypy311_pp73-musllinux_1_1_x86_64.whl", hash = "sha256:f14f8f046c14563f8eb3f45f499cc658ab8d10072961e07225e507adb700e93f", size = 2316992 }, + { url = "https://files.pythonhosted.org/packages/36/c7/cfc8e811f061c841d7990b0201912c3556bfeb99cdcb7ed24adc8d6f8704/pydantic_core-2.41.5-pp311-pypy311_pp73-win_amd64.whl", hash = "sha256:56121965f7a4dc965bff783d70b907ddf3d57f6eba29b6d2e5dabfaf07799c51", size = 2145302 }, +] + +[[package]] +name = "pydantic-settings" +version = "2.12.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "pydantic" }, + { name = "python-dotenv" }, + { name = "typing-inspection" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/43/4b/ac7e0aae12027748076d72a8764ff1c9d82ca75a7a52622e67ed3f765c54/pydantic_settings-2.12.0.tar.gz", hash = "sha256:005538ef951e3c2a68e1c08b292b5f2e71490def8589d4221b95dab00dafcfd0", size = 194184 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c1/60/5d4751ba3f4a40a6891f24eec885f51afd78d208498268c734e256fb13c4/pydantic_settings-2.12.0-py3-none-any.whl", hash = "sha256:fddb9fd99a5b18da837b29710391e945b1e30c135477f484084ee513adb93809", size = 51880 }, +] + +[[package]] +name = "pyecharts" +version = "2.0.9" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "jinja2" }, + { name = "prettytable" }, + { name = "simplejson" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/99/34/4b051acc2339b1767bcb95f2a22154f5f7fa8cfcb1def092cff47415f06a/pyecharts-2.0.9.tar.gz", hash = "sha256:7b4e8620809c22e32a19d613542f47ea6efa02f1189e00f91134cb5225e8f3ec", size = 166212 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/37/b4/c95a324b12885a110dcb340bad5043a397bc58ecbbf920416eafebc5cd42/pyecharts-2.0.9-py3-none-any.whl", hash = "sha256:c2adada56931c29669bcf3a972137231625136337f874dbd617ccd752f1477b8", size = 153862 }, +] + +[[package]] +name = "pyee" +version = "13.0.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/95/03/1fd98d5841cd7964a27d729ccf2199602fe05eb7a405c1462eb7277945ed/pyee-13.0.0.tar.gz", hash = "sha256:b391e3c5a434d1f5118a25615001dbc8f669cf410ab67d04c4d4e07c55481c37", size = 31250 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/9b/4d/b9add7c84060d4c1906abe9a7e5359f2a60f7a9a4f67268b2766673427d8/pyee-13.0.0-py3-none-any.whl", hash = "sha256:48195a3cddb3b1515ce0695ed76036b5ccc2ef3a9f963ff9f77aec0139845498", size = 15730 }, +] + +[[package]] +name = "pyparsing" +version = "3.3.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/33/c1/1d9de9aeaa1b89b0186e5fe23294ff6517fce1bc69149185577cd31016b2/pyparsing-3.3.1.tar.gz", hash = "sha256:47fad0f17ac1e2cad3de3b458570fbc9b03560aa029ed5e16ee5554da9a2251c", size = 1550512 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/8b/40/2614036cdd416452f5bf98ec037f38a1afb17f327cb8e6b652d4729e0af8/pyparsing-3.3.1-py3-none-any.whl", hash = "sha256:023b5e7e5520ad96642e2c6db4cb683d3970bd640cdf7115049a6e9c3682df82", size = 121793 }, +] + +[[package]] +name = "pyreadline3" +version = "3.5.4" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/0f/49/4cea918a08f02817aabae639e3d0ac046fef9f9180518a3ad394e22da148/pyreadline3-3.5.4.tar.gz", hash = "sha256:8d57d53039a1c75adba8e50dd3d992b28143480816187ea5efbd5c78e6c885b7", size = 99839 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/5a/dc/491b7661614ab97483abf2056be1deee4dc2490ecbf7bff9ab5cdbac86e1/pyreadline3-3.5.4-py3-none-any.whl", hash = "sha256:eaf8e6cc3c49bcccf145fc6067ba8643d1df34d604a1ec0eccbf7a18e6d3fae6", size = 83178 }, +] + +[[package]] +name = "python-dateutil" +version = "2.9.0.post0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "six" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/66/c0/0c8b6ad9f17a802ee498c46e004a0eb49bc148f2fd230864601a86dcf6db/python-dateutil-2.9.0.post0.tar.gz", hash = "sha256:37dd54208da7e1cd875388217d5e00ebd4179249f90fb72437e91a35459a0ad3", size = 342432 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/ec/57/56b9bcc3c9c6a792fcbaf139543cee77261f3651ca9da0c93f5c1221264b/python_dateutil-2.9.0.post0-py2.py3-none-any.whl", hash = "sha256:a8b2bc7bffae282281c8140a97d3aa9c14da0b136dfe83f850eea9a5f7470427", size = 229892 }, +] + +[[package]] +name = "python-dotenv" +version = "1.2.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/f0/26/19cadc79a718c5edbec86fd4919a6b6d3f681039a2f6d66d14be94e75fb9/python_dotenv-1.2.1.tar.gz", hash = "sha256:42667e897e16ab0d66954af0e60a9caa94f0fd4ecf3aaf6d2d260eec1aa36ad6", size = 44221 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/14/1b/a298b06749107c305e1fe0f814c6c74aea7b2f1e10989cb30f544a1b3253/python_dotenv-1.2.1-py3-none-any.whl", hash = "sha256:b81ee9561e9ca4004139c6cbba3a238c32b03e4894671e181b671e8cb8425d61", size = 21230 }, +] + +[[package]] +name = "pytz" +version = "2025.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/f8/bf/abbd3cdfb8fbc7fb3d4d38d320f2441b1e7cbe29be4f23797b4a2b5d8aac/pytz-2025.2.tar.gz", hash = "sha256:360b9e3dbb49a209c21ad61809c7fb453643e048b38924c765813546746e81c3", size = 320884 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/81/c4/34e93fe5f5429d7570ec1fa436f1986fb1f00c3e0f43a589fe2bbcd22c3f/pytz-2025.2-py2.py3-none-any.whl", hash = "sha256:5ddf76296dd8c44c26eb8f4b6f35488f3ccbf6fbbd7adee0b7262d43f0ec2f00", size = 509225 }, +] + +[[package]] +name = "qrcode" +version = "8.2" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "colorama", marker = "sys_platform == 'win32'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/8f/b2/7fc2931bfae0af02d5f53b174e9cf701adbb35f39d69c2af63d4a39f81a9/qrcode-8.2.tar.gz", hash = "sha256:35c3f2a4172b33136ab9f6b3ef1c00260dd2f66f858f24d88418a015f446506c", size = 43317 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/dd/b8/d2d6d731733f51684bbf76bf34dab3b70a9148e8f2cef2bb544fccec681a/qrcode-8.2-py3-none-any.whl", hash = "sha256:16e64e0716c14960108e85d853062c9e8bba5ca8252c0b4d0231b9df4060ff4f", size = 45986 }, +] + +[package.optional-dependencies] +pil = [ + { name = "pillow" }, +] + +[[package]] +name = "simhash" +version = "2.1.2" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "numpy" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/84/2a/2bed94a174fb53931c67c3581869d9c485db7ea64e0c7e8a7d6018c6defe/simhash-2.1.2.tar.gz", hash = "sha256:533bc8cf41e4e6dd83f0b1847363516bf3323e0fa92e63d9e6df4e281e882e1b", size = 4660 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/2e/86/0f4a39a50fcf510753b7abd98b728e66c14fefe89bad9ef5824745bf7b9f/simhash-2.1.2-py3-none-any.whl", hash = "sha256:968de16c82c227a631aa00e57ab58f9e4ee9e47e8408486199a9eb59c1d6979b", size = 4714 }, +] + +[[package]] +name = "simplejson" +version = "3.20.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/41/f4/a1ac5ed32f7ed9a088d62a59d410d4c204b3b3815722e2ccfb491fa8251b/simplejson-3.20.2.tar.gz", hash = "sha256:5fe7a6ce14d1c300d80d08695b7f7e633de6cd72c80644021874d985b3393649", size = 85784 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b9/3e/96898c6c66d9dca3f9bd14d7487bf783b4acc77471b42f979babbb68d4ca/simplejson-3.20.2-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:06190b33cd7849efc413a5738d3da00b90e4a5382fd3d584c841ac20fb828c6f", size = 92633 }, + { url = "https://files.pythonhosted.org/packages/6b/a2/cd2e10b880368305d89dd540685b8bdcc136df2b3c76b5ddd72596254539/simplejson-3.20.2-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:4ad4eac7d858947a30d2c404e61f16b84d16be79eb6fb316341885bdde864fa8", size = 75309 }, + { url = "https://files.pythonhosted.org/packages/5d/02/290f7282eaa6ebe945d35c47e6534348af97472446951dce0d144e013f4c/simplejson-3.20.2-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:b392e11c6165d4a0fde41754a0e13e1d88a5ad782b245a973dd4b2bdb4e5076a", size = 75308 }, + { url = "https://files.pythonhosted.org/packages/43/91/43695f17b69e70c4b0b03247aa47fb3989d338a70c4b726bbdc2da184160/simplejson-3.20.2-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:51eccc4e353eed3c50e0ea2326173acdc05e58f0c110405920b989d481287e51", size = 143733 }, + { url = "https://files.pythonhosted.org/packages/9b/4b/fdcaf444ac1c3cbf1c52bf00320c499e1cf05d373a58a3731ae627ba5e2d/simplejson-3.20.2-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:306e83d7c331ad833d2d43c76a67f476c4b80c4a13334f6e34bb110e6105b3bd", size = 153397 }, + { url = "https://files.pythonhosted.org/packages/c4/83/21550f81a50cd03599f048a2d588ffb7f4c4d8064ae091511e8e5848eeaa/simplejson-3.20.2-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:f820a6ac2ef0bc338ae4963f4f82ccebdb0824fe9caf6d660670c578abe01013", size = 141654 }, + { url = "https://files.pythonhosted.org/packages/cf/54/d76c0e72ad02450a3e723b65b04f49001d0e73218ef6a220b158a64639cb/simplejson-3.20.2-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:21e7a066528a5451433eb3418184f05682ea0493d14e9aae690499b7e1eb6b81", size = 144913 }, + { url = "https://files.pythonhosted.org/packages/3f/49/976f59b42a6956d4aeb075ada16ad64448a985704bc69cd427a2245ce835/simplejson-3.20.2-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:438680ddde57ea87161a4824e8de04387b328ad51cfdf1eaf723623a3014b7aa", size = 144568 }, + { url = "https://files.pythonhosted.org/packages/60/c7/30bae30424ace8cd791ca660fed454ed9479233810fe25c3f3eab3d9dc7b/simplejson-3.20.2-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:cac78470ae68b8d8c41b6fca97f5bf8e024ca80d5878c7724e024540f5cdaadb", size = 146239 }, + { url = "https://files.pythonhosted.org/packages/79/3e/7f3b7b97351c53746e7b996fcd106986cda1954ab556fd665314756618d2/simplejson-3.20.2-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:7524e19c2da5ef281860a3d74668050c6986be15c9dd99966034ba47c68828c2", size = 154497 }, + { url = "https://files.pythonhosted.org/packages/1d/48/7241daa91d0bf19126589f6a8dcbe8287f4ed3d734e76fd4a092708947be/simplejson-3.20.2-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:0e9b6d845a603b2eef3394eb5e21edb8626cd9ae9a8361d14e267eb969dbe413", size = 148069 }, + { url = "https://files.pythonhosted.org/packages/e6/f4/ef18d2962fe53e7be5123d3784e623859eec7ed97060c9c8536c69d34836/simplejson-3.20.2-cp311-cp311-win32.whl", hash = "sha256:47d8927e5ac927fdd34c99cc617938abb3624b06ff86e8e219740a86507eb961", size = 74158 }, + { url = "https://files.pythonhosted.org/packages/35/fd/3d1158ecdc573fdad81bf3cc78df04522bf3959758bba6597ba4c956c74d/simplejson-3.20.2-cp311-cp311-win_amd64.whl", hash = "sha256:ba4edf3be8e97e4713d06c3d302cba1ff5c49d16e9d24c209884ac1b8455520c", size = 75911 }, + { url = "https://files.pythonhosted.org/packages/9d/9e/1a91e7614db0416885eab4136d49b7303de20528860ffdd798ce04d054db/simplejson-3.20.2-cp312-cp312-macosx_10_9_universal2.whl", hash = "sha256:4376d5acae0d1e91e78baeba4ee3cf22fbf6509d81539d01b94e0951d28ec2b6", size = 93523 }, + { url = "https://files.pythonhosted.org/packages/5e/2b/d2413f5218fc25608739e3d63fe321dfa85c5f097aa6648dbe72513a5f12/simplejson-3.20.2-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:f8fe6de652fcddae6dec8f281cc1e77e4e8f3575249e1800090aab48f73b4259", size = 75844 }, + { url = "https://files.pythonhosted.org/packages/ad/f1/efd09efcc1e26629e120fef59be059ce7841cc6e1f949a4db94f1ae8a918/simplejson-3.20.2-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:25ca2663d99328d51e5a138f22018e54c9162438d831e26cfc3458688616eca8", size = 75655 }, + { url = "https://files.pythonhosted.org/packages/97/ec/5c6db08e42f380f005d03944be1af1a6bd501cc641175429a1cbe7fb23b9/simplejson-3.20.2-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:12a6b2816b6cab6c3fd273d43b1948bc9acf708272074c8858f579c394f4cbc9", size = 150335 }, + { url = "https://files.pythonhosted.org/packages/81/f5/808a907485876a9242ec67054da7cbebefe0ee1522ef1c0be3bfc90f96f6/simplejson-3.20.2-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:ac20dc3fcdfc7b8415bfc3d7d51beccd8695c3f4acb7f74e3a3b538e76672868", size = 158519 }, + { url = "https://files.pythonhosted.org/packages/66/af/b8a158246834645ea890c36136584b0cc1c0e4b83a73b11ebd9c2a12877c/simplejson-3.20.2-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:db0804d04564e70862ef807f3e1ace2cc212ef0e22deb1b3d6f80c45e5882c6b", size = 148571 }, + { url = "https://files.pythonhosted.org/packages/20/05/ed9b2571bbf38f1a2425391f18e3ac11cb1e91482c22d644a1640dea9da7/simplejson-3.20.2-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:979ce23ea663895ae39106946ef3d78527822d918a136dbc77b9e2b7f006237e", size = 152367 }, + { url = "https://files.pythonhosted.org/packages/81/2c/bad68b05dd43e93f77994b920505634d31ed239418eb6a88997d06599983/simplejson-3.20.2-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:a2ba921b047bb029805726800819675249ef25d2f65fd0edb90639c5b1c3033c", size = 150205 }, + { url = "https://files.pythonhosted.org/packages/69/46/90c7fc878061adafcf298ce60cecdee17a027486e9dce507e87396d68255/simplejson-3.20.2-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:12d3d4dc33770069b780cc8f5abef909fe4a3f071f18f55f6d896a370fd0f970", size = 151823 }, + { url = "https://files.pythonhosted.org/packages/ab/27/b85b03349f825ae0f5d4f780cdde0bbccd4f06c3d8433f6a3882df887481/simplejson-3.20.2-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:aff032a59a201b3683a34be1169e71ddda683d9c3b43b261599c12055349251e", size = 158997 }, + { url = "https://files.pythonhosted.org/packages/71/ad/d7f3c331fb930638420ac6d236db68e9f4c28dab9c03164c3cd0e7967e15/simplejson-3.20.2-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:30e590e133b06773f0dc9c3f82e567463df40598b660b5adf53eb1c488202544", size = 154367 }, + { url = "https://files.pythonhosted.org/packages/f0/46/5c67324addd40fa2966f6e886cacbbe0407c03a500db94fb8bb40333fcdf/simplejson-3.20.2-cp312-cp312-win32.whl", hash = "sha256:8d7be7c99939cc58e7c5bcf6bb52a842a58e6c65e1e9cdd2a94b697b24cddb54", size = 74285 }, + { url = "https://files.pythonhosted.org/packages/fa/c9/5cc2189f4acd3a6e30ffa9775bf09b354302dbebab713ca914d7134d0f29/simplejson-3.20.2-cp312-cp312-win_amd64.whl", hash = "sha256:2c0b4a67e75b945489052af6590e7dca0ed473ead5d0f3aad61fa584afe814ab", size = 75969 }, + { url = "https://files.pythonhosted.org/packages/5e/9e/f326d43f6bf47f4e7704a4426c36e044c6bedfd24e072fb8e27589a373a5/simplejson-3.20.2-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:90d311ba8fcd733a3677e0be21804827226a57144130ba01c3c6a325e887dd86", size = 93530 }, + { url = "https://files.pythonhosted.org/packages/35/28/5a4b8f3483fbfb68f3f460bc002cef3a5735ef30950e7c4adce9c8da15c7/simplejson-3.20.2-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:feed6806f614bdf7f5cb6d0123cb0c1c5f40407ef103aa935cffaa694e2e0c74", size = 75846 }, + { url = "https://files.pythonhosted.org/packages/7a/4d/30dfef83b9ac48afae1cf1ab19c2867e27b8d22b5d9f8ca7ce5a0a157d8c/simplejson-3.20.2-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:6b1d8d7c3e1a205c49e1aee6ba907dcb8ccea83651e6c3e2cb2062f1e52b0726", size = 75661 }, + { url = "https://files.pythonhosted.org/packages/09/1d/171009bd35c7099d72ef6afd4bb13527bab469965c968a17d69a203d62a6/simplejson-3.20.2-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:552f55745044a24c3cb7ec67e54234be56d5d6d0e054f2e4cf4fb3e297429be5", size = 150579 }, + { url = "https://files.pythonhosted.org/packages/61/ae/229bbcf90a702adc6bfa476e9f0a37e21d8c58e1059043038797cbe75b8c/simplejson-3.20.2-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:c2da97ac65165d66b0570c9e545786f0ac7b5de5854d3711a16cacbcaa8c472d", size = 158797 }, + { url = "https://files.pythonhosted.org/packages/90/c5/fefc0ac6b86b9108e302e0af1cf57518f46da0baedd60a12170791d56959/simplejson-3.20.2-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:f59a12966daa356bf68927fca5a67bebac0033cd18b96de9c2d426cd11756cd0", size = 148851 }, + { url = "https://files.pythonhosted.org/packages/43/f1/b392952200f3393bb06fbc4dd975fc63a6843261705839355560b7264eb2/simplejson-3.20.2-cp313-cp313-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:133ae2098a8e162c71da97cdab1f383afdd91373b7ff5fe65169b04167da976b", size = 152598 }, + { url = "https://files.pythonhosted.org/packages/f4/b4/d6b7279e52a3e9c0fa8c032ce6164e593e8d9cf390698ee981ed0864291b/simplejson-3.20.2-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:7977640af7b7d5e6a852d26622057d428706a550f7f5083e7c4dd010a84d941f", size = 150498 }, + { url = "https://files.pythonhosted.org/packages/62/22/ec2490dd859224326d10c2fac1353e8ad5c84121be4837a6dd6638ba4345/simplejson-3.20.2-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:b530ad6d55e71fa9e93e1109cf8182f427a6355848a4ffa09f69cc44e1512522", size = 152129 }, + { url = "https://files.pythonhosted.org/packages/33/ce/b60214d013e93dd9e5a705dcb2b88b6c72bada442a97f79828332217f3eb/simplejson-3.20.2-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:bd96a7d981bf64f0e42345584768da4435c05b24fd3c364663f5fbc8fabf82e3", size = 159359 }, + { url = "https://files.pythonhosted.org/packages/99/21/603709455827cdf5b9d83abe726343f542491ca8dc6a2528eb08de0cf034/simplejson-3.20.2-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:f28ee755fadb426ba2e464d6fcf25d3f152a05eb6b38e0b4f790352f5540c769", size = 154717 }, + { url = "https://files.pythonhosted.org/packages/3c/f9/dc7f7a4bac16cf7eb55a4df03ad93190e11826d2a8950052949d3dfc11e2/simplejson-3.20.2-cp313-cp313-win32.whl", hash = "sha256:472785b52e48e3eed9b78b95e26a256f59bb1ee38339be3075dad799e2e1e661", size = 74289 }, + { url = "https://files.pythonhosted.org/packages/87/10/d42ad61230436735c68af1120622b28a782877146a83d714da7b6a2a1c4e/simplejson-3.20.2-cp313-cp313-win_amd64.whl", hash = "sha256:a1a85013eb33e4820286139540accbe2c98d2da894b2dcefd280209db508e608", size = 75972 }, + { url = "https://files.pythonhosted.org/packages/05/5b/83e1ff87eb60ca706972f7e02e15c0b33396e7bdbd080069a5d1b53cf0d8/simplejson-3.20.2-py3-none-any.whl", hash = "sha256:3b6bb7fb96efd673eac2e4235200bfffdc2353ad12c54117e1e4e2fc485ac017", size = 57309 }, +] + +[[package]] +name = "six" +version = "1.17.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/94/e7/b2c673351809dca68a0e064b6af791aa332cf192da575fd474ed7d6f16a2/six-1.17.0.tar.gz", hash = "sha256:ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81", size = 34031 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b7/ce/149a00dd41f10bc29e5921b496af8b574d8413afcd5e30dfa0ed46c2cc5e/six-1.17.0-py2.py3-none-any.whl", hash = "sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274", size = 11050 }, +] + +[[package]] +name = "sympy" +version = "1.14.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "mpmath" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/83/d3/803453b36afefb7c2bb238361cd4ae6125a569b4db67cd9e79846ba2d68c/sympy-1.14.0.tar.gz", hash = "sha256:d3d3fe8df1e5a0b42f0e7bdf50541697dbe7d23746e894990c030e2b05e72517", size = 7793921 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a2/09/77d55d46fd61b4a135c444fc97158ef34a095e5681d0a6c10b75bf356191/sympy-1.14.0-py3-none-any.whl", hash = "sha256:e091cc3e99d2141a0ba2847328f5479b05d94a6635cb96148ccb3f34671bd8f5", size = 6299353 }, +] + +[[package]] +name = "typing-extensions" +version = "4.15.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/72/94/1a15dd82efb362ac84269196e94cf00f187f7ed21c242792a923cdb1c61f/typing_extensions-4.15.0.tar.gz", hash = "sha256:0cea48d173cc12fa28ecabc3b837ea3cf6f38c6d1136f85cbaaf598984861466", size = 109391 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl", hash = "sha256:f0fa19c6845758ab08074a0cfa8b7aecb71c999ca73d62883bc25cc018c4e548", size = 44614 }, +] + +[[package]] +name = "typing-inspection" +version = "0.4.2" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/55/e3/70399cb7dd41c10ac53367ae42139cf4b1ca5f36bb3dc6c9d33acdb43655/typing_inspection-0.4.2.tar.gz", hash = "sha256:ba561c48a67c5958007083d386c3295464928b01faa735ab8547c5692e87f464", size = 75949 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/dc/9b/47798a6c91d8bdb567fe2698fe81e0c6b7cb7ef4d13da4114b41d239f65d/typing_inspection-0.4.2-py3-none-any.whl", hash = "sha256:4ed1cacbdc298c220f1bd249ed5287caa16f34d44ef4e9c3d0cbad5b521545e7", size = 14611 }, +] + +[[package]] +name = "tzdata" +version = "2025.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/5e/a7/c202b344c5ca7daf398f3b8a477eeb205cf3b6f32e7ec3a6bac0629ca975/tzdata-2025.3.tar.gz", hash = "sha256:de39c2ca5dc7b0344f2eba86f49d614019d29f060fc4ebc8a417896a620b56a7", size = 196772 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c7/b0/003792df09decd6849a5e39c28b513c06e84436a54440380862b5aeff25d/tzdata-2025.3-py2.py3-none-any.whl", hash = "sha256:06a47e5700f3081aab02b2e513160914ff0694bce9947d6b76ebd6bf57cfc5d1", size = 348521 }, +] + +[[package]] +name = "w3lib" +version = "2.3.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/bf/7d/1172cfaa1e29beb9bf938e484c122b3bdc82e8e37b17a4f753ba6d6e009f/w3lib-2.3.1.tar.gz", hash = "sha256:5c8ac02a3027576174c2b61eb9a2170ba1b197cae767080771b6f1febda249a4", size = 49531 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/58/dd/56f0d8af71e475ed194d702f8b4cf9cea812c95e82ad823d239023c6558c/w3lib-2.3.1-py3-none-any.whl", hash = "sha256:9ccd2ae10c8c41c7279cd8ad4fe65f834be894fe7bfdd7304b991fd69325847b", size = 21751 }, +] + +[[package]] +name = "wcwidth" +version = "0.2.14" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/24/30/6b0809f4510673dc723187aeaf24c7f5459922d01e2f794277a3dfb90345/wcwidth-0.2.14.tar.gz", hash = "sha256:4d478375d31bc5395a3c55c40ccdf3354688364cd61c4f6adacaa9215d0b3605", size = 102293 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/af/b5/123f13c975e9f27ab9c0770f514345bd406d0e8d3b7a0723af9d43f710af/wcwidth-0.2.14-py2.py3-none-any.whl", hash = "sha256:a7bb560c8aee30f9957e5f9895805edd20602f2d7f720186dfd906e82b4982e1", size = 37286 }, +] + +[[package]] +name = "win32-setctime" +version = "1.2.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/b3/8f/705086c9d734d3b663af0e9bb3d4de6578d08f46b1b101c2442fd9aecaa2/win32_setctime-1.2.0.tar.gz", hash = "sha256:ae1fdf948f5640aae05c511ade119313fb6a30d7eabe25fef9764dca5873c4c0", size = 4867 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e1/07/c6fe3ad3e685340704d314d765b7912993bcb8dc198f0e7a89382d37974b/win32_setctime-1.2.0-py3-none-any.whl", hash = "sha256:95d644c4e708aba81dc3704a116d8cbc974d70b3bdb8be1d150e36be6e9d1390", size = 4083 }, +] + +[[package]] +name = "wordcloud" +version = "1.9.5" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "matplotlib" }, + { name = "numpy" }, + { name = "pillow" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/1f/a2/108cd319f6315931708a7c03d0824cd8684eb56e0af56e375e61785e4b3c/wordcloud-1.9.5.tar.gz", hash = "sha256:6ac7c1378f2886d7e849600a306febd41d0d46b15ce876d665a3e549f5403b0b", size = 27563652 } +wheels = [ + { url = "https://files.pythonhosted.org/packages/7d/a2/319d4fac92cc9a943d86fd1feb39077e6ca74dfeca8b0bc5a5be409d235f/wordcloud-1.9.5-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:c4156064e1d20a553125fef32f7bb5b2333a1966661f71688e714380a88ca4ea", size = 168771 }, + { url = "https://files.pythonhosted.org/packages/b7/0c/d4a1510749489b1ac1390ec15f8f814923ae65014367533ebbc167222cc5/wordcloud-1.9.5-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:6ab1913657a2189b84ae9ab7eacb50516e37586f420ccdc6abdfaa23512e4424", size = 168402 }, + { url = "https://files.pythonhosted.org/packages/e9/03/1ff71d1ba850aa15f2c1959ead7142db781e4a767bb95045358fd0927290/wordcloud-1.9.5-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ed6dbd945c3e4a18b39823a7ffea839499e684c85f10d25cd7693d1b9892c52a", size = 547684 }, + { url = "https://files.pythonhosted.org/packages/bf/0c/606dde0beb4abd952a8d1631e4630fac70c51daedd94a204aa0086f3da6e/wordcloud-1.9.5-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:55c0a6d38ba4fef29738e4725142b4c0c3769f301047758d480eb110e6796060", size = 551670 }, + { url = "https://files.pythonhosted.org/packages/4f/74/2ebe53a215e7e88b89983f16b14fb85ab75ba94817fa11e466f51808ac12/wordcloud-1.9.5-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:b31c2568af3201806193e7ca5c05e7d66b60ba2bcd854b08501b2496d77ca2ff", size = 544097 }, + { url = "https://files.pythonhosted.org/packages/06/65/83525a140ed7b26b367e56010846a3e655f26053eb336112d8b6559e87a2/wordcloud-1.9.5-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:1a630bfff5614b90e3cbb4b6d8a74bb296dc247126868fe40c1e5678c3f2ca58", size = 555467 }, + { url = "https://files.pythonhosted.org/packages/41/6a/8d22301bc03756652cc7081f2f196614c1cd505e8cd22d771e6bc7530d6e/wordcloud-1.9.5-cp311-cp311-win32.whl", hash = "sha256:69ff2f262ca349ae59482b647aef63222d19aeba54f38a705ffeca558847826c", size = 295608 }, + { url = "https://files.pythonhosted.org/packages/14/9e/6d5357fe58af3a1c6f7e58eaa88e77fb86556ff4027e8ea90032a6185ba0/wordcloud-1.9.5-cp311-cp311-win_amd64.whl", hash = "sha256:ded40e3ebaaa96eaa7ae86df0bbe89da7ebf5301efcfe9e429145a2a04dc72fb", size = 306082 }, + { url = "https://files.pythonhosted.org/packages/f0/af/8ca23d9a29c7e646e9e21ca0c3f798b08dd3e58ea61b32f4431a13d27d41/wordcloud-1.9.5-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:ac56a1bdd961253f528d48044650ede52534a2fc47427c16729e8386a2beba29", size = 170100 }, + { url = "https://files.pythonhosted.org/packages/3a/18/4239c7a209a55a1dbb58d2bfca215d9a53500eab5b4386d1b5c44d47a073/wordcloud-1.9.5-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:daf0a1a80fdd8bb60619fa3e6855ecf862efaa6149e1c86fcbefe5db02354cd4", size = 168920 }, + { url = "https://files.pythonhosted.org/packages/bf/4f/dc24ca5c366a7f5ff2d7ea510cf50ed5b3773825ceba56af1cf9b803437e/wordcloud-1.9.5-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d36983701f194828fa99c1162cd16610f43959a5fe09f9f8b7b6619ab7390051", size = 548944 }, + { url = "https://files.pythonhosted.org/packages/90/99/a4bc45e087f7f3f11893b0a4feea5d9d72ecd75d9c615341e04de069023b/wordcloud-1.9.5-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b86143325c12479f02c7f994cf19afcc5bc194a2d74456adf03d71c91215bb99", size = 555208 }, + { url = "https://files.pythonhosted.org/packages/10/c0/021e86f11fe660adb88d58e7b3f66658ae9a93b02f1d75c5fb036d4a7359/wordcloud-1.9.5-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:599ff3e40121ad6b2714daf3fb19c094d3248de69429792a674321079dc93bfe", size = 539344 }, + { url = "https://files.pythonhosted.org/packages/7d/ef/a08ea52eb7649d9296abbc6319634b1ece7bc14b1080ac92a460d725a410/wordcloud-1.9.5-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:f32fbe231518534b20703dadc7e397cf6edee82c0ff572ce9193447e4418cf94", size = 554955 }, + { url = "https://files.pythonhosted.org/packages/a0/42/116c6f3365ad2f0d882ad68c33122e3a82c7503a6eadfff1286a59121efa/wordcloud-1.9.5-cp312-cp312-win32.whl", hash = "sha256:5a8954b28d5c9d515944343adecd9dfde8dbe723815768fdc5ae5eb541426f1e", size = 296178 }, + { url = "https://files.pythonhosted.org/packages/ba/66/04e0f33135d7b8d76bd1721c1c7a42a0cbe748ff48588ab6ea01316e1ab0/wordcloud-1.9.5-cp312-cp312-win_amd64.whl", hash = "sha256:790cf92513a1f5e4d65c801d9fe35c607a4219079075f342bb2fe32d427d64ce", size = 307255 }, + { url = "https://files.pythonhosted.org/packages/04/14/261b76055dda37c4adda27d81b4c4917c0c8c0beeb82bc17cc929112fd19/wordcloud-1.9.5-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:a8468c5943ca2c95a5785314dd8b9c2aff622a16459ec16f44fbabfdb47ed68d", size = 169342 }, + { url = "https://files.pythonhosted.org/packages/50/14/3d60c08364ae1a8c54ab7b1f326f69c681e7c59dff32081bf75adb2b2b26/wordcloud-1.9.5-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:cc68fc27f2fc613a903271c03ff9c47801bce9e4663cf940831d434cb8e7aae8", size = 168295 }, + { url = "https://files.pythonhosted.org/packages/08/55/275ded21f0b815c93d12d66b0425b1ee549b19bb5c5e8d60924e43f05b7e/wordcloud-1.9.5-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:000c8ee667a572334763de28d49838456a12642f272b97c5883550ba143f2d93", size = 543738 }, + { url = "https://files.pythonhosted.org/packages/e8/56/72d77bc4416a6aa97ffbba633ac6d8f75156cd593e5f559cad8d84553be4/wordcloud-1.9.5-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f5ae9697c4a6674fd19b16178f1c40b8f3445eb6324fa06cbe0ff053c03c3d61", size = 551843 }, + { url = "https://files.pythonhosted.org/packages/cb/6f/937f53365cc67f98325057490d63a27749c8526fd8bea8e4a1fbe74045e3/wordcloud-1.9.5-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:a16c7bc45b1625374620d0a859c0c6dea0b86a7161b054a9949deaabac5da3dc", size = 536626 }, + { url = "https://files.pythonhosted.org/packages/63/cc/d566ab24da637787a381a1bc9999a166932b53d2b73b12b989c8fb1fa595/wordcloud-1.9.5-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:ecd0400835ee92d9d335aac3d73b3a50cb44f3ba408f03132d5d0bd915b8feef", size = 552904 }, + { url = "https://files.pythonhosted.org/packages/fb/96/47bfce33702be594bad4c1f652e7e16c1d8efd5b8163f9389f38dfa4b3fa/wordcloud-1.9.5-cp313-cp313-win32.whl", hash = "sha256:dc2027675d9c8a72e6565844e442736ff55ce670421ddb02c77d5d9178a9a798", size = 296069 }, + { url = "https://files.pythonhosted.org/packages/c1/b9/440cf09c98680f15ccd83aa31a71d8789ad70ee65a731ead23a6ba8b169c/wordcloud-1.9.5-cp313-cp313-win_amd64.whl", hash = "sha256:b02205ff66f81bc6be1c418c98da08c353847196f2ccf945bd3ea5a52f22aec7", size = 307047 }, + { url = "https://files.pythonhosted.org/packages/29/44/59372ad37df4c93a662f405689ad35039a40c6a0e0dc72e420a6b9bf8aab/wordcloud-1.9.5-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:8cc6baa2f2a64cf6c65eb0034997b54e2a7538e14833582ad91f3686826ebb14", size = 169693 }, + { url = "https://files.pythonhosted.org/packages/72/b3/60481e917ce07f3139961eb1dc32f45746dda132fc919d27f2aefa6ca2a2/wordcloud-1.9.5-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:3e744e70d99b2083f7a544c77fea59e0df92053c3d310669c43235bbdaa9a6b9", size = 168940 }, + { url = "https://files.pythonhosted.org/packages/6a/c5/dd5e409a9ecb2e1fa026b061703b16282257b350dd5a15c5b8de682ecc04/wordcloud-1.9.5-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:508a569826751f52db1225c91e01f3ea4c725421431338a6cb17768aa29b3d15", size = 543226 }, + { url = "https://files.pythonhosted.org/packages/01/ab/9c24089ea2883403a3a1586745f827cb6866bd43b41276a0d00d12a4b978/wordcloud-1.9.5-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8e09563dd90aeb42d2ca034b39ba691298f783be373dc7d7c2d7430e729c1f41", size = 547357 }, + { url = "https://files.pythonhosted.org/packages/03/c2/7d24d1fdddf329da69a2c72a6204e40250e42478037c593545d70691539e/wordcloud-1.9.5-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:97d0ac557dc80710e56609760f733ae4f382fc93e4c807c4abcfacbfb1a7a4c3", size = 535710 }, + { url = "https://files.pythonhosted.org/packages/ba/b9/c474a1d651fe2d941a6f92ad65e8debaebc8da7df9773182002ccef9787a/wordcloud-1.9.5-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:7addfa9623317ff7b59de1afe308bfb42db575ef804b271b11777797c0699e02", size = 549386 }, + { url = "https://files.pythonhosted.org/packages/12/9c/026fee4942a6e21074d19929b4833b6d4c1019e7e225d73370e85cba3e91/wordcloud-1.9.5-cp314-cp314-win32.whl", hash = "sha256:0e8c8aab33d9b495656b1315715a31c222f1ec8c9ca40d719372e200fee0204c", size = 297175 }, + { url = "https://files.pythonhosted.org/packages/6b/4e/938315f85438df0e225cb613d783301585bf1adf8d5fe869dca18b029e71/wordcloud-1.9.5-cp314-cp314-win_amd64.whl", hash = "sha256:cda8de69df5fac5a90aea3646993b03b4a920d8aa6454b6f6e58c341397b9ca6", size = 308691 }, + { url = "https://files.pythonhosted.org/packages/55/a2/d04ca5669acddefe29faeb3d7103b6f735b23ebaad82cf73a067561c906b/wordcloud-1.9.5-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:6cee7e8668b1905844a4901597f2dc12ddb97f58586dc520eaf1016a6949cd6e", size = 174155 }, + { url = "https://files.pythonhosted.org/packages/68/05/f77b6ceb7eead741a3b2abbdbad5cb404f1d1297a9708766f452a115341b/wordcloud-1.9.5-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:4c3d3c2477613a61ea671ace84b0acde3b00d0e9afd49636d4bf3e504a3a8a05", size = 174214 }, + { url = "https://files.pythonhosted.org/packages/a6/2c/265936f5efc0edcd9204107f3f5ecf224514370fe4886fe1b7ae35018b63/wordcloud-1.9.5-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4f81f7978cf247a981764b15e90c46adc90a1af8c8983e376dbddd6a94137862", size = 559691 }, + { url = "https://files.pythonhosted.org/packages/31/74/ee7ea5117554e36fcb2ed878d4a271ef8c0af0c3cd4727694d67814c131a/wordcloud-1.9.5-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5c71a569259983733a496df6918c72fbecb480929f8b6514fbe754030b41ab7a", size = 552045 }, + { url = "https://files.pythonhosted.org/packages/55/71/674f39d3a766b1d89c56a9671746653e169fd84251617a848258167a4936/wordcloud-1.9.5-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:c3c1c9b865b8ddf22a9eb529d83395f73151117d06e3e9aeaa1ef0f6db1979af", size = 542971 }, + { url = "https://files.pythonhosted.org/packages/27/8c/613b2f63ed3231ac536e1efb45c0ab73037e53b5f00449ece9664df6b31f/wordcloud-1.9.5-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:c9bd4cbd662774e4d83ffb70cfb4a5de4107ae468247e8b68c4cc87a81dd4efc", size = 546190 }, + { url = "https://files.pythonhosted.org/packages/92/ca/0041d90657e2422c76d75a1f32cced0152251bf9d2e9005975cc69b3c953/wordcloud-1.9.5-cp314-cp314t-win32.whl", hash = "sha256:de4749944686c5cfc10143f718d24c965bfbff48d920273cd5b15e889b89a3ae", size = 306425 }, + { url = "https://files.pythonhosted.org/packages/ea/47/5f27d088000e301d174d33a6dd852f7ea6bae6e914e9971d24a9460fa35e/wordcloud-1.9.5-cp314-cp314t-win_amd64.whl", hash = "sha256:e19c3883165967ad4e0cb7baa9208fdca758cfd0f75d68743a9390269180d47a", size = 320479 }, +]