diff --git a/README.md b/README.md index 9bef71b..7b1cae7 100644 --- a/README.md +++ b/README.md @@ -1,82 +1,59 @@ -GMapiServer +# GMapiServer GMapiServer 是一个基于 Python 和 Flask 构建的高性能 API 服务器，支持通过 URL 输入文件并输出下载链接，提供同步和异步两种处理模式。内置缓存自动清理机制（默认保留 2 小时），支持多种工具接口调用，包括实时任务状态查询和视频帧处理功能。 -🚀 项目简介 +## 🚀 项目简介 GMapiServer 是一个轻量级、模块化的 API 服务框架，旨在为开发者提供以下核心功能： -• URL 输入与输出：支持通过 URL 提交文件，返回生成文件的下载链接。 +- **URL 输入与输出**：支持通过 URL 提交文件，返回生成文件的下载链接。 +- **异步处理**：任务在后台异步执行，不阻塞主进程，提升服务器并发性能。 +- **自动缓存清理**：内置定时清理机制，自动删除过期的缓存文件（包括输出文件），默认保留时间 2 小时。 +- **多工具接口集成**：集成 FFmpeg 和 Sanjuuni 工具接口，支持在线调用。 +- **实时任务状态**：提供任务状态查询接口，支持日志实时推送。 +- **视频帧提取**：支持从视频中提取指定分辨率、帧率的图片帧，可打包为二进制格式。 +- **硬件加速支持**：支持 NVIDIA CUDA、Intel QSV、VA-API、树莓派 v4l2m2m 等硬件加速方式。 -• 异步处理：任务在后台异步执行，不阻塞主进程，提升服务器并发性能。 +## 🛠️ 技术栈 -• 自动缓存清理：内置定时清理机制，自动删除过期的缓存文件（包括输出文件），默认保留时间 2 小时。 +- **编程语言**: Python 3.10+ +- **Web 框架**: Flask +- **异步处理**: 基于 threading +- **缓存清理**: 定时任务 +- **依赖管理**: pip +- **视频处理**: FFmpeg (支持硬件加速) -• 多工具接口集成：集成 FFmpeg 和 Sanjuuni 工具接口，支持在线调用。 +## 🧩 支持的接口 -• 实时任务状态：提供任务状态查询接口，支持日志实时推送。 +### 1. FFmpeg 工具接口 -• 视频帧提取：支持从视频中提取指定分辨率、帧率的图片帧，可打包为二进制格式。 +- **功能**: 在线调用 FFmpeg 工具进行视频/音频处理（如转码、裁剪、合并等）。 +- **接口文档**: +- **同步接口**: POST /api/ffmpeg +- **异步接口**: POST /api/ffmpeg/async -🛠️ 技术栈 +### 2. Sanjuuni 工具接口 -• 编程语言: Python +- **功能**: 在线调用 +- **接口文档**: +- **同步接口**: POST /api/sanjuuni +- **异步接口**: POST /api/sanjuuni/async -• Web 框架: Flask +### 3. 视频帧提取接口 -• 异步处理: 基于 threading +- **功能**: 从视频中提取指定分辨率、帧率的图片帧，支持强制分辨率调整和填充，支持B站BV号。 +- **异步接口**: POST /api/video\_frame/async +- **参数**: + - **video\_url**: 视频文件 URL (或使用 video\_bv 参数替代) + - **video\_bv**: B站视频BV号 (自动转换为URL) + - **w**: 输出宽度 + - **h**: 输出高度 + - **fps**: 输出帧率 (默认: 30) + - **force\_resolution**: 是否强制调整分辨率 (默认: false) + - **pad\_to\_target**: 是否填充到目标分辨率 (默认: false) + - **pix\_fmt**: 输出像素格式，可选 `pal8` 或 `rgb24`（默认: `pal8`） +- **返回示例**: -• 缓存清理: 定时任务 - -• 依赖管理: pip - -🧩 支持的接口 - -1. FFmpeg 工具接口 - -• 功能: 在线调用 FFmpeg 工具进行视频/音频处理（如转码、裁剪、合并等）。 - -• 接口文档: https://www.liulikeji.cn/archives/FFmpegApi - -• 同步接口: POST /api/ffmpeg - -• 异步接口: POST /api/ffmpeg/async - -2. Sanjuuni 工具接口 - -• 功能: 在线调用 https://github.com/MCJack123/sanjuuni/tree/master（具体功能需参考其官方文档）。 - -• 接口文档: https://www.liulikeji.cn/archives/SanjuuniApi - -• 同步接口: POST /api/sanjuuni - -• 异步接口: POST /api/sanjuuni/async - -3. 视频帧提取接口 - -• 功能: 从视频中提取指定分辨率、帧率的图片帧，支持强制分辨率调整和填充，支持B站BV号。 - -• 异步接口: POST /api/video_frame/async - -• 参数: - - • video_url: 视频文件 URL (或使用 video_bv 参数替代) - - • video_bv: B站视频BV号 (自动转换为URL) - - • w: 输出宽度 - - • h: 输出高度 - - • fps: 输出帧率 (默认: 30) - - • force_resolution: 是否强制调整分辨率 (默认: false) - - • pad_to_target: 是否填充到目标分辨率 (默认: false) - - • pix_fmt: 输出像素格式，可选 `pal8` 或 `rgb24`（默认: `pal8`） - -• 返回示例: ```json { "task_id": "abc12345", @@ -85,7 +62,8 @@ GMapiServer 是一个轻量级、模块化的 API 服务框架，旨在为开发 } ``` -• 实时任务状态响应（处理中）: +- **实时任务状态响应（处理中）**: + ```json { "task_id": "abc12345", @@ -111,7 +89,8 @@ GMapiServer 是一个轻量级、模块化的 API 服务框架，旨在为开发 } ``` -• 任务完成时结果: +- **任务完成时结果**: + ```json { "task_id": "abc12345", @@ -139,78 +118,122 @@ GMapiServer 是一个轻量级、模块化的 API 服务框架，旨在为开发 } ``` -4. 异步任务状态查询 +### 4. 异步任务状态查询 -• 功能: 查询异步任务状态和进度，支持日志实时推送。 +- **功能**: 查询异步任务状态和进度，支持日志实时推送。 +- **接口**: GET /api/task/\ +- **返回字段**: + - **status**: 任务状态（pending/running/completed/error） + - **progress**: 进度百分比 + - **new\_logs**: 新增的日志条目 + - **result**: 完成后的结果（包含下载URL） + - **error**: 错误信息（如果状态为error） -• 接口: GET /api/task/ +### 5. 帧打包接口 -• 返回字段: +- **功能**: 将多个帧图片打包为二进制格式，便于批量下载。 +- **接口**: + - `GET /api/framepack?--`（推荐） + - `POST /api/framepack`（兼容旧方式） +- **参数**: + - **新方式**: + - `task_id`: 视频帧任务ID + - `start`: 起始帧（从1开始） + - `end`: 结束帧（包含） + - **旧方式**: + - `urls`: 帧图片URL列表，格式为 `/frames//.png` +- **返回**: 二进制流，格式为帧数(4字节) + 各帧数据 - • status: 任务状态（pending/running/completed/error） +### 6. 文件下载接口 - • progress: 进度百分比 +- **功能**: 下载处理完成的文件。 +- **接口**: GET /download/\/ - • new_logs: 新增的日志条目 +### 7. 任务列表查询 - • result: 完成后的结果（包含下载URL） +- **功能**: 列出所有任务状态（调试用）。 +- **接口**: GET /api/tasks - • error: 错误信息（如果状态为error） +### 8. 健康检查 -5. 帧打包接口 +- **功能**: 服务健康检查。 +- **接口**: GET /health -• 功能: 将多个帧图片打包为二进制格式，便于批量下载。 +## 📦 部署与使用 -• 接口: - • `GET /api/framepack?--`（推荐） - • `POST /api/framepack`（兼容旧方式） - -• 参数: - - • 新方式: - • `task_id`: 视频帧任务ID - • `start`: 起始帧（从1开始） - • `end`: 结束帧（包含） - - • 旧方式: - • `urls`: 帧图片URL列表，格式为 `/frames//.png` - -• 返回: 二进制流，格式为帧数(4字节) + 各帧数据 - -6. 文件下载接口 - -• 功能: 下载处理完成的文件。 - -• 接口: GET /download// - -7. 任务列表查询 - -• 功能: 列出所有任务状态（调试用）。 - -• 接口: GET /api/tasks - -8. 健康检查 - -• 功能: 服务健康检查。 - -• 接口: GET /health - -📦 部署与使用 - -1. 安装依赖 +### 1. 安装依赖 +```bash pip install -r requirements.txt +``` +### 2. 配置文件 -2. 启动服务器 +编辑 `config.json` 配置各项参数： +```json +{ + "streaming": { + "enabled": true, + "buffer_size": 8192, + "chunk_delay_ms": 0 + }, + "server": { + "port": 5000, + "debug": false, + "threaded": true + }, + "cleanup": { + "interval_hours": 1, + "retention_hours": 2, + "download_cache_max_mb": 500, + "video_cache_max_mb": 2000, + "cache_retention_hours": 2 + }, + "logging": { + "level": "INFO", + "file_path": "data/server.log" + }, + "hwaccel": { + "enabled": true, + "method": "auto" + }, + "video_download": { + "limit_resolution": true, + "max_resolution": "720p", + "preferred_codec": "h264" + } +} +``` + +### 配置说明 + +| 配置项 | 说明 | 默认值 | +| --------------------------------- | --------------------------------------- | ---- | +| `hwaccel.enabled` | 是否启用硬件加速 | true | +| `hwaccel.method` | 硬件加速方式（auto/cuda/vaapi/qsv/v4l2m2m/drm） | auto | +| `video_download.limit_resolution` | 是否限制下载分辨率 | true | +| `video_download.max_resolution` | 最大下载分辨率（480p/720p/1080p/best） | 720p | +| `video_download.preferred_codec` | 首选视频编码（h264/h265/av1/vp9/auto） | h264 | + +### 3. 启动服务器 + +```bash +# 方式1：直接启动 +python main.py + +# 方式2：使用启动脚本 +python start_server.py + +# 方式3：指定端口 python main.py --port 5000 +``` +### 4. 异步任务使用示例 -3. 异步任务使用示例 - -创建异步FFmpeg任务 +**创建异步FFmpeg任务** +```bash curl -X POST http://localhost:5000/api/ffmpeg/async \ -H "Content-Type: application/json" \ -d '{ @@ -218,20 +241,23 @@ curl -X POST http://localhost:5000/api/ffmpeg/async \ "output_format": "mp4", "args": ["-c:v", "libx264", "-crf", "23"] }' +``` +**查询任务状态** -查询任务状态 - +```bash curl http://localhost:5000/api/task/ +``` - -4. 指定公网地址 +### 5. 指定公网地址 在相应工具模块中配置下载URL： + +```python # sanjuuni_utils.py return { 'status': 'success', - 'download_url': f"http://ffmpeg.liulikeji.cn/download/{output_id}/{output_filename}", # 外部访问地址 + 'download_url': f"http://ffmpeg.liulikeji.cn/download/{output_id}/{output_filename}", 'file_id': output_id, 'temp_dir': temp_dir } @@ -239,122 +265,150 @@ return { # ffmpeg_utils.py return { 'status': 'success', - 'download_url': f"http://ffmpeg.liulikeji.cn/download/{output_id}/{output_filename}", # 外部访问地址 + 'download_url': f"http://ffmpeg.liulikeji.cn/download/{output_id}/{output_filename}", 'file_id': output_id, 'temp_dir': temp_dir } +``` +## 🔧 硬件加速配置 -⏰ 自动缓存清理配置 +### 支持的硬件加速方式 -• 默认保留时间: 2 小时 +| 加速方式 | 适用平台 | 说明 | +| ---------------- | ---------------- | ---------------------- | +| **cuda** | NVIDIA GPU | 需要 NVIDIA 显卡和 CUDA 工具包 | +| **vaapi** | Intel/AMD Linux | 需要支持 VA-API 的显卡 | +| **qsv** | Intel Quick Sync | Intel 核显专用 | +| **videotoolbox** | macOS | Apple 芯片专用 | +| **v4l2m2m** | 树莓派/嵌入式 Linux | 支持 H.264 编码 | +| **drm** | Linux DRM/KMS | 树莓派官方系统推荐 | +| **mmal** | 树莓派（旧版） | 树莓派传统硬件加速 | +| **vdpau** | NVIDIA Linux | NVIDIA VDPAU 加速 | -• 自定义配置: - # 在 file_cleanup.py 中修改缓存清理时间 - CLEANUP_INTERVAL = 600 # 清理任务执行间隔(秒) - FILE_EXPIRY = 7200 # 文件记录过期时间(秒) - TASK_EXPIRY = 7200 # 任务记录过期时间(秒) +### 验证硬件加速 -🔄 最新缓存与恢复机制（2026-05 更新） +```bash +# 检查可用编码器 +ffmpeg -encoders | grep v4l2m2m -1. 持久化存储 +# 测试硬件编码 +ffmpeg -i input.mp4 -c:v h264_v4l2m2m output.mp4 -• 任务与缓存索引会写入 `cache_state/state.json`。 +# 检查硬件加速方式 +ffmpeg -hwaccels +``` -• 服务重启后自动恢复任务、文件注册表和视频帧缓存索引。 +## ⏰ 自动缓存清理配置 -2. 视频帧任务去重（全参数匹配） +- **默认保留时间**: 2 小时 +- **自定义配置**: 在 `config.json` 中修改 -• `/api/video_frame/async` 会对请求计算签名（包含 URL、分辨率、fps、pix_fmt、force_resolution、pad_to_target 等）。 +## 🔄 缓存与恢复机制（2026-05 更新） -• 若签名匹配到已有任务： - • 任务 `pending/running`：直接返回已有 `task_id`。 - • 任务 `completed` 且缓存有效：复用已有 `task_id` 并重置过期计时。 - • 任务 `error` / 缓存失效 / 缓存目录缺失：自动新建 `task_id` 重新转换。 +### 1. 持久化存储 -3. 重启中断识别 +- 任务与缓存索引会写入 `cache_state/state.json`。 +- 服务重启后自动恢复任务、文件注册表和视频帧缓存索引。 -• 启动时会将上次 `pending/running` 的任务标记为“因服务器重启中断”。 +### 2. 视频帧任务去重（全参数匹配） -• 这类任务不会被复用，同参数新请求会重新创建任务。 +- `/api/video_frame/async` 会对请求计算签名（包含 URL、分辨率、fps、pix\_fmt、force\_resolution、pad\_to\_target 等）。 +- 若签名匹配到已有任务： + - 任务 `pending/running`：直接返回已有 `task_id`。 + - 任务 `completed` 且缓存有效：复用已有 `task_id` 并重置过期计时。 + - 任务 `error` / 缓存失效 / 缓存目录缺失：自动新建 `task_id` 重新转换。 -4. 看门狗式缓存续期（2 小时） +### 3. 重启中断识别 -• 视频帧缓存采用看门狗语义：每次相关访问都会将过期时间重置为 `当前时间 + 2小时`。 +- 启动时会将上次 `pending/running` 的任务标记为“因服务器重启中断”。 +- 这类任务不会被复用，同参数新请求会重新创建任务。 -• 相关访问包括： - • `/api/video_frame/async` 命中已有任务 - • `/api/framepack` 请求了该 `job_id` 的帧 - • 视频帧任务转换完成后的结果写回 +### 4. 看门狗式缓存续期（2 小时） -• 若超过 2 小时没有相关访问，则按清理策略过期。 +- 视频帧缓存采用看门狗语义：每次相关访问都会将过期时间重置为 `当前时间 + 2小时`。 +- 相关访问包括： + - `/api/video_frame/async` 命中已有任务 + - `/api/framepack` 请求了该 `job_id` 的帧 + - 视频帧任务转换完成后的结果写回 +- 若超过 2 小时没有相关访问，则按清理策略过期。 -5. NoDelete 保护 +### 5. NoDelete 保护 -• 清理线程会检测 `frames//NoDelete` 文件。 +- 清理线程会检测 `frames//NoDelete` 文件。 +- 若存在该文件： + - 跳过该目录清理 + - 并重置该任务缓存计时器（`now + 2h`） -• 若存在该文件： - • 跳过该目录清理 - • 并重置该任务缓存计时器（`now + 2h`） +### 6. 双语日志 -6. 双语日志 +- `ffmpeg`、`sanjuuni`、`video_frame` 的业务日志已统一为中英双语格式：`中文 | English`。 -• `ffmpeg`、`sanjuuni`、`video_frame` 的业务日志已统一为中英双语格式：`中文 | English`。 - - -🔥 视频帧实时进度功能 +## 🔥 视频帧实时进度功能 GMapiServer 的视频帧提取功能支持实时进度跟踪，让客户端可以在任务处理过程中实时获取已生成的帧并进行下载。 -• 实时帧进度跟踪: 自动解析FFmpeg输出的进度信息，实时更新当前帧数 -• 智能进度计算: 基于帧数自动计算转换进度（20-80%） -• 音频预处理: 音频在视频处理前提取，客户端可提前下载音频文件 -• 实时URL返回: 在处理过程中不断返回已生成的帧URL列表 - -## 优势特性 +### 优势特性 1. **减少等待时间**: 客户端无需等待整个视频处理完成 2. **并行下载**: 可以在转换过程中并行下载已生成的帧 3. **实时反馈**: 用户可以看到实时处理进度和日志 4. **资源优化**: 避免了"堵车"效应，提高系统并发能力 -## 使用建议 +### 使用建议 -• 客户端可以轮询 `/api/task/` 接口获取最新状态 -• 通过检查 `result.current_frames` 与 `result.total_frames` 判断转换进度 -• 音频文件可以立即下载（音频处理优先级更高） +- 客户端可以轮询 `/api/task/` 接口获取最新状态 +- 通过检查 `result.current_frames` 与 `result.total_frames` 判断转换进度 +- 音频文件可以立即下载（音频处理优先级更高） +## 🔧 异步任务处理流程 -🔧 异步任务处理流程 +1. **任务创建**: 客户端提交任务请求，服务器返回任务ID和状态查询URL +2. **后台处理**: 任务在独立线程中执行，不阻塞主进程 +3. **状态查询**: 客户端可轮询状态接口获取任务进度和实时日志 +4. **结果获取**: 任务完成后可通过状态接口或文件下载接口获取结果 +5. **自动清理**: 任务结果文件在指定时间后自动清理 -1. 任务创建: 客户端提交任务请求，服务器返回任务ID和状态查询URL -2. 后台处理: 任务在独立线程中执行，不阻塞主进程 -3. 状态查询: 客户端可轮询状态接口获取任务进度和实时日志 -4. 结果获取: 任务完成后可通过状态接口或文件下载接口获取结果 -5. 自动清理: 任务结果文件在指定时间后自动清理 +## ⚠️ 注意事项 -⚠️ 注意事项 +1. **文件合法性**: 请确保上传文件符合法律法规，不得用于非法用途。 +2. **缓存文件**: 系统会自动清理过期文件，请及时下载生成的输出文件。 +3. **异步任务**: 任务状态已支持持久化恢复；但重启前处于运行中的任务会被标记为中断，需要重新发起。 +4. **资源限制**: 视频帧提取和转码操作可能消耗较多CPU和内存资源。 +5. **树莓派注意**: 硬件解码可能不可用，但硬件编码（h264\_v4l2m2m）可用，性能提升约3倍。 -1. 文件合法性: 请确保上传文件符合法律法规，不得用于非法用途。 -2. 缓存文件: 系统会自动清理过期文件，请及时下载生成的输出文件。 -3. 异步任务: 任务状态已支持持久化恢复；但重启前处于运行中的任务会被标记为中断，需要重新发起。 -4. 资源限制: 视频帧提取和转码操作可能消耗较多CPU和内存资源。 +## 📝 贡献与反馈 -📝 贡献与反馈 - -欢迎提交 Issue 或 Pull Request！ -如有问题，请联系：[xingluo01@liulikeji.cn] 或 [qq:180877430] - -📁 项目结构 +欢迎提交 Issue 或 Pull Request！\ +如有问题，请联系：\[] 或 \[qq:180877430] +## 📁 项目结构 +``` GMapiServer/ ├── main.py # 主程序入口 +├── start_server.py # 启动脚本（含运行库检查） +├── config.py # 配置管理 +├── config.json # 配置文件 ├── shared_utils.py # 共享工具函数 ├── ffmpeg_utils.py # FFmpeg处理模块 ├── sanjuuni_utils.py # Sanjuuni处理模块 ├── video_frame_utils.py # 视频帧提取模块 ├── file_cleanup.py # 文件清理模块 +├── hwaccel_detect.py # 硬件加速检测模块 +├── lib_downloader.py # 运行库下载器 ├── logging_config.py # 日志配置 +├── logo.py # Logo输出 ├── requirements.txt # 依赖包列表 +├── test/ # 测试工具目录 +│ └── decode_performance_test.py # 性能测试脚本 +├── lib/ # 本地运行库 +│ ├── ffmpeg/ +│ └── sanjuuni/ +├── data/ # 数据目录 +│ ├── ffmpeg/ +│ ├── sanjuuni/ +│ └── videoframe/ └── README.md # 项目说明文档 +``` + diff --git a/config.json b/config.json new file mode 100644 index 0000000..590405c --- /dev/null +++ b/config.json @@ -0,0 +1,89 @@ +{ + // ==================== 转码模式配置 ==================== + "streaming": { + // 转码模式（无法生效）： + // true - 流式处理（边转码边发送，适合高性能服务器） + // false - 非流式处理（转码完成后再发送，适合低性能服务器避免超时） + "enabled": true, + + // 流式传输缓冲区大小（字节） + "buffer_size": 8192, + + // 块发送延迟（毫秒），用于控制发送速率 + "chunk_delay_ms": 0 + }, + + // ==================== 服务器配置 ==================== + "server": { + // 服务监听端口 + "port": 5000, + + // 调试模式（开发环境使用） + "debug": false, + + // 多线程模式 + "threaded": true + }, + + // ==================== 自动清理配置 ==================== + "cleanup": { + // 清理任务执行间隔（小时） + "interval_hours": 1, + + // 下载临时文件保留时间（小时） + "retention_hours": 2, + + // 下载缓存最大大小（MB） + "download_cache_max_mb": 500, + + // 视频缓存最大大小（MB） + "video_cache_max_mb": 2000, + + // 缓存文件保留时间（小时） + "cache_retention_hours": 2 + }, + + // ==================== 日志配置 ==================== + "logging": { + // 日志级别：DEBUG, INFO, WARNING, ERROR + "level": "INFO", + + // 日志文件路径 + "file_path": "data/server.log" + }, + + // ==================== 硬件加速配置 ==================== + "hwaccel": { + // 是否启用硬件加速 + "enabled": true, + + // 硬件加速方式： + // "auto" - 自动检测最佳方案 + // "cuda" - NVIDIA GPU加速 + // "vaapi" - Intel/AMD Linux加速 + // "qsv" - Intel Quick Sync加速 + // "videotoolbox" - macOS加速 + // "mmal" - 树莓派MMAL硬件加速（需自定义编译FFmpeg） + // "v4l2m2m" - Linux V4L2 Memory-to-Memory加速（适用于树莓派/嵌入式设备） + // "drm" - Linux DRM/KMS加速（树莓派官方系统推荐） + // "vdpau" - NVIDIA VDPAU加速 + // "vulkan" - Vulkan通用GPU加速（跨平台） + "method": "auto" + }, + + // ==================== 视频下载配置 ==================== + "video_download": { + // 是否限制下载分辨率（减轻转码压力） + "limit_resolution": true, + + // 最大下载分辨率（当limit_resolution为true时生效） + // 可选值：480p, 720p, 1080p, 1440p, 2160p, 4320p + // 设置为"best"则下载最佳质量（不限制） + "max_resolution": "720p", + + // 首选视频编码（减轻解码压力） + // 可选值："h264", "h265", "av1", "vp9", "auto" + // 建议树莓派使用"h264"以支持硬件加速 + "preferred_codec": "h264" + } +} \ No newline at end of file diff --git a/config.py b/config.py new file mode 100644 index 0000000..1c08e4b --- /dev/null +++ b/config.py @@ -0,0 +1,174 @@ +import os +import json + +CONFIG_FILE = 'config.json' + +DEFAULT_CONFIG = { + 'streaming': { + 'enabled': True, + 'buffer_size': 8192, + 'chunk_delay_ms': 0 + }, + 'server': { + 'port': 5000, + 'debug': False, + 'threaded': True + }, + 'cleanup': { + 'interval_hours': 1, + 'retention_hours': 2, + 'download_cache_max_mb': 500, + 'video_cache_max_mb': 2000, + 'cache_retention_hours': 24 + }, + 'logging': { + 'level': 'INFO', + 'file_path': 'data/server.log' + }, + 'hwaccel': { + 'enabled': True, + 'method': 'auto' + }, + 'video_download': { + 'limit_resolution': True, + 'max_resolution': '720p', + 'preferred_codec': 'h264' + } +} + +_config = None + +def remove_comments(json_string): + """移除JSON字符串中的//注释""" + lines = json_string.split('\n') + result = [] + + for line in lines: + if '//' in line: + in_string = False + string_char = '' + escape = False + for i, char in enumerate(line): + if escape: + escape = False + continue + + if char == '\\' and in_string: + escape = True + continue + + if char in ('"', "'"): + if in_string and char == string_char: + in_string = False + elif not in_string: + in_string = True + string_char = char + + if not in_string and '//' in line[i:]: + line = line[:i] + break + + stripped = line.strip() + if stripped or result: + result.append(line) + + return '\n'.join(result) + +def load_config(): + global _config + if _config is not None: + return _config + + if os.path.exists(CONFIG_FILE): + try: + with open(CONFIG_FILE, 'r', encoding='utf-8') as f: + content = f.read() + + content = remove_comments(content) + _config = json.loads(content) + except Exception as e: + print(f"配置文件读取失败，使用默认配置: {e}") + _config = DEFAULT_CONFIG.copy() + else: + _config = DEFAULT_CONFIG.copy() + save_config() + + return _config + +def save_config(): + if _config is not None: + os.makedirs(os.path.dirname(CONFIG_FILE) or '.', exist_ok=True) + with open(CONFIG_FILE, 'w', encoding='utf-8') as f: + json.dump(_config, f, ensure_ascii=False, indent=2) + +def get_streaming_enabled(): + config = load_config() + return config.get('streaming', {}).get('enabled', True) + +def get_streaming_buffer_size(): + config = load_config() + return config.get('streaming', {}).get('buffer_size', 8192) + +def get_streaming_chunk_delay(): + config = load_config() + return config.get('streaming', {}).get('chunk_delay_ms', 0) + +def get_server_port(): + config = load_config() + return config.get('server', {}).get('port', 5000) + +def get_server_debug(): + config = load_config() + return config.get('server', {}).get('debug', False) + +def get_server_threaded(): + config = load_config() + return config.get('server', {}).get('threaded', True) + +def get_cleanup_interval(): + config = load_config() + return config.get('cleanup', {}).get('interval_hours', 1) + +def get_cleanup_retention(): + config = load_config() + return config.get('cleanup', {}).get('retention_hours', 2) + +def get_logging_level(): + config = load_config() + return config.get('logging', {}).get('level', 'INFO') + +def get_logging_file_path(): + config = load_config() + return config.get('logging', {}).get('file_path', 'data/server.log') + +def get_hwaccel_enabled(): + config = load_config() + return config.get('hwaccel', {}).get('enabled', True) + +def get_hwaccel_method(): + config = load_config() + return config.get('hwaccel', {}).get('method', 'auto') + +def get_download_cache_max_mb(): + config = load_config() + return config.get('cleanup', {}).get('download_cache_max_mb', 500) + +def get_video_cache_max_mb(): + config = load_config() + return config.get('cleanup', {}).get('video_cache_max_mb', 2000) + +def get_cache_retention_hours(): + config = load_config() + return config.get('cleanup', {}).get('cache_retention_hours', 24) + +def get_video_download_limit_resolution(): + config = load_config() + return config.get('video_download', {}).get('limit_resolution', True) + +def get_video_download_max_resolution(): + config = load_config() + return config.get('video_download', {}).get('max_resolution', '720p') + +def get_video_download_preferred_codec(): + config = load_config() + return config.get('video_download', {}).get('preferred_codec', 'h264') \ No newline at end of file diff --git a/ffmpeg_utils.py b/ffmpeg_utils.py index 168d0a7..085f3d9 100644 --- a/ffmpeg_utils.py +++ b/ffmpeg_utils.py @@ -9,6 +9,94 @@ import threading import time from contextlib import contextmanager from shared_utils import add_task_log, save_persistent_state +from config import get_streaming_enabled, get_streaming_buffer_size, get_hwaccel_enabled, get_hwaccel_method + +_hwaccel_args = None +_hwaccel_method = None +_ffmpeg_path = None + +def find_ffmpeg(): + """查找FFmpeg可执行文件路径""" + global _ffmpeg_path + + if _ffmpeg_path is not None: + return _ffmpeg_path + + if sys.platform.startswith('win'): + local_path = 'lib/ffmpeg/bin/ffmpeg.exe' + exe_name = 'ffmpeg.exe' + else: + local_path = 'lib/ffmpeg/bin/ffmpeg' + exe_name = 'ffmpeg' + + if os.path.exists(local_path): + _ffmpeg_path = local_path + return _ffmpeg_path + + if sys.platform.startswith('linux'): + try: + result = subprocess.run(['which', exe_name], capture_output=True, text=True) + if result.returncode == 0: + _ffmpeg_path = result.stdout.strip() + logging.info(bi(f"使用系统FFmpeg: {_ffmpeg_path}", f"Using system FFmpeg: {_ffmpeg_path}")) + return _ffmpeg_path + except Exception as e: + logging.warning(bi(f"查找系统FFmpeg失败: {e}", f"Failed to find system FFmpeg: {e}")) + + _ffmpeg_path = local_path + return _ffmpeg_path + +def init_hwaccel(): + """初始化硬件加速""" + global _hwaccel_args, _hwaccel_method + + if not get_hwaccel_enabled(): + logging.info(bi("硬件加速已禁用", "Hardware acceleration disabled")) + _hwaccel_args = [] + _hwaccel_method = None + return + + try: + from hwaccel_detect import get_working_hwaccel, get_hwaccel_args, test_hwaccel, detect_hardware_acceleration + + config_method = get_hwaccel_method() + available_methods = detect_hardware_acceleration() + + if config_method != 'auto': + if config_method not in available_methods: + _hwaccel_args = [] + _hwaccel_method = None + logging.warning(bi(f"配置指定的硬件加速方式 '{config_method}' 不在系统支持列表中。可用方式: {available_methods}，使用软件解码", f"Configured hardware acceleration method '{config_method}' not in system support list. Available methods: {available_methods}, using software decoding")) + return + + if test_hwaccel(config_method): + _hwaccel_method = config_method + _hwaccel_args = get_hwaccel_args(config_method) + logging.info(bi(f"使用配置指定的硬件加速方式: {_hwaccel_method}", f"Using configured hardware acceleration method: {_hwaccel_method}")) + else: + _hwaccel_args = [] + _hwaccel_method = None + logging.warning(bi(f"配置指定的硬件加速方式 {config_method} 测试失败，使用软件解码", f"Configured hardware acceleration method {config_method} test failed, using software decoding")) + return + else: + _hwaccel_method = get_working_hwaccel() + if _hwaccel_method: + _hwaccel_args = get_hwaccel_args(_hwaccel_method) + logging.info(bi(f"自动检测到硬件加速方式: {_hwaccel_method}", f"Automatically detected hardware acceleration method: {_hwaccel_method}")) + else: + _hwaccel_args = [] + logging.info(bi("未检测到可用的硬件加速，使用软件解码", "No available hardware acceleration detected, using software decoding")) + except Exception as e: + _hwaccel_args = [] + _hwaccel_method = None + logging.error(bi(f"初始化硬件加速失败: {e}", f"Failed to initialize hardware acceleration: {e}")) + +def get_hwaccel_args(): + """获取硬件加速参数""" + global _hwaccel_args + if _hwaccel_args is None: + init_hwaccel() + return _hwaccel_args UPLOAD_FOLDER = os.path.join('data', 'ffmpeg') os.makedirs(UPLOAD_FOLDER, exist_ok=True) @@ -79,25 +167,25 @@ def execute_ffmpeg(input_path, output_path, ffmpeg_args, task_id=None, task_regi filtered_args = [arg for arg in ffmpeg_args if not (arg.lower() in ['-i', '-input'] or (ffmpeg_args.index(arg) > 0 and ffmpeg_args[ffmpeg_args.index(arg) - 1].lower() == '-i'))] - cmd = ['lib/ffmpeg/bin/ffmpeg', '-i', input_path] + filtered_args + [output_path] + ffmpeg_bin = find_ffmpeg() + hwaccel_args = get_hwaccel_args() + cmd = [ffmpeg_bin] + hwaccel_args + ['-i', input_path] + filtered_args + [output_path] if task_id and task_registry and task_lock: add_task_log(task_id, bi(f"执行FFmpeg命令: {' '.join(cmd)}", f"Execute FFmpeg command: {' '.join(cmd)}"), task_registry, task_lock) logging.info(bi(f"执行FFmpeg命令: {' '.join(cmd)}", f"Execute FFmpeg command: {' '.join(cmd)}")) - # 使用Popen来实时捕获输出 process = subprocess.Popen( cmd, stdout=subprocess.PIPE, - stderr=subprocess.STDOUT, # 将stderr重定向到stdout + stderr=subprocess.STDOUT, universal_newlines=True, encoding='utf-8', errors='replace', bufsize=1 ) - # 实时读取输出 while True: output = process.stdout.readline() if output == '' and process.poll() is not None: @@ -106,14 +194,11 @@ def execute_ffmpeg(input_path, output_path, ffmpeg_args, task_id=None, task_regi output = output.strip() if output and task_id and task_registry and task_lock: add_task_log(task_id, output, task_registry, task_lock) - # 简单的进度解析（可以根据FFmpeg的输出格式进行优化） if 'time=' in output and 'bitrate=' in output: - # 这里可以添加更复杂的进度解析逻辑 if task_id and task_registry and task_lock: with task_lock: if task_id in task_registry: - # 处理进度从50%开始到100% - task_registry[task_id]['progress'] = 50 + 50 // 2 # 简单示例 + task_registry[task_id]['progress'] = 50 + 50 // 2 returncode = process.poll() if returncode != 0: @@ -135,7 +220,78 @@ def execute_ffmpeg(input_path, output_path, ffmpeg_args, task_id=None, task_regi logging.error(error_msg) raise +def execute_ffmpeg_streaming(input_path, ffmpeg_args, task_id=None, task_registry=None, task_lock=None): + try: + filtered_args = [arg for arg in ffmpeg_args if not (arg.lower() in ['-i', '-input'] or + (ffmpeg_args.index(arg) > 0 and ffmpeg_args[ffmpeg_args.index(arg) - 1].lower() == '-i'))] + + ffmpeg_bin = find_ffmpeg() + hwaccel_args = get_hwaccel_args() + cmd = [ffmpeg_bin] + hwaccel_args + ['-i', input_path] + filtered_args + ['-f', 'mp4', '-'] + + if task_id and task_registry and task_lock: + add_task_log(task_id, bi(f"执行FFmpeg流式命令: {' '.join(cmd)}", f"Execute FFmpeg streaming command: {' '.join(cmd)}"), task_registry, task_lock) + + logging.info(bi(f"执行FFmpeg流式命令: {' '.join(cmd)}", f"Execute FFmpeg streaming command: {' '.join(cmd)}")) + + process = subprocess.Popen( + cmd, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + bufsize=get_streaming_buffer_size() + ) + + buffer = b'' + while True: + chunk = process.stdout.read(get_streaming_buffer_size()) + if chunk: + buffer += chunk + while len(buffer) >= get_streaming_buffer_size(): + yield buffer[:get_streaming_buffer_size()] + buffer = buffer[get_streaming_buffer_size():] + + if not chunk and process.poll() is not None: + break + + stderr_line = process.stderr.readline() + if stderr_line: + stderr_line = stderr_line.decode('utf-8', errors='replace').strip() + if stderr_line and task_id and task_registry and task_lock: + add_task_log(task_id, stderr_line, task_registry, task_lock) + if 'time=' in stderr_line and 'bitrate=' in stderr_line and task_id and task_registry and task_lock: + with task_lock: + if task_id in task_registry: + task_registry[task_id]['progress'] = 50 + 50 // 2 + + if buffer: + yield buffer + + returncode = process.poll() + if returncode != 0: + stderr_output = process.stderr.read().decode('utf-8', errors='replace') if process.stderr else '' + error_msg = bi(f"FFmpeg流式处理失败，返回码: {returncode}", f"FFmpeg streaming processing failed, return code: {returncode}") + if stderr_output: + error_msg += f" - {stderr_output[:200]}" + if task_id and task_registry and task_lock: + add_task_log(task_id, error_msg, task_registry, task_lock) + logging.error(error_msg) + raise Exception(error_msg) + + if task_id and task_registry and task_lock: + add_task_log(task_id, bi("FFmpeg流式处理成功完成", "FFmpeg streaming processing completed successfully"), task_registry, task_lock) + + logging.info(bi("FFmpeg流式处理成功完成", "FFmpeg streaming processing completed successfully")) + + except Exception as e: + error_msg = bi(f"执行FFmpeg流式处理时出错: {e}", f"Error while executing FFmpeg streaming: {e}") + if task_id and task_registry and task_lock: + add_task_log(task_id, error_msg, task_registry, task_lock) + logging.error(error_msg) + raise + def process_ffmpeg(data, file_registry, file_lock, task_id=None, task_registry=None, task_lock=None): + streaming = get_streaming_enabled() + try: dir_name = task_id if task_id else str(uuid.uuid4())[:8] temp_dir = os.path.join(UPLOAD_FOLDER, dir_name) @@ -156,12 +312,29 @@ def process_ffmpeg(data, file_registry, file_lock, task_id=None, task_registry=N output_path = os.path.join(temp_dir, output_filename) if task_id and task_registry and task_lock: - add_task_log(task_id, bi(f"开始FFmpeg处理，输出格式: {output_format}", f"Start FFmpeg processing, output format: {output_format}"), task_registry, task_lock) + mode_text = "流式" if streaming else "非流式" + add_task_log(task_id, bi(f"开始FFmpeg{mode_text}处理，输出格式: {output_format}", f"Start FFmpeg {mode_text} processing, output format: {output_format}"), task_registry, task_lock) with task_lock: if task_id in task_registry: - task_registry[task_id]['progress'] = 50 # 开始处理，进度50% + task_registry[task_id]['progress'] = 50 - execute_ffmpeg(input_path, output_path, ffmpeg_args, task_id, task_registry, task_lock) + if streaming: + if task_id and task_registry and task_lock: + add_task_log(task_id, bi("使用流式处理模式", "Using streaming mode"), task_registry, task_lock) + logging.info(bi("使用流式处理模式", "Using streaming mode")) + + output_data = b'' + for chunk in execute_ffmpeg_streaming(input_path, ffmpeg_args, task_id, task_registry, task_lock): + output_data += chunk + + with open(output_path, 'wb') as f: + f.write(output_data) + else: + if task_id and task_registry and task_lock: + add_task_log(task_id, bi("使用非流式处理模式（转码完成后发送）", "Using non-streaming mode (send after completion)"), task_registry, task_lock) + logging.info(bi("使用非流式处理模式（转码完成后发送）", "Using non-streaming mode (send after completion)")) + + execute_ffmpeg(input_path, output_path, ffmpeg_args, task_id, task_registry, task_lock) with file_lock: file_registry[output_id] = { @@ -176,11 +349,10 @@ def process_ffmpeg(data, file_registry, file_lock, task_id=None, task_registry=N add_task_log(task_id, bi(f"已注册新文件ID: {output_id}, 路径: {output_path}", f"Registered new file ID: {output_id}, path: {output_path}"), task_registry, task_lock) with task_lock: if task_id in task_registry: - task_registry[task_id]['progress'] = 100 # 处理完成，进度100% + task_registry[task_id]['progress'] = 100 logging.info(bi(f"已注册新文件ID: {output_id}, 路径: {output_path}", f"Registered new file ID: {output_id}, path: {output_path}")) - # 返回临时目录路径以便在主函数中删除 return { 'status': 'success', 'download_url': f"http://newgmapi.liulikeji.cn/download/{output_id}/{output_filename}", diff --git a/hwaccel_detect.py b/hwaccel_detect.py new file mode 100644 index 0000000..74f89b8 --- /dev/null +++ b/hwaccel_detect.py @@ -0,0 +1,251 @@ +import subprocess +import os +import sys +import logging + +def is_raspberry_pi(): + try: + with open('/proc/cpuinfo', 'r') as f: + content = f.read() + if 'Raspberry Pi' in content or 'BCM2835' in content or 'BCM2711' in content: + return True + except Exception: + pass + return False + +def find_v4l2_device(): + for i in range(10, 16): + device_path = f'/dev/video{i}' + if os.path.exists(device_path): + try: + result = subprocess.run( + ['v4l2-ctl', '--device', device_path, '--all'], + capture_output=True, + text=True, + timeout=5 + ) + if result.returncode == 0 and 'codec' in result.stdout.lower(): + return device_path + except Exception: + pass + return device_path + + for i in range(32): + device_path = f'/dev/video{i}' + if os.path.exists(device_path): + return device_path + + return '/dev/video11' + +def detect_hardware_acceleration(): + hwaccel_methods = [] + + if sys.platform.startswith('win'): + ffmpeg_path = 'lib/ffmpeg/bin/ffmpeg.exe' + else: + ffmpeg_path = 'lib/ffmpeg/bin/ffmpeg' + + if not os.path.exists(ffmpeg_path): + try: + result = subprocess.run(['which', 'ffmpeg'], capture_output=True, text=True) + if result.returncode == 0: + ffmpeg_path = result.stdout.strip() + else: + logging.warning(bi("FFmpeg未找到，跳过硬件加速检测", "FFmpeg not found, skipping hardware acceleration detection")) + return [] + except Exception: + logging.warning(bi("FFmpeg未找到，跳过硬件加速检测", "FFmpeg not found, skipping hardware acceleration detection")) + return [] + + try: + result = subprocess.run( + [ffmpeg_path, '-hwaccels'], + capture_output=True, + text=True, + timeout=10 + ) + + if result.returncode == 0: + output = result.stdout.strip() + for line in output.split('\n'): + line = line.strip() + if line and line != 'Hardware acceleration methods:': + hwaccel_methods.append(line) + + logging.info(bi(f"检测到可用的硬件加速方式: {hwaccel_methods}", f"Detected available hardware acceleration methods: {hwaccel_methods}")) + except Exception as e: + logging.error(bi(f"检测硬件加速时出错: {e}", f"Error detecting hardware acceleration: {e}")) + + if is_raspberry_pi(): + result = subprocess.run( + [ffmpeg_path, '-hide_banner', '-encoders'], + capture_output=True, + text=True, + timeout=10 + ) + + if result.returncode == 0: + if 'h264_v4l2m2m' in result.stdout: + if 'v4l2m2m' not in hwaccel_methods: + hwaccel_methods.append('v4l2m2m') + logging.info(bi("树莓派特殊处理：检测到 v4l2m2m 编码器，添加到可用硬件加速列表", "Raspberry Pi special handling: detected v4l2m2m encoder, added to available hardware acceleration list")) + + return hwaccel_methods + +def get_best_hwaccel(): + available = detect_hardware_acceleration() + + if is_raspberry_pi(): + priority_order = ['v4l2m2m', 'drm', 'mmal', 'vulkan', 'vdpau', 'vaapi', 'cuda', 'qsv'] + else: + priority_order = ['cuda', 'vaapi', 'qsv', 'videotoolbox', 'dxva2', 'd3d11va', 'amf', 'v4l2m2m', 'drm', 'vulkan', 'vdpau'] + + for method in priority_order: + if method in available: + logging.info(bi(f"选择硬件加速方式: {method}", f"Selected hardware acceleration method: {method}")) + return method + + return None + +def get_hwaccel_args(hwaccel_method=None): + if not hwaccel_method: + hwaccel_method = get_best_hwaccel() + + if not hwaccel_method: + return [] + + args = [] + + if hwaccel_method == 'vaapi': + args.extend(['-hwaccel', 'vaapi', '-hwaccel_device', '/dev/dri/renderD128']) + elif hwaccel_method == 'qsv': + args.extend(['-hwaccel', 'qsv', '-hwaccel_device', 'auto']) + elif hwaccel_method == 'mmal': + args.extend(['-hwaccel', 'mmal', '-hwaccel_device', 'vc.smem']) + elif hwaccel_method == 'v4l2m2m': + device_path = find_v4l2_device() + args.extend(['-hwaccel_device', device_path]) + elif hwaccel_method == 'drm': + args.extend(['-hwaccel', 'drm', '-hwaccel_device', '/dev/dri/renderD128']) + elif hwaccel_method == 'vdpau': + args.extend(['-hwaccel', 'vdpau', '-hwaccel_device', 'auto']) + elif hwaccel_method == 'vulkan': + args.extend(['-hwaccel', 'vulkan', '-hwaccel_device', 'auto']) + + return args + +def test_hwaccel(hwaccel_method): + if sys.platform.startswith('win'): + ffmpeg_path = 'lib/ffmpeg/bin/ffmpeg.exe' + else: + ffmpeg_path = 'lib/ffmpeg/bin/ffmpeg' + + if not os.path.exists(ffmpeg_path): + try: + result = subprocess.run(['which', 'ffmpeg'], capture_output=True, text=True) + if result.returncode == 0: + ffmpeg_path = result.stdout.strip() + else: + logging.warning(bi(f"本地路径未找到FFmpeg，跳过{hwaccel_method}测试", f"FFmpeg not found in local path, skipping test for {hwaccel_method}")) + return True + except Exception: + logging.warning(bi(f"无法找到FFmpeg，跳过{hwaccel_method}测试", f"Cannot find FFmpeg, skipping test for {hwaccel_method}")) + return True + + try: + if hwaccel_method == 'v4l2m2m': + device_path = find_v4l2_device() + cmd = [ + ffmpeg_path, + '-f', 'lavfi', + '-i', 'color=c=black:s=1920x1080:r=1', + '-t', '1', + '-c:v', 'h264_v4l2m2m', + '-f', 'null', + '-' + ] + elif hwaccel_method == 'vaapi': + cmd = [ + ffmpeg_path, + '-hwaccel', 'vaapi', + '-hwaccel_device', '/dev/dri/renderD128', + '-f', 'lavfi', + '-i', 'color=c=black:s=1920x1080:r=1', + '-t', '1', + '-c:v', 'h264_vaapi', + '-f', 'null', + '-' + ] + elif hwaccel_method == 'drm': + cmd = [ + ffmpeg_path, + '-hwaccel', 'drm', + '-hwaccel_device', '/dev/dri/renderD128', + '-f', 'lavfi', + '-i', 'color=c=black:s=1920x1080:r=1', + '-t', '1', + '-c:v', 'h264', + '-f', 'null', + '-' + ] + elif hwaccel_method == 'mmal': + cmd = [ + ffmpeg_path, + '-hwaccel', 'mmal', + '-hwaccel_device', 'vc.smem', + '-f', 'lavfi', + '-i', 'color=c=black:s=1920x1080:r=1', + '-t', '1', + '-c:v', 'h264', + '-f', 'null', + '-' + ] + else: + cmd = [ + ffmpeg_path, + '-hwaccel', hwaccel_method, + '-f', 'lavfi', + '-i', 'color=c=black:s=1920x1080:r=1', + '-t', '1', + '-c:v', 'h264', + '-f', 'null', + '-' + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + timeout=20 + ) + + if result.returncode == 0: + logging.info(bi(f"硬件加速 {hwaccel_method} 测试通过", f"Hardware acceleration {hwaccel_method} test passed")) + return True + + error_stderr = result.stderr.lower() + if 'invalid device' in error_stderr or 'not found' in error_stderr or \ + 'no such device' in error_stderr or 'failed to open' in error_stderr or \ + 'could not find' in error_stderr or 'permission denied' in error_stderr: + logging.warning(bi(f"硬件加速 {hwaccel_method} 设备无效", f"Hardware acceleration {hwaccel_method} device invalid: {result.stderr.strip()[:100]}")) + return False + + logging.warning(bi(f"硬件加速 {hwaccel_method} 测试返回码非零但非致命: {result.returncode}", f"Hardware acceleration {hwaccel_method} test returned non-zero but non-fatal: {result.returncode}")) + return True + except Exception as e: + logging.error(bi(f"测试硬件加速 {hwaccel_method} 失败: {e}", f"Failed to test hardware acceleration {hwaccel_method}: {e}")) + return False + +def get_working_hwaccel(): + available = detect_hardware_acceleration() + + if is_raspberry_pi(): + priority_order = ['v4l2m2m', 'drm', 'mmal', 'vulkan', 'vdpau', 'vaapi', 'cuda', 'qsv'] + else: + priority_order = ['cuda', 'vaapi', 'qsv', 'videotoolbox', 'dxva2', 'd3d11va', 'amf', 'v4l2m2m', 'drm', 'vulkan', 'vdpau'] + + for method in priority_order: + if method in available and test_hwaccel(method): + return method + + return None \ No newline at end of file diff --git a/lib_downloader.py b/lib_downloader.py new file mode 100644 index 0000000..06a8cd9 --- /dev/null +++ b/lib_downloader.py @@ -0,0 +1,644 @@ +import os +import sys +import platform +import tarfile +import zipfile +import logging +import hashlib +import time +import subprocess +import shutil +from pathlib import Path +from datetime import datetime + +LIB_DIR = os.path.join(os.path.dirname(__file__), 'lib') +DOWNLOAD_CACHE = os.path.join(LIB_DIR, '.download_cache') + +DATA_DIR = os.path.join(os.path.dirname(__file__), 'data') + +def bi(zh, en): + return f"{zh} | {en}" + +def get_directory_size(directory): + """计算目录大小（字节）""" + total_size = 0 + for dirpath, dirnames, filenames in os.walk(directory): + for f in filenames: + try: + fp = os.path.join(dirpath, f) + total_size += os.path.getsize(fp) + except Exception: + pass + return total_size + +def format_size(bytes_size): + """格式化文件大小""" + if bytes_size < 1024: + return f"{bytes_size} B" + elif bytes_size < 1024 * 1024: + return f"{bytes_size / 1024:.2f} KB" + elif bytes_size < 1024 * 1024 * 1024: + return f"{bytes_size / (1024 * 1024):.2f} MB" + else: + return f"{bytes_size / (1024 * 1024 * 1024):.2f} GB" + +def cleanup_download_cache(): + """清理下载缓存（仅按时间清理）""" + if not os.path.exists(DOWNLOAD_CACHE): + return + + try: + from config import get_cache_retention_hours + max_age_hours = get_cache_retention_hours() + except Exception: + max_age_hours = 24 + + current_size = get_directory_size(DOWNLOAD_CACHE) + + print(f"\n🧹 清理下载缓存") + print(f" 当前缓存大小: {format_size(current_size)}") + print(f" 保留时间: {max_age_hours} 小时") + + files_to_delete = [] + for filename in os.listdir(DOWNLOAD_CACHE): + filepath = os.path.join(DOWNLOAD_CACHE, filename) + if os.path.isfile(filepath): + try: + mtime = os.path.getmtime(filepath) + age_hours = (time.time() - mtime) / 3600 + files_to_delete.append((mtime, filepath, age_hours)) + except Exception: + pass + + files_to_delete.sort(key=lambda x: x[0]) + + deleted_size = 0 + deleted_count = 0 + + for mtime, filepath, age_hours in files_to_delete: + if age_hours > max_age_hours: + try: + file_size = os.path.getsize(filepath) + os.remove(filepath) + deleted_size += file_size + deleted_count += 1 + print(f" 删除: {os.path.basename(filepath)} ({format_size(file_size)})") + except Exception as e: + print(f" 删除失败: {os.path.basename(filepath)} - {e}") + + if deleted_count > 0: + print(f" 已清理 {deleted_count} 个文件，释放 {format_size(deleted_size)} 空间") + else: + print(f" 无需清理") + + logging.info(bi(f"下载缓存清理完成: 删除 {deleted_count} 个文件，释放 {format_size(deleted_size)}", f"Download cache cleanup completed: deleted {deleted_count} files, freed {format_size(deleted_size)}")) + +def cleanup_video_cache(): + """清理视频帧缓存（按时间和大小清理）""" + try: + from config import get_cache_retention_hours, get_video_cache_max_mb + max_age_hours = get_cache_retention_hours() + max_size_mb = get_video_cache_max_mb() + except Exception: + max_age_hours = 24 + max_size_mb = 2000 + + videoframe_dir = os.path.join(DATA_DIR, 'videoframe') + + if not os.path.exists(videoframe_dir): + return + + current_size = get_directory_size(videoframe_dir) + current_size_mb = current_size / (1024 * 1024) + + print(f"\n🧹 清理视频帧缓存") + print(f" 当前缓存大小: {format_size(current_size)}") + print(f" 最大缓存: {max_size_mb} MB") + print(f" 保留时间: {max_age_hours} 小时") + + dirs_to_delete = [] + + for item in os.listdir(videoframe_dir): + item_path = os.path.join(videoframe_dir, item) + if os.path.isdir(item_path): + try: + mtime = os.path.getmtime(item_path) + age_hours = (time.time() - mtime) / 3600 + dir_size = get_directory_size(item_path) + dirs_to_delete.append((mtime, item_path, age_hours, dir_size)) + except Exception: + pass + + dirs_to_delete.sort(key=lambda x: x[0]) + + deleted_size = 0 + deleted_count = 0 + + for mtime, dir_path, age_hours, dir_size in dirs_to_delete: + if age_hours > max_age_hours or current_size_mb > max_size_mb: + try: + shutil.rmtree(dir_path) + deleted_size += dir_size + deleted_count += 1 + current_size_mb -= dir_size / (1024 * 1024) + print(f" 删除: {os.path.basename(dir_path)} ({format_size(dir_size)})") + + if current_size_mb <= max_size_mb: + break + except Exception as e: + print(f" 删除失败: {os.path.basename(dir_path)} - {e}") + + if deleted_count > 0: + print(f" 已清理 {deleted_count} 个任务目录，释放 {format_size(deleted_size)} 空间") + else: + print(f" 无需清理") + + logging.info(bi(f"视频帧缓存清理完成: 删除 {deleted_count} 个任务，释放 {format_size(deleted_size)}", f"Video frame cache cleanup completed: deleted {deleted_count} tasks, freed {format_size(deleted_size)}")) + +def detect_linux_distro(): + """检测Linux发行版""" + if os.path.exists('/etc/os-release'): + with open('/etc/os-release', 'r') as f: + for line in f: + if line.startswith('ID='): + distro = line.strip().split('=')[1].strip('"') + return distro.lower() + return 'unknown' + +def run_command(cmd, description=""): + """运行命令并返回结果""" + try: + print(f" [{description}] 执行命令: {' '.join(cmd)}") + result = subprocess.run(cmd, capture_output=True, text=True, check=True) + print(f" [{description}] 命令执行成功") + return True, result.stdout + except subprocess.CalledProcessError as e: + print(f" [{description}] 命令执行失败: {e.stderr}") + return False, e.stderr + except Exception as e: + print(f" [{description}] 命令执行异常: {e}") + return False, str(e) + +def install_with_package_manager(lib_name): + """使用包管理器安装依赖""" + distro = detect_linux_distro() + print(f" [{lib_name}] 检测到发行版: {distro}") + + package_commands = { + 'debian': { + 'update': ['apt-get', 'update'], + 'ffmpeg': ['apt-get', 'install', '-y', 'ffmpeg'], + 'sanjuuni': None + }, + 'ubuntu': { + 'update': ['apt-get', 'update'], + 'ffmpeg': ['apt-get', 'install', '-y', 'ffmpeg'], + 'sanjuuni': None + }, + 'arch': { + 'update': ['pacman', '-Syu', '--noconfirm'], + 'ffmpeg': ['pacman', '-S', '--noconfirm', 'ffmpeg'], + 'sanjuuni': ['pacman', '-S', '--noconfirm', 'sanjuuni'] + }, + 'fedora': { + 'update': ['dnf', 'check-update'], + 'ffmpeg': ['dnf', 'install', '-y', 'ffmpeg'], + 'sanjuuni': None + }, + 'centos': { + 'update': ['yum', 'check-update'], + 'ffmpeg': ['yum', 'install', '-y', 'ffmpeg'], + 'sanjuuni': None + }, + 'opensuse': { + 'update': ['zypper', 'refresh'], + 'ffmpeg': ['zypper', 'install', '-y', 'ffmpeg'], + 'sanjuuni': None + } + } + + if distro not in package_commands: + print(f" [{lib_name}] 不支持的发行版: {distro}，使用下载方式安装") + return False + + commands = package_commands[distro] + cmd = commands.get(lib_name) + + if cmd is None: + print(f" [{lib_name}] {distro} 不支持通过包管理器安装，使用下载方式") + return False + + if os.geteuid() != 0: + print(f" [{lib_name}] 需要root权限，使用sudo") + cmd = ['sudo'] + cmd + + success, output = run_command(cmd, lib_name) + return success + +ARCHITECTURE_MAP = { + ('Windows', 'AMD64'): 'windows-x86_64', + ('Windows', 'x86'): 'windows-x86', + ('Linux', 'x86_64'): 'linux-x86_64', + ('Linux', 'aarch64'): 'linux-aarch64', + ('Linux', 'armv7l'): 'linux-armv7', + ('Darwin', 'x86_64'): 'macos-x86_64', + ('Darwin', 'arm64'): 'macos-arm64', +} + +DEFAULT_DOWNLOAD_URLS = { + 'ffmpeg': { + 'windows-x86_64': 'https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-win64-gpl.zip', + 'windows-x86': 'https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-win32-gpl.zip', + 'linux-x86_64': 'https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-linux64-gpl.tar.xz', + 'linux-aarch64': 'https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-linuxarm64-gpl.tar.xz', + 'macos-x86_64': 'https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-macos64-gpl.tar.xz', + 'macos-arm64': 'https://github.com/BtbN/FFmpeg-Builds/releases/download/latest/ffmpeg-master-latest-macosarm64-gpl.tar.xz', + }, + 'sanjuuni': { + 'windows-x86_64': 'https://github.com/MCJack123/sanjuuni/releases/latest/download/sanjuuni-Win64.zip', + } +} + +def get_system_architecture(): + system = platform.system() + machine = platform.machine() + + if system == 'Linux' and machine == 'armv7l': + arch = 'armv7l' + else: + arch = machine + + key = (system, arch) + return ARCHITECTURE_MAP.get(key, None) + +def format_speed(bytes_per_second): + if bytes_per_second < 1024: + return f"{bytes_per_second} B/s" + elif bytes_per_second < 1024 * 1024: + return f"{bytes_per_second / 1024:.2f} KB/s" + else: + return f"{bytes_per_second / (1024 * 1024):.2f} MB/s" + +def wait_for_input(prompt="按 Enter 键继续..."): + try: + input(prompt) + except EOFError: + pass + +def is_library_installed(lib_name, target_arch): + if lib_name == 'ffmpeg': + bin_dir = os.path.join(LIB_DIR, 'ffmpeg', 'bin') + if sys.platform.startswith('win'): + exe_path = os.path.join(bin_dir, 'ffmpeg.exe') + else: + exe_path = os.path.join(bin_dir, 'ffmpeg') + + if os.path.exists(exe_path): + return True + + if sys.platform.startswith('linux'): + try: + result = subprocess.run(['which', 'ffmpeg'], capture_output=True) + if result.returncode == 0: + print(f" [FFmpeg] 在系统路径中找到: {result.stdout.decode().strip()}") + return True + except Exception: + pass + + return False + + elif lib_name == 'sanjuuni': + if sys.platform.startswith('win'): + exe_name = 'sanjuuni.exe' + else: + exe_name = 'sanjuuni' + + lib_path = os.path.join(LIB_DIR, 'sanjuuni', exe_name) + if os.path.exists(lib_path): + return True + + for path in os.environ.get('PATH', '').split(os.pathsep): + exe_path = os.path.join(path, exe_name) + if os.path.exists(exe_path): + return True + + return False + + return False + +def download_file_with_retry(url, dest_path, lib_name="", max_retries=3, retry_delay=5): + import requests + + for attempt in range(max_retries): + try: + logging.info(bi(f"[{lib_name}] 开始下载 (尝试 {attempt + 1}/{max_retries}): {url}", f"[{lib_name}] Starting download (attempt {attempt + 1}/{max_retries}): {url}")) + print(f" [{lib_name}] 正在下载... (尝试 {attempt + 1}/{max_retries})") + + response = requests.get(url, stream=True, timeout=30) + response.raise_for_status() + + total_size = int(response.headers.get('content-length', 0)) + downloaded_size = 0 + start_time = time.time() + last_time = start_time + last_downloaded = 0 + progress_bar_length = 40 + + os.makedirs(os.path.dirname(dest_path), exist_ok=True) + + if total_size > 0: + print(f" [{lib_name}] 文件大小: {format_size(total_size)}") + + with open(dest_path, 'wb') as f: + for chunk in response.iter_content(chunk_size=8192): + if chunk: + f.write(chunk) + downloaded_size += len(chunk) + + current_time = time.time() + if current_time - last_time >= 0.5 or downloaded_size == total_size: + elapsed_time = current_time - start_time + speed = downloaded_size / elapsed_time if elapsed_time > 0 else 0 + + if total_size > 0: + progress = downloaded_size / total_size + progress_percent = int(progress * 100) + filled_length = int(progress_bar_length * progress) + bar = '█' * filled_length + '░' * (progress_bar_length - filled_length) + + eta_seconds = (total_size - downloaded_size) / speed if speed > 0 else 0 + eta_minutes = int(eta_seconds // 60) + eta_remaining = int(eta_seconds % 60) + + print(f"\r [{lib_name}] [{bar}] {progress_percent}% ({format_size(downloaded_size)}/{format_size(total_size)}) - {format_speed(speed)} - ETA: {eta_minutes:02d}:{eta_remaining:02d}", end='') + else: + print(f"\r [{lib_name}] 已下载: {format_size(downloaded_size)} - {format_speed(speed)}", end='') + + last_time = current_time + last_downloaded = downloaded_size + + elapsed_time = time.time() - start_time + avg_speed = downloaded_size / elapsed_time if elapsed_time > 0 else 0 + + print() + print(f" [{lib_name}] 下载完成!") + print(f" [{lib_name}] 文件保存到: {dest_path}") + print(f" [{lib_name}] 下载大小: {format_size(downloaded_size)}") + print(f" [{lib_name}] 平均速度: {format_speed(avg_speed)}") + print(f" [{lib_name}] 耗时: {int(elapsed_time // 60)}分{int(elapsed_time % 60)}秒") + + logging.info(bi(f"[{lib_name}] 下载完成: {dest_path} ({format_size(downloaded_size)})", f"[{lib_name}] Download completed: {dest_path} ({format_size(downloaded_size)})")) + return dest_path + + except Exception as e: + error_msg = f"[{lib_name}] 下载失败 (尝试 {attempt + 1}/{max_retries}): {e}" + print(f"\n [{lib_name}] ❌ {error_msg}") + print(f" [{lib_name}] 错误类型: {type(e).__name__}") + logging.error(error_msg) + + if os.path.exists(dest_path): + os.remove(dest_path) + + if attempt < max_retries - 1: + print(f" [{lib_name}] 将在 {retry_delay} 秒后重试...") + time.sleep(retry_delay) + else: + print(f" [{lib_name}] 已达到最大重试次数 ({max_retries}次)") + wait_for_input("按 Enter 键继续...") + raise + +def download_file(url, dest_path, lib_name=""): + return download_file_with_retry(url, dest_path, lib_name, max_retries=3, retry_delay=5) + +def extract_archive(file_path, dest_dir, lib_name=""): + try: + file_size = os.path.getsize(file_path) + print(f" [{lib_name}] 开始解压...") + print(f" [{lib_name}] 压缩包大小: {format_size(file_size)}") + logging.info(bi(f"[{lib_name}] 开始解压: {file_path}", f"[{lib_name}] Starting extraction: {file_path}")) + + start_time = time.time() + + if file_path.endswith('.zip'): + with zipfile.ZipFile(file_path, 'r') as zip_ref: + file_count = len(zip_ref.namelist()) + print(f" [{lib_name}] 包含 {file_count} 个文件") + zip_ref.extractall(dest_dir) + elif file_path.endswith('.tar.gz'): + with tarfile.open(file_path, 'r:gz') as tar_ref: + file_count = len(tar_ref.getmembers()) + print(f" [{lib_name}] 包含 {file_count} 个文件") + tar_ref.extractall(dest_dir) + elif file_path.endswith('.tar.xz'): + with tarfile.open(file_path, 'r:xz') as tar_ref: + file_count = len(tar_ref.getmembers()) + print(f" [{lib_name}] 包含 {file_count} 个文件") + tar_ref.extractall(dest_dir) + else: + raise ValueError(f"[{lib_name}] 不支持的压缩格式: {file_path}") + + elapsed_time = time.time() - start_time + + print(f" [{lib_name}] 解压完成!") + print(f" [{lib_name}] 解压到: {dest_dir}") + print(f" [{lib_name}] 耗时: {int(elapsed_time // 60)}分{int(elapsed_time % 60)}秒") + logging.info(bi(f"[{lib_name}] 解压完成: {dest_dir}", f"[{lib_name}] Extraction completed: {dest_dir}")) + + except Exception as e: + error_msg = f"[{lib_name}] 解压失败: {e}" + print(f" [{lib_name}] ❌ {error_msg}") + print(f" [{lib_name}] 错误类型: {type(e).__name__}") + logging.error(error_msg) + wait_for_input("按 Enter 键继续...") + raise + +def install_ffmpeg(target_arch): + ffmpeg_dir = os.path.join(LIB_DIR, 'ffmpeg') + bin_dir = os.path.join(ffmpeg_dir, 'bin') + + print(f"\n📦 安装 FFmpeg ({target_arch})") + print("-" * 50) + + if os.path.exists(bin_dir): + import shutil + print(f" [FFmpeg] 检测到已安装，正在清理旧版本...") + logging.info(bi("[FFmpeg] 检测到已安装，清理旧版本", "[FFmpeg] Already installed, cleaning up old version")) + shutil.rmtree(ffmpeg_dir) + + if sys.platform.startswith('linux'): + print(f" [FFmpeg] Linux系统，尝试使用包管理器安装...") + if install_with_package_manager('ffmpeg'): + print(f" [FFmpeg] ✅ 通过包管理器安装成功") + return + print(f" [FFmpeg] 包管理器安装失败，使用下载方式") + + url = DEFAULT_DOWNLOAD_URLS['ffmpeg'].get(target_arch) + if not url: + raise ValueError(f"[FFmpeg] 不支持当前架构: {target_arch}") + + print(f" [FFmpeg] 下载地址: {url}") + cache_file = os.path.join(DOWNLOAD_CACHE, f"ffmpeg-{target_arch}.zip") + + download_file(url, cache_file, "FFmpeg") + extract_archive(cache_file, ffmpeg_dir, "FFmpeg") + + extracted_contents = os.listdir(ffmpeg_dir) + if len(extracted_contents) == 1 and os.path.isdir(os.path.join(ffmpeg_dir, extracted_contents[0])): + import shutil + inner_dir = os.path.join(ffmpeg_dir, extracted_contents[0]) + print(f" [FFmpeg] 整理文件结构...") + for item in os.listdir(inner_dir): + src = os.path.join(inner_dir, item) + dst = os.path.join(ffmpeg_dir, item) + if os.path.exists(dst): + if os.path.isdir(dst): + shutil.rmtree(dst) + else: + os.remove(dst) + shutil.move(src, ffmpeg_dir) + shutil.rmtree(inner_dir) + + ffmpeg_bin = 'ffmpeg.exe' if sys.platform.startswith('win') else 'ffmpeg' + final_path = os.path.join(bin_dir, ffmpeg_bin) + + if not os.path.exists(final_path): + raise RuntimeError(f"[FFmpeg] 安装失败，未找到可执行文件: {final_path}") + + os.chmod(final_path, 0o755) + + final_size = sum(os.path.getsize(os.path.join(bin_dir, f)) for f in os.listdir(bin_dir) if os.path.isfile(os.path.join(bin_dir, f))) + print(f" [FFmpeg] ✅ 安装成功!") + print(f" [FFmpeg] 可执行文件: {final_path}") + print(f" [FFmpeg] 安装大小: {format_size(final_size)}") + logging.info(bi(f"[FFmpeg] 安装成功: {final_path}", f"[FFmpeg] Installation successful: {final_path}")) + print("-" * 50) + +def install_sanjuuni(target_arch): + sanjuuni_dir = os.path.join(LIB_DIR, 'sanjuuni') + + print(f"\n📦 安装 Sanjuuni ({target_arch})") + print("-" * 50) + + url = DEFAULT_DOWNLOAD_URLS['sanjuuni'].get(target_arch) + + if not url: + print(f" [Sanjuuni] ⚠️ 当前架构不支持自动下载") + print(f" [Sanjuuni] 请手动安装 sanjuuni:") + print() + print(f" [Sanjuuni] Arch Linux (AUR):") + print(f" yay -S sanjuuni") + print() + print(f" [Sanjuuni] Nix/NixOS:") + print(f" nix-channel --update") + print(f" nix-shell -p sanjuuni") + print() + print(f" [Sanjuuni] 其他系统请从源码编译:") + print(f" git clone https://github.com/MCJack123/sanjuuni") + print(f" cd sanjuuni") + print(f" mkdir build && cd build") + print(f" cmake ..") + print(f" make") + print() + print(f" [Sanjuuni] 安装完成后，请确保 sanjuuni 可执行文件在系统 PATH 中") + logging.warning(bi(f"[Sanjuuni] 当前架构 {target_arch} 需要手动安装", f"[Sanjuuni] Current architecture {target_arch} requires manual installation")) + wait_for_input("按 Enter 键继续...") + return + + if os.path.exists(sanjuuni_dir): + import shutil + print(f" [Sanjuuni] 检测到已安装，正在清理旧版本...") + logging.info(bi("[Sanjuuni] 检测到已安装，清理旧版本", "[Sanjuuni] Already installed, cleaning up old version")) + shutil.rmtree(sanjuuni_dir) + + print(f" [Sanjuuni] 下载地址: {url}") + cache_file = os.path.join(DOWNLOAD_CACHE, f"sanjuuni-{target_arch}.zip") + + download_file(url, cache_file, "Sanjuuni") + extract_archive(cache_file, sanjuuni_dir, "Sanjuuni") + + sanjuuni_bin = 'sanjuuni.exe' if sys.platform.startswith('win') else 'sanjuuni' + final_path = os.path.join(sanjuuni_dir, sanjuuni_bin) + + if not os.path.exists(final_path): + extracted_contents = os.listdir(sanjuuni_dir) + print(f" [Sanjuuni] 解压内容: {extracted_contents}") + for item in extracted_contents: + item_path = os.path.join(sanjuuni_dir, item) + if os.path.isdir(item_path): + inner_bin = os.path.join(item_path, sanjuuni_bin) + if os.path.exists(inner_bin): + import shutil + for file in os.listdir(item_path): + src = os.path.join(item_path, file) + dst = os.path.join(sanjuuni_dir, file) + if os.path.exists(dst): + if os.path.isdir(dst): + shutil.rmtree(dst) + else: + os.remove(dst) + shutil.move(src, sanjuuni_dir) + shutil.rmtree(item_path) + final_path = os.path.join(sanjuuni_dir, sanjuuni_bin) + break + + if not os.path.exists(final_path): + raise RuntimeError(f"[Sanjuuni] 安装失败，未找到可执行文件: {final_path}") + + os.chmod(final_path, 0o755) + + final_size = sum(os.path.getsize(os.path.join(sanjuuni_dir, f)) for f in os.listdir(sanjuuni_dir) if os.path.isfile(os.path.join(sanjuuni_dir, f))) + print(f" [Sanjuuni] ✅ 安装成功!") + print(f" [Sanjuuni] 可执行文件: {final_path}") + print(f" [Sanjuuni] 安装大小: {format_size(final_size)}") + logging.info(bi(f"[Sanjuuni] 安装成功: {final_path}", f"[Sanjuuni] Installation successful: {final_path}")) + print("-" * 50) + +def check_and_install_libraries(force=False): + print("\n🔍 运行库检查") + print("=" * 50) + + target_arch = get_system_architecture() + + if not target_arch: + error_msg = f"无法识别当前系统架构: {platform.system()} {platform.machine()}" + print(f"❌ {error_msg}") + logging.error(error_msg) + wait_for_input("按 Enter 键继续...") + return False + + print(f"系统架构: {platform.system()} {platform.machine()}") + print(f"目标架构: {target_arch}") + logging.info(bi(f"检测到系统架构: {target_arch}", f"Detected system architecture: {target_arch}")) + + os.makedirs(DOWNLOAD_CACHE, exist_ok=True) + print(f"缓存目录: {DOWNLOAD_CACHE}") + + needs_install = False + + if not is_library_installed('ffmpeg', target_arch) or force: + print(f"FFmpeg: ❌ 未安装") + install_ffmpeg(target_arch) + needs_install = True + else: + print(f"FFmpeg: ✅ 已安装") + + if not is_library_installed('sanjuuni', target_arch) or force: + print(f"Sanjuuni: ❌ 未安装") + install_sanjuuni(target_arch) + needs_install = True + else: + print(f"Sanjuuni: ✅ 已安装") + + if needs_install: + print("\n✅ 所有运行库已安装完成") + logging.info(bi("所有运行库已安装完成", "All runtime libraries installed successfully")) + cleanup_download_cache() + cleanup_video_cache() + elif not force: + print("\n✅ 所有运行库已安装完成") + logging.info(bi("所有运行库已安装完成", "All runtime libraries installed successfully")) + cleanup_download_cache() + cleanup_video_cache() + + print("=" * 50) + return True \ No newline at end of file diff --git a/main.py b/main.py index 6b2c7f2..1c6a821 100644 --- a/main.py +++ b/main.py @@ -1,6 +1,9 @@ import logging import json import os + +def bi(zh, en): + return f"{zh} | {en}" import hashlib from flask import Flask, request, jsonify import threading @@ -36,7 +39,7 @@ try: from video_frame_utils import process_video_frame_extraction from file_cleanup import start_cleanup_thread except ImportError as e: - logging.error(f"导入模块时出错: {e}") + logging.error(bi(f"导入模块时出错: {e}", f"Error importing module: {e}")) # 定义空函数作为备用 def process_ffmpeg(*args, **kwargs): return {'error': 'FFmpeg模块未正确导入'} @@ -134,10 +137,10 @@ def validate_request(data, api_name): for key in enter_parameter_table[api_name]: is_optional = key in optional_parameters.get(api_name, set()) if key not in data and key != "subtitle" and not is_optional: - logging.warning(f"请求中没有提供{key}参数") + logging.warning(bi(f"请求中没有提供{key}参数", f"Request missing {key} parameter")) return jsonify({'error': f"未提供{key}参数"}), 400 if key in data and isinstance(data[key], enter_parameter_table[api_name][key]) == False: - logging.warning(f"请求中{key}参数类型错误，应为{enter_parameter_table[api_name][key]}") + logging.warning(bi(f"请求中{key}参数类型错误，应为{enter_parameter_table[api_name][key]}", f"Request {key} parameter type error, expected {enter_parameter_table[api_name][key]}")) return jsonify({'error': f"{key}参数类型错误，您输入为{type(data[key])}，应为{enter_parameter_table[api_name][key]}" }), 400 return None, None @@ -172,7 +175,7 @@ def run_async_task(task_id, process_func, data): save_persistent_state() except Exception as e: - logging.error(f"任务 {task_id} 执行异常: {e}") + logging.error(bi(f"任务 {task_id} 执行异常: {e}", f"Task {task_id} execution exception: {e}")) with task_lock: if task_id in task_registry: task_registry[task_id]['status'] = 'error' @@ -186,7 +189,7 @@ def run_async_task(task_id, process_func, data): @app.route('/api/ffmpeg/async', methods=['POST']) def ffmpeg_async_api(): """创建异步FFmpeg任务""" - logging.info("收到异步FFmpeg API请求") + logging.info(bi("收到异步FFmpeg API请求", "Received async FFmpeg API request")) data = request.get_json() # 检测参数类型 @@ -222,7 +225,7 @@ def ffmpeg_async_api(): status_url = f"{scheme}://{host}/api/task/{task_id}" - logging.info(f"创建异步FFmpeg任务: {task_id}") + logging.info(bi(f"创建异步FFmpeg任务: {task_id}", f"Created async FFmpeg task: {task_id}")) return jsonify({ 'status': 'success', 'task_id': task_id, @@ -233,7 +236,7 @@ def ffmpeg_async_api(): @app.route('/api/sanjuuni/async', methods=['POST']) def sanjuuni_async_api(): """创建异步Sanjuuni任务""" - logging.info("收到异步Sanjuuni API请求") + logging.info(bi("收到异步Sanjuuni API请求", "Received async Sanjuuni API request")) data = request.get_json() # 检测参数类型 @@ -269,7 +272,7 @@ def sanjuuni_async_api(): status_url = f"{scheme}://{host}/api/task/{task_id}" - logging.info(f"创建异步Sanjuuni任务: {task_id}") + logging.info(bi(f"创建异步Sanjuuni任务: {task_id}", f"Created async Sanjuuni task: {task_id}")) return jsonify({ 'status': 'success', 'task_id': task_id, @@ -280,7 +283,7 @@ def sanjuuni_async_api(): @app.route('/api/video_frame/async', methods=['POST']) def video_frame_async_api(): """创建异步视频帧提取任务""" - logging.info("收到异步视频帧提取API请求") + logging.info(bi("收到异步视频帧提取API请求", "Received async video frame extraction API request")) data = request.get_json() # 检测参数类型 @@ -320,7 +323,7 @@ def video_frame_async_api(): renew_video_frame_task_expiry(task_info, now) task_info['create_time'] = now save_persistent_state() - logging.info(f"命中视频帧缓存，复用任务: {cached_task_id}, signature: {request_signature}") + logging.info(bi(f"命中视频帧缓存，复用任务: {cached_task_id}, signature: {request_signature}", f"Video frame cache hit, reusing task: {cached_task_id}, signature: {request_signature}")) return jsonify({ 'status': 'success', 'task_id': cached_task_id, @@ -394,7 +397,7 @@ def video_frame_async_api(): status_url = f"{scheme}://{host}/api/task/{task_id}" - logging.info(f"创建异步视频帧提取任务: {task_id}") + logging.info(bi(f"创建异步视频帧提取任务: {task_id}", f"Created async video frame extraction task: {task_id}")) return jsonify({ 'status': 'success', 'task_id': task_id, @@ -472,7 +475,7 @@ def get_task_status(task_id): # 原有的同步接口保持不变 @app.route('/api/ffmpeg', methods=['POST']) def ffmpeg_api(): - logging.info("收到FFmpeg API请求") + logging.info(bi("收到FFmpeg API请求", "Received FFmpeg API request")) data = request.get_json() # 检测参数类型 @@ -494,10 +497,10 @@ def ffmpeg_api(): # 等待处理结果 result = result_queue.get() if 'error' in result: - logging.error(f"处理过程中出错: {result['error']}") + logging.error(bi(f"处理过程中出错: {result['error']}", f"Error during processing: {result['error']}")) return jsonify({'status': 'error', 'error': result['error']}), 500 else: - logging.info(f"处理成功，返回下载URL: {result['download_url']}") + logging.info(bi(f"处理成功，返回下载URL: {result['download_url']}", f"Processing successful, returning download URL: {result['download_url']}")) return jsonify({ 'status': 'success', 'download_url': result['download_url'], @@ -506,7 +509,7 @@ def ffmpeg_api(): @app.route('/api/sanjuuni', methods=['POST']) def sanjuuni_api(): - logging.info("收到Sanjuuni API请求") + logging.info(bi("收到Sanjuuni API请求", "Received Sanjuuni API request")) data = request.get_json() # 检测参数类型 @@ -528,10 +531,10 @@ def sanjuuni_api(): # 等待处理结果 result = result_queue.get() if 'error' in result: - logging.error(f"处理过程中出错: {result['error']}") + logging.error(bi(f"处理过程中出错: {result['error']}", f"Error during processing: {result['error']}")) return jsonify({'status': 'error', 'error': result['error']}), 500 else: - logging.info(f"处理成功，返回下载URL: {result['download_url']}") + logging.info(bi(f"处理成功，返回下载URL: {result['download_url']}", f"Processing successful, returning download URL: {result['download_url']}")) return jsonify({ 'status': 'success', 'download_url': result['download_url'], @@ -540,10 +543,10 @@ def sanjuuni_api(): @app.route('/download//', methods=['GET']) def download_file_endpoint(file_id, filename): - logging.info(f"收到文件下载请求 - 文件ID: {file_id}, 文件名: {filename}") + logging.info(bi(f"收到文件下载请求 - 文件ID: {file_id}, 文件名: {filename}", f"Received file download request - File ID: {file_id}, filename: {filename}")) with file_lock: if file_id not in file_registry: - logging.warning(f"文件ID: {file_id} 不存在") + logging.warning(bi(f"文件ID: {file_id} 不存在", f"File ID: {file_id} does not exist")) return jsonify({'error': '文件不存在'}), 404 file_info = file_registry[file_id] file_info['last_access'] = time.time() @@ -555,7 +558,7 @@ def download_file_endpoint(file_id, filename): file_data = f.read() return file_data, 200, {'Content-Disposition': f'attachment; filename={filename}'} except Exception as e: - logging.error(f"下载文件时出错: {e}") + logging.error(bi(f"下载文件时出错: {e}", f"Error downloading file: {e}")) return jsonify({'status': 'error', 'error': str(e)}), 500 @app.route('/frames//', methods=['GET']) @@ -579,7 +582,7 @@ def serve_video_frames(job_id, filename): try: return send_from_directory(dir_path, safe_file) except Exception as e: - logging.error(f"提供文件时出错: {e}") + logging.error(bi(f"提供文件时出错: {e}", f"Error serving file: {e}")) return jsonify({"error": "文件访问失败"}), 500 @@ -594,9 +597,39 @@ def create_framepack(): touched_job_ids = set() file_paths = [] - # 新模式：GET /api/framepack?-- + # 获取查询字符串（可能是范围表达式或第一个URL） range_expr = request.query_string.decode('utf-8').strip() if request.query_string else '' - if range_expr: + + # 获取POST请求体 + data = request.get_json(silent=True) or {} + + # 检查是否是客户端格式：查询参数是第一个URL，body包含完整URL列表 + # 格式: POST /api/framepack?/frames/task_id/frame_000001.png Body: {"urls": [...]} + if range_expr.startswith('/frames/') and 'urls' in data: + # 这是客户端正常格式，使用body中的urls列表 + urls = data.get('urls', []) + if not isinstance(urls, list) or len(urls) == 0: + return jsonify({'error': 'Missing or empty urls list'}), 400 + + for url in urls: + if not url.startswith('/frames/'): + return jsonify({'error': f'Invalid URL prefix: {url}'}), 400 + parts = url[len('/frames/'):].split('/', 1) + if len(parts) != 2: + return jsonify({'error': f'Malformed URL: {url}'}), 400 + job_id, filename = parts + touched_job_ids.add(os.path.basename(job_id)) + safe_job = os.path.basename(job_id) + safe_file = os.path.basename(filename) + if not safe_file.endswith('.png'): + return jsonify({'error': f'Only .png allowed: {filename}'}), 400 + full_path = os.path.join(FRAMES_ROOT, safe_job, safe_file) + if not os.path.isfile(full_path): + return jsonify({'error': f'File not found: {full_path}'}), 404 + file_paths.append(full_path) + + # 新模式：GET /api/framepack?-- + elif range_expr: try: parts = range_expr.split('-') if len(parts) != 3: @@ -724,6 +757,16 @@ if __name__ == '__main__': parser.add_argument('--port', type=int, default=5000, help='服务监听的端口号（默认: 5000）') args = parser.parse_args() + print("🔍 正在检查运行库...") + try: + from lib_downloader import check_and_install_libraries + check_and_install_libraries() + print("✅ 运行库检查完成") + except Exception as e: + logging.error(bi(f"运行库安装失败: {e}", f"Runtime library installation failed: {e}")) + print(f"❌ 运行库安装失败: {e}") + sys.exit(1) + load_persistent_state() interrupted_count = mark_interrupted_tasks_on_startup() if interrupted_count > 0: @@ -732,7 +775,7 @@ if __name__ == '__main__': from logo import print_logo print_logo() except Exception as e: - logging.warning(f"LOGO显示失败: {e}") - logging.info(f"启动应用程序，端口: {args.port}...") + logging.warning(bi(f"LOGO显示失败: {e}", f"LOGO display failed: {e}")) + logging.info(bi(f"启动应用程序，端口: {args.port}...", f"Starting application, port: {args.port}...")) start_cleanup_thread() app.run(host='0.0.0.0', port=args.port, threaded=True) diff --git a/sanjuuni_utils.py b/sanjuuni_utils.py index d555358..6cc27c4 100644 --- a/sanjuuni_utils.py +++ b/sanjuuni_utils.py @@ -62,9 +62,27 @@ def download_file(url, temp_dir, task_id=None, task_registry=None, task_lock=Non logging.error(error_msg) raise +def find_sanjuuni_binary(): + if sys.platform.startswith('win'): + exe_name = 'sanjuuni.exe' + else: + exe_name = 'sanjuuni' + + local_path = os.path.join('lib', 'sanjuuni', exe_name) + if os.path.exists(local_path): + return local_path + + for path in os.environ.get('PATH', '').split(os.pathsep): + exe_path = os.path.join(path, exe_name) + if os.path.exists(exe_path): + return exe_path + + return 'sanjuuni' + def execute_sanjuuni(input_path, output_path, sanjuuni_args, task_id=None, task_registry=None, task_lock=None): try: - cmd = ['lib/sanjuuni/sanjuuni', '-i', input_path] + sanjuuni_args + ['-o', output_path] + sanjuuni_bin = find_sanjuuni_binary() + cmd = [sanjuuni_bin, '-i', input_path] + sanjuuni_args + ['-o', output_path] if task_id and task_registry and task_lock: add_task_log(task_id, bi(f"执行Sanjuuni命令: {' '.join(cmd)}", f"Execute Sanjuuni command: {' '.join(cmd)}"), task_registry, task_lock) diff --git a/start_server.py b/start_server.py index 20f80da..1a5e0cd 100644 --- a/start_server.py +++ b/start_server.py @@ -7,19 +7,34 @@ import os import sys import logging -from main import app -from shared_utils import load_persistent_state, mark_interrupted_tasks_on_startup, save_persistent_state +from lib_downloader import check_and_install_libraries from logo import print_logo +def bi(zh, en): + return f"{zh} | {en}" +from config import ( + load_config, + get_server_port, + get_server_debug, + get_server_threaded, + get_cleanup_interval, + get_cleanup_retention, + get_streaming_enabled, + get_logging_level, + get_logging_file_path, + get_hwaccel_enabled, + get_hwaccel_method +) + os.makedirs('data', exist_ok=True) -# 配置日志 +logging_level = getattr(logging, get_logging_level()) logging.basicConfig( - level=logging.INFO, + level=logging_level, format='%(asctime)s - %(levelname)s - %(message)s', handlers=[ logging.StreamHandler(sys.stdout), - logging.FileHandler(os.path.join('data', 'server.log')) + logging.FileHandler(get_logging_file_path()) ] ) @@ -27,8 +42,20 @@ def main(): try: print_logo() except Exception as e: - logging.warning(f"LOGO显示失败: {e}") + logging.warning(bi(f"LOGO显示失败: {e}", f"LOGO display failed: {e}")) + print("🔍 正在检查运行库...") + try: + check_and_install_libraries() + print("✅ 运行库检查完成") + except Exception as e: + logging.error(bi(f"运行库安装失败: {e}", f"Runtime library installation failed: {e}")) + print(f"❌ 运行库安装失败: {e}") + sys.exit(1) + + port = get_server_port() + streaming_mode = "流式" if get_streaming_enabled() else "非流式" + print("🚀 GMapiServer 启动中...") print("=" * 50) print("📋 可用功能:") @@ -37,11 +64,25 @@ def main(): print(" • 视频帧提取 (异步,支持B站BV号)") print() print("🔧 配置信息:") - print(f" 端口: 5000") + print(f" 端口: {port}") + print(f" 转码模式: {streaming_mode}") + + hwaccel_enabled = get_hwaccel_enabled() + hwaccel_method = get_hwaccel_method() + + if hwaccel_enabled: + import ffmpeg_utils + ffmpeg_utils.init_hwaccel() + actual_method = ffmpeg_utils._hwaccel_method if ffmpeg_utils._hwaccel_method else "检测失败(使用软件解码)" + print(f" 硬件加速: 已启用 ({actual_method})") + else: + print(f" 硬件加速: 已禁用") + print(f" FFmpeg缓存目录: data/ffmpeg/") print(f" Sanjuuni缓存目录: data/sanjuuni/") print(f" 视频帧目录: data/videoframe/") - print(f" 清理间隔: 1小时") + print(f" 清理间隔: {get_cleanup_interval()}小时") + print(f" 文件保留时间: {get_cleanup_retention()}小时") print() print("🔗 API 端点:") print(" 同步接口:") @@ -64,20 +105,22 @@ def main(): print(" GET /api/tasks - 任务列表(调试)") print("=" * 50) print() - print("🌐 服务器地址: http://0.0.0.0:5000") - print("🔄 自动清理机制已启用 - 临时文件将在2小时后自动删除") + print(f"🌐 服务器地址: http://0.0.0.0:{port}") + print(f"🔄 自动清理机制已启用 - 临时文件将在{get_cleanup_retention()}小时后自动删除") print() + from main import app + from shared_utils import load_persistent_state, mark_interrupted_tasks_on_startup, save_persistent_state + load_persistent_state() interrupted_count = mark_interrupted_tasks_on_startup() if interrupted_count > 0: save_persistent_state() - # 启动Flask应用 from file_cleanup import start_cleanup_thread start_cleanup_thread() - app.run(host='0.0.0.0', port=5000, debug=False, threaded=True) + app.run(host='0.0.0.0', port=port, debug=get_server_debug(), threaded=get_server_threaded()) if __name__ == '__main__': main() diff --git a/video_frame_utils.py b/video_frame_utils.py index 1058fcb..611e4b9 100644 --- a/video_frame_utils.py +++ b/video_frame_utils.py @@ -11,6 +11,87 @@ import glob import requests import logging from shared_utils import task_registry, file_registry, file_lock, task_lock, add_task_log, save_persistent_state +from config import get_video_download_limit_resolution, get_video_download_max_resolution, get_video_download_preferred_codec + +def find_yt_dlp_executable(): + result = subprocess.run(['which', 'yt-dlp'], capture_output=True, text=True) + if result.returncode == 0: + return result.stdout.strip() + return None + +def detect_video_codec(file_path): + try: + result = subprocess.run( + ['ffprobe', '-v', 'quiet', '-print_format', 'json', '-show_streams', file_path], + capture_output=True, + text=True, + timeout=30 + ) + if result.returncode == 0: + info = json.loads(result.stdout) + for stream in info.get('streams', []): + if stream.get('codec_type') == 'video': + return stream.get('codec_name') + except Exception as e: + logging.warning(bi(f"检测视频编码失败: {e}", f"Failed to detect video codec: {e}")) + return None + +def ensure_yt_dlp(): + try: + import yt_dlp + return True, None + except ImportError: + pass + + print("🔧 yt-dlp 未安装，正在尝试自动安装...") + logging.info(bi("yt-dlp 未安装，正在尝试自动安装", "yt-dlp not installed, attempting automatic installation")) + + venv_python = os.path.join(os.path.dirname(sys.executable), '..', '..', 'bin', 'python3') + if os.path.exists(venv_python): + venv_python = os.path.abspath(venv_python) + + install_methods = [ + ([sys.executable, '-m', 'pip', 'install', 'yt-dlp', '--quiet'], "直接安装"), + ([sys.executable, '-m', 'pip', 'install', 'yt-dlp', '--quiet', '--break-system-packages'], "使用 --break-system-packages"), + ] + + if os.path.exists(venv_python) and venv_python != sys.executable: + install_methods.insert(0, ([venv_python, '-m', 'pip', 'install', 'yt-dlp', '--quiet'], "虚拟环境安装")) + + if sys.platform.startswith('linux'): + try: + result = subprocess.run(['which', 'apt-get'], capture_output=True) + if result.returncode == 0: + install_methods.append((['sudo', 'apt-get', 'install', '-y', 'yt-dlp'], "使用 apt-get")) + except Exception: + pass + + for cmd, method in install_methods: + try: + print(f" 尝试 {method}: {' '.join(cmd)}") + subprocess.check_call(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) + print("✅ yt-dlp 安装成功") + logging.info(bi(f"yt-dlp 安装成功（{method}）", f"yt-dlp installed successfully ({method})")) + return True, None + except subprocess.CalledProcessError as e: + print(f" ❌ {method} 失败") + logging.warning(bi(f"yt-dlp {method} 安装失败: {e}", f"yt-dlp {method} installation failed: {e}")) + except Exception as e: + print(f" ❌ {method} 异常: {e}") + logging.warning(bi(f"yt-dlp {method} 安装异常: {e}", f"yt-dlp {method} installation exception: {e}")) + + yt_dlp_path = find_yt_dlp_executable() + if yt_dlp_path: + print(f"⚠️ 检测到系统中有 yt-dlp 命令: {yt_dlp_path}") + logging.info(bi(f"检测到系统中有 yt-dlp 命令: {yt_dlp_path}", f"Found yt-dlp executable in system: {yt_dlp_path}")) + return True, yt_dlp_path + + print("\n❌ yt-dlp 安装失败，请手动安装：") + print(" 方法1 (推荐): sudo apt-get install yt-dlp") + print(" 方法2: pip install yt-dlp --break-system-packages") + print(" 方法3: 创建虚拟环境后安装") + logging.error(bi("yt-dlp 所有安装方法均失败，请手动安装", "All yt-dlp installation methods failed, please install manually")) + return False, None def bi(zh, en): return f"{zh} | {en}" @@ -34,12 +115,30 @@ def renew_video_frame_task_expiry(task_info, now): # 确保frames目录存在 os.makedirs(FRAMES_ROOT, exist_ok=True) -if sys.platform.startswith('win'): - FFMPEG_PATH = os.path.join(BASE_DIR, 'lib', 'ffmpeg', 'bin', 'ffmpeg.exe') - FFPROBE_PATH = os.path.join(BASE_DIR, 'lib', 'ffmpeg', 'bin', 'ffprobe.exe') -else: - FFMPEG_PATH = os.path.join(BASE_DIR, 'lib', 'ffmpeg', 'bin', 'ffmpeg') - FFPROBE_PATH = os.path.join(BASE_DIR, 'lib', 'ffmpeg', 'bin', 'ffprobe') +def find_ffmpeg_tool(tool_name): + """查找FFmpeg工具路径（优先系统路径）""" + if sys.platform.startswith('win'): + exe_name = f"{tool_name}.exe" + local_path = os.path.join(BASE_DIR, 'lib', 'ffmpeg', 'bin', exe_name) + else: + exe_name = tool_name + local_path = os.path.join(BASE_DIR, 'lib', 'ffmpeg', 'bin', tool_name) + + if os.path.exists(local_path): + return local_path + + if sys.platform.startswith('linux'): + try: + result = subprocess.run(['which', exe_name], capture_output=True, text=True) + if result.returncode == 0: + return result.stdout.strip() + except Exception: + pass + + return local_path + +FFMPEG_PATH = find_ffmpeg_tool('ffmpeg') +FFPROBE_PATH = find_ffmpeg_tool('ffprobe') def parse_ffmpeg_frame_progress(line, task_id, task_registry, task_lock): """解析FFmpeg进度输出并更新任务状态""" @@ -165,16 +264,64 @@ def process_video_frame_extraction(data, file_registry, file_lock, task_id=None, if task_id in task_registry: task_registry[task_id]['progress'] = 20 # 开始下载，进度20% - yt_dlp_cmd = [ - sys.executable, '-m', 'yt_dlp', - video_url, - '-o', temp_base, - '--no-warnings', - '--progress', # 启用进度显示 - '--newline', # 确保换行符正常 - '--console-title', # 确保进度信息正确输出 - '--no-colors', # 禁用颜色，避免控制字符干扰 - ] + yt_dlp_ok, yt_dlp_path = ensure_yt_dlp() + if not yt_dlp_ok: + raise Exception(bi("yt-dlp 安装失败，请手动安装: pip install yt-dlp", "yt-dlp installation failed, please install manually: pip install yt-dlp")) + + limit_resolution = get_video_download_limit_resolution() + max_resolution = get_video_download_max_resolution() + preferred_codec = get_video_download_preferred_codec() + + format_spec = '' + filters = [] + + if preferred_codec != 'auto': + codec_map = { + 'h264': 'avc1', + 'h265': 'hvc1', + 'av1': 'av01', + 'vp9': 'vp09' + } + codec_id = codec_map.get(preferred_codec) + if codec_id: + filters.append(f'vcodec^={codec_id}') + logging.info(bi(f"使用首选编码: {preferred_codec} (vcodec^={codec_id})", f"Using preferred codec: {preferred_codec} (vcodec^={codec_id})")) + + if limit_resolution and max_resolution != 'best': + filters.append(f'height<={max_resolution[:-1]}') + logging.info(bi(f"限制下载分辨率为: {max_resolution}", f"Limit download resolution to: {max_resolution}")) + + if filters: + format_spec = f'bv*[{"][".join(filters)}]+ba*' + else: + format_spec = 'bestvideo+bestaudio' + + if yt_dlp_path: + yt_dlp_cmd = [ + yt_dlp_path, + video_url, + '-o', temp_base, + '-f', format_spec, + '--no-playlist', + '--no-warnings', + '--progress', + '--newline', + '--console-title', + '--no-colors', + ] + else: + yt_dlp_cmd = [ + sys.executable, '-m', 'yt_dlp', + video_url, + '-o', temp_base, + '-f', format_spec, + '--no-playlist', + '--no-warnings', + '--progress', + '--newline', + '--console-title', + '--no-colors', + ] # 使用 Popen 实时捕获 stdout 和 stderr proc = subprocess.Popen( @@ -306,9 +453,16 @@ def process_video_frame_extraction(data, file_registry, file_lock, task_id=None, if task_id in task_registry: task_registry[task_id]['progress'] = 40 + hwaccel_args_audio = [] + codec = detect_video_codec(temp_audio) + if codec == 'av1': + logging.info(bi("检测到AV1编码视频，音频提取时禁用硬件加速", "Detected AV1 encoded video, disabling hardware acceleration for audio extraction")) + hwaccel_args_audio = ['-hwaccel', 'none'] + # 公共参数 dfpwm_args = [ - FFMPEG_PATH, '-y', + FFMPEG_PATH, '-y' + ] + hwaccel_args_audio + [ '-i', temp_audio, '-vn', '-ar', '48000', @@ -381,8 +535,21 @@ def process_video_frame_extraction(data, file_registry, file_lock, task_id=None, frame_pattern = os.path.join(job_dir, "frame_%06d.png") + try: + from ffmpeg_utils import get_hwaccel_args + hwaccel_args = get_hwaccel_args() + + codec = detect_video_codec(temp_video) + if codec == 'av1': + logging.info(bi("检测到AV1编码视频，禁用硬件加速", "Detected AV1 encoded video, disabling hardware acceleration")) + hwaccel_args = [] + except Exception as e: + logging.warning(bi(f"获取硬件加速参数失败: {e}", f"Failed to get hardware acceleration args: {e}")) + hwaccel_args = [] + ffmpeg_cmd = [ - FFMPEG_PATH, '-y', + FFMPEG_PATH, '-y' + ] + hwaccel_args + [ '-i', temp_video, '-vf', vf, '-pix_fmt', pix_fmt,