Compare commits
4 Commits
85ee16d1e6
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b318144230 | ||
|
|
56490460fa | ||
|
|
f422eb32c1 | ||
|
|
cad3a033a3 |
1
.gitignore
vendored
1
.gitignore
vendored
@@ -2,3 +2,4 @@
|
||||
__pycache__/
|
||||
app.log
|
||||
temp_files
|
||||
frames
|
||||
|
||||
250
README.md
250
README.md
@@ -1,97 +1,215 @@
|
||||
---
|
||||
GMapiServer
|
||||
|
||||
# GMapiServer
|
||||
GMapiServer 是一个基于 Python 和 Flask 构建的高性能 API 服务器,支持通过 URL 输入文件并输出下载链接,提供同步和异步两种处理模式。内置缓存自动清理机制(默认保留 2 小时),支持多种工具接口调用,包括实时任务状态查询和视频帧处理功能。
|
||||
|
||||
**GMapiServer** 是一个基于 Python 和 Flask 构建的高性能 API 服务器,支持通过 URL 输入文件并输出下载链接,提供异步处理功能,确保主进程不被阻塞。内置缓存自动清理机制(默认保留 2 小时),支持多种工具接口调用。
|
||||
🚀 项目简介
|
||||
|
||||
---
|
||||
GMapiServer 是一个轻量级、模块化的 API 服务框架,旨在为开发者提供以下核心功能:
|
||||
|
||||
## 🚀 项目简介
|
||||
• URL 输入与输出:支持通过 URL 提交文件,返回生成文件的下载链接。
|
||||
|
||||
**GMapiServer** 是一个轻量级、模块化的 API 服务框架,旨在为开发者提供以下核心功能:
|
||||
• 异步处理:任务在后台异步执行,不阻塞主进程,提升服务器并发性能。
|
||||
|
||||
- **URL 输入与输出**:支持通过 URL 提交文件,返回生成文件的下载链接。
|
||||
- **异步处理**:任务在后台异步执行,不阻塞主进程,提升服务器并发性能。
|
||||
- **自动缓存清理**:内置定时清理机制,自动删除过期的缓存文件(包括输出文件),默认保留时间 2 小时。
|
||||
- **多工具接口集成**:集成 FFmpeg 和 Sanjuuni 工具接口,支持在线调用。
|
||||
• 自动缓存清理:内置定时清理机制,自动删除过期的缓存文件(包括输出文件),默认保留时间 2 小时。
|
||||
|
||||
---
|
||||
• 多工具接口集成:集成 FFmpeg 和 Sanjuuni 工具接口,支持在线调用。
|
||||
|
||||
## 🛠️ 技术栈
|
||||
• 实时任务状态:提供任务状态查询接口,支持日志实时推送。
|
||||
|
||||
- **编程语言**: Python
|
||||
- **Web 框架**: Flask
|
||||
- **异步处理**: 基于 `threading`
|
||||
- **缓存清理**: 定时任务
|
||||
- **依赖管理**: pip
|
||||
• 视频帧提取:支持从视频中提取指定分辨率、帧率的图片帧,可打包为二进制格式。
|
||||
|
||||
---
|
||||
🛠️ 技术栈
|
||||
|
||||
## 🧩 支持的接口
|
||||
• 编程语言: Python
|
||||
|
||||
### 1. FFmpeg 工具接口
|
||||
- **功能**: 在线调用 FFmpeg 工具进行视频/音频处理(如转码、裁剪、合并等)。
|
||||
- **接口文档**: [FFmpegApi 文档](https://www.liulikeji.cn/archives/FFmpegApi)
|
||||
• Web 框架: Flask
|
||||
|
||||
### 2. Sanjuuni 工具接口
|
||||
- **功能**: 在线调用 [Sanjuuni 工具](https://github.com/MCJack123/sanjuuni/tree/master)(具体功能需参考其官方文档)。
|
||||
- **接口文档**: [SanjuuniApi 文档](https://www.liulikeji.cn/archives/SanjuuniApi)
|
||||
• 异步处理: 基于 threading
|
||||
|
||||
• 缓存清理: 定时任务
|
||||
|
||||
### 新增异步接口调用
|
||||
- **接口文档**: [异步任务处理接口文档](https://www.liulikeji.cn/archives/wei-ming-ming-wen-zhang)
|
||||
• 依赖管理: pip
|
||||
|
||||
---
|
||||
🧩 支持的接口
|
||||
|
||||
## 📦 部署与使用
|
||||
1. FFmpeg 工具接口
|
||||
|
||||
• 功能: 在线调用 FFmpeg 工具进行视频/音频处理(如转码、裁剪、合并等)。
|
||||
|
||||
• 接口文档: https://www.liulikeji.cn/archives/FFmpegApi
|
||||
|
||||
• 同步接口: POST /api/ffmpeg
|
||||
|
||||
• 异步接口: POST /api/ffmpeg/async
|
||||
|
||||
2. Sanjuuni 工具接口
|
||||
|
||||
• 功能: 在线调用 https://github.com/MCJack123/sanjuuni/tree/master(具体功能需参考其官方文档)。
|
||||
|
||||
• 接口文档: https://www.liulikeji.cn/archives/SanjuuniApi
|
||||
|
||||
• 同步接口: POST /api/sanjuuni
|
||||
|
||||
• 异步接口: POST /api/sanjuuni/async
|
||||
|
||||
3. 视频帧提取接口
|
||||
|
||||
• 功能: 从视频中提取指定分辨率、帧率的图片帧,支持强制分辨率调整和填充。
|
||||
|
||||
• 异步接口: POST /api/video_frame/async
|
||||
|
||||
• 参数:
|
||||
|
||||
• video_url: 视频文件 URL
|
||||
|
||||
• w: 输出宽度
|
||||
|
||||
• h: 输出高度
|
||||
|
||||
• fps: 输出帧率
|
||||
|
||||
• force_resolution: 是否强制调整分辨率
|
||||
|
||||
• pad_to_target: 是否填充到目标分辨率
|
||||
|
||||
4. 异步任务状态查询
|
||||
|
||||
• 功能: 查询异步任务状态和进度,支持日志实时推送。
|
||||
|
||||
• 接口: GET /api/task/<task_id>
|
||||
|
||||
• 返回字段:
|
||||
|
||||
• status: 任务状态(pending/running/completed/error)
|
||||
|
||||
• progress: 进度百分比
|
||||
|
||||
• new_logs: 新增的日志条目
|
||||
|
||||
• result: 完成后的结果(包含下载URL)
|
||||
|
||||
• error: 错误信息(如果状态为error)
|
||||
|
||||
5. 帧打包接口
|
||||
|
||||
• 功能: 将多个帧图片打包为二进制格式,便于批量下载。
|
||||
|
||||
• 接口: POST /api/framepack
|
||||
|
||||
• 参数:
|
||||
|
||||
• urls: 帧图片URL列表,格式为 /frames/<job_id>/<filename>.png
|
||||
|
||||
• 返回: 二进制流,格式为帧数(4字节) + 各帧数据
|
||||
|
||||
6. 文件下载接口
|
||||
|
||||
• 功能: 下载处理完成的文件。
|
||||
|
||||
• 接口: GET /download/<file_id>/<filename>
|
||||
|
||||
7. 任务列表查询
|
||||
|
||||
• 功能: 列出所有任务状态(调试用)。
|
||||
|
||||
• 接口: GET /api/tasks
|
||||
|
||||
8. 健康检查
|
||||
|
||||
• 功能: 服务健康检查。
|
||||
|
||||
• 接口: GET /health
|
||||
|
||||
📦 部署与使用
|
||||
|
||||
1. 安装依赖
|
||||
|
||||
### 1. 安装依赖
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
### 2. 启动服务器
|
||||
```bash
|
||||
|
||||
2. 启动服务器
|
||||
|
||||
python main.py --port 5000
|
||||
```
|
||||
### 3. 指定公网地址
|
||||
```python
|
||||
##sanjuuni_utils.py
|
||||
return {
|
||||
'status': 'success',
|
||||
'download_url': f"http://ffmpeg.liulikeji.cn/download/{output_id}/{output_filename}", #外部访问地址
|
||||
'file_id': output_id,
|
||||
'temp_dir': temp_dir # 返回临时目录路径
|
||||
}
|
||||
|
||||
##ffmpeg_utils.py
|
||||
return {
|
||||
'status': 'success',
|
||||
'download_url': f"http://ffmpeg.liulikeji.cn/download/{output_id}/{output_filename}", #外部访问地址
|
||||
'file_id': output_id,
|
||||
'temp_dir': temp_dir # 返回临时目录路径
|
||||
}
|
||||
```
|
||||
---
|
||||
|
||||
## ⏰ 自动缓存清理配置
|
||||
- **默认保留时间**: 2 小时
|
||||
- **自定义配置**:
|
||||
```python
|
||||
|
||||
3. 异步任务使用示例
|
||||
|
||||
创建异步FFmpeg任务
|
||||
|
||||
curl -X POST http://localhost:5000/api/ffmpeg/async \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"input_url": "http://example.com/input.mp4",
|
||||
"output_format": "mp4",
|
||||
"args": ["-c:v", "libx264", "-crf", "23"]
|
||||
}'
|
||||
|
||||
|
||||
查询任务状态
|
||||
|
||||
curl http://localhost:5000/api/task/<task_id>
|
||||
|
||||
|
||||
4. 指定公网地址
|
||||
|
||||
在相应工具模块中配置下载URL:
|
||||
# sanjuuni_utils.py
|
||||
return {
|
||||
'status': 'success',
|
||||
'download_url': f"http://ffmpeg.liulikeji.cn/download/{output_id}/{output_filename}", # 外部访问地址
|
||||
'file_id': output_id,
|
||||
'temp_dir': temp_dir
|
||||
}
|
||||
|
||||
# ffmpeg_utils.py
|
||||
return {
|
||||
'status': 'success',
|
||||
'download_url': f"http://ffmpeg.liulikeji.cn/download/{output_id}/{output_filename}", # 外部访问地址
|
||||
'file_id': output_id,
|
||||
'temp_dir': temp_dir
|
||||
}
|
||||
|
||||
|
||||
⏰ 自动缓存清理配置
|
||||
|
||||
• 默认保留时间: 2 小时
|
||||
|
||||
• 自定义配置:
|
||||
# 在logging_config.py文件中修改缓存清理时间
|
||||
CLEANUP_INTERVAL = 3600 # 清理临时文件的间隔(秒)
|
||||
FILE_EXPIRY = 7200 # 文件过期时间(秒)
|
||||
```
|
||||
---
|
||||
|
||||
|
||||
## ⚠️ 注意事项
|
||||
1. **文件合法性**: 请确保上传文件符合法律法规,不得用于非法用途。
|
||||
2. **缓存文件**: 系统会自动清理过期文件,请及时下载生成的输出文件。
|
||||
🔧 异步任务处理流程
|
||||
|
||||
---
|
||||
1. 任务创建: 客户端提交任务请求,服务器返回任务ID和状态查询URL
|
||||
2. 后台处理: 任务在独立线程中执行,不阻塞主进程
|
||||
3. 状态查询: 客户端可轮询状态接口获取任务进度和实时日志
|
||||
4. 结果获取: 任务完成后可通过状态接口或文件下载接口获取结果
|
||||
5. 自动清理: 任务结果文件在指定时间后自动清理
|
||||
|
||||
⚠️ 注意事项
|
||||
|
||||
1. 文件合法性: 请确保上传文件符合法律法规,不得用于非法用途。
|
||||
2. 缓存文件: 系统会自动清理过期文件,请及时下载生成的输出文件。
|
||||
3. 异步任务: 异步任务结果在服务器重启后会丢失,请及时处理完成的任务。
|
||||
4. 资源限制: 视频帧提取和转码操作可能消耗较多CPU和内存资源。
|
||||
|
||||
📝 贡献与反馈
|
||||
|
||||
## 📝 贡献与反馈
|
||||
欢迎提交 Issue 或 Pull Request!
|
||||
如有问题,请联系:[xingluo01@liulikeji.cn] 或 [qq:180877430]
|
||||
|
||||
---
|
||||
📁 项目结构
|
||||
|
||||
|
||||
GMapiServer/
|
||||
├── main.py # 主程序入口
|
||||
├── shared_utils.py # 共享工具函数
|
||||
├── ffmpeg_utils.py # FFmpeg处理模块
|
||||
├── sanjuuni_utils.py # Sanjuuni处理模块
|
||||
├── video_frame_utils.py # 视频帧提取模块
|
||||
├── file_cleanup.py # 文件清理模块
|
||||
├── logging_config.py # 日志配置
|
||||
├── requirements.txt # 依赖包列表
|
||||
└── README.md # 项目说明文档
|
||||
@@ -6,28 +6,50 @@ import logging
|
||||
from shared_utils import file_registry, file_lock, task_registry, task_lock
|
||||
|
||||
UPLOAD_FOLDER = 'temp_files'
|
||||
FRAMES_FOLDER = 'frames' # 视频帧提取任务文件夹
|
||||
CLEANUP_INTERVAL = 3600 # 清理临时文件的间隔(秒)
|
||||
FILE_EXPIRY = 7200 # 文件过期时间(秒)
|
||||
TASK_EXPIRY = 7200 # 任务记录过期时间(秒)
|
||||
|
||||
def cleanup_temp_files():
|
||||
current_time = time.time()
|
||||
expired_folders = []
|
||||
|
||||
for folder_name in os.listdir(UPLOAD_FOLDER):
|
||||
folder_path = os.path.join(UPLOAD_FOLDER, folder_name)
|
||||
|
||||
if not os.path.isdir(folder_path):
|
||||
continue
|
||||
# 清理temp_files目录
|
||||
if os.path.exists(UPLOAD_FOLDER):
|
||||
for folder_name in os.listdir(UPLOAD_FOLDER):
|
||||
folder_path = os.path.join(UPLOAD_FOLDER, folder_name)
|
||||
|
||||
try:
|
||||
folder_mtime = os.path.getmtime(folder_path)
|
||||
|
||||
if current_time - folder_mtime > FILE_EXPIRY:
|
||||
expired_folders.append(folder_path)
|
||||
logging.info(f"检测到过期文件夹: {folder_path} (修改时间: {time.ctime(folder_mtime)})")
|
||||
except Exception as e:
|
||||
logging.error(f"获取文件夹 {folder_path} 信息失败: {e}")
|
||||
if not os.path.isdir(folder_path):
|
||||
continue
|
||||
|
||||
try:
|
||||
folder_mtime = os.path.getmtime(folder_path)
|
||||
|
||||
if current_time - folder_mtime > FILE_EXPIRY:
|
||||
expired_folders.append(folder_path)
|
||||
logging.info(f"检测到过期文件夹: {folder_path} (修改时间: {time.ctime(folder_mtime)})")
|
||||
except Exception as e:
|
||||
logging.error(f"获取文件夹 {folder_path} 信息失败: {e}")
|
||||
|
||||
# 清理frames目录
|
||||
if os.path.exists(FRAMES_FOLDER):
|
||||
for folder_name in os.listdir(FRAMES_FOLDER):
|
||||
folder_path = os.path.join(FRAMES_FOLDER, folder_name)
|
||||
|
||||
if not os.path.isdir(folder_path):
|
||||
continue
|
||||
|
||||
try:
|
||||
folder_mtime = os.path.getmtime(folder_path)
|
||||
|
||||
if current_time - folder_mtime > FILE_EXPIRY:
|
||||
expired_folders.append(folder_path)
|
||||
logging.info(f"检测到过期视频帧文件夹: {folder_path} (修改时间: {time.ctime(folder_mtime)})")
|
||||
except Exception as e:
|
||||
logging.error(f"获取视频帧文件夹 {folder_path} 信息失败: {e}")
|
||||
|
||||
# 删除所有过期文件夹
|
||||
for folder_path in expired_folders:
|
||||
try:
|
||||
shutil.rmtree(folder_path)
|
||||
@@ -36,18 +58,36 @@ def cleanup_temp_files():
|
||||
logging.error(f"删除文件夹 {folder_path} 失败: {e}")
|
||||
|
||||
logging.info(f"清理完成,共删除 {len(expired_folders)} 个过期文件夹")
|
||||
|
||||
# 同时清理过期的任务记录(超过2小时)
|
||||
|
||||
# 清理过期的任务记录和文件记录
|
||||
cleanup_expired_registries(current_time)
|
||||
|
||||
def cleanup_expired_registries(current_time):
|
||||
"""清理过期的任务记录和文件记录"""
|
||||
# 清理过期的任务记录
|
||||
with task_lock:
|
||||
expired_tasks = []
|
||||
for task_id, task_info in list(task_registry.items()):
|
||||
if current_time - task_info['create_time'] > FILE_EXPIRY:
|
||||
if current_time - task_info['create_time'] > TASK_EXPIRY:
|
||||
expired_tasks.append(task_id)
|
||||
|
||||
|
||||
for task_id in expired_tasks:
|
||||
del task_registry[task_id]
|
||||
logging.info(f"已清理过期任务记录: {task_id}")
|
||||
|
||||
# 清理过期的文件记录(基于最后访问时间)
|
||||
with file_lock:
|
||||
expired_files = []
|
||||
for file_id, file_info in list(file_registry.items()):
|
||||
if current_time - file_info['last_access'] > FILE_EXPIRY:
|
||||
expired_files.append(file_id)
|
||||
|
||||
for file_id in expired_files:
|
||||
del file_registry[file_id]
|
||||
logging.info(f"已清理过期文件记录: {file_id}")
|
||||
|
||||
logging.info(f"共清理 {len(expired_tasks)} 个任务记录和 {len(expired_files)} 个文件记录")
|
||||
|
||||
def start_cleanup_thread():
|
||||
def cleanup_loop():
|
||||
while True:
|
||||
|
||||
148
main.py
148
main.py
@@ -15,6 +15,7 @@ app = Flask(__name__)
|
||||
try:
|
||||
from ffmpeg_utils import process_ffmpeg
|
||||
from sanjuuni_utils import process_sanjuuni
|
||||
from video_frame_utils import process_video_frame_extraction
|
||||
from file_cleanup import start_cleanup_thread
|
||||
except ImportError as e:
|
||||
logging.error(f"导入模块时出错: {e}")
|
||||
@@ -23,6 +24,8 @@ except ImportError as e:
|
||||
return {'error': 'FFmpeg模块未正确导入'}
|
||||
def process_sanjuuni(*args, **kwargs):
|
||||
return {'error': 'Sanjuuni模块未正确导入'}
|
||||
def process_video_frame_extraction(*args, **kwargs):
|
||||
return {'error': '视频帧提取模块未正确导入'}
|
||||
def start_cleanup_thread():
|
||||
pass
|
||||
|
||||
@@ -37,6 +40,14 @@ enter_parameter_table = {
|
||||
"input_url": str,
|
||||
"output_format": str,
|
||||
"args": list
|
||||
},
|
||||
"video_frame": {
|
||||
"video_url": str,
|
||||
"w": int,
|
||||
"h": int,
|
||||
"fps": int,
|
||||
"force_resolution": bool,
|
||||
"pad_to_target": bool
|
||||
}
|
||||
}
|
||||
|
||||
@@ -140,11 +151,11 @@ def sanjuuni_async_api():
|
||||
|
||||
# 生成任务ID
|
||||
task_id = str(uuid.uuid4())[:8]
|
||||
|
||||
|
||||
# 获取请求的主机信息用于构建URL
|
||||
host = request.host
|
||||
scheme = request.scheme
|
||||
|
||||
|
||||
# 初始化任务状态
|
||||
with task_lock:
|
||||
task_registry[task_id] = {
|
||||
@@ -155,7 +166,7 @@ def sanjuuni_async_api():
|
||||
'last_returned_index': 0, # 记录最后返回的日志索引
|
||||
'progress': 0
|
||||
}
|
||||
|
||||
|
||||
# 启动异步任务
|
||||
thread = threading.Thread(
|
||||
target=run_async_task,
|
||||
@@ -163,9 +174,9 @@ def sanjuuni_async_api():
|
||||
)
|
||||
thread.daemon = True
|
||||
thread.start()
|
||||
|
||||
|
||||
status_url = f"{scheme}://{host}/api/task/{task_id}"
|
||||
|
||||
|
||||
logging.info(f"创建异步Sanjuuni任务: {task_id}")
|
||||
return jsonify({
|
||||
'status': 'success',
|
||||
@@ -174,6 +185,53 @@ def sanjuuni_async_api():
|
||||
'message': '任务已创建,请使用状态URL查询进度'
|
||||
}), 202
|
||||
|
||||
@app.route('/api/video_frame/async', methods=['POST'])
|
||||
def video_frame_async_api():
|
||||
"""创建异步视频帧提取任务"""
|
||||
logging.info("收到异步视频帧提取API请求")
|
||||
data = request.get_json()
|
||||
|
||||
# 检测参数类型
|
||||
error_response, status_code = validate_request(data, "video_frame")
|
||||
if error_response:
|
||||
return error_response, status_code
|
||||
|
||||
# 生成任务ID
|
||||
task_id = str(uuid.uuid4())[:8]
|
||||
|
||||
# 获取请求的主机信息用于构建URL
|
||||
host = request.host
|
||||
scheme = request.scheme
|
||||
|
||||
# 初始化任务状态
|
||||
with task_lock:
|
||||
task_registry[task_id] = {
|
||||
'status': 'pending',
|
||||
'type': 'video_frame',
|
||||
'create_time': time.time(),
|
||||
'logs': [],
|
||||
'last_returned_index': 0, # 记录最后返回的日志索引
|
||||
'progress': 0
|
||||
}
|
||||
|
||||
# 启动异步任务
|
||||
thread = threading.Thread(
|
||||
target=run_async_task,
|
||||
args=(task_id, process_video_frame_extraction, data)
|
||||
)
|
||||
thread.daemon = True
|
||||
thread.start()
|
||||
|
||||
status_url = f"{scheme}://{host}/api/task/{task_id}"
|
||||
|
||||
logging.info(f"创建异步视频帧提取任务: {task_id}")
|
||||
return jsonify({
|
||||
'status': 'success',
|
||||
'task_id': task_id,
|
||||
'status_url': status_url,
|
||||
'message': '任务已创建,请使用状态URL查询进度'
|
||||
}), 202
|
||||
|
||||
@app.route('/api/task/<task_id>', methods=['GET'])
|
||||
def get_task_status(task_id):
|
||||
"""查询任务状态和进度(自动返回新增日志)"""
|
||||
@@ -307,6 +365,86 @@ def download_file_endpoint(file_id, filename):
|
||||
logging.error(f"下载文件时出错: {e}")
|
||||
return jsonify({'status': 'error', 'error': str(e)}), 500
|
||||
|
||||
@app.route('/frames/<job_id>/<filename>', methods=['GET'])
|
||||
def serve_video_frames(job_id, filename):
|
||||
"""提供视频帧和音频文件访问"""
|
||||
import os
|
||||
from flask import send_from_directory
|
||||
from video_frame_utils import FRAMES_ROOT
|
||||
|
||||
safe_job = os.path.basename(job_id)
|
||||
safe_file = os.path.basename(filename)
|
||||
dir_path = os.path.join(FRAMES_ROOT, safe_job)
|
||||
|
||||
if not os.path.isdir(dir_path):
|
||||
return jsonify({"error": "文件不存在"}), 404
|
||||
|
||||
allowed = safe_file.endswith(('.png', '.dfpwm'))
|
||||
if not allowed:
|
||||
return jsonify({"error": "文件类型不支持"}), 400
|
||||
|
||||
try:
|
||||
return send_from_directory(dir_path, safe_file)
|
||||
except Exception as e:
|
||||
logging.error(f"提供文件时出错: {e}")
|
||||
return jsonify({"error": "文件访问失败"}), 500
|
||||
|
||||
|
||||
@app.route('/api/framepack', methods=['POST'])
|
||||
def create_framepack():
|
||||
"""接收一组相对路径(如 /frames/abc123/frame_0001.png),打包返回二进制"""
|
||||
import struct
|
||||
from video_frame_utils import FRAMES_ROOT
|
||||
import os
|
||||
|
||||
data = request.get_json()
|
||||
if not data or 'urls' not in data or not isinstance(data['urls'], list):
|
||||
return jsonify({'error': 'Missing urls list'}), 400
|
||||
|
||||
# 构建完整本地路径,验证安全性
|
||||
file_paths = []
|
||||
for url in data['urls']:
|
||||
# 只允许以 /frames/ 开头的路径
|
||||
if not url.startswith('/frames/'):
|
||||
return jsonify({'error': f'Invalid URL prefix: {url}'}), 400
|
||||
# 提取 job_id 和 filename
|
||||
parts = url[len('/frames/'):].split('/', 1)
|
||||
if len(parts) != 2:
|
||||
return jsonify({'error': f'Malformed URL: {url}'}), 400
|
||||
job_id, filename = parts
|
||||
# 安全路径拼接
|
||||
safe_job = os.path.basename(job_id)
|
||||
safe_file = os.path.basename(filename)
|
||||
if not safe_file.endswith('.png'):
|
||||
return jsonify({'error': f'Only .png allowed: {filename}'}), 400
|
||||
full_path = os.path.join(FRAMES_ROOT, safe_job, safe_file)
|
||||
if not os.path.isfile(full_path):
|
||||
return jsonify({'error': f'File not found: {full_path}'}), 404
|
||||
file_paths.append(full_path)
|
||||
|
||||
# 构建 FramePack 二进制
|
||||
pack_data = bytearray()
|
||||
frame_count = len(file_paths)
|
||||
pack_data.extend(struct.pack('>I', frame_count)) # big-endian uint32
|
||||
|
||||
# 先读所有文件到内存(小批量安全)
|
||||
frames = []
|
||||
for path in file_paths:
|
||||
with open(path, 'rb') as f:
|
||||
frames.append(f.read())
|
||||
|
||||
for frame in frames:
|
||||
pack_data.extend(struct.pack('>I', len(frame)))
|
||||
pack_data.extend(frame)
|
||||
|
||||
response = app.response_class(
|
||||
response=bytes(pack_data),
|
||||
status=200,
|
||||
mimetype='application/octet-stream'
|
||||
)
|
||||
response.headers['Content-Disposition'] = 'inline; filename="framepack.bin"'
|
||||
return response
|
||||
|
||||
@app.route('/health', methods=['GET'])
|
||||
def health_check():
|
||||
"""健康检查接口,直接返回 'ok'"""
|
||||
|
||||
@@ -1,2 +1,3 @@
|
||||
Flask==3.1.0
|
||||
Requests==2.32.3
|
||||
yt-dlp
|
||||
155640
server.log
Normal file
155640
server.log
Normal file
File diff suppressed because it is too large
Load Diff
68
start_server.py
Normal file
68
start_server.py
Normal file
@@ -0,0 +1,68 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
启动GMapiServer的启动脚本
|
||||
包含所有功能:FFmpeg处理、Sanjuuni处理、视频帧提取
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import logging
|
||||
from main import app
|
||||
|
||||
# 配置日志
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format='%(asctime)s - %(levelname)s - %(message)s',
|
||||
handlers=[
|
||||
logging.StreamHandler(sys.stdout),
|
||||
logging.FileHandler('server.log')
|
||||
]
|
||||
)
|
||||
|
||||
def main():
|
||||
print("🚀 GMapiServer 启动中...")
|
||||
print("=" * 50)
|
||||
print("📋 可用功能:")
|
||||
print(" • FFmpeg媒体处理 (同步/异步)")
|
||||
print(" • Sanjuuni工具处理 (同步/异步)")
|
||||
print(" • 视频帧提取 (异步,支持B站BV号)")
|
||||
print()
|
||||
print("🔧 配置信息:")
|
||||
print(f" 端口: 5000")
|
||||
print(f" 临时文件目录: temp_files/")
|
||||
print(f" 视频帧目录: frames/")
|
||||
print(f" 清理间隔: 1小时")
|
||||
print()
|
||||
print("🔗 API 端点:")
|
||||
print(" 同步接口:")
|
||||
print(" POST /api/ffmpeg - FFmpeg处理")
|
||||
print(" POST /api/sanjuuni - Sanjuuni处理")
|
||||
print()
|
||||
print(" 异步接口:")
|
||||
print(" POST /api/ffmpeg/async - 异步FFmpeg处理")
|
||||
print(" POST /api/sanjuuni/async - 异步Sanjuuni处理")
|
||||
print(" POST /api/video_frame/async - 异步视频帧提取")
|
||||
print(" GET /api/task/<task_id> - 查询异步任务状态")
|
||||
print(" GET /api/tasks - 所有任务列表")
|
||||
print()
|
||||
print(" 文件下载:")
|
||||
print(" GET /download/<file_id>/<filename> - 下载处理后的文件")
|
||||
print(" GET /frames/<job_id>/<filename> - 下载视频帧/音频文件")
|
||||
print()
|
||||
print(" 其他:")
|
||||
print(" GET /health - 健康检查")
|
||||
print(" GET /api/tasks - 任务列表(调试)")
|
||||
print("=" * 50)
|
||||
print()
|
||||
print("🌐 服务器地址: http://0.0.0.0:5000")
|
||||
print("🔄 自动清理机制已启用 - 临时文件将在2小时后自动删除")
|
||||
print()
|
||||
|
||||
# 启动Flask应用
|
||||
from file_cleanup import start_cleanup_thread
|
||||
start_cleanup_thread()
|
||||
|
||||
app.run(host='0.0.0.0', port=5000, debug=False, threaded=True)
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
391
video_frame_utils.py
Normal file
391
video_frame_utils.py
Normal file
@@ -0,0 +1,391 @@
|
||||
import os
|
||||
import sys
|
||||
import uuid
|
||||
import subprocess
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
from datetime import datetime, timedelta
|
||||
import json
|
||||
import glob
|
||||
import requests
|
||||
import logging
|
||||
from shared_utils import task_registry, file_registry, file_lock, task_lock, add_task_log
|
||||
|
||||
# ======================
|
||||
# 配置
|
||||
# ======================
|
||||
BASE_DIR = os.path.dirname(os.path.abspath(__file__))
|
||||
FRAMES_ROOT = os.path.abspath("./frames")
|
||||
TASK_TIMEOUT_HOURS = 1 # 任务过期时间(小时)
|
||||
|
||||
# 确保frames目录存在
|
||||
os.makedirs(FRAMES_ROOT, exist_ok=True)
|
||||
|
||||
if sys.platform.startswith('win'):
|
||||
FFMPEG_PATH = os.path.join(BASE_DIR, 'lib', 'ffmpeg', 'bin', 'ffmpeg.exe')
|
||||
FFPROBE_PATH = os.path.join(BASE_DIR, 'lib', 'ffmpeg', 'bin', 'ffprobe.exe')
|
||||
else:
|
||||
FFMPEG_PATH = os.path.join(BASE_DIR, 'lib', 'ffmpeg', 'bin', 'ffmpeg')
|
||||
FFPROBE_PATH = os.path.join(BASE_DIR, 'lib', 'ffmpeg', 'bin', 'ffprobe')
|
||||
|
||||
def log_subprocess_output(pipe, task_id, task_registry, task_lock, prefix=""):
|
||||
"""从管道实时读取并记录日志"""
|
||||
if not pipe:
|
||||
return
|
||||
try:
|
||||
for line in iter(pipe.readline, ''):
|
||||
if line:
|
||||
clean_line = line.strip()
|
||||
if clean_line and task_id and task_registry and task_lock:
|
||||
add_task_log(task_id, f"{prefix}{clean_line}", task_registry, task_lock)
|
||||
except Exception as e:
|
||||
if task_id and task_registry and task_lock:
|
||||
add_task_log(task_id, f"[日志读取错误] {e}", task_registry, task_lock)
|
||||
finally:
|
||||
pipe.close()
|
||||
|
||||
# ======================
|
||||
# 任务处理函数
|
||||
# ======================
|
||||
|
||||
def process_video_frame_extraction(data, file_registry, file_lock, task_id=None, task_registry=None, task_lock=None):
|
||||
"""处理视频帧提取任务"""
|
||||
try:
|
||||
if task_id and task_registry and task_lock:
|
||||
add_task_log(task_id, "开始处理视频帧提取任务", task_registry, task_lock)
|
||||
|
||||
logging.info(f"开始处理视频帧提取任务: {task_id}")
|
||||
|
||||
# 提取参数
|
||||
video_url = data.get('video_url')
|
||||
video_bv = data.get('video_bv') # 支持BV号
|
||||
w = data.get('w')
|
||||
h = data.get('h')
|
||||
fps = data.get('fps', 30)
|
||||
force_resolution = data.get('force_resolution', False)
|
||||
pad_to_target = data.get('pad_to_target', False)
|
||||
|
||||
# 验证参数
|
||||
if not video_url and not video_bv:
|
||||
raise Exception("缺少必要参数: video_url 或 video_bv")
|
||||
if w is None or h is None:
|
||||
raise Exception("缺少必要参数: w, h")
|
||||
|
||||
try:
|
||||
w = int(w)
|
||||
h = int(h)
|
||||
fps = float(fps)
|
||||
if w <= 0 or h <= 0 or fps <= 0:
|
||||
raise ValueError("参数必须为正数")
|
||||
except (ValueError, TypeError) as e:
|
||||
raise Exception(f"参数无效: {str(e)}")
|
||||
|
||||
# 如果提供了BV号,自动生成Bilibili URL
|
||||
if not video_url and video_bv:
|
||||
video_bv = str(video_bv).strip()
|
||||
if not video_bv.startswith(('BV', 'bv')):
|
||||
raise Exception("video_bv 必须以 BV 或 bv 开头")
|
||||
video_url = f"https://www.bilibili.com/video/{video_bv.upper()}"
|
||||
|
||||
# 准备目录
|
||||
job_dir = os.path.join(FRAMES_ROOT, task_id)
|
||||
os.makedirs(job_dir, exist_ok=True)
|
||||
|
||||
if task_id and task_registry and task_lock:
|
||||
add_task_log(task_id, f"创建任务目录: {job_dir}", task_registry, task_lock)
|
||||
with task_lock:
|
||||
if task_id in task_registry:
|
||||
task_registry[task_id]['progress'] = 10
|
||||
|
||||
# 下载视频
|
||||
temp_base = os.path.join(tempfile.gettempdir(), task_id)
|
||||
|
||||
# === 替换 yt-dlp 下载部分 ===
|
||||
if task_id and task_registry and task_lock:
|
||||
add_task_log(task_id, f"开始下载视频: {video_url}", task_registry, task_lock)
|
||||
|
||||
yt_dlp_cmd = [
|
||||
sys.executable, '-m', 'yt_dlp',
|
||||
video_url,
|
||||
'-o', temp_base,
|
||||
]
|
||||
|
||||
# 使用 Popen 实时捕获 stderr
|
||||
proc = subprocess.Popen(
|
||||
yt_dlp_cmd,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
text=True,
|
||||
bufsize=1,
|
||||
universal_newlines=True
|
||||
)
|
||||
|
||||
# 启动日志线程(只读 stderr,因为 yt-dlp 进度在 stderr)
|
||||
stderr_thread = threading.Thread(
|
||||
target=log_subprocess_output,
|
||||
args=(proc.stderr, task_id, task_registry, task_lock, "[yt-dlp] "),
|
||||
daemon=True
|
||||
)
|
||||
stderr_thread.start()
|
||||
|
||||
try:
|
||||
proc.wait(timeout=600)
|
||||
if proc.returncode != 0:
|
||||
raise subprocess.CalledProcessError(proc.returncode, yt_dlp_cmd)
|
||||
except subprocess.TimeoutExpired:
|
||||
proc.kill()
|
||||
raise Exception("yt-dlp 下载超时(超过10分钟)")
|
||||
finally:
|
||||
stderr_thread.join(timeout=5) # 等待日志线程结束
|
||||
|
||||
# 查找实际生成的文件
|
||||
candidates = glob.glob(temp_base + ".*")
|
||||
if not candidates:
|
||||
raise Exception("yt-dlp 执行成功但未生成任何视频文件")
|
||||
|
||||
# 按修改时间取最新
|
||||
temp_video = max(candidates, key=os.path.getmtime)
|
||||
|
||||
if task_id and task_registry and task_lock:
|
||||
add_task_log(task_id, "下载完成,开始获取视频信息", task_registry, task_lock)
|
||||
with task_lock:
|
||||
if task_id in task_registry:
|
||||
task_registry[task_id]['progress'] = 40
|
||||
|
||||
# 获取视频信息
|
||||
duration = get_video_duration(temp_video)
|
||||
|
||||
# 构建滤镜
|
||||
vf = build_video_filter(w, h, fps, force_resolution, pad_to_target)
|
||||
|
||||
# 提取帧
|
||||
if task_id and task_registry and task_lock:
|
||||
add_task_log(task_id, "开始提取视频帧", task_registry, task_lock)
|
||||
with task_lock:
|
||||
if task_id in task_registry:
|
||||
task_registry[task_id]['progress'] = 50
|
||||
|
||||
frame_pattern = os.path.join(job_dir, "frame_%06d.png")
|
||||
|
||||
ffmpeg_cmd = [
|
||||
FFMPEG_PATH, '-y',
|
||||
'-i', temp_video,
|
||||
'-vf', vf,
|
||||
'-pix_fmt', 'rgb24',
|
||||
'-stats', # 关键:启用进度统计
|
||||
'-v', 'info', # 确保输出级别足够
|
||||
frame_pattern
|
||||
]
|
||||
|
||||
proc = subprocess.Popen(
|
||||
ffmpeg_cmd,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
text=True,
|
||||
bufsize=1,
|
||||
universal_newlines=True
|
||||
)
|
||||
|
||||
# FFmpeg 的进度在 stderr
|
||||
stderr_thread = threading.Thread(
|
||||
target=log_subprocess_output,
|
||||
args=(proc.stderr, task_id, task_registry, task_lock, "[FFmpeg] "),
|
||||
daemon=True
|
||||
)
|
||||
stderr_thread.start()
|
||||
|
||||
try:
|
||||
proc.wait()
|
||||
if proc.returncode != 0:
|
||||
raise subprocess.CalledProcessError(proc.returncode, ffmpeg_cmd)
|
||||
finally:
|
||||
stderr_thread.join(timeout=5)
|
||||
|
||||
# 获取输出分辨率
|
||||
if task_id and task_registry and task_lock:
|
||||
add_task_log(task_id, "分析输出结果", task_registry, task_lock)
|
||||
with task_lock:
|
||||
if task_id in task_registry:
|
||||
task_registry[task_id]['progress'] = 80
|
||||
|
||||
out_w, out_h = get_output_resolution(job_dir)
|
||||
total_frames = len([f for f in os.listdir(job_dir) if f.endswith('.png')])
|
||||
|
||||
# 检查音频
|
||||
audio_exists = has_audio(temp_video)
|
||||
|
||||
if audio_exists:
|
||||
if task_id and task_registry and task_lock:
|
||||
add_task_log(task_id, "提取音频(左/右/混合声道)", task_registry, task_lock)
|
||||
with task_lock:
|
||||
if task_id in task_registry:
|
||||
task_registry[task_id]['progress'] = 90
|
||||
|
||||
# 公共参数
|
||||
dfpwm_args = [
|
||||
FFMPEG_PATH, '-y',
|
||||
'-i', temp_video,
|
||||
'-vn',
|
||||
'-ar', '48000',
|
||||
'-ac', '1',
|
||||
'-f', 'dfpwm'
|
||||
]
|
||||
|
||||
# 混合声道
|
||||
dfpwm_path_mix = os.path.join(job_dir, "audio.dfpwm")
|
||||
subprocess.run(
|
||||
dfpwm_args + [dfpwm_path_mix],
|
||||
check=True, capture_output=True
|
||||
)
|
||||
|
||||
# 左声道
|
||||
dfpwm_path_left = os.path.join(job_dir, "audio_left.dfpwm")
|
||||
subprocess.run(
|
||||
dfpwm_args + ['-af', 'pan=mono|c0=c0'] + [dfpwm_path_left],
|
||||
check=True, capture_output=True
|
||||
)
|
||||
|
||||
# 右声道
|
||||
dfpwm_path_right = os.path.join(job_dir, "audio_right.dfpwm")
|
||||
subprocess.run(
|
||||
dfpwm_args + ['-af', 'pan=mono|c0=c1'] + [dfpwm_path_right],
|
||||
check=True, capture_output=True
|
||||
)
|
||||
|
||||
# 生成结果
|
||||
result_data = {
|
||||
"duration_seconds": round(duration, 3),
|
||||
"total_frames": total_frames,
|
||||
"fps": fps,
|
||||
"output_resolution": {"w": out_w, "h": out_h},
|
||||
"frame_urls": [f"/frames/{task_id}/frame_{i:06d}.png" for i in range(1, total_frames + 1)],
|
||||
"audio_dfpwm_url": f"/frames/{task_id}/audio.dfpwm" if has_audio else None,
|
||||
"audio_dfpwm_left_url": f"/frames/{task_id}/audio_left.dfpwm" if has_audio else None,
|
||||
"audio_dfpwm_right_url": f"/frames/{task_id}/audio_right.dfpwm" if has_audio else None
|
||||
}
|
||||
|
||||
# 注册文件到文件注册表
|
||||
with file_lock:
|
||||
for frame_file in glob.glob(os.path.join(job_dir, "frame_*.png")):
|
||||
filename = os.path.basename(frame_file)
|
||||
file_id = f"frame_{task_id}_{filename}"
|
||||
file_registry[file_id] = {
|
||||
'path': os.path.abspath(frame_file),
|
||||
'filename': filename,
|
||||
'last_access': time.time(),
|
||||
'download_count': 0
|
||||
}
|
||||
|
||||
if audio_exists:
|
||||
for audio_file in ["audio.dfpwm", "audio_left.dfpwm", "audio_right.dfpwm"]:
|
||||
audio_path = os.path.join(job_dir, audio_file)
|
||||
if os.path.exists(audio_path):
|
||||
file_id = f"audio_{task_id}_{audio_file}"
|
||||
file_registry[file_id] = {
|
||||
'path': os.path.abspath(audio_path),
|
||||
'filename': audio_file,
|
||||
'last_access': time.time(),
|
||||
'download_count': 0
|
||||
}
|
||||
|
||||
if task_id and task_registry and task_lock:
|
||||
add_task_log(task_id, "视频帧提取任务完成", task_registry, task_lock)
|
||||
with task_lock:
|
||||
if task_id in task_registry:
|
||||
task_registry[task_id]['progress'] = 100
|
||||
|
||||
return {
|
||||
'status': 'success',
|
||||
'result': result_data,
|
||||
'task_id': task_id,
|
||||
'temp_dir': job_dir
|
||||
}
|
||||
|
||||
except subprocess.CalledProcessError as e:
|
||||
stderr_str = e.stderr.strip() if e.stderr else ""
|
||||
stdout_str = e.stdout.strip() if e.stdout else ""
|
||||
error_output = stderr_str or stdout_str or str(e)
|
||||
error_msg = error_output[-500:] if len(error_output) > 500 else error_output
|
||||
|
||||
if task_id and task_registry and task_lock:
|
||||
add_task_log(task_id, f"FFmpeg处理失败: {error_msg}", task_registry, task_lock)
|
||||
|
||||
return {'error': f'FFmpeg处理失败: {error_msg}'}
|
||||
except requests.RequestException as e:
|
||||
error_msg = f'视频下载失败: {str(e)}'
|
||||
if task_id and task_registry and task_lock:
|
||||
add_task_log(task_id, error_msg, task_registry, task_lock)
|
||||
return {'error': error_msg}
|
||||
except Exception as e:
|
||||
error_msg = f'处理失败: {str(e)}'
|
||||
if task_id and task_registry and task_lock:
|
||||
add_task_log(task_id, error_msg, task_registry, task_lock)
|
||||
return {'error': error_msg}
|
||||
finally:
|
||||
# 清理临时视频文件
|
||||
for f in glob.glob(os.path.join(tempfile.gettempdir(), f"{task_id}.*")):
|
||||
try:
|
||||
os.remove(f)
|
||||
except:
|
||||
pass
|
||||
|
||||
# ======================
|
||||
# 辅助函数
|
||||
# ======================
|
||||
|
||||
def get_video_duration(video_path):
|
||||
"""获取视频时长"""
|
||||
result = subprocess.run([
|
||||
FFPROBE_PATH, '-v', 'error',
|
||||
'-show_entries', 'format=duration',
|
||||
'-of', 'csv=p=0',
|
||||
video_path
|
||||
], capture_output=True, text=True, check=True)
|
||||
|
||||
duration_str = result.stdout.strip()
|
||||
return float(duration_str) if duration_str and duration_str != 'N/A' else 0.0
|
||||
|
||||
def build_video_filter(w, h, fps, force_resolution, pad_to_target):
|
||||
"""构建视频滤镜"""
|
||||
if force_resolution:
|
||||
return f"scale={w}:{h},fps={fps}"
|
||||
else:
|
||||
scale_expr = f"scale='min({w},iw*min(1,{h}/ih))':'min({h},ih*min(1,{w}/iw))'"
|
||||
if pad_to_target:
|
||||
return f"{scale_expr},pad={w}:{h}:(ow-iw)/2:(oh-ih)/2,fps={fps}"
|
||||
else:
|
||||
return f"{scale_expr},fps={fps}"
|
||||
|
||||
def get_output_resolution(job_dir):
|
||||
"""获取输出分辨率"""
|
||||
first_frame = None
|
||||
for f in sorted(os.listdir(job_dir)):
|
||||
if f.endswith('.png'):
|
||||
first_frame = os.path.join(job_dir, f)
|
||||
break
|
||||
|
||||
if first_frame:
|
||||
probe_res = subprocess.run([
|
||||
FFPROBE_PATH, '-v', 'error',
|
||||
'-select_streams', 'v:0',
|
||||
'-show_entries', 'stream=width,height',
|
||||
'-of', 'csv=p=0',
|
||||
first_frame
|
||||
], capture_output=True, text=True, check=True)
|
||||
|
||||
out_w, out_h = map(int, probe_res.stdout.strip().split(','))
|
||||
return out_w, out_h
|
||||
|
||||
return 0, 0
|
||||
|
||||
def has_audio(video_path):
|
||||
"""检查视频是否有音频"""
|
||||
probe_streams = subprocess.run([
|
||||
FFPROBE_PATH, '-v', 'error',
|
||||
'-show_entries', 'stream=codec_type',
|
||||
'-of', 'csv=p=0',
|
||||
video_path
|
||||
], capture_output=True, text=True)
|
||||
|
||||
return 'audio' in probe_streams.stdout.lower()
|
||||
Reference in New Issue
Block a user