djangoadmin/DjangoAdmin_FastAPI_EleVue
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(phase5): 解耦与收尾清理

Browse Source 
- core/redis_client.py 移除 scheduler 耦合，lifespan 由 core/app.py 组合（Redis→调度器→Redis）
- main.py 改从 core.app 导入 create_app
- 删除 apps/、extensions/ 目录；utils 删除已迁移垫片与死代码 formatter.py
- 新增 deploy/（nginx、supervisor）、scripts/（init_db、seed）、logs/（结构化日志落盘）
- core/app.py 接入 core.logger setup_logging
- 更新 CLAUDE.md 反映 core/shared/modules 新结构

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Claude
2026-08-07 10:16:44 +08:00
co-authored by Claude
parent 64a5504396
commit 63bebedf05
99 changed files with 229 additions and 631 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+16 -16
View File
@@ -53,30 +53,30 @@ async def add(request: Request, data: LevelForm):
return await level.LevelAdd(request, data)
```
Validation errors are caught by a global `RequestValidationError` handler registered in `apps/__init__.py` and converted to the standard response shape `{"code": -1, "data": null, "msg": "field: message/..."}` with HTTP 200. Services receive the validated Pydantic instance and use `data.model_dump()` (full dict), `data.model_dump(exclude={'id'})` (for updates), or direct attribute access (`data.id`, `data.status`).
Validation errors are caught by a global `RequestValidationError` handler registered in `core/app.py` and converted to the standard response shape `{"code": -1, "data": null, "msg": "field: message/..."}` with HTTP 200. Services receive the validated Pydantic instance and use `data.model_dump()` (full dict), `data.model_dump(exclude={'id'})` (for updates), or direct attribute access (`data.id`, `data.status`).
### Key Directories
- **`config/`** - Configuration split into sub-modules: `app.py` (app settings, CORS, upload limits), `database.py` (DB connection, pool config, `SQLALCHEMY_MYSQL_URL`), `redis.py`, `auth.py` (JWT), `captcha.py`, `email.py`, `sms.py`. All loaded from `.env` via `python-dotenv`. `config/__init__.py` loads `.env` first, then re-exports everything.
- **`extensions/`** - Framework extensions: SQLAlchemy engine/session/contextvars (`extensions/sqlalchemy.py`), custom exception class (`extensions/exception.py`), `get_db_session()` accessor.
- **`routers/v1.py`** - Central route registration. All module routers are included here with prefix paths.
- **`apps/middleware/`** - `login_required` (JWT auth dependency, ignores `/login` and `/captcha`), `permission_required` (decorator checking user's permission nodes; admin userId=1 bypasses all checks), `check_demo` (decorator that blocks write operations when `FASTAPI_DEMO=True`), `db_session` (Starlette middleware setting request-scoped DB session via contextvars).
- **`utils/`** - `response` (JSON response helpers: `R.ok()`, `R.failed()`), `jwt` (JWT create/parse/refresh/blacklist with HS256), `password` (bcrypt-based password hashing with salt), `security` (`get_user_id(request)`, `get_username(request)` — reads from `request.state` set by `login_required`), `request` (`parse_batch_ids`, `like_escape`, `like_contains` for safe LIKE queries), `captcha` (verification code generation), `file` (file operations), `redis_ctx` (Redis connection pool lifecycle and request-scoped context via `get_redis()`), `ip`, `formatter`, `rich_text`.
- **`apps/constants/`** - `constants.py` (enum dicts for gender, dept types, menu types, etc.), `message.py` (default page size, messages, `SERIALIZE_EXCLUDE_FIELDS`).
- **`core/`** - Core infrastructure: `database.py` (engine/SessionLocal/`_db_ctx`/`get_db_session()`/`commit_independently`/`Base`), `redis_client.py` (`get_redis()`/`redis_lifespan`/`redis_middleware`), `jwt.py` (JWT create/parse/refresh/blacklist with HS256), `password.py` (bcrypt hashing with salt), `response.py` (`R.ok()`/`R.failed()`), `exceptions.py` (AuthorizationException/BusinessException), `security.py` (`get_user_id`/`get_username`), `middleware.py` (`login_required`/`permission_required`/`check_demo`/`db_session_middleware`/`operation_log_middleware`/`register_cros`), `deps.py` (DI), `logger.py` (structured logging), `app.py` (create_app factory + lifespan).
- **`shared/`** - Cross-module shared layer: `enums.py` (enum dicts), `constants.py` (PAGE_SIZE/DEFAULT_PASSWORD/SERIALIZE_EXCLUDE_FIELDS), `pagination.py`, `base_model.py`, `base_db.py`, `base_repository.py` (Generic repository base), `base_service.py`.
- **`modules/`** - Business modules, each a directory with `router.py` (may export multiple routers, e.g. `dict_router` + `dict_item_router`), `schemas.py`, `service.py`, `repository.py`, `models.py`. Module groups: `auth/`, `system/` (user, role, menu, dept, position, level), `dictionary/`, `configuration/`, `cms/` (item, item_cate, ad, ad_sort), `member/`, `region/`, `link/`, `notice/`, `job/` (job + job_log + scheduler + job_executor), `log/`, `upload/`, `index/`.
- **`routers/`** - `v1.py` central route registration (imports routers from `modules/`), `__init__.py` mounts login (免认证) on app and v1 behind `login_required`.
- **`utils/`** - Remaining utilities that stay here: `request` (`parse_batch_ids`, `like_escape`, `like_contains`), `captcha`, `file`, `ip`/`ip2region`, `ua`, `rich_text`, `perm_cache`. `R`/`password` are re-exported from `core` via `utils/__init__.py` for back-compat.
### Database Access
Database sessions are request-scoped via `contextvars` + Starlette middleware (`apps/middleware/db_session.py`). Services call `db = get_db_session()` to get the current request's session. Models use `self.save()` / `self.delete()` from `base_db`. The middleware auto-closes the session after each request (with rollback). Soft deletes via `is_delete` flag (0=active, 1=deleted).
Database sessions are request-scoped via `contextvars` + Starlette middleware (`core/middleware.py: db_session_middleware`). Services call `db = get_db_session()` (from `core.database`) to get the current request's session. Models use `self.save()` / `self.delete()` from `shared/base_db.py`. The middleware auto-closes the session after each request (with rollback). Soft deletes via `is_delete` flag (0=active, 1=deleted).
### Redis Access
Redis uses async client via `utils/redis_ctx.py`. Connection pool created in `redis_lifespan` (app lifespan), injected per-request via `redis_middleware` into contextvars. Business code calls `get_redis()` to get the current request's async Redis client.
Redis uses async client via `core/redis_client.py`. Connection pool created in `redis_lifespan` (composed with scheduler startup in `core/app.py` lifespan), injected per-request via `redis_middleware` into contextvars. Business code calls `get_redis()` to get the current request's async Redis client.
### Authentication & Authorization
- JWT tokens via `utils/jwt.py` (HS256, configurable expiry via JWT_EXPIRE_MINUTES, secret via JWT_SALT env var). Token sent as `Authorization: Bearer <token>` header. Supports token refresh and blacklist (logout) via Redis.
- JWT tokens via `core/jwt.py` (HS256, configurable expiry via JWT_EXPIRE_MINUTES, secret via JWT_SALT env var). Token sent as `Authorization: Bearer <token>` header. Supports token refresh and blacklist (logout) via Redis.
- `login_required` is a FastAPI `Depends` applied globally to the v1 router. On success, it sets `request.state.user_id`, `request.state.username`, `request.state.realname`.
- `permission_required` is a per-endpoint decorator checking permission strings like `"sys:level:list"`. User ID 1 (admin) bypasses all permission checks.
- `permission_required` is a per-endpoint decorator checking permission strings like `"sys:level:list"`. User ID 1 (admin) bypasses all permission checks. It lazily queries `modules.system.menu.service.get_permissions_list_sync`.
- `check_demo` is a per-endpoint decorator that returns `R.failed("演示环境，暂无操作权限")` when `FASTAPI_DEMO=True`. Apply it after `@permission_required` on all write endpoints (add, update, delete, status, import, export).
### API Response Format
@@ -98,12 +98,12 @@ Single delete: `DELETE /delete/{id}` where id can be comma-separated (e.g., `"1,
## Adding a New Module
1. Create `apps/models/{name}.py` - model class inheriting `base_model` and `base_db`, set `__tablename__ = DB_PREFIX + "{name}"`
2. Create `apps/forms/{name}.py` - Pydantic `BaseModel` validation classes with `Field(...)` constraints and `@field_validator` for custom checks
3. Create `apps/services/{name}.py` - CRUD service functions following the `get_level_page`/`get_level_detail`/`add_level`/`update_level`/`delete_level`/`update_level_status` pattern
4. Create `apps/views/{name}.py` - APIRouter with endpoints, apply `@permission_required` and `@check_demo` on write endpoints
5. Register the router in `routers/v1.py` - `v1.include_router({name}.router, prefix="/{prefix}", tags=["标签"])`
6. Import the view in `routers/v1.py`
1. Create `modules/{group}/{name}/models.py` - model class inheriting `base_model` and `base_db` (from `shared/`), set `__tablename__ = DB_PREFIX + "{name}"`
2. Create `modules/{group}/{name}/schemas.py` - Pydantic `BaseModel` validation classes with `Field(...)` constraints and `@field_validator` for custom checks
3. Create `modules/{group}/{name}/repository.py` - subclass `shared.base_repository.BaseRepository` + module-level instance
4. Create `modules/{group}/{name}/service.py` - CRUD service functions following the `get_level_page`/`get_level_detail`/`add_level`/`update_level`/`delete_level`/`update_level_status` pattern
5. Create `modules/{group}/{name}/router.py` - APIRouter with endpoints, apply `@permission_required` and `@check_demo` on write endpoints
6. Register the router in `routers/v1.py` - `v1.include_router({name}_router, prefix="/{prefix}", tags=["标签"])`
## Key Dependencies
apps/__init__.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：应用工厂已迁移到 core/app.py，保留旧导入路径
from core.app import create_app
apps/constants/__init__.py
View File
apps/constants/constants.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 shared/enums.py 迁移，保留旧导入路径
from shared.enums import * # noqa: F401,F403
apps/constants/message.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 shared/constants.py 迁移，保留旧导入路径
from shared.constants import * # noqa: F401,F403
apps/forms/__init__.py
View File
apps/forms/ad.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.cms.ad.schemas 迁移
from modules.cms.ad.schemas import * # noqa: F401,F403
apps/forms/ad_sort.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.cms.ad_sort.schemas 迁移
from modules.cms.ad_sort.schemas import * # noqa: F401,F403
apps/forms/city.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.region.schemas 迁移
from modules.region.schemas import * # noqa: F401,F403
apps/forms/config.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.configuration.schemas 迁移
from modules.configuration.schemas import * # noqa: F401,F403
apps/forms/config_item.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.configuration.schemas 迁移
from modules.configuration.schemas import * # noqa: F401,F403
apps/forms/dept.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.system.dept.schemas 迁移
from modules.system.dept.schemas import * # noqa: F401,F403
apps/forms/dict.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.dictionary.schemas 迁移
from modules.dictionary.schemas import * # noqa: F401,F403
apps/forms/dict_item.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.dictionary.schemas 迁移
from modules.dictionary.schemas import * # noqa: F401,F403
apps/forms/item.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.cms.item.schemas 迁移
from modules.cms.item.schemas import * # noqa: F401,F403
apps/forms/item_cate.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.cms.item_cate.schemas 迁移
from modules.cms.item_cate.schemas import * # noqa: F401,F403
apps/forms/job.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.job.schemas 迁移
from modules.job.schemas import * # noqa: F401,F403
apps/forms/level.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.system.level.schemas 迁移
from modules.system.level.schemas import * # noqa: F401,F403
apps/forms/link.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.link.schemas 迁移
from modules.link.schemas import * # noqa: F401,F403
apps/forms/login.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.auth.schemas 迁移
from modules.auth.schemas import * # noqa: F401,F403
apps/forms/member.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.member.schemas 迁移
from modules.member.schemas import * # noqa: F401,F403
apps/forms/member_level.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.member.schemas 迁移
from modules.member.schemas import * # noqa: F401,F403
apps/forms/menu.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.system.menu.schemas 迁移
from modules.system.menu.schemas import * # noqa: F401,F403
apps/forms/notice.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.notice.schemas 迁移
from modules.notice.schemas import * # noqa: F401,F403
apps/forms/position.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.system.position.schemas 迁移
from modules.system.position.schemas import * # noqa: F401,F403
apps/forms/role.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.system.role.schemas 迁移
from modules.system.role.schemas import * # noqa: F401,F403
apps/forms/role_menu.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.system.role.schemas 迁移
from modules.system.role.schemas import * # noqa: F401,F403
apps/forms/user.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.system.user.schemas 迁移
from modules.system.user.schemas import * # noqa: F401,F403
apps/middleware/.gitignore
-2
View File
@@ -1,2 +0,0 @@
*
!.gitignore
apps/models/.gitignore
-2
View File
@@ -1,2 +0,0 @@
*
!.gitignore
apps/services/__init__.py
View File
apps/services/ad.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.cms.ad.service 迁移
from modules.cms.ad.service import * # noqa: F401,F403
apps/services/ad_sort.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.cms.ad_sort.service 迁移
from modules.cms.ad_sort.service import * # noqa: F401,F403
apps/services/city.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.region.service 迁移
from modules.region.service import * # noqa: F401,F403
apps/services/config.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.configuration.service 迁移
from modules.configuration.service import * # noqa: F401,F403
apps/services/config_item.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.configuration.service 迁移
from modules.configuration.service import * # noqa: F401,F403
apps/services/dept.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.system.dept.service 迁移
from modules.system.dept.service import * # noqa: F401,F403
apps/services/dict.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.dictionary.service 迁移
from modules.dictionary.service import * # noqa: F401,F403
apps/services/dict_item.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.dictionary.service 迁移
from modules.dictionary.service import * # noqa: F401,F403
apps/services/item.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.cms.item.service 迁移
from modules.cms.item.service import * # noqa: F401,F403
apps/services/item_cate.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.cms.item_cate.service 迁移
from modules.cms.item_cate.service import * # noqa: F401,F403
apps/services/job.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.job.service 迁移
from modules.job.service import * # noqa: F401,F403
apps/services/job_executor.py
-57
View File
@@ -1,57 +0,0 @@
"""
任务执行器模块
根据触发器标识执行对应任务的功能。
"""
import logging
from typing import Any, Dict
logger = logging.getLogger(__name__)
class JobExecutor:
"""任务执行器：根据触发器标识调用对应的内部方法执行任务"""
def __init__(self):
self._handlers = {}
self._register_default_handlers()
def _register_default_handlers(self):
"""注册默认的任务处理器"""
self.register('send_sms', self._send_sms)
self.register('send_email', self._send_email)
self.register('sync_data', self._sync_data)
def register(self, trigger: str, handler):
"""注册任务处理器"""
self._handlers[trigger] = handler
logger.info(f"注册任务处理器: {trigger}")
def execute(self, trigger: str, **kwargs) -> Any:
"""执行任务"""
if trigger not in self._handlers:
raise KeyError(f"未注册的任务触发器: {trigger}（可用 job_executor.register() 注册处理器）")
handler = self._handlers[trigger]
logger.info(f"执行任务: {trigger}, 参数: {kwargs}")
return handler(**kwargs)
def _send_sms(self, **kwargs) -> Dict:
"""发送短信任务"""
logger.info(f"发送短信 [任务ID: {kwargs.get('job_id')}]")
return {'success': True, 'message': '短信发送成功'}
def _send_email(self, **kwargs) -> Dict:
"""发送邮件任务"""
logger.info(f"发送邮件 [任务ID: {kwargs.get('job_id')}]")
return {'success': True, 'message': '邮件发送成功'}
def _sync_data(self, **kwargs) -> Dict:
"""数据同步任务"""
logger.info(f"数据同步 [任务ID: {kwargs.get('job_id')}]")
return {'success': True, 'message': '数据同步成功'}
# 全局执行器实例
job_executor = JobExecutor()
apps/services/job_log.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.job.service 迁移
from modules.job.service import * # noqa: F401,F403
apps/services/level.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.system.level.service 迁移
from modules.system.level.service import * # noqa: F401,F403
apps/services/link.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.link.service 迁移
from modules.link.service import * # noqa: F401,F403
apps/services/login.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.auth.service 迁移
from modules.auth.service import * # noqa: F401,F403
apps/services/login_log.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.log.service 迁移
from modules.log.service import * # noqa: F401,F403
apps/services/member.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.member.service 迁移
from modules.member.service import * # noqa: F401,F403
apps/services/member_level.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.member.service 迁移
from modules.member.service import * # noqa: F401,F403
apps/services/menu.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.system.menu.service 迁移
from modules.system.menu.service import * # noqa: F401,F403
apps/services/notice.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.notice.service 迁移
from modules.notice.service import * # noqa: F401,F403
apps/services/operation_log.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.log.service 迁移
from modules.log.service import * # noqa: F401,F403
apps/services/position.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.system.position.service 迁移
from modules.system.position.service import * # noqa: F401,F403
apps/services/role.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.system.role.service 迁移
from modules.system.role.service import * # noqa: F401,F403
apps/services/role_menu.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.system.role.service 迁移
from modules.system.role.service import * # noqa: F401,F403
apps/services/scheduler.py
-368
View File
@@ -1,368 +0,0 @@
"""
定时任务调度核心模块
使用 APScheduler 管理和执行定时任务。
"""
import json
import logging
import threading
from datetime import datetime
from typing import Dict, Optional
import requests
from apscheduler.schedulers.background import BackgroundScheduler
from apscheduler.triggers.cron import CronTrigger
from apscheduler.jobstores.base import JobLookupError
from config import FASTAPI_HOST, FASTAPI_PORT
logger = logging.getLogger(__name__)
# 调度器实例
_scheduler: Optional[BackgroundScheduler] = None
def get_scheduler() -> BackgroundScheduler:
"""获取调度器单例"""
global _scheduler
if _scheduler is None:
_scheduler = BackgroundScheduler(timezone='Asia/Shanghai')
return _scheduler
def build_full_url(url: str) -> str:
"""构建完整URL，将相对路径拼接为带域名的URL"""
if not url:
return url
if url.startswith(('http://', 'https://')):
return url
if url.startswith('/'):
url = f"http://{FASTAPI_HOST}:{FASTAPI_PORT}{url}"
return url
# ============================================================
# 任务执行核心函数
# ============================================================
def execute_job(
job_id: int,
url: Optional[str] = None,
is_sync: int = 0,
job_trigger: Optional[str] = None,
params: Optional[Dict] = None
) -> Dict:
"""执行任务（调度器调用入口），执行完成后写入任务日志"""
start_time = datetime.now()
result = {'success': False, 'error': '未知错误'}
try:
logger.info(f"执行定时任务 [ID: {job_id}, URL: {url}, Trigger: {job_trigger}]")
if url:
result = _execute_by_url(job_id, url, is_sync, params)
elif job_trigger:
result = _execute_by_trigger(job_trigger, job_id, params)
else:
result = {'success': False, 'error': '任务既没有URL也没有触发器'}
return result
except Exception as e:
logger.error(f"执行任务异常 [ID: {job_id}]: {str(e)}")
result = {'success': False, 'error': str(e)}
return result
finally:
_record_execution_log(job_id, result, start_time)
def _record_execution_log(job_id: int, result: Dict, start_time: datetime):
"""记录一次任务执行日志"""
try:
from apps.services.job_log import record_job_log
end_time = datetime.now()
success = bool(result.get('success'))
status = 0 if success else 1
if not success:
job_message = '任务执行失败'
exception_info = str(result.get('error'))
elif result.get('status') == 'triggered':
job_message = '异步任务已触发'
exception_info = None
else:
job_message = '任务执行成功'
exception_info = None
record_job_log(job_id, start_time, end_time, status,
exception_info=exception_info, job_message=job_message)
except Exception as e:
logger.error(f"记录任务日志失败 [ID: {job_id}]: {str(e)}")
def _execute_by_url(job_id: int, url: str, is_sync: int, params: Optional[Dict] = None) -> Dict:
"""通过URL方式执行任务"""
full_url = build_full_url(url)
request_data = {
'job_id': job_id,
'timestamp': datetime.now().isoformat(),
'params': params or {}
}
if is_sync == 1:
return _send_sync_request(full_url, request_data, job_id)
else:
return _send_async_request(full_url, request_data, job_id)
def _send_sync_request(url: str, data: Dict, job_id: int) -> Dict:
"""发送同步HTTP请求"""
try:
logger.info(f"发送同步请求 [ID: {job_id}, URL: {url}]")
headers = {'Content-Type': 'application/json', 'User-Agent': 'Scheduler/1.0'}
response = requests.post(url, json=data, timeout=30, headers=headers)
response.raise_for_status()
try:
result_data = response.json()
except Exception:
result_data = {'text': response.text[:200]}
return {'success': True, 'status_code': response.status_code, 'data': result_data}
except requests.exceptions.RequestException as e:
return {'success': False, 'error': str(e)}
def _send_async_request(url: str, data: Dict, job_id: int) -> Dict:
"""发送异步HTTP请求（独立线程）"""
def async_request():
try:
headers = {'Content-Type': 'application/json'}
resp = requests.post(url, json=data, timeout=30, headers=headers)
logger.info(f"异步任务完成 [ID: {job_id}, 状态码: {resp.status_code}]")
except Exception as e:
logger.error(f"异步任务异常 [ID: {job_id}]: {str(e)}")
thread = threading.Thread(target=async_request)
thread.start()
return {'success': True, 'status': 'triggered'}
def _execute_by_trigger(trigger: str, job_id: int, params: Optional[Dict] = None) -> Dict:
"""通过触发器方式执行任务"""
try:
from apps.services.job_executor import job_executor
# 剔除 job_id，避免与 execute(job_id=...) 关键字参数冲突
trigger_params = dict(params or {})
trigger_params.pop('job_id', None)
result = job_executor.execute(trigger, job_id=job_id, **trigger_params)
# 处理器返回 success=False 时透传失败，而非一律视为成功
if isinstance(result, dict) and result.get('success') is False:
error = result.get('message') or result.get('error') or '任务执行失败'
return {'success': False, 'trigger': trigger, 'error': error}
return {'success': True, 'trigger': trigger, 'data': result}
except Exception as e:
return {'success': False, 'error': str(e)}
# ============================================================
# 调度器任务管理函数
# ============================================================
def _parse_job_params(params: Optional[str]) -> Dict:
"""解析任务参数（JSON字符串 -> dict），容错非法输入"""
if not params:
return {}
try:
parsed = json.loads(params)
if isinstance(parsed, dict):
return parsed
except Exception:
logger.warning(f"任务参数不是合法JSON，已忽略: {str(params)[:200]}")
return {}
def _build_misfire_config(execute_policy) -> Dict:
"""根据执行策略映射 APScheduler 的 misfire/coalesce 参数"""
if execute_policy == 3:
# 放弃执行：错过的运行不补
return {'misfire_grace_time': 0, 'coalesce': True}
if execute_policy == 2:
# 执行一次：短时间内错过的补跑一次
return {'misfire_grace_time': 600, 'coalesce': True}
# 立即执行（默认）：错过立即补跑一次
return {'misfire_grace_time': 3600, 'coalesce': True}
def add_job_to_scheduler(job_dict: dict) -> bool:
"""添加任务到调度器"""
try:
job_id = job_dict.get('id')
job_status = job_dict.get('status')
job_cron = job_dict.get('cron_expression')
job_url = job_dict.get('url')
job_sync = job_dict.get('is_sync', 0)
job_trigger = job_dict.get('job_trigger')
job_name = job_dict.get('job_name', '')
job_params = _parse_job_params(job_dict.get('params'))
if job_status != 1:
return False
if not job_cron:
logger.warning(f"跳过添加任务: Cron表达式为空 (id={job_id})")
return False
# 验证Cron表达式
try:
trigger = CronTrigger.from_crontab(job_cron)
next_run = trigger.get_next_fire_time(None, datetime.now())
if next_run is None:
logger.warning(f"Cron表达式不会触发 (id={job_id})")
return False
except Exception as e:
logger.error(f"Cron表达式无效 (id={job_id}): {str(e)}")
return False
scheduler = get_scheduler()
scheduler_job_id = f"job_{job_id}"
# 移除已存在的同名任务
try:
scheduler.remove_job(scheduler_job_id)
except JobLookupError:
pass
misfire_config = _build_misfire_config(job_dict.get('execute_policy', 1))
scheduler.add_job(
func=execute_job,
trigger=trigger,
args=[job_id, job_url, job_sync, job_trigger, job_params],
id=scheduler_job_id,
name=job_name,
replace_existing=True,
**misfire_config
)
logger.info(f"任务已添加到调度器 [ID: {job_id}, cron: {job_cron}]")
return True
except Exception as e:
logger.error(f"添加任务到调度器异常: {str(e)}")
return False
def remove_job_from_scheduler(job_id: int) -> bool:
"""从调度器移除任务"""
try:
get_scheduler().remove_job(f"job_{job_id}")
return True
except JobLookupError:
return True
except Exception as e:
logger.error(f"移除任务异常: {str(e)}")
return False
def pause_job_in_scheduler(job_id: int) -> bool:
"""暂停调度器中的任务"""
try:
get_scheduler().pause_job(f"job_{job_id}")
return True
except JobLookupError:
return True
except Exception as e:
logger.error(f"暂停任务异常: {str(e)}")
return False
def resume_job_in_scheduler(job_id: int) -> bool:
"""恢复调度器中的任务"""
try:
get_scheduler().resume_job(f"job_{job_id}")
return True
except JobLookupError:
return True
except Exception as e:
logger.error(f"恢复任务异常: {str(e)}")
return False
def run_job_once(job_id: int, params: Optional[Dict] = None) -> Dict:
"""立即执行一次任务（不依赖调度器）"""
try:
from apps.models.job import Job
from extensions import get_db_session
db = get_db_session()
job = db.query(Job).filter(Job.id == job_id, Job.is_delete == 0).first()
if not job:
return {'success': False, 'error': '任务不存在'}
return execute_job(job.id, job.url, job.is_sync, job.job_trigger, params or {})
except Exception as e:
return {'success': False, 'error': str(e)}
# ============================================================
# 调度器初始化
# ============================================================
def init_scheduler() -> int:
"""初始化调度器，启动并从数据库加载所有启用的任务"""
try:
from apps.models.job import Job
from extensions.sqlalchemy import SessionLocal
scheduler = get_scheduler()
if not scheduler.running:
scheduler.start()
logger.info("调度器已启动")
# lifespan 启动阶段无请求上下文，需用独立会话而非请求级 get_db_session()
session = SessionLocal()
try:
jobs = session.query(Job).filter(Job.status == 1, Job.is_delete == 0).all()
# 转为字典避免 Session 分离问题
job_dicts = [job.to_dict() for job in jobs]
finally:
session.close()
success_count = 0
for job_dict in job_dicts:
if add_job_to_scheduler(job_dict):
success_count += 1
logger.info(f"调度器初始化完成，加载 {success_count}/{len(job_dicts)} 个任务")
return success_count
except Exception as e:
logger.error(f"初始化调度器异常: {str(e)}")
return 0
def shutdown_scheduler():
"""关闭调度器"""
try:
scheduler = get_scheduler()
if scheduler.running:
scheduler.shutdown(wait=False)
logger.info("调度器已关闭")
except Exception as e:
logger.error(f"关闭调度器异常: {str(e)}")
def reload_scheduler() -> int:
"""重新加载调度器：清空所有任务，重新从数据库加载"""
try:
scheduler = get_scheduler()
scheduler.remove_all_jobs()
return init_scheduler()
except Exception as e:
logger.error(f"重新加载调度器异常: {str(e)}")
return 0
apps/services/upload.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.upload.service 迁移
from modules.upload.service import * # noqa: F401,F403
apps/services/user.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.system.user.service 迁移
from modules.system.user.service import * # noqa: F401,F403
apps/services/user_role.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.system.user.service 迁移
from modules.system.user.service import * # noqa: F401,F403
apps/views/__init__.py
View File
apps/views/ad.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.cms.ad.router 迁移
from modules.cms.ad.router import router
apps/views/ad_sort.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.cms.ad_sort.router 迁移
from modules.cms.ad_sort.router import router
apps/views/city.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.region.router 迁移
from modules.region.router import router
apps/views/config.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.configuration.router 迁移
from modules.configuration.router import config_router as router
apps/views/config_item.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.configuration.router 迁移
from modules.configuration.router import config_item_router as router
apps/views/dept.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.system.dept.router 迁移
from modules.system.dept.router import router
apps/views/dict.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.dictionary.router 迁移
from modules.dictionary.router import dict_router as router
apps/views/dict_item.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.dictionary.router 迁移
from modules.dictionary.router import dict_item_router as router
apps/views/index.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.index.router 迁移
from modules.index.router import router
apps/views/item.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.cms.item.router 迁移
from modules.cms.item.router import router
apps/views/item_cate.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.cms.item_cate.router 迁移
from modules.cms.item_cate.router import router
apps/views/job.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.job.router 迁移
from modules.job.router import job_router as router
apps/views/job_log.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.job.router 迁移
from modules.job.router import job_log_router as router
apps/views/level.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.system.level.router 迁移
from modules.system.level.router import router
apps/views/link.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.link.router 迁移
from modules.link.router import router
apps/views/login.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.auth.router 迁移
from modules.auth.router import router
apps/views/login_log.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.log.router 迁移
from modules.log.router import login_log_router as router
apps/views/member.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.member.router 迁移
from modules.member.router import member_router as router
apps/views/member_level.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.member.router 迁移
from modules.member.router import member_level_router as router
apps/views/menu.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.system.menu.router 迁移
from modules.system.menu.router import router
apps/views/notice.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.notice.router 迁移
from modules.notice.router import router
apps/views/operation_log.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.log.router 迁移
from modules.log.router import operation_log_router as router
apps/views/position.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.system.position.router 迁移
from modules.system.position.router import router
apps/views/role.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.system.role.router 迁移
from modules.system.role.router import role_router as router
apps/views/role_menu.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.system.role.router 迁移
from modules.system.role.router import role_menu_router as router
apps/views/upload.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.upload.router 迁移
from modules.upload.router import router
apps/views/user.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 modules.system.user.router 迁移
from modules.system.user.router import router
core/app.py
+23 -1
View File
@@ -8,7 +8,9 @@
"""应用工厂：create_app 组装中间件、路由、异常处理与 lifespan。"""
import logging
import time
from contextlib import asynccontextmanager
from fastapi import FastAPI
from fastapi.exceptions import RequestValidationError
@@ -18,6 +20,7 @@ from starlette.responses import JSONResponse
from config.app import FASTAPI_DEBUG
from core.exceptions import AuthorizationException, BusinessException
from core.logger import setup_logging
from core.middleware import (
register_middleware,
register_cros,
@@ -28,12 +31,31 @@ from core.redis_client import redis_lifespan, redis_middleware
from routers import register_router
@asynccontextmanager
async def _combined_lifespan(app: FastAPI):
"""组合 lifespan：先起 Redis，再启动定时任务调度器；退出时先停调度器再关 Redis。"""
async with redis_lifespan(app):
# 懒加载调度器，避免 core → 业务模块顶层依赖
from modules.job.scheduler import init_scheduler, shutdown_scheduler
job_count = init_scheduler()
logging.getLogger(__name__).info(
f"定时任务调度器已启动，加载 {job_count} 个任务")
try:
yield
finally:
shutdown_scheduler()
def create_app() -> FastAPI:
# 初始化结构化日志（控制台 + logs/app.log）
setup_logging()
app = FastAPI(
title="FastAPI",
description="FastAPI",
version="v1",
lifespan=redis_lifespan,
lifespan=_combined_lifespan,
)
# 注册跨域中间件
core/redis_client.py
-7
View File
@@ -56,14 +56,7 @@ async def redis_lifespan(app: FastAPI):
print(f"Redis 连接成功 --> {REDIS_HOST}:{REDIS_PORT}/{REDIS_INDEX}")
logger.info(f"Redis 连接成功: {REDIS_HOST}:{REDIS_PORT}/{REDIS_INDEX}")
# startup: 初始化定时任务调度器
from modules.job.scheduler import init_scheduler, shutdown_scheduler
job_count = init_scheduler()
logger.info(f"定时任务调度器已启动，加载 {job_count} 个任务")
yield
# shutdown: 关闭调度器
shutdown_scheduler()
# shutdown: 关闭连接
await redis.aclose()
deploy/nginx.conf
+27
View File
@@ -0,0 +1,27 @@
# Nginx 反向代理示例配置
# 将 8031 端口的 FastAPI 服务暴露到 80 端口
# 部署时将 server_name 与 proxy_pass 替换为实际值
server {
listen 80;
server_name _;
# 上传文件大小上限（与 .env 中 UPLOAD_* 保持一致）
client_max_body_size 50m;
# 静态资源（如前端 build 产物）可选配置
# location /assets/ {
# root /var/www/frontend/dist;
# }
# API 反向代理
location / {
proxy_pass http://127.0.0.1:8031;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 120s;
}
}
deploy/supervisor.conf
+17
View File
@@ -0,0 +1,17 @@
; Supervisor 进程管理示例配置
; 将 command / directory 替换为实际路径后复制到 /etc/supervisor/conf.d/
[program:djangoadmin-fastapi]
; 启动命令：使用项目虚拟环境中的 uvicorn
command=/path/to/venv/bin/uvicorn main:app --host 127.0.0.1 --port 8031
; 项目根目录
directory=/path/to/DjangoAdmin_FastAPI_EleVue
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
redirect_stderr=true
stdout_logfile=/path/to/logs/supervisor.out.log
stdout_logfile_maxbytes=50MB
stdout_logfile_backups=10
environment=PYTHONUNBUFFERED=1
extensions/__init__.py
-6
View File
@@ -1,6 +0,0 @@
# 兼容垫片：由 core/ 迁移，保留旧导入路径
from core.database import get_db_session, commit_independently
def register_extends(app):
pass
extensions/exception.py
-2
View File
@@ -1,2 +0,0 @@
# 兼容垫片：由 core/exceptions.py 迁移，保留旧导入路径
from core.exceptions import AuthorizationException, BusinessException
extensions/sqlalchemy.py
-9
View File
@@ -1,9 +0,0 @@
# 兼容垫片：由 core/database.py 迁移，保留旧导入路径
from core.database import (
engine,
SessionLocal,
_db_ctx,
get_db_session,
commit_independently,
Base,
)
main.py
+1 -1
View File
@@ -39,7 +39,7 @@
# | 许可协议》附件为准。
# +======================================================================
from apps import create_app
from core.app import create_app
from config import FASTAPI_HOST, FASTAPI_PORT
app = create_app()
scripts/init_db.py
+97
View File
@@ -0,0 +1,97 @@
"""
数据库初始化脚本
=============================================================
功能：
1. 创建数据库（默认 utf8mb4）
2. 导入 document/djangoadmin.fastapi.elevue.sql（若存在）
3. 用模型 metadata 补建 SQL 中缺失的表（如 fastapi_login_log / fastapi_operation_log）
用法：
python scripts/init_db.py # 使用 .env 中的数据库配置
python scripts/init_db.py --drop-first # 先删除已存在的同名库再重建
"""
import argparse
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
def split_statements(content: str):
"""按 ';\\n' 切分语句，并剥离 -- 注释行，避免注释+语句整块被跳过。"""
for block in content.split(";\n"):
body = []
for line in block.splitlines():
s = line.strip()
if s.startswith("--") or s.startswith("#"):
continue
body.append(line)
stmt = "\n".join(body).strip()
if stmt:
yield stmt
def main():
parser = argparse.ArgumentParser(description="初始化数据库")
parser.add_argument("--drop-first", action="store_true", help="先删除已存在库再重建")
args = parser.parse_args()
# 加载 .env 数据库配置
from config import DB_HOST, DB_PORT, DB_DATABASE, DB_USERNAME, DB_PASSWORD, DB_PREFIX
import pymysql
from sqlalchemy import create_engine
from core.database import Base
# 加载全部模型（导入 create_app 会连带导入所有模块）
from core.app import create_app
create_app()
root_conn = pymysql.connect(host=DB_HOST, port=int(DB_PORT), user=DB_USERNAME, password=DB_PASSWORD)
cur = root_conn.cursor()
if args.drop_first:
cur.execute(f"DROP DATABASE IF EXISTS `{DB_DATABASE}`")
cur.execute(
f"CREATE DATABASE IF NOT EXISTS `{DB_DATABASE}` "
f"DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci"
)
cur.execute(f"USE `{DB_DATABASE}`")
print(f"database {DB_DATABASE} ready (prefix={DB_PREFIX})")
sql_path = os.path.join("document", "djangoadmin.fastapi.elevue.sql")
if os.path.exists(sql_path):
with open(sql_path, "r", encoding="utf-8") as f:
content = f.read()
done = errors = 0
for stmt in split_statements(content):
try:
cur.execute(stmt)
done += 1
except Exception as e:
errors += 1
print(f"[SKIP] {e}\n {stmt[:100]}")
root_conn.commit()
print(f"imported {done} statements, errors={errors}")
else:
print(f"!! 未找到 {sql_path}，跳过 SQL 导入")
# 用模型补建缺失表
engine = create_engine(
f"mysql+pymysql://{DB_USERNAME}:{DB_PASSWORD}@{DB_HOST}:{DB_PORT}/{DB_DATABASE}?charset=utf8mb4"
)
Base.metadata.create_all(engine)
from sqlalchemy import inspect
actual = set(inspect(engine).get_table_names())
missing = set(Base.metadata.tables) - actual
print(f"tables in DB: {len(actual)}; still missing: {sorted(missing)}")
engine.dispose()
cur.close()
root_conn.close()
if __name__ == "__main__":
main()
scripts/seed.py
+46
View File
@@ -0,0 +1,46 @@
"""
数据种子脚本：重置指定用户密码为 bcrypt 哈希。
用法：
python scripts/seed.py admin 123456 # 重置 admin 用户密码为 123456
python scripts/seed.py admin 123456 3 # 指定用户 ID 3
"""
import argparse
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
def main():
parser = argparse.ArgumentParser(description="重置用户密码")
parser.add_argument("username", help="用户名")
parser.add_argument("password", nargs="?", default="123456", help="新密码（默认 123456）")
parser.add_argument("user_id", nargs="?", type=int, default=None, help="用户 ID（可选，避免同名冲突）")
args = parser.parse_args()
from config import DB_HOST, DB_PORT, DB_DATABASE, DB_USERNAME, DB_PASSWORD
from core.password import generate_salt, encrypt_password
import pymysql
salt = generate_salt()
hashed = encrypt_password(args.password, salt)
conn = pymysql.connect(host=DB_HOST, port=int(DB_PORT), user=DB_USERNAME,
password=DB_PASSWORD, database=DB_DATABASE)
cur = conn.cursor()
if args.user_id:
cur.execute("UPDATE fastapi_user SET salt=%s, password=%s WHERE id=%s",
(salt, hashed, args.user_id))
else:
cur.execute("UPDATE fastapi_user SET salt=%s, password=%s WHERE username=%s",
(salt, hashed, args.username))
conn.commit()
print(f"updated {cur.rowcount} row(s) for username={args.username}")
cur.close()
conn.close()
if __name__ == "__main__":
main()
tests/contract/helpers.py
+2 -2
View File
@@ -21,7 +21,7 @@ UPDATE_GOLDEN = os.getenv("UPDATE_GOLDEN") == "1"
def collect_routes():
"""通过 openapi schema 收集全部 (METHOD /path)，去重排序。"""
from apps import create_app
from core.app import create_app
app = create_app()
spec = app.openapi()
@@ -44,7 +44,7 @@ def collect_permissions():
def collect_tables():
"""从 SQLAlchemy metadata 收集全部表名。"""
from extensions.sqlalchemy import Base
from core.database import Base
return sorted(Base.metadata.tables.keys())