This commit is contained in:
dlandy
2026-03-27 10:18:26 +08:00
commit 40be11adbf
116 changed files with 26138 additions and 0 deletions

View File

@@ -0,0 +1,746 @@
# ToolsShow NestJS Backend Design (v1.3 - Hybrid Access: Web + Download)
## 1. Overview
Current project is a static frontend (`index.html + app.js`) with in-memory tool data.
A new business constraint is introduced:
- some tools are opened directly via web URL (no download)
- some tools still require package download
This design keeps SQLite + self-hosted GitLab storage, and upgrades backend to support both access modes in one unified model.
## 2. Scope
### 2.1 In Scope (v1.3)
- Public APIs for tools/categories/keywords/overview
- Hybrid tool access:
- `web` mode: open target URL
- `download` mode: ticket + GitLab-backed file stream
- Admin backend APIs:
- admin login/logout/token refresh
- tool/category/tag/keyword management
- tool access-mode management
- artifact upload/version management for download-mode tools
- audit log query
- SQLite schema design and migration plan
- GitLab integration design for artifact upload/download
### 2.2 Out of Scope (v1.3)
- Multi-tenant architecture
- Fine-grained role/permission system (all active admins share capability)
- Recommendation engine
## 3. Tech Stack Selection
| Category | Selection | Reason |
|---|---|---|
| Runtime | Node.js 20 LTS | stable NestJS ecosystem |
| Framework | NestJS + TypeScript | modular architecture + DI |
| ORM | Prisma | schema/migration/type-safe client |
| Database | SQLite (`dev.db`) | low ops overhead and enough for current scale |
| File Storage | Self-hosted GitLab (Generic Package Registry) | artifact versioning and centralized storage |
| Auth | JWT (admin only) | simple and mature |
| Validation | `class-validator` + `class-transformer` | DTO safety |
| API Docs | Swagger | clear FE/BE contract |
| Logging | `pino` (`nestjs-pino`) | structured logging |
### 3.1 SQLite Decisions
- Enable WAL mode (`PRAGMA journal_mode=WAL`) for read/write concurrency.
- Use one writable instance in v1 to reduce lock contention.
- Use `TEXT` ids for flexible business identifiers.
- Use `DATETIME` and ISO-8601 API serialization.
### 3.2 Access Mode Strategy
- `access_mode = web`:
- tool is opened by URL
- no artifact required
- `access_mode = download`:
- tool requires at least one active artifact
- artifact stored in GitLab
Publish constraints:
- tool can be `published` only when mode requirements are satisfied.
## 4. Architecture Design
### 4.1 Layered Structure
- Controller Layer: route + DTO validation + response mapping
- Application Layer: use-case orchestration (query, launch, upload, download)
- Domain Layer: mode constraints, publish constraints, version constraints
- Infrastructure Layer: Prisma repository, GitLab client, cache, auth, logging
### 4.2 Module Breakdown
| Module | Responsibility | Depends On |
|---|---|---|
| `ToolModule` | public tool list/detail/search | Prisma, Cache |
| `CategoryModule` | category list + count | Prisma, Cache |
| `KeywordModule` | hot keywords | Prisma |
| `OverviewModule` | KPI aggregation | Prisma, Cache |
| `AccessModule` | unified launch entry for web/download modes | Prisma, GitlabStorage |
| `DownloadModule` | consume download ticket and stream package | Prisma, GitlabStorage |
| `ArtifactModule` | artifact metadata query | Prisma |
| `GitlabStorageModule` | GitLab upload/download encapsulation | HTTP client, Config |
| `AdminAuthModule` | admin auth | Prisma, JWT |
| `AdminToolModule` | tool CRUD + access mode setup | Prisma, AdminAuth |
| `AdminArtifactModule` | artifact upload/version management | Prisma, GitlabStorage |
| `AdminCategoryModule` | category CRUD/reorder | Prisma, AdminAuth |
| `AdminTagModule` | tag CRUD/binding | Prisma, AdminAuth |
| `AdminKeywordModule` | keyword management | Prisma, AdminAuth |
| `AdminUserModule` | admin user management | Prisma, AdminAuth |
| `AdminAuditModule` | audit log query | Prisma, AdminAuth |
| `HealthModule` | liveness/readiness | DB/GitLab |
### 4.3 Unified Launch Flow
1. Frontend calls `POST /tools/:id/launch`.
2. Backend checks `access_mode`:
- `web`: return target URL and record open event
- `download`: create short-lived ticket and return download URL
3. Frontend follows returned action URL.
4. For download mode, `GET /downloads/:ticket` streams file from GitLab.
## 5. API Contract
Base path: `/api/v1`
### 5.1 Unified Response Format
```json
{
"code": 0,
"message": "ok",
"data": {},
"traceId": "7f9b4c8f-3fdf-4f9f-9d2c-8d969ad4c5f1",
"timestamp": "2026-03-26T10:10:00.000Z"
}
```
### 5.2 Public APIs
#### 1) Query tools
- `GET /tools`
- Query:
- `query` (optional)
- `category` (optional, default `all`)
- `sortBy` (`popular|latest|rating|name`)
- `page` (default `1`)
- `pageSize` (default `6`, max `50`)
- Each tool includes:
- `accessMode`: `web | download`
- `openUrl` (nullable; present in `web` mode)
- `hasArtifact` (boolean; meaningful in `download` mode)
#### 2) Tool detail
- `GET /tools/:id`
- Returns mode-specific usage hints:
- `web`: `openUrl`
- `download`: `latestVersion`, `fileSize`, `downloadReady`
#### 3) Category list
- `GET /categories`
#### 4) Hot keywords
- `GET /keywords/hot`
#### 5) Site overview KPI
- `GET /overview`
- Includes:
- `toolTotal`
- `categoryTotal`
- `downloadTotal`
- `openTotal`
### 5.3 Public Launch + Download APIs
#### 1) Unified launch endpoint
- `POST /tools/:id/launch`
- Body (optional):
```json
{
"channel": "official",
"clientVersion": "web-1.0.0"
}
```
- Web mode response example:
```json
{
"mode": "web",
"actionUrl": "https://example-tool.com/app",
"openIn": "new_tab"
}
```
- Download mode response example:
```json
{
"mode": "download",
"ticket": "dl_tk_7f8a2b...",
"expiresInSec": 120,
"actionUrl": "/api/v1/downloads/dl_tk_7f8a2b..."
}
```
#### 2) Consume download ticket
- `GET /downloads/:ticket`
- Behavior:
- validate ticket and expiration
- resolve artifact metadata
- stream file from GitLab
- write `download_records` and increment download counters
### 5.4 Admin Auth APIs
All admin APIs use `/admin` prefix.
- `POST /admin/auth/login`
- `POST /admin/auth/refresh`
- `POST /admin/auth/logout`
- `GET /admin/auth/me`
Login response example:
```json
{
"accessToken": "jwt-access-token",
"refreshToken": "jwt-refresh-token",
"expiresIn": 7200,
"profile": {
"id": "u_admin_001",
"username": "admin",
"displayName": "System Admin"
}
}
```
### 5.5 Admin Tool APIs
- `GET /admin/tools`
- `POST /admin/tools`
- `GET /admin/tools/:id`
- `PATCH /admin/tools/:id`
- `PATCH /admin/tools/:id/status`
- `PATCH /admin/tools/:id/access-mode`
- `DELETE /admin/tools/:id` (soft delete)
`POST/PATCH /admin/tools` request core fields:
- `name`
- `categoryId`
- `description`
- `tags`
- `features`
- `accessMode` (`web|download`)
- `openUrl` (required when `accessMode=web`)
### 5.6 Admin Artifact APIs (Download Mode Only)
#### 1) Upload artifact file
- `POST /admin/tools/:id/artifacts`
- Content-Type: `multipart/form-data`
- Form fields:
- `file` (required)
- `version` (required)
- `releaseNotes` (optional)
- `isLatest` (optional, default `true`)
Validation:
- tool must be `accessMode=download`
- version must be unique within tool
- file type/size must pass policy
#### 2) List tool artifacts
- `GET /admin/tools/:id/artifacts`
#### 3) Set latest artifact
- `PATCH /admin/tools/:id/artifacts/:artifactId/latest`
#### 4) Deprecate artifact
- `PATCH /admin/tools/:id/artifacts/:artifactId/status`
#### 5) Delete artifact metadata
- `DELETE /admin/tools/:id/artifacts/:artifactId`
### 5.7 Admin Taxonomy APIs
- `GET /admin/categories`
- `POST /admin/categories`
- `PATCH /admin/categories/:id`
- `DELETE /admin/categories/:id`
- `PATCH /admin/categories/reorder`
- `GET /admin/tags`
- `POST /admin/tags`
- `PATCH /admin/tags/:id`
- `DELETE /admin/tags/:id`
- `GET /admin/keywords/hot`
- `PUT /admin/keywords/hot`
### 5.8 Admin User and Audit APIs
- `GET /admin/users`
- `POST /admin/users`
- `PATCH /admin/users/:id`
- `PATCH /admin/users/:id/status`
- `GET /admin/audit-logs`
### 5.9 Error Codes
| Code | Meaning |
|---|---|
| `1001` | validation failed |
| `1002` | unauthorized |
| `1003` | forbidden |
| `1004` | resource not found |
| `1005` | conflict (duplicate/version conflict) |
| `1010` | invalid credentials |
| `1011` | token invalid/expired |
| `1201` | GitLab upload failed |
| `1202` | GitLab download failed |
| `1203` | artifact not available |
| `1204` | download ticket invalid/expired |
| `1210` | tool access mode mismatch |
| `1211` | web open URL not configured |
| `1500` | internal server error |
## 6. Data Model Design (SQLite)
SQLite file: `server/prisma/dev.db`
### 6.1 Table: `tools`
| Field | Type | Constraint | Description |
|---|---|---|---|
| `id` | TEXT | PK | business id (`tool_1`) |
| `name` | TEXT | not null | tool name |
| `slug` | TEXT | unique | URL-friendly id |
| `category_id` | TEXT | FK -> categories.id | category |
| `description` | TEXT | not null | summary |
| `rating` | REAL | check 0~5 | score |
| `download_count` | INTEGER | default 0 | successful downloads |
| `open_count` | INTEGER | default 0 | successful web opens |
| `access_mode` | TEXT | not null default `download` | `web|download` |
| `open_url` | TEXT | nullable | target URL for web mode |
| `open_in_new_tab` | INTEGER | default 1 | 1/0 |
| `latest_artifact_id` | TEXT | nullable | FK -> tool_artifacts.id |
| `status` | TEXT | default `draft` | `draft/published/archived` |
| `updated_at` | TEXT | not null | `YYYY-MM-DD` |
| `is_deleted` | INTEGER | default 0 | soft delete flag |
| `created_at` | DATETIME | default current_timestamp | created time |
| `modified_at` | DATETIME | default current_timestamp | updated time |
Recommended checks:
- `access_mode IN ('web','download')`
- `status IN ('draft','published','archived')`
- `access_mode != 'web' OR open_url IS NOT NULL`
Indexes:
- `idx_tools_category_id`
- `idx_tools_status`
- `idx_tools_access_mode`
- `idx_tools_download_count`
- `idx_tools_open_count`
- `idx_tools_updated_at`
- `idx_tools_rating`
- `idx_tools_name`
### 6.2 Table: `tool_artifacts`
| Field | Type | Constraint | Description |
|---|---|---|---|
| `id` | TEXT | PK | artifact id |
| `tool_id` | TEXT | FK -> tools.id | owner tool |
| `version` | TEXT | not null | version |
| `file_name` | TEXT | not null | package filename |
| `file_size_bytes` | INTEGER | not null | size |
| `sha256` | TEXT | not null | checksum |
| `mime_type` | TEXT | nullable | content type |
| `gitlab_project_id` | INTEGER | not null | GitLab project id |
| `gitlab_package_name` | TEXT | not null | package path segment |
| `gitlab_package_version` | TEXT | not null | usually equals `version` |
| `gitlab_file_path` | TEXT | not null | package file path |
| `status` | TEXT | default `active` | `active/deprecated/deleted` |
| `release_notes` | TEXT | nullable | release notes |
| `uploaded_by` | TEXT | FK -> admin_users.id | operator |
| `created_at` | DATETIME | default current_timestamp | upload time |
Unique / Index:
- `uk_tool_version (tool_id, version)`
- `idx_artifact_tool_id`
- `idx_artifact_status`
### 6.3 Table: `categories`
| Field | Type | Constraint |
|---|---|---|
| `id` | TEXT | PK |
| `name` | TEXT | unique, not null |
| `sort_order` | INTEGER | default 100 |
| `is_deleted` | INTEGER | default 0 |
### 6.4 Table: `tags`
| Field | Type | Constraint |
|---|---|---|
| `id` | TEXT | PK |
| `name` | TEXT | unique, not null |
| `is_deleted` | INTEGER | default 0 |
### 6.5 Table: `tool_tags`
| Field | Type | Constraint |
|---|---|---|
| `tool_id` | TEXT | FK -> tools.id |
| `tag_id` | TEXT | FK -> tags.id |
| `(tool_id, tag_id)` | - | composite PK |
### 6.6 Table: `tool_features`
| Field | Type | Constraint |
|---|---|---|
| `id` | TEXT | PK |
| `tool_id` | TEXT | FK -> tools.id |
| `feature_text` | TEXT | not null |
| `sort_order` | INTEGER | default 100 |
### 6.7 Table: `hot_keywords`
| Field | Type | Constraint |
|---|---|---|
| `id` | TEXT | PK |
| `keyword` | TEXT | unique, not null |
| `sort_order` | INTEGER | default 100 |
| `is_active` | INTEGER | default 1 |
### 6.8 Table: `download_tickets`
| Field | Type | Constraint | Description |
|---|---|---|---|
| `id` | INTEGER | PK AUTOINCREMENT | internal id |
| `ticket` | TEXT | unique, not null | public token |
| `tool_id` | TEXT | FK -> tools.id | target tool |
| `artifact_id` | TEXT | FK -> tool_artifacts.id | target artifact |
| `channel` | TEXT | nullable | source |
| `client_version` | TEXT | nullable | app version |
| `request_ip` | TEXT | nullable | requester ip |
| `expires_at` | DATETIME | not null | expiry |
| `consumed_at` | DATETIME | nullable | consume time |
| `created_at` | DATETIME | default current_timestamp | create time |
### 6.9 Table: `download_records`
| Field | Type | Constraint | Description |
|---|---|---|---|
| `id` | INTEGER | PK AUTOINCREMENT | record id |
| `tool_id` | TEXT | FK -> tools.id | downloaded tool |
| `artifact_id` | TEXT | FK -> tool_artifacts.id | downloaded artifact |
| `ticket` | TEXT | nullable | download ticket |
| `downloaded_at` | DATETIME | default current_timestamp | event time |
| `client_ip` | TEXT | nullable | requester ip |
| `user_agent` | TEXT | nullable | requester ua |
| `channel` | TEXT | nullable | source |
| `client_version` | TEXT | nullable | frontend version |
| `status` | TEXT | default `success` | `success/failed` |
| `error_message` | TEXT | nullable | failure reason |
### 6.10 Table: `open_records`
| Field | Type | Constraint | Description |
|---|---|---|---|
| `id` | INTEGER | PK AUTOINCREMENT | record id |
| `tool_id` | TEXT | FK -> tools.id | opened tool |
| `opened_at` | DATETIME | default current_timestamp | event time |
| `client_ip` | TEXT | nullable | requester ip |
| `user_agent` | TEXT | nullable | requester ua |
| `channel` | TEXT | nullable | source |
| `client_version` | TEXT | nullable | frontend version |
| `referer` | TEXT | nullable | referer URL |
### 6.11 Table: `admin_users`
| Field | Type | Constraint |
|---|---|---|
| `id` | TEXT | PK |
| `username` | TEXT | unique, not null |
| `password_hash` | TEXT | not null |
| `display_name` | TEXT | nullable |
| `status` | TEXT | default `active` |
| `last_login_at` | DATETIME | nullable |
| `created_at` | DATETIME | default current_timestamp |
| `modified_at` | DATETIME | default current_timestamp |
### 6.12 Table: `admin_audit_logs`
| Field | Type | Constraint | Description |
|---|---|---|---|
| `id` | INTEGER | PK AUTOINCREMENT | log id |
| `admin_user_id` | TEXT | FK -> admin_users.id | operator |
| `action` | TEXT | not null | e.g. `artifact.upload` |
| `resource_type` | TEXT | not null | `tool/artifact/category` |
| `resource_id` | TEXT | nullable | target id |
| `request_method` | TEXT | not null | `POST/PATCH/DELETE` |
| `request_path` | TEXT | not null | route path |
| `request_body` | TEXT | nullable | masked json |
| `ip` | TEXT | nullable | operator ip |
| `user_agent` | TEXT | nullable | operator ua |
| `created_at` | DATETIME | default current_timestamp | op time |
## 7. Access Mode Business Rules
### 7.1 Rule Matrix
| Scenario | Required Fields | Allowed Operations |
|---|---|---|
| `access_mode=web` | `open_url` | launch as URL open, no artifact upload required |
| `access_mode=download` | at least one `active` artifact | launch as download ticket |
### 7.2 Publish Validation
Tool can be published only when:
- common fields valid (`name/category/description`)
- if `web` mode: `open_url` is valid URL
- if `download` mode: has active `latest_artifact_id`
### 7.3 Mode Switch Rules
- `web -> download`:
- must upload at least one artifact before publish
- `download -> web`:
- `open_url` required
- existing artifacts can be retained for history but not used in launch path
## 8. GitLab Upload/Download Design
### 8.1 Required Environment Variables
- `GITLAB_BASE_URL` (e.g. `https://gitlab.company.local`)
- `GITLAB_API_BASE` (e.g. `https://gitlab.company.local/api/v4`)
- `GITLAB_PROJECT_ID`
- `GITLAB_TOKEN` (PAT/Project Access Token/Deploy Token)
- `GITLAB_PACKAGE_NAME_PREFIX` (default `toolsshow`)
- `DOWNLOAD_TICKET_TTL_SEC` (default `120`)
- `UPLOAD_MAX_SIZE_MB` (default `512`)
### 8.2 Upload Flow (Download Mode Only)
1. Admin calls `POST /admin/tools/:id/artifacts` with file + version.
2. Backend validates tool mode is `download`.
3. Backend validates file policy and version uniqueness.
4. Backend computes SHA-256 checksum.
5. Backend uploads file to GitLab Generic Package Registry:
- `PUT /projects/:id/packages/generic/:packageName/:version/:fileName`
6. Backend stores artifact metadata in SQLite.
7. Optionally sets this artifact as latest.
8. Writes admin audit log.
### 8.3 Download Flow
1. Client calls `POST /tools/:id/launch`.
2. For download mode, backend creates ticket and returns `actionUrl`.
3. Client calls `GET /downloads/:ticket`.
4. Backend validates ticket and streams file from GitLab.
5. Backend writes `download_records` and increments `download_count`.
### 8.4 Web Open Flow
1. Client calls `POST /tools/:id/launch`.
2. For web mode, backend returns `open_url`.
3. Backend writes `open_records` and increments `open_count`.
4. Frontend opens URL in browser.
## 9. Admin Backend Design (No Roles)
### 9.1 Capability List
| Capability | Description |
|---|---|
| Dashboard | view KPI and trends |
| Tool Management | create/edit/publish/archive/delete tools |
| Access Mode Management | configure web/download mode and constraints |
| Artifact Management | upload and maintain versions for download tools |
| Category/Tag/Keyword | maintain taxonomy and hot keywords |
| Admin User Management | create/disable admin accounts |
| Audit Logs | query write-operation logs |
### 9.2 Auth and Authorization
- Only `JwtAuthGuard` for admin-protected APIs.
- No role/permission table and no RBAC.
- All active admins have same capability.
- Disabled admins cannot login or refresh token.
### 9.3 Admin Write Workflow
1. Request passes JWT auth and admin status check.
2. Service validates business rules (including access mode rules).
3. Service writes DB and optionally calls GitLab API.
4. Audit interceptor records operation.
5. Cache keys are invalidated.
## 10. Security, Reliability, and Performance
- Public endpoints:
- anonymous read for query APIs
- rate limit for `launch` and `downloads` endpoints
- Admin endpoints:
- password hashed with `argon2id`
- login failure counter and temporary lock
- Upload security:
- extension/MIME whitelist
- max size limit
- checksum verification
- Web URL security:
- validate URL format and optional domain whitelist
- block private-network targets if needed
- Error handling:
- global exception filter with stable error schema
- SQLite reliability:
- periodic `dev.db` backup
- lock latency monitoring
## 11. Caching Strategy
- Cache targets:
- tool list query (`query+category+sort+page+pageSize`)
- overview KPI
- categories and hot keywords
- TTL:
- tools list: 60s
- overview/categories/keywords: 120s
- Invalidation:
- tool mode/status updates
- artifact upload/status updates
- category/tag/keyword writes
- counter updates (`download_count/open_count`)
## 12. Recommended Project Structure
```text
server/
src/
main.ts
app.module.ts
common/
filters/
interceptors/
guards/
decorators/
constants/
modules/
health/
tools/
categories/
keywords/
overview/
access/
downloads/
artifacts/
gitlab-storage/
admin-auth/
admin-tools/
admin-artifacts/
admin-categories/
admin-tags/
admin-keywords/
admin-users/
admin-audit/
prisma/
prisma.service.ts
prisma/
schema.prisma
migrations/
seed.ts
test/
public-tools.e2e-spec.ts
public-launch.e2e-spec.ts
public-download.e2e-spec.ts
admin-auth.e2e-spec.ts
admin-tools.e2e-spec.ts
admin-artifacts.e2e-spec.ts
```
## 13. Frontend Integration Mapping
Public frontend changes:
- replace local `tools` with `GET /api/v1/tools`
- each tool card reads `accessMode`
- click primary action:
- call `POST /api/v1/tools/:id/launch`
- if response `mode=web`, use `window.open(actionUrl, '_blank')`
- if response `mode=download`, navigate to returned download URL
Admin frontend changes:
- tool form adds `accessMode` selector (`web|download`)
- when mode is `web`: show `openUrl` field, hide artifact upload block
- when mode is `download`: show artifact upload/version block
- mode switch prompts validation hints before publish
## 14. Implementation Plan
1. Initialize NestJS app in `server/` with Prisma(SQLite).
2. Build schema and seed base data.
3. Implement public query APIs (`tools/categories/keywords/overview`).
4. Implement unified launch endpoint with mode branching.
5. Implement GitLab client + download-mode artifact upload APIs.
6. Implement download ticket consumption and stream proxy.
7. Implement admin auth, tool mode management, taxonomy, audit logs.
8. Add Swagger and unit/e2e tests.
## 15. Test Strategy
- Unit tests:
- tool query and sorting logic
- launch service mode branching (`web` vs `download`)
- artifact upload validation + checksum + metadata persistence
- ticket create/consume/expire logic
- E2E tests:
- `GET /tools` returns mode-specific fields
- `POST /tools/:id/launch` for web mode returns URL and writes open record
- `POST /tools/:id/launch` for download mode returns ticket
- `GET /downloads/:ticket` streams file and writes download record
- `POST /admin/tools/:id/artifacts` rejects when tool mode is `web`
## 16. Risks and Open Questions
- Confirm whether all web URLs are external only, or include internal SSO links.
- Confirm whether web-open events need anti-abuse strategy similar to download.
- Confirm max artifact size and whether chunk upload is required.
- Confirm whether artifact deletion should also trigger GitLab deletion immediately.
- Confirm whether mode switch should be restricted once tool is published.
## 17. Delivery Note
This design is updated for:
- SQLite database
- admin backend without role/permission model
- mixed tool access modes (`web` + `download`)
- GitLab-based upload/download for download-mode tools only
After confirmation, next step is `代码实现` (NestJS scaffold + hybrid launch flow + GitLab integration baseline).