chore: added documentation

This commit is contained in:
2026-03-09 22:45:24 +01:00
parent dd0d0cfde6
commit ec8d5a6124
11 changed files with 963 additions and 0 deletions

92
docs/job-management.md Normal file
View File

@@ -0,0 +1,92 @@
# Job Management
## Overview
Downloads run as background jobs. Each job is tracked in-memory during execution and persisted to the database on completion. The frontend polls for status updates every ~13 seconds.
---
## Job Lifecycle
```
User Request → Create Job (queued) → Spawn Thread → status: running
→ [completed | failed | cancelled] → Upsert to DB
```
1. A job record is created in the in-memory `jobs` dict with `status: queued`.
2. A Python thread is spawned to run the download function.
3. The thread updates `status`, `output`, and `return_code` in-memory as it runs.
4. On finish (success, failure, or cancellation), the job is upserted into SQLite.
---
## In-Memory Job Structure
```python
{
"id": str, # UUID
"user_id": str, # Owner
"urls": list[str], # Spotify URLs
"options": dict, # Download parameters
"status": str, # queued | running | completed | failed | cancelled
"output": list[str], # Log lines (capped at 500)
"command": str, # CLI command string (Votify jobs only)
"return_code": int, # Process exit code
"process": Popen, # Subprocess handle (for cancellation)
"created_at": float, # Unix timestamp
}
```
`process` is only present while the job is running; it is not persisted to the database.
---
## Output Streaming
- The subprocess stdout is read line-by-line in the runner thread.
- Each line is appended to `job["output"]`.
- The list is capped at 500 entries (oldest lines are dropped first).
- The frontend reads output via `GET /api/jobs/<id>` and displays the log incrementally.
---
## Cancellation
1. Frontend calls `POST /api/jobs/<id>/cancel`.
2. Job `status` is set to `cancelled` in-memory.
3. `process.terminate()` is called on the Popen handle.
4. The runner thread detects the cancellation flag and logs a cancellation message.
5. The job is then upserted to the database with `status: cancelled`.
---
## Job Expiry
A background daemon thread runs hourly and calls `delete_jobs_older_than(days)` where `days` comes from the `job_expiry_days` setting (default: 30). This removes old job records from the database but does **not** delete downloaded files.
---
## API Endpoints
| Endpoint | Method | Purpose |
|----------|--------|---------|
| `/api/jobs` | GET | List all jobs for the current user |
| `/api/jobs/<id>` | GET | Get a single job (status + output) |
| `/api/jobs/<id>/cancel` | POST | Cancel a running job |
| `/api/jobs/<id>` | DELETE | Delete a completed/failed/cancelled job record |
| `/api/admin/users/<id>/jobs` | GET | Admin view of any user's jobs |
---
## Database Persistence
Jobs are only written to SQLite when they reach a terminal state (`completed`, `failed`, `cancelled`). In-memory jobs from before a restart are lost, but completed jobs survive restarts via the database. The `GET /api/jobs` endpoint merges both sources: in-memory (for live jobs) and database (for historical).
---
## Key Files
| File | Relevance |
|------|-----------|
| [app.py](../app.py) | Job dict, route handlers, runner threads, expiry daemon |
| [db.py](../db.py) | `upsert_job`, `get_job`, `list_jobs_for_user`, `delete_jobs_older_than` |