chore: added documentation
This commit is contained in:
92
docs/job-management.md
Normal file
92
docs/job-management.md
Normal file
@@ -0,0 +1,92 @@
|
||||
# Job Management
|
||||
|
||||
## Overview
|
||||
|
||||
Downloads run as background jobs. Each job is tracked in-memory during execution and persisted to the database on completion. The frontend polls for status updates every ~1–3 seconds.
|
||||
|
||||
---
|
||||
|
||||
## Job Lifecycle
|
||||
|
||||
```
|
||||
User Request → Create Job (queued) → Spawn Thread → status: running
|
||||
→ [completed | failed | cancelled] → Upsert to DB
|
||||
```
|
||||
|
||||
1. A job record is created in the in-memory `jobs` dict with `status: queued`.
|
||||
2. A Python thread is spawned to run the download function.
|
||||
3. The thread updates `status`, `output`, and `return_code` in-memory as it runs.
|
||||
4. On finish (success, failure, or cancellation), the job is upserted into SQLite.
|
||||
|
||||
---
|
||||
|
||||
## In-Memory Job Structure
|
||||
|
||||
```python
|
||||
{
|
||||
"id": str, # UUID
|
||||
"user_id": str, # Owner
|
||||
"urls": list[str], # Spotify URLs
|
||||
"options": dict, # Download parameters
|
||||
"status": str, # queued | running | completed | failed | cancelled
|
||||
"output": list[str], # Log lines (capped at 500)
|
||||
"command": str, # CLI command string (Votify jobs only)
|
||||
"return_code": int, # Process exit code
|
||||
"process": Popen, # Subprocess handle (for cancellation)
|
||||
"created_at": float, # Unix timestamp
|
||||
}
|
||||
```
|
||||
|
||||
`process` is only present while the job is running; it is not persisted to the database.
|
||||
|
||||
---
|
||||
|
||||
## Output Streaming
|
||||
|
||||
- The subprocess stdout is read line-by-line in the runner thread.
|
||||
- Each line is appended to `job["output"]`.
|
||||
- The list is capped at 500 entries (oldest lines are dropped first).
|
||||
- The frontend reads output via `GET /api/jobs/<id>` and displays the log incrementally.
|
||||
|
||||
---
|
||||
|
||||
## Cancellation
|
||||
|
||||
1. Frontend calls `POST /api/jobs/<id>/cancel`.
|
||||
2. Job `status` is set to `cancelled` in-memory.
|
||||
3. `process.terminate()` is called on the Popen handle.
|
||||
4. The runner thread detects the cancellation flag and logs a cancellation message.
|
||||
5. The job is then upserted to the database with `status: cancelled`.
|
||||
|
||||
---
|
||||
|
||||
## Job Expiry
|
||||
|
||||
A background daemon thread runs hourly and calls `delete_jobs_older_than(days)` where `days` comes from the `job_expiry_days` setting (default: 30). This removes old job records from the database but does **not** delete downloaded files.
|
||||
|
||||
---
|
||||
|
||||
## API Endpoints
|
||||
|
||||
| Endpoint | Method | Purpose |
|
||||
|----------|--------|---------|
|
||||
| `/api/jobs` | GET | List all jobs for the current user |
|
||||
| `/api/jobs/<id>` | GET | Get a single job (status + output) |
|
||||
| `/api/jobs/<id>/cancel` | POST | Cancel a running job |
|
||||
| `/api/jobs/<id>` | DELETE | Delete a completed/failed/cancelled job record |
|
||||
| `/api/admin/users/<id>/jobs` | GET | Admin view of any user's jobs |
|
||||
|
||||
---
|
||||
|
||||
## Database Persistence
|
||||
|
||||
Jobs are only written to SQLite when they reach a terminal state (`completed`, `failed`, `cancelled`). In-memory jobs from before a restart are lost, but completed jobs survive restarts via the database. The `GET /api/jobs` endpoint merges both sources: in-memory (for live jobs) and database (for historical).
|
||||
|
||||
---
|
||||
|
||||
## Key Files
|
||||
|
||||
| File | Relevance |
|
||||
|------|-----------|
|
||||
| [app.py](../app.py) | Job dict, route handlers, runner threads, expiry daemon |
|
||||
| [db.py](../db.py) | `upsert_job`, `get_job`, `list_jobs_for_user`, `delete_jobs_older_than` |
|
||||
Reference in New Issue
Block a user