93 lines
3.1 KiB
Markdown
93 lines
3.1 KiB
Markdown
# Job Management
|
||
|
||
## Overview
|
||
|
||
Downloads run as background jobs. Each job is tracked in-memory during execution and persisted to the database on completion. The frontend polls for status updates every ~1–3 seconds.
|
||
|
||
---
|
||
|
||
## Job Lifecycle
|
||
|
||
```
|
||
User Request → Create Job (queued) → Spawn Thread → status: running
|
||
→ [completed | failed | cancelled] → Upsert to DB
|
||
```
|
||
|
||
1. A job record is created in the in-memory `jobs` dict with `status: queued`.
|
||
2. A Python thread is spawned to run the download function.
|
||
3. The thread updates `status`, `output`, and `return_code` in-memory as it runs.
|
||
4. On finish (success, failure, or cancellation), the job is upserted into SQLite.
|
||
|
||
---
|
||
|
||
## In-Memory Job Structure
|
||
|
||
```python
|
||
{
|
||
"id": str, # UUID
|
||
"user_id": str, # Owner
|
||
"urls": list[str], # Spotify URLs
|
||
"options": dict, # Download parameters
|
||
"status": str, # queued | running | completed | failed | cancelled
|
||
"output": list[str], # Log lines (capped at 500)
|
||
"command": str, # CLI command string (Votify jobs only)
|
||
"return_code": int, # Process exit code
|
||
"process": Popen, # Subprocess handle (for cancellation)
|
||
"created_at": float, # Unix timestamp
|
||
}
|
||
```
|
||
|
||
`process` is only present while the job is running; it is not persisted to the database.
|
||
|
||
---
|
||
|
||
## Output Streaming
|
||
|
||
- The subprocess stdout is read line-by-line in the runner thread.
|
||
- Each line is appended to `job["output"]`.
|
||
- The list is capped at 500 entries (oldest lines are dropped first).
|
||
- The frontend reads output via `GET /api/jobs/<id>` and displays the log incrementally.
|
||
|
||
---
|
||
|
||
## Cancellation
|
||
|
||
1. Frontend calls `POST /api/jobs/<id>/cancel`.
|
||
2. Job `status` is set to `cancelled` in-memory.
|
||
3. `process.terminate()` is called on the Popen handle.
|
||
4. The runner thread detects the cancellation flag and logs a cancellation message.
|
||
5. The job is then upserted to the database with `status: cancelled`.
|
||
|
||
---
|
||
|
||
## Job Expiry
|
||
|
||
A background daemon thread runs hourly and calls `delete_jobs_older_than(days)` where `days` comes from the `job_expiry_days` setting (default: 30). This removes old job records from the database but does **not** delete downloaded files.
|
||
|
||
---
|
||
|
||
## API Endpoints
|
||
|
||
| Endpoint | Method | Purpose |
|
||
|----------|--------|---------|
|
||
| `/api/jobs` | GET | List all jobs for the current user |
|
||
| `/api/jobs/<id>` | GET | Get a single job (status + output) |
|
||
| `/api/jobs/<id>/cancel` | POST | Cancel a running job |
|
||
| `/api/jobs/<id>` | DELETE | Delete a completed/failed/cancelled job record |
|
||
| `/api/admin/users/<id>/jobs` | GET | Admin view of any user's jobs |
|
||
|
||
---
|
||
|
||
## Database Persistence
|
||
|
||
Jobs are only written to SQLite when they reach a terminal state (`completed`, `failed`, `cancelled`). In-memory jobs from before a restart are lost, but completed jobs survive restarts via the database. The `GET /api/jobs` endpoint merges both sources: in-memory (for live jobs) and database (for historical).
|
||
|
||
---
|
||
|
||
## Key Files
|
||
|
||
| File | Relevance |
|
||
|------|-----------|
|
||
| [app.py](../app.py) | Job dict, route handlers, runner threads, expiry daemon |
|
||
| [db.py](../db.py) | `upsert_job`, `get_job`, `list_jobs_for_user`, `delete_jobs_older_than` |
|