# Ansible Deployment for YT-DLP Cluster

This document provides an overview of the Ansible playbooks used to deploy and manage the YT-DLP Airflow cluster.

## Main Playbooks

These are the primary entry points for cluster management.

- `playbook-full-with-proxies.yml`: **(Recommended Entry Point)** Deploys shadowsocks proxies and then the entire application stack.
- `playbook-full.yml`: Deploys the entire application stack (master and workers) without touching proxies.
- `playbook-master.yml`: Deploys/updates only the Airflow master node.
- `playbook-worker.yml`: Deploys/updates all Airflow worker nodes.
- `playbook-proxies.yml`: Deploys/updates only the shadowsocks proxy services on all nodes.

## Component & Utility Playbooks

These playbooks are used for more specific tasks or are called by the main playbooks.

### Core Deployment Logic
- `roles/airflow-master/tasks/main.yml`: Contains all tasks for setting up the Airflow master services.
- `roles/airflow-worker/tasks/main.yml`: Contains all tasks for setting up the Airflow worker services.
- `roles/ytdlp-master/tasks/main.yml`: Contains tasks for setting up the YT-DLP management services on the master.
- `roles/ytdlp-worker/tasks/main.yml`: Contains tasks for setting up YT-DLP, Camoufox, and other worker-specific services.

### Utility & Maintenance
- `playbook-dags.yml`: Quickly syncs only the `dags/` and `config/` directories to all nodes.
- `playbook-hook.yml`: Syncs Airflow custom hooks and restarts relevant services.
- `playbook-sync-local.yml`: Syncs local development files (e.g., `ytops_client`, `pangramia`) to workers.
- `playbooks/pause_worker.yml`: Pauses a worker by creating a lock file, preventing it from taking new tasks.
- `playbooks/resume_worker.yml`: Resumes a paused worker by removing the lock file.
- `playbooks/playbook-bgutils-start.yml`: Starts the `bgutil-provider` container.
- `playbooks/playbook-bgutils-stop.yml`: Stops the `bgutil-provider` container.
- `playbook-update-s3-vars.yml`: Updates the `s3_delivery_connection` in Airflow.
- `playbook-update-regression-script.yml`: Updates the `regression.py` script on the master.

### Deprecated
- `playbook-dl.yml`: Older worker deployment logic. Superseded by `playbook-worker.yml`.
- `playbook-depricated.dl.yml`: Older worker deployment logic. Superseded by `playbook-worker.yml`.

## Current Goal: Disable Camoufox & Enable Aria2

The current objective is to modify the worker deployment (`playbook-worker.yml` and its role `roles/ytdlp-worker/tasks/main.yml`) to:
1.  **Disable Camoufox**: Prevent the build, configuration generation, and startup of all `camoufox` services.
2.  **Enable Aria2**: Ensure the `aria2-pro` service is built and started correctly on worker nodes.

The `playbook-worker.yml` has already been updated to build the `aria2-pro` image. The next steps will involve modifying `roles/ytdlp-worker/tasks/main.yml` to remove the Camoufox-related tasks.