# YTDLP Client Side Integration

This document describes how to integrate and use the YTDLP client with the token service.  The client DAG is `ytdlp_client_dag`.

## Building & Testing on the Remote Worker Host

1. **(OPTIONAL if done) Start server if needed:**
   ```bash
   cd /srv/airflow_worker/  
   docker login pangramia #if not performed auth before
   docker compose -f docker-compose-ytdlp-ops.yaml up -d
   docker compose -f docker-compose-ytdlp-ops.yaml logs -f
   ```
   The server is bound to a certain proxy, like "socks5://sslocal-rust-1084:1084".

2. **Rebuild worker with custom base:**
   ```bash
   cd /srv/airflow_worker/  
   docker compose build airflow-worker
   docker compose down airflow-worker
   docker compose up -d --no-deps airflow-worker
   ```

3. **Test the Client Inside a Worker Container:**
   ```bash
   # Show client help
   docker compose exec airflow-worker python /app/ytdlp_ops_client.py --help 

   # Get token and info.json 
   docker compose exec airflow-worker python /app/ytdlp_ops_client.py --host 85.192.30.55 --port 9090 getToken --url 'https://www.youtube.com/watch?v=vKTVLpmvznI'

   # List formats using saved info.json 
   docker compose exec airflow-worker yt-dlp --load-info-json "latest.json"  -F

   # Simulate download using saved info.json 
   docker compose exec airflow-worker yt-dlp --load-info-json "latest.json" --proxy "socks5://sslocal-rust-1084:1084" --simulate --verbose 

   # Extract metadata and download URLs using jq 
   docker compose exec airflow-worker jq -r '"Title: \(.title)", "Date: \(.upload_date | strptime("%Y%m%d") | strftime("%Y-%m-%d"))", "Author: \(.uploader)", "Length: \(.duration_string)", "", "Download URLs:", (.formats[] | select(.vcodec != "none" or .acodec != "none") | .url)' latest.json            
   ```

4. **Instructions to run DAGs:**

   To run the `ytdlp_client_dag` DAG:

   Set up required Airflow variables 
    ```bash                                                                                                                                                        
    docker compose exec airflow-worker airflow variables set DOWNLOAD_OPTIONS '{"formats": ["bestvideo[height<=1080]+bestaudio/best[height<=1080]"]}'                                                  
    docker compose exec airflow-worker airflow variables set DOWNLOADS_TEMP '/opt/airflow/downloadfiles'
    docker compose exec airflow-worker airflow variables set DOWNLOADS_PATH '/opt/airflow/downloadfiles'     
     
    docker compose exec airflow-worker airflow variables  list
    docker compose exec airflow-worker airflow variables set TOKEN_TIMEOUT '300'  

    docker compose exec airflow-worker airflow  connections import /opt/airflow/config/docker_hub_repo.json
    docker compose exec airflow-worker airflow  connections delete redis_default
    docker compose exec airflow-worker airflow  connections import /opt/airflow/config/redis_default_conn.json
   ```
 

   **Using direct connection:**
   ```bash
   docker compose exec airflow-worker airflow db reset
   docker compose exec airflow-worker airflow dags reserialize

   docker compose exec airflow-worker airflow dags list
   docker compose exec airflow-worker airflow  dags list-import-errors
   docker compose exec airflow-worker airflow tasks test ytdlp_client_dag_v2.1 get_token $(date -u +"%Y-%m-%dT%H:%M:%S+00:00") --task-params '{"url": "https://www.youtube.com/watch?v=sOlTX9uxUtM", "redis_enabled": false, "service_ip": "85.192.30.55", "service_port": 9090}'
   docker compose exec airflow-worker  yt-dlp --load-info-json /opt/airflow/downloadfiles/latest.json --proxy "socks5://sslocal-rust-1084:1084" --verbose --simulate

   docker compose exec airflow-worker  airflow dags list-runs -d ytdlp_client_dag 
                                                                             



 
   ```


   or deploy using trigger
    ```bash
   docker compose exec airflow-worker airflow dags list
   docker compose exec airflow-worker airflow dags unpause ytdlp_client_dag_v2.1

   // Check if works from server deploy
   docker compose exec airflow-worker airflow dags trigger ytdlp_client_dag_v2.1 -c '{"url": "https://www.youtube.com/watch?v=sOlTX9uxUtM", "account_id": "test_direct", "redis_enabled": false, "service_ip": "85.192.30.55", "service_port": 9090}'                                              

    ```

                                                                                                                                                                                                   
   Check Redis for stored data 
   ```bash  
    docker compose exec  redis redis-cli -a XXXXXX -h 89.253.221.173 -p 52909 HGETALL "token_info:sOlTX9uxUtM" | jq -R -s 'split("\n") | del(.[] | select(. == "")) | [.[range(0;length;2)]]'
   ```