From 8390c8172ac14b1322069a168b713f313f08dc1a Mon Sep 17 00:00:00 2001 From: aperez Date: Thu, 3 Apr 2025 13:33:36 +0300 Subject: [PATCH] Update readme with more explicit instructions on pull and build --- README-ytdlp-ops-auth.md | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/README-ytdlp-ops-auth.md b/README-ytdlp-ops-auth.md index a620d5e..06ffb16 100644 --- a/README-ytdlp-ops-auth.md +++ b/README-ytdlp-ops-auth.md @@ -1,19 +1,21 @@ # YTDLP Client Side Integration -This document describes how to integrate and use the YTDLP client with the token service. The client DAG is `ytdlp_client_dag`. +This document describes how to integrate and use the YTDLP client with the token service. -## Building & Testing on the Remote Worker Host +## Build -1. **(OPTIONAL if done) Start server if needed:** +1. **Pull, configure and start server if needed:** ```bash cd /srv/airflow_worker/ - docker login pangramia #if not performed auth before + docker login pangramia # It used to be performed beforehand otherwise ask pull password docker compose -f docker-compose-ytdlp-ops.yaml up -d docker compose -f docker-compose-ytdlp-ops.yaml logs -f ``` The server is bound to a certain proxy, like "socks5://sslocal-rust-1084:1084". -2. **Rebuild worker with custom base:** + Also check that redis in bind to 0.0.0.0 in config + +2. **Build airflow-worker with custom dependencies:** ```bash cd /srv/airflow_worker/ docker compose build airflow-worker @@ -21,7 +23,7 @@ This document describes how to integrate and use the YTDLP client with the token docker compose up -d --no-deps airflow-worker ``` -3. **Test the Client Inside a Worker Container:** +3. **Test the built-in client:** ```bash # Show client help docker compose exec airflow-worker python /app/ytdlp_ops_client.py --help @@ -39,9 +41,9 @@ This document describes how to integrate and use the YTDLP client with the token docker compose exec airflow-worker jq -r '"Title: \(.title)", "Date: \(.upload_date | strptime("%Y%m%d") | strftime("%Y-%m-%d"))", "Author: \(.uploader)", "Length: \(.duration_string)", "", "Download URLs:", (.formats[] | select(.vcodec != "none" or .acodec != "none") | .url)' latest.json ``` -4. **Instructions to run DAGs:** +4. **Test Airflow task:** - To run the `ytdlp_client_dag` DAG: + To run the `ytdlp_client_dag_v2.1` DAG: Set up required Airflow variables ```bash @@ -58,7 +60,7 @@ This document describes how to integrate and use the YTDLP client with the token ``` - **Using direct connection:** + **Using direct connection with task test:** ```bash docker compose exec airflow-worker airflow db reset docker compose exec airflow-worker airflow dags reserialize @@ -82,13 +84,13 @@ This document describes how to integrate and use the YTDLP client with the token docker compose exec airflow-worker airflow dags list docker compose exec airflow-worker airflow dags unpause ytdlp_client_dag_v2.1 - // Check if works from server deploy - docker compose exec airflow-worker airflow dags trigger ytdlp_client_dag_v2.1 -c '{"url": "https://www.youtube.com/watch?v=sOlTX9uxUtM", "account_id": "test_direct", "redis_enabled": false, "service_ip": "85.192.30.55", "service_port": 9090}' + // Try UI or recheck if works from server deploy + docker compose exec airflow-worker airflow dags trigger ytdlp_client_dag_v2.1 -c '{"url": "https://www.youtube.com/watch?v=sOlTX9uxUtM", "redis_enabled": false, "service_ip": "85.192.30.55", "service_port": 9090}' ``` - Check Redis for stored data + Check Redis for stored data by videoID ```bash docker compose exec redis redis-cli -a XXXXXX -h 89.253.221.173 -p 52909 HGETALL "token_info:sOlTX9uxUtM" | jq -R -s 'split("\n") | del(.[] | select(. == "")) | [.[range(0;length;2)]]' ```