alex_p/yt-dlp-dags
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: alex_p:e284099eeef73b2eca5448a7a9d10555dda50eb7
Branches Tags
alex_p:main
..
compare: alex_p:561eac8a7e71a54519b4b18806d326ae37c366fb
Branches Tags
alex_p:main

No commits in common. "e284099eeef73b2eca5448a7a9d10555dda50eb7" and "561eac8a7e71a54519b4b18806d326ae37c366fb" have entirely different histories.
e284099eee ... 561eac8a7e

3 changed files with 0 additions and 77 deletions
Show Stats Expand all files Collapse all files

1
.vault_pass.example
View File

@ -1 +0,0 @@
PASS_TO_UNLOCK_host_vars_encrypted

44
README.md
View File

@ -1,44 +0,0 @@
Deploy with ansible from af-jump
```
ssh user@af-jump
cp cluster.dummy.yml cluster.stress.yml
vi cluster.stress.yml
./tools/generate-inventory.py cluster.stress.yml
ansible-playbook ansible/playbook-XXX -i ansible/inventory.stress.ini
playbook-base-system.yml
playbook-proxies.yml
playbook-stress-sync-code.yml
playbook-stress-install-deps.yml
playbook-stress-generate-env.yml
playbook-docker-services-setup.yml
```
Code updates only of ytops
```
#git pull or ./tools/sync-to-jump.sh
playbook-stress-sync-code.yml
```
Running
```
ssh user@af-green
cd /srv/airflow_master
./bin/build-yt-dlp-image
bin/ytops-client setup-profiles --policy policies/6_profile_setup_policy.yaml --cleanup-all
bin/ytops-client profile list --auth-env sim_auth --download-env sim_download --live --no-blink --show-proxy-activity
bin/ytops-client policy-enforcer --policy policies/8_unified_simulation_enforcer.yaml --live
bin/ytops-client stress-policy --policy policies/10_direct_docker_auth_simulation.yaml --verbose --set execution_control.workers=1 --set settings.urls_file=inputfiles/urls.rt3700.txt
bin/ytops-client stress-policy --policy policies/11_direct_docker_download_simulation.yaml --set execution_control.workers=1 --verbose
```

32
cluster.dummy.yml
View File

@ -1,32 +0,0 @@
global_vars:
# --- Core Ansible & Directory Settings ---
ansible_user: "alex_p"
deploy_group: "ytdl"
airflow_master_dir: "/srv/airflow_master"
airflow_worker_dir: "/srv/airflow_dl_worker"
# --- Docker & Network Settings ---
docker_network_name: "airflow_proxynet"
# --- Shadowsocks Proxy Configuration ---
shadowsocks_image: "ghcr.io/shadowsocks/sslocal-rust:v1.22.0"
shadowsocks_cipher_method: "aes-256-gcm"
shadowsocks_local_address: "0.0.0.0"
shadowsocks_timeout: 20
shadowsocks_fast_open: true
shadowsocks_mode: "tcp_and_udp"
# List of all available proxies to be deployed on all nodes.
# The key (e.g., 'sslocal-rust-1087') becomes the service name.
shadowsocks_proxies:
sslocal-rust-1084:
server: px012
server_port: 8388
local_port: 1092
vault_password_key: vault_ss_password_1
master:
af-green:
ip: x.x.x.x
port: 22