Proxy Setup
If you’re experiencing connection issues or want to improve download reliability, you can configure the scraper to use proxies.
When to Use Proxies
Section titled “When to Use Proxies”You might want to use proxies if:
- You’re getting rate limited (downloads fail frequently)
- Your IP is blocked by Kemono/Coomer
- You want to distribute requests across multiple IPs
- You need to route traffic through a specific location
Configuring Proxies
Section titled “Configuring Proxies”Proxies are configured in your config.yaml file.
Basic Proxy Setup
Section titled “Basic Proxy Setup”# Proxy configurationproxies: - type: http host: proxy.example.com port: 8080
# Your creatorscreators: - service: patreon userId: "12345678"Proxy with Authentication
Section titled “Proxy with Authentication”If your proxy requires a username and password:
proxies: - type: http host: proxy.example.com port: 8080 username: your-username password: your-passwordMultiple Proxies
Section titled “Multiple Proxies”The scraper will rotate between proxies automatically:
proxies: - type: http host: proxy1.example.com port: 8080
- type: http host: proxy2.example.com port: 8080
- type: socks5 host: socks.example.com port: 1080Proxy Types
Section titled “Proxy Types”| Type | Description | Example |
|---|---|---|
http | Standard HTTP proxy | Most common |
https | HTTPS proxy | For encrypted proxy connections |
socks5 | SOCKS5 proxy | More versatile, supports all traffic |
HTTP Proxy Example
Section titled “HTTP Proxy Example”proxies: - type: http host: 192.168.1.100 port: 3128SOCKS5 Proxy Example
Section titled “SOCKS5 Proxy Example”proxies: - type: socks5 host: socks.example.com port: 1080 username: user password: passProxy Rotation
Section titled “Proxy Rotation”When you have multiple proxies, the scraper automatically:
- Rotates between proxies (round-robin)
- Detects unhealthy proxies
- Cools down failed proxies temporarily
- Falls back to direct connection if all proxies fail
Rotation Mode
Section titled “Rotation Mode”proxyRotation: round_robin # DefaultDebugging Proxy Issues
Section titled “Debugging Proxy Issues”Enable verbose proxy logging to see what’s happening:
DEBUG_PROXY=1 ./kemono-scraper -c config.yamlThis shows:
- Which proxy is being used for each request
- Proxy failures and cooldowns
- Fallback to direct connections
Complete Example
Section titled “Complete Example”Here’s a full config with proxies:
# Kemono Scraper Configuration with Proxies
host: kemono.croutputDir: downloads-%username%maxPosts: 5000
# Proxy configurationproxyRotation: round_robinproxies: # Fast HTTP proxy - type: http host: fast-proxy.example.com port: 8080
# Backup SOCKS5 proxy - type: socks5 host: backup.example.com port: 1080 username: myuser password: mypass
# Creators to downloadcreators: - service: patreon userId: "12345678"
- service: onlyfans userId: "someuser" host: coomer.stCommon Proxy Issues
Section titled “Common Proxy Issues””Proxy connection refused”
Section titled “”Proxy connection refused””- Check that the proxy host and port are correct
- Make sure the proxy is running and accessible
- Verify any firewall rules allow the connection
”Proxy authentication required”
Section titled “”Proxy authentication required””Add username and password to your proxy config:
proxies: - type: http host: proxy.example.com port: 8080 username: your-username password: your-password“All proxies failing”
Section titled ““All proxies failing””- The scraper will fall back to direct connections
- Check your proxy credentials and connectivity
- Try reducing concurrent downloads:
maxConcurrentDownloads: 1
Proxy is slow
Section titled “Proxy is slow”- Try a different proxy provider
- Use a proxy closer to your location
- Consider using fewer concurrent downloads
Where to Get Proxies
Section titled “Where to Get Proxies”There are many proxy providers available. Some options:
- Residential proxies - Look like regular home connections
- Datacenter proxies - Faster but may be blocked more often
- Free proxies - Not recommended (unreliable and insecure)
Running Without Proxies
Section titled “Running Without Proxies”To disable proxies, either:
- Remove the
proxiessection from your config - Leave it empty:
proxies: []The scraper works fine without proxies for most users.