Project

General

Profile

Feature #6623

Feature #5379: Remove Polipo or replace it with Privoxy

Have wget directly use the Tor SOCKS proxy

Added by sajolida almost 4 years ago. Updated almost 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
-
Target version:
Start date:
01/27/2014
Due date:
% Done:

80%

QA Check:
Pass
Feature Branch:
feature/6623-wget-socks
Type of work:
Code
Blueprint:
Starter:
Yes
Affected tool:

Description

We should add a wget wrapper, that runs it with torsocks instead of using the HTTP proxy. See comment 2 for implementation ideas.


Related issues

Related to Tails - Bug #8603: Unexpected redirection while sending Host header with wget Resolved 01/07/2015

Associated revisions

Revision d42ade06 (diff)
Added by Tails about 3 years ago

Wrap wget with torsocks, instead of configuring it to use Polipo (Closes: #6623).

This is yet another step on the road towards removing Polipo.

Revision 52cd1acb (diff)
Added by Tails about 3 years ago

Wrap wget with torsocks, instead of configuring it to use Polipo (Closes: #6623).

This is yet another step on the road towards removing Polipo.

History

#1 Updated by sajolida almost 4 years ago

Issue has been updated by winterfairy.

At least in the past I had some problem with getting wget to use the socks proxy. Whatever I tried I only got it to work through polipo, which has a download limit (either bug or feature) of 50 MB.

Instead I use curl with resume support as follows (where $URL is the URL to download):

curl -C - -O --socks5-hostname 127.0.0.1:9050 $URL

If wget is still broken in this regard (I haven't tried), maybe creating a "torget" that does the curl command above is a better solution?

#2 Updated by sajolida almost 4 years ago

Issue #6600 has been updated by intrigeri.

At least in the past I had some problem with getting wget to use the socks proxy.
Whatever I tried I only got it to work through polipo, which has a download limit
(either bug or feature) of 50 MB.

This makes wget connect to the Tor SOCKS proxy instead of Polipo:

  1. Comment out the http_proxy and https_proxy options in /etc/wgetrc (note that we add these options ourselves at build time)
  2. Run http_proxy= HTTP_PROXY= https_proxy= HTTPS_PROXY= torsocks wget URL

So I guess we could very well drop our wgetrc patching, and add a wget
wrapper that does the above.

This probably breaks downloading stuff from the LAN, but I'm not sure
if our current configuration allows it anyway.

#3 Updated by intrigeri almost 4 years ago

Tails 0.22.1: wget can fetch files from a FTP server on the LAN, but not from a HTTP server on the LAN. In other words, the wget wrapper idea is doable.

#4 Updated by intrigeri almost 4 years ago

  • Tracker changed from Bug to Feature
  • Subject changed from Document the possibility to use wget in Tails to Have wget use the SOCKS proxy
  • Status changed from New to Confirmed
  • Type of work changed from Documentation to Code

#5 Updated by intrigeri almost 4 years ago

  • Priority changed from Low to Normal

#6 Updated by intrigeri over 3 years ago

  • Status changed from Confirmed to In Progress
  • % Done changed from 0 to 10

#7 Updated by intrigeri over 3 years ago

  • Description updated (diff)
  • Starter changed from No to Yes

#8 Updated by intrigeri over 3 years ago

  • Target version set to Sustainability_M1
  • Parent task set to #5379

Flagging for 2.0, as it's a subtask of #5379 that is itself a 2.0 goal.

#9 Updated by BitingBird over 3 years ago

  • Target version changed from Sustainability_M1 to Hardening_M1

Right, except it's 3.0 :)

#10 Updated by intrigeri about 3 years ago

  • Subject changed from Have wget use the SOCKS proxy to Have wget directly use the Tor SOCKS proxy

#11 Updated by intrigeri about 3 years ago

  • Assignee set to intrigeri
  • Feature Branch set to feature/6623-wget-socks

#12 Updated by intrigeri about 3 years ago

  • Assignee deleted (intrigeri)
  • Target version changed from Hardening_M1 to Tails_1.3
  • % Done changed from 10 to 50
  • QA Check set to Ready for QA

#13 Updated by sajolida about 3 years ago

In Tails 1.2.1, to have wget work fine with torsocks you can do:

torsocks wget --no-proxy URL

#14 Updated by alant almost 3 years ago

  • Assignee set to intrigeri
  • QA Check changed from Ready for QA to Dev Needed

/usr/local/bin/wget includes

unset http_proxy
unset HTTP_PROXY
unset https_proxy
unset HTTPS_PROX

while these variables are not set anymore (see 32897af7). Why? Please provide a reason or remove these lines.

#15 Updated by intrigeri almost 3 years ago

  • Assignee changed from intrigeri to alant
  • % Done changed from 50 to 80
  • QA Check changed from Dev Needed to Ready for QA

alant wrote:

/usr/local/bin/wget includes

[...]

while these variables are not set anymore (see 32897af7). Why? Please provide a reason or remove these lines.

The following command-line depends on the fact that these environment variables are not set. Keeping these unset makes this clear, and is a potentially useful safety net IMO, e.g. in case:

  • one manually sets one of these envvars for some reason;
  • our build system or test suite or something sets one of these envvars, and uses this wget wrapper.

#16 Updated by alant almost 3 years ago

The following command-line depends on the fact that these environment variables are not set. Keeping these unset makes this clear, and is a potentially useful safety net IMO, e.g. in case:

  • one manually sets one of these envvars for some reason;
  • our build system or test suite or something sets one of these envvars, and uses this wget wrapper.

OK.

#17 Updated by alant almost 3 years ago

  • Status changed from In Progress to Fix committed
  • Assignee deleted (alant)
  • QA Check changed from Ready for QA to Pass

#18 Updated by sajolida almost 3 years ago

  • Related to Bug #8603: Unexpected redirection while sending Host header with wget added

#19 Updated by BitingBird almost 3 years ago

  • Status changed from Fix committed to Resolved

Also available in: Atom PDF