Project

General

Profile

Feature #7893

Bug #7755: Migrate WhisperBack to Python 3

Test updated WhisperBack

Added by alant almost 4 years ago. Updated over 3 years ago.

Status:
Resolved
Priority:
Elevated
Assignee:
-
Category:
-
Target version:
Start date:
09/13/2014
Due date:
% Done:

100%

QA Check:
Feature Branch:
Type of work:
Test
Blueprint:
Starter:
Affected tool:
WhisperBack

History

#1 Updated by BitingBird almost 4 years ago

  • Category set to 174
  • Status changed from New to Confirmed

#2 Updated by intrigeri almost 4 years ago

  • Assignee set to alant

#3 Updated by intrigeri almost 4 years ago

  • Target version set to Tails_2.0

(as parent ticket..)

#4 Updated by intrigeri over 3 years ago

  • Priority changed from Normal to Elevated

#5 Updated by alant over 3 years ago

  • % Done changed from 0 to 20

Tested whisperback from commit f4ef4ac by installing the .deb in tails/jessie ISO from commit 3a15e4e.

It claims to work fine once correcting /etc/whisperback/config.py but I never recieve the emails.

@@ -139,7 +139,7 @@
       amnesia_version_process = subprocess.Popen ("tails-version",
                                                  stdout=subprocess.PIPE)
       amnesia_version_process.wait()
-      amnesia_version = amnesia_version_process.stdout.read()
+      amnesia_version = amnesia_version_process.stdout.read().decode("utf-8")
     except OSError:
       amnesia_version = "tails-version command not found" 
     except subprocess.CalledProcessError:
@@ -164,7 +164,7 @@
         process = subprocess.Popen (["sudo", "/usr/local/sbin/tails-debugging-i
nfo"],
                                     stdout=subprocess.PIPE)
         for line in process.stdout:
-            debugging_info += line
+            debugging_info += line.decode("utf-8")
         process.wait()
     except OSError:
         debugging_info += "sudo command not found\n" 

#6 Updated by alant over 3 years ago

I never recieve the emails

According to its logs, the email is recieved by the hidden service and relayed to boum.org.

#7 Updated by alant over 3 years ago

Emails without "Debugging info" pass well, while emails including them are lost between the HS relay and tails-bugs.

Ideas:

  • maybe the messages are formatted in a way that makes Schleuder crazy?
  • have WB save an email it has generated on Wheezy and Jessie, and compare
  • the admins of the schleuder list may have received Error messages -> ask them

#8 Updated by alant over 3 years ago

  • % Done changed from 20 to 40

maybe the messages are formatted in a way that makes Schleuder crazy?
have WB save an email it has generated on Wheezy and Jessie, and compare

I did set up an hidden service SMTP server and configured a python3 whisperback running on Tails/Jessie as well as the released whisperback on Tails 1.2.2 to use it. The emails I recieved from the two whisperback were formatted the same way (inline OpenPGP, same header) and I achieved to decrypt both without errors. The content looks very similar, unless the size of the email which is a lot bigger for Jessie, due to a longer syslog.

1.2.2:

Content-Length: 40934
Lines: 633

tails/jessie:

Content-Length: 140169
Lines: 2160

the admins of the schleuder list may have received Error messages -> ask them

They have recieved such error messages:

2015-01-02 00:00:02 tails-bugs@boum.org FATAL   Caught RegexpError: Stack overflow in regexp matcher: /\A(?:
                    [\x00-\x7f]                                     |
                    [\xc2-\xdf] [\x80-\xbf]                         |
                    \xe0        [\xa0-\xbf] [\x80-\xbf]             |
                    [\xe1-\xef] [\x80-\xbf] [\x80-\xbf]             |
                    \xf0        [\x90-\xbf] [\x80-\xbf] [\x80-\xbf] |
                    [\xf1-\xf3] [\x80-\xbf] [\x80-\xbf] [\x80-\xbf] |
                    \xf4        [\x80-\x8f] [\x80-\xbf] [\x80-\xbf]
                   )*\z/xn
        /usr/lib/ruby/1.8/kconv.rb:260:in `match'
        /usr/lib/ruby/1.8/kconv.rb:260:in `isutf8'
        /usr/lib/ruby/1.8/kconv.rb:366:in `isutf8'
        /usr/lib/ruby/vendor_ruby/tmail/quoting.rb:112:in `convert_to'
        /usr/lib/ruby/vendor_ruby/tmail/quoting.rb:35:in `unquoted_body'
        /usr/lib/ruby/vendor_ruby/tmail/quoting.rb:61:in `body'
        /usr/lib/ruby/vendor_ruby/1.8/schleuder/mail.rb:545:in `add_metadata!'

I thus suspect schleuder to become crazy when it gets long inline OpenPGP emails, which is not a regression in WhisperBack.

I wanted to try to sent Jessie logs from Tails 1.2.2 but the gpg process forked by whisperback hangs in proc.handles['stdin'].write(data) with these long data and I'm not into debugging that now (this issue doesn't happen with the updated whisperback on Jessie).

Please confirm or infirm my analysis before I close this ticket and create a new one about schleuder being crazy.

#9 Updated by intrigeri over 3 years ago

Please confirm or infirm my analysis before I close this ticket and create a new one about schleuder being crazy.

Your analysis seems to make sense, but it also seems unrealistic to expect this to be fixed on the Schleuder side.

#10 Updated by intrigeri over 3 years ago

I thus suspect schleuder to become crazy when it gets long inline OpenPGP emails, [...]

inline?

#11 Updated by BitingBird over 3 years ago

  • Affected tool set to WhisperBack

#12 Updated by alant over 3 years ago

  • Assignee changed from alant to intrigeri

intrigeri wrote:

I thus suspect schleuder to become crazy when it gets long inline OpenPGP emails, [...]

inline?

WhisperBack send inline OpenPGP, not PGP/MIME. I can try to implement PGP/MIME and hope Schleuder prefers, or we can filter logs to remove duplicated/not relevant lines to shorten them. Thought?

#13 Updated by intrigeri over 3 years ago

Given:

  • inline OpenPGP is a deprecated thing that has never been properly standardized;
  • it's a well-known fact that Schleuder is über-buggy with inline OpenPGP attachments (thanks to your experiments, we know that it apparently depends on their size -- I didn't know that, and am happy to understand this problem slightly better :)
  • I've no idea what's the maximum inline attachment size that doesn't make Schleuder freak out;
  • I've no idea how to trim the attached logs and make sure they'll be short enough not to trigger Schleuder bugs, without losing valuable info;
  • I believe that Schleuder works mostly fine with well-formatted PGP/MIME;

=> I say let's not add kludges over kludges to make PGP/inline kind of work a bit longer => I think the way to go is PGP/MIME.

#14 Updated by alant over 3 years ago

  • Status changed from Confirmed to Resolved
  • Assignee deleted (intrigeri)
  • % Done changed from 40 to 100

I think the way to go is PGP/MIME.

Made #6183 a subtask of #7756 and assigned it to me.

Also available in: Atom PDF