Project

General

Profile

Bug #8834

new duplicity loses option --ssh-backend causing backups to fail

Added by micah almost 3 years ago. Updated over 1 year ago.

Status:
In Progress
Priority:
Normal
Assignee:
Category:
duplicity handler
Target version:
Start date:
02/02/2015
Due date:
% Done:

0%

QA Check:
Ready for QA

Description

I noticed that duplicity backups started to fail because of:

Warning: Duplicity cleanup failed.
Warning: Duplicity remove-older-than failed.
Error: Usage:
Error: duplicity [full|incremental] [options] source_dir target_url
Error: duplicity [restore] [options] source_url target_dir
Error: duplicity verify [options] source_url target_dir
Error: duplicity collection-status [options] target_url
Error: duplicity list-current-files [options] target_url
Error: duplicity cleanup [options] target_url
Error: duplicity remove-older-than time [options] target_url
Error: duplicity remove-all-but-n-full count [options] target_url
Error: duplicity remove-all-inc-of-but-n-full count [options] target_url
Error: Backends and their URL formats:
Error: cf+http://container_name
Error: file:///some_dir
Error: ftp://user[:password]@other.host[:port]/some_dir
Error: ftps://user[:password]@other.host[:port]/some_dir
Error: hsi://user[:password]@other.host[:port]/some_dir
Error: imap://user[:password]@other.host[:port]/some_dir
Error: rsync://user[:password]@other.host[:port]::/module/some_dir
Error: rsync://user[:password]@other.host[:port]/relative_path
Error: rsync://user[:password]@other.host[:port]//absolute_path
Error: s3://other.host/bucket_name[/prefix]
Error: s3+http://bucket_name[/prefix]
Error: scp://user[:password]@other.host[:port]/some_dir
Error: ssh://user[:password]@other.host[:port]/some_dir
Error: swift://container_name
Error: tahoe://alias/directory
Error: webdav://user[:password]@other.host/some_dir
Error: webdavs://user[:password]@other.host/some_dir
Error: gdocs://user[:password]@other.host/some_dir
Error: mega://user[:password]@other.host/some_dir
Error: copy://user[:password]@other.host/some_dir
Error: dpbx:///some_dir
Error: onedrive://some_dir
Error: azure://container_name
Error: Commands:
Error: cleanup <target_url>
Error: collection-status <target_url>
Error: full <source_dir> <target_url>
Error: incr <source_dir> <target_url>
Error: list-current-files <target_url>
Error: restore <target_url> <source_dir>
Error: remove-older-than <time> <target_url>
Error: remove-all-but-n-full <count> <target_url>
Error: remove-all-inc-of-but-n-full <count> <target_url>
Error: verify <target_url> <source_dir>
Error: duplicity: error: no such option: --ssh-backend
Fatal: Duplicity failed.

This is with duplicity 0.7.01-1

History

#1 Updated by micah almost 3 years ago

It appears duplicity 0.7.01-1 was uploaded to debian unstable on January 25th, which is the very night that my backups started to fail with that error.

The man page still lists --ssh-backend as an option, but duplicity --help does not.

According to the upstream changelog:

* Merged in lp:~ed.so/duplicity/lftp.ncftp.and.prefixes
  - retire --ssh-backend, --use-scp parameters
  - introduce scheme prefixes for alternative backend selection
    e.g. ncftp+ftp://, see manpage
  - scp is now selected via scheme e.g. scp://
  - added lftp fish, webdav(s), sftp support
<pre>

#2 Updated by ulrich over 1 year ago

Hell micah,

based on your report I wrote a patch that adds a check for that version and option. Now it is easier to understand the error.
I will have a look at the --use-scp parameters and try to figure out how do solve this issue.

Kind regards,
ulrich

#3 Updated by ulrich over 1 year ago

Sorry, I forgot to remove the declaration of the version of duplicity.

#4 Updated by intrigeri over 1 year ago

  • Status changed from New to In Progress
  • QA Check set to Ready for QA

#5 Updated by ulrich over 1 year ago

I read through the manpages of pre and post 0.7.01 duplicity and I think we have to introduce a new config option for the new backend management. Before 0.7.01 you could specify a backend (pexpect for example) via --ssh-backend pexpect. Version 0.7.01 and onwards you have to specify it via a scheme prefix pexpect+scp://.
At the moment the duplicity handler sets the serverpart of the execstring directly to scp://$destuser$desthost/$destdir@, if desturl is unset. So if you don't config desturl with a scheme prefix in your config file you won't be able to choose an other backend.

The attached patch introduces a new option called sshbackend. The option is also included in a patched ninjahelper. It works the following way:
The argument of --ssh-backend is used for the scheme prefix and a warning is printed, if a version >= 0.7.01 is used. This way backward compatibility should be maintained. If the options doesn't contain a --ssh-backend argument, the new sshbackend variable from the config is used.
For duplicity versions < 0.7.01 the --ssh-backend argument is used as it is. Furthermore you can use the new sshbackend option. It will be used to include a --ssh-backend $sshbackend to the options.
The explicit (old) --ssh-backend always overrides the new sshbackend option.

Same configuration file can be used for duplicity < or >= 0.7.01.

I tested the new patch with debian stable, latest git master of backupninja and duplicity 0.6.24 (stable) and 0.7.06 (unstable). Combinations of options with and without --ssh-backend and with the new sshbackend option set and unset worked as expected.

I didn't test it with version below 0.6.17 and I didn't changed the code of that part. The new config variable will not work with that versions. I don't know if it had the --ssh-backend. Is support for that versions still needed?

Kind regards,
ulrich

#6 Updated by ulrich over 1 year ago

The patches can also be found at my git repository (https://github.com/UlrichViefhaus/backupninja) in the branch Bug8834: https://github.com/UlrichViefhaus/backupninja/tree/Bug8834

I hope it helps to integrate the patches in the next release. Can someone test it, please?

Also available in: Atom PDF