CedarBackup3.tools package

Official Cedar Backup Tools

This package provides official Cedar Backup tools. Tools are things that feel a little like extensions, but don’t fit the normal mold of extensions. For instance, they might not be intended to run from cron, or might need to interact dynamically with the user (i.e. accept user input).

Tools are usually scripts that are run directly from the command line, just like the main cback3 script. Like the cback3 script, the majority of a tool is implemented in a .py module, and then the script just invokes the module’s cli() function. The actual scripts for tools are distributed in the util/ directory.

author:Kenneth J. Pronovici <pronovic@ieee.org>

Submodules

CedarBackup3.tools.amazons3 module

Synchonizes a local directory with an Amazon S3 bucket.

No configuration is required; all necessary information is taken from the command-line. The only thing configuration would help with is the path resolver interface, and it doesn’t seem worth it to require configuration just to get that.

author:Kenneth J. Pronovici <pronovic@ieee.org>
class CedarBackup3.tools.amazons3.Options(argumentList=None, argumentString=None, validate=True)[source]

Bases: object

Class representing command-line options for the cback3-amazons3-sync script.

The Options class is a Python object representation of the command-line options of the cback3-amazons3-sync script.

The object representation is two-way: a command line string or a list of command line arguments can be used to create an Options object, and then changes to the object can be propogated back to a list of command-line arguments or to a command-line string. An Options object can even be created from scratch programmatically (if you have a need for that).

There are two main levels of validation in the Options class. The first is field-level validation. Field-level validation comes into play when a given field in an object is assigned to or updated. We use Python’s property functionality to enforce specific validations on field values, and in some places we even use customized list classes to enforce validations on list members. You should expect to catch a ValueError exception when making assignments to fields if you are programmatically filling an object.

The second level of validation is post-completion validation. Certain validations don’t make sense until an object representation of options is fully “complete”. We don’t want these validations to apply all of the time, because it would make building up a valid object from scratch a real pain. For instance, we might have to do things in the right order to keep from throwing exceptions, etc.

All of these post-completion validations are encapsulated in the Options.validate method. This method can be called at any time by a client, and will always be called immediately after creating a Options object from a command line and before exporting a Options object back to a command line. This way, we get acceptable ease-of-use but we also don’t accept or emit invalid command lines.

Note: Lists within this class are “unordered” for equality comparisons.

__init__(argumentList=None, argumentString=None, validate=True)[source]

Initializes an options object.

If you initialize the object without passing either argumentList or argumentString, the object will be empty and will be invalid until it is filled in properly.

No reference to the original arguments is saved off by this class. Once the data has been parsed (successfully or not) this original information is discarded.

The argument list is assumed to be a list of arguments, not including the name of the command, something like sys.argv[1:]. If you pass sys.argv instead, things are not going to work.

The argument string will be parsed into an argument list by the util.splitCommandLine function (see the documentation for that function for some important notes about its limitations). There is an assumption that the resulting list will be equivalent to sys.argv[1:], just like argumentList.

Unless the validate argument is False, the Options.validate method will be called (with its default arguments) after successfully parsing any passed-in command line. This validation ensures that appropriate actions, etc. have been specified. Keep in mind that even if validate is False, it might not be possible to parse the passed-in command line, so an exception might still be raised.

Note: The command line format is specified by the _usage function. Call _usage to see a usage statement for the cback3-amazons3-sync script.

Note: It is strongly suggested that the validate option always be set to True (the default) unless there is a specific need to read in invalid command line arguments.

Parameters:
  • argumentList (List of arguments, i.e. sys.argv) – Command line for a program
  • argumentString (String, i.e. "cback3-amazons3-sync --verbose stage store") – Command line for a program
  • validate (Boolean true/false) – Validate the command line after parsing it
Raises:
  • getopt.GetoptError – If the command-line arguments could not be parsed
  • ValueError – If the command-line arguments are invalid
buildArgumentList(validate=True)[source]

Extracts options into a list of command line arguments.

The original order of the various arguments (if, indeed, the object was initialized with a command-line) is not preserved in this generated argument list. Besides that, the argument list is normalized to use the long option names (i.e. –version rather than -V). The resulting list will be suitable for passing back to the constructor in the argumentList parameter. Unlike buildArgumentString, string arguments are not quoted here, because there is no need for it.

Unless the validate parameter is False, the Options.validate method will be called (with its default arguments) against the options before extracting the command line. If the options are not valid, then an argument list will not be extracted.

Note: It is strongly suggested that the validate option always be set to True (the default) unless there is a specific need to extract an invalid command line.

Parameters:validate (Boolean true/false) – Validate the options before extracting the command line
Returns:List representation of command-line arguments
Raises:ValueError – If options within the object are invalid
buildArgumentString(validate=True)[source]

Extracts options into a string of command-line arguments.

The original order of the various arguments (if, indeed, the object was initialized with a command-line) is not preserved in this generated argument string. Besides that, the argument string is normalized to use the long option names (i.e. –version rather than -V) and to quote all string arguments with double quotes ("). The resulting string will be suitable for passing back to the constructor in the argumentString parameter.

Unless the validate parameter is False, the Options.validate method will be called (with its default arguments) against the options before extracting the command line. If the options are not valid, then an argument string will not be extracted.

Note: It is strongly suggested that the validate option always be set to True (the default) unless there is a specific need to extract an invalid command line.

Parameters:validate (Boolean true/false) – Validate the options before extracting the command line
Returns:String representation of command-line arguments
Raises:ValueError – If options within the object are invalid
debug

Command-line debug (-d,--debug) flag.

diagnostics

Command-line diagnostics (-D,--diagnostics) flag.

help

Command-line help (-h,--help) flag.

ignoreWarnings

Command-line ignoreWarnings (-w,--ignoreWarnings) flag

logfile

Command-line logfile (-l,--logfile) parameter.

mode

Command-line mode (-m,--mode) parameter.

output

Command-line output (-O,--output) flag.

owner

Command-line owner (-o,--owner) parameter, as tuple (user,group).

quiet

Command-line quiet (-q,--quiet) flag.

s3BucketUrl

Command-line s3BucketUrl, target of sync.

sourceDir

Command-line sourceDir, source of sync.

stacktrace

Command-line stacktrace (-s,--stack) flag.

validate()[source]

Validates command-line options represented by the object.

Unless --help or --version are supplied, at least one action must be specified. Other validations (as for allowed values for particular options) will be taken care of at assignment time by the properties functionality.

Note: The command line format is specified by the _usage function. Call _usage to see a usage statement for the cback3-amazons3-sync script.

Raises:ValueError – If one of the validations fails
verbose

Command-line verbose (-b,--verbose) flag.

verifyOnly

Command-line verifyOnly (-v,--verifyOnly) flag.

version

Command-line version (-V,--version) flag.

CedarBackup3.tools.amazons3.cli()[source]

Implements the command-line interface for the cback3-amazons3-sync script.

Essentially, this is the “main routine” for the cback3-amazons3-sync script. It does all of the argument processing for the script, and then also implements the tool functionality.

This function looks pretty similiar to CedarBackup3.cli.cli(). It’s not easy to refactor this code to make it reusable and also readable, so I’ve decided to just live with the duplication.

A different error code is returned for each type of failure:

  • 1: The Python interpreter version is < 3.4
  • 2: Error processing command-line arguments
  • 3: Error configuring logging
  • 5: Backup was interrupted with a CTRL-C or similar
  • 6: Error executing other parts of the script

Note: This script uses print rather than logging to the INFO level, because it is interactive. Underlying Cedar Backup functionality uses the logging mechanism exclusively.

Returns:Error code as described above

CedarBackup3.tools.span module

Spans staged data among multiple discs

This is the Cedar Backup span tool. It is intended for use by people who stage more data than can fit on a single disc. It allows a user to split staged data among more than one disc. It can’t be an extension because it requires user input when switching media.

Most configuration is taken from the Cedar Backup configuration file, specifically the store section. A few pieces of configuration are taken directly from the user.

author:Kenneth J. Pronovici <pronovic@ieee.org>
class CedarBackup3.tools.span.SpanOptions(argumentList=None, argumentString=None, validate=True)[source]

Bases: CedarBackup3.cli.Options

Tool-specific command-line options.

Most of the cback3 command-line options are exactly what we need here – logfile path, permissions, verbosity, etc. However, we need to make a few tweaks since we don’t accept any actions.

Also, a few extra command line options that we accept are really ignored underneath. I just don’t care about that for a tool like this.

validate()[source]

Validates command-line options represented by the object. There are no validations here, because we don’t use any actions. :raises: ValueError – If one of the validations fails

CedarBackup3.tools.span.cli()[source]

Implements the command-line interface for the cback3-span script.

Essentially, this is the “main routine” for the cback3-span script. It does all of the argument processing for the script, and then also implements the tool functionality.

This function looks pretty similiar to CedarBackup3.cli.cli(). It’s not easy to refactor this code to make it reusable and also readable, so I’ve decided to just live with the duplication.

A different error code is returned for each type of failure:

  • 1: The Python interpreter version is < 3.4
  • 2: Error processing command-line arguments
  • 3: Error configuring logging
  • 4: Error parsing indicated configuration file
  • 5: Backup was interrupted with a CTRL-C or similar
  • 6: Error executing other parts of the script

Note: This script uses print rather than logging to the INFO level, because it is interactive. Underlying Cedar Backup functionality uses the logging mechanism exclusively.

Returns:Error code as described above