How to Create a Custom Transport Script for Backups

backups whmadvanced

How to Create a Custom Transport Script for Backups

Valid for versions 82 through the latest version

Version:

82

Last modified: June 21, 2021

Overview

Warning:

Only advanced users should create custom transport scripts. Other users can try one of WHM’s other options:

A local directory
Amazon S3™
Backblaze B2
FTP
Google Drive™
Rsync
S3 Compatible
SFTP
WebDAV

The Backup Configuration feature allows users to create a custom destination for their backups.

Create a custom transport script

The custom transport script is a script that you must provide for each custom backup destination that you set up in WHM’s Backup Configuration interface (WHM » Home » Backups » Backup Configuration). You can enter the transport script’s absolute path via the Script setting for the Custom destination type in the Additional Destinations section.

Script operation

The following rules affect how the script interacts with the system:

The script runs once per command.
The script cannot save state information between commands.
The system does not reuse the connection between commands. Instead, each time that the script runs, the system creates the connection to the remote custom destination, and then drops it after the script runs.

The system passes information to the script, through the command line, in the following order:

Command name.
Current directory.
Command specific parameters.
Host.
Username.

The system passes the password information to the script through the environment.

Script commands

The script must implement the following commands:

Command	Description	Parameters
`chdir`	This command changes directories on the remote destination. It is equivalent to the `cd` command on the command line.	`$path` — A file path.
`delete`	This command deletes an individual file on the remote destination.	`$path` — A file path.
`get`	This command copies a remote file to a local destination.	`$dest_root_dir` — The remote directory. `$dest_file` — The remote file name. `$local_file` — The full path to the local file.
`ls`	This command prints identical output to the `ls -l` command.	`$path` — A file path.
`mkdir`	This command creates a directory on the remote destination.	`$path` — A file path.
`put`	This command copies a local file to a remote destination.	`$dest_root_dir` — The remote directory. `$dest_file` — The remote file name. `$local_file` — The full path to the local file.
`rmdir`	This command deletes a directory on the remote destination. We strongly recommend that you verify the path that you plan to delete. If you pass the root directory (`/`) as the path to delete, your system will experience serious problems.	`$path` — A file path.

Backups run each of these commands individually while the system transports the backup file and validates the destination.

Your script should return any output to STDOUT to return data to the user.

Note:

If the script fails, it prints the output to STDERR. The system logs any data that the script returns to STDERR as part of the failure.

Templates

You can use the /usr/local/cpanel/scripts/custom_backup_destination.pl.skeleton script in cPanel & WHM as a template to create your own custom_backup_destination.pl script. For a sample backup transport script, see the /usr/local/cpanel/scripts/custom_backup_destination.pl.sample script.

Click to view...

#!/usr/local/cpanel/3rdparty/bin/perl

# cpanel - scripts/custom_backup_destination.pl.skeleton      Copyright 2021 cPanel, L.L.C
#                                                           All rights Reserved.
# [email protected]                                         http://cpanel.net
# This code is subject to the cPanel license. Unauthorized copying is prohibited

use strict;
use warnings;

# These are the commands that a custom destination script must process
my %commands = (
    put    => \&my_put,
    get    => \&my_get,
    ls     => \&my_ls,
    mkdir  => \&my_mkdir,
    chdir  => \&my_chdir,
    rmdir  => \&my_rmdir,
    delete => \&my_delete,
);

# There must be at least the command and the local directory
usage() if ( @ARGV < 2 );

#
# The command line arguments passed to the script will be in the following order:
# command, local_directory, command arguments, and optionally, host, user password
# The local directory is passed in so we know from which directory to run the command
# We need to pass this in each time since we start the script fresh for each command
#
my ( $cmd, $local_dir, @args ) = @ARGV;

# complain if the command does not exist
usage() unless exists $commands{$cmd};

# Run our command
$commands{$cmd}->(@args);

#
# This script should only really be executed by the custom backup destination type
# If someone executes it directly out of curiosity, give them usage info
#
sub usage {
    my @cmds = sort keys %commands;
    print STDERR "This script is for implementing a custom backup destination\n";
    print STDERR "It requires the following arguments:  cmd, local_dir, cmd_args\n";
    print STDERR "These are the valid commands:  @cmds\n";
    exit 1;
}

#
# This portion contains the implementations for the various commands
# that the script needs to support in order to implement a custom destination
#

#
# Copy a local file to a remote destination
#
sub my_put {
    my ( $local, $remote, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};
    return;
}

#
# Copy a remote file to a local destination
#
sub my_get {
    my ( $remote, $local, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};
    return;
}

#
# Print out the results of doing an ls operation
# The calling program will expect the data to be
# in the format supplied by 'ls -l' and have it
# printed to STDOUT
#
sub my_ls {
    my ( $path, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};
    return;
}

#
# Create a directory on the remote destination
#
sub my_mkdir {
    my ( $path, $recurse, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};
    return;
}

#
# Change into a directory on the remote destination
# This does not have the same meaning as it normally would since the script
# is run anew for each command call.
# This needs to do the operation to ensure it doesn't fail
# then print the new resulting directory that the calling program
# will pass in as the local directory for subsequent calls
#
sub my_chdir {
    my ( $path, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};
    return;
}

#
# Recursively delete a directory on the remote destination
#
sub my_rmdir {
    my ( $path, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};
    return;
}

#
# Delete an individual file on the remote destination
#
sub my_delete {
    my ( $path, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};
    return;
}

View the custom_backup_destination.pl.sample script...

#!/usr/local/cpanel/3rdparty/bin/perl

# cpanel - scripts/custom_backup_destination.pl.sample      Copyright 2013 cPanel, L.L.C
#                                                           All rights Reserved.
# [email protected]                                         http://cpanel.net
# This code is subject to the cPanel license. Unauthorized copying is prohibited

use strict;
use warnings;
use Cwd qw(getcwd abs_path);
use File::Spec;
use File::Copy;
use File::Path qw(make_path remove_tree);
use autodie qw(:all copy);

# These are the commands that a custom destination script must process
my %commands = (
    put    => \&my_put,
    get    => \&my_get,
    ls     => \&my_ls,
    mkdir  => \&my_mkdir,
    chdir  => \&my_chdir,
    rmdir  => \&my_rmdir,
    delete => \&my_delete,
);

# There must be at least the command and the local directory
usage() if ( @ARGV < 2 );

#
# The command line arguments passed to the script will be in the following order:
# command, local_directory, command arguments, and optionally, host and user
# The local directory is passed in so we know from which directory to run the command
# we need to pass this in each time since we start the script fresh for each command
#
my ( $cmd, $local_dir, @args ) = @ARGV;

# complain if the command does not exist
usage() unless exists $commands{$cmd};

# For this example transport, we are going to simply copy everything under this directory
my $dest_root_dir = '/custom_transport_demo';
mkdir $dest_root_dir unless -d $dest_root_dir;

# Step into the local directory
# This will be under the directory that we have as the file destination
$local_dir = File::Spec->catdir( $dest_root_dir, $local_dir );
make_path($local_dir) unless -d $local_dir;
chdir $local_dir;

# Run our command
$commands{$cmd}->(@args);

#
# This script should only really be executed by the custom backup destination type
# If someone executes it directly out of curiosity, give them usage info
#
sub usage {
    my @cmds = sort keys %commands;
    print STDERR "This script is for implementing a custom backup destination\n";
    print STDERR "It requires the following arguments:  cmd, local_dir, cmd_args\n";
    print STDERR "These are the valid commands:  @cmds\n";
    exit 1;
}

#
# Convert a path to be under our destination directory
# Absolute paths will be directly under it,
# relative paths will be relative to the local directory
#
sub convert_path {
    my ($path) = @_;

    if ( $path =~ m|^/| ) {
        $path = File::Spec->catdir( $dest_root_dir, $path );
    }
    else {
        $path = File::Spec->catdir( $local_dir, $path );
    }

    return $path;
}

#
# Convert a full path to the path under the the directory
# where we copy all the files
#
sub get_sub_directory {
    my ($path) = @_;

    # The first part will be the destination root directory,
    # Remove that part of the path and we will have the subdirectory
    $path =~ s|^$dest_root_dir||;

    return $path;
}

#
# This portion contains the implementations for the various commands
# that the script needs to support in order to implement a custom destination
#

#
# Copy a local file to a remote destination
#
sub my_put {
    my ( $local, $remote, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};

    $remote = convert_path($remote);

    # Make sure the full destination directory exists
    my ( undef, $dir, undef ) = File::Spec->splitpath($remote);
    make_path($dir) unless ( $dir and -d $dir );
    copy( $local, $remote );
    return;
}

#
# Copy a remote file to a local destination
#
sub my_get {
    my ( $remote, $local, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};

    $remote = convert_path($remote);

    copy( $remote, $local );
    return;
}

#
# Print out the results of doing an ls operation
# The calling program will expect the data to be
# in the format supplied by 'ls -l' and have it
# printed to STDOUT
#
sub my_ls {
    my ( $path, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};

    $path = convert_path($path);

    # Cheesy, but this is a demo
    my $ls = `ls -al $path`;

    # Remove the annoying 'total' line
    $ls =~ s|^total[^\n]*\n||;

    print $ls;
    return;
}

#
# Create a directory on the remote destination
#
sub my_mkdir {
    my ( $path, $recurse, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};

    $path = convert_path($path);

    make_path($path);

    die "Failed to create $path" unless -d $path;
    return;
}

#
# Change into a directory on the remote destination
# This does not have the same meaning as it normally would since the script
# is run anew for each command call.
# This needs to do the operation to ensure it doesn't fail
# then print the new resulting directory that the calling program
# will pass in as the local directory for subsequent calls
#

sub my_chdir {
    my ( $path, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};

    $path = convert_path($path);
    chdir $path;

    print get_sub_directory( getcwd() ) . "\n";
    return;
}

#
# Recursively delete a directory on the remote destination
#
sub my_rmdir {
    my ( $path, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};

    $path = convert_path($path);

    remove_tree($path);

    die "$path still exists" if -d $path;
    return;
}

#
# Delete an individual file on the remote destination
#
sub my_delete {
    my ( $path, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};

    $path = convert_path($path);

    unlink $path;
    return;
}

The system passes most variables as arguments to the command line. If your script does not pass one of the hardcoded arguments to the core functions, the system will display all valid arguments in the global %commands hash.

Code examples

Use statements

Begin the script with the standard use statements. Include any modules that you may need for your transport:

1
2
3
4
5
6
7


use strict;
use warnings;
use Cwd qw(getcwd abs_path);
use File::Spec;
use File::Copy;
use File::Path qw(make_path remove_tree);
use autodie qw(:all copy);

The %commands list

Note:

The script can only process the following commands.

1
2
3
4
5
6
7
8
9


my %commands = (
    put    => \&my_put,
    get    => \&my_get,
    ls     => \&my_ls,
    mkdir  => \&my_mkdir,
    chdir  => \&my_chdir,
    rmdir  => \&my_rmdir,
    delete => \&my_delete,
);

The %command subroutines

Every call to the script begins with a command and a local directory. You must pass the command line arguments in the following order:

$cmd — The command.
$local_dir — The local directory.
@args — The command’s arguments.
$host — Optional. The remote destination’s hostname or IP address.
$user — Optional. The remote destination’s account username.
$password — Optional. The remote destination’s password.

Use the arguments that are specific to each of the commands and variables.

Note:

You should only include the optional $host, $user, and $password values if you configured them in the transport.
If you include the $password value, you must use the ENV hash.
You must include the $local_dir variable in every command subroutine that you create because the script calls each command individually.

1
2
3


my ( $cmd, $local_dir, @args, $host, $user ) = @ARGV;
my $password = $ENV{'PASSWORD'};
 usage() unless exists $commands{$cmd};

The put functions

The put function directs the script to upload or copy a local file to a remote destination. This function works similarly to the FTP put command.

Note:

For more robust transports, we strongly recommend that you perform several error checks for each step to ensure that the system reports all errors back properly.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


sub my_put {
    my ( $local, $remote, $host, $user ) = @_;   # Required argument order

    my $password = $ENV{'PASSWORD'};     # Enclose the password in the ENV hash

    $remote = convert_path($remote);     # the remote file's variable

    my ( undef, $dir, undef ) = File::Spec->splitpath($remote); # Make sure the full destination directory exists
    make_path($dir) unless ( $dir and -d $dir );

    copy( $local, $remote ); # copy the local file to the remote file
    return;
}

The get function

The get function directs the script to download or retrieve a local file from a remote destination. This function works similarly to the FTP get command.

1
2
3
4
5
6
7
8


sub my_get {
    my ( $remote, $local, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};
    $remote = convert_path($remote);

    copy( $remote, $local );
    return;
}

The ls function

The ls function pulls the listing of a remote file or directory, similar to FTP or a local ls argument on the command line.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


sub my_ls {
    my ( $path, $host, $user ) = @_;

    my $password = $ENV{'PASSWORD'};

    $path = convert_path($path);

    my $ls = `ls -al $path`;

    # Remove the annoying 'total' line
    $ls =~ s|^total[^\n]*\n||;

    print $ls;
    return;
}

The mkdir function

The mkdir function ensures that a directory exists on the remote machine and that the system uploads the backup to a real path.

Note:

Not all transports use a feature like the mkdir function, however, you must include this function in the script.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


sub my_mkdir {
    my ( $path, $recurse, $host, $user ) = @_;

    my $password = $ENV{'PASSWORD'};

    $path = convert_path($path);

    make_path($path);

    die "Failed to create $path" unless -d $path;
    return;
}

The chdir function

The chdir function allows you to store the working directory and keep the session information between operations.

Note:

Because this is a custom transport script, the system will not keep session information between operations by a single active process.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


sub my_chdir {
    my ( $path, $host, $user ) = @_;
    my $password = $ENV{'PASSWORD'};

    $path = convert_path($path);
    chdir $path;

    print get_sub_directory( getcwd() ) . "\n";
    return;
}

The rmdir function

The rmdir function removes a directory and recursively deletes everything below the given directory. Based on which transport you use, you may need to remove all the files and directories below the given directory before the system can execute this function.

Warning:

We strongly recommend that you verify the path that you plan to recursively delete. If you pass the root (/) directory as the path to delete, your system will experience serious issues.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


sub my_rmdir {
    my ( $path, $host, $user ) = @_;

    my $password = $ENV{'PASSWORD'};

    $path = convert_path($path);

    remove_tree($path);

    die "$path still exists" if -d $path;
    return;
}

The delete function

The delete function deletes a single file.

Note:

We strongly recommend that you ensure the path that you use, relative or full, is appropriate for the transport. If your transport does not provide an error status check, use the ls function on the file to ensure the system deleted it.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


sub my_delete {
    my ( $path, $host, $user ) = @_;

    my $password = $ENV{'PASSWORD'};

    $path = convert_path($path);

    unlink $path;
    return;
}

Basic error check

Note:

We strongly recommend that you:

Perform a basic error check to ensure that the script receives the proper arguments when the system calls it. At a minimum, you must pass the usage() if ( @ARGV < 2 ) command.
Construct a built-in description of what the script is, what it does, and its purpose so that you can identify the script.

1
2


sub my_delete {
usage() if ( @ARGV < 2 );

1
2
3
4
5
6
7


sub usage {
    my @cmds = sort keys %commands;
    print STDERR "This script is for implementing a custom backup destination\n";
    print STDERR "It requires the following arguments:  cmd, local_dir, cmd_args\n";
    print STDERR "These are the valid commands:  @cmds\n";
    exit 1;
}

The $cmd command

Use the %command hash to call each command’s specific code block.

$commands{$cmd}->(@args);

How to Create a Custom Transport Script for Backups

Version:

82

Overview

Create a custom transport script

Script operation

Script commands

Templates

Code examples

Use statements

The %commands list

The %command subroutines

The put functions

The get function

The ls function

The mkdir function

The chdir function

The rmdir function

The delete function

Basic error check

The $cmd command

Additional Documentation