Capture stdout/stderr and exit code in perl

The modern recommended way:

use strict;
use warnings;

use Capture::Tiny qw/capture/;

my $cmd = "ls -l /tmp /non_existant_path";
my ($stdout, $stderr, $exitcode) = capture {
    system( $cmd );
};
$exitcode >>= 8;

print "STDOUT:\n", $stdout, "\n";
print "STDERR:\n", $stderr, "\n";
print "Exit code: $exitcode\n";

If Capture::Tiny is not available, IPC::Open3 might be ( but this has some caveats ).

use strict;
use warnings;

use IPC::Open3;

local(*MY_STDIN, *MY_STDOUT, *MY_STDERR);

my $cmd = "ls -l /tmp /nonexistant_path";

my $childpid = open3(*MY_STDIN, *MY_STDOUT, *MY_STDERR, $cmd);
my @outlines = <MY_STDOUT>;
my @errlines = <MY_STDERR>;  # XXX: block potential if massive
close MY_STDOUT;
close MY_STDERR;

print "STDOUT:\n", @outlines, "\n";
print "STDERR:\n", @errlines, "\n";
waitpid($childpid, 0);
my $exitcode = $? >> 8;
print "Exit code $exitcode\n";

Using open3 may fail if the output being thrown in stdout/stderr is too large and cannot be buffered. This can be made to work with IO::Select. See also http://docstore.mik.ua/orelly/perl2/prog/ch32_31.htm .

Capture stdout/stderr and exit code in perl

Speed up ssh logins

In this article we are going to explore two ways to speed up ssh logins. Reusing existing connections and disabling GSSAPI authentication.

The changes can be made on the ssh client side by adding the relevant config to the ssh config file at $HOME/.ssh/config .

Reusing existing connections

If you login to the same host using the same credentials multiple times (ie: different windows), one way to speed up connections is to share the same ssh connection. This means only the first connection actually needs to login. This also speeds things up if you open an ssh session, and later use scp/rsync to copy files across. Instead of opening a new connection, scp/rsync will reuse the existing ssh connection.

Host *
    ControlMaster auto
    ControlPath /tmp/ssh_mux_%h_%p_%r

Disabling GSSAPI Authentication

The GSSAPI is a IETF standard that provides an alternative to public/private keys as a means of authentication, suitable for doing strong encrypted authentication1. Sometimes ssh takes a long time figuring out if it needs to use GSSAPIAuthentication or not. If you don’t use it, disabling it can speed things up.

Host *
    GSSAPIAuthentication no

Footnotes

1. Using GSSAPI authentication at SLAC

Speed up ssh logins

Force cfengine client to download/execute new promises

CFEngine is a configuration management system, once upon a time famous for being used by facebook (who since migrated to chef).

CFEngine has a server component where configuration changes are written to, and a client side component that downloads and applies them. When testing around, it’s useful to be able to tell the client to download and execute these “promises” (as they are called in cfengine speak) immediately rather than waiting for the various daemons to kick in. This is how it can be done:

On the hub:

cf-agent -K -f failsafe.cf
cf-promises

On the client

rm -f /var/cfengine/inputs/cf_promises_validated
cf-agent -K -f failsafe.cf   # Download new promises
cf-agent -vK                     #execute new promises in verbose mode

Happy configuration managing !

Force cfengine client to download/execute new promises

Ignore SSL certificates in LWP

Ever since release 6.00, Perl’s LWP validates the server’s SSL certificate on HTTPS requests. By default, LWP will use the certificate bundle provided by Mozilla::CA to verify the server certificate. This is detailed in the changelog for 6.00.

There are two way of reverting to the old behaviour (ignore server certificate):

Setting an environment variable, ie:

PERL_LWP_SSL_VERIFY_HOSTNAME=0

Passing an option to the LWP::UserAgent object asking for certificate validation to be ignored, ie:

    use LWP::UserAgent;
    use IO::Socket::SSL qw();

    my $ua = LWP::UserAgent->new(
        ssl_opts => {
            SSL_verify_mode => IO::Socket::SSL::SSL_VERIFY_NONE,
            verify_hostname => 0,
#            SSL_hostname => '',# Set SSL_hostname if you do want to verify the hostname
                                # (ie, when using SNI https://en.wikipedia.org/wiki/Server_Name_Indication)
        }
    );

The SSL_hostname option is only required if you intend to fake the “Host” HTTP header ( so that it doesn’t mismatch with a similar header sent in the SSL handshake, see Server Name Indication ).

Ignore SSL certificates in LWP

Using tcpdump to capture network traffic on the command line

Looking at low level network traffic is often useful to diagnose application/system problems.

This is easy to do in modern browsers using devtools network capture widgets and extensions such as postman, but sometimes you need to diagnose traffic between servers in a live application, as opposed to being able to do the requests yourself from your browser. In these cases, tcpdump shines.

This article will cover how to use tcpdump to diagnose non-encrypted traffic. For encrypted traffic, ssldump can be used provided you have access to the private key used to encrypt the traffic.

To listen for traffic and write to stdout, the snippet is:

tcpdump -i any -s 65535 -n -A expression
  • -i The interface id to listen on ( run tcpdump -D to get a list of available interfaces, or just use the keyword “any” to listen in all network interfaces )
  • -n Don’t convert host addresses to names (avoid dns resolution, makes things faster)
  • -A Print packets in ASCII, excluding link level headers. Use -X for both hex and ASCII printout.
  • -s By default tcpdump only captures the first 68 bytes of each packet. This option allows you specify how much of each packet to capture. The maximum IP packet size is 65535.

See man pcap-filter for a full description of ‘expression’

Examples include:

tcpdump -i 11 -n -A -s 65535 port 80
tcpdump -i 11 -n -A -s 65535 dst host 10.0.0.1

Other useful flags:

  • -w write packets to a file which can later be replayed
  • -r read packets written with -w

Examples:

tcpdump -i any -n -A -s 65535 -w network_traffic.capture
tcpdump -n -r network_traffic.capture -A port 80

The examples above will get you started, but for more detail, check the man pages and cheat sheet:

Happy sniffing!

Using tcpdump to capture network traffic on the command line