Automating Interactive Shell Operations with pexpect in Python

Linux’s Bash shell has a built-in command called expect. This is used for automating command-line work by automatically inputting interactive shells such as password input prompts. As a Python wrapper for this expect, there is the pexpect package.

Install pexpect with the following:

pip install pexpect

As an example of using pexpect, I’ll show the steps up to pushing to Github and automatically inputting the password prompt using pexpect.

First, to execute simple commands like git add . in python, do the following: The following is an example using Python 3.6.7. If the python version is different, incompatibilities around shell=True may cause errors. To install and switch between multiple python versions, it’s recommended to use a version management system like pyenv. pyenv is explained in other articles. Please use the search function on this blog.

# Python 3.6.7

import pexpect
import subprocess
import os

cmd = subprocess.run("git add .", shell=True, stderr=subprocess.PIPE, stdout=subprocess.PIPE)
if ( cmd.stdout ) : print( cmd.stdout.decode('utf-8') )
if ( cmd.stderr ) : print( cmd.stderr.decode('utf-8') )

Continuing, let’s proceed to git commit.

cmd = subprocess.run("git status", shell=True, stderr=subprocess.PIPE, stdout=subprocess.PIPE)
if ( cmd.stdout ) : print( cmd.stdout.decode('utf-8') )
if ( cmd.stderr ) : print( cmd.stderr.decode('utf-8') )

cmd = subprocess.run("git commit -m .", shell=True, stderr=subprocess.PIPE, stdout=subprocess.PIPE)
if ( cmd.stdout ) : print( cmd.stdout.decode('utf-8') )
if ( cmd.stderr ) : print( cmd.stderr.decode('utf-8') )

The following is a specific example of using pexpect. It automatically inputs the username and password prompts. You need to set environment variables for username and password respectively.

child = pexpect.spawn('git push -u origin gh-pages')
child.expect('Username .*')
child.sendline(os.environ['GITHUB_USERNAME'])
child.expect('Password .*')
child.sendline(os.environ['GITHUB_PASSWORD'])
child.expect(pexpect.EOF, timeout=None)
cmd_show_data = child.before
print( cmd_show_data.decode() )

The overall script looks like this:

# Python 3.6.7

import pexpect
import subprocess
import os

cmd = subprocess.run("git add .", shell=True, stderr=subprocess.PIPE, stdout=subprocess.PIPE)
if ( cmd.stdout ) : print( cmd.stdout.decode('utf-8') )
if ( cmd.stderr ) : print( cmd.stderr.decode('utf-8') )

cmd = subprocess.run("git status", shell=True, stderr=subprocess.PIPE, stdout=subprocess.PIPE)
if ( cmd.stdout ) : print( cmd.stdout.decode('utf-8') )
if ( cmd.stderr ) : print( cmd.stderr.decode('utf-8') )

cmd = subprocess.run("git commit -m .", shell=True, stderr=subprocess.PIPE, stdout=subprocess.PIPE)
if ( cmd.stdout ) : print( cmd.stdout.decode('utf-8') )
if ( cmd.stderr ) : print( cmd.stderr.decode('utf-8') )

child = pexpect.spawn('git push -u origin gh-pages')
child.expect('Username .*')
child.sendline(os.environ['GITHUB_USERNAME'])
child.expect('Password .*')
child.sendline(os.environ['GITHUB_PASSWORD'])
child.expect(pexpect.EOF, timeout=None)
cmd_show_data = child.before
print( cmd_show_data.decode() )