functions in bash

Well, at work we often take a dated backup copy of a file that we might be editing. Some time ago I wrote a perl script that does some things to the file that is backed up. For example, if I was editing a zone file, why not increment the serial number inside it? This worked fine for a while, but later I decided it wasn't such a good idea to have this on every single host that I work on. So, to cut a long story short, here's the essential functionality implemented in bash so that it can exist in a .bashrc file, which is much easier to manage.

function bkp() { N=0; DATE=$( /bin/date +%Y%m%d ); for i in $@ ; do D=$( /usr/bin/dirname $i ); F=$( /bin/echo $i | /bin/sed -e 's/.*\///g' ); if [ ! -d $D/old ] ; then /bin/mkdir -p $D/old; fi ; until [ 1 -eq 2 ] ; do let “N += 1″; C=$( /usr/bin/printf “%s/old/%s-%s-%.2d-%s” $D $F $DATE $N $USER ); if [ ! -f $C ] ; then break ; fi ; done ; /bin/cp -p $i $C; done; }

Hope this can be of some use to you too. This does have some obvious dependencies but this shouldn't be too major.

time saving tricks

Another great time saver is alt-backspace. Give that a go, it deletes as far as the first non-alphanumeric character. Alt . is also useful, returns the last argument on the previous command.

Don't forget about the curly bracket expansion:

user@laptop:~$ echo file{a,b,c} filea fileb filec

This is notably very useful when you intend to create a large tree of items with identical nodes within.

$ mkdir -p {beta,current,release}/{code,scripts,html,images}

This would create the nodes code, scripts, html and images within the top directories beta, current, release. Much more useful than creating these individual items.

useful keystrokes

Most systems default to emacs input. These keystrokes require the use of ctrl/alt in a way that should seem familiar to a emacs user. Below there are a few of the vi alternatives.

emacs combination	vi	result
alt-.		last arg from previous command
alt-d	^[dw	delete the next word on the line
alt-backspace	^[db	delete backwards one word
ctrl-w	^[dT	delete backwards a whole word until white space/punctuation
ctrl-k	^[D	delete from the cursor to the end of the line
ctrl-u	^[d^	delete from the cursor to the start of the line
alt-f	^[E	move to the end of the word
alt-b	^[w	move to the start of the word
ctrl-e	^[$	move to the very end of the line
ctrl-a	^[^	move to the very start of the line

There are some pros and cons to both. If you wish to use the vi bindings then simply type set -o vi from a bash prompt. To revert type set -o emacs.

Some of the pros of using vi bindings are that you can use the movement keys and repeating them doesn't feel like getting finger strain from keyboard gymnastics. One of the things I dislike about the vi bindings is that there is no simple way of recalling the last word from the previous command history. This is highly useful in the emacs mode (alt-.).

subshells in practice

Something useful that's worth remembering is that () can create a sub-shell where the output can be piped elsewhere.

( cmd1 ; cmd2 ) | cmd3

sends output from both cmd1 and cmd2 are sent as stdin to cmd3.

This can be incredibly useful, especially if you're playing around mailing things via /usr/sbin/sendmail, for example, if you wish to send the contents of some output to someone/something and you wish to customise the headers a bit.

( /bin/echo -e "Subject: My favourite file\n\n" ; cat ~/.vimrc ) \ | /usr/sbin/sendmail -fme@example.test me@example.test

The huge advantage here is that the headers can be specified through the echo output, which will appear in front of the file contents when the sendmail program reads standard in.

input

Although not entirely related to bash, it's highly useful to add the following to your ~/.inputrc file so that annoying things like ~ don't appear when you press the delete key. This is often the case when the system you connect to does not have a friendly /etc/inputrc

"\e[3~": delete-char "\e[1;5C": forward-word "\e[1;5D": backward-word "\e[5C": forward-word "\e[5D": backward-word "\e\e[C": forward-word "\e\e[D": backward-word

This should enable the delete button for you, along with the ctrl-[left]/[right] buttons to go forward/backward a word. You might be more at home using alt-f/b key combinations however.

If you'd like to use vim style bindings at the command line then put the following at the end of your ~/.inputrc file:

#vi mode
$if mode=vi
set keymap vi-command
Control-l: clear-screen
"#": insert-comment
".": "i !*\r"
"|": "A | "
"D":kill-line
"C": "Da"
"dw": kill-word
"dd": kill-whole-line
"db": backward-kill-word
"cc": "ddi"
"cw": "dwi"
"cb": "dbi"
"daw": "lbdW"
"yaw": "lbyW"
"caw": "lbcW"
"diw": "lbdw"
"yiw": "lbyw"
"ciw": "lbcw"
"da\"": "lF\"df\""
"di\"": "lF\"lmtf\"d`t"
"ci\"": "di\"i"
"ca\"": "da\"i"
"da'": "lF'df'"
"di'": "lF'lmtf'd`t"
"ci'": "di'i"
"ca'": "da'i"
"da`": "lF\`df\`"
"di`": "lF\`lmtf\`d`t"
"ci`": "di`i"
"ca`": "da`i"
"da(": "lF(df)"
"di(": "lF(lmtf)d`t"
"ci(": "di(i"
"ca(": "da(i"
"da)": "lF(df)"
"di)": "lF(lmtf)d`t"
"ci)": "di(i"
"ca)": "da(i"
"da{": "lF{df}"
"di{": "lF{lmtf}d`t"
"ci{": "di{i"
"ca{": "da{i"
"da}": "lF{df}"
"di}": "lF{lmtf}d`t"
"ci}": "di}i"
"ca}": "da}i"
"da[": "lF[df]"
"di[": "lF[lmtf]d`t"
"ci[": "di[i"
"ca[": "da[i"
"da]": "lF[df]"
"di]": "lF[lmtf]d`t"
"ci]": "di]i"
"ca]": "da]i"
"da<": "lF<df>"
"di<": "lF<lmtf>d`t"
"ci<": "di<i"
"ca<": "da<i"
"da>": "lF<df>"
"di>": "lF<lmtf>d`t"
"ci>": "di>i"
"ca>": "da>i"
"da/": "lF/df/"
"di/": "lF/lmtf/d`t"
"ci/": "di/i"
"ca/": "da/i"
"da:": "lF:df:"
"di:": "lF:lmtf:d`t"
"ci:": "di:i"
"ca:": "da:i"
"gg": beginning-of-history
"G": end-of-history
?: reverse-search-history
/: forward-search-history

set keymap vi-insert
Control-l: clear-screen
"\C-a": beginning-of-line
"\C-e": end-of-line
"\e[A": history-search-backward
"\e[B": history-search-forward
$endif

input repetition

Another time save is the ability to repeat a command many times. This can be done by pressing alt-N (where N is a numeric). Follow this with a command or text and it will be repeated N number of times.

$ [alt-10]a $ aaaaaaaaaa

You don't just have to limit to text, you can for example remove 10 words by pressing alt-10 ^w (^w being ctrl-w)

The huge benefit of things like this is that most of the functionality of the BASH command line is due to libreadline which is linked in many other applications.

bash redirection

One of the very useful things you can do from within a script is capture it's output, rather than from the caller, which may become a bit of command line clobber that you don't want to record in your crontab.

$ my_script 2>&1 > /var/tmp/my_script_${HOSTNAME}_$$_${USER}.log

You can put all this into the script itself

#!/bin/bash

LOG=/var/tmp/my_script_${HOSTNAME}_$$_${USER}.log

exec 2>&1 1>${LOG}

So, everything beneath these lines will be captured via the LOG file.

bash in vi mode

Of late I have found many benefits for using bash in vi mode. For those who don't know, bash by default uses emacs bindings, one presses alt+[b|d|f|w] etc to mode around on the command line or ctrl-[p|n] to search through history.

Well, if you're using a slow connection or have a very limited keyboard it can be quite tedious to do these key strokes, for example, you may not have an alt, or even escape key on your handset.

If you're lacking both alt and escape buttons then you have to press ctrl+[ to emulate the escape button, followed by the alt combination button (b, d, f, w etc). So, to go forward one word, you'd have to press ctrl+[ followed by w, to go forward three words you'd have to repeat that three times. You could

In vi mode you can press ctrl+[ to enter command mode, followed by 3 followed by w. Bingo.

bind '"\e."':yank-last-arg bind '"\C-l"':clear-screen

To summarise, if using a limited keyboard, you may wish to have set -o vi in your environment (and set yank-last-arg too).

on the subject of subshells and descriptors

An interesting thing happened today with a peculiarity with Solaris Tar not behaving the same way as GNU Tar with regards to reading a list of input files from stdin.

With GNU Tar it's perfectly acceptable to feed tar a list of files as stdin, like so:

find /etc -type f | tar -cvf etc.tar -T -

Solaris tar uses -I to specify the list of input files, so as normal we'd use:

find /etc -type f | tar -cvf etc.tar -I -

Yet this fails to work, it reports that – cannot be opened, it appears it is trying to actually open -. After reading some pages online about how to do this with Solaris tar I find that the following does work:

tar -cvf etc -I <( find /etc -type f)

This surprises me greatly as I don't see any difference at first glance. The reason for this is that I mentally read the final component as $( find /etc -type f), "send the output of this process as input to another".

So lets inspect this a little:

$ cat <( ls ) deb Desktop Documents ...

Looks normal at the moment, that's what I'd expect from $ echo $( ls ) too.

$ echo <( ls ) /dev/fd/63

What is actually happening is that the output of the program is being stored in a file descriptor, the descriptor is then read as an argument, in the above case as a list of files for the tar program. Rather neat and not something that I'd taken correct attention of in the past particularly when doing things like:

$ diff <( echo ls | ssh host -t /bin/bash ) <( echo ls | ssh host -t /bin/bash )

Given the above, it all make a lot more sense now!

duplicating to one or more process

Sometimes you may have a system where you take data as an input, do something with that data and then write the results. Once complete, you may need to do something with the original data again, which would need opening the original data, which of course would require another disk read. This is obviously not efficient.

Before learning this trick, I would have passed the data to a perl script to do the duplication. There is however, a much easier way. If you noted above that <() creates a descriptor, then we can use this to our advantage with tee.

Doing the below will duplicate output to two files, one via stdout.

$ cat /etc/hosts | tee /var/tmp/hosts > /var/tmp/hosts.2

The exact same result can be done with cat, too:

$ cat /etc/hosts | tee >( cat > /var/tmp/hosts) > /var/tmp/hosts.2

What happens here, is that >() creates a file descriptor which is passed as an argument to tee, pretty handy really and allows you to reduce overhead of using an external process to do the duplication effort.

solaris tar -T - alike

Sadly Solaris tar does not have the ability to read files names on standard in, but you can tell it to read the files from the I parameter.

For instance, find . | tar cvf - -I - will fail, it will try to read the file list -. To accomplish our desire, you need to make use of intermediate files, like so:

tar cvf - -I <( find . )

bash will substitute the <() with a temporary file.

This could also be accomplished using a file of your own naming.

function exploit

Here's a simple test that I found on a discussion forum to see if your bash interpreter is subject to env x='() { :;}; echo vulnerable' bash -c "echo this is a test" This will echo vulnerable to the terminal if so, or will print to inform you of a function definition attempt.

fullscreen edit

In emacs mode (set -o emacs) you can use a full screen editor to compose your command:

^X ^E

or in vi mode (set -o vi), just press 'v'. You'll have to be in command mode (press escape, or ctrl [)first, though.

This is highly useful if you wish to build up a complex command line or if you wish to script something that requires some indentation where only a full screen editor will do.