WiredRevolution.com » BASH

Have a Bash script determine it’s own location

Ryan — Wed, 15 Dec 2010 18:49:37 +0000

At some point it may be helpful to have a BASH script dynamically determine the location of itself when executed from anywhere on the system. The following code will produce the canonicalized absolute pathname of the script, as well as the directory that it resides in.

The script first determines if the the first argument which is a path to the script is a relative or absolute path. If it is a relative path it will prepend the current working directory. Finally the entire path is canonicalized which resolves all symlinks and removes the extra “./”, “../” and ‘/’ characters.

Here is the script:

if [[ "$0" =~ ^/ ]] then SCRIPT_LOCATION=$(readlink -f "$0") else SCRIPT_LOCATION=$(readlink -f "$(pwd)/""$0") fi SCRIPT_DIR=$(dirname $SCRIPT_LOCATION) echo $SCRIPT_LOCATION echo $SCRIPT_DIR

Now for a quick test. Lets assume our script is located here:
/home/ryan/scripts/mylocation.sh

Executing the script in the same directory:
$ ./mylocation/home/ryan/scripts/mylocation.sh /home/ryan/scripts

Executing in another directory with the absolute path produces the same results:
$ /home/ryan//////scripts/../scripts/mylocation.sh/home/ryan/scripts/mylocation.sh /home/ryan/scripts

Convert text files within a directory from Windows to Unix format

Ryan — Wed, 07 Jan 2009 14:14:33 +0000

When a file is saved in Windows and then moved to a Linux system the formatting differences can cause a variety of problems. To effectively use these files you will need to change the format from Windows/DOS to Unix. This conversion occurs by simply removing the Windows carriage return characters.

I have explained how to use the tr command remove these windows carriage returns, but when you have a large amount of files to convert this can become tedious. As a solution to this I have written a BASH script to convert all text files within a directory.

#!/bin/bash

for file in /directory/to/convert/*
do
  if [[ -f $file && `file $file | grep text` ]]
  then
    tr -d '\r' < $file > "$file"_clear
    mv "$file"_clear $file
  fi
done

Command substitution in BASH

Ryan — Fri, 07 Nov 2008 13:42:59 +0000

BASH has the ability to execute a command string and replace that string with the output of the command. Or to say it another way, the output of a command will replace the command itself.

There are 2 ways to accomplish this.

The first is to surround the command with backticks or blockquotes.
`command`

You can also substitute with dollar symbol and parenthesis.
$(command)

The following will do a direct substitution of the current username on the command line.

$ echo /home/$(whoami)

/home/ryan

Another great use is the ability to assign a command to an environment variable.

$ MYDATE=`date`

or using the other format

$ MYDATE=$(date)

Delete a specific line from a text file with sed

Ryan — Sun, 26 Oct 2008 09:05:21 +0000

At some point you may have the need to remove all lines within a text file that match a certain pattern. Accomplishing this is easy with the sed command.

Here is the command format.

$ sed -i '/PATTERN/ d' file.txt

The ‘-i‘ option allows you to edit the specified file in place.
PATTERN is a regular expression
d is a command which instructs sed to delete the line if it matches the PATTERN

Here is an example.

$ cat file.txt

one
two
three
one hundred

We want to remove the lines containing the word “one”

$ sed -i '/one/ d' file.txt

$ cat file.txt

two
three

If your pattern contains an environment variable, you can surround it with double quotes instead to force the shell to expand it.

$ sed -i "/$MYVAR/ d" file.txt

How to correctly use LD_LIBRARY_PATH

Ryan — Wed, 22 Oct 2008 12:32:13 +0000

The LD_LIBRARY_PATH environment variable contains a colon separated list of paths that the linker uses to resolve library dependencies of ELF executables at run-time. These paths will be given priority over the standard library paths /lib and /usr/lib. The standard paths will still be searched, but only after the list of paths in LD_LIBRARY_PATH has been exhausted.

The best way to use LD_LIBRARY_PATH is to set it on the command line or script immediately before executing the program. This way you can keep the new LD_LIBRARY_PATH isolated from the rest of your system.

$ export LD_LIBRARY_PATH="/list/of/library/paths:/another/path"
$ ./program

In general it is not a good practice to have LD_LIBRARY_PATH permanently set in your environment. This could lead to unintended side effects as programs can link to unintended libraries producing strange results or unexpectedly crashing. There is also the possibility introducing potential security threats.

All those warnings aside, if you are using BASH you can set it permanently by placing a line similar to this in your .bashrc in your home directory.

export LIBRARY_PATH="/list/of/library/paths:/another/path"

A common case for setting LD_LIBRARY_PATH is when you have an application that requires dynamic libraries which were not installed in the standard library locations.

You can check if the linker can locate all the required libraries by running the ldd command.

$ ldd ~/myprogram

	librt.so.1 => /lib/librt.so.1 (0x00002b4eca08e000)
	libc.so.6 => /lib/libc.so.6 (0x00002b4eca49f000)
	libpthread.so.0 => /lib/libpthread.so.0 (0x00002b4eca7df000)
	/lib64/ld-linux-x86-64.so.2 (0x00002b4ec9e72000)
	libmylib.so.1 => not found

The linker cannot find libmylib.so.1.

Let’s assume this library exists here “~/myprogdir/lib/libmylib.so.1″. We have to set LD_LIBRARY_PATH to include this path for the application to successfully run.

$ export LD_LIBRARY_PATH="~/myprogdir/lib/:$LD_LIBRARY_PATH"
$ ldd ~/myprogram

	librt.so.1 => /lib/librt.so.1 (0x00002b4eca08e000)
	libc.so.6 => /lib/libc.so.6 (0x00002b4eca49f000)
	libpthread.so.0 => /lib/libpthread.so.0 (0x00002b4eca7df000)
	/lib64/ld-linux-x86-64.so.2 (0x00002b4ec9e72000)
	libmylib.so.1 => ~/myprogdir/lib/libmylib.so.1 (0x00002b4eca9fa000)

The linker has now found all the required libraries.

Customize the BASH PS1 command prompt

Ryan — Sun, 19 Oct 2008 14:45:22 +0000

The PS1 environment controls the appearance of the BASH command line prompt. There are a variety of default prompts but they usually include username, hostname, and working directory. You can easily customize your prompt to display information important to you as well as add color and style formatting.

Here we can see the default prompt is “username@hostname working_directory $”.

ryan@workstation ~ $

You can echo the PS1 environment variable and see the character codes which are used to create the output.

ryan@workstation ~ $ echo $PS1

\u@\h \w \$

Here is a list of all the special character codes you can use to build your PS1 command prompt.

Special Character Codes

\a – an ASCII bell character (07)
\d – the date in “Weekday Month Date” format (e.g., “Tue May 26″)
\D{format} – the format is passed to strftime(3) and the result is inserted into the prompt string; an empty format results in a locale-specific time representation. The braces are required
\e – an ASCII escape character (033)
\h – the hostname up to the first `.’
\H – the hostname
\j – the number of jobs currently managed by the shell
\l – the basename of the shell’s terminal device name
\n – newline
\r – carriage return
\s – the name of the shell, the basename of $0 (the portion following the final slash)
\t – the current time in 24-hour HH:MM:SS format
\T – the current time in 12-hour HH:MM:SS format
\@ – the current time in 12-hour am/pm format
\A – the current time in 24-hour HH:MM format
\u – the username of the current user
\v – the version of bash (e.g., 2.00)
\V – the release of bash, version + patchelvel (e.g., 2.00.0)
\w – the current working directory
\W – the basename of the current working directory
\! – the history number of this command
\# – the command number of this command
\$ – if the effective UID is 0, a #, otherwise a $
\nnn – the character corresponding to the octal number nnn
\\ – a backslash
\[ - begin a sequence of non-printing characters, which could be used to embed a terminal control sequence into the prompt
\] – end a sequence of non-printing characters

Lets try a simple change. Make sure you use double quotes so the escape characters get passed on to the variable and not expanded in the shell.

ryan@workstation ~ $ PS1="My Prompt: $ "

My Prompt: $

As you can see the prompt will immediately change to reflect the changes.

We can add the date to the prompt.

ryan@workstation ~ $ PS1="\d \u@\h \w \$ "

Sun Oct 19 ryan@workstation ~ $

Now lets add some color to our original prompt.

ryan@workstation ~ $ PS1="\[\e[1;32m\]\u@\h\[\e[1;34m\] \w \$\[\e[0m\] "

Let go over this in detail.

A style block will display all text that follows with the style it defines. The block contains 3 elements.

Lets look at the first block.
“\[\e[1;32m\]”

“\[" - begin a sequence of non-printing characters
"\e[1;32m" - the semicolon separated list of style/color codes (in this case bold;green)
"\]” – end a sequence of non-printing characters

The first block will display the following username@hostname text in BOLD green.
“\[\e[1;32m\]\u@\h\[\e[1;34m\] \w \$\[\e[0m\] ”

The second block will display the following working_directory text in BOLD blue.
“\[\e[1;32m\]\u@\h\[\e[1;34m\] \w \$\[\e[0m\] ”

The final block resets the colors.
“\[\e[1;32m\]\u@\h\[\e[1;34m\] \w \$\[\e[0m\] ”

Here is a rundown of all the color and style codes.

Color and Style Codes

Style
0 – default
1 – bold
4 – underline
7 – inverse
9 – strikeout

Foreground Colors
30 – foreground Black
31 – foreground Red
32 – foreground Green
33 – foreground Yellow
34 – foreground Blue
35 – foreground Magenta
36 – foreground Cyan
37 – foreground White

Background Colors
40 – background Black
41 – background Red
42 – background Green
43 – background Yellow
44 – background Blue
45 – background Magenta
46 – background Cyan
47 – background White

You can combine multiple codes to create the exact style you want.
“\[\e[1;4;36;47m\]”
This style block defines bold and underlined cyan text with a white background.

To make these changes permanent place your new prompt string like the one below in your ~/.bashrc file in your home directory.

PS1="\[\e[1;32m\]\u@\h\[\e[1;34m\] \w \$ \[\e[0m\]"

This file is sourced every time you start a new interactive shell.

I/O redirection in BASH

Ryan — Thu, 16 Oct 2008 12:50:37 +0000

One of the best features of the Linux command line is the ability to efficiently direct input and output to and from files and other programs.

Every program begins with 3 open file streams.

stdin (file descriptor 0) – input from the user, usually keyboard
stdout (file descriptor 1) – standard output that is displayed on the screen
stderr (file descriptor 2) – standard error output, also displayed on the screen

We can redirect the stdout from a command to a file. The file is created if it doesn’t exist and is truncated if it does.

$ command > outfile

Any error messages produced by the command will still be sent to the command line. To redirect both stdout and stderr you can do this.

$ command &> outfile

Lets say you want to suppress the error messages from a command and view only the successful output on the command line.

$ command 2> /dev/null

The error messages in this case will be sent to ‘/dev/null‘ which is a special device that discards all data sent to it.

If you want to append stdout to a file. The file is still created if it doesn’t exists.

$ command >> outfile

To append stderr to a file.

$ command 2>> outfile

Its a little more complex to append both stdout and stderr to a file.

$ command >> outfile 2>&1

Here stdout is appended to file and stderr is redirected to stdout.

If you want to redirect stdin from a file to a command.

$ command < inputfile

We can combine redirection of stdin from the inputfile and stdout to the outfile.

$ command < inputfile > outfile

A more complex example where all 3 streams are redirected.

$ command < inputfile > outfile 2>&1

So far I have only showed how to direct file streams from program to a file and vice versa. You can also redirect the output from one command to the input of another. This is one of the most powerful features of Linux, and learning to do it well will make you a better Linux user.

Lets direct the stdout from one command to the stdin of another. Chaining commands togather is done with the pipe character '|'.

$ command1 | command2

Here we "pipe" the output from command1 to command 2.

As you can imagine we can get really creative and combine all of this into a single command line statement.

$ command1 2>~/errorfile | command2 > ~/resultfile 2>> ~/errorfile

Lets go over this.

command1 will truncate and write stderr to 'errorfile' in your home directory
stdout is piped to the stdin of command2
the final results from stdout of command2 are saved in 'resultfile'
command2 stderr will be appended to 'errorfile'

Single versus double quotes in BASH

Ryan — Thu, 16 Oct 2008 03:50:30 +0000

Understanding the difference between double versus single quotes is important when using BASH. Many times you may have seen them being used interchangeably. The basic difference is that variable names will be expanded within double quotes but not within single ones.

This example shows the normal output with no quotes.

$ echo $USER

ryan

As you can see the shell will expand the variable.

In this example we will surround the variable with double quotes.

$ echo "$USER"

ryan

We see the same result as before.

This time with single quotes.

$ echo '$USER'

$USER

The variable name is not expanded.

Here is another interesting case. The single quotes are inside the double quotes.

$ echo "'$USER'"

'ryan'

The variable is expanded and the single quotes are retained.

Here I flip the order with the single quotes on the outside.

$ echo '"$USER"'

"$USER"

The variable is not expanded but the double quotes are retained.