WiredRevolution.com » tr

Convert text files within a directory from Windows to Unix format

Ryan — Wed, 07 Jan 2009 14:14:33 +0000

When a file is saved in Windows and then moved to a Linux system the formatting differences can cause a variety of problems. To effectively use these files you will need to change the format from Windows/DOS to Unix. This conversion occurs by simply removing the Windows carriage return characters.

I have explained how to use the tr command remove these windows carriage returns, but when you have a large amount of files to convert this can become tedious. As a solution to this I have written a BASH script to convert all text files within a directory.

#!/bin/bash

for file in /directory/to/convert/*
do
  if [[ -f $file && `file $file | grep text` ]]
  then
    tr -d '\r' < $file > "$file"_clear
    mv "$file"_clear $file
  fi
done

View process environment details with proc

Ryan — Wed, 10 Dec 2008 12:42:25 +0000

The process environment consists of all the individual environment variables which are passed on to the program by the shell when the program is launched. This environment can be read and changed by the program during its execution, and can affect how a program is linked or how it executes. For all these reasons it may be important at times to see exactly what the environment contains. The proc filesystem provides the interface to make this happen.

The proc pseudo-file system is used as an interface to kernel data structures. It is commonly mounted at /proc and most of the files inside are read-only. Within proc there exists a subdirectory for every running process on the system identified by its process ID (PID).

Inside each process directory is an environ file which contains the current environment for that process.

/proc//environ

This file is limited at this point in time to 4096 characters. Therefore if you have a process with an unusually large environment the data beyond this limit will be truncated.

The environment variable entries are separated by null characters ‘\0‘, which makes it difficult to view the file. The best way to get around this is with the tr command which will replace the the null characters with newline characters ‘\n‘.

$ cat /proc/12345/environ | tr "\000" "\n"

Remove Windows carriage returns with tr

Ryan — Sun, 30 Nov 2008 15:46:17 +0000

The end of a line in a UNIX text file is designated with the newline character. In Windows, a line ends with both newline and carriage return ASCII characters. If a file is saved in Windows and then moved to a Linux system these carriage returns can cause all sorts of problems. If the text file is a BASH script for example it will not run correctly since it doesn’t know how to interpret these characters.

You can verify that a text file has these Windows carriage returns by running the cat command with the ‘-v‘ option which shows the non-printing characters.

$ cat -v inputfile | head

first line^M
second line^M

You can see the carriage return characters, “^M” (Cntl-M).

CRLF = Carriage Return Line Feed

There are various was to remove these carriage returns. You can use the dos2unix command but this is rarely installed by default on a a Linux system. The easiest way then is to use the “tr” command utility which always comes standard.

The command uses the tr command which translates and removes characters. This will remove the carriage return characters.

$ tr -d '\r' < inputfile > outputfile

You can verify they are really gone by running the same cat command again.

$ cat -v outputfile | head

first line
second line

You can also run the file command.

$ file inputfile

inputfile:             ASCII text

Optionally you can now overwrite the original file.

$ mv outputfile inputfile