When a file is saved in Windows and then moved to a Linux system the formatting differences can cause a variety of problems. To effectively use these files you will need to change the format from Windows/DOS to Unix. This conversion occurs by simply removing the Windows carriage return characters.

I have explained how to use the tr command remove these windows carriage returns, but when you have a large amount of files to convert this can become tedious. As a solution to this I have written a BASH script to convert all text files within a directory.

#!/bin/bash

for file in /directory/to/convert/*
do
  if [[ -f $file && `file $file | grep text` ]]
  then
    tr -d '\r' < $file > "$file"_clear
    mv "$file"_clear $file
  fi
done

No related posts.

The end of a line in a UNIX text file is designated with the newline character. In Windows, a line ends with both newline and carriage return ASCII characters. If a file is saved in Windows and then moved to a Linux system these carriage returns can cause all sorts of problems. If the text file is a BASH script for example it will not run correctly since it doesn’t know how to interpret these characters.

You can verify that a text file has these Windows carriage returns by running the cat command with the ‘-v‘ option which shows the non-printing characters.

$ cat -v inputfile | head

first line^M
second line^M

You can see the carriage return characters, “^M” (Cntl-M).

CRLF = Carriage Return Line Feed

There are various was to remove these carriage returns. You can use the dos2unix command but this is rarely installed by default on a a Linux system. The easiest way then is to use the “tr” command utility which always comes standard.

The command uses the tr command which translates and removes characters. This will remove the carriage return characters.

$ tr -d '\r' < inputfile > outputfile

You can verify they are really gone by running the same cat command again.

$ cat -v outputfile | head

first line
second line

You can also run the file command.

$ file inputfile

inputfile:             ASCII text

Optionally you can now overwrite the original file.

$ mv outputfile inputfile

No related posts.