WiredRevolution.com » system administration

Install Linux development man pages in Ubuntu

Ryan — Wed, 13 May 2009 14:53:15 +0000

By default Ubuntu doesn’t install the manual pages used for Linux software development which is an invaluable reference for any system developer. These man pages describe the Linux programming interface including Linux system calls as well as system library calls.

To install these man pages simply install the manpages-dev package.

$ sudo apt-get install manpages-dev

You can also install the POSIX specific interfaces by installing the manpages-posix-dev package.

$ sudo apt-get install manpages-posix-dev

Setup user login restrictions with SSH

Ryan — Wed, 04 Feb 2009 15:52:57 +0000

At various times it is necessary to restrict the users which can access a certain host. If your network relies on SSH it is as simple as changing an option in the sshd_config configuration file. You will of course need root access to make the necessary changes to this file and eventually reset the SSH daemon.

This configuration file is usually located here.
/etc/ssh/sshd_config

Open the file as root in order to make changes.

$ sudo vim /etc/ssh/sshd_config

You need to set the AllowUsers keyword followed by the users you want to have access to the machine.

AllowUsers	ryan joe

If you want to do something more complex here is the output from the man page:

AllowUsers
This keyword can be followed by a list of user name patterns, separated by spaces. If specified, login is allowed only for user names that match one of the patterns. ‘*’ and ‘?’ can be used as wildcards in the patterns. Only user names are valid; a numerical user ID is not recognized. By default, login is allowed for all users. If the pattern takes the form USER@HOST then USER and HOST are separately checked, restricting logins to particular users from particular hosts.

Another helpful to set the PermitRootLogin to ‘no’ so that the root account is inaccessible.

PermitRootLogin  no

When these settings have been changed go ahead and restart the SSH daemon.

$ sudo /etc/init.d/sshd restart

There are of course ways around this if other users have access to sudo or the root account. But for the most part it is a good way to restrict user access.

Skip network filesystems when searching with find

Ryan — Mon, 19 Jan 2009 16:45:06 +0000

When you run the find command on the root directory of a system you may want to exclude all network filesystems and confine your search to only your local machine. The benefits of this are obvious as it will save you a great amount of time especially if the mounted filesystem is very large.

Luckily the find command provides this ability with the ‘-xdev‘ or ‘-mount‘ options.

For example:

$ find / -xdev -name myfile.txt

This command will recursively search for myfile.txt starting in your root directory and skip all externally mounted filesystems.

View process environment details with proc

Ryan — Wed, 10 Dec 2008 12:42:25 +0000

The process environment consists of all the individual environment variables which are passed on to the program by the shell when the program is launched. This environment can be read and changed by the program during its execution, and can affect how a program is linked or how it executes. For all these reasons it may be important at times to see exactly what the environment contains. The proc filesystem provides the interface to make this happen.

The proc pseudo-file system is used as an interface to kernel data structures. It is commonly mounted at /proc and most of the files inside are read-only. Within proc there exists a subdirectory for every running process on the system identified by its process ID (PID).

Inside each process directory is an environ file which contains the current environment for that process.

/proc//environ

This file is limited at this point in time to 4096 characters. Therefore if you have a process with an unusually large environment the data beyond this limit will be truncated.

The environment variable entries are separated by null characters ‘\0‘, which makes it difficult to view the file. The best way to get around this is with the tr command which will replace the the null characters with newline characters ‘\n‘.

$ cat /proc/12345/environ | tr "\000" "\n"

View system information with /proc/cpuinfo and /proc/meminfo

Ryan — Tue, 09 Dec 2008 12:36:26 +0000

There are 2 files available on a Linux system which will give a user a wealth of information about a system. These files, /proc/cpuinfo and /proc/meminfo are written by the Linux kernel and tell you important details about your cpu and memory.

Both files are read-only and can be viewed with any text editor. This information may be overkill for most users but there are a variety of important fields to look for.

The cpuinfo file provides important stats including model name, cpu cores, cpu MHz, cache size, and flags which show what instruction sets are available on the processor. Systems with multiple processors or multiple cores will have separate entries for each.

Here for example is the output of my /proc/cpuinfo file:

$ cat /proc/cpuinfo

processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 23
model name	: Intel(R) Core(TM)2 Duo CPU     T8300  @ 2.40GHz
stepping	: 6
cpu MHz	: 800.000
cache size	: 3072 KB
physical id	: 0
siblings	: 2
core id	: 0
cpu cores	: 2
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 lahf_lm ida
bogomips	: 4787.93
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 23
model name	: Intel(R) Core(TM)2 Duo CPU     T8300  @ 2.40GHz
stepping	: 6
cpu MHz	: 800.000
cache size	: 3072 KB
physical id	: 0
siblings	: 2
core id	: 1
cpu cores	: 2
apicid		: 1
initial apicid	: 1
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 lahf_lm ida
bogomips	: 4787.96
clflush size	: 64
cache_alignment	: 64
address sizes	: 36 bits physical, 48 bits virtual
power management:

The important fields to look for in the meminfo file include MemTotal, MemFree, and SwapTotal.

The output of the /proc/meminfo file :

$ cat /proc/meminfo

MemTotal:      4054116 kB
MemFree:       3187648 kB
Buffers:         19852 kB
Cached:         288692 kB
SwapCached:          0 kB
Active:         527556 kB
Inactive:       196060 kB
SwapTotal:           0 kB
SwapFree:            0 kB
Dirty:             208 kB
Writeback:           0 kB
AnonPages:      415096 kB
Mapped:         103684 kB
Slab:            50592 kB
SReclaimable:    23924 kB
SUnreclaim:      26668 kB
PageTables:      15756 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
WritebackTmp:        0 kB
CommitLimit:   2027056 kB
Committed_AS:   870676 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    115908 kB
VmallocChunk: 34359621627 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
HugePages_Surp:      0
Hugepagesize:     2048 kB
DirectMap4k:     86472 kB
DirectMap2M:   4106240 kB

How to X session forwarding over SSH

Ryan — Thu, 04 Dec 2008 14:05:41 +0000

SSH allows secure (encrypted and authenticated) connections between two hosts. These connections include terminal sessions, file transfers, TCP port forwarding, as well as X window forwarding which I will be covering here. X forwarding is a form of tunneling that allows you to run a GUI application on a remote machine but let you view and interact with it on your local machine.

To try this out you will need both X and SSH installed on your local and remote machines. Make sure that you are able to log into the the remote machine over SSH before you continue.

Simple SSH command, ensure that this works before continuing. You may have to enter a password for the user before it will allow you access.

$ ssh user@remotehost

The next step is to add the ‘-X‘ option. This will turn on X forwarding and allow you to remotely run X programs. In this case we will run xclock.

$ ssh -X user@remotehost xclock

You should see the xclock window appear on your screen. You can interact with it like any other local application window. Close it when you are done.

If you have a slower connection you can turn on compression by adding the ‘-C‘ option to the command above. This will compress all data communications with the gzip algorithm.

$ ssh -C -X user@remotehost xclock

If you are experiencing any problems turn on verbose output with the ‘-v‘ option. This will give you a lot more output and tell you what is going on underneath.

$ ssh -v -X user@remotehost xclock

If you are still having issues look in the ssh configuration file here/etc/ssh/ssh_config, and make sure that you don’t have X forwarding settings disabled.

Firefox Keyboard Shortcuts

Ryan — Sat, 29 Nov 2008 03:49:05 +0000

This is the list of the most useful Firefox keyboard shortcuts. It is amazing how much more efficient you can be when your not always reaching for your mouse.

Controlling Tabs

New Tab – Ctrl+T
Close Tab – Ctrl+W or Ctrl+F4
Next Tab – Ctrl+Tab or Ctrl+PageDown
Previous Tab – Ctrl+Shft+Tab or Ctrl+PageUp
Select Tab [1 to 9] – Ctrl+[1 to 9]

Searching Text

Find in This Page – Ctrl+F
Find Again – Ctrl+G or F3
Find Previous – Ctrl+Shft+G or Shft+F3

Other Commands

Full Screen – F11
Reload – F5 or Ctrl+R
Reload (override cache) – Ctrl+F5 or Ctrl+Shft+R
Select All – Ctrl+A

Install the GRUB boot loader to the MBR

Ryan — Sat, 29 Nov 2008 02:58:03 +0000

You might need to reinstall the boot loader to the master boot record (MBR) if it becomes corrupted or erased. This can occur for example if Windows is installed on another partition which overwrites the MBR.

There are two ways to reinstall grub, automatically using grub-install, and manually through grub itself. grub-install is a script which calls grub underneath and in most cases it is your best solution. Sometimes the manual approach is better if you have a complex setup or the first option fails.

The GRUB naming scheme for hard drives and partitions is a bit confusing at first. Hard drive and partition numbers are ordererd with the count starting at zero. They are ordered based on the boot sequence in BIOS, so different types (IDE, SCSI, or RAID) my be intermixed.

For example a system with 2 PATA disks and 1 SATA disk might have the following drive mapping.
/dev/hda = (hd0) = drive 1
/dev/hda1 = (hd0,0) = drive 1, partition 1
/dev/hdb2 = (hd1,1) = drive 2, partition 2
/dev/sda1 = (hd2,0) = drive 3, partition 1

These steps require root privileges so switch to root or use sudo.

Install automatically with grub-install. The first argument is the hard drive where the boot loader will installed onto the MBR. It will use your current root directory partition for the rest of the installation. Change “/dev/sda” to the correct device for your system.

# grub-install /dev/sda

If you have a separate boot partition use the –root-directory option to install GRUB on that device instead of your root partition.

# grub-install --root-directory=/boot /dev/sda

The second method will allow you to install the boot loader manually with grub. Start by launching the GRUB command shell.

# grub

If your not sure of the device to specify for the installation, you can ask GRUB to tell you. Run this command and GRUB will attempt to find your GRUB installation on all available device partitions on the system. If found GRUB will output all the partitions containing GRUB installations in GRUB naming format. You can select the correct device and use in the next step.

grub> find /boot/grub/menu.lst

 (hd0,0)

Set your root partition which contains the boot directory as well as the GRUB installation. You can use the output from the step above.

grub> root (hd0,0)

Install GRUB to the MBR of your first hard drive device.

grub> setup (hd0)

Exit Grub.

grub> quit

Reboot, you should hopefully see the menu on start up.

# reboot

Free ext3 reserved blocks with tune2fs

Ryan — Sun, 23 Nov 2008 16:02:03 +0000

As a security measure the ext3 file system reserves 5% of device space for administrative processes. This protects the system by allowing root processes to continue using the disk if a user process runs wild and fills it up. With today’s larger disk capacities, 5% equates into gigabytes of arguably wasted space. Thankfully with the tune2fs command you can reduce this percentage and free most if not all of the reserved space.

The tune2fs command utility operates exclusively on ext2/ext3 file systems.

To run these commands you must run the command as root or use sudo.

You can run tune2fs on the ext3 partition with the ‘-l‘ option to show you all the filesystem details. The important information to focus on are the “Reserved block count” and “Block size” lines. Multiply these lines together to see how many bytes are currently reserved on the filesystem.

$ tune2fs -l /dev/sda1

...
Reserved block count:     1929908
...
Block size:               4096
...

That’s more than 7GB of space that is reserved.

Run the df command to see the current used space on the device before you make any changes.

$ df -h

Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1             145G  3.1G  135G   3% /

Run the tune2fs command to change your reserved block percentage. The ‘-m‘ option sets the new reserved percentage.

In this example I am setting the percentage to 0 and completely removing the reserved space. This will effectively disable the security feature but free the most space. You may choose to reduce the reserved percentage instead in order to preserve the security benefit while still freeing some space.

$ sudo tune2fs -m 0 /dev/sda1

You can confirm the changes have taken effect by viewing the filesystem details.

$ tune2fs -l /dev/sda1

...
Reserved block count:     0
...

You can directly see the changes by looking at your system disk space usage. We now have an additional 7GB available.

$ df -h

Filesystem            Size  Used Avail Use% Mounted on
/dev/sda1             145G  3.1G  142G   3% /

Change MIME settings in Firefox 3

Ryan — Thu, 20 Nov 2008 12:46:54 +0000

In Firefox 3 it is easy to edit the MIME settings. These settings are used to control what program is used by default to open each file type you encounter while browsing.

To edit these settings in Firefox 3 navigate these menus:

Edit -> Preferences -> Applications

For each file type on the left you can designate a program to open that file type on the right. You can also avoid opening a specific file type and choose to save it to disk by default instead.

Creating a Windows Boot Disk with a Linux Machine

Ryan — Mon, 17 Nov 2008 12:36:27 +0000

The standard way to create a Windows boot disk is to have access to a Windows system. If you have access to a Linux system instead, and know a couple tricks, you can create boot disk as well.

The first thing to do is find a website that has the boot disks available for download. There are plenty of these out there so take your pick. The only thing you must specifically do is make sure that you download the disk image and not a self extracting executable file which must be run under Windows.

If the image is zipped unzip it.

$ unzip cdboot1.zip

Archive:  cdboot1.zip
  inflating: CDBOOT1.IMG

Check that the disk image is the correct size.

$ ls -lh CDBOOT1.IMG

-r--r--r-- 1 ryan ryan 1.5M 1999-12-07 12:00 CDBOOT1.IMG

Finally, you will use the dd command to copy the disk image to a blank floppy disk.

$ dd if=CDBOOT1.IMG of=/dev/fd0

Find your MAC address with ifconfig

Ryan — Thu, 30 Oct 2008 11:06:53 +0000

You can find the MAC address of your system at the command line by using the ifconfig command. The ifconfig command will show you information on all your network interfaces. If no arguments are given ifconfig displays the status of the currently active interfaces.

You have to run this command as root or use sudo.

# ifconfig

wlan0     Link encap:Ethernet  HWaddr a1:b2:c3:d4:e5:f6  
          inet addr:192.168.2.11  Bcast:192.168.2.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1492  Metric:1
          RX packets:4007 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4246 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:3292271 (3.1 MiB)  TX bytes:994922 (971.6 KiB)

Your MAC address is displayed in the HWaddr field.

Find your IP address with ifconfig

Ryan — Thu, 30 Oct 2008 10:48:07 +0000

You can find the IP address of your system at the command line by using the ifconfig command. The ifconfig command will show you information on all your network interfaces. If no arguments are given ifconfig displays the status of the currently active interfaces.

You have to run this command as root or use sudo.

# ifconfig

wlan0     Link encap:Ethernet  HWaddr a1:b2:c3:d4:e5:f6
          inet addr:192.168.2.11  Bcast:192.168.2.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1492  Metric:1
          RX packets:4007 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4246 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:3292271 (3.1 MiB)  TX bytes:994922 (971.6 KiB)

Your IP address is displayed in the inet addr field.

Disable ext3 boot-time check with tune2fs

Ryan — Sun, 26 Oct 2008 19:09:18 +0000

The ext3 file system forces an fsck once it has been mounted a certain number of times. By default this maximum mount count is usually set between 20-30. On many systems such as laptops which can be rebooted quite often this can quickly become a problem. To turn off this checking you can use the tune2fs command.

The tune2fs command utility operates exclusively on ext2/ext3 file systems.

To run these commands you must run the command as root or use sudo. You must also make sure that your filesystem is unmounted before making any changes. If you are doing this on your root partition the best solution is to use a LiveCD.

You can run tune2fs on the ext3 partition with the ‘-l‘ option to view what your current and maximum mount count is set to currently.

tune2fs -l /dev/sda1

...
Mount count:              2
Maximum mount count:      25
...

To turn off this check set the maximum count to 0 with the ‘-c‘ option.

# tune2fs -c 0 /dev/sda1

If you do not want to completely disable the file system checking, you can also increase the maximum count.

# tune2fs -c 100 /dev/sda1

How to correctly use LD_LIBRARY_PATH

Ryan — Wed, 22 Oct 2008 12:32:13 +0000

The LD_LIBRARY_PATH environment variable contains a colon separated list of paths that the linker uses to resolve library dependencies of ELF executables at run-time. These paths will be given priority over the standard library paths /lib and /usr/lib. The standard paths will still be searched, but only after the list of paths in LD_LIBRARY_PATH has been exhausted.

The best way to use LD_LIBRARY_PATH is to set it on the command line or script immediately before executing the program. This way you can keep the new LD_LIBRARY_PATH isolated from the rest of your system.

$ export LD_LIBRARY_PATH="/list/of/library/paths:/another/path"
$ ./program

In general it is not a good practice to have LD_LIBRARY_PATH permanently set in your environment. This could lead to unintended side effects as programs can link to unintended libraries producing strange results or unexpectedly crashing. There is also the possibility introducing potential security threats.

All those warnings aside, if you are using BASH you can set it permanently by placing a line similar to this in your .bashrc in your home directory.

export LIBRARY_PATH="/list/of/library/paths:/another/path"

A common case for setting LD_LIBRARY_PATH is when you have an application that requires dynamic libraries which were not installed in the standard library locations.

You can check if the linker can locate all the required libraries by running the ldd command.

$ ldd ~/myprogram

	librt.so.1 => /lib/librt.so.1 (0x00002b4eca08e000)
	libc.so.6 => /lib/libc.so.6 (0x00002b4eca49f000)
	libpthread.so.0 => /lib/libpthread.so.0 (0x00002b4eca7df000)
	/lib64/ld-linux-x86-64.so.2 (0x00002b4ec9e72000)
	libmylib.so.1 => not found

The linker cannot find libmylib.so.1.

Let’s assume this library exists here “~/myprogdir/lib/libmylib.so.1″. We have to set LD_LIBRARY_PATH to include this path for the application to successfully run.

$ export LD_LIBRARY_PATH="~/myprogdir/lib/:$LD_LIBRARY_PATH"
$ ldd ~/myprogram

	librt.so.1 => /lib/librt.so.1 (0x00002b4eca08e000)
	libc.so.6 => /lib/libc.so.6 (0x00002b4eca49f000)
	libpthread.so.0 => /lib/libpthread.so.0 (0x00002b4eca7df000)
	/lib64/ld-linux-x86-64.so.2 (0x00002b4ec9e72000)
	libmylib.so.1 => ~/myprogdir/lib/libmylib.so.1 (0x00002b4eca9fa000)

The linker has now found all the required libraries.

Select threading implementation using LD_ASSUME_KERNEL

Ryan — Mon, 13 Oct 2008 12:37:09 +0000

For backwards compatibility, many Linux distributions support both the older LinuxThreads implementation as well as the newer Native POSIX Thread Library (NPTL). By setting the LD_ASSUME_KERNEL environment variable you can tell the dynamic linker to assume that it is running on top of a particular kernel version. This will override the dynamic linker’s default choice of threading implementation (usually NPTL) and force the use of the older LinuxThreads implementation at run-time.

Many 32-bit systems have 3 separate glibc versions which have support for a minimum OS Application Binary Interface (ABI), designated by a kernel version and offer support for a particular thread implementation.

/lib/tls/libc.so.6

minimum ABI = 2.4.20
Native POSIX Thread Library (NPTL)

/lib/i686/libc.so.6

minimum ABI = 2.4.1
standard LinuxThreads

/lib/libc.so.6

minimum ABI = 2.2.5
early LinuxThreads code which had fixed size threads

It is worth noting that 64-bit systems do not support the older 2.2.5 LinuxThreads implementation.

You can run the following commands to see what thread implementations are supported by each separate glibc library.

$ /lib/tls/libc.so.6 | grep [T|t]hreads

	Native POSIX Threads Library by Ulrich Drepper

$ /lib/i686/libc.so.6 | grep [T|t]hreads

	linuxthreads-0.10 by Xavier Leroy

$ /lib/libc.so.6 | grep [T|t]hreads

	linuxthreads-0.10 by Xavier Leroy

A shared library can tell the dynamic linker which minimum OS ABI version is needed for it to successfully run. At run-time the linker will start at the most recent library path and walk backwards in this order: /lib/tls, /lib/i686, /lib. It will either link to the first library it finds or fail and terminate the program if the minimum ABI is reached before it finds a library.

Setting LD_ASSUME_KERNEL below the minimum threshold ABI version of any dynamic library will cause the linker to skip that library and try to find an older implementation

Set the environment variable by assigning it to a specific ABI/kernel version.

$ export LD_ASSUME_KERNEL=X.Y.Z

You must be careful not to set the the ABI below 2.2.5 in your environment as all programs which require dynamic linking will probably fail to run. Luckily you are still be able to unset or change it back.

It is useless to set LD_ASSUME_KERNEL for most programs as they will run correctly no matter what underlying implementation is used since the API is the same. However there are some programs that may depend on some non-conformant behavior requiring LinuxThreads, which will fail with NPTL.