Monday, 15 August 2011

How to Find Out Which Process Is Listening Upon a Port in Linux


How do I find out running processes were associated with each open port? How do I find out what process has open tcp port 111 or udp port 7000 under Linux?

You can the following programs to find out about port numbers and its associated process:
  1. netstat - a command-line tool that displays network connections, routing tables, and a number of network interface statistics.
  2. fuser - a command line tool to identify processes using files or sockets.
  3. lsof - a command line tool to list open files under Linux / UNIX to report a list of all open files and the processes that opened them.
  4. /proc/$pid/ file system - Under Linux /proc includes a directory for each running process (including kernel processes) at /proc/PID, containing information about that process, notably including the processes name that opened port.
You must run above command as root.

netstat example

Type the following command:
# netstat -tulpnSample outputs:
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.0.1:3306          0.0.0.0:*               LISTEN      1138/mysqld
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      850/portmap
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      1607/apache2
tcp        0      0 0.0.0.0:55091           0.0.0.0:*               LISTEN      910/rpc.statd
tcp        0      0 192.168.122.1:53        0.0.0.0:*               LISTEN      1467/dnsmasq
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      992/sshd
tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN      1565/cupsd
tcp        0      0 0.0.0.0:7000            0.0.0.0:*               LISTEN      3813/transmission
TCP port 3306 was opened by mysqld process having PID # 1138. You can verify this using /proc, enter:
# ls -l /proc/1138/exe
Sample outputs:
lrwxrwxrwx 1 root root 0 2010-10-29 10:20 /proc/1138/exe -> /usr/sbin/mysqld
You can use grep command to filter out information:
# netstat -tulpn | grep :80
Sample outputs:
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      1607/apache2

fuser command

Find out the processes PID that opened tcp port 7000, enter:
# fuser 7000/tcp
Sample outputs:
7000/tcp:             3813
Finally, find out process name associated with PID # 3813, enter:
# ls -l /proc/3813/exe
Sample outputs:
lrwxrwxrwx 1 vivek vivek 0 2010-10-29 11:00 /proc/3813/exe -> /usr/bin/transmission
/usr/bin/transmission is a bittorrent client, enter:
# man transmission
OR
# whatis transmission
Sample outputs:
transmission (1)     - a bittorrent client

Task: Find Out Current Working Directory Of a Process

To find out current working directory of a process called bittorrent or pid 3813, enter:
# ls -l /proc/3813/cwd
Sample outputs:
lrwxrwxrwx 1 vivek vivek 0 2010-10-29 12:04 /proc/3813/cwd -> /home/vivek
OR use pwdx command, enter:
# pwdx 3813
Sample outputs:
3813: /home/vivek

Task: Find Out Owner Of a Process

Use the following command to find out the owner of a process PID called 3813:
# ps aux | grep 3813
OR
# ps aux | grep '[3]813'
Sample outputs:
vivek     3813  1.9  0.3 188372 26628 ?        Sl   10:58   2:27 transmission
OR try the following ps command:
# ps -eo pid,user,group,args,etime,lstart | grep '[3]813'
Sample outputs:
3813 vivek    vivek    transmission                   02:44:05 Fri Oct 29 10:58:40 2010
Another option is /proc/$PID/environ, enter:
# cat /proc/3813/environ
OR
# grep --color -w -a USER /proc/3813/environ
Sample outputs (note --colour option):
Fig.01: grep output
lsof Command Example
Type the command as follows:
lsof -i :portNumber
lsof -i tcp:portNumber
lsof -i udp:portNumber
lsof -i :80
lsof -i :80 | grep LISTEN
Sample outputs:
apache2   1607     root    3u  IPv4   6472      0t0  TCP *:www (LISTEN)
apache2   1616 www-data    3u  IPv4   6472      0t0  TCP *:www (LISTEN)
apache2   1617 www-data    3u  IPv4   6472      0t0  TCP *:www (LISTEN)
apache2   1618 www-data    3u  IPv4   6472      0t0  TCP *:www (LISTEN)
apache2   1619 www-data    3u  IPv4   6472      0t0  TCP *:www (LISTEN)
apache2   1620 www-data    3u  IPv4   6472      0t0  TCP *:www (LISTEN)
Now, you get more information about pid # 1607 or 1616 and so on:
# ps aux | grep '[1]616'
Sample outputs:
www-data 1616 0.0 0.0 35816 3880 ? S 10:20 0:00 /usr/sbin/apache2 -k start
I recommend the following command to grab info about pid # 1616:
# ps -eo pid,user,group,args,etime,lstart | grep '[1]616'
Sample outputs:
1616 www-data www-data /usr/sbin/apache2 -k start     03:16:22 Fri Oct 29 10:20:17 2010
Where,
  • 1616 : PID
  • www-date : User name (owner - EUID)
  • www-date : Group name (group - EGID)
  • /usr/sbin/apache2 -k start : The command name and its args
  • 03:16:22 : Elapsed time since the process was started, in the form [[dd-]hh:]mm:ss.
  • Fri Oct 29 10:20:17 2010 : Time the command started.

Help: I Discover an Open Port Which I Don't Recognize At All

The file /etc/services is used to map port numbers and protocols to service names. Try matching port numbers:
$ grep port /etc/services
$ grep 443 /etc/services

Sample outputs:
https		443/tcp				# http protocol over TLS/SSL
https		443/udp

Check For rootkit

I strongly recommend that you find out which processes are really running, especially servers connected to the high speed Internet access. You can look for rootkit which is a program designed to take fundamental control (in Linux / UNIX terms "root" access, in Windows terms "Administrator" access) of a computer system, without authorization by the system's owners and legitimate managers. See how to detecting / checking rootkits under Linux.

Keep an Eye On Your Bandwidth Graphs

Usually, rooted servers are used to send a large number of spam or malware or DoS style attacks on other computers.

See also:

See the following man pages for more information:
$ man ps
$ man grep
$ man lsof
$ man netstat
$ man fuser

Find All File Descriptors Used By a Process


How can I find all the file descriptors used by a process such as httpd (Apache web server) or mysqld (MySQL Database server)?

You can use /proc file system or the lsof command to find all the file descriptors used by a process.

/proc File System Example

First, find out process ID using the ps command, enter:
# ps aux | grep processName
# ps aux | grep mysqld

Sample outputs:
root      3632  0.0  0.0  65944  1296 ?        S    Jun05   0:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --socket=/var/lib/mysql/mysql.sock --log-error=/var/log/mysqld.log --pid-file=/var/run/mysqld/mysqld.pid --user=mysql
mysql     3679  9.0  8.9 1393476 733756 ?      Sl   Jun05 3448:12 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-external-locking --socket=/var/lib/mysql/mysql.sock
root     44723  0.0  0.0  61188   820 pts/0    S+   17:48   0:00 grep mysqld
You can also use the pidof command as follows to find out the process id’s (pids) of the named programs:
# pidof processName
# pidof mysqld

Sample outputs:
3679
Note down the PID # 3679.

/proc/$pid/fd Directory Example

Type the following command:
# ls /proc/3679/fd
# ls -l /proc/3679/fd

Sample outputs:
total 0
lr-x------ 1 root root 64 Jul  1 13:09 0 -> /dev/null
l-wx------ 1 root root 64 Jul  1 13:09 1 -> /var/log/mysqld.log
lrwx------ 1 root root 64 Jul  1 13:09 10 -> socket:[14667]
lrwx------ 1 root root 64 Jul  1 13:09 100 -> /var/lib/mysql/blogdb/wp_term_relationships.MYI
lrwx------ 1 root root 64 Jul  1 13:09 101 -> /var/lib/mysql/blogdb/wp_postmeta.MYI
lrwx------ 1 root root 64 Jul  1 13:09 102 -> /var/lib/mysql/blogdb/wp_terms.MYI
lrwx------ 1 root root 64 Jul  1 13:09 103 -> /var/lib/mysql/orders/discussionread.MYD
lrwx------ 1 root root 64 Jul  1 13:09 104 -> /var/lib/mysql/orders/comments.MYI
lrwx------ 1 root root 64 Jul  1 13:09 105 -> /var/lib/mysql/orders/useractivation.MYI
lrwx------ 1 root root 64 Jul  1 13:09 106 -> /var/lib/mysql/orders/postparsed.MYI
lrwx------ 1 root root 64 Jul  1 13:09 107 -> /var/lib/mysql/orders/sigparsed.MYI
lrwx------ 1 root root 64 Jul  1 13:09 108 -> /var/lib/mysql/support/forum.MYI
......
.......
...
(output truncated)

lsof command Example

Type the command as follows:
# lsof -a -p {PID-HERE}
# lsof -a -p 3679

Sample outputs:
COMMAND  PID  USER   FD      TYPE             DEVICE     SIZE     NODE NAME
mysqld  3679 mysql  cwd       DIR                8,2     4096  7667720 /var/lib/mysql
mysqld  3679 mysql  rtd       DIR                8,2     4096        2 /
mysqld  3679 mysql  txt       REG                8,2  7721832 15185707 /usr/libexec/mysqld
mysqld  3679 mysql  mem       REG                8,2          13074466 /lib64/ld-2.5.so (path inode=13074445)
mysqld  3679 mysql  mem       REG                8,2          13074602 /lib64/libc-2.5.so (path inode=13074454)
mysqld  3679 mysql  mem       REG                8,2          13074603 /lib64/libdl-2.5.so (path inode=13074461)
mysqld  3679 mysql  mem       REG                8,2          13074608 /lib64/libm-2.5.so (path inode=13074464)
mysqld  3679 mysql  mem       REG                8,2          13074612 /lib64/libpthread-2.5.so (path inode=13074491)
mysqld  3679 mysql  mem       REG                8,2    85928 15193367 /usr/lib64/libz.so.1.2.3
mysqld  3679 mysql  mem       REG                8,2   247496 13074620 /lib64/libsepol.so.1
mysqld  3679 mysql  mem       REG                8,2    95464 13074623 /lib64/libselinux.so.1
mysqld  3679 mysql  mem       REG                8,2          13074615 /lib64/librt-2.5.so (path inode=13074500)
mysqld  3679 mysql  mem       REG                8,2          13074618 /lib64/libnsl-2.5.so (path inode=13074469)
mysqld  3679 mysql  mem       REG                8,2          13074633 /lib64/libcrypt-2.5.so (path inode=13074459)
mysqld  3679 mysql  mem       REG                8,2  1366272 13074653 /lib64/libcrypto.so.0.9.8e
mysqld  3679 mysql  mem       REG                8,2          13074655 /lib64/libresolv-2.5.so (path inode=13074496)
mysqld  3679 mysql  mem       REG                8,2    58400 13074610 /lib64/libgcc_s-4.1.2-20080825.so.1
mysqld  3679 mysql  mem       REG                8,2    10000 13074666 /lib64/libcom_err.so.2.1
mysqld  3679 mysql  mem       REG                8,2          15193399 /usr/lib64/libk5crypto.so.3.1 (path inode=15186140)
mysqld  3679 mysql  mem       REG                8,2     9728 13074658 /lib64/libkeyutils-1.2.so
mysqld  3679 mysql  mem       REG                8,2          15193402 /usr/lib64/libgssapi_krb5.so.2.2 (path inode=15186013)
mysqld  3679 mysql  mem       REG                8,2          15193397 /usr/lib64/libkrb5support.so.0.1 (path inode=15186147)
mysqld  3679 mysql  mem       REG                8,2          15193400 /usr/lib64/libkrb5.so.3.3 (path inode=15184571)
mysqld  3679 mysql  mem       REG                8,2   976312 15180443 /usr/lib64/libstdc++.so.6.0.8
mysqld  3679 mysql  mem       REG                8,2   315032 13074667 /lib64/libssl.so.0.9.8e
mysqld  3679 mysql  mem       REG                8,2          13074479 /lib64/libnss_files-2.5.so (path inode=13074476)
mysqld  3679 mysql    0r      CHR                1,3              4102 /dev/null
.....
..
...
mysqld  3679 mysql  234u      REG                8,2        0  7700867 /var/lib/mysql/order/subscribediscussion.MYD
mysqld  3679 mysql  238u      REG                8,2     2048  7700637 /var/lib/mysql/email/mailqueue.MYI
mysqld  3679 mysql  239u      REG                8,2    79060  7700593 /var/lib/mysql/email/mailqueue.MYD
mysqld  3679 mysql  340u      REG                8,2 17690624  7766217 /var/lib/mysql/blogdb/wp_posts.MYI
To see statistics on file descriptors used, enter:
# cat /proc/3679/net/sockstat
Sample outputs:
sockets: used 183
TCP: inuse 5 orphan 0 tw 0 alloc 9 mem 2
UDP: inuse 9 mem 1
UDPLITE: inuse 0
RAW: inuse 0
FRAG: inuse 0 memory 0
Please note that on older kernel such as those shipped with RHEL 5 or Suse 9 or Debian 4 or 5 use system wild statistics file descriptors file called /proc/net/sockstat:
# cat /proc/net/sockstat
Sample outputs:
sockets: used 455
TCP: inuse 146 orphan 0 tw 673 alloc 146 mem 9
UDP: inuse 14 mem 0
RAW: inuse 0
FRAG: inuse 0 memory 0

yum command: Update / Install Packages ( Redhat Enterprise / CentOS Linux Version 5.x)


How do I use yum command to update and patch my Red hat Enterprise Linux / CentOS Linux version 5.x server via RHN / Internet? Can I use up2date command under RHEL 5?

up2date command was part of RHEL v4.x or older version. You need to use yum command to update and patch the system using RHN or Internet. Use yum command to install critical and non-critical security updates as well as binary packages. Login as the root user to install and update the system.

Task: Register my system with RHN

To register your system with RHN type the following command and just follow on screen instructions (CentOS user skip to next step):
# rhn_register
WARNING! These examples only works with RHEL / CentOS Linux version 5.x or above. For RHEL 4.x and older version use up2date command.

Task: Display list of updated software (security fix)

Type the following command at shell prompt:
# yum list updates

Task: Patch up system by applying all updates

To download and install all updates type the following command:
# yum update

Task: List all installed packages

List all installed packages, enter:
# rpm -qa
# yum list installed

Find out if httpd package installed or not, enter:
# rpm -qa | grep httpd*
# yum list installed httpd

Task: Check for and update specified packages

# yum update {package-name-1}
To check for and update httpd package, enter:
# yum update httpd

Task: Search for packages by name

Search httpd and all matching perl packages, enter:
# yum list {package-name}
# yum list {regex}
# yum list httpd
# yum list perl*

Sample output:
Loading "installonlyn" plugin
Loading "security" plugin
Setting up repositories
Reading repository metadata in from local files
Installed Packages
perl.i386                                4:5.8.8-10.el5_0.2     installed
perl-Archive-Tar.noarch                  1.30-1.fc6             installed
perl-BSD-Resource.i386                   1.28-1.fc6.1           installed
perl-Compress-Zlib.i386                  1.42-1.fc6             installed
perl-DBD-MySQL.i386                      3.0007-1.fc6           installed
perl-DBI.i386                            1.52-1.fc6             installed
perl-Digest-HMAC.noarch                  1.01-15                installed
perl-Digest-SHA1.i386                    2.11-1.2.1             installed
perl-HTML-Parser.i386                    3.55-1.fc6             installed
.....
.......
..
perl-libxml-perl.noarch                  0.08-1.2.1             base
perl-suidperl.i386                       4:5.8.8-10.el5_0.2     updates 

Task: Install the specified packages [ RPM(s) ]

Install package called httpd:
# yum install {package-name-1} {package-name-2}
# yum install httpd

Task: Remove / Uninstall the specified packages [ RPM(s) ]

Remove package called httpd, enter:
# yum remove {package-name-1} {package-name-2}
# yum remove httpd

Task: Display the list of available packages

# yum list all

Task: Display list of group software

Type the following command:
# yum grouplist
Output:
Installed Groups:
   Engineering and Scientific
   MySQL Database
   Editors
   System Tools
   Text-based Internet
   Legacy Network Server
   DNS Name Server
   Dialup Networking Support
   FTP Server
   Network Servers
   Legacy Software Development
   Legacy Software Support
   Development Libraries
   Graphics
   Web Server
   Ruby
   Printing Support
   Mail Server
   Server Configuration Tools
   PostgreSQL Database
Available Groups:
   Office/Productivity
   Administration Tools
   Beagle
   Development Tools
   GNOME Software Development
   X Software Development
   Virtualization
   GNOME Desktop Environment
   Authoring and Publishing
   Mono
   Games and Entertainment
   XFCE-4.4
   Tomboy
   Java
   Java Development
   Emacs
   X Window System
   Windows File Server
   KDE Software Development
   KDE (K Desktop Environment)
   Horde
   Sound and Video
   FreeNX and NX
   News Server
   Yum Utilities
   Graphical Internet
Done

Task: Install all the default packages by group

Install all 'Development Tools' group packages, enter:
# yum groupinstall "Development Tools"

Task: Update all the default packages by group

Update all 'Development Tools' group packages, enter:
# yum groupupdate "Development Tools"

Task: Remove all packages in a group

Remove all 'Development Tools' group packages, enter:
# yum groupremove "Development Tools"

Task: Install particular architecture package

If you are using 64 bit RHEL version it is possible to install 32 packages:
# yum install {package-name}.{architecture}
# yum install mysql.i386

Task: Display packages not installed via official RHN subscribed repos

Show all packages not available via subscribed channels or repositories i.e show packages installed via other repos:
# yum list extras
Sample output:
Loading "installonlyn" plugin
Loading "security" plugin
Setting up repositories
Reading repository metadata in from local files
Extra Packages
DenyHosts.noarch                         2.6-python2.4          installed
VMwareTools.i386                         6532-44356             installed
john.i386                                1.7.0.2-3.el5.rf       installed
kernel.i686                              2.6.18-8.1.15.el5      installed
kernel-devel.i686                        2.6.18-8.1.15.el5      installed
lighttpd.i386                            1.4.18-1.el5.rf        installed
lighttpd-fastcgi.i386                    1.4.18-1.el5.rf        installed
psad.i386                                2.1-1                  installed
rssh.i386                                2.3.2-1.2.el5.rf       installed

Task: Display what package provides the file

You can easily find out what RPM package provides the file. For example find out what provides the /etc/passwd file:
# yum whatprovides /etc/passwd
Sample output:
Loading "installonlyn" plugin
Loading "security" plugin
Setting up repositories
Reading repository metadata in from local files

setup.noarch                             2.5.58-1.el5           base
Matched from:
/etc/passwd

setup.noarch                             2.5.58-1.el5           installed
Matched from:
/etc/passwd
You can use same command to list packages that satisfy dependencies:
# yum whatprovides {dependency-1} {dependency-2}
Refer yum command man page for more information:
# man yum

yum Command Hangs on Running Transaction Test Message Error and Solution


When I run 'yum install package' or 'yum update' commands the yum command works normally until Running Transaction Test and hangs until the yum process is killed manually using the kill command. How do I fix this problem under CentOS Linux 5.x AMD64 running on HP server?
This error indicates that other duplicate yum processes running on the system or older yum process causing problem. To fix this problem either reboot the server or type the following commands:
# /bin/rm /var/lib/rpm/__db.*
# cd /var/cache/yum
# /bin/rm -rf *
# yum clean all
# yum update

If you still get the same problem just reboot the server.

Find Out: Dell Server / Desktop / Laptop Service Tag Command -Linux


 You need to find out the service tag of my Dell server or computer. How do  find out my nique five- to seven- digit alphanumeric (letter and number) service tag code without looking on the back of the system or opening door of bezel?

You need to use the dmidecode command. It is a tool for dumping a computer's DMI (some say SMBIOS) table contents in a human-readable format. This tool can display serial number or dell service tag without looking on the back of the server. Type the command as follows as root user:
# dmidecode | grep --color -i serial
OR
# dmidecode -t 1
Sample outputs:
# dmidecode 2.9
SMBIOS 2.6 present.

Handle 0x0100, DMI type 1, 27 bytes
System Information
	Manufacturer: Dell Inc.
	Product Name: Precision M6500
	Version: Not Specified
	Serial Number: 12AB69D
	UUID: 44454A4C-3100-109C-1032-XXXXXXXXXX
	Wake-up Type: Power Switch
	SKU Number: Not Specified
	Family:  Precision Workstation
You can use Dell service tag # 12AB69D for downloading drivers or for other support related issues on http://dell.com/ itself.
Browser Name:
Browser Version:
Browser Code Name:
User-Agent: