GlusterFS and rpcbind / portmap – DDoS Reflection Attacks

If you use GlusterFS then you are also using rpcbind. The linux man page provides the following description for rpcbind:

DESCRIPTION
     The rpcbind utility is a server that converts RPC program numbers into universal addresses.  It must be running on the host to be able to make RPC
     calls on a server on that machine.

     When an RPC service is started, it tells rpcbind the address at which it is listening, and the RPC program numbers it is prepared to serve.  When a
     client wishes to make an RPC call to a given program number, it first contacts rpcbind on the server machine to determine the address where RPC
     requests should be sent.

 

GlusterFS (3.6.9) on Ubuntu 14.04 requires rpcbind. By default rpcbind is listening on all interfaces. The linux man page states that this can be changed using the -h option at startup:

     -h      Specify specific IP addresses to bind to for UDP requests.  This option may be specified multiple times and is typically necessary when run‐
             ning on a multi-homed host.  If no -h option is specified, rpcbind will bind to INADDR_ANY, which could lead to problems on a multi-homed host
             due to rpcbind returning a UDP packet from a different IP address than it was sent to.  Note that when specifying IP addresses with -h,
             rpcbind will automatically add 127.0.0.1 and if IPv6 is enabled, ::1 to the list.

 

You can list the interfaces rpcbind is listening on using netstat:

netstat -aunlp | grep rpcbind
udp        0      0 0.0.0.0:111             0.0.0.0:*                           8861/rpcbind
udp        0      0 0.0.0.0:980             0.0.0.0:*                           8861/rpcbind
udp6       0      0 :::111                  :::*                                8861/rpcbind
udp6       0      0 :::980                  :::*                                8861/rpcbind

 

The startup script in /etc/init.d/rpcbind looks for a configuration file in /etc/default/rpcbind or /etc/rpcbind.conf (Ubuntu 14.04) but these do not exist. A bug was filed for this with a suggested rpcbind configuration. I created my /etc/rpcbind.conf based on the suggestion:

# /etc/init.d/rpcbind

OPTIONS=""

# Cause rpcbind to do a "warm start" utilizing a state file (default)
# OPTIONS="-w "

# Uncomment the following line to restrict rpcbind to localhost only for UDP requests
OPTIONS="${OPTIONS} -h 192.168.166.108"

# Uncomment the following line to enable libwrap TCP-Wrapper connection logging
# OPTIONS="${OPTIONS} -l "

 

After restarting rpcbind with the new configuration file netstat listed the following:

netstat -aunlp | grep rpcbind
udp        0      0 192.168.166.108:111     0.0.0.0:*                           8890/rpcbind    
udp        0      0 127.0.0.1:111           0.0.0.0:*                           8890/rpcbind    
udp        0      0 0.0.0.0:1009            0.0.0.0:*                           8890/rpcbind    
udp6       0      0 ::1:111                 :::*                                8890/rpcbind    
udp6       0      0 :::1009                 :::*                                8890/rpcbind 

 

You can also use rpcinfo -p from an external machine to check if rpcbind is open to connections (replace XXX.XXX.XXX.XXX with your server IP):

rpcinfo -p XXX.XXX.XXX.XXX
   program vers proto   port  service
    100000    4   tcp    111  portmapper
    100000    3   tcp    111  portmapper
    100000    2   tcp    111  portmapper
    100000    4   udp    111  portmapper
    100000    3   udp    111  portmapper
    100000    2   udp    111  portmapper
    100024    1   udp  33218  status
    100024    1   tcp  57664  status

 

That wasn’t good to see. I am not the only one that has come across this issue as reported here.

To secure rpcbind and restrict access to specific hosts you can add entries to /etc/hosts.allow as such (replace XXX.XXX.XXX.XXX with your local server IP):

# /etc/hosts.allow: list of hosts that are allowed to access the system.
#                   See the manual pages hosts_access(5) and hosts_options(5).
#
# Example:    ALL: LOCAL @some_netgroup
#             ALL: .foobar.edu EXCEPT terminalserver.foobar.edu
#
# If you're going to protect the portmapper use the name "rpcbind" for the
# daemon name. See rpcbind(8) and rpc.mountd(8) for further information.
#
rpcbind: XXX.XXX.XXX.XXX
rpcbind: 127.0.0.1

 

Then edit /etc/hosts.deny and add:

# /etc/hosts.deny: list of hosts that are _not_ allowed to access the system.
#                  See the manual pages hosts_access(5) and hosts_options(5).
#
# Example:    ALL: some.host.name, .some.domain
#             ALL EXCEPT in.fingerd: other.host.name, .other.domain
#
# If you're going to protect the portmapper use the name "rpcbind" for the
# daemon name. See rpcbind(8) and rpc.mountd(8) for further information.
#
# The PARANOID wildcard matches any host whose name does not match its
# address.
#
# You may wish to enable this to ensure any programs that don't
# validate looked up hostnames still leave understandable logs. In past
# versions of Debian this has been the default.
# ALL: PARANOID
rpcbind: ALL

 

Now you can try again from an external machine using rpcinfo:

rpcinfo -p 104.237.142.239
rpcinfo: can't contact portmapper: RPC: Authentication error; why = Client credential too weak

Success!

You can find some additional information about this new DDoS reflection attack on the Level 3 Communication blog.

Be the first to write a comment.

Your feedback