Debugging iptables rules with tcpdump and wireshark

Debugging iptables rules with tcpdump and wireshark

While reading libpcap code I bumped on very interesting feature that I though was not possible doing with pcap library. What I learned was that libpcap supports listening on netlink sockets, more precisely - listening on iptables rules which have NFLOG target.

Lets start from the beginning, libpcap is packet capture (pcap) library which consists of API for capturing network traffic. Capturing and analyzing network traffic can be very demanding process because there can be hundreds of thousands packets at every second coming to interface. Fortunately, Linux socket filter mechanism allows user to specify string filter (in BPF syntax) which is then compiled and stored in socket structure and with kernel functions it can be executed. Latter mechanism is used by libpcap. Whole process of capturing network traffic consists of opening socket and then reading from it (perhaps filtering if filter is specified). Netlink is socket family used for inter-process communication on same machine and it can be used for communication between kernel and user space. For netlink sockets AF_NETLINK domain is used. I assume that if you read this far, you already know what iptables stands for. But just in case - iptables is part of netfilter framework which main objective is to provide necessary features to build a firewall. Firewall built with iptables consists of series of rules which are used to compare packet with them when packet gets to that rule. When a packet matches a rule the action is performed (called target in netfilter terms). There are various targets but the simplest ones are DROP and ACCEPT. Target which is in focus of this blog post is NFLOG target. Task of NFLOG target is to copy packets and send them to specified netlink socket. Some parameters which can be passed to NFLOG are (copied from ulogd2 documentation):

--nflog-group N

The number of the netlink multicast group to which NFLOGed packets are sent. 

--nflog-range N

Copyrange. This works like the snaplen parameter of tcpdump. You can specify a number of bytes up to which the packet is copied. If you say 40, you will receive the first forty bytes of every packet. Leave it to 0 to dump the whole packet.

--nflog-threshold N

Queue threshold. If a packet is matched by the iptables rule, and already N packets are in the queue, the queue is flushed to userspace. You can use this to implement a policy like: Use a big queue in order to gain high performance, but still have certain packets logged immediately to userspace.

--nflog-prefix STRING

A string that is associated with every packet logged by this rule. You can use this option to later tell from which rule the packet was logged. 

So now lets see how is capturing  network packets that pass through iptables rules done. First it is necessary that at least one NFLOG rule is present somewhere in netfilter chains.

# iptables -A INPUT -j NFLOG --nflog-group 10

Shown rule has no filter attached to it, it sends all packets to netlink group 10 except packets that were dropped by some previous rule in a chain. After adding rule with NFLOG target it is possible to track packets that hit that rule using tcpdump.

# tcpdump -i nflog:10 -w <filename>

For unknown reasons** it is not possible to output packet info on screen, so filename must be specified and packet information is stored in pcap format. Putting filter on shown tcpdump command didnt work also for unknown reasons, because clearly it does support it (found the part in libpcap in nflog_activate function).

Wireshark tool also supports listening on NFLOG target. After starting wireshark there will be NFLOG interface shown in the list of all interfaces. After choosing NFLOG interface some options can be set for interface. One of them is setting BPF filter - which didnt work here neither. After typing few letters of filter wireshark just crashed. In interface options I didnt find option of setting netlink group and wireshark didnt capture packets with previously defined iptables rule. After adding:

# iptables -A INPUT -j NFLOG

wireshark started to capture packets.
Debugging of iptables rules with tcpdump can consists of two iptables rules which are added before and after some rule which we want to inspect. Those rules should have different netlink group values. Starting two tcpdump processes to capture packets on different netlink groups, and then diffing those two files, it is possible to get packets that are dropped by rule which is inspected. It should be noted that described procedure is somewhat clumsy to use when there are hundreds of iptables rules. Another issue is performance issue - NFLOG packet copying is very time expensive action and it can be bottleneck when dealing with high amount of incoming or outgoing packets.

**After further examination of tcpdump and libpcap it was revealed why tpcudmp cant print NFLOG packet information on screen. Printing packet information is implemented in tcpdump not libpcap. So for printing on screen there must be special function for every protocol used so that the headers of various protocols can be printed. One such print function for NFLOG interface does exist so tcpdump is unable to print packet information on screen.

EDIT: further reading about implementing tcpdump printing capability for NFLOG can be done here.

download file now