Zeek XDP Filtering

[!NOTE] This plugin uses features that are not yet in a Zeek release. The zeekctl support is included in this directory. Since this is intended to make it into Zeek as well, it is also in a branch. Do not use both zeekctl plugins at the same time.

Uses XDP in order to "shunt" traffic - that way, Zeek doesn't waste its time analyzing packets that won't get any useful information.

The provided scripts do a lot of heavy lifting, with some configuration knobs for common patterns. If you just want to connect to a loaded XDP program with some simple shunting automatically done, just add this to local.zeek (or a test script):

# local.zeek
@load xdp
@load xdp/shunt/policy

This will connect to the loaded XDP program and shunt some traffic. Any shunted traffic will appear in xdp_shunt.log once it is unshunted (which is based on timeouts, so feedback may not be immediate). Use the shunted_conn event for more immediate feedback.

Clusters need to load the XDP program before starting. The XDP program should be magically found, but you must enable zeekctl loading the XDP plugin (in zeekctl.cfg):

# zeekctl.cfg
xdp.enabled = True

This will load the given XDP program on start and unload it on stop.

To change the loaded XDP program, set the corresponding option in zeekctl.cfg. This may be necessary if the magic location does not work.

# zeekctl.cfg
xdp.Program = /home/etyp/src/zeek-xdp-shunter/build/bpf/filter.o

If you want to just load the XDP program on a standalone instance, you can tell the shunter to do so. This means that Zeek itself will load the program, so it must have necessary permissions. This option is primarily for testing. Just add this to the local.zeek file in order to tell Zeek to load the XDP program on init:

# local.zeek
redef XDP::start_new_xdp = T;

However, this might cause issues in clusters, where not every Zeek process is listening on an interface.

For more complex environments, some combination of loading the XDP programs before with xdp-loader pinned at /sys/fs/bpf/zeek on each interface and simply loading with @load xdp should work.

Building

You may need a couple of packages in order to properly build the XDP/BPF programs. For example, on Ubuntu:

sudo apt install bpftool libxdp-dev clang

It might also be useful to have certain tools in order to load the programs and analyze them:

sudo apt install xdp-tools

[!NOTE] Loading an XDP program to the kernel is a privileged operation, so always expect to require root if loading. However, it's not necessary for zeek to load the XDP program (though it is possible). All Zeek needs is to access the BPF map.

In order to achieve this, first you must still have cap_net_raw just to run Zeek (via sudo setcap cap_not_raw+ep <zeekinstall>/bin/zeek).

Then, you may be able to just set permissions for maps via chmod after the XDP program has loaded with:

$ sudo chmod 755 /sys/fs/bpf
$ sudo xdp-loader load <interface> build/bpf/filter.o -p /sys/fs/bpf/zeek
$ sudo chmod 755 /sys/fs/bpf/zeek
$ sudo chmod 666 /sys/fs/bpf/zeek/*

However, since it's mounted, you may also need to remount with those permissions first with sudo mount -o remount,mode=755 /sys/fs/bpf.

When using this non-root way, do not start the XDP shunter with redef XDP::start_new_xdp = T

Manual Shunting

You may also choose to manually shunt connections based on your own criteria. You can use the provided functions for that.

Flow-based shunting

This shunts encrypted sessions, then unshunts when it is inactive for a default amount of time:

@load xdp

# Tell Zeek to start a new XDP program, not reconnect
redef XDP::start_new_xdp = T;

event XDP::Shunt::ConnID::unshunted_conn(cid: conn_id, stats: XDP::ShuntedStats)
	{
	assert stats$present;
	print fmt("Unshunted connection from %s:%d<->%s:%d. Transmitted %d bytes and %d packets.",
	    cid$orig_h, cid$orig_p, cid$resp_h, cid$resp_p, stats$bytes_from_1 +
	    stats$bytes_from_2, stats$packets_from_1 + stats$packets_from_2);

	if ( stats?$timestamp )
		print fmt("Last packet was at %s.", stats$timestamp);
	}

event XDP::Shunt::ConnID::shunted_conn(cid: conn_id)
	{
	print fmt("Shunted connection from %s:%d<->%s:%d", cid$orig_h, cid$orig_p,
	    cid$resp_h, cid$resp_p);
	}

# This is exactly what is in the ssl.zeek shunting policy!
event ssl_established(c: connection)
	{
	XDP::Shunt::ConnID::shunt(c);
	}

event zeek_done()
	{
	XDP::end_shunt();
	}

The flow-based shunting relies on timeouts to unshunt connections. Because Zeek never sees the connection, Zeek attempts to time it out after the default timeout interval. This is always going to be necessary for many connections. We then add special logic that checks the shunted connection: if the last shunted packet is in the provided timeout range, then we can continue to timing the connection out. Otherwise, the shunter will block Zeek from timing out the connection.

IP Pairs

Shunting with IP pairs is similar:

@load xdp

# Tell Zeek to start a new XDP program, not reconnect
redef XDP::start_new_xdp = T;

event XDP::Shunt::IPPair::unshunted_pair(pair: XDP::ip_pair,
    stats: XDP::ShuntedStats)
	{
	assert stats$present;
	print fmt("Unshunted connection from %s<->%s. Transmitted %d bytes and %d packets.",
	    pair$ip1, pair$ip2, stats$bytes_from_1 + stats$bytes_from_2,
	    stats$packets_from_1 + stats$packets_from_2);

	if ( stats?$timestamp )
		print fmt("Last packet was at %s.", stats$timestamp);
	}

event XDP::Shunt::IPPair::shunted_pair(pair: XDP::ip_pair)
	{
	print fmt("Shunted connection from %s<->%s", pair$ip1, pair$ip2);
	}

event ssl_established(c: connection)
	{
	XDP::Shunt::IPPair::shunt([ $ip1=c$id$orig_h, $ip2=c$id$resp_h ]);
	}

# IP pairs do not normally unshunt on connection remove, but we will here for
# demonstration. Note that conn_id has special handling to remove only after
# a given time, but this will not do so.
event connection_state_remove(c: connection)
	{
	XDP::Shunt::IPPair::unshunt([ $ip1=c$id$orig_h, $ip2=c$id$resp_h ]);
	}

event zeek_done()
	{
	XDP::end_shunt();
	}

IP pairs will not log to the same log, since unshunting an IP pair is a different idea from shunting a particular connection. You may add your own log if you choose.

VLANs

VLANs are automatically dealt with if you load the corresponding connection key plugin:

@load policy/frameworks/conn_key/vlan_fivetuple

With those fields in the conn_id_ctx, the XDP shunter will automatically construct flows properly. It will also load the XDP program with this support, if necessary.

A VLAN of 0 is considered the same as a VLAN with no ID for the purposes of shunting.

You can force not using VLANs, regardless of the conn_id_ctx, by setting:

redef XDP::force_no_vlans = T;

Internals

This plugin uses XDP (eXpress Data Path) in order to filter traffic before it reaches user applications, like Zeek. This is filtering purely on the network device that Zeek sees traffic from.

The XDP program itself simply checks if a certain IP pair or 5-tuple are in the "shunting map." This map is managed entirely from user space, that is, the Zeek program. You may also, optionally, make it VLAN-aware (see above).

Any shunted connections keep state about certain statistics, such as the last packet seen and how many bytes/packets were seen from each direction. This may help the user determine how effective the shunting was, or simply unshunt connections after a certain time elapsed without any traffic.

Here is a diagram, with some parts slightly simplified:

flowchart TD
    subgraph driver
        stuff
        XDP
    end

    subgraph Zeek
        direction LR
        subgraph ssl
            direction TB
            established["<code>event ssl_established</code"] -->
            shunt-ssl["<code>shunt(conn_id)</code>"]
        end
        subgraph bulk
            direction TB
            newconn["<code>event new_connection</code>"] -->
            watch["<code>ConnPolling::watch(conn_callback, ...)</code>"] -->
            callback["<code>function conn_callback(c: connection)</code>"] -->
            size_check["<code>if ( c$orig$size > size_threshold || c$resp$size > size_threshold )</code>"] -->
            shunt-bulk["<code>shunt(c$id);</code>"]
        end
    end

    subgraph kernel    
        subgraph bpf[BPF map]
            filter_map
            ip_pair_map
        end
        
        network-stack["network stack"]
    end

    NIC --> XDP
    XDP -- "3. (if in map) XDP_DROP" --> trash[(trash)]
    XDP -- "3. (if not in map) XDP_PASS" --> stuff
    stuff --> network-stack --> Zeek

    XDP -- "1. Lookup" --> bpf
    bpf -- "2. Provide stats" --> XDP

    shunt-ssl --> add:::hidden
    shunt-bulk --> add:::hidden
    add -- "Add to map" --> bpf

    NIC