Zeek XDP Filtering

[!NOTE] This plugin uses features that are not yet in a Zeek release. Currently the zeekctl support is in progress in a branch. Be sure to use current Zeek master and that branch of zeekctl in order to use this plugin.

Uses XDP in order to "shunt" traffic - that way, Zeek doesn't waste its time analyzing packets that won't get any useful information.

The provided scripts do a lot of heavy lifting, with some configuration knobs for common patterns. If you just want to connect to a loaded XDP program with some simple shunting automatically done, just add this to local.zeek (or a test script):

@load xdp
@load xdp/shunt/policy

This will connect to the loaded XDP program and shunt some traffic. Any shunted traffic will appear in xdp_shunt.log once it is unshunted (which is based on timeouts, so feedback may not be immediate). Use the shunted_conn event for more immediate feedback.

Clusters need to load the XDP program before starting. To do so, simply add the XDP program path that should be loaded to the zeekctl.cfg, like:

XDPProgram = /home/etyp/src/zeek-xdp-shunter/build/bpf/filter.o

This will load the given XDP program on start and unload it on stop.

If you want to just load the XDP program on a standalone instance, you can tell the shunter to do so:

redef XDP::start_new_xdp = T;

However, this might cause issues in clusters, where not every Zeek process is listening on an interface.

Manual Shunting

You may also choose to manually shunt connections based on your own criteria. You can use the provided functions for that.

Flow-based shunting

This shunts encrypted sessions, then unshunts when it is inactive for a default amount of time:

@load xdp

# Tell Zeek to start a new XDP program, not reconnect
redef XDP::start_new_xdp = T;

event XDP::Shunt::ConnID::unshunted_conn(cid: conn_id, stats: XDP::ShuntedStats)
	{
	assert stats$present;
	print fmt("Unshunted connection from %s:%d<->%s:%d. Transmitted %d bytes and %d packets.",
	    cid$orig_h, cid$orig_p, cid$resp_h, cid$resp_p, stats$bytes_from_1 +
	    stats$bytes_from_2, stats$packets_from_1 + stats$packets_from_2);

	if ( stats?$timestamp )
		print fmt("Last packet was at %s.", stats$timestamp);
	}

event XDP::Shunt::ConnID::shunted_conn(cid: conn_id)
	{
	print fmt("Shunted connection from %s:%d<->%s:%d", cid$orig_h, cid$orig_p,
	    cid$resp_h, cid$resp_p);
	}

# This is exactly what is in the ssl.zeek shunting policy!
event ssl_established(c: connection)
	{
	XDP::Shunt::ConnID::shunt(c$id);
	}

event zeek_done()
	{
	XDP::end_shunt();
	}

The flow-based shunting relies on timeouts to unshunt connections. Because Zeek never sees the connection, Zeek attempts to time it out after the default timeout interval. This is always going to be necessary for many connections. We then add special logic that checks the shunted connection: if the last shunted packet is in the provided timeout range, then we can continue to timing the connection out. Otherwise, the shunter will block Zeek from timing out the connection.

IP Pairs

Shunting with IP pairs is similar:

@load xdp

# Tell Zeek to start a new XDP program, not reconnect
redef XDP::start_new_xdp = T;

event XDP::Shunt::IPPair::unshunted_pair(pair: XDP::ip_pair,
    stats: XDP::ShuntedStats)
	{
	assert stats$present;
	print fmt("Unshunted connection from %s<->%s. Transmitted %d bytes and %d packets.",
	    pair$ip1, pair$ip2, stats$bytes_from_1 + stats$bytes_from_2,
	    stats$packets_from_1 + stats$packets_from_2);

	if ( stats?$timestamp )
		print fmt("Last packet was at %s.", stats$timestamp);
	}

event XDP::Shunt::IPPair::shunted_pair(pair: XDP::ip_pair)
	{
	print fmt("Shunted connection from %s<->%s", pair$ip1, pair$ip2);
	}

event ssl_established(c: connection)
	{
	XDP::Shunt::IPPair::shunt([ $ip1=c$id$orig_h, $ip2=c$id$resp_h ]);
	}

# IP pairs do not normally unshunt on connection remove, but we will here for
# demonstration. Note that conn_id has special handling to remove only after
# a given time, but this will not do so.
event connection_state_remove(c: connection)
	{
	XDP::Shunt::IPPair::unshunt([ $ip1=c$id$orig_h, $ip2=c$id$resp_h ]);
	}

event zeek_done()
	{
	XDP::end_shunt();
	}

IP pairs will not log to the same log, since unshunting an IP pair is a different idea from shunting a particular connection. You may add your own log if you choose.

VLANs

By default, the XDP program will parse only a static number of VLAN headers, defined within the file. This should be adequate for most use cases. It will always parse the VLAN headers, but it will only use them if set with the include_vlan config option, like so:

redef XDP::include_vlan = T;

[!NOTE] The following about canonical IDs is outdated, since unshunt uses conn_id now. We should add a hook to add them to the canonical_id or something.

Then, the "canonical" ID does not add the VLAN by default, so the user has to add it. You can ask Zeek to bring the VLANs in the conn_id by loading a policy script:

@load policy/frameworks/conn_key/vlan_fivetuple

Then add the information to the canonical ID when shunting and unshunting:

event connection_state_remove(c: connection)
	{
	local new_id = XDP::conn_id_to_canonical(c$id);

	if ( c$id$ctx?$vlan )
		new_id$outer_vlan_id = c$id$ctx$vlan;

	if ( c$id$ctx?$inner_vlan )
		new_id$inner_vlan_id = c$id$ctx$inner_vlan;

	XDP::Shunt::ConnID::unshunt(new_id);
	}

# ...

event ssl_established(c: connection)
	{
	local new_id = XDP::conn_id_to_canonical(c$id);

	if ( c$id$ctx?$vlan )
		new_id$outer_vlan_id = c$id$ctx$vlan;

	if ( c$id$ctx?$inner_vlan )
		new_id$inner_vlan_id = c$id$ctx$inner_vlan;

	XDP::Shunt::ConnID::shunt(new_id);
	}

A VLAN of 0 is considered the same as a VLAN with no ID for the purposes of shunting.

Internals

This plugin uses XDP (eXpress Data Path) in order to filter traffic before it reaches user applications, like Zeek. This is filtering purely on the network device that Zeek sees traffic from.

The XDP program itself simply checks if a certain IP pair or 5-tuple are in the "shunting map." This map is managed entirely from user space, that is, the Zeek program. You may also, optionally, make it VLAN-aware (see above).

Any shunted connections keep state about certain statistics, such as the last packet seen and how many bytes/packets were seen from each direction. This may help the user determine how effective the shunting was, or simply unshunt connections after a certain time elapsed without any traffic.

Here is a diagram, with some parts slightly simplified:

flowchart TD
    subgraph driver
        stuff
        XDP
    end

    subgraph Zeek
        direction LR
        subgraph ssl
            direction TB
            established["<code>event ssl_established</code"] -->
            shunt-ssl["<code>shunt(conn_id)</code>"]
        end
        subgraph bulk
            direction TB
            newconn["<code>event new_connection</code>"] -->
            watch["<code>ConnPolling::watch(conn_callback, ...)</code>"] -->
            callback["<code>function conn_callback(c: connection)</code>"] -->
            size_check["<code>if ( c$orig$size > size_threshold || c$resp$size > size_threshold )</code>"] -->
            shunt-bulk["<code>shunt(c$id);</code>"]
        end
    end

    subgraph kernel    
        subgraph bpf[BPF map]
            filter_map
            ip_pair_map
        end
        
        network-stack["network stack"]
    end

    NIC --> XDP
    XDP -- "3. (if in map) XDP_DROP" --> trash[(trash)]
    XDP -- "3. (if not in map) XDP_PASS" --> stuff
    stuff --> network-stack --> Zeek

    XDP -- "1. Lookup" --> bpf
    bpf -- "2. Provide stats" --> XDP

    shunt-ssl --> add:::hidden
    shunt-bulk --> add:::hidden
    add -- "Add to map" --> bpf

    NIC