Zeek XDP Filtering

Uses XDP in order to "shunt" traffic - that way, Zeek doesn't waste its time analyzing packets that won't get any useful information.

You can use the provided configuration that is similar in functionality to bro-react:

@load xdp/shunt/bulk

This provides a few configuration knobs, but it should shunt connections that are deemed "too large" and unshunt it when a certain time elapses without the connection terminating.

The XDP program will still forward certain TCP packets, namely FIN, RST, SYN, and ACK packets. This way, Zeek can tell when a connection is complete and properly terminate it.

Manual Shunting

You may also choose to manually shunt connections based on your own criteria. You can use the provided BIFs for that.

Attaching to existing XDP programs

In a Zeek cluster (and many environments), the preferred approach is for Zeek to know nothing about XDP, but just add and remove flows/pairs to a map. The following examples give info on how to start and stop the program for shunting, but in a cluster it may be better to reuse the same map for all, and to have an external program manage the XDP logic. This way, if a node crashes, it will keep it's shunting map, and the Zeek processes don't need to worry about stopping the shunt.

For this, see the provided reconnect instead of start_shunt. Then, if you need to stop adding and removing from the map, use disconnect. This does not do much, since the XDP program remains the same. All you need to do is connect to the pinned maps.

Flow-based shunting

This shunts encrypted sessions, then unshunts when it sees a FIN/RST TCP packet:

@load xdp
@load xdp/shunt/conn_id

global xdp_prog: opaque of XDP::Program;

event zeek_init()
	{
	local opts: XDP::ShuntOptions = [
		$attach_mode=XDP::SKB, # SKB is more for testing, NATIVE is better
		$conn_id_map_max_size=131072, # A bit over the max number of shunted connections
		$ip_pair_map_max_size=1, # We can't remove the map, so make it very small
	];
	xdp_prog = XDP::start_shunt(opts);
	}

event connection_state_remove(c: connection)
	{
	XDP::Shunt::ConnID::unshunt(xdp_prog, XDP::conn_id_to_canonical(c$id));
	}

event XDP::Shunt::ConnID::unshunted_conn(cid: XDP::canonical_id,
    stats: XDP::ShuntedStats)
	{
	assert stats$present;
	print fmt("Unshunted connection from %s:%d<->%s:%d. Transmitted %d bytes and %d packets.",
	    cid$ip1, cid$ip1_port, cid$ip2, cid$ip2_port, stats$bytes_from_1 +
	    stats$bytes_from_2, stats$packets_from_1 + stats$packets_from_2);

	if ( stats?$timestamp )
		print fmt("Last packet was at %s.", stats$timestamp);
	}

event XDP::Shunt::ConnID::shunted_conn(cid: XDP::canonical_id)
	{
	print fmt("Shunted connection from %s:%d<->%s:%d", cid$ip1, cid$ip1_port,
	    cid$ip2, cid$ip2_port);
	}

event ssl_established(c: connection)
	{
	XDP::Shunt::ConnID::shunt(xdp_prog, XDP::conn_id_to_canonical(c$id));
	}

event zeek_done()
	{
	XDP::end_shunt(xdp_prog);
	}

IP Pairs

Shunting with IP pairs is similar:

@load xdp
@load xdp/shunt/ip_pair

global xdp_prog: opaque of XDP::Program;

event zeek_init()
	{
	local opts: XDP::ShuntOptions = [
		$attach_mode=XDP::SKB, # SKB is more for testing, NATIVE is better
		$conn_id_map_max_size=1, # We can't remove the map, so make it very small
		$ip_pair_map_max_size=131072, # A bit over the max number of shunted connections
	];
	xdp_prog = XDP::start_shunt(opts);
	}

event connection_state_remove(c: connection)
	{
	XDP::Shunt::IPPair::unshunt(xdp_prog, [ $ip1=c$id$orig_h, $ip2=c$id$resp_h ]);
	}

event XDP::Shunt::IPPair::unshunted_pair(pair: XDP::ip_pair,
    stats: XDP::ShuntedStats)
	{
	assert stats$present;
	print fmt("Unshunted connection from %s<->%s. Transmitted %d bytes and %d packets.",
	    pair$ip1, pair$ip2, stats$bytes_from_1 + stats$bytes_from_2,
	    stats$packets_from_1 + stats$packets_from_2);

	if ( stats?$timestamp )
		print fmt("Last packet was at %s.", stats$timestamp);
	}

event XDP::Shunt::IPPair::shunted_pair(pair: XDP::ip_pair)
	{
	print fmt("Shunted connection from %s<->%s", pair$ip1, pair$ip2);
	}

event ssl_established(c: connection)
	{
	XDP::Shunt::IPPair::shunt(xdp_prog, [ $ip1=c$id$orig_h, $ip2=c$id$resp_h ]);
	}

event zeek_done()
	{
	XDP::end_shunt(xdp_prog);
	}

VLANs

By default, the XDP program will parse only a static number of VLAN headers, defined within the file. This should be adequate for most use cases. It will always parse the VLAN headers, but it will only use them if set with the include_vlan config option, like so:

event zeek_init()
	{
	local opts: XDP::ShuntOptions = [
		$attach_mode=XDP::SKB,
		$conn_id_map_max_size=131072,
		$ip_pair_map_max_size=1,
		$include_vlan=T, # Added this!
	];
	xdp_prog = XDP::start_shunt(opts);
	}

Then, the "canonical" ID does not add the VLAN by default, so the user has to add it. You can ask Zeek to bring the VLANs in the conn_id by loading a policy script:

@load policy/frameworks/conn_key/vlan_fivetuple

Then add the information to the canonical ID when shunting and unshunting:

event connection_state_remove(c: connection)
	{
	local new_id = XDP::conn_id_to_canonical(c$id);

	if ( c$id$ctx?$vlan )
		new_id$outer_vlan_id = c$id$ctx$vlan;

	if ( c$id$ctx?$inner_vlan )
		new_id$inner_vlan_id = c$id$ctx$inner_vlan;

	XDP::Shunt::ConnID::unshunt(xdp_prog, new_id);
	}

# ...

event ssl_established(c: connection)
	{
	local new_id = XDP::conn_id_to_canonical(c$id);

	if ( c$id$ctx?$vlan )
		new_id$outer_vlan_id = c$id$ctx$vlan;

	if ( c$id$ctx?$inner_vlan )
		new_id$inner_vlan_id = c$id$ctx$inner_vlan;

	XDP::Shunt::ConnID::shunt(xdp_prog, new_id);
	}

A VLAN of 0 is considered the same as a VLAN with no ID for the purposes of shunting.

Internals

This plugin uses XDP (eXpress Data Path) in order to filter traffic before it reaches user applications, like Zeek. This is filtering purely on the network device that Zeek sees traffic from.

The XDP program itself simply checks if a certain IP pair or 5-tuple are in the "shunting map." This map is managed entirely from user space, that is, the Zeek program. You may also, optionally, make it VLAN-aware (see above).

Any shunted connections keep state about certain statistics, such as the last packet seen and how many bytes/packets were seen from each direction. This may help the user determine how effective the shunting was, or simply unshunt connections after a certain time elapsed without any traffic.

TCP control packets, like FIN and RST, are forwarded to Zeek regardless of content. This allows Zeek to determine that the connection was terminated by a FIN or RST rather than by timeout.

Here is a diagram, with some parts slightly simplified:

flowchart TD
    subgraph driver
        stuff
        XDP
    end

    subgraph Zeek
        direction LR
        subgraph ssl
            direction TB
            established["<code>event ssl_established</code"] -->
            shunt-ssl["<code>shunt(conn_id)</code>"]
        end
        subgraph bulk
            direction TB
            newconn["<code>event new_connection</code>"] -->
            watch["<code>ConnPolling::watch(conn_callback, ...)</code>"] -->
            callback["<code>function conn_callback(c: connection)</code>"] -->
            size_check["<code>if ( c$orig$size > size_threshold || c$resp$size > size_threshold )</code>"] -->
            shunt-bulk["<code>shunt(c$id);</code>"]
        end
    end

    subgraph kernel    
        subgraph bpf[BPF map]
            filter_map
            ip_pair_map
        end
        
        network-stack["network stack"]
    end

    NIC --> XDP
    XDP -- "3. (if in map) XDP_DROP" --> trash[(trash)]
    XDP -- "3. (if not in map) XDP_PASS" --> stuff
    stuff --> network-stack --> Zeek

    XDP -- "1. Lookup" --> bpf
    bpf -- "2. Provide stats" --> XDP

    shunt-ssl --> add:::hidden
    shunt-bulk --> add:::hidden
    add -- "Add to map" --> bpf

    NIC