A Netflow Parser library for Cisco V5, V7, V9, and IPFIX written in Rust. Supports chaining of multiple versions in the same stream.
- Example
- Serialization (JSON)
- Filtering for a Specific Version
- Parsing Out Unneeded Versions
- Error Handling Configuration
- Netflow Common
- Re-Exporting Flows
- V9/IPFIX Notes
- Features
- Included Examples
use netflow_parser::{NetflowParser, NetflowPacket};
// 0000 00 05 00 01 03 00 04 00 05 00 06 07 08 09 00 01 ................
// 0010 02 03 04 05 06 07 08 09 00 01 02 03 04 05 06 07 ................
// 0020 08 09 00 01 02 03 04 05 06 07 08 09 00 01 02 03 ................
// 0030 04 05 06 07 08 09 00 01 02 03 04 05 06 07 08 09 ................
// 0040 00 01 02 03 04 05 06 07 ........
let v5_packet = [0, 5, 0, 1, 3, 0, 4, 0, 5, 0, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7,];
match NetflowParser::default().parse_bytes(&v5_packet).first() {
Some(NetflowPacket::V5(v5)) => assert_eq!(v5.header.version, 5),
Some(NetflowPacket::Error(e)) => println!("{:?}", e),
_ => (),
}Structures fully support serialization. Below is an example using the serde_json macro:
use serde_json::json;
use netflow_parser::NetflowParser;
// 0000 00 05 00 01 03 00 04 00 05 00 06 07 08 09 00 01 ................
// 0010 02 03 04 05 06 07 08 09 00 01 02 03 04 05 06 07 ................
// 0020 08 09 00 01 02 03 04 05 06 07 08 09 00 01 02 03 ................
// 0030 04 05 06 07 08 09 00 01 02 03 04 05 06 07 08 09 ................
// 0040 00 01 02 03 04 05 06 07 ........
let v5_packet = [0, 5, 0, 1, 3, 0, 4, 0, 5, 0, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7,];
println!("{}", json!(NetflowParser::default().parse_bytes(&v5_packet)).to_string());[
{
"V5": {
"header": {
"count": 1,
"engine_id": 7,
"engine_type": 6,
"flow_sequence": 33752069,
"sampling_interval": 2057,
"sys_up_time": { "nanos": 672000000, "secs": 50332 },
"unix_nsecs": 134807553,
"unix_secs": 83887623,
"version": 5
},
"sets": [
{
"d_octets": 66051,
"d_pkts": 101124105,
"dst_addr": "4.5.6.7",
"dst_as": 515,
"dst_mask": 5,
"dst_port": 1029,
"first": { "nanos": 87000000, "secs": 67438 },
"input": 515,
"last": { "nanos": 553000000, "secs": 134807 },
"next_hop": "8.9.0.1",
"output": 1029,
"pad1": 6,
"pad2": 1543,
"protocol_number": 8,
"protocol_type": "Egp",
"src_addr": "0.1.2.3",
"src_as": 1,
"src_mask": 4,
"src_port": 515,
"tcp_flags": 7,
"tos": 9
}
]
}
}
]use netflow_parser::{NetflowParser, NetflowPacket};
// 0000 00 05 00 01 03 00 04 00 05 00 06 07 08 09 00 01 ................
// 0010 02 03 04 05 06 07 08 09 00 01 02 03 04 05 06 07 ................
// 0020 08 09 00 01 02 03 04 05 06 07 08 09 00 01 02 03 ................
// 0030 04 05 06 07 08 09 00 01 02 03 04 05 06 07 08 09 ................
// 0040 00 01 02 03 04 05 06 07 ........
let v5_packet = [0, 5, 0, 1, 3, 0, 4, 0, 5, 0, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7,];
let parsed = NetflowParser::default().parse_bytes(&v5_packet);
let v5_parsed: Vec<NetflowPacket> = parsed.into_iter().filter(|p| p.is_v5()).collect();If you only care about a specific version or versions you can specify allowed_versions:
use netflow_parser::{NetflowParser, NetflowPacket};
// 0000 00 05 00 01 03 00 04 00 05 00 06 07 08 09 00 01 ................
// 0010 02 03 04 05 06 07 08 09 00 01 02 03 04 05 06 07 ................
// 0020 08 09 00 01 02 03 04 05 06 07 08 09 00 01 02 03 ................
// 0030 04 05 06 07 08 09 00 01 02 03 04 05 06 07 08 09 ................
// 0040 00 01 02 03 04 05 06 07 ........
let v5_packet = [0, 5, 0, 1, 3, 0, 4, 0, 5, 0, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7,];
let mut parser = NetflowParser::default();
parser.allowed_versions = [7, 9].into();
let parsed = parser.parse_bytes(&v5_packet);This code will return an empty Vec as version 5 is not allowed.
To prevent memory exhaustion from malformed packets, the parser limits the size of error buffer samples. By default, only the first 256 bytes of unparseable data are stored in error messages. You can customize this limit for all parsers:
use netflow_parser::NetflowParser;
let mut parser = NetflowParser::default();
// Configure maximum error buffer size for the main parser (default: 256 bytes)
// This applies to generic parsing errors
parser.max_error_sample_size = 512;
// Configure maximum error buffer size for V9 (default: 256 bytes)
parser.v9_parser.max_error_sample_size = 512;
// Configure maximum error buffer size for IPFIX (default: 256 bytes)
parser.ipfix_parser.max_error_sample_size = 512;
let parsed = parser.parse_bytes(&some_packet);This setting helps prevent memory exhaustion when processing malformed or malicious packets while still providing enough context for debugging.
We have included a NetflowCommon and NetflowCommonFlowSet structure.
This will allow you to use common fields without unpacking values from specific versions.
If the packet flow does not have the matching field it will simply be left as None.
use std::net::IpAddr;
use netflow_parser::protocol::ProtocolTypes;
#[derive(Debug, Default)]
pub struct NetflowCommon {
pub version: u16,
pub timestamp: u32,
pub flowsets: Vec<NetflowCommonFlowSet>,
}
#[derive(Debug, Default)]
struct NetflowCommonFlowSet {
src_addr: Option<IpAddr>,
dst_addr: Option<IpAddr>,
src_port: Option<u16>,
dst_port: Option<u16>,
protocol_number: Option<u8>,
protocol_type: Option<ProtocolTypes>,
first_seen: Option<u32>,
last_seen: Option<u32>,
src_mac: Option<String>,
dst_mac: Option<String>,
}use netflow_parser::{NetflowParser, NetflowPacket};
// 0000 00 05 00 01 03 00 04 00 05 00 06 07 08 09 00 01 ................
// 0010 02 03 04 05 06 07 08 09 00 01 02 03 04 05 06 07 ................
// 0020 08 09 00 01 02 03 04 05 06 07 08 09 00 01 02 03 ................
// 0030 04 05 06 07 08 09 00 01 02 03 04 05 06 07 08 09 ................
// 0040 00 01 02 03 04 05 06 07 ........
let v5_packet = [0, 5, 0, 1, 3, 0, 4, 0, 5, 0, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3,
4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1,
2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7];
let netflow_common = NetflowParser::default()
.parse_bytes(&v5_packet)
.first()
.unwrap()
.as_netflow_common()
.unwrap();
for common_flow in netflow_common.flowsets.iter() {
println!("Src Addr: {} Dst Addr: {}", common_flow.src_addr.unwrap(), common_flow.dst_addr.unwrap());
}To gather all flowsets from all packets into a flattened vector:
use netflow_parser::NetflowParser;
let flowsets = NetflowParser::default().parse_bytes_as_netflow_common_flowsets(&v5_packet);By default, NetflowCommon maps standard IANA fields to the common structure. However, you can customize which fields are used for V9 and IPFIX packets using configuration structs. This is useful when:
- You want to prefer IPv6 addresses over IPv4
- Your vendor uses non-standard field mappings
- You need to extract data from vendor-specific enterprise fields
use netflow_parser::netflow_common::{NetflowCommon, V9FieldMappingConfig};
use netflow_parser::variable_versions::v9_lookup::V9Field;
// Create a custom configuration that prefers IPv6 addresses
let mut config = V9FieldMappingConfig::default();
config.src_addr.primary = V9Field::Ipv6SrcAddr;
config.src_addr.fallback = Some(V9Field::Ipv4SrcAddr);
config.dst_addr.primary = V9Field::Ipv6DstAddr;
config.dst_addr.fallback = Some(V9Field::Ipv4DstAddr);
// Use with a parsed V9 packet
// let common = NetflowCommon::from_v9_with_config(&v9_packet, &config);use netflow_parser::netflow_common::{NetflowCommon, IPFixFieldMappingConfig};
use netflow_parser::variable_versions::ipfix_lookup::{IPFixField, IANAIPFixField};
// Create a custom configuration that prefers IPv6 addresses
let mut config = IPFixFieldMappingConfig::default();
config.src_addr.primary = IPFixField::IANA(IANAIPFixField::SourceIpv6address);
config.src_addr.fallback = Some(IPFixField::IANA(IANAIPFixField::SourceIpv4address));
config.dst_addr.primary = IPFixField::IANA(IANAIPFixField::DestinationIpv6address);
config.dst_addr.fallback = Some(IPFixField::IANA(IANAIPFixField::DestinationIpv4address));
// Use with a parsed IPFIX packet
// let common = NetflowCommon::from_ipfix_with_config(&ipfix_packet, &config);Both V9FieldMappingConfig and IPFixFieldMappingConfig support configuring:
| Field | Description | Default V9 Field | Default IPFIX Field |
|---|---|---|---|
src_addr |
Source IP address | Ipv4SrcAddr (fallback: Ipv6SrcAddr) | SourceIpv4address (fallback: SourceIpv6address) |
dst_addr |
Destination IP address | Ipv4DstAddr (fallback: Ipv6DstAddr) | DestinationIpv4address (fallback: DestinationIpv6address) |
src_port |
Source port | L4SrcPort | SourceTransportPort |
dst_port |
Destination port | L4DstPort | DestinationTransportPort |
protocol |
Protocol number | Protocol | ProtocolIdentifier |
first_seen |
Flow start time | FirstSwitched | FlowStartSysUpTime |
last_seen |
Flow end time | LastSwitched | FlowEndSysUpTime |
src_mac |
Source MAC address | InSrcMac | SourceMacaddress |
dst_mac |
Destination MAC address | InDstMac | DestinationMacaddress |
Each field mapping has a primary field (always checked first) and an optional fallback field (used if primary is not present in the flow record).
Parsed V5, V7, V9, and IPFIX packets can be re-exported back into bytes.
Note: For V9/IPFIX, we only export the original padding we dissected and do not calculate/align the flowset padding ourselves. If you modify an existing V9/IPFIX flow or create your own, you must manually adjust the padding.
// 0000 00 05 00 01 03 00 04 00 05 00 06 07 08 09 00 01 ................
// 0010 02 03 04 05 06 07 08 09 00 01 02 03 04 05 06 07 ................
// 0020 08 09 00 01 02 03 04 05 06 07 08 09 00 01 02 03 ................
// 0030 04 05 06 07 08 09 00 01 02 03 04 05 06 07 08 09 ................
// 0040 00 01 02 03 04 05 06 07 ........
let packet = [
0, 5, 0, 1, 3, 0, 4, 0, 5, 0, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3,
4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1,
2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7,
];
if let NetflowPacket::V5(v5) = NetflowParser::default()
.parse_bytes(&packet)
.first()
.unwrap()
{
assert_eq!(v5.to_be_bytes(), packet);
}Parse the data (&[u8]) like any other version. The parser (NetflowParser) caches parsed templates, so you can send header/data flowset combos and it will use the cached templates. To see cached templates, use the parser for the correct version (v9_parser for V9, ipfix_parser for IPFIX).
IPFIX Note: We only parse sequence number and domain id, it is up to you if you wish to validate it.
use netflow_parser::NetflowParser;
let parser = NetflowParser::default();
dbg!(parser.v9_parser.templates);
dbg!(parser.v9_parser.options_templates);To access templates flowset of a processed V9/IPFix flowset you can find the flowsets attribute on the Parsed Record. In there you can find Templates, Option Templates, and Data Flowsets.
Parsers (NetflowParser, V9Parser, IPFixParser) are not thread-safe and should not be shared across threads without external synchronization. Each parser maintains internal state (template caches) that is modified during parsing.
Recommended pattern for multi-threaded applications:
- Create one parser instance per thread
- Each thread processes packets from a single router/source
- See
examples/netflow_udp_listener_multi_threaded.rsfor implementation example
This library includes several performance optimizations:
- Single-pass field caching - NetflowCommon conversions use efficient single-pass lookups
- Minimal cloning - Template storage avoids unnecessary vector clones
- Optimized string processing - Single-pass filtering and prefix stripping
- Capacity pre-allocation - Vectors pre-allocate when sizes are known
- Bounded error buffers - Error handling limits memory consumption to prevent exhaustion
Best practices for optimal performance:
- Reuse parser instances instead of creating new ones for each packet
- Use
parse_bytes_as_netflow_common_flowsets()when you only need flow data - For V9/IPFIX, batch process packets from the same source to maximize template cache hits
parse_unknown_fields- When enabled fields not listed in this library will attempt to be parsed as a Vec of bytes and the field_number listed. When disabled an error is thrown when attempting to parse those fields. Enabled by default.netflow_common- When enabled providesNetflowCommonandNetflowCommonFlowSetstructures for working with common fields across different Netflow versions. Disabled by default.
Examples have been included mainly for those who want to use this parser to read from a Socket and parse netflow. In those cases with V9/IPFix it is best to create a new parser for each router. There are both single threaded and multi-threaded examples in the examples directory.
To run:
cargo run --example netflow_udp_listener_multi_threaded
or
cargo run --example netflow_udp_listener_single_threaded
or
cargo run --example netflow_udp_listener_tokio
or
cargo run --example netflow_pcap
The pcap example also shows how to cache flows that have not yet discovered a template.
If you find my work helpful, consider supporting me!
