Skip to main content
Version: 1.4.0

NetFlow

Push

Synopsis​

Creates a NetFlow v5 collector that accepts flow data over UDP connections. Supports High-Volume collection with multiple workers and configurable buffer sizes.

warning

The collector only supports legacy NetFlow types, such as NetFlow v5. For NetFlow v9, use the ipfix collector type.

For details, see Appendix.

Schema​

- id: <numeric>
  name: <string>
  description: <string>
  type: netflow
  tags: <string[]>
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    address: <string>
    port: <numeric>
    reuse: <boolean>
    buffer_size: <numeric>

Configuration​

The following fields are used to define the device:

Device​

FieldRequiredDefaultDescription
idYUnique identifier
nameYDevice name
descriptionN-Optional description
typeYMust be netflow
tagsN-Optional tags
pipelinesN-Optional pre-processor pipelines
statusNtrueEnable/disable the device

Connection​

FieldRequiredDefaultDescription
addressN"0.0.0.0"Listen address
portN2055Listen port
reuseNtrueEnable socket address reuse

Performance​

FieldRequiredDefaultDescription
buffer_sizeN9000Network read buffer size in bytes

Key Features​

The following are unique features that Director offers.

Multiple Workers​

When reuse is enabled, the collector automatically spawns multiple workers which maintain their own UDP listeners, process flows independently, and write to dedicated queue files.

note

The collector scales up to use all available CPU cores.

Flows​

The collector supports fixed format NetFlow v5 records, application identification, port-based protocol mapping, flow state tracking, and statistical aggregation.

Examples​

The following are commonly used configuration types.

Basic​

A basic collector can be created easily:

Creating a simple NetFlow collector...

devices:
  - id: 1
    name: basic_netflow
    type: netflow
    properties:
      port: 2055

High-Volume​

Performance can be enhanced for high flow volumes:

Optimizing for high message volumes...

devices:
  - id: 2
    name: performant_netflow
    type: netflow
    properties:
      address: "0.0.0.0"
      port: 2055
      reuse: true
      buffer_size: 32768

Legacy Networks​

Collecting flows from older network devices is possible:

Collecting from legacy network devices...

devices:
  - id: 3
    name: legacy_netflow
    type: netflow
    properties:
      address: "0.0.0.0"
      port: 2055
      reuse: true
      buffer_size: 16384

Application Protocols​

The collector supports application-based identification, TCP port mapping, and UDP port mapping. The definition files for these respectively are placed in three locations under <vm_root>:

  • /user/definitions/app-definitions-{device-id}.csv (device-specific)
  • /user/definitions/app-definitions.csv (user-defined)
  • /package/definitions/app-definitions.csv (system defaults)
  • /user/definitions/tcp-definitions-{device-id}.csv (device-specific)
  • /user/definitions/tcp-definitions.csv (user-defined)
  • /package/definitions/tcp-definitions.csv (system defaults)
  • /user/definitions/udp-definitions-{device-id}.csv (device-specific)
  • /user/definitions/udp-definitions.csv (user-defined)
  • /package/definitions/udp-definitions.csv (system defaults):::warning

Definition files must be in CSV format with exactly two columns per row. :::

The contents of these files are:

app-definitions.csv:

SSH,Secure Shell
RDP,Remote Desktop
HTTP,Web Browsing
HTTPS,Secure Web

tcp-definitions.csv:

22,SSH
3389,RDP
80,HTTP
443,HTTPS

udp-definitions.csv:

53,DNS
67,DHCP
123,NTP
161,SNMP
note

The collector will fall back on the system defaults if the custom definition files are not found.

Using application identification...

devices:
  - id: 4
    name: app_aware_netflow
    type: netflow
    properties:
      port: 2055
      reuse: true
      buffer_size: 16384