NetFlow: Prerequisites and configuration

The NetFlow plugin allows you to monitor the throughput generated by an application, a source IP or a destination IP and generate alerts if defined thresholds are exceeded, it also creates data and performance graphs in the same way as other plugins.

Use cases and good practices for using plugins:

The plugin has been created so that it can meet specific needs. It offers different fields to fill in to target bandwidth consumption.

Ideally, each instantiated plugin should serve a particular need. For example, measuring the throughput generated by a messaging service. In this case, the user will fill in the different fields required for this measurement (destination mail server IP, SMTP port 25 ….).

NetFlow

Introduction

NetFlow is a network protocol used to count IP network traffic. It was developed by Cisco Systems. Nowadays, NetFlow has become an industry standard supported by many devices. There are several versions of the protocol, but the most common versions are versions 5 and 9.

Network flows

NetFlow uses the concept of a stream to capture network behaviour data, such as the source and destination of network traffic, applications using the network, and the amount of bandwidth utilised by those applications.

A stream is a unidirectional sequence of packets between a source and a given destination, defined by a 7-tuple key comprising the following fields:

  • Source IP address
  • Destination IP address
  • Source Port
  • The port of destination
  • IP protocol
  • Input interface
  • Type of IP service

NetFlow records

The NetFlow information collected by the Flow Publisher is managed by creating records for each feed. Each record is managed in the NetFlow cache. When packets are captured, statistics for active flows are updated. Once a stream has been created and placed in the NetFlow cache, it persists until it times out. After the flow has elapsed, the flow record is added to a NetFlow export datagram for transmission to the NetFlow collector.

NetFlow support

In addition to Cisco, many network equipment manufacturers offer NetFlow support on their enclosures. The list includes Juniper, Alcatel-Lucent and Nortel, among others. For software platforms, there is support for VMware servers and Linux.

Some manufacturers use alternative names for this technology:

  • Jflow or cflowd at Juniper Networks
  • NetStream at 3Com / HP
  • NetStream at Huawei Technologies
  • Cflowd at Alcatel-Lucent
  • Rflow at Ericsson
  • AppFlow at Citrix

Architecture

Network elements (switches and routers) establish statistics on the network flow data they export to collectors. These detailed statistics can include numbers of packets and bytes, application ports, IP addresses, quality of service fields, interfaces through which they pass, and so on.

The architecture for collecting information on IP network traffic is as follows:

https://coservit.com/servicenav/wp-content/uploads/sites/3/2018/09/Capture-300x251.png

  • NetFlow Exporter: Monitors packet data, creates records of monitored network traffic, and passes this data to the NetFlow Collector.
  • NetFlow Collector: Collects the records sent by the exporter, stores them in a local database.
  • ServiceNav BOX: Retrieves information collected by the NetFlow Collector according to the requirements set in NetFlow plugin parameters
  • SNP (monitoring platform) allows for the visualisation the collected NetFlow data reported via the ServiceNav BOX

Configuration of prerequisites

Setting up the NetFlow Collector Storage

Depending on your network analysis requirements, it is possible to use a dedicated NetFlow Collector Storage server or to use one of your ServiceNav Boxes already in service.

Sizing the NetFlow Collector Storage

How much disk space should an average NetFlow deployment consume? One of the biggest concerns is that exporting NetFlow will impact available bandwidth, CPU overhead on devices, or the hard drives that store the data.

It is important to note that an export of network stream data may contain records relating to up to 30 conversations or streams. This is important because the average NetFlow volume is directly proportional to the number of unique TCP / UDP sockets created by network clients and servers.

This aggregate nature of NetFlow and the fact that NetFlow packets consist only of IP header information (ie not the packet payload itself), explains why the export does not consume than 1-2% of the interface bandwidth. Since 2004, Cisco’s NetFlow experts have maintained a basic rule that NetFlow will only create 1 to 1.5% throughput on the interface on which it is exported.

What is the typical flow volume per PC? The answer is … “it depends”, but the trend seems to be about 100 streams / minute per computer, peaking out at around 350.

As an example, a company has 1000 nodes and each node generates 200 streams per minute. This equates to about 200,000 streams in a minute, or about 3300 streams per second. Why such volumes?

Applications generate a lot of unique streams, especially web browsers and most applications. Here are some typically ‘chatty’ applications:

  • Java, Adobe, Anti-virus, web browsers
  • Skype is very talkative and causes a good deal of DNS traffic
  • Web pages, generating images, ads, etc.
  • Email constantly checking the inbox
  • NetBIOS

A stream stored on NetFlow Collector Storage occupies 150 bytes of disk space, so it is recommended to provision 2 GB per day and per 100 nodes:

Cpu(s) 4x vCPU
RAM 8GB
Disk Space 20GB + 2GB per day and per 100 nodes
Network Interface 1Gbps

NetFlow Collector Storage Configuration:

The NetFlow Collector Storage is created from an SNB master, this server must be dedicated to collecting NetFlow exports and must not be used as a regular monitoring box.

  • Download the SNB Master:
    • FTP site: software.servicenav.io (contact support for access credentials)
    • Select the SNB master in the directory SNB-SNM – ServiceNav Box/4.0/SNM_MASTER_OVF_2019_01_24_V4_0.0.zip
  • Target network interfaces to meet your analysis needs
  • Log in SSH to Netflow Collector Storage
  • Create a destination directory of NetFlow exports, for example you can create a generic path ~/network_analysis/netflow and created under this one as much directory as network interface to monitor, these directories will be intended to store exports (NetFlow Exporter ). You can name them according to interface ip or others.
  • Ex: mkdir ~ / netflow / 192.168.0.1
  • Define a listening port for the network equipment listener on which NetFlow will be activated, for example 9995 for Router A and 9996 for Router B
  • Create an ACL that allows Netflow Collector Storage to connect to the listening ports:
    • iptables -A INPUT -p udp -dport 9995 -j ACCEPT
    • iptables -A INPUT -p udp -dport 9996 -j ACCEPT
  • Launch the listener via the following command:
    • nfcapd -p 9995 -l /usr/local/nagios/libexec/netflow/RouterA_192.168.0.1 -D
    • nfcapd -p 9996 -l /usr/local/nagios/libexec/netflow/RouterB_192.168.1.1 -D
      • -p sets UDP listening port (9995 in our Cisco router configuration)
      • -l sets the directory where the data will be stored (collector location)
      • -w makes sure that the collection will be made every n minutes (n = 5 by default) with values, 5,10 …
      • -D allows nfcapd to start as a daemon (in background)
  • Initialize the NetFlow Collector Stotage to benefit from the updates: Chapter 2.2 of the following procedure: https://coservit.com/servicenav/en/documentation/service-service-of-box-servicenav/

By default, exports in nfcapd.YYYYMMddhhmm formats are deleted every 24 hours via a scheduled task launched every day at 0h00:

/ Root / crontabRoot

0 0 * * * /usr/local/nagios/libexec/nfcapd_deleteCache.sh> / dev / null 2> & 1

NOTE, the functionality to delete recordings in the format nfcapd.xxx will be available only from version 4.1. In the meantime, in order to avoid the exhausting of disk space, the recordings must be deleted at least every 24 hours. You can contact the Coservit support to install the patch in advance of the scheduled release or to assist you in setting up a scheduled delete file task.

network device configuration

Connect to the network device on which NetFlow is to be enabled and perform these steps to configure NetFlow and NetFlow Data Export.

The following is an example configuration for a CISCO Switch / Router using the version 9 export format:

  1. enable

  2. configure terminal

  3. ip flow-export destination {ip-address | hostname} udp-port

  4. Repeat Step 3 once to configure a second NetFlow export destination.

  5. ip flow-export version 9

  6. interface interface-type interface-number

  7. ip flow {ingress | egress}

  8. exit

  9. Repeat Steps 6 through 8 to enable NetFlow on other interfaces

  10. end

 

Step details

Step Command or Action Objective
Step 1 Activate

Example:

Router> enable

(Required) Enables the privileged EXEC mode.

Enter your password if prompted.

Step 2 configure the terminal

Example:

Router # configure terminal

(Required) Provides access to the global configuration mode.
Step 3 ip flow-export destination {ip-address | hostname} udp-port

Example:

Router (config) # ip flow-export destination 172.16.10.2 9995

IP address or host name of the Netflow Collector Storage the UDP listening port.
Step 4 Repeat step 3 once to set up a second NetFlow export destination. (Optional) You can configure up to two export destinations for NetFlow.
Step 5 ip flow-export version 9

Example:

Router (config) # ip flow-export version 9

(Optional) Enables the export of information in NetFlow cache entries.

The release 9 keyword indicates that the export package uses the version 9 format.

Step 6 interface interface number interface type

Example:

Router (config) # ethernet interface 0/0

(Required) Specifies the interface on which you want to enable NetFlow and enters the interface configuration mode.
Step 7 ip flow {ingress | egress}

Example:

Router (config-if) #ip flow ingressRouter (config-if) # ip flow egress

(Required) Activates NetFlow on the interface.

ingress – Captures the traffic received by the interface.

egress – Retrieves the traffic transmitted by the interface.

Step 8 exit

Example:

Router (config-if) # exit

(Optional) Exit the interface configuration mode and return to the global configuration mode.

Note:

You only need to use this command to enable NetFlow on another interface.

Step 9 end

Example:

Router (config-if) # end

(Required) Exits the current configuration mode and returns to the privileged execution mode

Verification of NetFlow operation and display of NetFlow statistics

Check that NetFlow is correctly configured.

Use the show ip flow interface command to display the NetFlow configuration for an interface. Here is an example of this command output:

Router# show ip flow interface Ethernet0/0 ip flow ingress

Use the show ip cache flow command to verify that NetFlow is operational and to view a summary of NetFlow statistics. Here is an example of this command output:

Router# show ip cache flow

IP packet size distribution (1103746 total packets):

1-32   64   96  128  160  192  224  256  288  320  352  384  416  448  480

.249 .694 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000 .000

512  544  576 1024 1536 2048 2560 3072 3584 4096 4608

.000 .000 .027 .000 .027 .000 .000 .000 .000 .000 .000

IP Flow Switching Cache, 278544 bytes

35 active, 4061 inactive, 980 added

2921778 ager polls, 0 flow alloc failures

Active flows timeout in 30 minutes

Inactive flows timeout in 15 seconds

IP Sub Flow Cache, 21640 bytes

0 active, 1024 inactive, 0 added, 0 added to flow

0 alloc failures, 0 force free

1 chunk, 1 chunk added

last clearing of statistics never

Protocol         Total    Flows   Packets Bytes  Packets Active(Sec) Idle(Sec)

——–         Flows     /Sec     /Flow  /Pkt     /Sec     /Flow     /Flow

TCP-FTP            108      0.0      1133    40      2.4    1799.6       0.9

TCP-FTPD           108      0.0      1133    40      2.4    1799.6       0.9

TCP-WWW             54      0.0      1133    40      1.2    1799.6       0.8

TCP-SMTP            54      0.0      1133    40      1.2    1799.6       0.8

Verifying that the NetFlow data export is operational

Verify that the NetFlow data export is operational by displaying NetFlow data export statistics.

Use the show ip flow export command to display NetFlow data export statistics, including netflow collector storage’s ip / port udp and statistics for the primary cache and all other enabled caches. Here is an example of this command output:

Router# show ip flow export

Flow export v9 is enabled for main cache

Exporting flows to 172.16.10.2 (9995) (ip/port udp du Netflow collector storage)

Exporting using source interface Ethernet0/0

Version 9 flow records

0 flows exported in 0 udp datagrams

0 flows failed due to lack of export packet

0 export packets were sent up to process level

0 export packets were dropped due to no fib

0 export packets were dropped due to adjacency issues

0 export packets were dropped due to fragmentation failures

0 export packets were dropped due to encapsulation fixup failures

Verifying that the NetFlow data is stored on the Netflow Collector Storage

Connect to the NetFlow Collector Storage, position yourself on the previously created directory corresponding to the interface on which Netflow was activated and check for the presence of files in the format nfcapd.YYYYMMddhhmm (nfcapd.201709181140).

https://coservit.com/servicenav/wp-content/uploads/sites/3/2018/09/Capture.11PNG-300x198.png

If this is the case, the configuration is operational, you can proceed to the next step, otherwise repeat the previous configuration steps.

Monitoring the NetFlow Collector Storage

Netflow Collector Storage is central to your NetFlow architecture, so it’s critical to monitor the load and processes running.

Use the host template System Linux server:

  • CPU
  • LIN-DiskIO
  • LIN-Diskspace
  • LIN-Network_Traffic
  • LIN-RAM
  • LIN-Swap

In addition to these service models already integrated into the host template, use the following service templates:

LIN-DirectorySize to monitor the size of your destination directories.

Lin-ProcessName to monitor the proper execution of nfcapd processes

Finally, use the action templates to restart the nfcapd processes in the event of an interruption by following the procedure below: https://coservit.com/servicenav/en/documentation/using-model-daction/

Configuring the NetworkAnalysis-NetFlow Service Template

In best practice, the NetworkAnalysis-NetFlow service template should be linked to the NetFlow Exporter (the switch or router that exports the NetFlow data) but you can also link it to any other host or System Up if needed.

After you have instantiated the NetworkAnalysis-NetFlow service, you need to configure the service based on the particular analysis requirement. As a reminder, the plugin has been designed to monitor the throughput of an application.

The following mandatory fields must be completed:

  • Collector Storage: Collector Storage Address
  • Bandwidth Allocated: Value in the chosen unit
  • Unit: Output Unit: kbps, Mbps, Gbps
  • Alert threshold: Alert threshold in%
  • Critical threshold: Critical threshold in%
  • Directory Name: The path of the directory containing the exports related to an interface
  • Absence status: Status to give in case of inactivity, eg 0 for OK

The other fields to be informed make it possible to target the flow to be monitored according to the need.

Example of a configuration targeting the throughput generated by a mail server:

The service will provide you with the following information:

  • A status according to the set thresholds
  • The flow rate generated in the chosen unit
  • Performance data
  • Metrics in absolute values and percentage of use

https://coservit.com/servicenav/wp-content/uploads/sites/3/2018/09/Captureaa-300x87.png

Metrics in absolute values:

https://coservit.com/servicenav/wp-content/uploads/sites/3/2018/09/Capturebb-300x203.png

Metrics in %:

https://coservit.com/servicenav/wp-content/uploads/sites/3/2018/09/Captureccc-300x208.png

Dashboard configuration

After you have instantiated as many NetworkAnalysis-NetFlow service as you have streams to monitor, you can create one or more dashboards to display the use of bandwidth, by function, and be alerted according to defined thresholds.

Here is a dashboard example:

https://coservit.com/servicenav/wp-content/uploads/sites/3/2018/09/Capture-3-300x130.png

 

Download Nulled WordPress Themes
Download Premium WordPress Themes Free
Download WordPress Themes
Download Premium WordPress Themes Free
udemy course download free
download micromax firmware
Download Nulled WordPress Themes
lynda course free download

UK ServiceNav Product Development Manager; my priority is to be needful of the particular requirements of all ‘English-speaking’ markets where ServiceNav is sold. I have over 20 years experience of the IT monitoring field - covering a wide variety of products and technologies.