SDN Traffic Measurement Solutions: A Brief Overview

October 3, 2019 in Blogs

Traffic measurement is a crucial task in traffic engineering. It includes three main subtasks: network topology measurement, network traffic measurement, and network performance measurement. My authors and I have extensively studied the Topology Measurement Problem in SDN in our papers [1] and [2]. In this short post, I will briefly review the SDN traffic and performance measurement literature.

In order to react to traffic changes, the SDN management applications require accurate and timely statistics on network resources at different aggregation levels (such as flow, packet and port). Hence the SDN controller must continuously monitor traffic counters and performance metrics to quickly adapt forwarding rules in switches.

However, many SDN architectures use traditional monitoring solutions that either require complex additional modules at the switch or impose significant measurement overhead. For instance, Cisco’s NetFlow [3] installs probes at switches as special modules to collect either complete or sampled traffic statistics, and send them to a central collector. Another flow sampling tool is sFlow [4], which samples traffic in a time-based fashion. Another flow sampling tool is Juniper’s JFlow [5] which is quite similar to NetFlow. These approaches introduce significant overhead incurred by statistics collection from the whole network. To cope with this problem, The following solutions were proposed to provide more efficient monitoring with higher accuracy and lower overhead.

PayLess [6] is a query-based monitoring framework designed as a component of the OpenFlow controller. PayLess provides a flexible RESTful API, translating request commands from applications into flow statistics collection at different aggregation levels (such as flow, packet and port), where it performs highly accurate information gathering in real-time without incurring significant network overhead. Instead of making the controller continuously polling switches, PayLess uses an adaptive scheduling algorithm that achieves the same level of accuracy as continuous polling with much less communication overhead. The evaluation results show that PayLess sends only 6.6 monitoring messages per second on average, compared to the controller’s periodic polling, which has an overhead of 13:5 monitoring messages per second on average.

OpenTM [7]is a query-based monitoring system that aims to measure the traffic matrix (TM) for OpenFlow networks. It keeps track of all active flows in the network. First, it gets the routing information, including routing paths, from the routing applications, then it periodically polls flow byte and packet-count counters from switches on each flow path. Using the polled statistics, OpenTM constructs the traffic matrix which represents the added amounts of traffic measured from each source to each destination in the network. In general, the number of network flows is very big, but available measurement resources, namely Ternary Content Addressable Memory (TCAM), are expensive, power hungry and hence limited, so in practice, it is impossible to obtain the traffic matrix by measuring the size of each flow directly. To solve this problem, selective and random switch polling approaches were proposed. Usually, random switch polling is not effective because selected switches might not be of significant importance in term of traffic volume. Selective switch polling however are more efficient.

iSTAMP [8] is an intelligent Traffic (de)Aggregation and Measurement Paradigm. iSTAMP partitions TCAM entries of switches into two parts (1) wildcard rules for aggregate measurements, and (2) fine grained rules to de-aggregate and directly measure the most informative flows for per-flow measurements. iSTAMP then processes these aggregate and per- flow measurements to estimate individual network flows using a variety of optimization techniques. iSTAMP seems to find balance between limitations of network resources and measurement accuracy, however, it does not consider routing and flow aggregation feasibility when designing optimal flow aggregates and only focuses on single-switch scenario. [9] extended iSTAMP framework to multi-switch scenario.

OpenMeasure [10]: assumes that the aggregation matrix is given based on the underlying routing and flow aggregation rules. OpenMeasure leverages the global view of SDN controller to identify the available monitoring resources, employs an online learning algorithm to determine the most informative flows for sampling and places flow sampling rules in selected SDN switches. Furthermore, OpenMeasure is light-weight and compatible with hybrid SDN networks.

In contrast to the on-demand query-based approaches, passive push-based monitoring methods have been proposed to analyze control messages between the controller and switches. These methods use the controller messages to monitor and measure network utilization, such as bandwidth consumption, without inducing additional overhead.

FlowSense [11] analyzes dynamic changes in network flows using messages received by the controller. For example, FlowSense uses PacketIn and FlowRemoved messages in OpenFlows networks to estimate per flow link utilization. The evaluation results show that FlowSense has higher accuracy compared to the request-based methods.

OpenSketch [12] is a software defined traffic measurement architecture, which separates the measurement data plane from the control plane. OpenSketch is generic and designed to allow more customized operations. It can perform efficient data collection with respect to flow selection by using both hashing and wildcard rules. In the data plane, OpenSketch provides a three-stage pipeline (hashing, filtering, and counting), which can be implemented with commodity switch components and support many measurement tasks. OpenSketch provides a library that contains a list of sketches, the sketch manager, and the resource allocator. Sketches can be used for different measurement programs such as heavy hitter detection and fine-grained delay measurement. The skecth library makes measurement programming easier by freeing operators from understanding the complex switch implementations and parameter tuning in diverse sketches. The measurement library automatically configures the data plane pipeline for different sketches and allocates resources (switch memory) across tasks to maximize accuracy.

OpenSample [13] proposed by IBM, is a sampling-based SDN measurement system. Rather than using the expensive OpenFlow rules which, OpenSample leverages sFlow, which is present in most switches, to capture packet header samples from the network with low overhead and uses TCP sequence numbers from the captured headers to measure accurate flow statistics. From random samples, OpenSample can infer a variety of information about the network including the elephant flows and link utilization.

References

[1] Azzouni, Abdelhadi, et al. “Limitations of openflow topology discovery protocol.” arXiv preprint arXiv:1705.00706 (2017).

[2] Azzouni, Abdelhadi, et al. “sOFTDP: Secure and Efficient Topology Discovery Protocol for SDN.” arXiv preprint arXiv:1705.04527 (2017).

[3] http://www.cisco.com/en/US/prod/collateral/iosswrel/ ps6537/ps6555/ps6601/prod_white_paper0900aecd80406232.html

[4] https://sflow.org/sFlowOverview.pdf

[5] A.C. Myers, Jflow: practical mostly-static information flow control, in: Proceedings of the 26th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL’99, January 1999, pp. 228–241

[6] S.R. Chowdhury, M.F. Bari, R. Ahmed, R. Boutaba, Payless: a low cost network monitoring framework for software defined networks, in: Proceedings of the 14th IEEE/IFIP Network Operations and Management Symposium, NOMS’14, May 2014.

[7] A. Tootoonchian, M. Ghobadi, Y. Ganjali, Opentm: traffic matrix estimator for openflow networks, in: Proceedings of the 11th International Conference on Passive and Active Measurement, PAM’10, April 2010, pp. 201–210.

[8] M. Malboubi, L. Wang, C.-N. Chuah, and P. Sharma, “Intelligent SDN based traffic (de) aggregation and measurement paradigm (iSTAMP),” in Proc. IEEE Conf. Comput. Commun. (INFOCOM), Apr./May 2014, pp. 934–942.

[9] Gong, Yanlei, et al. “Towards accurate online traffic matrix estimation in software-defined networks.” Proceedings of the 1st ACM SIGCOMM Symposium on Software Defined Net- working Research. ACM, 2015.

[10] Liu, Chang, AMehdi Malboubi, and Chen-Nee Chuah. “OpenMeasure: Adaptive flow measurement & inference with online learning in SDN.” Computer Communications Workshops (INFOCOM WKSHPS), 2016 IEEE Conference on. IEEE, 2016.

[11] C. Yu, C. Lumezanu, Y. Zhang, V. Singh, G. Jiang, H.V. Madhyastha, Flowsense: monitoring network utilization with zero measurement cost, in: Proceedings of the 14th International Conference on Passive and Active Measurement, PAM’13, March 2013, pp. 31–41.

[12] M. Yu, L. Jose, R. Miao, Software defined traffic measurement with opensketch, in: Proceedings of the 10th USENIX Symposium on Networked Systems Design and Implementation, NSDI’13, vol. 13, April 2013, pp. 29–42.

[13] J. Suh, T. Kwon, C. Dixon, W. Felter, and J. Carter, “Opensample: A low-latency, sampling-based measurement platform for sdn,” IBM Research Report, January 2014.