In order for a technology to be fully understood, it needs to be tested and measured. In NFV, verification is needed to check network operator requirements and erroneous configuration of virtualized network functions (VNFs) can cause network infrastructure breakdowns. Robust verification is also an important step towards establishing and maintaining interoperability between NFV hardware and software components.

The Yardstick project is used for verifying OPNFV infrastructure compliance when running VNF applications. The Yardstick framework is deployed in the OPNFV Pharos community test labs and is installer, infrastructure and application independent. The project’s scope is the development of a test framework, test cases and test stimuli to enable Network Function Virtualization Infrastructure (NFVI) verification.

Yardstick implements system-level validation with baseline testing requirements as outlined in the ETSI TST 001 spec where use cases show a large variety of applications–each defining specific requirements and complex configuration of the underlying infrastructure and test tools. The Yardstick concept decomposes typical VNF workload performance metrics into a number of characteristics/performance vectors, which each of them can be represented by distinct test cases.

Project Goals

  • Verify infrastructure compliance, from the perspective of a VNF
  • Help to find infrastructure faults and thresholds at an earlier stage by utilizing simple test cases that examine basic infrastructure metrics
  • Enable verification of more complex test cases by developing functionality to run parallel testing, inject fault, test multiple topologies, and test scenarios

Progress

The Yardstick project now has 45 test cases for NFV infrastructure benchmarks and has integrated with the OPNFV StorPerf project and VSperf project. The Yardstick framework now supports multiple target VMs and automated test workflow enables more efficient resource utilization, integration between the Yardstick and VSPERF and StorPerf projects, additional test cases for security, VPN, IPv6, and other features.

The Yardstick IPv6 test case is now ready for Open Stack Mitaka. Moreover, Yardstick CI status reporting can show the status of the last 10 days’ scenarios for four installers (TripleO, Compass, Fuel, Joid).

In the future, Yardstick will pursue integration with other OPNFV test projects, look to provide REST API support, more generic test cases, and look to make results more meaningful.

Learn More