Packet Deduplication Generator Tap For Network Visibility And Data Optimization

Packet Deduplication Generator TAP for Network Visibility and Data Optimization

Data De-duplication is a popular and popular storage technology that optimizes storage capacity.It eliminates redundant data by removing duplicate data from the dataset, leaving only one copy.As shown in the figure below.This technology can greatly reduce the need for physical storage space to meet the growing demand for data storage.Dedupe technology can bring many practical benefits, mainly including the following aspects:

Dedupe technology is widely used in data backup and archiving systems, because there are a lot of duplicate data after multiple backups of data, which is very suitable for this technology.In fact, dedupe technology can be used in many situations, including online data, near-line data, and offline data storage systems. It can be implemented in file systems, volume managers, NAS, and sans.Dedupe can also be used for data disaster recovery, data transmission and synchronization, as a data compression technology can be used for data packaging.Dedupe technology can help many applications reduce data storage, save network bandwidth, improve storage efficiency, reduce the backup window, and save costs.

Dedupe has two main dimensions: deduplocation ratios and performance.Dedupe performance depends on the specific implementation technology, while Dedupe rate is determined by the characteristics of the data itself and application patterns, as shown in the table below.Storage vendors currently report deduplication rates ranging from 20:1 to 500:1.

Dedupe Implementation Points

Various factors should be considered when developing or applying Dedupe technology, as these factors directly affect its performance and effectiveness.

Dedupe Key Technology

Deduplication process of storage system in general is this: first of all the data file is divided into a set of data, for each block of data to calculate the fingerprint, and then based on fingerprint Hash search keywords, matching indicates the data for the duplicate data blocks, only stores data block index number, otherwise it means the data block is the only piece of a new, storage of data block and create relevant meta information.Thus, a physical file in the storage system corresponds to a logical representation of a set of FP metadata.When reading the file, first read the logical file, then according to the FP sequence, take out the corresponding data block from the storage system, restore the copy of the physical file.It can be seen from the above process that the key technologies of Dedupe mainly include file data block segmentation, data block fingerprint calculation and data block retrieval.

(1) File data block segmentation

(2) Data block fingerprint calculation

(3) Data block retrieval

What's the Network Packet Broker and Network Taps to do?
With the acceleration of carrier broadband construction and the rapid development of 3G, 4G, and 5G networks, and with the popularity and development of big data and cloud computing technologies, data traffic is growing, and the bandwidth requirements for backbone networks and core networks are increasing. Since 2013, the major operators have begun to gradually upgrade the backbone network. The core backbone network transmission link media starts to be expanded from 10GPOS and 40GPOS links to 100G Ethernet links. It will inevitably lead to the continuous updating of big data collection and analysis technology. In the face of 100GE link environment with high bandwidth capacity, how to effectively meet the requirements of various national security, network security monitoring, operator intelligent pipeline DPI data collection and analysis will become a hot spot in the field of data acquisition and analysis.
Chengdu Shuwei Communication is following the development direction of the Internet and develop the MATRIX-TCA-CG Traffic Acquisition(Traffic Capture) Visibility System, and dedicated to the traffic collection and visual analysis application requirements on 1GE, 10GE, 10G/40G POS, 40GE and 100GE links. The MATRIX-TCA-CG integrates free-steering output functions such as network flow collection/acquisition, aggregation, filtering, forwarding/distribution, and load balancing and provides an efficient solution for the flow analysis.
 
Matrix-TCA-CG is a network data visualization control device for high density and large flow of 10G/40G/100G.
Matrix-TCA-CG is specially designed for the data collection of 10GE 40GE and 100GE links distributed intensively in operators mobile Internet export, IDC export, provincial network and backbone.
Based on ATCA standard architecture, MATRIX-TCA-CG has high carrier-level reliability and strong expansibility which can smoothly configure the functional board CARDS to cope with port demand scenarios of various specifications and sizes.

  
SDN Network Packet Broker Applications:

After sorting out the main participating entities of the network, the application scenarios of SDN basically focus on telecom operators, government and enterprise customers, data center service providers and Internet companies.The application scenarios of SDN mainly focus on: data center network, interconnection between data centers, government-enterprise network, telecom operator network, and business deployment of Internet companies.

Network Packet Broker Specification and Module Type

NT-FTAP-32QCX Network TAP NPB.pdf