DISTRIBUTED DATA PROTECTION: New Approaches to an Old Challenge

by Greg Schulz
April 1, 2005

Data compression and network bandwidth acceleration technology, including appliances and software, can help make better use of available network bandwidth. Compressing the data means that either less bandwidth is needed to do a given amount of work in a given timeframe (lower latency or response time), or that more data can be transmitted in a given timeframe.

Data compression is only a partial solution. Most data compression technologies don’t address a root challenge of moving data over distance and that is the underlying protocol efficiency. For example, if you have a 1MB file to move over a network, data compression technology can reduce the amount of actual data transmitted to perhaps 75 percent of its original size. However, the data still needs to be transmitted using multiple packets and data frames. While the size of the packets and frames can be altered, there’s still the “handshake” protocol communication involved with moving the data. When other protocols and applications are layered on top of a network, additional layers of management and data integrity traffic are added. This “chattiness” can end up being a large portion of data transfer time over a network, even with compression.

Data caching isn’t a new technology to enhance performance for data access over distance; in fact, many of the distributed applications and storage have been deployed to act as a form of data cache. Some new technologies such as Wide Area File Services (WAFS) combine multiple techniques to help address data protection and storage access over distance. WAFS solutions improve performance (latency) of remotely accessed, centralized storage using several techniques. Some performance-enhancing techniques of WAFS solutions include:

-        Data caching locally of remote (distributed or centralized) data

-        Bandwidth compression and optimization to improve network utilization

-        Protocol optimization to reduce overhead and bandwidth inefficiency

-        Read-ahead, pre-fetching of data to speed up data access

-        Coordinated file shares and locking for data coherency and consistency

-        File differencing and journaling to track and transmit only changed data.

The result is that data can be accessed faster using less network bandwidth. For some applications, this type of technology can be used to accelerate and improve remote data backup and, for other environments, data traditionally stored on remote file servers could be moved to a centralized file server. A benefit of moving some data to a centralized file server is enhanced data consistency and coherency via coordinated file sharing and locking information.