org.apache.hadoop.hdfs.protocol.datatransfer

Interface DataTransferProtocol

All Known Implementing Classes:

Receiver, Sender

@InterfaceAudience.Private
@InterfaceStability.Evolving
public interface DataTransferProtocol

Transfer data to/from datanode using a streaming protocol.

Field Summary

Fields

Modifier and Type	Field and Description
static int	DATA_TRANSFER_VERSION Version for data transfers between clients and datanodes This should change when serialization of DatanodeInfo, not just when protocol changes.
static org.apache.commons.logging.Log	LOG

Method Summary

Methods

Modifier and Type	Method and Description
void	blockChecksum(ExtendedBlock blk, Token<BlockTokenIdentifier> blockToken) Get block checksum (MD5 of CRC32).
void	copyBlock(ExtendedBlock blk, Token<BlockTokenIdentifier> blockToken) Copy a block.
void	readBlock(ExtendedBlock blk, Token<BlockTokenIdentifier> blockToken, String clientName, long blockOffset, long length, boolean sendChecksum, CachingStrategy cachingStrategy) Read a block.
void	replaceBlock(ExtendedBlock blk, Token<BlockTokenIdentifier> blockToken, String delHint, DatanodeInfo source) Receive a block from a source datanode and then notifies the namenode to remove the copy from the original datanode.
void	requestShortCircuitFds(ExtendedBlock blk, Token<BlockTokenIdentifier> blockToken, int maxVersion) Request short circuit access file descriptors from a DataNode.
void	transferBlock(ExtendedBlock blk, Token<BlockTokenIdentifier> blockToken, String clientName, DatanodeInfo[] targets) Transfer a block to another datanode.
void	writeBlock(ExtendedBlock blk, Token<BlockTokenIdentifier> blockToken, String clientName, DatanodeInfo[] targets, DatanodeInfo source, BlockConstructionStage stage, int pipelineSize, long minBytesRcvd, long maxBytesRcvd, long latestGenerationStamp, DataChecksum requestedChecksum, CachingStrategy cachingStrategy) Write a block to a datanode pipeline.

Field Detail

LOG

static final org.apache.commons.logging.Log LOG

DATA_TRANSFER_VERSION

static final int DATA_TRANSFER_VERSION

Version for data transfers between clients and datanodes This should change when serialization of DatanodeInfo, not just when protocol changes. It is not very obvious.

See Also:

Constant Field Values

Method Detail

readBlock

void readBlock(ExtendedBlock blk,
             Token<BlockTokenIdentifier> blockToken,
             String clientName,
             long blockOffset,
             long length,
             boolean sendChecksum,
             CachingStrategy cachingStrategy)
               throws IOException

Read a block.

Parameters:

blk - the block being read.

blockToken - security token for accessing the block.

clientName - client's name.

blockOffset - offset of the block.

length - maximum number of bytes for this read.

sendChecksum - if false, the DN should skip reading and sending checksums

cachingStrategy - The caching strategy to use.

Throws:

IOException

writeBlock

void writeBlock(ExtendedBlock blk,
              Token<BlockTokenIdentifier> blockToken,
              String clientName,
              DatanodeInfo[] targets,
              DatanodeInfo source,
              BlockConstructionStage stage,
              int pipelineSize,
              long minBytesRcvd,
              long maxBytesRcvd,
              long latestGenerationStamp,
              DataChecksum requestedChecksum,
              CachingStrategy cachingStrategy)
                throws IOException

Write a block to a datanode pipeline.

Parameters:

blk - the block being written.

blockToken - security token for accessing the block.

clientName - client's name.

targets - target datanodes in the pipeline.

source - source datanode.

stage - pipeline stage.

pipelineSize - the size of the pipeline.

minBytesRcvd - minimum number of bytes received.

maxBytesRcvd - maximum number of bytes received.

latestGenerationStamp - the latest generation stamp of the block.

Throws:

IOException

transferBlock

void transferBlock(ExtendedBlock blk,
                 Token<BlockTokenIdentifier> blockToken,
                 String clientName,
                 DatanodeInfo[] targets)
                   throws IOException

Transfer a block to another datanode. The block stage must be either BlockConstructionStage.TRANSFER_RBW or BlockConstructionStage.TRANSFER_FINALIZED.

Parameters:

blk - the block being transferred.

blockToken - security token for accessing the block.

clientName - client's name.

targets - target datanodes.

Throws:

IOException

requestShortCircuitFds

void requestShortCircuitFds(ExtendedBlock blk,
                          Token<BlockTokenIdentifier> blockToken,
                          int maxVersion)
                            throws IOException

Request short circuit access file descriptors from a DataNode.

Parameters:

blk - The block to get file descriptors for.

blockToken - Security token for accessing the block.

maxVersion - Maximum version of the block data the client can understand.

Throws:

IOException

replaceBlock

void replaceBlock(ExtendedBlock blk,
                Token<BlockTokenIdentifier> blockToken,
                String delHint,
                DatanodeInfo source)
                  throws IOException

Receive a block from a source datanode and then notifies the namenode to remove the copy from the original datanode. Note that the source datanode and the original datanode can be different. It is used for balancing purpose.

Parameters:

blk - the block being replaced.

blockToken - security token for accessing the block.

delHint - the hint for deleting the block in the original datanode.

source - the source datanode for receiving the block.

Throws:

IOException

copyBlock

void copyBlock(ExtendedBlock blk,
             Token<BlockTokenIdentifier> blockToken)
               throws IOException

Copy a block. It is used for balancing purpose.

Parameters:

blk - the block being copied.

blockToken - security token for accessing the block.

Throws:

IOException

blockChecksum

void blockChecksum(ExtendedBlock blk,
                 Token<BlockTokenIdentifier> blockToken)
                   throws IOException

Get block checksum (MD5 of CRC32).

Parameters:

blk - a block.

blockToken - security token for accessing the block.

Throws:

IOException