org.apache.hadoop.fs

Class FSOutputSummer

java.lang.Object

java.io.OutputStream

org.apache.hadoop.fs.FSOutputSummer

All Implemented Interfaces:

Closeable, Flushable, AutoCloseable

Direct Known Subclasses:

DFSOutputStream

@InterfaceAudience.LimitedPrivate(value="HDFS")
@InterfaceStability.Unstable
public abstract class FSOutputSummer
extends OutputStream

This is a generic output stream for generating checksums for data before it is written to the underlying stream

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	FSOutputSummer(Checksum sum, int maxChunkSize, int checksumSize)

Method Summary

Methods

Modifier and Type	Method and Description
protected abstract void	checkClosed() Check if the implementing OutputStream is closed and should no longer accept writes.
static byte[]	convertToByteStream(Checksum sum, int checksumSize) Converts a checksum integer value to a byte stream
protected void	flushBuffer()
protected void	flushBuffer(boolean keep)
protected int	getBufferedDataSize() Return the number of valid bytes currently in the buffer.
protected void	resetChecksumChunk(int size) Resets existing buffer with a new one of the specified size.
void	write(byte[] b, int off, int len) Writes len bytes from the specified byte array starting at offset off and generate a checksum for each data chunk.
void	write(int b) Write one byte
protected abstract void	writeChunk(byte[] b, int offset, int len, byte[] checksum)

Methods inherited from class java.io.OutputStream

close, flush, write

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FSOutputSummer

protected FSOutputSummer(Checksum sum,
              int maxChunkSize,
              int checksumSize)

Method Detail

writeChunk

protected abstract void writeChunk(byte[] b,
              int offset,
              int len,
              byte[] checksum)
                            throws IOException

Throws:

IOException

checkClosed

protected abstract void checkClosed()
                             throws IOException

Check if the implementing OutputStream is closed and should no longer accept writes. Implementations should do nothing if this stream is not closed, and should throw an IOException if it is closed.

Throws:

IOException - if this stream is already closed.

write

public void write(int b)
           throws IOException

Write one byte

Specified by:

write in class OutputStream

Throws:

IOException

write

public void write(byte[] b,
         int off,
         int len)
           throws IOException

Writes len bytes from the specified byte array starting at offset off and generate a checksum for each data chunk.

This method stores bytes from the given array into this stream's buffer before it gets checksumed. The buffer gets checksumed and flushed to the underlying output stream when all data in a checksum chunk are in the buffer. If the buffer is empty and requested length is at least as large as the size of next checksum chunk size, this method will checksum and write the chunk directly to the underlying output stream. Thus it avoids uneccessary data copy.

Overrides:

write in class OutputStream

Parameters:

b - the data.

off - the start offset in the data.

len - the number of bytes to write.

Throws:

IOException - if an I/O error occurs.

flushBuffer

protected void flushBuffer()
                    throws IOException

Throws:

IOException

flushBuffer

protected void flushBuffer(boolean keep)
                    throws IOException

Throws:

IOException

getBufferedDataSize

protected int getBufferedDataSize()

Return the number of valid bytes currently in the buffer.

convertToByteStream

public static byte[] convertToByteStream(Checksum sum,
                         int checksumSize)

Converts a checksum integer value to a byte stream

resetChecksumChunk

protected void resetChecksumChunk(int size)

Resets existing buffer with a new one of the specified size.