org.apache.hadoop.hdfs.qjournal.client

Class QuorumJournalManager

java.lang.Object

org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager

All Implemented Interfaces:

Closeable, AutoCloseable, Storage.FormatConfirmable, JournalManager

@InterfaceAudience.Private
public class QuorumJournalManager
extends Object
implements JournalManager

A JournalManager that writes to a set of remote JournalNodes, requiring a quorum of nodes to ack each write.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.hdfs.server.namenode.JournalManager

JournalManager.CorruptionException

Constructor Summary

Constructors

Constructor and Description
QuorumJournalManager(Configuration conf, URI uri, NamespaceInfo nsInfo)

Method Summary

Methods

Modifier and Type	Method and Description
static void	checkJournalId(String jid)
void	close() Close the journal manager, freeing any resources it may hold.
protected List<org.apache.hadoop.hdfs.qjournal.client.AsyncLogger>	createLoggers(AsyncLogger.Factory factory)
void	finalizeLogSegment(long firstTxId, long lastTxId) Mark the log segment that spans from firstTxId to lastTxId as finalized and complete.
void	format(NamespaceInfo nsInfo) Format the underlying storage, removing any previously stored data.
boolean	hasSomeData()
void	purgeLogsOlderThan(long minTxIdToKeep) Remove all edit logs with transaction IDs lower than the given transaction ID.
void	recoverUnfinalizedSegments() Recover segments which have not been finalized.
void	selectInputStreams(Collection<EditLogInputStream> streams, long fromTxnId, boolean inProgressOk)
void	selectInputStreams(Collection<EditLogInputStream> streams, long fromTxnId, boolean inProgressOk, boolean forReading) Get a list of edit log input streams.
void	setOutputBufferCapacity(int size) Set the amount of memory that this stream should use to buffer edits
EditLogOutputStream	startLogSegment(long txId) Begin writing to a new segment of the log stream, which starts at the given transaction ID.
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

QuorumJournalManager

public QuorumJournalManager(Configuration conf,
                    URI uri,
                    NamespaceInfo nsInfo)
                     throws IOException

Throws:

IOException

Method Detail

createLoggers

protected List<org.apache.hadoop.hdfs.qjournal.client.AsyncLogger> createLoggers(AsyncLogger.Factory factory)
                                                                          throws IOException

Throws:

IOException

checkJournalId

public static void checkJournalId(String jid)

format

public void format(NamespaceInfo nsInfo)
            throws IOException

Description copied from interface: JournalManager

Format the underlying storage, removing any previously stored data.

Specified by:

format in interface JournalManager

Throws:

IOException

hasSomeData

public boolean hasSomeData()
                    throws IOException

Specified by:

hasSomeData in interface Storage.FormatConfirmable

Returns:

true if the storage seems to have some valid data in it, and the user should be required to confirm the format. Otherwise, false.

Throws:

IOException - if the storage cannot be accessed at all.

startLogSegment

public EditLogOutputStream startLogSegment(long txId)
                                    throws IOException

Description copied from interface: JournalManager

Begin writing to a new segment of the log stream, which starts at the given transaction ID.

Specified by:

startLogSegment in interface JournalManager

Throws:

IOException

finalizeLogSegment

public void finalizeLogSegment(long firstTxId,
                      long lastTxId)
                        throws IOException

Description copied from interface: JournalManager

Mark the log segment that spans from firstTxId to lastTxId as finalized and complete.

Specified by:

finalizeLogSegment in interface JournalManager

Throws:

IOException

setOutputBufferCapacity

public void setOutputBufferCapacity(int size)

Description copied from interface: JournalManager

Set the amount of memory that this stream should use to buffer edits

Specified by:

setOutputBufferCapacity in interface JournalManager

purgeLogsOlderThan

public void purgeLogsOlderThan(long minTxIdToKeep)
                        throws IOException

Remove all edit logs with transaction IDs lower than the given transaction ID.

Parameters:

minTxIdToKeep - the lowest transaction ID that should be retained

Throws:

IOException - in the event of error

recoverUnfinalizedSegments

public void recoverUnfinalizedSegments()
                                throws IOException

Description copied from interface: JournalManager

Recover segments which have not been finalized.

Specified by:

recoverUnfinalizedSegments in interface JournalManager

Throws:

IOException

close

public void close()
           throws IOException

Description copied from interface: JournalManager

Close the journal manager, freeing any resources it may hold.

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Specified by:

close in interface JournalManager

Throws:

IOException

selectInputStreams

public void selectInputStreams(Collection<EditLogInputStream> streams,
                      long fromTxnId,
                      boolean inProgressOk)
                        throws IOException

Throws:

IOException

selectInputStreams

public void selectInputStreams(Collection<EditLogInputStream> streams,
                      long fromTxnId,
                      boolean inProgressOk,
                      boolean forReading)
                        throws IOException

Get a list of edit log input streams. The list will start with the stream that contains fromTxnId, and continue until the end of the journal being managed.

fromTxnId - the first transaction id we want to read

inProgressOk - whether or not in-progress streams should be returned

forReading - whether or not the caller intends to read from the edit logs

Throws:

IOException - if the underlying storage has an error or is otherwise inaccessible

toString

public String toString()

Specified by:

toString in interface Storage.FormatConfirmable

Overrides:

toString in class Object

Returns:

a string representation of the formattable item, suitable for display to the user inside a prompt