org.apache.hadoop.hdfs.server.namenode

Class FileJournalManager

java.lang.Object

org.apache.hadoop.hdfs.server.namenode.FileJournalManager

All Implemented Interfaces:

Closeable, AutoCloseable, Storage.FormatConfirmable, JournalManager

@InterfaceAudience.Private
public class FileJournalManager
extends Object
implements JournalManager

Journal manager for the common case of edits files being written to a storage directory. Note: this class is not thread-safe and should be externally synchronized.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	FileJournalManager.EditLogFile Record of an edit log that has been located and had its filename parsed.

Nested classes/interfaces inherited from interface org.apache.hadoop.hdfs.server.namenode.JournalManager

JournalManager.CorruptionException

Constructor Summary

Constructors

Constructor and Description
FileJournalManager(Configuration conf, Storage.StorageDirectory sd, StorageErrorReporter errorReporter)

Method Summary

Methods

Modifier and Type	Method and Description
void	close() Close the journal manager, freeing any resources it may hold.
void	finalizeLogSegment(long firstTxId, long lastTxId) Mark the log segment that spans from firstTxId to lastTxId as finalized and complete.
void	format(NamespaceInfo ns) Format the underlying storage, removing any previously stored data.
static FileJournalManager.EditLogFile	getLogFile(File dir, long startTxId)
FileJournalManager.EditLogFile	getLogFile(long startTxId)
List<FileJournalManager.EditLogFile>	getLogFiles(long fromTxId)
List<RemoteEditLog>	getRemoteEditLogs(long firstTxId, boolean forReading, boolean inProgressOk) Find all editlog segments starting at or above the given txid.
Storage.StorageDirectory	getStorageDirectory()
boolean	hasSomeData()
static List<FileJournalManager.EditLogFile>	matchEditLogs(File logDir) returns matching edit logs via the log directory.
void	purgeLogsOlderThan(long minTxIdToKeep) Remove all edit logs with transaction IDs lower than the given transaction ID.
void	recoverUnfinalizedSegments() Recover segments which have not been finalized.
void	selectInputStreams(Collection<EditLogInputStream> streams, long fromTxId, boolean inProgressOk, boolean forReading) Get a list of edit log input streams.
void	setOutputBufferCapacity(int size) Set the amount of memory that this stream should use to buffer edits
EditLogOutputStream	startLogSegment(long txid) Begin writing to a new segment of the log stream, which starts at the given transaction ID.
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

FileJournalManager

public FileJournalManager(Configuration conf,
                  Storage.StorageDirectory sd,
                  StorageErrorReporter errorReporter)

Method Detail

close

public void close()
           throws IOException

Description copied from interface: JournalManager

Close the journal manager, freeing any resources it may hold.

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Specified by:

close in interface JournalManager

Throws:

IOException

format

public void format(NamespaceInfo ns)
            throws IOException

Description copied from interface: JournalManager

Format the underlying storage, removing any previously stored data.

Specified by:

format in interface JournalManager

Throws:

IOException

hasSomeData

public boolean hasSomeData()

Specified by:

hasSomeData in interface Storage.FormatConfirmable

Returns:

true if the storage seems to have some valid data in it, and the user should be required to confirm the format. Otherwise, false.

startLogSegment

public EditLogOutputStream startLogSegment(long txid)
                                    throws IOException

Description copied from interface: JournalManager

Begin writing to a new segment of the log stream, which starts at the given transaction ID.

Specified by:

startLogSegment in interface JournalManager

Throws:

IOException

finalizeLogSegment

public void finalizeLogSegment(long firstTxId,
                      long lastTxId)
                        throws IOException

Description copied from interface: JournalManager

Mark the log segment that spans from firstTxId to lastTxId as finalized and complete.

Specified by:

finalizeLogSegment in interface JournalManager

Throws:

IOException

getStorageDirectory

public Storage.StorageDirectory getStorageDirectory()

setOutputBufferCapacity

public void setOutputBufferCapacity(int size)

Description copied from interface: JournalManager

Set the amount of memory that this stream should use to buffer edits

Specified by:

setOutputBufferCapacity in interface JournalManager

purgeLogsOlderThan

public void purgeLogsOlderThan(long minTxIdToKeep)
                        throws IOException

Remove all edit logs with transaction IDs lower than the given transaction ID.

Parameters:

minTxIdToKeep - the lowest transaction ID that should be retained

Throws:

IOException - in the event of error

getRemoteEditLogs

public List<RemoteEditLog> getRemoteEditLogs(long firstTxId,
                                    boolean forReading,
                                    boolean inProgressOk)
                                      throws IOException

Find all editlog segments starting at or above the given txid.

Parameters:

fromTxId - the txnid which to start looking

forReading - whether or not the caller intends to read from the edit logs

inProgressOk - whether or not to include the in-progress edit log segment

Returns:

a list of remote edit logs

Throws:

IOException - if edit logs cannot be listed.

matchEditLogs

public static List<FileJournalManager.EditLogFile> matchEditLogs(File logDir)
                                                          throws IOException

returns matching edit logs via the log directory. Simple helper function that lists the files in the logDir and calls matchEditLogs(File[])

Parameters:

logDir - directory to match edit logs in

Returns:

matched edit logs

Throws:

IOException - IOException thrown for invalid logDir

selectInputStreams

public void selectInputStreams(Collection<EditLogInputStream> streams,
                      long fromTxId,
                      boolean inProgressOk,
                      boolean forReading)
                        throws IOException

Get a list of edit log input streams. The list will start with the stream that contains fromTxnId, and continue until the end of the journal being managed.

fromTxId - the first transaction id we want to read

inProgressOk - whether or not in-progress streams should be returned

forReading - whether or not the caller intends to read from the edit logs

Throws:

IOException - if the underlying storage has an error or is otherwise inaccessible

recoverUnfinalizedSegments

public void recoverUnfinalizedSegments()
                                throws IOException

Description copied from interface: JournalManager

Recover segments which have not been finalized.

Specified by:

recoverUnfinalizedSegments in interface JournalManager

Throws:

IOException

getLogFiles

public List<FileJournalManager.EditLogFile> getLogFiles(long fromTxId)
                                                 throws IOException

Throws:

IOException

getLogFile

public FileJournalManager.EditLogFile getLogFile(long startTxId)
                                          throws IOException

Throws:

IOException

getLogFile

public static FileJournalManager.EditLogFile getLogFile(File dir,
                                        long startTxId)
                                                 throws IOException

Throws:

IOException

toString

public String toString()

Specified by:

toString in interface Storage.FormatConfirmable

Overrides:

toString in class Object

Returns:

a string representation of the formattable item, suitable for display to the user inside a prompt