org.apache.hadoop.mapred

Class MapOutputFile

java.lang.Object

org.apache.hadoop.mapred.MapOutputFile

All Implemented Interfaces:

Configurable

Direct Known Subclasses:

MROutputFiles, YarnOutputFiles

@InterfaceAudience.Private
@InterfaceStability.Unstable
public abstract class MapOutputFile
extends Object
implements Configurable

Manipulate the working area for the transient store for maps and reduces. This class is used by map and reduce tasks to identify the directories that they need to write to/read from for intermediate files. The callers of these methods are from child space and see mapreduce.cluster.local.dir as taskTracker/jobCache/jobId/attemptId This class should not be used from TaskTracker space.

Constructor Summary

Constructors

Constructor and Description
MapOutputFile()

Method Summary

Methods

Modifier and Type	Method and Description
Configuration	getConf() Return the configuration used by this object.
abstract Path	getInputFile(int mapId) Return a local reduce input file created earlier
abstract Path	getInputFileForWrite(TaskID mapId, long size) Create a local reduce input file name.
abstract Path	getOutputFile() Return the path to local map output file created earlier
abstract Path	getOutputFileForWrite(long size) Create a local map output file name.
abstract Path	getOutputFileForWriteInVolume(Path existing) Create a local map output file name on the same volume.
abstract Path	getOutputIndexFile() Return the path to a local map output index file created earlier
abstract Path	getOutputIndexFileForWrite(long size) Create a local map output index file name.
abstract Path	getOutputIndexFileForWriteInVolume(Path existing) Create a local map output index file name on the same volume.
abstract Path	getSpillFile(int spillNumber) Return a local map spill file created earlier.
abstract Path	getSpillFileForWrite(int spillNumber, long size) Create a local map spill file name.
abstract Path	getSpillIndexFile(int spillNumber) Return a local map spill index file created earlier
abstract Path	getSpillIndexFileForWrite(int spillNumber, long size) Create a local map spill index file name.
abstract void	removeAll() Removes all of the files related to a task.
void	setConf(Configuration conf) Set the configuration to be used by this object.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MapOutputFile

public MapOutputFile()

Method Detail

getOutputFile

public abstract Path getOutputFile()
                            throws IOException

Return the path to local map output file created earlier

Returns:

path

Throws:

IOException

getOutputFileForWrite

public abstract Path getOutputFileForWrite(long size)
                                    throws IOException

Create a local map output file name.

Parameters:

size - the size of the file

Returns:

path

Throws:

IOException

getOutputFileForWriteInVolume

public abstract Path getOutputFileForWriteInVolume(Path existing)

Create a local map output file name on the same volume.

getOutputIndexFile

public abstract Path getOutputIndexFile()
                                 throws IOException

Return the path to a local map output index file created earlier

Returns:

path

Throws:

IOException

getOutputIndexFileForWrite

public abstract Path getOutputIndexFileForWrite(long size)
                                         throws IOException

Create a local map output index file name.

Parameters:

size - the size of the file

Returns:

path

Throws:

IOException

getOutputIndexFileForWriteInVolume

public abstract Path getOutputIndexFileForWriteInVolume(Path existing)

Create a local map output index file name on the same volume.

getSpillFile

public abstract Path getSpillFile(int spillNumber)
                           throws IOException

Return a local map spill file created earlier.

Parameters:

spillNumber - the number

Returns:

path

Throws:

IOException

getSpillFileForWrite

public abstract Path getSpillFileForWrite(int spillNumber,
                        long size)
                                   throws IOException

Create a local map spill file name.

Parameters:

spillNumber - the number

size - the size of the file

Returns:

path

Throws:

IOException

getSpillIndexFile

public abstract Path getSpillIndexFile(int spillNumber)
                                throws IOException

Return a local map spill index file created earlier

Parameters:

spillNumber - the number

Returns:

path

Throws:

IOException

getSpillIndexFileForWrite

public abstract Path getSpillIndexFileForWrite(int spillNumber,
                             long size)
                                        throws IOException

Create a local map spill index file name.

Parameters:

spillNumber - the number

size - the size of the file

Returns:

path

Throws:

IOException

getInputFile

public abstract Path getInputFile(int mapId)
                           throws IOException

Return a local reduce input file created earlier

Parameters:

mapId - a map task id

Returns:

path

Throws:

IOException

getInputFileForWrite

public abstract Path getInputFileForWrite(TaskID mapId,
                        long size)
                                   throws IOException

Create a local reduce input file name.

Parameters:

mapId - a map task id

size - the size of the file

Returns:

path

Throws:

IOException

removeAll

public abstract void removeAll()
                        throws IOException

Removes all of the files related to a task.

Throws:

IOException

setConf

public void setConf(Configuration conf)

Description copied from interface: Configurable

Set the configuration to be used by this object.

Specified by:

setConf in interface Configurable

getConf

public Configuration getConf()

Description copied from interface: Configurable

Return the configuration used by this object.

Specified by:

getConf in interface Configurable