org.apache.hadoop.mapreduce.v2.app.job.impl

Class JobImpl

java.lang.Object

org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl

All Implemented Interfaces:

Job, EventHandler<JobEvent>

public class JobImpl
extends Object
implements Job, EventHandler<JobEvent>

Implementation of Job interface. Maintains the state machines of Job. The read and write calls use ReadWriteLock for concurrency.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	JobImpl.InitTransition
static class	JobImpl.StartTransition

Field Summary

Fields

Modifier and Type	Field and Description
JobConf	conf
Path	remoteJobConfFile
protected static StateMachineFactory<JobImpl,JobStateInternal,JobEventType,JobEvent>	stateMachineFactory

Constructor Summary

Constructors

Constructor and Description

JobImpl(JobId jobId, ApplicationAttemptId applicationAttemptId, Configuration conf, EventHandler eventHandler, TaskAttemptListener taskAttemptListener, JobTokenSecretManager jobTokenSecretManager, Credentials jobCredentials, Clock clock, Map<TaskId,JobHistoryParser.TaskInfo> completedTasksFromPreviousRun, MRAppMetrics metrics, OutputCommitter committer, boolean newApiCommitter, String userName, long appSubmitTime, List<AMInfo> amInfos, AppContext appContext, JobStateInternal forcedState, String forcedDiagnostic)

Method Summary

Methods

Modifier and Type	Method and Description
protected void	addDiagnostic(String diag)
protected void	addTask(Task task)
boolean	checkAccess(UserGroupInformation callerUGI, JobACL jobOperation)
protected JobStateInternal	checkReadyForCommit()
void	constructFinalFullcounters()
static String	escapeString(String data)
Counters	getAllCounters() Get all the counters of this job.
List<AMInfo>	getAMInfos()
int	getCompletedMaps()
int	getCompletedReduces()
Path	getConfFile()
List<String>	getDiagnostics()
protected FileSystem	getFileSystem(Configuration conf) Create the default file System for this job.
JobId	getID()
JobStateInternal	getInternalState()
Map<JobACL,AccessControlList>	getJobACLs()
TaskCompletionEvent[]	getMapAttemptCompletionEvents(int startIndex, int maxEvents)
String	getName()
float	getProgress()
String	getQueueName()
JobReport	getReport()
JobState	getState()
protected StateMachine<JobStateInternal,JobEventType,JobEvent>	getStateMachine()
Task	getTask(TaskId taskID)
TaskAttemptCompletionEvent[]	getTaskAttemptCompletionEvents(int fromEventId, int maxEvents)
Map<TaskId,Task>	getTasks()
Map<TaskId,Task>	getTasks(TaskType taskType)
int	getTotalMaps()
int	getTotalReduces()
String	getUserName()
void	handle(JobEvent event)
static Counters	incrTaskCounters(Counters counters, Collection<Task> tasks)
boolean	isUber()
Configuration	loadConfFile()
protected void	scheduleTasks(Set<TaskId> taskIDs, boolean recoverTaskOutput)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

conf

public JobConf conf

remoteJobConfFile

public Path remoteJobConfFile

stateMachineFactory

protected static final StateMachineFactory<JobImpl,JobStateInternal,JobEventType,JobEvent> stateMachineFactory

Constructor Detail

JobImpl

public JobImpl(JobId jobId,
       ApplicationAttemptId applicationAttemptId,
       Configuration conf,
       EventHandler eventHandler,
       TaskAttemptListener taskAttemptListener,
       JobTokenSecretManager jobTokenSecretManager,
       Credentials jobCredentials,
       Clock clock,
       Map<TaskId,JobHistoryParser.TaskInfo> completedTasksFromPreviousRun,
       MRAppMetrics metrics,
       OutputCommitter committer,
       boolean newApiCommitter,
       String userName,
       long appSubmitTime,
       List<AMInfo> amInfos,
       AppContext appContext,
       JobStateInternal forcedState,
       String forcedDiagnostic)

Method Detail

getStateMachine

protected StateMachine<JobStateInternal,JobEventType,JobEvent> getStateMachine()

getID

public JobId getID()

Specified by:

getID in interface Job

checkAccess

public boolean checkAccess(UserGroupInformation callerUGI,
                  JobACL jobOperation)

Specified by:

checkAccess in interface Job

getTask

public Task getTask(TaskId taskID)

Specified by:

getTask in interface Job

getCompletedMaps

public int getCompletedMaps()

Specified by:

getCompletedMaps in interface Job

getCompletedReduces

public int getCompletedReduces()

Specified by:

getCompletedReduces in interface Job

isUber

public boolean isUber()

Specified by:

isUber in interface Job

getAllCounters

public Counters getAllCounters()

Description copied from interface: Job

Get all the counters of this job. This includes job-counters aggregated together with the counters of each task. This creates a clone of the Counters, so use this judiciously.

Specified by:

getAllCounters in interface Job

Returns:

job-counters and aggregate task-counters

incrTaskCounters

public static Counters incrTaskCounters(Counters counters,
                        Collection<Task> tasks)

getTaskAttemptCompletionEvents

public TaskAttemptCompletionEvent[] getTaskAttemptCompletionEvents(int fromEventId,
                                                          int maxEvents)

Specified by:

getTaskAttemptCompletionEvents in interface Job

getMapAttemptCompletionEvents

public TaskCompletionEvent[] getMapAttemptCompletionEvents(int startIndex,
                                                  int maxEvents)

Specified by:

getMapAttemptCompletionEvents in interface Job

getDiagnostics

public List<String> getDiagnostics()

Specified by:

getDiagnostics in interface Job

getReport

public JobReport getReport()

Specified by:

getReport in interface Job

getProgress

public float getProgress()

Specified by:

getProgress in interface Job

getTasks

public Map<TaskId,Task> getTasks()

Specified by:

getTasks in interface Job

getTasks

public Map<TaskId,Task> getTasks(TaskType taskType)

Specified by:

getTasks in interface Job

getState

public JobState getState()

Specified by:

getState in interface Job

scheduleTasks

protected void scheduleTasks(Set<TaskId> taskIDs,
                 boolean recoverTaskOutput)

handle

public void handle(JobEvent event)

Specified by:

handle in interface EventHandler<JobEvent>

getInternalState

@InterfaceAudience.Private
public JobStateInternal getInternalState()

addTask

protected void addTask(Task task)

getFileSystem

protected FileSystem getFileSystem(Configuration conf)
                            throws IOException

Create the default file System for this job.

Parameters:

conf - the conf object

Returns:

the default filesystem for this job

Throws:

IOException

checkReadyForCommit

protected JobStateInternal checkReadyForCommit()

getUserName

public String getUserName()

Specified by:

getUserName in interface Job

getQueueName

public String getQueueName()

Specified by:

getQueueName in interface Job

getConfFile

public Path getConfFile()

Specified by:

getConfFile in interface Job

Returns:

a path to where the config file for this job is located.

getName

public String getName()

Specified by:

getName in interface Job

getTotalMaps

public int getTotalMaps()

Specified by:

getTotalMaps in interface Job

getTotalReduces

public int getTotalReduces()

Specified by:

getTotalReduces in interface Job

getJobACLs

public Map<JobACL,AccessControlList> getJobACLs()

Specified by:

getJobACLs in interface Job

Returns:

the ACLs for this job for each type of JobACL given.

getAMInfos

public List<AMInfo> getAMInfos()

Specified by:

getAMInfos in interface Job

Returns:

information for MR AppMasters (previously failed and current)

escapeString

public static String escapeString(String data)

constructFinalFullcounters

@InterfaceAudience.Private
public void constructFinalFullcounters()

addDiagnostic

protected void addDiagnostic(String diag)

loadConfFile

public Configuration loadConfFile()
                           throws IOException

Specified by:

loadConfFile in interface Job

Returns:

a parsed version of the config files pointed to by Job.getConfFile().

Throws:

IOException - on any error trying to load the conf file.