p2pMapReduce.mapreduceModule
Class JobContextImpl

java.lang.Object
  extended by p2pMapReduce.mapreduceModule.JobContextImpl
All Implemented Interfaces:
JobContext, MRJobConfig
Direct Known Subclasses:
Job, TaskAttemptContextImpl

public class JobContextImpl
extends java.lang.Object
implements JobContext

A read-only view of the job that is provided to the tasks while they are running.


Field Summary
protected  JobConf conf
           
 
Fields inherited from interface p2pMapReduce.mapreduceModule.MRJobConfig
CACHE_ARCHIVES, CACHE_ARCHIVES_SIZES, CACHE_ARCHIVES_TIMESTAMPS, CACHE_ARCHIVES_VISIBILITIES, CACHE_FILE_TIMESTAMPS, CACHE_FILE_VISIBILITIES, CACHE_FILES, CACHE_FILES_SIZES, CACHE_LOCALARCHIVES, CACHE_LOCALFILES, CACHE_SYMLINK, CLASSPATH_ARCHIVES, CLASSPATH_FILES, COMBINE_CLASS_ATTR, COMPLETED_MAPS_FOR_REDUCE_SLOWSTART, END_NOTIFICATION_RETRIE_INTERVAL, END_NOTIFICATION_RETRIES, END_NOTIFICATION_URL, GROUP_COMPARATOR_CLASS, HISTORY_LOCATION, ID, INPUT_FORMAT_CLASS_ATTR, INPUTSPLIT_CLASS_ATTR, INPUTSPLIT_SIZE, IO_SORT_FACTOR, IO_SORT_MB, JAR, JAR_UNPACK_PATTERN, JOB_ACL_MODIFY_JOB, JOB_ACL_VIEW_JOB, JOB_CANCEL_DELEGATION_TOKEN, JOB_DIR_ATTR, JOB_INPUT_DIR_ATTR, JOB_INPUTSPLIT_DIR_ATTR, JOB_JAR_DIR_ATTR, JOB_JOBTRACKER_ID, JOB_LOCAL_DIR, JOB_NAME, JOB_NAMENODES, JOB_OUTPUT_DIR_ATTR, JVM_NUMTASKS_TORUN, KEY_COMPARATOR, MAP_CLASS_ATTR, MAP_COMBINE_MIN_SPILLS, MAP_DEBUG_SCRIPT, MAP_ENV, MAP_FAILURES_MAX_PERCENT, MAP_INPUT_FILE, MAP_INPUT_PATH, MAP_INPUT_START, MAP_JAVA_OPTS, MAP_LOG_LEVEL, MAP_MAX_ATTEMPTS, MAP_MEMORY_MB, MAP_MEMORY_PHYSICAL_MB, MAP_OUTPUT_COMPRESS, MAP_OUTPUT_COMPRESS_CODEC, MAP_OUTPUT_DIR_ATTR, MAP_OUTPUT_KEY_CLASS, MAP_OUTPUT_KEY_FIELD_SEPERATOR, MAP_OUTPUT_KEY_VALUE_SEPARATOR_ATTR, MAP_OUTPUT_VALUE_CLASS, MAP_SKIP_INCR_PROC_COUNT, MAP_SKIP_MAX_RECORDS, MAP_SORT_SPILL_PERCENT, MAP_SPECULATIVE, MAP_ULIMIT, MAX_TASK_FAILURES_PER_TRACKER, NUM_MAP_PROFILES, NUM_MAPS, NUM_REDUCE_PROFILES, NUM_REDUCES, OUTPUT_FORMAT_CLASS_ATTR, OUTPUT_KEY_CLASS, OUTPUT_VALUE_CLASS, PARTITIONER_CLASS_ATTR, PRESERVE_FAILED_TASK_FILES, PRESERVE_FILES_PATTERN, PRIORITY, QUEUE_NAME, RECORDS_BEFORE_PROGRESS, REDUCE_CLASS_ATTR, REDUCE_DEBUG_SCRIPT, REDUCE_ENV, REDUCE_FAILURES_MAXPERCENT, REDUCE_INPUT_BUFFER_PERCENT, REDUCE_INPUT_DIR_ATTR, REDUCE_JAVA_OPTS, REDUCE_LOG_LEVEL, REDUCE_MARKRESET_BUFFER_PERCENT, REDUCE_MARKRESET_BUFFER_SIZE, REDUCE_MAX_ATTEMPTS, REDUCE_MEMORY_MB, REDUCE_MEMORY_PHYSICAL_MB, REDUCE_MEMORY_TOTAL_BYTES, REDUCE_MEMTOMEM_ENABLED, REDUCE_MEMTOMEM_THRESHOLD, REDUCE_MERGE_INMEM_THRESHOLD, REDUCE_OUTPUT_DIR_ATTR, REDUCE_SKIP_INCR_PROC_COUNT, REDUCE_SKIP_MAXGROUPS, REDUCE_SPECULATIVE, REDUCE_ULIMIT, SETUP_CLEANUP_NEEDED, SHUFFLE_CONNECT_TIMEOUT, SHUFFLE_FETCH_FAILURES, SHUFFLE_INPUT_BUFFER_PERCENT, SHUFFLE_MERGE_EPRCENT, SHUFFLE_NOTIFY_READERROR, SHUFFLE_PARALLEL_COPIES, SHUFFLE_READ_TIMEOUT, SKIP_OUTDIR, SKIP_RECORDS, SKIP_START_ATTEMPTS, SPECULATIVE_SLOWNODE_THRESHOLD, SPECULATIVE_SLOWTASK_THRESHOLD, SPECULATIVECAP, SPLIT_FILE, TASK_ATTEMPT_ID, TASK_DEBUGOUT_LINES, TASK_ID, TASK_ISMAP, TASK_OUTPUT_DIR, TASK_PARTITION, TASK_PROFILE, TASK_PROFILE_PARAMS, TASK_TEMP_DIR, TASK_TIMEOUT, TASK_USERLOG_LIMIT, USER_LOG_RETAIN_HOURS, USER_NAME, WORKING_DIR
 
Constructor Summary
JobContextImpl(Configuration conf, JobID jobId)
           
 
Method Summary
 java.lang.Class<? extends Reducer<?,?,?,?>> getCombinerClass()
          Get the combiner class for the job.
 Configuration getConfiguration()
          Return the configuration for the job.
 RawComparator<?> getGroupingComparator()
          Get the user defined RawComparator comparator for grouping keys of inputs to the reduce.
 java.lang.Class<? extends InputFormat<?,?>> getInputFormatClass()
          Get the InputFormat class for the job.
 java.lang.String getJar()
          Get the pathname of the job's jar.
 JobID getJobID()
          Get the unique ID for the job.
 java.lang.String getJobName()
          Get the user-specified job name.
 boolean getJobSetupCleanupNeeded()
          Get whether job-setup and job-cleanup is needed for the job
 java.lang.Class<?> getMapOutputKeyClass()
          Get the key class for the map output data.
 java.lang.Class<?> getMapOutputValueClass()
          Get the value class for the map output data.
 java.lang.Class<? extends Mapper<?,?,?,?>> getMapperClass()
          Get the Mapper class for the job.
 int getMaxMapAttempts()
          Get the configured number of maximum attempts that will be made to run a map task, as specified by the mapred.map.max.attempts property.
 int getMaxReduceAttempts()
          Get the configured number of maximum attempts that will be made to run a reduce task, as specified by the mapred.reduce.max.attempts property.
 int getNumReduceTasks()
          Get configured the number of reduce tasks for this job.
 java.lang.Class<? extends OutputFormat<?,?>> getOutputFormatClass()
          Get the OutputFormat class for the job.
 java.lang.Class<?> getOutputKeyClass()
          Get the key class for the job output data.
 java.lang.Class<?> getOutputValueClass()
          Get the value class for job outputs.
 java.lang.Class<? extends Partitioner<?,?>> getPartitionerClass()
          Get the Partitioner class for the job.
 boolean getProfileEnabled()
          Get whether the task profiling is enabled.
 java.lang.String getProfileParams()
          Get the profiler configuration arguments.
 Configuration.IntegerRanges getProfileTaskRange(boolean isMap)
          Get the range of maps or reduces to profile.
 java.lang.Class<? extends Reducer<?,?,?,?>> getReducerClass()
          Get the Reducer class for the job.
 RawComparator<?> getSortComparator()
          Get the RawComparator comparator used to compare keys.
 java.lang.String getUser()
          Get the reported username for this job.
 Path getWorkingDirectory()
          Get the current working directory for the default file system.
 void setJobID(JobID jobId)
          Set the JobID.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

conf

protected final JobConf conf
Constructor Detail

JobContextImpl

public JobContextImpl(Configuration conf,
                      JobID jobId)
Method Detail

getConfiguration

public Configuration getConfiguration()
Return the configuration for the job.

Specified by:
getConfiguration in interface JobContext
Returns:
the shared configuration object

getJobID

public JobID getJobID()
Get the unique ID for the job.

Specified by:
getJobID in interface JobContext
Returns:
the object with the job id

setJobID

public void setJobID(JobID jobId)
Set the JobID.


getNumReduceTasks

public int getNumReduceTasks()
Get configured the number of reduce tasks for this job. Defaults to 1.

Specified by:
getNumReduceTasks in interface JobContext
Returns:
the number of reduce tasks for this job.

getWorkingDirectory

public Path getWorkingDirectory()
                         throws java.io.IOException
Get the current working directory for the default file system.

Specified by:
getWorkingDirectory in interface JobContext
Returns:
the directory name.
Throws:
java.io.IOException

getOutputKeyClass

public java.lang.Class<?> getOutputKeyClass()
Get the key class for the job output data.

Specified by:
getOutputKeyClass in interface JobContext
Returns:
the key class for the job output data.

getOutputValueClass

public java.lang.Class<?> getOutputValueClass()
Get the value class for job outputs.

Specified by:
getOutputValueClass in interface JobContext
Returns:
the value class for job outputs.

getMapOutputKeyClass

public java.lang.Class<?> getMapOutputKeyClass()
Get the key class for the map output data. If it is not set, use the (final) output key class. This allows the map output key class to be different than the final output key class.

Specified by:
getMapOutputKeyClass in interface JobContext
Returns:
the map output key class.

getMapOutputValueClass

public java.lang.Class<?> getMapOutputValueClass()
Get the value class for the map output data. If it is not set, use the (final) output value class This allows the map output value class to be different than the final output value class.

Specified by:
getMapOutputValueClass in interface JobContext
Returns:
the map output value class.

getJobName

public java.lang.String getJobName()
Get the user-specified job name. This is only used to identify the job to the user.

Specified by:
getJobName in interface JobContext
Returns:
the job's name, defaulting to "".

getInputFormatClass

public java.lang.Class<? extends InputFormat<?,?>> getInputFormatClass()
                                                                throws java.lang.ClassNotFoundException
Get the InputFormat class for the job.

Specified by:
getInputFormatClass in interface JobContext
Returns:
the InputFormat class for the job.
Throws:
java.lang.ClassNotFoundException

getMapperClass

public java.lang.Class<? extends Mapper<?,?,?,?>> getMapperClass()
                                                          throws java.lang.ClassNotFoundException
Get the Mapper class for the job.

Specified by:
getMapperClass in interface JobContext
Returns:
the Mapper class for the job.
Throws:
java.lang.ClassNotFoundException

getCombinerClass

public java.lang.Class<? extends Reducer<?,?,?,?>> getCombinerClass()
                                                             throws java.lang.ClassNotFoundException
Get the combiner class for the job.

Specified by:
getCombinerClass in interface JobContext
Returns:
the combiner class for the job.
Throws:
java.lang.ClassNotFoundException

getReducerClass

public java.lang.Class<? extends Reducer<?,?,?,?>> getReducerClass()
                                                            throws java.lang.ClassNotFoundException
Get the Reducer class for the job.

Specified by:
getReducerClass in interface JobContext
Returns:
the Reducer class for the job.
Throws:
java.lang.ClassNotFoundException

getOutputFormatClass

public java.lang.Class<? extends OutputFormat<?,?>> getOutputFormatClass()
                                                                  throws java.lang.ClassNotFoundException
Get the OutputFormat class for the job.

Specified by:
getOutputFormatClass in interface JobContext
Returns:
the OutputFormat class for the job.
Throws:
java.lang.ClassNotFoundException

getPartitionerClass

public java.lang.Class<? extends Partitioner<?,?>> getPartitionerClass()
                                                                throws java.lang.ClassNotFoundException
Get the Partitioner class for the job.

Specified by:
getPartitionerClass in interface JobContext
Returns:
the Partitioner class for the job.
Throws:
java.lang.ClassNotFoundException

getSortComparator

public RawComparator<?> getSortComparator()
Get the RawComparator comparator used to compare keys.

Specified by:
getSortComparator in interface JobContext
Returns:
the RawComparator comparator used to compare keys.

getJar

public java.lang.String getJar()
Get the pathname of the job's jar.

Specified by:
getJar in interface JobContext
Returns:
the pathname

getGroupingComparator

public RawComparator<?> getGroupingComparator()
Get the user defined RawComparator comparator for grouping keys of inputs to the reduce.

Returns:
comparator set by the user for grouping values.
See Also:
for details.

getJobSetupCleanupNeeded

public boolean getJobSetupCleanupNeeded()
Get whether job-setup and job-cleanup is needed for the job

Specified by:
getJobSetupCleanupNeeded in interface JobContext
Returns:
boolean

getMaxMapAttempts

public int getMaxMapAttempts()
Get the configured number of maximum attempts that will be made to run a map task, as specified by the mapred.map.max.attempts property. If this property is not already set, the default is 4 attempts.

Specified by:
getMaxMapAttempts in interface JobContext
Returns:
the max number of attempts per map task.

getMaxReduceAttempts

public int getMaxReduceAttempts()
Get the configured number of maximum attempts that will be made to run a reduce task, as specified by the mapred.reduce.max.attempts property. If this property is not already set, the default is 4 attempts.

Specified by:
getMaxReduceAttempts in interface JobContext
Returns:
the max number of attempts per reduce task.

getProfileEnabled

public boolean getProfileEnabled()
Get whether the task profiling is enabled.

Specified by:
getProfileEnabled in interface JobContext
Returns:
true if some tasks will be profiled

getProfileParams

public java.lang.String getProfileParams()
Get the profiler configuration arguments. The default value for this property is "-agentlib:hprof=cpu=samples,heap=sites,force=n,thread=y,verbose=n,file=%s"

Specified by:
getProfileParams in interface JobContext
Returns:
the parameters to pass to the task child to configure profiling

getProfileTaskRange

public Configuration.IntegerRanges getProfileTaskRange(boolean isMap)
Get the range of maps or reduces to profile.

Specified by:
getProfileTaskRange in interface JobContext
Parameters:
isMap - is the task a map?
Returns:
the task ranges

getUser

public java.lang.String getUser()
Get the reported username for this job.

Specified by:
getUser in interface JobContext
Returns:
the username