p2pMapReduce.mapreduceModule
Interface JobContext
- All Superinterfaces:
- MRJobConfig
- All Known Subinterfaces:
- MapContext<KEYIN,VALUEIN,KEYOUT,VALUEOUT>, ReduceContext<KEYIN,VALUEIN,KEYOUT,VALUEOUT>, TaskAttemptContext, TaskInputOutputContext<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
- All Known Implementing Classes:
- Job, JobContextImpl, MapContextImpl, Mapper.Context, ReduceContextImpl, Reducer.Context, TaskAttemptContextImpl, TaskInputOutputContextImpl, WrappedMapper.Context, WrappedReducer.Context
public interface JobContext
- extends MRJobConfig
A read-only view of the job that is provided to the tasks while they
are running.
Fields inherited from interface p2pMapReduce.mapreduceModule.MRJobConfig |
CACHE_ARCHIVES, CACHE_ARCHIVES_SIZES, CACHE_ARCHIVES_TIMESTAMPS, CACHE_ARCHIVES_VISIBILITIES, CACHE_FILE_TIMESTAMPS, CACHE_FILE_VISIBILITIES, CACHE_FILES, CACHE_FILES_SIZES, CACHE_LOCALARCHIVES, CACHE_LOCALFILES, CACHE_SYMLINK, CLASSPATH_ARCHIVES, CLASSPATH_FILES, COMBINE_CLASS_ATTR, COMPLETED_MAPS_FOR_REDUCE_SLOWSTART, END_NOTIFICATION_RETRIE_INTERVAL, END_NOTIFICATION_RETRIES, END_NOTIFICATION_URL, GROUP_COMPARATOR_CLASS, HISTORY_LOCATION, ID, INPUT_FORMAT_CLASS_ATTR, INPUTSPLIT_CLASS_ATTR, INPUTSPLIT_SIZE, IO_SORT_FACTOR, IO_SORT_MB, JAR, JAR_UNPACK_PATTERN, JOB_ACL_MODIFY_JOB, JOB_ACL_VIEW_JOB, JOB_CANCEL_DELEGATION_TOKEN, JOB_DIR_ATTR, JOB_INPUT_DIR_ATTR, JOB_INPUTSPLIT_DIR_ATTR, JOB_JAR_DIR_ATTR, JOB_JOBTRACKER_ID, JOB_LOCAL_DIR, JOB_NAME, JOB_NAMENODES, JOB_OUTPUT_DIR_ATTR, JVM_NUMTASKS_TORUN, KEY_COMPARATOR, MAP_CLASS_ATTR, MAP_COMBINE_MIN_SPILLS, MAP_DEBUG_SCRIPT, MAP_ENV, MAP_FAILURES_MAX_PERCENT, MAP_INPUT_FILE, MAP_INPUT_PATH, MAP_INPUT_START, MAP_JAVA_OPTS, MAP_LOG_LEVEL, MAP_MAX_ATTEMPTS, MAP_MEMORY_MB, MAP_MEMORY_PHYSICAL_MB, MAP_OUTPUT_COMPRESS, MAP_OUTPUT_COMPRESS_CODEC, MAP_OUTPUT_DIR_ATTR, MAP_OUTPUT_KEY_CLASS, MAP_OUTPUT_KEY_FIELD_SEPERATOR, MAP_OUTPUT_KEY_VALUE_SEPARATOR_ATTR, MAP_OUTPUT_VALUE_CLASS, MAP_SKIP_INCR_PROC_COUNT, MAP_SKIP_MAX_RECORDS, MAP_SORT_SPILL_PERCENT, MAP_SPECULATIVE, MAP_ULIMIT, MAX_TASK_FAILURES_PER_TRACKER, NUM_MAP_PROFILES, NUM_MAPS, NUM_REDUCE_PROFILES, NUM_REDUCES, OUTPUT_FORMAT_CLASS_ATTR, OUTPUT_KEY_CLASS, OUTPUT_VALUE_CLASS, PARTITIONER_CLASS_ATTR, PRESERVE_FAILED_TASK_FILES, PRESERVE_FILES_PATTERN, PRIORITY, QUEUE_NAME, RECORDS_BEFORE_PROGRESS, REDUCE_CLASS_ATTR, REDUCE_DEBUG_SCRIPT, REDUCE_ENV, REDUCE_FAILURES_MAXPERCENT, REDUCE_INPUT_BUFFER_PERCENT, REDUCE_INPUT_DIR_ATTR, REDUCE_JAVA_OPTS, REDUCE_LOG_LEVEL, REDUCE_MARKRESET_BUFFER_PERCENT, REDUCE_MARKRESET_BUFFER_SIZE, REDUCE_MAX_ATTEMPTS, REDUCE_MEMORY_MB, REDUCE_MEMORY_PHYSICAL_MB, REDUCE_MEMORY_TOTAL_BYTES, REDUCE_MEMTOMEM_ENABLED, REDUCE_MEMTOMEM_THRESHOLD, REDUCE_MERGE_INMEM_THRESHOLD, REDUCE_OUTPUT_DIR_ATTR, REDUCE_SKIP_INCR_PROC_COUNT, REDUCE_SKIP_MAXGROUPS, REDUCE_SPECULATIVE, REDUCE_ULIMIT, SETUP_CLEANUP_NEEDED, SHUFFLE_CONNECT_TIMEOUT, SHUFFLE_FETCH_FAILURES, SHUFFLE_INPUT_BUFFER_PERCENT, SHUFFLE_MERGE_EPRCENT, SHUFFLE_NOTIFY_READERROR, SHUFFLE_PARALLEL_COPIES, SHUFFLE_READ_TIMEOUT, SKIP_OUTDIR, SKIP_RECORDS, SKIP_START_ATTEMPTS, SPECULATIVE_SLOWNODE_THRESHOLD, SPECULATIVE_SLOWTASK_THRESHOLD, SPECULATIVECAP, SPLIT_FILE, TASK_ATTEMPT_ID, TASK_DEBUGOUT_LINES, TASK_ID, TASK_ISMAP, TASK_OUTPUT_DIR, TASK_PARTITION, TASK_PROFILE, TASK_PROFILE_PARAMS, TASK_TEMP_DIR, TASK_TIMEOUT, TASK_USERLOG_LIMIT, USER_LOG_RETAIN_HOURS, USER_NAME, WORKING_DIR |
Method Summary |
java.lang.Class<? extends Reducer<?,?,?,?>> |
getCombinerClass()
Get the combiner class for the job. |
Configuration |
getConfiguration()
Return the configuration for the job. |
java.lang.Class<? extends InputFormat<?,?>> |
getInputFormatClass()
Get the InputFormat class for the job. |
java.lang.String |
getJar()
Get the pathname of the job's jar. |
JobID |
getJobID()
Get the unique ID for the job. |
java.lang.String |
getJobName()
Get the user-specified job name. |
boolean |
getJobSetupCleanupNeeded()
Get whether job-setup and job-cleanup is needed for the job |
java.lang.Class<?> |
getMapOutputKeyClass()
Get the key class for the map output data. |
java.lang.Class<?> |
getMapOutputValueClass()
Get the value class for the map output data. |
java.lang.Class<? extends Mapper<?,?,?,?>> |
getMapperClass()
Get the Mapper class for the job. |
int |
getMaxMapAttempts()
Get the configured number of maximum attempts that will be made to run a
map task, as specified by the mapred.map.max.attempts
property. |
int |
getMaxReduceAttempts()
Get the configured number of maximum attempts that will be made to run a
reduce task, as specified by the mapred.reduce.max.attempts
property. |
int |
getNumReduceTasks()
Get configured the number of reduce tasks for this job. |
java.lang.Class<? extends OutputFormat<?,?>> |
getOutputFormatClass()
Get the OutputFormat class for the job. |
java.lang.Class<?> |
getOutputKeyClass()
Get the key class for the job output data. |
java.lang.Class<?> |
getOutputValueClass()
Get the value class for job outputs. |
java.lang.Class<? extends Partitioner<?,?>> |
getPartitionerClass()
Get the Partitioner class for the job. |
boolean |
getProfileEnabled()
Get whether the task profiling is enabled. |
java.lang.String |
getProfileParams()
Get the profiler configuration arguments. |
Configuration.IntegerRanges |
getProfileTaskRange(boolean isMap)
Get the range of maps or reduces to profile. |
java.lang.Class<? extends Reducer<?,?,?,?>> |
getReducerClass()
Get the Reducer class for the job. |
RawComparator<?> |
getSortComparator()
Get the RawComparator comparator used to compare keys. |
java.lang.String |
getUser()
Get the reported username for this job. |
Path |
getWorkingDirectory()
Get the current working directory for the default file system. |
getConfiguration
Configuration getConfiguration()
- Return the configuration for the job.
- Returns:
- the shared configuration object
getJobID
JobID getJobID()
- Get the unique ID for the job.
- Returns:
- the object with the job id
getNumReduceTasks
int getNumReduceTasks()
- Get configured the number of reduce tasks for this job. Defaults to
1
.
- Returns:
- the number of reduce tasks for this job.
getWorkingDirectory
Path getWorkingDirectory()
throws java.io.IOException
- Get the current working directory for the default file system.
- Returns:
- the directory name.
- Throws:
java.io.IOException
getInputFormatClass
java.lang.Class<? extends InputFormat<?,?>> getInputFormatClass()
throws java.lang.ClassNotFoundException
- Get the
InputFormat
class for the job.
- Returns:
- the
InputFormat
class for the job.
- Throws:
java.lang.ClassNotFoundException
getOutputFormatClass
java.lang.Class<? extends OutputFormat<?,?>> getOutputFormatClass()
throws java.lang.ClassNotFoundException
- Get the
OutputFormat
class for the job.
- Returns:
- the
OutputFormat
class for the job.
- Throws:
java.lang.ClassNotFoundException
getMapOutputKeyClass
java.lang.Class<?> getMapOutputKeyClass()
- Get the key class for the map output data. If it is not set, use the
(final) output key class. This allows the map output key class to be
different than the final output key class.
- Returns:
- the map output key class.
getMapOutputValueClass
java.lang.Class<?> getMapOutputValueClass()
- Get the value class for the map output data. If it is not set, use the
(final) output value class This allows the map output value class to be
different than the final output value class.
- Returns:
- the map output value class.
getOutputKeyClass
java.lang.Class<?> getOutputKeyClass()
- Get the key class for the job output data.
- Returns:
- the key class for the job output data.
getOutputValueClass
java.lang.Class<?> getOutputValueClass()
- Get the value class for job outputs.
- Returns:
- the value class for job outputs.
getMapperClass
java.lang.Class<? extends Mapper<?,?,?,?>> getMapperClass()
throws java.lang.ClassNotFoundException
- Get the
Mapper
class for the job.
- Returns:
- the
Mapper
class for the job.
- Throws:
java.lang.ClassNotFoundException
getCombinerClass
java.lang.Class<? extends Reducer<?,?,?,?>> getCombinerClass()
throws java.lang.ClassNotFoundException
- Get the combiner class for the job.
- Returns:
- the combiner class for the job.
- Throws:
java.lang.ClassNotFoundException
getPartitionerClass
java.lang.Class<? extends Partitioner<?,?>> getPartitionerClass()
throws java.lang.ClassNotFoundException
- Get the
Partitioner
class for the job.
- Returns:
- the
Partitioner
class for the job.
- Throws:
java.lang.ClassNotFoundException
getReducerClass
java.lang.Class<? extends Reducer<?,?,?,?>> getReducerClass()
throws java.lang.ClassNotFoundException
- Get the
Reducer
class for the job.
- Returns:
- the
Reducer
class for the job.
- Throws:
java.lang.ClassNotFoundException
getMaxMapAttempts
int getMaxMapAttempts()
- Get the configured number of maximum attempts that will be made to run a
map task, as specified by the
mapred.map.max.attempts
property. If this property is not already set, the default is 4 attempts.
- Returns:
- the max number of attempts per map task.
getMaxReduceAttempts
int getMaxReduceAttempts()
- Get the configured number of maximum attempts that will be made to run a
reduce task, as specified by the
mapred.reduce.max.attempts
property. If this property is not already set, the default is 4 attempts.
- Returns:
- the max number of attempts per reduce task.
getUser
java.lang.String getUser()
- Get the reported username for this job.
- Returns:
- the username
getJobName
java.lang.String getJobName()
- Get the user-specified job name. This is only used to identify the
job to the user.
- Returns:
- the job's name, defaulting to "".
getSortComparator
RawComparator<?> getSortComparator()
- Get the
RawComparator
comparator used to compare keys.
- Returns:
- the
RawComparator
comparator used to compare keys.
getJar
java.lang.String getJar()
- Get the pathname of the job's jar.
- Returns:
- the pathname
getJobSetupCleanupNeeded
boolean getJobSetupCleanupNeeded()
- Get whether job-setup and job-cleanup is needed for the job
- Returns:
- boolean
getProfileEnabled
boolean getProfileEnabled()
- Get whether the task profiling is enabled.
- Returns:
- true if some tasks will be profiled
getProfileParams
java.lang.String getProfileParams()
- Get the profiler configuration arguments.
The default value for this property is
"-agentlib:hprof=cpu=samples,heap=sites,force=n,thread=y,verbose=n,file=%s"
- Returns:
- the parameters to pass to the task child to configure profiling
getProfileTaskRange
Configuration.IntegerRanges getProfileTaskRange(boolean isMap)
- Get the range of maps or reduces to profile.
- Parameters:
isMap
- is the task a map?
- Returns:
- the task ranges