p2pMapReduce.mapreduceModule
Class ReduceContextImpl<KEYIN,VALUEIN,KEYOUT,VALUEOUT>

java.lang.Object
  extended by p2pMapReduce.mapreduceModule.JobContextImpl
      extended by p2pMapReduce.mapreduceModule.TaskAttemptContextImpl
          extended by p2pMapReduce.mapreduceModule.TaskInputOutputContextImpl<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
              extended by p2pMapReduce.mapreduceModule.ReduceContextImpl<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
All Implemented Interfaces:
JobContext, MRJobConfig, ReduceContext<KEYIN,VALUEIN,KEYOUT,VALUEOUT>, TaskAttemptContext, TaskInputOutputContext<KEYIN,VALUEIN,KEYOUT,VALUEOUT>

public class ReduceContextImpl<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
extends TaskInputOutputContextImpl<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
implements ReduceContext<KEYIN,VALUEIN,KEYOUT,VALUEOUT>

NOTE: the format of intermediate result depends on MapContext#write(K,V) method that refer to RecordWriter


Nested Class Summary
protected  class ReduceContextImpl.ValueIterable
           
protected  class ReduceContextImpl.ValueIterator
           
 
Field Summary
 
Fields inherited from class p2pMapReduce.mapreduceModule.JobContextImpl
conf
 
Fields inherited from interface p2pMapReduce.mapreduceModule.MRJobConfig
CACHE_ARCHIVES, CACHE_ARCHIVES_SIZES, CACHE_ARCHIVES_TIMESTAMPS, CACHE_ARCHIVES_VISIBILITIES, CACHE_FILE_TIMESTAMPS, CACHE_FILE_VISIBILITIES, CACHE_FILES, CACHE_FILES_SIZES, CACHE_LOCALARCHIVES, CACHE_LOCALFILES, CACHE_SYMLINK, CLASSPATH_ARCHIVES, CLASSPATH_FILES, COMBINE_CLASS_ATTR, COMPLETED_MAPS_FOR_REDUCE_SLOWSTART, END_NOTIFICATION_RETRIE_INTERVAL, END_NOTIFICATION_RETRIES, END_NOTIFICATION_URL, GROUP_COMPARATOR_CLASS, HISTORY_LOCATION, ID, INPUT_FORMAT_CLASS_ATTR, INPUTSPLIT_CLASS_ATTR, INPUTSPLIT_SIZE, IO_SORT_FACTOR, IO_SORT_MB, JAR, JAR_UNPACK_PATTERN, JOB_ACL_MODIFY_JOB, JOB_ACL_VIEW_JOB, JOB_CANCEL_DELEGATION_TOKEN, JOB_DIR_ATTR, JOB_INPUT_DIR_ATTR, JOB_INPUTSPLIT_DIR_ATTR, JOB_JAR_DIR_ATTR, JOB_JOBTRACKER_ID, JOB_LOCAL_DIR, JOB_NAME, JOB_NAMENODES, JOB_OUTPUT_DIR_ATTR, JVM_NUMTASKS_TORUN, KEY_COMPARATOR, MAP_CLASS_ATTR, MAP_COMBINE_MIN_SPILLS, MAP_DEBUG_SCRIPT, MAP_ENV, MAP_FAILURES_MAX_PERCENT, MAP_INPUT_FILE, MAP_INPUT_PATH, MAP_INPUT_START, MAP_JAVA_OPTS, MAP_LOG_LEVEL, MAP_MAX_ATTEMPTS, MAP_MEMORY_MB, MAP_MEMORY_PHYSICAL_MB, MAP_OUTPUT_COMPRESS, MAP_OUTPUT_COMPRESS_CODEC, MAP_OUTPUT_DIR_ATTR, MAP_OUTPUT_KEY_CLASS, MAP_OUTPUT_KEY_FIELD_SEPERATOR, MAP_OUTPUT_KEY_VALUE_SEPARATOR_ATTR, MAP_OUTPUT_VALUE_CLASS, MAP_SKIP_INCR_PROC_COUNT, MAP_SKIP_MAX_RECORDS, MAP_SORT_SPILL_PERCENT, MAP_SPECULATIVE, MAP_ULIMIT, MAX_TASK_FAILURES_PER_TRACKER, NUM_MAP_PROFILES, NUM_MAPS, NUM_REDUCE_PROFILES, NUM_REDUCES, OUTPUT_FORMAT_CLASS_ATTR, OUTPUT_KEY_CLASS, OUTPUT_VALUE_CLASS, PARTITIONER_CLASS_ATTR, PRESERVE_FAILED_TASK_FILES, PRESERVE_FILES_PATTERN, PRIORITY, QUEUE_NAME, RECORDS_BEFORE_PROGRESS, REDUCE_CLASS_ATTR, REDUCE_DEBUG_SCRIPT, REDUCE_ENV, REDUCE_FAILURES_MAXPERCENT, REDUCE_INPUT_BUFFER_PERCENT, REDUCE_INPUT_DIR_ATTR, REDUCE_JAVA_OPTS, REDUCE_LOG_LEVEL, REDUCE_MARKRESET_BUFFER_PERCENT, REDUCE_MARKRESET_BUFFER_SIZE, REDUCE_MAX_ATTEMPTS, REDUCE_MEMORY_MB, REDUCE_MEMORY_PHYSICAL_MB, REDUCE_MEMORY_TOTAL_BYTES, REDUCE_MEMTOMEM_ENABLED, REDUCE_MEMTOMEM_THRESHOLD, REDUCE_MERGE_INMEM_THRESHOLD, REDUCE_OUTPUT_DIR_ATTR, REDUCE_SKIP_INCR_PROC_COUNT, REDUCE_SKIP_MAXGROUPS, REDUCE_SPECULATIVE, REDUCE_ULIMIT, SETUP_CLEANUP_NEEDED, SHUFFLE_CONNECT_TIMEOUT, SHUFFLE_FETCH_FAILURES, SHUFFLE_INPUT_BUFFER_PERCENT, SHUFFLE_MERGE_EPRCENT, SHUFFLE_NOTIFY_READERROR, SHUFFLE_PARALLEL_COPIES, SHUFFLE_READ_TIMEOUT, SKIP_OUTDIR, SKIP_RECORDS, SKIP_START_ATTEMPTS, SPECULATIVE_SLOWNODE_THRESHOLD, SPECULATIVE_SLOWTASK_THRESHOLD, SPECULATIVECAP, SPLIT_FILE, TASK_ATTEMPT_ID, TASK_DEBUGOUT_LINES, TASK_ID, TASK_ISMAP, TASK_OUTPUT_DIR, TASK_PARTITION, TASK_PROFILE, TASK_PROFILE_PARAMS, TASK_TEMP_DIR, TASK_TIMEOUT, TASK_USERLOG_LIMIT, USER_LOG_RETAIN_HOURS, USER_NAME, WORKING_DIR
 
Constructor Summary
ReduceContextImpl(Configuration conf, TaskAttemptID taskid, java.lang.String partitionFile, RecordWriter<KEYOUT,VALUEOUT> output, OutputCommitter committer, StatusReporter reporter, java.lang.Class<KEYIN> keyClass, java.lang.Class<VALUEIN> valueClass)
           
 
Method Summary
 KEYIN getCurrentKey()
          Get the current key.
 VALUEIN getCurrentValue()
          Get the current value.
 java.lang.Iterable<VALUEIN> getValues()
          Iterate through the values for the current key, reusing the same value object, which is stored in the context.
 boolean nextKey()
          Start processing next unique key.
 boolean nextKeyValue()
          Advance to the next key/value pair.
 
Methods inherited from class p2pMapReduce.mapreduceModule.TaskInputOutputContextImpl
getCounter, getCounter, getOutputCommitter, write
 
Methods inherited from class p2pMapReduce.mapreduceModule.TaskAttemptContextImpl
getStatus, getTaskAttemptID, setStatus
 
Methods inherited from class p2pMapReduce.mapreduceModule.JobContextImpl
getCombinerClass, getConfiguration, getGroupingComparator, getInputFormatClass, getJar, getJobID, getJobName, getJobSetupCleanupNeeded, getMapOutputKeyClass, getMapOutputValueClass, getMapperClass, getMaxMapAttempts, getMaxReduceAttempts, getNumReduceTasks, getOutputFormatClass, getOutputKeyClass, getOutputValueClass, getPartitionerClass, getProfileEnabled, getProfileParams, getProfileTaskRange, getReducerClass, getSortComparator, getUser, getWorkingDirectory, setJobID
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 
Methods inherited from interface p2pMapReduce.mapreduceModule.TaskInputOutputContext
getCounter, getCounter, getOutputCommitter, write
 
Methods inherited from interface p2pMapReduce.mapreduceModule.TaskAttemptContext
getStatus, getTaskAttemptID, setStatus
 
Methods inherited from interface p2pMapReduce.mapreduceModule.JobContext
getCombinerClass, getConfiguration, getInputFormatClass, getJar, getJobID, getJobName, getJobSetupCleanupNeeded, getMapOutputKeyClass, getMapOutputValueClass, getMapperClass, getMaxMapAttempts, getMaxReduceAttempts, getNumReduceTasks, getOutputFormatClass, getOutputKeyClass, getOutputValueClass, getPartitionerClass, getProfileEnabled, getProfileParams, getProfileTaskRange, getReducerClass, getSortComparator, getUser, getWorkingDirectory
 

Constructor Detail

ReduceContextImpl

public ReduceContextImpl(Configuration conf,
                         TaskAttemptID taskid,
                         java.lang.String partitionFile,
                         RecordWriter<KEYOUT,VALUEOUT> output,
                         OutputCommitter committer,
                         StatusReporter reporter,
                         java.lang.Class<KEYIN> keyClass,
                         java.lang.Class<VALUEIN> valueClass)
                  throws java.io.FileNotFoundException,
                         java.io.IOException
Throws:
java.io.FileNotFoundException
java.io.IOException
Method Detail

nextKey

public boolean nextKey()
                throws java.io.IOException,
                       java.lang.InterruptedException
Description copied from interface: ReduceContext
Start processing next unique key.

Specified by:
nextKey in interface ReduceContext<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
Throws:
java.io.IOException
java.lang.InterruptedException

nextKeyValue

public boolean nextKeyValue()
                     throws java.io.IOException,
                            java.lang.InterruptedException
Advance to the next key/value pair.

Specified by:
nextKeyValue in interface TaskInputOutputContext<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
Specified by:
nextKeyValue in class TaskInputOutputContextImpl<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
Returns:
the key object that was read into, or null if no more
Throws:
java.io.IOException
java.lang.InterruptedException

getCurrentKey

public KEYIN getCurrentKey()
Description copied from class: TaskInputOutputContextImpl
Get the current key.

Specified by:
getCurrentKey in interface TaskInputOutputContext<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
Specified by:
getCurrentKey in class TaskInputOutputContextImpl<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
Returns:
the current key object or null if there isn't one

getCurrentValue

public VALUEIN getCurrentValue()
Description copied from class: TaskInputOutputContextImpl
Get the current value.

Specified by:
getCurrentValue in interface TaskInputOutputContext<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
Specified by:
getCurrentValue in class TaskInputOutputContextImpl<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
Returns:
the value object that was read into

getValues

public java.lang.Iterable<VALUEIN> getValues()
                                      throws java.io.IOException,
                                             java.lang.InterruptedException
Iterate through the values for the current key, reusing the same value object, which is stored in the context.

Specified by:
getValues in interface ReduceContext<KEYIN,VALUEIN,KEYOUT,VALUEOUT>
Returns:
the series of values associated with the current key. All of the objects returned directly and indirectly from this method are reused.
Throws:
java.io.IOException
java.lang.InterruptedException