|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectp2pMapReduce.mapreduceModule.input.InputFormat<java.lang.Long,java.lang.String>
p2pMapReduce.mapreduceModule.input.TextFileInputFormat
public class TextFileInputFormat
Read input text files and generate the InputSplit
Each InputSplit is a TextFileInputSplit
and always terminate with a whole
text line.
Provide also a LineRecordReader
Field Summary | |
---|---|
static java.lang.String |
INPUT_PATHS_ATTR
Deprecated. |
static java.lang.String |
INPUT_PATHS_SEPARATOR
|
static java.lang.String |
INPUTSPLIT_SUBDIR_ATTR
|
static java.lang.String |
LOCAL_INPUT_PATHS_ATTR
|
Constructor Summary | |
---|---|
TextFileInputFormat()
|
Method Summary | |
---|---|
static void |
addInputPath_old(Job job,
java.lang.String path)
Deprecated. |
static void |
addInputPath(Job job,
java.lang.String path)
|
static void |
addLocalInputPath(Job job,
java.lang.String path)
Add a path to an input of the job. |
RecordReader<java.lang.Long,java.lang.String> |
createRecordReader(InputSplit split,
TaskAttemptContext context)
Create a record reader for a given split. |
static java.lang.String |
getLocalInputPath(Job job)
|
java.util.List<InputSplit> |
getSplits(JobContext jobContext)
Logically split the set of input files for the job. |
static void |
main(java.lang.String[] args)
|
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
public static final java.lang.String LOCAL_INPUT_PATHS_ATTR
@Deprecated public static final java.lang.String INPUT_PATHS_ATTR
public static final java.lang.String INPUTSPLIT_SUBDIR_ATTR
public static final java.lang.String INPUT_PATHS_SEPARATOR
Constructor Detail |
---|
public TextFileInputFormat()
Method Detail |
---|
public static void main(java.lang.String[] args) throws java.io.IOException
java.io.IOException
public static void addLocalInputPath(Job job, java.lang.String path)
job
- path
- public static java.lang.String getLocalInputPath(Job job)
public static void addInputPath_old(Job job, java.lang.String path)
Path
to the list of inputs for the map-reduce job.
job
- The Job
to modifypath
- Path
to be added to the list of inputs for
the map-reduce job.public static void addInputPath(Job job, java.lang.String path)
public RecordReader<java.lang.Long,java.lang.String> createRecordReader(InputSplit split, TaskAttemptContext context) throws java.io.IOException, java.lang.InterruptedException
InputFormat
RecordReader.initialize(InputSplit, TaskAttemptContext)
before
the split is used.
createRecordReader
in class InputFormat<java.lang.Long,java.lang.String>
split
- the split to be readcontext
- the information about the task
java.io.IOException
java.lang.InterruptedException
public java.util.List<InputSplit> getSplits(JobContext jobContext) throws java.io.IOException, java.lang.InterruptedException
InputFormat
Each InputSplit
is then assigned to an individual Mapper
for processing.
Note: The split is a logical split of the inputs and the
input files are not physically split into chunks. For e.g. a split could
be <input-file-path, start, offset> tuple. The InputFormat
also creates the RecordReader
to read the InputSplit
.
getSplits
in class InputFormat<java.lang.Long,java.lang.String>
jobContext
- job configuration.
InputSplit
s for the job.
java.io.IOException
java.lang.InterruptedException
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |