org.apache.parquet.hadoop

Class ParquetInputSplit

java.lang.Object

org.apache.hadoop.mapreduce.InputSplit

org.apache.hadoop.mapreduce.lib.input.FileSplit

org.apache.parquet.hadoop.ParquetInputSplit

All Implemented Interfaces:

org.apache.hadoop.io.Writable

@InterfaceAudience.Private
public class ParquetInputSplit
extends org.apache.hadoop.mapreduce.lib.input.FileSplit
implements org.apache.hadoop.io.Writable

An input split for the Parquet format It contains the information to read one block of the file. This class is private to the ParquetInputFormat. Backward compatibility is not maintained.

Author:

Julien Le Dem

Constructor Summary

Constructors

Constructor and Description
ParquetInputSplit() Writables must have a parameterless constructor
ParquetInputSplit(org.apache.hadoop.fs.Path file, long start, long end, long length, String[] hosts, long[] rowGroupOffsets)
ParquetInputSplit(org.apache.hadoop.fs.Path path, long start, long length, String[] hosts, List<BlockMetaData> blocks, String requestedSchema, String fileSchema, Map<String,String> extraMetadata, Map<String,String> readSupportMetadata) Deprecated.

Method Summary

Methods

Modifier and Type	Method and Description
List<BlockMetaData>	getBlocks() Deprecated. the file footer is no longer read before creating input splits
long	getEnd()
Map<String,String>	getExtraMetadata() Deprecated. the file footer is no longer read before creating input splits
String	getFileSchema() Deprecated. the file footer is no longer read before creating input splits
long[]	getRowGroupOffsets()
void	readFields(DataInput hin)
String	toString()
void	write(DataOutput hout)

Methods inherited from class org.apache.hadoop.mapreduce.lib.input.FileSplit

getLength, getLocations, getPath, getStart

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

ParquetInputSplit

public ParquetInputSplit()

Writables must have a parameterless constructor

ParquetInputSplit

@Deprecated
public ParquetInputSplit(org.apache.hadoop.fs.Path path,
                            long start,
                            long length,
                            String[] hosts,
                            List<BlockMetaData> blocks,
                            String requestedSchema,
                            String fileSchema,
                            Map<String,String> extraMetadata,
                            Map<String,String> readSupportMetadata)

Deprecated.

For compatibility only use ParquetInputSplit(Path, long, long, long, String[], long[])

Parameters:

path -

start -

length -

hosts -

blocks -

requestedSchema -

fileSchema -

extraMetadata -

readSupportMetadata -

ParquetInputSplit

public ParquetInputSplit(org.apache.hadoop.fs.Path file,
                 long start,
                 long end,
                 long length,
                 String[] hosts,
                 long[] rowGroupOffsets)

Parameters:

file - the path of the file for that split

start - the start offset in the file

end - the end offset in the file

length - the actual size in bytes that we expect to read

hosts - the hosts with the replicas of this data

rowGroupOffsets - the offsets of the rowgroups selected if loaded on the client

Method Detail

getBlocks

@Deprecated
public List<BlockMetaData> getBlocks()

Deprecated. the file footer is no longer read before creating input splits

Returns:

the block meta data

getFileSchema

@Deprecated
public String getFileSchema()

Deprecated. the file footer is no longer read before creating input splits

Returns:

the file schema

getEnd

public long getEnd()

Returns:

the end offset of that split

getExtraMetadata

@Deprecated
public Map<String,String> getExtraMetadata()

Deprecated. the file footer is no longer read before creating input splits

Returns:

app specific metadata from the file

getRowGroupOffsets

public long[] getRowGroupOffsets()

Returns:

the offsets of the row group selected if this has been determined on the client side

toString

public String toString()

Overrides:

toString in class org.apache.hadoop.mapreduce.lib.input.FileSplit

readFields

public void readFields(DataInput hin)
                throws IOException

Specified by:

readFields in interface org.apache.hadoop.io.Writable

Overrides:

readFields in class org.apache.hadoop.mapreduce.lib.input.FileSplit

Throws:

IOException

write

public void write(DataOutput hout)
           throws IOException

Specified by:

write in interface org.apache.hadoop.io.Writable

Overrides:

write in class org.apache.hadoop.mapreduce.lib.input.FileSplit

Throws:

IOException