org.apache.parquet.hadoop.example

Class ExampleOutputFormat

java.lang.Object

org.apache.hadoop.mapreduce.OutputFormat<K,V>

org.apache.hadoop.mapreduce.lib.output.FileOutputFormat<Void,T>

org.apache.parquet.hadoop.ParquetOutputFormat<Group>

org.apache.parquet.hadoop.example.ExampleOutputFormat

public class ExampleOutputFormat
extends ParquetOutputFormat<Group>

An example output format must be provided the schema up front

Author:

Julien Le Dem

See Also:

setSchema(Job, MessageType), GroupWriteSupport.PARQUET_EXAMPLE_SCHEMA

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat

org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.Counter

Field Summary

Fields inherited from class org.apache.parquet.hadoop.ParquetOutputFormat

BLOCK_SIZE, COMPRESSION, DICTIONARY_PAGE_SIZE, ENABLE_DICTIONARY, ENABLE_JOB_SUMMARY, ESTIMATE_PAGE_SIZE_CHECK, MAX_PADDING_BYTES, MAX_ROW_COUNT_FOR_PAGE_SIZE_CHECK, MEMORY_POOL_RATIO, MIN_MEMORY_ALLOCATION, MIN_ROW_COUNT_FOR_PAGE_SIZE_CHECK, PAGE_SIZE, VALIDATION, WRITE_SUPPORT_CLASS, WRITER_VERSION

Fields inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat

BASE_OUTPUT_NAME, COMPRESS, COMPRESS_CODEC, COMPRESS_TYPE, OUTDIR, PART

Constructor Summary

Constructors

Constructor and Description
ExampleOutputFormat()

Method Summary

Methods

Modifier and Type	Method and Description
static MessageType	getSchema(org.apache.hadoop.mapreduce.Job job) retrieve the schema from the conf
static void	setSchema(org.apache.hadoop.mapreduce.Job job, MessageType schema) set the schema being written to the job conf

Methods inherited from class org.apache.parquet.hadoop.ParquetOutputFormat

getBlockSize, getBlockSize, getCompression, getCompression, getDictionaryPageSize, getDictionaryPageSize, getEnableDictionary, getEnableDictionary, getEstimatePageSizeCheck, getLongBlockSize, getMaxRowCountForPageSizeCheck, getMemoryManager, getMinRowCountForPageSizeCheck, getOutputCommitter, getPageSize, getPageSize, getRecordWriter, getRecordWriter, getRecordWriter, getValidation, getValidation, getWriterVersion, getWriteSupport, getWriteSupportClass, isCompressionSet, isCompressionSet, setBlockSize, setCompression, setDictionaryPageSize, setEnableDictionary, setMaxPaddingSize, setMaxPaddingSize, setPageSize, setValidation, setValidation, setWriteSupportClass, setWriteSupportClass

Methods inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat

checkOutputSpecs, getCompressOutput, getDefaultWorkFile, getOutputCompressorClass, getOutputName, getOutputPath, getPathForWorkFile, getUniqueFile, getWorkOutputPath, setCompressOutput, setOutputCompressorClass, setOutputName, setOutputPath

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ExampleOutputFormat

public ExampleOutputFormat()

Method Detail

setSchema

public static void setSchema(org.apache.hadoop.mapreduce.Job job,
             MessageType schema)

set the schema being written to the job conf

Parameters:

job -

schema - the schema of the data

getSchema

public static MessageType getSchema(org.apache.hadoop.mapreduce.Job job)

retrieve the schema from the conf

Parameters:

job -

Returns:

the schema