org.apache.flink.table.planner.plan.nodes.exec.stream

Class StreamExecMiniBatchAssigner

java.lang.Object

org.apache.flink.table.planner.plan.nodes.exec.ExecNodeBase<org.apache.flink.table.data.RowData>

org.apache.flink.table.planner.plan.nodes.exec.stream.StreamExecMiniBatchAssigner

All Implemented Interfaces:

ExecNode<org.apache.flink.table.data.RowData>, ExecNodeTranslator<org.apache.flink.table.data.RowData>, SingleTransformationTranslator<org.apache.flink.table.data.RowData>, StreamExecNode<org.apache.flink.table.data.RowData>

@ExecNodeMetadata(name="stream-exec-mini-batch-assigner",
                  version=1,
                  producedTransformations="mini-batch-assigner",
                  minPlanVersion=v1_15,
                  minStateVersion=v1_15)
public class StreamExecMiniBatchAssigner
extends ExecNodeBase<org.apache.flink.table.data.RowData>
implements StreamExecNode<org.apache.flink.table.data.RowData>, SingleTransformationTranslator<org.apache.flink.table.data.RowData>

Stream ExecNode which injects a mini-batch event in the streaming data. The mini-batch event will be recognized as a boundary between two mini-batches. The following operators will keep track of the mini-batch events and trigger mini-batch once the mini-batch id is advanced.

NOTE: currently, we leverage the runtime watermark mechanism to achieve the mini-batch, because runtime doesn't support customized events and the watermark mechanism fully meets mini-batch needs.

Field Summary

Fields

Modifier and Type	Field and Description
static String	FIELD_NAME_MINI_BATCH_INTERVAL
static String	MINI_BATCH_ASSIGNER_TRANSFORMATION

Fields inherited from interface org.apache.flink.table.planner.plan.nodes.exec.ExecNode

FIELD_NAME_CONFIGURATION, FIELD_NAME_DESCRIPTION, FIELD_NAME_ID, FIELD_NAME_INPUT_PROPERTIES, FIELD_NAME_OUTPUT_TYPE, FIELD_NAME_TYPE

Constructor Summary

Constructors

Constructor and Description
StreamExecMiniBatchAssigner(int id, ExecNodeContext context, org.apache.flink.configuration.ReadableConfig persistedConfig, MiniBatchInterval miniBatchInterval, List<InputProperty> inputProperties, org.apache.flink.table.types.logical.RowType outputType, String description)
StreamExecMiniBatchAssigner(org.apache.flink.configuration.ReadableConfig tableConfig, MiniBatchInterval miniBatchInterval, InputProperty inputProperty, org.apache.flink.table.types.logical.RowType outputType, String description)

Method Summary

Modifier and Type	Method and Description
protected org.apache.flink.api.dag.Transformation<org.apache.flink.table.data.RowData>	translateToPlanInternal(org.apache.flink.table.planner.delegation.PlannerBase planner, ExecNodeConfig config) Internal method, translates this node into a Flink operator.

Methods inherited from class org.apache.flink.table.planner.plan.nodes.exec.ExecNodeBase

accept, createFormattedTransformationDescription, createFormattedTransformationName, createTransformationDescription, createTransformationMeta, createTransformationMeta, createTransformationName, createTransformationUid, getContextFromAnnotation, getDescription, getId, getInputEdges, getInputProperties, getOutputType, getPersistedConfig, getSimplifiedName, inputsContainSingleton, replaceInputEdge, setCompiled, setInputEdges, translateToPlan

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.table.planner.plan.nodes.exec.ExecNode

accept, getDescription, getId, getInputEdges, getInputProperties, getOutputType, replaceInputEdge, setCompiled, setInputEdges

Methods inherited from interface org.apache.flink.table.planner.plan.nodes.exec.ExecNodeTranslator

translateToPlan

Field Detail

MINI_BATCH_ASSIGNER_TRANSFORMATION

public static final String MINI_BATCH_ASSIGNER_TRANSFORMATION

See Also:

Constant Field Values

FIELD_NAME_MINI_BATCH_INTERVAL

public static final String FIELD_NAME_MINI_BATCH_INTERVAL

See Also:

Constant Field Values

Constructor Detail

StreamExecMiniBatchAssigner

public StreamExecMiniBatchAssigner(org.apache.flink.configuration.ReadableConfig tableConfig,
                                   MiniBatchInterval miniBatchInterval,
                                   InputProperty inputProperty,
                                   org.apache.flink.table.types.logical.RowType outputType,
                                   String description)

StreamExecMiniBatchAssigner

public StreamExecMiniBatchAssigner(int id,
                                   ExecNodeContext context,
                                   org.apache.flink.configuration.ReadableConfig persistedConfig,
                                   MiniBatchInterval miniBatchInterval,
                                   List<InputProperty> inputProperties,
                                   org.apache.flink.table.types.logical.RowType outputType,
                                   String description)

Method Detail

translateToPlanInternal

protected org.apache.flink.api.dag.Transformation<org.apache.flink.table.data.RowData> translateToPlanInternal(org.apache.flink.table.planner.delegation.PlannerBase planner,
                                                                                                               ExecNodeConfig config)

Description copied from class: ExecNodeBase

Internal method, translates this node into a Flink operator.

Specified by:

translateToPlanInternal in class ExecNodeBase<org.apache.flink.table.data.RowData>

Parameters:

planner - The planner.

config - per-ExecNode configuration that contains the merged configuration from various layers which all the nodes implementing this method should use, instead of retrieving configuration from the planner. For more details check ExecNodeConfig.