Skip to main content
Version: v1.6.0

NiFi Connection

NifiConnection is the Schema for the NiFi connection API.

apiVersion: nifi.konpyutaika.com/v1alpha1
kind: NifiConnection
metadata:
name: connection
namespace: instances
spec:
source:
name: input
namespace: instances
subName: output_1
type: dataflow
destination:
name: output
namespace: instances
subName: input_1
type: dataflow
configuration:
flowFileExpiration: 1 hour
backPressureDataSizeThreshold: 100 GB
backPressureObjectThreshold: 10000
loadBalanceStrategy: PARTITION_BY_ATTRIBUTE
loadBalancePartitionAttribute: partition_attribute
loadBalanceCompression: DO_NOT_COMPRESS
prioritizers:
- NewestFlowFileFirstPrioritizer
- FirstInFirstOutPrioritizer
labelIndex: 0
bends:
- posX: 550
posY: 550
- posX: 550
posY: 440
- posX: 550
posY: 88
updateStrategy: drain

NifiDataflow

FieldTypeDescriptionRequiredDefault
metadataObjectMetadatais metadata that all persisted resources must have, which includes all objects dataflows must create.Nonil
specNifiConnectionSpecdefines the desired state of NifiDataflow.Nonil
statusNifiConnectionStatusdefines the observed state of NifiDataflow.Nonil

NifiConnectionSpec

FieldTypeDescriptionRequiredDefault
sourceComponentReferencethe Source component of the connection.Yes-
destinationComponentReferencethe Destination component of the connection.Yes-
configurationConnectionConfigurationthe version of the flow to run.Yes-
updateStrategyComponentUpdateStrategydescribes the way the operator will deal with data when a connection will be deleted: Drop or DrainYesdrain

NifiConnectionStatus

FieldTypeDescriptionRequiredDefault
connectionIDstringconnection ID.Yes-
stateConnectionStatethe connection current state.Yes-

ComponentUpdateStrategy

NameValueDescription
DrainStrategydrainleads to block stopping of input/output component until they are empty.
DropStrategydropleads to dropping all flowfiles from the connection.

ConnectionState

NameValueDescription
ConnectionStateCreatedCreateddescribes the status of a NifiConnection as created.
ConnectionStateOutOfSyncOutOfSyncdescribes the status of a NifiConnection as out of sync.
ConnectionStateInSyncInSyncdescribes the status of a NifiConnection as in sync.

ComponentReference

NameValueDescriptionRequiredDefault
namestringthe name of the component.Yes-
namespacestringthe namespace of the component.Yes-
typeComponentTypethe type of the component (e.g. nifidataflow).Yes-
subNamestringthe name of the sub component (e.g. queue or port name).No-

ComponentType

NameValueDescription
ComponentDataflowdataflowindicates that the component is a NifiDataflow.
ComponentInputPortinput-portindicates that the component is a NifiInputPort. (not implemented)
ComponentOutputPortoutput-portindicates that the component is a NifiOutputPort. (not implemented)
ComponentProcessorprocessorindicates that the component is a NifiProcessor. (not implemented)
ComponentFunnelfunnelindicates that the component is a NifiFunnel. (not implemented)
ComponentProcessGroupprocess-groupindicates that the component is a NifiProcessGroup. (not implemented)

ConnectionConfiguration

NameValueDescriptionRequiredDefault
flowFileExpirationstringthe maximum amount of time an object may be in the flow before it will be automatically aged out of the flow.No-
backPressureDataSizeThresholdstringthe maximum data size of objects that can be queued before back pressure is applied.No1 GB
backPressureObjectThreshold*int64the maximum number of objects that can be queued before back pressure is applied.No10000
loadBalanceStrategyConnectionLoadBalanceStrategyhow to load balance the data in this Connection across the nodes in the cluster.NoDO_NOT_LOAD_BALANCE
loadBalancePartitionAttributestringthe FlowFile Attribute to use for determining which node a FlowFile will go to.No-
loadBalanceCompressionConnectionLoadBalanceCompressionwhether or not data should be compressed when being transferred between nodes in the cluster.NoDO_NOT_COMPRESS
prioritizers[ ]ConnectionPrioritizerthe comparators used to prioritize the queue.No-
labelIndex*int32the index of the bend point where to place the connection label.No-
bends[ ]ConnectionBendthe bend points on the connection.No-

ConnectionLoadBalanceStrategy

NameValueDescription
StrategyDoNotLoadBalanceDO_NOT_LOAD_BALANCEdo not load balance FlowFiles between nodes in the cluster.
StrategyPartitionByAttributePARTITION_BY_ATTRIBUTEdetermine which node to send a given FlowFile to based on the value of a user-specified FlowFile Attribute. All FlowFiles that have the same value for said Attribute will be sent to the same node in the cluster.
StrategyRoundRobinROUND_ROBINflowFiles will be distributed to nodes in the cluster in a Round-Robin fashion. However, if a node in the cluster is not able to receive data as fast as other nodes, that node may be skipped in one or more iterations in order to maximize throughput of data distribution across the cluster.
StrategySingleSINGLEall FlowFiles will be sent to the same node. Which node they are sent to is not defined.

ConnectionLoadBalanceCompression

NameValueDescription
CompressionDoNotCompressDO_NOT_COMPRESSflowFiles will not be compressed.
CompressionCompressAttributesOnlyCOMPRESS_ATTRIBUTES_ONLYflowFiles' attributes will be compressed, but the flowFiles' contents will not be.
CompressionCompressAttributesAndContentCOMPRESS_ATTRIBUTES_AND_CONTENTflowFiles' attributes and content will be compressed.

ConnectionPrioritizer

NameValueDescription
PrioritizerFirstInFirstOutPrioritizerFirstInFirstOutPrioritizergiven two FlowFiles, the one that reached the connection first will be processed first.
PrioritizerNewestFlowFileFirstPrioritizerNewestFlowFileFirstPrioritizergiven two FlowFiles, the one that is newest in the dataflow will be processed first.
PrioritizerOldestFlowFileFirstPrioritizerOldestFlowFileFirstPrioritizergiven two FlowFiles, the one that is oldest in the dataflow will be processed first. 'This is the default scheme that is used if no prioritizers are selected'.
PrioritizerPriorityAttributePrioritizerPriorityAttributePrioritizergiven two FlowFiles, an attribute called “priority” will be extracted. The one that has the lowest priority value will be processed first.

ConnectionBend

NameValueDescriptionRequiredDefault
posX*int64the x coordinate.No-
posY*int64the y coordinate.No-