Skip to main content
Version: v1.4.0

NiFi cluster

NifiCluster describes the desired state of the NiFi cluster we want to setup through the operator.

apiVersion: nifi.konpyutaika.com/v1
kind: NifiCluster
metadata:
name: simplenifi
spec:
service:
headlessEnabled: true
annotations:
tyty: ytyt
labels:
tete: titi
pod:
annotations:
toto: tata
labels:
titi: tutu
zkAddress: 'zookeepercluster-client.zookeeper:2181'
zkPath: '/simplenifi'
clusterImage: 'apache/nifi:1.11.3'
oneNifiNodePerNode: false
nodeConfigGroups:
default_group:
isNode: true
podMetadata:
annotations:
node-annotation: "node-annotation-value"
labels:
node-label: "node-label-value"
externalVolumeConfigs:
- name: example-volume
mountPath: "/opt/nifi/example"
secret:
secretName: "raw-controller"
storageConfigs:
- mountPath: '/opt/nifi/nifi-current/logs'
name: logs
# Metadata to attach to the PVC that gets created
metadata:
labels:
my-label: my-value
annotations:
my-annotation: my-value
pvcSpec:
accessModes:
- ReadWriteOnce
storageClassName: 'standard'
resources:
requests:
storage: 10Gi
serviceAccountName: 'default'
resourcesRequirements:
limits:
cpu: '2'
memory: 3Gi
requests:
cpu: '1'
memory: 1Gi
nodes:
- id: 1
nodeConfigGroup: 'default_group'
- id: 2
nodeConfigGroup: 'default_group'
propagateLabels: true
nifiClusterTaskSpec:
retryDurationMinutes: 10
listenersConfig:
internalListeners:
- type: 'http'
name: 'http'
containerPort: 8080
- type: 'cluster'
name: 'cluster'
containerPort: 6007
- type: 's2s'
name: 's2s'
containerPort: 10000
externalServices:
- name: 'clusterip'
spec:
type: ClusterIP
portConfigs:
- port: 8080
internalListenerName: 'http'
metadata:
annotations:
toto: tata
labels:
titi: tutu

NifiCluster

FieldTypeDescriptionRequiredDefault
metadataObjectMetadatais metadata that all persisted resources must have, which includes all objects users must create.Nonil
specNifiClusterSpecdefines the desired state of NifiCluster.Nonil
statusNifiClusterStatusdefines the observed state of NifiCluster.Nonil

NifiClusterSpec

FieldTypeDescriptionRequiredDefault
clientTypeEnum=basicdefines if the operator will use basic or tls authentication to query the NiFi cluster.Notls
typeEnum=internaldefines if the cluster is internal (i.e manager by the operator) or external.Nointernal
nodeURITemplatestringused to dynamically compute node uri.if external type-
nifiURIstringused access through a LB uri.if external type-
rootProcessGroupIdstringcontains the uuid of the root process group for this cluster.if external type-
secretRef[ ]SecretReferencereference the secret containing the informations required to authentiticate to the cluster.if external type-
proxyUrlstringdefines the proxy required to query the NiFi cluster.if external type-
serviceServicePolicydefines the policy for services owned by NiFiKop operator.No-
podPodPolicydefines the policy for pod owned by NiFiKop operator.No-
zkAddressstringspecifies the ZooKeeper connection string in the form hostname:port where host and port are those of a Zookeeper server.No""
zkPathstringspecifies the Zookeeper chroot path as part of its Zookeeper connection string which puts its data under same path in the global ZooKeeper namespace.Yes"/"
initContainerImagestringcan override the default image used into the init container to check if ZoooKeeper server is reachable.Yes"bash"
initContainers[ ]stringdefines additional initContainers configurations.No[ ]
clusterImagestringcan specify the whole nificluster image in one place.No""
oneNifiNodePerNodebooleanif set to true every nifi node is started on a new node, if there is not enough node to do that it will stay in pending state. If set to false the operator also tries to schedule the nifi node to a unique node but if the node number is insufficient the nifi node will be scheduled to a node where a nifi node is already running.Nonil
propagateLabelsbooleanwhether the labels defined on the NifiCluster metadata will be propagated to resources created by the operator or not.Yesfalse
managedAdminUsers[ ]ManagedUsercontains the list of users that will be added to the managed admin group (with all rights).No[]
managedReaderUsers[ ]ManagedUsercontains the list of users that will be added to the managed admin group (with all rights).No[]
readOnlyConfigReadOnlyConfigspecifies the read-only type Nifi config cluster wide, all theses will be merged with node specified readOnly configurations, so it can be overwritten per node.Nonil
nodeUserIdentityTemplatestringspecifies the template to be used when naming the node user identity (e.g. node-%d-mysuffix)Yes"node-%d-<cluster-name>"
nodeConfigGroupsmap[string]NodeConfigspecifies multiple node configs with unique nameNonil
nodes[ ]Nodespecifies the list of cluster nodes, all node requires an image, unique id, and storageConfigs settingsYesnil
disruptionBudgetDisruptionBudgetdefines the configuration for PodDisruptionBudget.Nonil
ldapConfigurationLdapConfigurationspecifies the configuration if you want to use LDAP.Nonil
singleUserConfigurationSingleUserConfigurationspecifies the configuration if you want to use SingleUser.Nonil
nifiClusterTaskSpecNifiClusterTaskSpecspecifies the configuration of the nifi cluster Tasks.Nonil
listenersConfigListenersConfigspecifies nifi's listener specifig configs.No-
sidecarConfigs[ ]ContainerDefines additional sidecar configurations. [Check documentation for more informations]
externalServices[ ]ExternalServiceConfigsspecifies settings required to access nifi externally.No-
topologySpreadConstraints[ ]TopologySpreadConstraintspecifies any TopologySpreadConstraint objects to be applied to all nodes.Nonil
nifiControllerTemplatestringNifiControllerTemplate specifies the template to be used when naming the node controller (e.g. %s-mysuffix) Warning: once defined don't change this value either the operator will no longer be able to manage the clusterYes"%s-controller"
controllerUserIdentitystringControllerUserIdentity specifies what to call the static admin user's identity Warning: once defined don't change this value either the operator will no longer be able to manage the clusterYesfalse

NifiClusterStatus

FieldTypeDescriptionRequiredDefault
nodesStatemap[string]NodeStateStore the state of each nifi node.No-
StateClusterStateStore the state of each nifi node.Yes-
rootProcessGroupIdstringcontains the uuid of the root process group for this cluster.No-

ServicePolicy

FieldTypeDescriptionRequiredDefault
headlessEnabledbooleanspecifies if the cluster should use headlessService for Nifi or individual services using service per nodes may come an handy case of service mesh.Yesfalse
serviceTemplatestringspecifies the template to be used when naming the service.YesIf headlessEnabled = true ? "%s-headless" = "%s-all-node"
annotationsmap[string]stringAnnotations specifies the annotations to attach to services the NiFiKop operator createsNo-
labelsmap[string]stringLabels specifies the labels to attach to services the NiFiKop operator createsNo-

PodPolicy

FieldTypeDescriptionRequiredDefault
annotationsmap[string]stringAnnotations specifies the annotations to attach to pods the NiFiKop operator createsNo-
labelsmap[string]stringLabels specifies the Labels to attach to pods the NiFiKop operator createsNo-
hostAliases[ ]HostAliasA list of host aliases to include in every pod's /etc/hosts configuration in the scenario where DNS is not available.No[ ]
readinessProbeProbeThe readiness probe that the Pod is configured with. If not provided, a default will be used.Nonil
livenessProbeProbeThe liveness probe that the Pod is configured with. If not provided, a default will be used.Nonil

ManagedUsers

FieldTypeDescriptionRequiredDefault
identitystringidentity field is use to define the user identity on NiFi cluster side, it use full when the user's name doesn't suite with Kubernetes resource name.No-
namestringname field is use to name the NifiUser resource, if not identity is provided it will be used to name the user on NiFi cluster side.Yes-

DisruptionBudget

FieldTypeDescriptionRequiredDefault
createboolif set to true, will create a podDisruptionBudget.No-
budgetstringthe budget to set for the PDB, can either be static number or a percentage.Yes-

LdapConfiguration

FieldTypeDescriptionRequiredDefault
enabledbooleanif set to true, we will enable ldap usage into nifi.properties configuration.Nofalse
urlstringspace-separated list of URLs of the LDAP servers (i.e. ldap://${hostname}:${port}).No""
searchBasestringbase DN for searching for users (i.e. CN=Users,DC=example,DC=com).No""
searchFilterstringFilter for searching for users against the 'User Search Base'. (i.e. sAMAccountName=0). The user specified name is inserted into '0'.No""

SingleUserConfiguration

FieldTypeDescriptionRequiredDefault
enabledbooleanspecifies whether or not the cluster should use single user authentication for NifiNofalse
authorizerEnabledbooleanspecifies if the cluster should use use the single-user-authorizer instead of the managed-authorizer (if enabled, the creation of users and user groups will not work in NiFi, and the single user will have no rights by default.)Notrue
secretRefSecretReferencereferences the secret containing the informations required to authentiticate to the clusterNonil
secretKeysUserSecretKeysreferences the keys from the secret containing the user name and password.No{username:"username", password:"password"}

NifiClusterTaskSpec

FieldTypeDescriptionRequiredDefault
retryDurationMinutesintdescribes the time the operator waits before going back and retrying a cluster task, which can be: scale up, scale down, rolling upgrade..Yes5

ClusterState

NameValueDescription
NifiClusterInitializingClusterInitializingstates that the cluster is still in initializing stage
NifiClusterInitializedClusterInitializedstates that the cluster is initialized
NifiClusterReconcilingClusterReconcilingstates that the cluster is still in reconciling stage
NifiClusterRollingUpgradingClusterRollingUpgradingstates that the cluster is rolling upgrading
NifiClusterRunningClusterRunningstates that the cluster is in running state
NifiClusterNoNodesNifiClusterNoNodesstates that the cluster has no nodes

UserSecretKeys

FieldTypeDescriptionRequiredDefault
usernamestringspecifies he name of the secret key to retrieve the user name.Nousername
passwordstringspecifies he name of the secret key to retrieve the user password.Nopassword