What is Apache Nifi?
Apache NiFi is a dataflow system based on the concepts of flow-based programming. It allows almost any data ingestion or refinement like transformation, data routing or logic mediation.
NIFI key features include:
- Web Interface to design, control and monitor the flows without writing a single line of code.
- Extensive configuration that makes your installation fit perfectly your data needs like: low latency vs. high throughput, loss tolerant vs guaranteed delivery or back pressure
- Data Provenance to track data from the beginning of the flow to the end.
- Designed for extension no matter where your dataflow starts or ends, connectors can be written and plugged-in.
Two key concepts of Apache Nifi are:
- FlowFile: Piece of data within NiFi. It consists of two parts: the content and its attributes.
- Processor: NiFi component responsible for processing FlowFiles. These may be creating, sending, receiving, transforming, routing, splitting, merging, and processing FlowFiles.
Apache Nifi installation
Standalone single node Apache Nifi can be easily installed locally on Linux, Windows or Mac. For Mac this can be as easy as:
brew install nifi nifi start
You may also need to set JAVA_HOME path as:
JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_172.jdk/Contents/Home/ export JAVA_HOME;
It can be also done with Docker on Linux and Mac:
docker run -d -P -h nifi -p 8080:8080 -p 8181:8181 --name nifi_1.6.0 --memory=4g -v /docker/nifiapache/nifi apache/nifi:1.6.0
It takes just a moment untlil localhost:8080/nifi shows Nifi Web UI.