The OpenSky Network makes air traffic data public. It is a community-based receiver network that publishes data realtime. In this post we show, how to fetch the data with Apache NIFI and push it into a Kafka cluster. OpenSky Data and its API We will use OpenSky's REST API to fetch the data. Requesting an URL: https://opensky-network.org/api/states/all?lamin=.. fetches … Continue reading Fetching OpenSky airspace to Kafka with Apache NIFI
How to fetch Bitcoin transactions with Apache Nifi? Neat and keep-it-simple solution based on BigQuery public dataset.
Step by step tutorial to export Google Analytics data to Apache Hive and integrate clickstream with other data at Your company.
Synergy between BigQuery datasets and datasets outside it. Quick glance on our custom executeBigQueryProcessor fetch public Open Air Quality dataset from BigQuery.
Google BigQuery Public Datasets delivers valuable data like Open Air Quality. This and other tables in BiqQuery can be fetched with Apache Nifi.
Whatever your business is, weather conditions affect its performance. Apache Nifi and its variety of connectors seems to be a perfect tool to ingest weather observations from public datasets like Deutscher Wetterdienst.
Fetching Facebook data can be extremely easy with Nifi and custom GetFacebookProcessor.
Analysing social media data can help you drive the business. In many cases, the data is freely available. The only missing gap is a process for migrating it to an ecosystem where it can be analysed.
Apache NiFi is a dataflow system based on the concepts of flow-based programming. It allows almost any data ingestion or refinement like transformation, data routing or logic mediation.
According to Gartner definition, Big Data stands also for high-variety of data sources and requires innovative forms for data pipelines.