Cloud Foundry · Microservices

Rapid Application Development with Cloud Foundry

Cloud Foundry is a Platform as a Service, that is PaaS, it helps in rapid application development by freeing developer from the hassle of application deployment and subsequent management of the run time. There are numerous tutorials out on internet to learn Cloud Foundry architecture and concepts therefore I will not focus on explaining CF… Continue reading Rapid Application Development with Cloud Foundry

Spark

Streaming analysis of Twitter data using HDP2.4, Spark, Mongodb on Azure

This post is about streaming analysis of live twitter data on Azure using spark on HORTONWORKS and persisting data to mongodb. With this post also the main idea is to help out flocks in troubleshooting. I will post issues which came up with this approach. In my previous post I used AWS and HBase with… Continue reading Streaming analysis of Twitter data using HDP2.4, Spark, Mongodb on Azure

Code · Spark

Spark streaming troubleshooting on cloudera 5.9 AWS cluster:Part3

This post is third in the series ‘Spark streaming troubleshooting on cloudera 5.9 AWS cluster’. Here I have integrated HBase with the spark streaming. Relevant data has been extracted from the live twitter tweets related with USA election results and saved to table electionSentiments on HBase. For this I used 5 worker nodes, 1 gateway… Continue reading Spark streaming troubleshooting on cloudera 5.9 AWS cluster:Part3

Code · Spark

Spark streaming troubleshooting on cloudera 5.9 AWS cluster:Part-2

This is second part of spark streaming troubleshooting on cloudera hadoop cluster on aws. Here I will focus on the code- I created an application to process the live tweets which contains words related with USA 2016 elections. Below program connects to twitter and fetches tweets which have words such as Hillary, Donad, Trump, Obama, Clinton… Continue reading Spark streaming troubleshooting on cloudera 5.9 AWS cluster:Part-2