GoLang ORM – GORM

In this blogs, I am creating a Kubernetes Client using GoLang to add custom feature which is not present in Kubernetes Vanilla Installations or Kubernetes…

Making an automatic ‘ON’ Switch using only relay and a charger!!!

Materials needed: Hi everyone I will be showing you how to make a switch that automatically switches on when there is a power outage if…

Kubernetes Operations Console

We need a framework using which we can build and manage a Kubernetes Cluster. When it comes to building a Kubernetes cluster, Kos Console provides you…

My Big Data solution using AWS services…

Goal A global advertising agency that manages marketing for different customers in Asia, Europe and US required the solution on development of a Big Data…

Part -2: Operators teach Kubernetes how to simplify stateful application…

I hope you have enjoyed my first article(link below) on Operator extension and Kubernetes Introduction. Now level up to this series another article where I’ll…

Operators teach Kubernetes how to simplify the stateful application…

This is the first article to a series of articles to showcase how we use Operator that can leverage Kubernetes to create a stateful application…

LAMP stack in Cloud: Building a Scalable, Secure and Highly Available architecture using AWS

1. Requirement Overview The acronym LAMP (Linux, Apache, MySQL, PHP) refers to an open-source stack, used to run dynamic and static content of servers. A…

Reference architecture of bigdata solution in GCP and Azure…

This article is a showcase of a Reference architecture approach for the financial sector where stream and batch processing is a common part of its…

Error resolution of Zalando Research Flair NLP package installation on Centos 7, “Failed building wheel for regex…”​

I was working on an NLP tool for evaluation purposes and found an issue in creating the environment. They had set up everything on Ubuntu…

How to install and create Python 3.6 virtualenv on HDP 3.0.1

Many times the default version bundled with HDP i.e 2.7 is not sufficient to explore a few libraries and it required python to add an…

How to create an Apache Beam data pipeline and deploy it using Cloud Dataflow in Java

Cloud Dataflow is a fully managed google service for executing data processing pipelines using Apache Beam. What do you mean by fully managed? Cloud dataflow…

Google Dataflow Python ValueError: Unable to get the Filesystem for path gs://myprojetc/digport/ports.csv.gz

I am using google cloud to create an event on Cloud Storage to Big Query using Apache Beam pythons library. I was executing an ETL…

Python: Stream the ingest of data into the database in real-time using dataflow.

In my previous articles, we solve real-time data ingestion problems using various tools like Apache Kafka, Storm, Flink and Spark. I have shown you in…

Sample Java Program on Google Cloud Pub/Sub

Overview This article contains a sample java program on Google Cloud’s Pub/Sub to publish messages from google store. The solution is simple to set up…

Sample Dataflow Pipeline featuring Cloud Pub/Sub, Dataflow, and BigQuery…

Streaming data in Google Cloud Platform is typically published to Cloud Pub/Sub, a serverless real-time messaging service. Cloud Pub/Sub provides reliable delivery and can scale…

Solved: Protocol tcp Port Exclusion issues when running Hadoop on Windows Docker

If you’re looking for simple and painless Hadoop deployment, Docker is the right tool for you. deployment. We mostly use Docker community edition-CE (https://docs.docker.com/docker-for-windows/install/) on…

Technical debt : Understand and manage it…

As we go on writing a piece of code or building a solution we always create some sort of technical debt. It is not always…

PowerShell script wrappers using the Microsoft Azure AzCopy.exe tool

Use case We are working on building data lake in Azure using Azure container, ADF, Azure DWH, Databricks and many other services of Azure. After…

Azure Arc – redefine hybrid cloud…

Azure delivered 59% revenue growth in the latest quarter which is more than expected from its other Microsoft products. MSFT introducing various new cloud services…

Leader in Me!!

Information cascade I learned a TON very quickly and everyday about completely new stuff, and was able to do so because I could easily feel…

My experience with HCL interview…

I am having 13+ years of experience and got the call from HR located in Hyderabad bspraviya_b@hcl.com for the interview for the position of Solution…

Apache Storm key takeaways…

Hadoop moves the code to the data, Storm moves the data to the code. This behavior makes more sense in a stream-processing system, because the…

Approach to execute Machine Learning project, “Halt the Hate”…

Disclaimer: The analysis was done in this project touches a sensitive issue in India. So I never convince anybody to trust my model. A real…

Fundamantals of Apache Spark…

You can view my other articles on Spark RDD at below links… Apache Spark RDD API using Pyspark… Tips and Tricks for Apache Spark RDD…

Bayesian-posterior imagination and applications…

Before going into Bayes and posterior probability let us first understand few terms we going to use:- Conditional Probability:- Conditional Probability and Independence:- A conditional…

Tao: The Balance, Simplicity, Oneness…

Thanks to James Quirk for sharing this book… To read full article please visit to my Linkedin Page..

Tips and Tricks for Apache Spark RDD API, Dataframe API- Part -1

I am planning to share my knowledge on Apache Spark RDD, Dataframes API and some tips and tricks. If I combine everything into one then it…

Better late then never : Time to replace your micro-service architecture with Kafka…

Kafka already spawns and facilitated many organizations on micro-services architecture world. If Kafka is still not part of your infrastructure, its high time for you…

In-depth Kafka Message queue principles of high-reliability

 At present many open source distributed processing systems such as Cloudera, Apache Storm, Spark and others support the integration with Kafka. Kafka is increasingly being…

Short introduction to pandas…

Please visit my git for the complete exercise at the location.

Python numpy excercise…

Python numpy exercises are present in my git repository at location. Happy Machine Learning…

Connecting Two HC-05 Bluetooth Together Using Arduino

Hey friends I am going to connect two Arduino bluetooths(HC-05) together. We don’t need much just an Arduino, two Bluetooths, and a set of wires.…

JavaScript Issue resolution in JupyterLab Notebook

The graphs are not appearing in JupyterLab Notebook and the error message says “JavaScript output is disabled in JupyterLab”. At first, it seems that from…

Understanding distribution functions…

This article helps to understand distribution functions and its usage in Exploratory Data Analysis in Data Science. In next article, I’ll take you to some…

Arduino Radar Scanner

Hello I am using Arduino, Ultrasonic Sensor, and Servo Tower Pro to make a simple radar scanner. Also, we’ll be needing a free software named…

My intuition to understand eigenvalues and eigenvectors…

One of my biggest hurdles learning linear algebra was getting the intuition of learning Algebra. Eigenvalues and eigenvectors are one of those things that pop…

How to convert Python list, tuples, strings to each other…

There are three built-in functions in Python : lists, tuples, and strings. The three functions, str (), tuple (), and list (), convert to each other…

Tips and Tricks for Apache Spark RDD API, Dataframe API- Part -1

I am planning to share my knowledge on Apache Spark RDD, Dataframes API and some tips and tricks. If I combine everything into one then it…

Better late then never : Time to replace your micro-service architecture with Kafka…

Kafka already spawns and facilitated many organizations on micro-services architecture world. If Kafka is still not part of your infrastructure, its high time for you…

In-depth Kafka Message queue principles of high-reliability

At present many open source distributed processing systems such as Cloudera, Apache Storm, Spark and others support the integration with Kafka. Kafka is increasingly being…

SolrCloud : CAP theorem world, this makes Solr a CP system, and keep availability in certain circumstances.

A SolrCloud cluster holds one or more distributed indexes which are called Collections. Each Collection is divided into shards (to increase write capacity) and each…

Google Cloud Platform(GCP) overview

Google Cloud Platform – GCP is a collection of various services of SaaS, PaaS, and IaaS, and new services are still being launched day by…

Consume JSON Messages From Kafka Using Kafka-Python’s Deserializer

Hope you are here when you want to take a ride on Python and Apache Kafka. Kafka-Python is most popular python library for Python. For…

Apache Eagle: Real-time security monitoring solution

On January 10, 2017, the Apache Software Foundation, which consists of more than 350 open source projects and innovation initiatives, all developed by volunteer, governance…

Hbase Administration using HBaseFsck (hbck) and other tools…

HBaseFsck (hbck) is a tool for checking for region consistency and table integrity problems and repairing a corrupted HBase. Sometime we need to run hbck…

Advertisement attributes or Ad Attributes…An Idea!!!

Some time ago i was working on an idea called as Ad Attributes or Advertisement attributes. I’d like to share my thoughts on this idea with audience.…

We just need to be better to each other before talking to AI

I’m not going to talk about statistics, machine learning, or AI, not even comparing any database which shows error made by AI work vs Manual…

ZeroMQ Part-1

  Programs like people need to communicate, and for them we have the UDP, TCP, HTTP, IPX, WebSocket protocol to connect and other related applications.…

Book Review : The Folly of Fools

The Folly of Fools: The Logic of Deceit and Self-Deception in Human Life Finished reading this books and developed friendships with author “Robert L. Trivers”…

My advise for work under Zombie project and toxic workplace…

I would like to say that there’s a chance of salvaging the Zombie project but there probably isn’t and its not your fault. I have…