# Introduction to Statistics (Part-II)

Standard Deviation

Standard deviation tells us about the concentration of data around the mean of the data set. In simple words, we can say how much data is deviated from the mean. Its symbol is σ (the Greek letter sigma). It is the square root of the Variance. So now you ask, “What is the Variance?”

Variance

The average of the squared differences from the Mean.

To calculate the variance follow these steps:

• Work out the Mean (the simple average of the numbers)
• Then for each number: subtract the Mean and square the result (the squared difference).
• Then work out the average of those squared differences. …

# Introduction to Statistics (Part-I)

Scope of Statistics

In the real world, the data plays a huge role to participate in real-time transactions. To deal with the data we need to know How the data is structured? or How the data can be interpreted for the next level of study?. Statistics used to organize, collect and analyze the data. From the end able to study the data structure. How do the data look like? Is the data is well enough to participate in machine learning.

Statistics divided into two broad categories, namely Descriptive and Inferential.

Descriptive Statistics

It is the method of organizing and summarizing data in an informative way. Say example consider the retail company to…

# Scrap amazon products and pricing data with Heroku deployment

What is the need for scrapping?

Today’s world the data plays a vital role in every business perspective. With the help of data, we can satisfy the business requirements. The data can be any form either in Pdf, Excel, Word, webpage data. Some of the files have come in the form of handy. Where you can read and extract the data in a very simple form. In cases of reading the webpage data, How are we going to achieve it? Save the data in another form? Handwritten document based on the webpage data. All the ideas are coming to the mind. This comes to the web scrapping plays a role. From the web scrapping able to extract the data from web sites and save in another format that can be any either in XLS, PDF, CSV. Using the Python we can learn and implement the web scrapping within an hour. …

Flask is a micro web framework written in Python

What is a web framework?

Web Application Framework represents a collection of libraries and modules that enables a web application developer to write applications without having to bother about low-level details such as protocols, thread management etc.

Using the web framework, we can develop dynamic websites, web services and web applications. For each web framework having their own standard for development and deployment in server. In today’s world, there is a lot of web framework plays a vital role. We going to cover this section on Flask. …

# What is MongoDB?

MongoDB is a NoSQL database — a NoSQL database is one where you don’t query the database with SQL. Other than that NoSQL really means nothing to define a database. So let’s have another go at defining MongoDB.
MongoDB is a JSON document datastore. It allows you to store and query JSON style documents with a few smarts on top. This means that you can store objects with nested data all in one collection (collections are like tables for MongoDB).
It is generally faster than traditional SQL databases for transactional stores but lacks the query power for Analytical usage.

# Why MongoDB?

When we needed a NoSQL database we chose MongoDB for a number of…

# Importing data from a MySQL database into Pandas data frame

This article illustrates the basic operation of how the dataset imported from the table. The database is taken as MySQL.

Is database is an essential thing for DataScience?

Yes, It is an essential thing. Without the data, the data analysis and forecasting can’t be done. Right!. The data can be any kind of format that may be in CSV, XLS. Consider the retail organization selling their multiple products from the past 5 years. The company decided to forecast the prediction of sales for the next two years.

I grasped what you thinking? Basically, try the data export to CSV file. That can be easily handled in the pandas data frame. But the things not working in such way because prediction does not rely on a single table. It depends on multiple tables say transaction, sales, customer review, the revenue of product from 5 years those need to look in to determine the prediction. The best-opted way will be directly importing the table to the data frame. …

# Data Visualization using Matplotlib and Seaborn

What is Data Visualization?

Today’s world a lot of data is going everywhere. The data is getting increasing every day. We can see in real-time right from the mobile. Using social media, Mails, Bank transactions keep increasing day by day. Is it possible to view the massive data in the formal way of representation? Yes, we can do via data visualization. The data visualization is a graphical representation of data. In the big data world, there are several data visualization tools capable of analysing the massive data used for decision making.

Today, we are going to implement data visualization using a dataset from UCI. …

# Visualizing Geospatial Data with Python using Folium

Data visualization is a broader term that describes any effort to help people understand the importance of data by placing it in a visual context. Patterns, trends, and correlations can be easily shown visually which otherwise might go unnoticed in textual data. It is a fundamental part of the data scientist’s toolkit. Creating visualisations is pretty easy but creating good ones is much harder. It requires an eye for detail and a good amount of expertise to create visualisations which are simple yet effective. Powerful visualisation tools and libraries are available today which have redefined the meaning of visualisation.

The beauty of using Python is that it offers libraries for every data visualisation need. One such library is Folium which comes in handy for visualising Geographic data (Geodata). Geographic data (Geodata) science is a subset of data science that deals with location-based data i.e description of objects and their relationship in space. …

# Reading data from PDF using tabula-py

Do you think really need PDF in Data science?

Yes, In real-world scenarios there are chances of having dataset in any formats. We should be knowing How to tackle/read the datasets in such scenarios.

Today we are going to see how to read the data from PDF file?

To achieve we need to install the library that supports reading the PDF file. Yes, the answer is here. From tabula-py, we can read the PDF and do a lot more of manipulations using PDF.

tabula-py Installation

Go to Anaconda command prompt, try using below command

`pip install tabula-py`

Finally, you will be getting the screen as below. …

# A walkthrough in pandas

In today’s world technology plays a vital role to change our lives. Especially the machine learning the next era of advanced computing. Do you think it’s the next era? It’s already started across the various fields as banking, retail, financial, medical, agriculture.

Well, I am too excited about the stuff which I have learnt in Data Science. So, I will be sharing my knowledge of pandas manipulation.

I am using Jupyter notebook for Pandas. Here I will start the basic thing right from the beginning of reading dataset. To be expertise in anything we need to have hands-on experience. So, I grasped the idea of why don’t we use the real-time dataset rather render in CSV directly. I’ve visited the site named as University of California, Irvine. They have enclosed a lot of dataset across various streams. …