Pandas Tutorial

In this Python Pandas tutorial, you will learn how to install and import Pandas, create Series and DataFrame objects, select and filter data, handle missing values, group rows, combine datasets, and read or write common file formats. The examples use small datasets so that each operation and its result are easy to follow.

What Is Pandas in Python?

Pandas is a Python library for working with structured and tabular data. It provides labelled data structures and functions for cleaning, transforming, analysing, reshaping, and exporting datasets.

Pandas is commonly used with data from CSV files, spreadsheets, SQL queries, JSON documents, and Python collections. Its two primary data structures are Series for one-dimensional labelled data and DataFrame for two-dimensional tabular data.

Pandas DataFrame Tutorial Topics

DataFrame

Create

• Pandas – Create an Empty DataFrame
• Pandas – Create DataFrame from Dictionary

Access

• Pandas – Access Value at specific Row/Column of DataFrame
• Pandas – Get Column Names of DataFrame
• Pandas – Set Column Names of DataFrame
• Pandas – Get Shape or Dimensions of DataFrame
• Pandas – Get Index of DataFrame
• Pandas – Set Index for DataFrame

Checks

• Pandas – Check if DataFrame is Empty

Install Pandas Library

To install pandas, use the following pip command.

 pip install pandas

If your system has multiple Python installations, run pip through the interpreter that will execute the program:

python -m pip install pandas

On systems where the Python 3 executable is named python3, use:

python3 -m pip install pandas

You can verify the installed version from a Python program or interactive shell.

import pandas as pd

print(pd.__version__)

Import Pandas

To import pandas python library, use the following statement in your program before using pandas classes.

 import pandas

Usually an alias is used for pandas while using in a program. Use the following import statement.

 import pandas as pd

The alias pd is a common convention. It keeps expressions such as pd.DataFrame() and pd.read_csv() concise while making it clear that the functions come from Pandas.

Pandas Datastructure

Pandas has two types of Datastructures to deal with the data. They are:

1. DataFrame
2. Series

A Series contains one-dimensional labelled values. A DataFrame contains rows and columns, and each DataFrame column is itself a Series.

Create a Pandas Series from a Python List

The following example creates a Series from a list of monthly sales values. Pandas assigns the default integer index 0, 1, and 2.

import pandas as pd

sales = pd.Series([120, 150, 135])
print(sales)

0    120
1    150
2    135
dtype: int64

You can provide meaningful labels through the index argument.

import pandas as pd

sales = pd.Series(
    [120, 150, 135],
    index=["January", "February", "March"]
)

print(sales["February"])

150

Pandas DataFrame

Pandas DataFrame is similar to R DataFrame. It stores two dimensional data with the structure similar to that of a table in databases.

Following is an example DataFrame.

     names  us  india  china
0   Google  68     84     78
1    Apple  74     56     88
2  Samsung  77     73     82
3  OnePlus  78     69     87

Each row has an index, shown on the left, and each column has a label such as names, us, india, or china. Columns can contain different data types, although values within an individual column usually share one dtype.

Create a Pandas DataFrame from a Dictionary

A dictionary of equal-length lists is a direct way to create a DataFrame. Dictionary keys become column names, and list elements become column values.

import pandas as pd

data = {
    "product": ["Keyboard", "Mouse", "Monitor"],
    "price": [45.50, 18.00, 220.00],
    "stock": [12, 30, 8]
}

products = pd.DataFrame(data)
print(products)

    product  price  stock
0  Keyboard   45.5     12
1     Mouse   18.0     30
2   Monitor  220.0      8

Inspect Pandas DataFrame Rows, Columns, and Data Types

Before analysing a dataset, inspect its shape, labels, data types, and a small sample of rows. This helps identify incorrect column types, unexpected missing values, or malformed input.

import pandas as pd

data = {
    "product": ["Keyboard", "Mouse", "Monitor", "Webcam"],
    "price": [45.50, 18.00, 220.00, 62.00],
    "stock": [12, 30, 8, 15]
}

products = pd.DataFrame(data)

print(products.head())
print(products.shape)
print(products.columns)
print(products.dtypes)

• head() returns the first five rows by default.
• tail() returns the last five rows by default.
• shape returns a tuple containing the row and column counts.
• columns returns the column labels.
• dtypes reports the dtype assigned to each column.
• info() prints a compact summary that includes non-null counts and memory usage.

Select Pandas DataFrame Columns and Rows

Select one column with square brackets. The result is a Series.

product_names = products["product"]
print(product_names)

Select multiple columns by passing a list of column labels. The result is a DataFrame.

summary = products[["product", "price"]]
print(summary)

Select Pandas Rows and Columns with loc

Use loc for label-based selection. The first argument identifies rows, and the second identifies columns.

selected = products.loc[1:2, ["product", "stock"]]
print(selected)

   product  stock
1    Mouse     30
2  Monitor      8

When labels are used in a loc slice, both the start and stop labels are included.

Select Pandas Rows and Columns with iloc

Use iloc for integer-position selection. Standard Python slicing rules apply, so the stop position is excluded.

selected = products.iloc[0:2, 0:2]
print(selected)

    product  price
0  Keyboard   45.5
1     Mouse   18.0

Filter Pandas DataFrame Rows with Conditions

A Boolean condition produces a Series of True and False values. Use that Boolean Series inside square brackets to keep only matching rows.

expensive_products = products[products["price"] > 50]
print(expensive_products)

   product  price  stock
2  Monitor  220.0      8
3   Webcam   62.0     15

Combine conditions with & for logical AND and | for logical OR. Place each comparison inside parentheses.

filtered = products[
    (products["price"] > 40) & (products["stock"] >= 10)
]

print(filtered)

Add, Update, Rename, and Remove Pandas Columns

You can create a new column from existing columns. Pandas performs arithmetic element by element.

products["inventory_value"] = products["price"] * products["stock"]
print(products)

Update a column by assigning a new Series or calculated result.

products["price"] = products["price"].round(2)

Rename one or more columns with rename().

products = products.rename(
    columns={"product": "product_name", "stock": "units_in_stock"}
)

Remove columns with drop(). Assign the result or use inplace=True when an in-place modification is appropriate.

products = products.drop(columns=["inventory_value"])

Handle Missing Values in a Pandas DataFrame

Missing values may appear as NaN, None, or missing datetime values, depending on the column dtype. Use isna() to locate them.

import pandas as pd

data = {
    "name": ["Asha", "Ben", "Chen", "Dia"],
    "score": [88, None, 91, None]
}

results = pd.DataFrame(data)
print(results.isna().sum())

name     0
score    2
dtype: int64

Use fillna() when a defensible replacement value is available. The following example fills missing scores with the mean of the existing scores.

mean_score = results["score"].mean()
results["score"] = results["score"].fillna(mean_score)

print(results)

Use dropna() to remove rows or columns containing missing values. Do this only when dropping the records is suitable for the analysis.

clean_rows = results.dropna()
clean_columns = results.dropna(axis="columns")

Sort Pandas Rows and Remove Duplicate Records

Use sort_values() to order rows by one or more columns.

sorted_products = products.sort_values(
    by=["price", "stock"],
    ascending=[False, True]
)

print(sorted_products)

Use duplicated() to identify duplicate rows and drop_duplicates() to remove them.

duplicate_mask = products.duplicated()
print(duplicate_mask)

unique_products = products.drop_duplicates()

Pass a column list to subset when duplicate detection should consider only selected fields.

unique_products = products.drop_duplicates(subset=["product"])

Calculate Pandas Summary Statistics

Pandas provides aggregation methods such as sum(), mean(), median(), min(), max(), and count().

print(products["price"].mean())
print(products["stock"].sum())
print(products[["price", "stock"]].max())

Use describe() for a compact statistical summary of numeric columns.

print(products.describe())

For categorical columns, value_counts() reports the frequency of each distinct value.

orders = pd.DataFrame({
    "status": ["Shipped", "Pending", "Shipped", "Cancelled", "Pending"]
})

print(orders["status"].value_counts())

status
Shipped      2
Pending      2
Cancelled    1
Name: count, dtype: int64

Group and Aggregate Data with Pandas groupby()

The groupby() method divides rows into groups based on one or more columns. An aggregation is then applied to each group.

import pandas as pd

sales = pd.DataFrame({
    "region": ["East", "West", "East", "West", "East"],
    "salesperson": ["Ana", "Ben", "Ana", "Cara", "Dev"],
    "amount": [1200, 950, 800, 1100, 700]
})

regional_totals = sales.groupby("region", as_index=False)["amount"].sum()
print(regional_totals)

  region  amount
0   East    2700
1   West    2050

Use agg() to calculate several summaries in one operation.

regional_summary = sales.groupby("region")["amount"].agg(
    total="sum",
    average="mean",
    orders="count"
)

print(regional_summary)

Combine Pandas DataFrames with concat() and merge()

Stack Pandas DataFrames with concat()

Use pd.concat() to combine objects along rows or columns. The following example stacks two DataFrames vertically.

import pandas as pd

january = pd.DataFrame({
    "product": ["Keyboard", "Mouse"],
    "units": [12, 18]
})

february = pd.DataFrame({
    "product": ["Monitor", "Webcam"],
    "units": [7, 11]
})

combined = pd.concat([january, february], ignore_index=True)
print(combined)

Join Related Pandas Tables with merge()

Use merge() to join DataFrames through a shared key, similar to a relational database join.

customers = pd.DataFrame({
    "customer_id": [1, 2, 3],
    "customer_name": ["Asha", "Ben", "Chen"]
})

orders = pd.DataFrame({
    "order_id": [101, 102, 103],
    "customer_id": [1, 3, 1],
    "amount": [75.00, 120.00, 49.50]
})

order_details = orders.merge(
    customers,
    on="customer_id",
    how="left"
)

print(order_details)

   order_id  customer_id  amount customer_name
0       101            1    75.0          Asha
1       102            3   120.0          Chen
2       103            1    49.5          Asha

Read and Write CSV Files with Pandas

Use pd.read_csv() to load comma-separated data into a DataFrame.

data = pd.read_csv("sales.csv")

Useful arguments include usecols for selecting columns, dtype for specifying data types, parse_dates for parsing date columns, and na_values for recognising additional missing-value markers.

data = pd.read_csv(
    "sales.csv",
    usecols=["date", "region", "amount"],
    parse_dates=["date"],
    dtype={"region": "string"}
)

Use to_csv() to save a DataFrame. Set index=False when the DataFrame index should not become an extra file column.

data.to_csv("cleaned_sales.csv", index=False)

Read and Write Excel Files with Pandas

Use pd.read_excel() to read a worksheet into a DataFrame. An Excel engine package may be required depending on the file format and environment.

data = pd.read_excel("sales.xlsx", sheet_name="January")

Use to_excel() to export a DataFrame.

data.to_excel("cleaned_sales.xlsx", index=False)

Use ExcelWriter when several DataFrames must be written to separate worksheets in one workbook.

with pd.ExcelWriter("report.xlsx") as writer:
    january.to_excel(writer, sheet_name="January", index=False)
    february.to_excel(writer, sheet_name="February", index=False)

Convert Pandas Columns to Numbers, Strings, and Dates

Imported columns are not always assigned the intended dtype. Use conversion functions before performing calculations or date operations.

data["amount"] = pd.to_numeric(data["amount"], errors="coerce")
data["order_date"] = pd.to_datetime(data["order_date"], errors="coerce")
data["region"] = data["region"].astype("string")

With errors="coerce", invalid numeric or datetime values become missing values. Inspect and handle those values before continuing the analysis.

Work with Text Columns Using Pandas str Methods

The str accessor applies string operations to a Series while preserving the index.

customers = pd.DataFrame({
    "name": ["  Asha Rao ", "BEN LEE", "Chen Wu"]
})

customers["clean_name"] = (
    customers["name"]
    .str.strip()
    .str.title()
)

customers["starts_with_a"] = customers["clean_name"].str.startswith("A")
print(customers)

Other useful string methods include str.contains(), str.replace(), str.split(), str.lower(), and str.upper().

Work with Dates Using the Pandas dt Accessor

After converting a column to a datetime dtype, use the dt accessor to obtain components such as year, month, day, weekday, or quarter.

orders = pd.DataFrame({
    "order_date": ["2026-01-05", "2026-02-12", "2026-02-28"]
})

orders["order_date"] = pd.to_datetime(orders["order_date"])
orders["month"] = orders["order_date"].dt.month
orders["weekday"] = orders["order_date"].dt.day_name()

print(orders)

Pandas Method Chaining for Readable Data Transformations

Method chaining expresses a sequence of transformations without repeatedly creating temporary variables. Place each method on a separate line to keep the steps readable.

summary = (
    sales
    .query("amount >= 800")
    .assign(tax=lambda frame: frame["amount"] * 0.08)
    .groupby("region", as_index=False)
    .agg(total_amount=("amount", "sum"), total_tax=("tax", "sum"))
    .sort_values("total_amount", ascending=False)
)

print(summary)

Method chaining is most useful when each operation has a clear purpose. For complex business rules, intermediate variables may be easier to debug.

Pandas and NumPy: When to Use Each Library

NumPy is centred on efficient multidimensional arrays and numerical operations. Pandas builds labelled Series and DataFrame structures that are convenient for tables, mixed column types, indexes, missing data, joins, and group-based analysis.

The libraries are complementary rather than interchangeable. Many Pandas operations use NumPy concepts internally, and Pandas columns can often be converted to NumPy arrays when lower-level numerical work is needed.

values = products["price"].to_numpy()
print(values)

Common Pandas Mistakes for Beginners

• Confusing labels with positions: use loc for labels and iloc for integer positions.
• Using and or or for Series conditions: use & and |, and wrap each comparison in parentheses.
• Ignoring column dtypes: convert text values to numeric or datetime types before calculating, sorting, or grouping them.
• Modifying a filtered slice without care: create an explicit copy when a filtered result will be changed.
• Using Python loops for routine column operations: prefer vectorised expressions, Series methods, or group operations when they directly express the required transformation.
• Saving the index unintentionally: pass index=False to file-writing methods when the index is not part of the dataset.

The following pattern creates a separate DataFrame before updating filtered rows:

available = products.loc[products["stock"] > 0].copy()
available["status"] = "Available"

Pandas Tutorial FAQs

Is Pandas difficult to learn for a Python beginner?

The basic Pandas workflow is manageable after learning core Python concepts such as variables, lists, dictionaries, functions, conditions, and imports. Beginners should first practise creating DataFrames, selecting rows and columns, filtering, handling missing values, and reading CSV files before moving to reshaping and multi-index operations.

Should I learn Python before learning Pandas?

Yes. You do not need advanced Python, but you should understand basic syntax, data types, collections, functions, and error messages. Pandas examples are easier to understand when normal Python expressions and indexing are already familiar.

What is the difference between a Pandas Series and DataFrame?

A Series is a one-dimensional labelled collection of values. A DataFrame is a two-dimensional table made of rows and named columns. Selecting one DataFrame column usually returns a Series, while selecting several columns returns another DataFrame.

Should I use Pandas or NumPy for data analysis?

Use Pandas when the data is organised into labelled rows and columns, contains mixed types, or requires joins, grouping, missing-value handling, and file input or output. Use NumPy when the work primarily involves homogeneous multidimensional arrays and numerical computations. Many projects use both.

How do I start practising Pandas with a real dataset?

Start with a small CSV file. Load it with pd.read_csv(), inspect it with head(), shape, info(), and isna().sum(), then practise selecting columns, filtering rows, converting dtypes, grouping values, and saving a cleaned result with to_csv().

Editorial QA Checklist for This Pandas Tutorial

• Run every Pandas example with Python 3 and confirm that each output matches the current code and sample data.
• Verify that loc examples use labels and iloc examples use integer positions.
• Check that Boolean filters use parentheses with & or | instead of Python’s scalar and or or.
• Confirm that missing-value examples explain whether values are filled, removed, or converted with errors="coerce".
• Test CSV and Excel snippets with representative files and verify whether optional spreadsheet engine dependencies are needed in the publishing environment.
• Review DataFrame outputs after any Pandas version update that changes display formatting, dtype labels, or aggregation output names.

Next Steps for Learning Pandas DataFrames

After learning the operations in this Pandas tutorial, practise with a dataset that includes numeric, text, datetime, and missing values. A useful workflow is to load the data, inspect its structure, clean its columns, filter invalid rows, calculate grouped summaries, combine related tables, and export the final DataFrame.

Continue with the DataFrame tutorials linked above for focused examples on creating DataFrames, accessing cells, working with column names, checking dimensions, and managing the index.