All roadmaps
MASTERY CHECKLIST · PYTHON

Python for Data Analytics

The complete Python journey for aspiring data analysts. Phase 1 (0–2 yrs) and Phase 2 (2–3+ yrs) topics, 17 priority-ranked topics, a 30-60-90 day plan, 5 portfolio projects, real-world scenarios, and interview prep.

Beginner → Senior ~200 hrs totalIndustry-focused · Interview-ready
Progress0 / 17
0
Done
17
Topics
8
Phase 1
9
Phase 2
2
Easy
12
Medium
3
Hard

Priority-Based Topic Checklist

Organized by Phase (1 and 2). Expand each for context; check off as you master them.

  • 01.Python Fundamentals — Syntax, Data Structures, Functions

    Most Critical
    EasyPhase 1 2–3 weeksOrder #1● Critical

  • 02.Pandas & NumPy — The Core Analytics Engine

    Most Critical
    EasyPhase 1 4–5 weeksOrder #2● Critical

  • 03.Data Cleaning — Real-World Dirty Data Workflows

    Must Know
    MediumPhase 1 3–4 weeksOrder #3● Critical

  • 04.Data Visualization — Matplotlib, Seaborn & Plotly

    Must Know
    MediumPhase 1 3–4 weeksOrder #4● High

  • 05.Daily Analyst Skills — Excel Automation, CSV, Reports

    Must Know
    MediumPhase 1 2–3 weeksOrder #5● Critical

  • 06.Git & Version Control — Non-Negotiable

    Must Know Google Critical
    MediumPhase 1 1–2 weeksOrder #6● Critical

  • 07.Project Structure, venv & Reproducible Environments

    Must Know
    MediumPhase 1 1 weekOrder #7● Critical

  • 08.Testing, Type Hints & Code Quality — Senior-Level Craft

    Senior Level
    MediumPhase 1 2 weeksOrder #8● High

  • 09.Automation — Reporting, Email, Scheduling, ETL Pipelines

    Must Know
    MediumPhase 2 3–4 weeksOrder #9● High

  • 10.SQL + Python Integration — Databases, Queries, Workflows

    Must Know
    MediumPhase 2 3–4 weeksOrder #10● Critical

  • 11.APIs & Web Scraping — External Data Ingestion

    Must Know
    MediumPhase 2 2–3 weeksOrder #11● High

  • 12.Performance — Large Datasets, Memory, Optimization

    Senior Level
    HardPhase 2 3–4 weeksOrder #12● High

  • 13.BI Integration — Power BI, Dashboards, Data Refresh

    Must Know
    MediumPhase 2 2–3 weeksOrder #13● High

  • 14.Advanced Analytics — Statistics, A/B Testing, Forecasting, ML Basics

    Senior Level
    HardPhase 2 4–6 weeksOrder #14● High

  • 15.BigQuery + Python — Google's Core Analytics Stack

    Senior Level Google Critical
    HardPhase 2 3–4 weeksOrder #15● High

  • 16.Jupyter Best Practices — Papermill, nbconvert & Production Notebooks

    Must Know
    MediumPhase 2 1–2 weeksOrder #16● High

  • 17.Automated EDA — ydata-profiling, Looker & Google Workspace APIs

    Google Critical
    MediumPhase 2 1–2 weeksOrder #17● High
Phase 1 · 0–2 Years

Must Know for Early Data Analysts

Core Python skills every data analyst must master in their first two years. Tested in interviews and used daily.

Python Fundamentals

  • ›Variables and data types: int, float, str, bool, None
  • ›Collections: lists, tuples, dictionaries, sets
  • ›Control flow: if/elif/else, for loops, while loops
  • ›Functions: def, args/kwargs, return, lambda
  • ›List comprehensions and dict comprehensions
  • ›File handling: open, read, write, with context manager
  • ›Exception handling: try/except/finally
  • ›Modules and imports (os, sys, datetime, re)
  • ›OOP basics: class, __init__, methods, inheritance

Pandas Core Operations

  • ›DataFrame and Series creation, indexing, slicing
  • ›loc vs iloc — label vs position based selection
  • ›Filtering: boolean masks, query(), isin()
  • ›Sorting: sort_values(), sort_index()
  • ›GroupBy: .groupby().agg(), transform(), apply()
  • ›Merge, join, concat — combining DataFrames
  • ›Pivot tables: pivot_table(), crosstab()
  • ›Date/time: pd.to_datetime(), .dt accessor
  • ›String operations: .str accessor
  • ›Missing values: isnull(), fillna(), dropna()

Data Cleaning Patterns

  • ›Duplicate removal: drop_duplicates(subset=, keep=)
  • ›Text cleaning: .str.strip(), .str.lower(), .str.replace()
  • ›Type casting: pd.to_numeric(errors='coerce')
  • ›Outlier detection: IQR method, Z-score, clip()
  • ›Standardization: regex-based normalization
  • ›Data validation: pandera, great_expectations
  • ›Encoding issues: encoding='utf-8-sig'
  • ›Production cleaning pipeline with copy()

Visualization Libraries

  • ›Matplotlib: foundation, full control, static reports
  • ›Seaborn: statistical charts, heatmaps, distribution plots
  • ›Plotly: interactive charts for notebooks and web dashboards
  • ›Plotly Express: high-level API, one-line charts
  • ›Chart selection: trend→line, comparison→bar, distribution→histogram
  • ›Best practices: labels, titles, units, consistent colors

Excel & File Automation

  • ›openpyxl — read/write Excel with formatting
  • ›xlsxwriter — create formatted Excel reports
  • ›pandas.ExcelWriter — multi-sheet Excel output
  • ›pandas.read_csv with all parameters mastered
  • ›glob, os — process multiple files
  • ›Batch CSV processing pattern

Git & Version Control

  • ›git init, clone, status, add, commit — daily workflow
  • ›git branch, checkout, merge — feature branching
  • ›git pull, push, fetch — remote repos (GitHub/GitLab)
  • ›git log --oneline, git diff — inspect history
  • ›git stash — save work in progress
  • ›.gitignore — never commit credentials or large files
  • ›Conventional commit messages: feat:, fix:, chore:, docs:
  • ›nbstripout — strip notebook outputs before commit
Phase 2 · 2–3+ Years

Advanced Python Skills for Senior Analysts

Automation, SQL integration, APIs, performance optimization, BI tools, and advanced analytics.

Automation & ETL

  • ›Email: smtplib, email.mime — send reports with attachments
  • ›Scheduling: schedule library, cron (Linux), Windows Task Scheduler
  • ›ETL: Extract → Transform → Load pipeline pattern
  • ›Logging: Python logging module — essential for production
  • ›Config files: .env, YAML, JSON — never hardcode
  • ›Error alerts: try/except + email notification on failure
  • ›Production pipeline pattern with logging

SQL + Python Integration

  • ›sqlalchemy — database engine, works with all databases
  • ›psycopg2 — PostgreSQL native driver
  • ›pyodbc — SQL Server connection
  • ›pandas.read_sql() — query to DataFrame directly
  • ›DataFrame.to_sql() — write results back to DB
  • ›Parameterized queries — SQL injection prevention
  • ›Connection pooling for repeated queries
  • ›Environment variables for credentials

APIs & Web Scraping

  • ›requests — HTTP requests, REST APIs
  • ›BeautifulSoup4 — HTML parsing
  • ›Scrapy — production web scraping
  • ›httpx — async HTTP (faster for many requests)
  • ›json — parse API responses
  • ›pandas.json_normalize() — flatten nested JSON
  • ›Pagination pattern for collecting all pages
  • ›Rate limiting and exponential backoff

Performance Optimization

  • ›Dtype optimization: int32 vs int64, category for strings
  • ›Chunked reading: pd.read_csv(chunksize=50000)
  • ›Vectorization: replace apply() with numpy/pandas operations
  • ›Parquet format: 10× faster than CSV, 5× smaller
  • ›Polars library: Rust-based DataFrame, 5–50× faster than Pandas
  • ›DuckDB: in-process SQL on DataFrames
  • ›numba @jit: JIT-compile Python loops to machine code

BI Integration & Dashboards

  • ›Python in Power Query: custom transforms using Python script
  • ›Power BI REST API: trigger refresh, manage datasets
  • ›Streamlit: build Python-native interactive dashboards in hours
  • ›Dash (Plotly): full production dashboard apps
  • ›Parquet/Delta files: Python writes → Power BI reads
  • ›Streamlit dashboard with KPI cards and interactive filters

Advanced Analytics

  • ›Statistical tests: scipy.stats — t-test, chi-square, ANOVA
  • ›A/B testing: power analysis, p-value, confidence intervals
  • ›Forecasting: Prophet, statsmodels ARIMA, exponential smoothing
  • ›Regression: scikit-learn LinearRegression, feature importance
  • ›Clustering: K-Means for customer segmentation
  • ›Correlation analysis: Pearson, Spearman, Kendall

BigQuery + Python

  • ›google-cloud-bigquery — official BQ Python client
  • ›pandas-gbq — lightweight BQ ↔ DataFrame bridge
  • ›google-cloud-storage — read/write GCS buckets
  • ›Cost-aware queries: use dry_run=True first
  • ›Partitioned tables: always filter on partition column
  • ›Clustering: cluster on high-cardinality columns
  • ›pd.read_gbq() — single-line BQ → DataFrame

Jupyter Production Workflows

  • ›papermill — execute notebooks with parameters from CLI or Python
  • ›nbconvert — convert notebooks to HTML, PDF, Markdown, slides
  • ›nbstripout — strip outputs before git commit
  • ›Notebook structure: title → imports → parameters → sections → output
  • ›Parameters cell: tag one cell as 'parameters' for Papermill
  • ›Automated notebook execution over multiple parameters

Automated EDA & Google APIs

  • ›ydata-profiling — full EDA report in 2 lines
  • ›Minimal mode for large datasets (>500K rows)
  • ›Google Sheets API (gspread) — Python to Sheets
  • ›Looker/Looker Studio — BQ-native dashboards
  • ›Google Workspace APIs — Drive, Sheets automation
  • ›Automated data exports to Google Sheets
Structured

30-60-90 Day Python Learning Plan

For a working analyst putting in 2–3 hrs/day. Phase 1 in Days 1–60. Phase 2 in Days 61–90 and beyond.

Days 1–30

Phase 1 Foundation

Python Fundamentals, Pandas, NumPy, Data Cleaning

Days 1–6
Python Fundamentals
Data types, loops, functions, list comprehensions, exception handling
Days 7–12
NumPy + Pandas I
DataFrames, loc/iloc, filtering, sorting, reading CSV/Excel
Days 13–18
Pandas II — Analysis
GroupBy, merge, pivot tables, date operations, apply/map
Days 19–24
Data Cleaning
Duplicates, text cleaning, outliers, validation, reusable pipelines
Days 25–30
Weekend Project
Sales analysis pipeline: CSV → clean → analyze → Excel report
Days 31–60

Phase 1 Complete

Visualization, Excel Automation, Daily Analyst Tasks

Days 31–36
Data Visualization
Matplotlib, Seaborn, Plotly, chart selection, business storytelling
Days 37–42
Excel Automation
openpyxl, ExcelWriter, multi-sheet reports, conditional formatting
Days 43–48
Daily Analyst Tasks
Scheduled reports, KPI tracking, ad-hoc analysis workflows
Days 49–54
Project 2
Automated EDA report generator class
Days 55–60
Phase 1 Review
Interview prep: 20 Pandas questions cold, code review, portfolio polish
Days 61–90

Phase 2 — Advanced

SQL, Automation, APIs, Performance, Advanced Analytics

Days 61–66
SQL + Python
SQLAlchemy, parameterized queries, read_sql, to_sql, DB patterns
Days 67–72
Automation + ETL
Logging, scheduling, email automation, full pipeline design
Days 73–78
Project 3
Automated weekly report system: DB → transform → Excel → email
Days 79–84
APIs + Performance
requests, pagination, dtype optimization, chunking, DuckDB
Days 85–90
Advanced Analytics + Dashboard
A/B testing, forecasting basics, Streamlit dashboard — Project 4
Interview Prep

4-Phase Interview Preparation Roadmap

Expect live coding exercises — practice for speed and accuracy.

1

Phase 1 — Fundamentals Cold

Week 1 · Daily 1.5 hrs
Must write from memory
  • ›List vs tuple vs dict vs set — when to use each
  • ›loc vs iloc — demonstrate both with examples
  • ›GroupBy + agg vs groupBy + transform — business examples
  • ›Clean a column with mixed formats to numeric
  • ›Write a function with proper type hints and docstring
Practice method
  • ›LeetCode Python — Easy/Medium problems timed
  • ›Write Pandas operations without documentation
  • ›Explain code out loud as you write it
  • ›Practice with real datasets (Kaggle, StrataScratch)
2

Pandas & Data Manipulation Sprint

Week 2 · Daily 2 hrs
Master these patterns
  • ›Filtering with boolean masks and query()
  • ›Merge, join, concat — know all 3 and when to use
  • ›Pivot tables and crosstab for business reporting
  • ›Date operations: to_datetime, .dt accessor, date arithmetic
  • ›Apply vs vectorized operations — performance implications
Interview tips
  • ›Explain SettingWithCopyWarning and how to avoid it
  • ›Mention memory optimization (dtype downcasting)
  • ›Discuss when to use apply() and when not to
  • ›Always mention .copy() when working with slices
3

Phase 2 — Automation & ETL

Week 3 · Daily 1.5 hrs
Production patterns
  • ›Design an ETL pipeline with logging and error handling
  • ›Parameterized SQL queries in Python (security)
  • ›API integration with pagination and rate limiting
  • ›Email automation with attachments
  • ›Scheduling with schedule library or cron
Standout elements
  • ›Show logging in every pipeline
  • ›Use .env files for credentials
  • ›Write tests for ETL functions
  • ›Document with READMEs and docstrings
  • ›Version control with clean git commits
4

Performance & Advanced Analytics

Week 4 · 4–6 hrs per task
Senior-level skills
  • ›Optimize a 50M row DataFrame (dtypes, chunking, DuckDB)
  • ›Build an A/B test analysis function
  • ›Create a forecasting model (Prophet or ARIMA)
  • ›Build a Streamlit dashboard with interactive filters
  • ›Integrate BigQuery with Python (cost-aware queries)
Portfolio talking points
  • ›Real project where Python saved significant time
  • ›Quantify impact: "4-hour manual task → 30-second script"
  • ›Production features: logging, error handling, config files
  • ›Testing and code quality practices
  • ›Git workflow with feature branches and PRs

Top Python Interview Questions

Common questions asked at data analyst interviews across tech, finance, and consulting.

Python Fundamentals

  1. What is the GIL in Python and how does it affect multi-threaded data processing?
  2. What are list comprehensions? Write one to extract emails from a list of strings.
  3. What is the difference between a shallow copy and a deep copy? When does it matter in Pandas?
  4. How do decorators work? Write a timing decorator for an analytics function.
  5. What is the difference between a list and a tuple? When would you use each?

Pandas Core

  1. What is SettingWithCopyWarning? How do you avoid it?
  2. What is the difference between merge() and join()? When would you use each?
  3. How does groupby + transform differ from groupby + agg? Give a business example.
  4. How would you find and handle outliers in a revenue column? Write the code.
  5. How do you calculate a 7-day rolling average in Pandas?
  6. What is pd.cut() and pd.qcut()? When would you use each for analytics?

Data Cleaning

  1. A column contains '£1,200', '1200.00', 'N/A'. Clean it to float in one pipeline.
  2. How would you standardize inconsistent date formats across 50,000 rows?
  3. What is the difference between fillna(method='ffill') and fillna(value=0)?

Automation & Production

  1. Design a Python script that reads SQL, creates a report, and emails it every Monday.
  2. How do you prevent hardcoded credentials in Python scripts?
  3. What logging best practices do you follow for production analytics scripts?
  4. How do you handle an API that returns paginated results?

Performance

  1. A Pandas apply() on 10M rows takes 45 minutes. How do you fix it?
  2. How do you process a 5GB CSV that doesn't fit in RAM?
  3. When would you use Polars instead of Pandas?

Analytics & Statistics

  1. What is p-value? In an A/B test, p=0.04 with 10 variants. Is the result significant?
  2. How do you calculate customer lifetime value in Python from a transaction dataset?
  3. What is the difference between correlation and causation? Give a real analytics example.
  4. Write a Python function to calculate month-over-month revenue growth for each product.
Portfolio

Hands-On Projects & Portfolio Ideas

Build these to prove real-world Python mastery. Each covers multiple topic areas.

P1Beginner 4–6hrs

Sales Analysis Pipeline — E-Commerce Dataset

Read CSV, clean data (duplicates, nulls, outliers), calculate KPIs (revenue, AOV, MoM growth), create 5 visualizations, output formatted Excel report.

P2Intermediate 6–8hrs

Automated EDA Report Generator

Build a DataProfiler class that accepts any DataFrame and outputs HTML EDA report: data profiling, correlation heatmap, outlier detection, and business commentary.

P3Advanced 10–12hrs

Automated Weekly Business Report System

ETL pipeline: connect to SQL, run queries, transform with Pandas, create Plotly charts, format Excel with openpyxl, email to stakeholders at 7am every Monday.

P4Advanced 12–15hrs

Customer Churn Analysis + A/B Test Framework

RFM segmentation, cohort retention heatmap, churn prediction with logistic regression, A/B test analysis, Streamlit dashboard with interactive filters.

P5Hard 15–20hrs

Supply Chain Demand Forecasting

Pull historical sales via API, enrich with external APIs, train Prophet/ARIMA model, build Streamlit dashboard with actuals vs forecast and confidence intervals.

Real-World

Business Scenarios You'll Actually Face

Common asks from actual analyst roles. Know how to solve each end-to-end.

📊Scenario 1 — 'Our weekly report takes 4 hours manually — automate it'

Current process
  • ›Download CSVs from 5 systems
  • ›Manual copy-paste into Excel
  • ›Create pivot tables
  • ›Format charts and KPIs
  • ›Email to 20 stakeholders
Python solution
  • ›Write a pandas script to combine sources
  • ›Power Query style cleaning pipeline
  • ›Automate pivot tables with groupby
  • ›Plotly charts with consistent branding
  • ›smtplib email with attachments
Business impact
  • ›4-hour manual → 30-second script
  • ›Zero copy-paste errors
  • ›Audit trail with logging
  • ›Stakeholder confidence in data quality
  • ›Free time for deeper analytics

🗄Scenario 2 — 'Query a database and build a dashboard for stakeholders'

Challenge
  • ›Stakeholders need daily KPI visibility
  • ›Data lives in SQL database
  • ›Dashboards must be interactive
  • ›No budget for Power BI licenses
Python solution
  • ›SQLAlchemy to connect and query
  • ›Pandas for transformation
  • ›Streamlit for interactive dashboard
  • ›Deploy on Streamlit Cloud for free
Output
  • ›Interactive dashboard with filters
  • ›KPI cards with metrics
  • ›Trend charts with date range
  • ›CSV download for stakeholders

🌐Scenario 3 — 'Pull data from an external API for competitive intelligence'

Requirements
  • ›Pull competitor pricing data daily
  • ›API has rate limits (100/min)
  • ›Need historical trend analysis
  • ›Alert when price changes >10%
Python solution
  • ›requests with exponential backoff
  • ›Pagination for all pages
  • ›Store in Parquet for efficiency
  • ›Schedule with cron job
Output
  • ›Daily competitor price snapshot
  • ›Trend analysis with Plotly
  • ›Automated price change alerts
  • ›Clean data ready for analysis

⚡Scenario 4 — 'Our 50M row Python script crashes with MemoryError'

Diagnosis
  • ›Dataset is 5GB CSV
  • ›Pandas loads entire file into RAM
  • ›Apply operations are slow
  • ›Script runs out of memory
Optimization
  • ›Use dtype downcasting (30-50% memory reduction)
  • ›Process in chunks with pd.read_csv(chunksize=)
  • ›Switch to Polars or DuckDB for 10× speedup
  • ›Save intermediate as Parquet
Results
  • ›Script runs on 4GB RAM
  • ›Processing time: 45min → 5min
  • ›No more memory errors
  • ›Pattern reusable for other datasets
Standards

Python Best Practices Used at Companies

Standards followed by professional analysts at product companies, consulting firms, and data-driven enterprises.

Code Organization

  • ›One function = one responsibility
  • ›Separate concerns: extract.py, transform.py, load.py
  • ›Config files (YAML/JSON) for all parameters
  • ›requirements.txt + virtual environment for every project
  • ›Follow PEP8: use flake8 or black formatter
  • ›Docstrings on every function: purpose, params, returns
  • ›Type hints everywhere

Security & Reliability

  • ›Never hardcode credentials — use python-dotenv + .env
  • ›Always use parameterized SQL — never string concatenation
  • ›Validate all inputs before processing
  • ›Log every pipeline step with timestamps and row counts
  • ›Test data quality assertions before downstream use

Performance Standards

  • ›Profile before optimizing — use %timeit and memory_profiler
  • ›Avoid apply() — use vectorized operations always
  • ›Set correct dtypes at read time — saves 50-70% memory
  • ›Use Parquet over CSV for intermediate files
  • ›Process in chunks for > 1M rows
  • ›Use DuckDB for complex SQL-like aggregations

Testing & Documentation

  • ›Write pytest tests for every transformation function
  • ›Test with edge cases: empty DataFrame, all-null column
  • ›README in every project: setup, usage, data sources
  • ›Jupyter Notebooks: use markdown to explain decisions
  • ›Version control: commit after every working feature

Anti-Patterns to Avoid

  • ›Chained assignment: df['a']['b'] = 1 — use .loc always
  • ›Iterating rows with iterrows() — use vectorized alternatives
  • ›apply() with a lambda that could be vectorized
  • ›Reading entire files when only specific columns needed
  • ›No error handling in production scripts
  • ›Magic numbers everywhere — define as named constants

Data Standards

  • ›Always validate row counts before/after merge
  • ›Log 'before' and 'after' counts for every cleaning step
  • ›Save intermediate checkpoints to Parquet
  • ›Never overwrite source files
  • ›Use UTC timestamps internally
  • ›Store raw, clean, and aggregated data at different stages
Level Up

Python Learning Resources

Free Learning Resources

Books & Reference

All roadmapsBrowse projects