Scripting with Python and SQL for Data Engineering

Ends in 3 days! Save 40% on your access to 10,000+ programs and make a real impact in your career. Save now.

Scripting with Python and SQL for Data Engineering

This course is part of Python, Bash and SQL Essentials for Data Engineering Specialization

Instructors: Alfredo Deza

16,232 already enrolled

Included with Learn more

Ask Coursera

4 modules

Gain insight into a topic and learn the fundamentals.

Intermediate level

Recommended experience

2 weeks to complete

at 10 hours a week

Flexible schedule

Learn at your own pace

4 modules

Gain insight into a topic and learn the fundamentals.

Intermediate level

Recommended experience

2 weeks to complete

at 10 hours a week

Flexible schedule

Learn at your own pace

What you'll learn

Extract data from different sources and map it to Python data structures.
Design Scripts to connect and query a SQL database from within Python.
Apply scraping techniques to read and extract data from a website.

Skills you'll gain

Tools you'll learn

Details to know

Shareable certificate

Add to your LinkedIn profile

Assessments

5 assignments

Taught in English

See how employees at top companies are mastering in-demand skills

Learn more about Coursera for Business

logos of Petrobras, TATA, Danone, Capgemini, P&G and L'Oreal

Build your subject-matter expertise

This course is part of the Python, Bash and SQL Essentials for Data Engineering Specialization

When you enroll in this course, you'll also be enrolled in this Specialization.

Learn new concepts from industry experts
Gain a foundational understanding of a subject or tool
Develop job-relevant skills with hands-on projects
Earn a shareable career certificate

There are 4 modules in this course

In this third course of the Python, Bash and SQL Essentials for Data Engineering Specialization, you will explore techniques to work effectively with Python and SQL. We will go through useful data structures in Python scripting and connect to databases like MySQL. Additionally, you will learn how to use a modern text editor to connect and run SQL queries against a real database, performing operations to load and extract data. Finally, you will use extracted data from websites using scraping techniques. These skills will allow you to work effectively when data is not readily available, or when spatial queries are required to extract useful information from databases.

In this module, you will learn how to effectively use Python’s data structures to load, persist, and iterate over data. You will apply these data structures to solve different problems when working with popular data formats like JSON.

What's included

19 videos9 readings2 assignments1 discussion prompt2 ungraded labs

19 videosTotal 62 minutes

Welcome to Scripting with Python and SQL for Data Engineering1 minute
Meet your Course Instructor: Alfredo Deza0 minutes
Overview of Key Concepts4 minutes
Introduction to Working with Data in Python0 minutes
Introduction to Python Data Structures1 minute
Using Lists to Save and Retrieve Data in Python5 minutes
Using Dictionaries to Save and Retrieve Data in Python7 minutes
Overview of Less Common Data Structures in Python4 minutes
Recap of Data Structures in Python1 minute
Introduction to Choosing Data Structures in Python1 minute
Iterating Over Lists and Dictionaries in Python8 minutes
Iterating Over Other Data Structures in Python2 minutes
Storing Data Between Data Structures in Python6 minutes
Recap of Mapping Data Structures in Python1 minute
Introduction to Data Sources and Formats in Python1 minute
Loading Data from Files and File Paths in Python6 minutes
Working with JSON in Python7 minutes
Saving Data from Python to Disk4 minutes
Recap of Persisting and Loading Data in Python1 minute

9 readingsTotal 90 minutes

Connect with your instructor10 minutes
Meet your Supporting Instructors: Kennedy Behrman and Noah Gift10 minutes
Course Structure and Discussion Etiquette10 minutes
Report a problem with the course10 minutes
Key Terms10 minutes
Python Lists and Dictionaries10 minutes
Key Terms10 minutes
Key Terms10 minutes
Build a notebook that reformats data into JSON10 minutes

2 assignmentsTotal 60 minutes

Using Data Structures in Python30 minutes
Working with Data in Python30 minutes

1 discussion promptTotal 10 minutes

Meet and Greet (optional)10 minutes

2 ungraded labsTotal 120 minutes

Looping Over Lists and Dictionaries in Python60 minutes
Loading and Saving JSON Files60 minutes

In this module, you will learn how to use Python scripting techniques to work with SQL. You will create a database, store data, and query that data with SQL from Python.

What's included

15 videos7 readings1 assignment3 ungraded labs

15 videosTotal 56 minutes

Introduction to Python Scripting and SQL1 minute
Introduction to Scripting in Python1 minute
Creating a Script as a Module in Python6 minutes
Traversing the File System with a Script in Python7 minutes
Recap of Python Scripting Basics1 minute
Introduction to Embedded Databases1 minute
What is SQLite?2 minutes
Creating and Connecting to a SQLite Database in Python6 minutes
Saving and Querying from a SQLite Database in Python10 minutes
Recap of SQLite and Python1 minute
Introduction to Querying with SQL in Python1 minute
Basic SQL Commands in Python7 minutes
Extracting Distinct Data using SQL in Python7 minutes
Searching with SQL in Python6 minutes
Recap of Querying with SQL in Python1 minute

7 readingsTotal 70 minutes

Key Terms10 minutes
Minimal Python, Chapter 2: Learn to Store Data10 minutes
Key Terms10 minutes
Example GitHub Repository 10 minutes
Key Terms10 minutes
Example GitHub Repository10 minutes
Create a Reporting Script for File Sizes10 minutes

1 assignmentTotal 30 minutes

Python Scripting and SQL30 minutes

3 ungraded labsTotal 180 minutes

Build a Script to Find Large Files60 minutes
Create a SQLite Database and Store Data60 minutes
Querying a SQLite Database60 minutes

In this module, you will learn how to efficiently extract data from the web. You will learn how to use a scraping library to read data from websites and identify and extract specific values from it.

What's included

15 videos7 readings1 assignment2 ungraded labs

15 videosTotal 65 minutes

Introduction to Web Scraping using Python1 minute
Introduction to Extracting Data from Unstructured HTML1 minute
Challenges with Web Data3 minutes
Parsing HTML with HTMLParser in Python5 minutes
Recap of Web Scraping Techniques in Python1 minute
Introduction to Scrapy and XPath in Python1 minute
Creating a Web Scraping Project with Scrapy in Python7 minutes
Parsing Data with XPath and Scrapy Shell12 minutes
Using Scrapy Spider for Web Scraping5 minutes
Recap of Scrapy and XPath in Python1 minute
Overview of Challenges with Web Scraping3 minutes
Scraping Locally9 minutes
Persisting Data in CSV and JSON Formats10 minutes
Persisting Data to a SQLite Database7 minutes
Recap of Persistence and Efficacy with Web Scraping1 minute

7 readingsTotal 70 minutes

Key Terms10 minutes
Parsing Techniques with HTMLParser10 minutes
Key Terms10 minutes
Codespaces Lab: Parse HTML with Scrapy and XPath10 minutes
Key Terms10 minutes
Efficient Scraping Techniques10 minutes
Build a Web Scraping Tool10 minutes

1 assignmentTotal 30 minutes

Web Scraping using Python30 minutes

2 ungraded labsTotal 120 minutes

Parse HTML with HTMLParser60 minutes
Parse HTML and Persist it to a SQLite Database60 minutes

In this module, you will learn how to use a modern development environment and editor to connect to a MySQL database server, execute queries and perform database operations.

What's included

18 videos8 readings1 assignment5 ungraded labs

18 videosTotal 70 minutes

Introduction to Working with MySQL1 minute
Introduction to VSCode with MySQL1 minute
Connecting to a MySQL Server8 minutes
Recap of VSCode with MySQL1 minute
Challenges of Running MySQL Queries2 minutes
Using VSCode to Execute MySQL Queries6 minutes
Recap of Running MySQL Queries1 minute
Challenges with Importing to Databases2 minutes
Importing CSV Data into MySQL7 minutes
Exporting Data from MySQL6 minutes
Recap of Importing and Exporting Data in MySQL1 minute
MySQL Hacking Overview3 minutes
MySQL from Terminal3 minutes
Archive and Drop Database5 minutes
Import external database Sakila7 minutes
Modify database Sakila5 minutes
Bash pipelines with MySQL5 minutes
MySQL to Python Standard Library Web Server4 minutes

8 readingsTotal 80 minutes

Key terms10 minutes
Key terms10 minutes
Working with VSCode and MySQL10 minutes
Key Terms10 minutes
Lesson Reflection10 minutes
Loading and Exporting Data using MySQL10 minutes
Next Steps10 minutes
Share your learning experience10 minutes

1 assignmentTotal 30 minutes

Working with MySQL30 minutes

5 ungraded labsTotal 300 minutes

Connect to a Running MySQL Server60 minutes
Running Queries Repository60 minutes
Export and Import Data in MySQL60 minutes
Linux Hacking with MySQL60 minutes
MySQL Sandbox60 minutes

Earn a career certificate

Add this credential to your LinkedIn profile, resume, or CV. Share it on social media and in your performance review.

Instructors

Alfredo Deza

Duke University

29 Courses188,442 learners

Offered by

Duke University

Explore more from Data Analysis

Status: Free Trial
Duke University
Python, Bash and SQL Essentials for Data Engineering
Specialization
Status: Free Trial
Duke University
Python and Pandas for Data Engineering
Course
Status: Free Trial
Duke University
Web Applications and Command-Line Tools for Data Engineering
Course
Status: Free Trial
Duke University
Linux and Bash for Data Engineering
Course

Why people choose Coursera for their career

Felipe M.

Learner since 2018

"To be able to take courses at my own pace and rhythm has been an amazing experience. I can learn whenever it fits my schedule and mood."

Jennifer J.

Learner since 2020

"I directly applied the concepts and skills I learned from my courses to an exciting new project at work."

Larry W.

Learner since 2021

"When I need courses on topics that my university doesn't offer, Coursera is one of the best places to go."

Chaitanya A.

"Learning isn't just about being better at your job: it's so much more than that. Coursera allows me to learn without limits."

Frequently asked questions

To access the course materials, assignments and to earn a Certificate, you will need to purchase the Certificate experience when you enroll in a course. You can try a Free Trial instead, or apply for Financial Aid. The course may offer 'Full Course, No Certificate' instead. This option lets you see all course materials, submit required assessments, and get a final grade. This also means that you will not be able to purchase a Certificate experience.

When you enroll in the course, you get access to all of the courses in the Specialization, and you earn a certificate when you complete the work. Your electronic Certificate will be added to your Accomplishments page - from there, you can print your Certificate or add it to your LinkedIn profile.

Yes. In select learning programs, you can apply for financial aid or a scholarship if you can’t afford the enrollment fee. If fin aid or scholarship is available for your learning program selection, you’ll find a link to apply on the description page.