Python for Data Science (CIS-2212) Home Page

This is the home page for Peter Chapin's Python for Data Science course notes for the Fall 2021 semester. Here you will find electronic versions of class handouts, homework assignments, lecture slides, and links to other references of interest. If you are a student taking Python for Data Science, consider bookmarking this page.

The course syllabus gives an overview of the course and its content, lists course resources, and describes the grading policy and related issues.
The homework submission area and grade book are on Canvas but all other course resources are here.
The Zoom meeting URL gives you access to the live lectures and the lab sessions.
(Optional) You can download and install Python for your platform from the main Python website. This site has links to a lot of educational material as well. Installing Python this way is optional because Anaconda (below) comes with its own Python interpreter.
We will be using Anaconda and Visual Studio Code to write our programs for this course. I have prepared some instructions about how to set up those tools.
I will distribute sample code via a Git repository on GitHub. You can use VS Code to clone that repository to your machine and then use the features of Git to manage your own branches, etc. I will discuss how to do this in class.
There are many Python books available in the O'Reilly eBook Collection via the Hartness Library. Start with Learning Python, Fifth Edition by Mark Lutz, particularly if you are entirely new to Python.
I've prepared some general information on submitting assignments.
My home page contains other resources of potential interest.

Lecture Topics

2021-08-24. Introduction to the course and available materials & tools.
2021-08-26. Detailed walk-through of using VSCode with Git and Python. Showed cloning a repository, pulling updates, creating branches, and making commits. Also showed how to run a Python program from inside VSCode.
2021-08-31. Started introducing Python basic concepts using a collection of sample programs. Briefly covered program structure, control flow, dynamic typing, lists, and tuples. Introduced classes.
2021-09-02. Continued reviewing the collection of basic sample programs. Discussed classes more, introduced text file I/O, using the Python documentation, and some important methods for the upcoming homework assignment.
2021-09-07. Discussed documenting Python code ("docstrings"), the type hints feature of Python 3.5 and later (and how to install the mypy type checker), and testing using PyTest. Also discussed further exceptions, lists, and tuples.
2021-09-09. Discussed Python dictionaries and some of Python's functional features: First class functions, higher order functions, and the map function.
2021-09-14. Demonstrated Jupyter notebooks, Jupyter Lab, and VSCode's Jupyter support.
2021-09-16. Discussed using the requests package to make queries against OpenStreetMap's RESTful API.
2021-09-21. Described Homework #2. Introduced Pandas.
2021-09-23. Discussed how to pull data from a DBMS into Pandas.
2021-09-28. Introduced NumPy.
2021-09-30. Demonstration of NumPy on the problem of solving simultaneous equations.
2021-10-12. Demonstration of the Python debugger in VSCode and review of Pandas.
2021-10-14. Discussed Homework #3.
2021-10-19. Introduced Hadoop.
2021-10-21. No class.
2021-10-26. More discussion about Hadoop.
2021-10-28. No class.
2021-11-02. Discussed Homework #4 and the various supporting materials related to it.
2021-11-04. Introduced Matplotlib.
2021-11-09. Introduced machine learning with scikit-learn.
2021-11-11. Introduced WSGI and using Python as a back-end language for web development.
021-11-16<. Demonstrated Homework #4. Discussed the Mira variable star data and how it will be provided for the upcoming Homework #5.
2021-11-18. Discussed Homework #5.
2021-11-30. Discussed how to use the Python lxml library to access data in an XML document (for example, using XPath).
2021-12-02. More discussion about using the Flask web application framework in Homework #5.
2021-12-07. Discussed the Fast Fourier Transform and how to calculate it using NumPy. This is also a part of Homework #5.
2021-12-09. Introduced the Julia programming language.

Slides

Introduction. These slides are a work in progress.
Dynamic Languages. Gives an overview of what it means to say a language is "dynamic" using Python examples.

Homework

Here is a checklist of things to do with VS Code to see if you are able to work with the tool, and with Git, in an effective way. This is not a graded activity. It is just intended to give your exploration of VSCode a little structure.
Homework #1 (Processing a CSV File). Due: 2021-09-10
Homework #2 (Using Nominatim). Due: 2021-10-01
Homework #3 (Using Pandas). Due: 2021-10-29
Homework #4 (Using Hadoop). Due: 2021-11-12
Homework #5 (Python XML/Web). Due: 2021-12-15

Resources/Articles

The Python Home Page contains documentation, tutorials, and a place to download the official ("stock") version of Python.
A nice description on how to include documentation in your Python programs.
A tutorial on PyTest.
How to change your Jupyter Notebook start-up folder. This is a Stack Overflow answer that seems to work.
The Nominatim documentation describes the API used by OpenStreetMap for searching.
The Pandas documentation contains a tutorial on using the Pandas library.
Tutorial material about NumPy.
To demonstrate NumPy, I used a program that solves a large system of simultaneous equations. The test system of 2000 equations and 2000 unknowns is here. A C version and a Fortran version of this program are also provided. Three different Python versions can be found in the samples repository for this course on GitHub.
PuTTY is a free SSH client program for Windows. It is relatively simple, yet provides a rich collection of features. I have a document that describes how to set up and use PuTTY to access Lemuria. If you are a Mac user, you might find these notes about SSH on the Mac useful.
I wrote an overview of Unix that provides some conceptual background and describes some basic commands. It is an old document, and rather too long, but you may still find it useful. There are many other descriptions of basic Linux commands online. There are also cheat sheets.
I wrote a brief description of Hadoop commands that you might find useful.
Raw weather Station data complements of the Northern Vermont University meteorology department. These scripts can be used to compute the average temperature by month using the Hadoop cluster on Lemuria.
A Twitter customer support database for use in the Hadoop assignment. This is already expanded on the Hadoop cluster's HDFS under /user/hadoop/Twitter-Customer-Support.csv. This data comes from Kaggle.
Machine learning with Python is often done with the scikit-learn package.
Two, popular Python web development frameworks include Django and Flask (there are many other frameworks as well). The links above point at tutorials for how to set up these frameworks inside Visual Studio Code.
Here is a nice tutorial on the XML parsing options available to Python programmers. We will use lxml (tutorial).
Information about the data fields in the AAVSO raw brightness data of Mira (Omicron Ceti). We will use the AOML version of this data in Homework #5.