Python and CSV files - Study Notes

Chapter Summary

This chapter explores the integration of Python with Comma-Separated Values (CSV) files. It contrasts the raw text-based CSV format with binary Excel (XLS/XLSX) sheets, highlighting how CSV processes data faster and consumes less memory. It covers how to read and write CSV files using Python's native csv module, utilizing classes like reader(), writer(), DictReader(), and DictWriter(). The text also explains formatting principles such as handling fields containing commas, quotes, and newlines by using custom dialects, delimiters, and quote characters.

Learning Objectives

• Differentiate between CSV and XLS file formats in terms of structure, storage, and performance.
• Understand the formatting rules governing a standard CSV flat file.
• Read CSV data row-by-row and map specific columns into Python lists.
• Write new tabular records and append rows to existing CSV files.
• Define and register custom dialects to customize delimiters, quoting, and line termination.
• Read and write CSV databases using Python dictionaries for key-value pair mapping.

Key Concepts and Definitions

• Flat File: A plain text file containing a single table of structured data without relational links or formatting metadata.
• Delimiter: A punctuation character (typically a comma) used to split fields in a text record.
• CRLF: Carriage Return and Line Feed (\r\n), which is the standard character sequence used to denote a line break in files.
• Dialect: A helper class in Python's csv module that groups together specific formatting parameters like delimiters, line terminators, and escape characters.
• skipinitialspace: A dialect parameter that ignores spaces immediately following a delimiter when set to true.
• DictReader: An object that maps the read CSV rows to structured dictionaries using the column headers as keys.

Worked Methods

Reading CSV Files

To read a comma-delimited file, open it in read text mode ('r') and pass the file object to csv.reader(). This splits each line into a list of strings representing the columns.

Registering a Dialect

When files use non-standard delimiters (like pipes |) or contain initial spaces, use csv.register_dialect('myDialect', delimiter='|', skipinitialspace=True) and apply it as a parameter in readers or writers.

Writing to a CSV

Open a file with write mode ('w') and newline='' to prevent double blank lines. Use csv.writer() combined with writerow() for single rows or writerows() for lists of lists.

Common Exam Traps

• Universal Newlines in Writers: Forgetting to set newline='' inside the file open() function when writing can result in unexpected blank lines between rows on Windows platforms.
• Omitting the Commas of Empty Fields: Removing separator commas when a cell is empty. All commas must remain in place to guarantee that the columns correspond accurately from row to row.
• Missing Quote Escapes: Failing to double internal quotation marks. If field values contain quotes, the entire field must be surrounded by quotes, and the internal quotes must be doubled (e.g., ""Cricket"").
• Append vs. Write Modes: Using the write mode 'w' instead of append mode 'a' when trying to add new lines to the end of an existing dataset. This accidentally deletes all pre-existing content.
• Incorrect Index Slicing: Slicing lists without remembering that Python's range limits are exclusive of the end index (running up to n-1).

Exam Tips

• Always import the csv module at the very top of your Python file before referencing any CSV classes.
• When reading a column for mathematical sorting, remember that Python parses all fields as text strings; you must explicitly cast them using int() or float() before comparison.
• Use the with open() as f: syntax. This handles clean close operations automatically even if an runtime exception is encountered.
• The next() function is invaluable for advancing the reader's cursor past the first line, allowing your loops to process data while skipping header strings.