🐍 Advanced Python Guide

Resources

Books

• Fluent Python: Advanced Python Programming

Youtube Channels

• ArjanCodes: software design
• mCoding: advanced python

Websites

• Python Changelog: Release highlights
• testdriven.io: APIs with docker
• Realpython.com: Python tutorials

Repositories

• Patterns
• Awesome Python
• My github template

Virtual environments

Python virtual environments allow you to isolate the packages and avoid conflicts between packages.

• venv + requirements.txt
  • In the standard library
  • python3 -m venv venv/
  • source venv/bin/activate
  • Add pip-tools if necessary
• Poetry
  • Can lock dependencies
  • Handle dev libraries
  • No need for setup.py if you want to make a library

IDE

PyCharm is a very powerful IDE for Python, it comes with:

• A default PEP-8 linter
• Code completion and type checking
• Easy documentation browser
• Virtual Environments manager
• Integrated terminal and version control utilities

VS Code is also a very good IDE, specially if you want to use the same editor for other languages.

Alternatives: jupyterlab and Google Colab (great for testing things if you don’t have a GPU locally).

CI

• black: code formatter
• isort: sort your imports
• flake8: style guide
• pyupgrade: upgrade python syntax
• mypy: static type checker
• bandit: find security issues
• safety: check dependencies

You can use Pre-commit hooks to enforce some of this.

Advanced Python

Typing

Enable type checking and autocompletion

def string_to_number(string: str) -> int:
    return int(string)

• Factory pattern using typing.Type
• Observer pattern using typing.Protocol

Concurrency

Use ThreadPoolExecutor and ProcessPoolExecutor from concurrent.futures

from concurrent.futures import ThreadPoolExecutor

with ThreadPoolExecutor(max_workers=32) as executor:
    executor.map(download_something, urls)

f-strings

Just put f before the quotes of your string and put your variables between brackets.

x, y, z = 1.0, 2.0, 3.0
# f-strings
print(f'The coordinates are: {x}, {y}, {z}')

.format() is still useful if you want to reuse templates.

Built-in Functions

The most common ones are:

• dict, int, float, len, list, print, range, set, str, sum, tuple

So let’s learn about some others:

• enumerate(): Allows looping over something and having an automatic counter.

my_list = ['a', 'b', 'c', 1, 2]

for idx, value in enumerate(my_list):
    print(f'Index: {idx}, Value: {value}')

• reversed(): Iterate in reverse order.
• min(), max(), sorted(): You can pass a key parameter

sorted(["Hello", "my", "name", "is", "Xiang"], key=len)
# ['is', 'my', 'name', 'Hello', 'Xiang']

• any() and all(): Check if any or all items match a condition.

numbers = [1, 2, 3, 4]
any(n%2 == 0 for n in numbers)  # True
all(n%2 == 0 for n in numbers)  # False

collections

A standard library module that provides some interesting datatypes.

• namedtuple(): quick way to create tuples with fields.

from collections import namedtuple

Point = namedtuple('Point', ['x', 'y'])
p = Point(11, y=22)

p[0] == p.x  # True

x, y = p  # (11, 22)

• Counter: dict subclass for counting hashable objects.

from collections import Counter

c = Counter(["a", "b", "b", "c", "d", "d", "d"])
# Counter({'d': 3, 'b': 2, 'a': 1, 'c': 1})

• defaultdict: dict subclass that returns a default value if a key is missing.

from collections import defaultdict

d = defaultdict(list)
for w in ["Paris", "Barcelona", "London", "Lisbon", "Bogotá"]:
    d[w[0]].append(w)  # does not raise any errors

argparse

Instead of using input() to ask for parameters for you script, consider using command-line options.

Comprehensions

Do not use filter nor map.

# List comprehensions
even_numbers = [i for i in range(10) if i % 2 == 0]  # filter
# [0, 2, 4, 6, 8]

squares = [i**2 for i in range(10)]  # map
# [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

# Generator comprehensions
squares = (i**2 for i in range(10))

# Set comprehensions
names = ['BoB', 'bob', 'BOB', 'Alice', 'Jordan', 'JORDan']
names_set = {name.lower() for name in names}
# {'jordan', 'alice', 'bob'}

# Dictionary comprehensions
length = {name: len(name) for name in names}
# {'BoB': 3, 'bob': 3, 'BOB': 3, 'Alice': 5, 'Jordan': 6, 'JORDan': 6}

Generators

TODO

Staticmethod vs Classmethod

TODO

Decorators

TODO

ContextVariables

TODO

Context managers

TODO

Dataclasses

TODO

Async / Await

TODO

Lambdas

TODO

args and kwargs

TODO