π Advanced Python Guide
Resources
Books
- Fluent Python: Advanced Python Programming
Youtube Channels
- ArjanCodes: software design
- mCoding: advanced python
Websites
- Python Changelog: Release highlights
- testdriven.io: APIs with docker
- Realpython.com: Python tutorials
Repositories
Virtual environments
Python virtual environments allow you to isolate the packages and avoid conflicts between packages.
- venv + requirements.txt
- In the standard library
python3 -m venv venv/
source venv/bin/activate
- Add pip-tools if necessary
- Poetry
- Can lock dependencies
- Handle dev libraries
- No need for
setup.py
if you want to make a library
IDE
PyCharm is a very powerful IDE for Python, it comes with:
- A default PEP-8 linter
- Code completion and type checking
- Easy documentation browser
- Virtual Environments manager
- Integrated terminal and version control utilities
VS Code is also a very good IDE, specially if you want to use the same editor for other languages.
Alternatives: jupyterlab and Google Colab (great for testing things if you donβt have a GPU locally).
CI
- black: code formatter
- isort: sort your imports
- flake8: style guide
- pyupgrade: upgrade python syntax
- mypy: static type checker
- bandit: find security issues
- safety: check dependencies
You can use Pre-commit hooks to enforce some of this.
Advanced Python
Typing
Enable type checking and autocompletion
def string_to_number(string: str) -> int:
return int(string)
- Factory pattern using typing.Type
- Observer pattern using typing.Protocol
Concurrency
Use ThreadPoolExecutor
and ProcessPoolExecutor
from concurrent.futures
from concurrent.futures import ThreadPoolExecutor
with ThreadPoolExecutor(max_workers=32) as executor:
executor.map(download_something, urls)
f-strings
Just put f
before the quotes of your string and put your variables between brackets.
x, y, z = 1.0, 2.0, 3.0
# f-strings
print(f'The coordinates are: {x}, {y}, {z}')
.format()
is still useful if you want to reuse templates.
Built-in Functions
The most common ones are:
- dict, int, float, len, list, print, range, set, str, sum, tuple
So letβs learn about some others:
- enumerate(): Allows looping over something and having an automatic counter.
my_list = ['a', 'b', 'c', 1, 2]
for idx, value in enumerate(my_list):
print(f'Index: {idx}, Value: {value}')
- reversed(): Iterate in reverse order.
- min(), max(), sorted(): You can pass a key parameter
sorted(["Hello", "my", "name", "is", "Xiang"], key=len)
# ['is', 'my', 'name', 'Hello', 'Xiang']
- any() and all(): Check if any or all items match a condition.
numbers = [1, 2, 3, 4]
any(n%2 == 0 for n in numbers) # True
all(n%2 == 0 for n in numbers) # False
collections
A standard library module that provides some interesting datatypes.
- namedtuple(): quick way to create tuples with fields.
from collections import namedtuple
Point = namedtuple('Point', ['x', 'y'])
p = Point(11, y=22)
p[0] == p.x # True
x, y = p # (11, 22)
- Counter: dict subclass for counting hashable objects.
from collections import Counter
c = Counter(["a", "b", "b", "c", "d", "d", "d"])
# Counter({'d': 3, 'b': 2, 'a': 1, 'c': 1})
- defaultdict: dict subclass that returns a default value if a key is missing.
from collections import defaultdict
d = defaultdict(list)
for w in ["Paris", "Barcelona", "London", "Lisbon", "BogotΓ‘"]:
d[w[0]].append(w) # does not raise any errors
argparse
Instead of using input()
to ask for parameters for you script, consider using command-line options.
Comprehensions
Do not use filter
nor map
.
# List comprehensions
even_numbers = [i for i in range(10) if i % 2 == 0] # filter
# [0, 2, 4, 6, 8]
squares = [i**2 for i in range(10)] # map
# [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
# Generator comprehensions
squares = (i**2 for i in range(10))
# Set comprehensions
names = ['BoB', 'bob', 'BOB', 'Alice', 'Jordan', 'JORDan']
names_set = {name.lower() for name in names}
# {'jordan', 'alice', 'bob'}
# Dictionary comprehensions
length = {name: len(name) for name in names}
# {'BoB': 3, 'bob': 3, 'BOB': 3, 'Alice': 5, 'Jordan': 6, 'JORDan': 6}
Generators
TODO
Staticmethod vs Classmethod
TODO
Decorators
TODO
ContextVariables
TODO
Context managers
TODO
Dataclasses
TODO
Async / Await
TODO
Lambdas
TODO
args and kwargs
TODO
Comments