Python Style Convention
- 1 Overview
- 2 Going against convention
- 3 Linters and formatters
- 3.1 Pylance
- 3.1.1 Disabling Pylance
- 3.2 Pylint
- 3.2.1 Disabling Pylint
- 3.3 Flake8
- 3.3.1 Disabling Flake8
- 3.4 Black
- 3.4.1 Disabling Black
- 3.1 Pylance
- 4 Formatting
- 4.1 Naming
- 4.2 Spacing
- 4.3 Miscellaneous
- 5 File structure
- 5.1 Docstring
- 5.2 Imports
- 5.2.1 Separation with previous section
- 5.2.2 Rules
- 5.3 Global constants
- 5.3.1 Separation with previous section
- 5.3.2 Rules
- 5.4 Top level calls
- 5.4.1 Separation with previous section
- 5.4.2 Rules
- 5.5 Functions
- 5.5.1 Separation with previous section
- 5.5.2 Rules
- 5.6 Classes
- 5.6.1 Separation with previous section
- 5.6.2 Rules
- 6 General
- 6.1 No dead code
- 6.2 Functions and methods
- 6.2.1 Parameters and return
- 6.2.2 Docstring
- 6.2.3 Code blocks
- 6.3 Classes
- 6.3.1 Docstring
- 6.3.2 Create key
- 6.3.3 Constants
- 6.3.4 Subclasses
- 6.3.5 Methods
- 6.4 Code blocks
- 6.4.1 Conditionals
- 6.4.2 Checks
- 6.4.3 Comments
- 6.4.4 File paths
- 6.4.5 Pythonic
- 6.5 No raised exceptions
- 6.6 Multiline
Overview
Code is read and used much more often than it is written. It is better to write good code slowly than bad code quickly.
The goal of style conventions is to increase code readability.
This style convention is applied all new Autonomy Python code (you do not need to worry about old code or libraries). If you’re uncertain, ask the Autonomy project managers and/or leads. In general, the style is PEP8 with a soft 100 character line limit.
Going against convention
System and installed modules do not follow this style convention, including their naming. It is acceptable to use module calls with their default names.
Otherwise, if you write code that goes against this convention:
Is going against the convention actually necessary? Why is it bad to follow the convention in this code?
If the answer to the above is yes, then write a code comment justifying the reason why
A PR comment is NOT an acceptable substitute
Linters and formatters
Autonomy uses linters and formatters which cover the majority of the style convention, but not all. You are responsible for resolving all linting and formatting issues before your code can be merged.
If there is a conflict between this style convention and the linters and formatters, follow the linters and formatters. Contact the Autonomy leads so that the conflict can be resolved.
More detail: Linters and formatters
Pylance
Pylance is the default Visual Studio Code Python checker by Microsoft. It is very useful for linting.
Disabling Pylance
You can disable Pylance errors by adding this to the end of the line: # type: ignore
. You do not need to add a comment explaining the disable.
The only time Pylance should be disabled is in tests and for library calls:
Fixtures:
yield
returns a generator containing the type, not the type itselfGenerators are a form of lazy evaluation (i.e. the actual object is created at the last possible moment)
Accessing a private member or method
Pylance is not recognizing a library access
Pylint
Pylint is a linter installed by pip
as part of Autonomy’s repositories. It can also be installed as a VSCode extension: https://marketplace.visualstudio.com/items?itemName=ms-python.pylint
Pylint covers:
Naming convention
Unused variables and imports
Docstring existence enforcement
Pythonic style enforcement (e.g. loop iteration)
Exception type handling enforcement
Function call correctness
Disabling Pylint
If a lint issue cannot be resolved by restructuring code, Pylint can be disabled. All Pylint disables MUST be accompanied by a comment explaining why this is necessary. The disabled section must be as short as possible.
Pylint can be disabled for a single line or for an entire block.
Flake8
Flake8 is another linter used by Autonomy.
While Flake8 is a full linter similar to Pylint, Autonomy only uses Flake8 to cover:
Usage of type annotations of function and method signatures
Flake8 enforces using
-> None
for functions that do not return anything
Flake8 does not cover type annotation of variable names.
Disabling Flake8
Disabling Flake8 is not supported. However, there are some workarounds involving default objects if they are unknown or it is not feasible to get the type (e.g. somewhere within a library). This MUST be accompanied by a comment explaining why this is necessary.
If the type is known but it is multiple types, instead use: |
Black
Black is a formatter that modifies the Python source code to follow the Black style guide.
Black covers:
Line spacing
Spaces within lines
Line length and multiline formatting
While Black modifies the code, the semantics (i.e. functionality) of the code does not change.
Disabling Black
Black can be disabled. Black is disabled for code that requires formatting to be retained (e.g. matrices). All Black disables MUST be accompanied by a comment explaining why this is necessary. The disabled section must be as short as possible.
Black can be disabled for a single line or for an entire block.
Formatting
Naming
Format:
variable_names
in snake_casefunction_names()
in snake_caseFunction and method names do not start with
test_
unless they are specifically part of a UNIT test
ClassNames
in PascalCase (UpperCamelCase)Class names do not start with
Test
unless they are specifically part of a UNIT test
CONSTANT_NAMES
in UPPER_SNAKE_CASEfile_names.extension
in snake_caseFile names do not start with
test_
unless they are specifically a unit or integration test file
Private functions and class members (methods and attributes) are prefixed with 2 underscores:
def __my_private_func():
__my_private_var = ...
Names are full length and not shortened. This is to increase readability:
Prefer
error_message
instead oferr_msg
Numbered variables with underscore surround the number with underscores, unless the letters before and/or after are specifically part of the number:
my_1_var
HEIGHT_3m
g2_star
Initialisms and acronyms:
PascalCase: 1st letter capitalized followed by lowercase:
NTFS
becomesNtfs
RAM
becomesRam
snake_case: All lowercase:
NTFS
becomesntfs
RAM
becomesram
All empty containers are type annotated so that it is clear what they eventually hold:
numbers: "list[float]" = []
names_to_values: "dict[str, int] = {}
Spacing
TODO Figure out which of these Black covers
Lines:
Indents are 4 spaces long
Empty lines do not have any space characters
No spaces at the end of a line
Spaces:
No extra spaces around parantheses:
Prefer
my_function(my_var)
instead ofmy_function (my_var)
ormy_function( my_var )
Operators:
1 space around mathematical (e.g.
+
,/
), boolean (e.g.^
), and assignment (e.g.=
,+=
) operatorsExcluding named arguments:
np.sum(arr, axis=1)
No spaces around the slicing operator:
my_list[0:2]
,my_numpy_array[:, 1:3]
1 space after comma:
my_function(var1, var2, var3)
1 space after
:
in type annotations:my_function(param: int) -> ...
No spaces around default parameters:
my_function(param: float=0.5)
Miscellaneous
Only use parantheses when necessary:
Prefer
while my_num < 3:
instead ofwhile (my_num < 3):
All strings are enclosed with double quotation marks:
print("Hello world!")
Prefer f-strings when variables are used:
text = f"Height: {height}, width: {width}"
Multiline strings are concatenated:
text = "Title\n" + "Header\n" + "Body\n"
The default encoding is UTF-8:
open(filename, encoding="utf-8")
File structure
File contents are arranged in the following sections:
Docstring
A docstring uses 3 consecutive marks, each on its own line. Documentation uses proper grammar, capitalizes the 1st letter (unless it is a name that specifically starts lowercase (e.g. variable name)), and ends with a period.
Use Canadian spelling.
Imports
Imports are used to bring in code from other libraries into the file.
Separation with previous section
1 empty line between system imports and file docstring.
Rules
Import order:
System modules: Anything that comes with Python by default
Installed modules: Anything that requires
pip install
Local modules: Anything written by/for WARG (including submodules)
Import groups are separated by 1 empty line, with module names in alphabetical order within the group along with the following rules:
Sort using the name immediately after the
import
orfrom
keywordDo not use the keyword to sort
Dot (
.
) is above a/AShorter is above longer (e.g.
hello
abovehello_world
)
No wildcards (asterisk/star) in from
imports. Explicitly state what needs to be imported. Prefer importing the module namespace instead of the object name directly.
Do not call private functions or members from an imported module.
Global constants
Global constants are used for various settings that are immutable when the code is running.
Global variables are NOT allowed.
Separation with previous section
2 empty lines between global constants and the previous section, except file docstring.
If there are no sections other than file docstring, 1 empty line between global constants and file docstring.
Rules
Global constant groups are separated by 1 empty line. Global constants within each group are ordered by earliest usage of the variable name in the code.
Global constants may involve operations and function calls, as long as it is single assignment.
Top level calls
Top level calls are rare. They are generally used to indicate certain things about the file itself. For example, skipping all unit tests in the file.
Top level calls include linter and formatter setting comments.
Separation with previous section
2 empty lines between top level calls and the previous section, except file docstring.
If there are no sections other than file docstring, 1 empty line between top level calls and file docstring.
Rules
Related top level calls are grouped. Every group has a comment explaining why the call(s) is necessary.
Top level call groups are separated by 1 empty line.
Functions
Functions are useful as helpers to call from other functions and methods.
Separation with previous section
2 empty lines between functions and the previous section, except file docstring.
If there are no sections other than file docstring, 1 empty line between functions and file docstring.
Rules
Functions are separated by 2 empty lines.
Functions and methods have additional general rules below.
Classes
Separation with previous section
2 empty lines between classes and the previous section, except file docstring.
If there are no sections other than file docstring, 1 empty line between top level functions and file docstring.
2 empty lines between classes.
Rules
Classes have additional general rules below.
There are no sections past the classes section.
General
No dead code
Dead code is code that is not used or is unreachable. Dead code is not allowed.
How does dead code occur?
Dead code occurs for a variety of reasons, but mainly the following:
Hoarding: Developers do not like writing code twice, so they keep it around as a reference and just in case it will be used again.
Laziness: When modifying code, developers want to change as little as possible. This is especially apparent in function and method signatures, which may no longer require some of the parameters.
How do I avoid dead code?
Delete old code, including unused parameters. Code is usually under version control (e.g. Git), so in the case that recovery or reference is required, it is still available. However, such cases are extremely rare. Old code generally has little to no value (e.g. inefficient, incompatible, deprecated by a library, does not conform to standards, hard to understand).
…you do use Git for your own code, right? If you don’t, you can use GitHub or the UW GitLab: https://git.uwaterloo.ca .
What about all the old WARG code for the previous competition?
We keep the previous year’s competition code as a starting point and reference for the current year’s competition. Any code before the previous year is removed.
Functions and methods
A method is a function that operates as part of a class object.
Functions are arranged in the following sections:
Parameters and return
All function parameters have their type annotated as well as the return. This is enforced with Flake8.
Type annotations of containers (e.g. list
, dict
, tuple
) are surrounded by marks.
Docstring
Separation with previous section:
0 empty lines between function docstring and function declaration
The function docstring contains a short description of what the function does, its parameters, and its return:
Parameters: Each parameter name is listed, followed by a short description of what the parameter represents, including units if any
Parameters may be grouped, separated by commas
Return:
Return:
followed by a short description of what the return represents
Code blocks
Code blocks have additional general rules below.
Separation with previous section:
0 empty lines between function code blocks and function docstring
Classes
Classes are arranged in the following sections:
Docstring
The class docstring is a short description of what the class is for and what it does. The class docstring follows the docstring rules.
Separation with previous section:
0 empty lines between class docstring and class declaration
Create key
The class create key is an optional class constant used to enforce a private class constructor.
Separation with previous section:
0 empty lines between class create key and class docstring
Constants
Class constants are private constants specific to the class.
Separation with previous section:
1 empty line between class constants and the previous section, except class docstring
If there are no sections other than class docstring, 0 empty lines between class constants and class docstring
1 empty line between groups of class constants.
Subclasses
Helper classes can be nested in the class. These are usually very simple. If a class constant requires the helper class, then it should be moved out as a separate class.
Separation with previous section:
1 empty line between subclasses and the previous section, except class docstring
If there are no sections other than class docstring, 0 empty lines between subclasses and class docstring
Methods
Class methods are functions that operate as part of the class object.
Separation with previous section:
1 empty line between class methods and the previous section, except class docstring
If there are no sections other than class docstring, 0 empty lines between class methods and class docstring
Methods always have self
as the 1st parameter, unless otherwise indicated.
Constructor:
The class constructor initializes the class into an object, including its attributes.
Method name is:
__init__()
All class attributes (i.e.
self.var_name
) are created in the constructor and nowhere elseIf a variable is not used until later, initialize it to a default value (e.g.
0
,[]
) orNone
Prefer default value instead of
None
, if possible
Private constructor:
The class constructor always succeeds (unless it throws an exception), so all checks must be done prior to calling the constructor.
The create() method checks the input:
Decorated with
@classmethod
Method name is:
create()
If there are multiple methods, suffix the method name appropriately
1st parameter is
cls
to allow access to class constantsReturns:
True, [ClassName]
on successFalse, None
on failure
The create() method is arranged in the following manner:
The constructor is modified to have class_private_create_key
as the 2nd parameter. This is a workaround to prevent the constructor from being called outside the class without accessing a private member.
Example:
Static methods:
Static methods are class methods that do not require the class object. They can be thought of as independent functions that happen to be in the class namespace.
Static methods are decorated with @staticmethod
and do not have self
or cls
as the 1st parameter.
Code blocks
Code blocks are separated by 1 empty line.
TODO More detail?
Conditionals
Conditionals have the value to be compared on the left and the value to be compared against on the right.
Checks
Prefer checking for failure and exit early, instead of putting large amounts of code within conditional blocks.
Assertions to quiet linter complaints about types are only allowed if they are impossible to fail.
Comments
Comments are used to summarize and/or provide reasons for a specific implementation.
Especially if you are going against convention or doing something unusual, add a comment explaining why
File paths
Use pathlib
for file paths: https://docs.python.org/3/library/pathlib.html
This library handles file paths in an OS agnostic manner. It also has some other niceties (e.g. checking path/file existence, creating paths).
Pythonic
Follow a Pythonic style of coding if possible.
No raised exceptions
Exceptions are expensive to detect and catch, so they are not allowed.
Exceptions (ha):
Unit and integration test assertions
Assertions after checks that are impossible to fail
Private constructors' create key assertions
Instead, use the result, value pattern, which returns:
True, [value]
on successFalse, None
on failure (including checks)
Multiline
Multiline is automatically formatted by Black.