In Python, a token is the smallest part of a program that the computer can understand. From a single dot to thousands of lines of code, everything is a token. Keywords, identifiers, literals, operators, and punctuators, these tokens are the building blocks used by the Python interpreter to parse and execute the code.
Table of Contents
1. Literals
Literals are the fixed values or constant values used directly in a program without any computation. Literals are immutable, meaning they cannot be changed. Python allows several kinds of literals.
String Literals
A string literal in Python is a sequence of characters enclosed within quotes, including single quotes, double quotes, or triple quotes.
- Single line string: As the name suggests, single line means a one-liner string that can be created by enclosing text in single quotes (‘ ‘) or double quotes (” “).
- Multi-line string: As the name suggests, this string contains multiple lines of text enclosed in triple single
'''or double quotes""". - Backslash characters: Backslash (
\) is used to include special characters in strings, known as escape sequences.
| Escape Sequence | Description | Example |
\' | Single quote | 'It\'s a sunny day!' |
\" | Double quote | "He said, \"Hello!\"" |
\\ | Backslash itself | "Path: C:\\folder" |
\n | Newline | "Hello\nWorld" |
\t | Tab space | "Hello\tWorld" |
\r | Carriage return (rarely used) | "Hello\rWorld" |
\b | Backspace | "Hello\bWorld" |
\f | Form feed | "Hello\fWorld" |
'Single Line'
"Single Line"
"""Multi
Line """
'''Multi
Line'''
'Backslash \' Character'Numeric Literals
The numeric literals in Python are nothing but just mathematical numbers and belong to any of the following three different numerical types.
- Integer number: integers are whole numbers without a decimal point and can be positive, negative, or zero. Example: -1, 10, 0. There can be several integer numbers with different base like octal number, hexadecimal number, binary number, etc.
- Float number: floats are numbers with a decimal point or written in scientific notation like exponent and mantissa. Example: -12.03, 3.1456, 0.1234
- Complex number: Complex numbers have two parts real part (a number like 2 or 3.5) and imaginary part (a number with
jlike 4j or -2.5j). Format:a + bj(whereais the real part, andbis the imaginary part). - Special literal None: Python has one special literal which is ‘None’. The ‘None’ literals is used to indicate absence of value.
Boolean literals
Boolean literals are special constants in Python that represent true or false values. These are used in programming for decision-making. Python uses True and False to represent these values (make sure T and F of True and False are capital letters). You can learn about data types in detail here.
2. Keyword
Keywords are the reserved word that has a predefined meaning and purpose. Keywords are part of the syntax and cannot be used as identifiers (such as variable names, function names, or class names).
Example: False, None, True, and, as, assert, async, await, break, class, continue, def, del, elif, else, except, finally, for, from, global, if, import, in, is, lambda, nonlocal, not, or, pass, raise, return, try, while, yield.
3. Identifiers
As the name identifier says something that identifies something. Identifiers are the names to identify variables, functions, classes, or other objects in a program. Identifiers are fundamental building blocks of a program. Identifiers forming rules of Python are being specified below:
- Identifiers must begin with a letter (
a-z,A-Z) or an underscore (_). - After the first character, identifiers can contain letters, digits (
0-9), and underscores (_). - Python identifiers are case-sensitive, meaning
myVariableandmyvariableare different identifiers. - Identifiers cannot be the same as Python keywords or reserved words. e.g.,
if,else,def,True,class. - Identifiers cannot contain spaces or special characters (such as
@,$,%, etc.).
Valid Identifiers:
x = 10 # valid identifier 'x'
name = "Alice" # valid identifier 'name'
total_amount = 50 # valid identifier 'total_amount'
_myvar = 100 # valid identifier '_myvar'Invalid Identifiers:
2value = 10 # Invalid, cannot start with a number
my-variable = 20 # Invalid, contains a special character '-'
if = 30 # Invalid, 'if' is a keyword4. Operators
Operators are special symbols or keywords or keywords used to perform operations on values or variables. So an operator requires some operands to work upon.
- Unary Operators: Unary operators are those operators that require one operand to operate upon. Following are some unary operators.
- + Unary plus
- – Unary minus
- ~ Bitwise complement
- not Logical negation
- Binary operators: Binary operators are those operators that require two operands to operate upon. Following are some binary operators.
- Arithmetic operators
- + Addition
- – Subtraction
- * Multiplication
- / Division
- % Remainder/Modulus
- ** exponent (raise to power)
- // Floor division
- Bitwise operators
- & Bitwise AND
- ^ Bitwise exclusive OR (XOR)
- | Bitwise OR
- Shift operators
- << Shift left
- >> Shift right
- Identity operators
- is is the identity same ?
- is not is the identity not same ?
- Relational operators
- < Less than
- > Greater than
- <= Less than or equal to
- >= Greater than or equal to
- == Equal to
- != Not equal to
- Assignment operators
- = Assignment
- /= Assign quotient
- += Assign sum
- *= Assign product
- %= Assign remainder
- -= Assign difference
- **= Assign Exponent
- //= Assign Floor division
- Logical operators
- and Logical AND
- or Logical or
- Membership operators
- in whether variable in sequence
- not in whether variable not in sequence
- Arithmetic operators
5. Punctuators
A punctuator in Python (and programming in general) refers to special symbols used to separate and organize different parts of the code. They play a significant role in the syntax and help the interpreter understand how to process the code.
'(Single Quote): Used to define string literals, e.g.,'hello'."(Double Quote): Also used for string literals, e.g.,"hello". Both single and double quotes are interchangeable.#(Hash): Used for writing comments, e.g.,# This is a comment.\(Backslash): Acts as an escape character, e.g.,\nfor a newline. It also allows line continuation.()(Parentheses): Used to group expressions, define function arguments, or call functions, e.g.,print("Hello").[](Square Brackets): Used for lists, indexing, and slicing, e.g.,[1, 2, 3].{}(Curly Braces): Used for dictionaries and sets, e.g.,{1: "one", 2: "two"}or{1, 2, 3}.@(At Symbol): Used for decorators in Python, e.g.,@staticmethod.,(Comma): Separates items in a list, tuple, or function arguments, e.g.,a, b = 1, 2.:(Colon): Used in function definitions, loops, conditional statements, and dictionaries, e.g.,def func():,if a > b:..(Dot): Accesses object attributes or methods, e.g.,object.method().`(Backtick): Rarely used in Python 3; was used in Python 2 for repr() in expressions.=(Equals): Used for assignment, e.g.,x = 10.
Final words
Tokens are the building blocks of any Python program, and understanding them is crucial for writing clean and efficient code. From literals to keywords, identifiers, operators, and punctuators, each token has a specific role in how Python interprets and executes your code. As a developer, getting comfortable with these basics not only helps you write better programs but also lays the groundwork for mastering more advanced concepts in Python. Remember, strong fundamentals lead to smoother coding and fewer errors down the road!
