Skip to content
Search

Python

lastmod 2026-09-11created 2014-02-08671 edits

Python Module Index

Python is a high-level programming language released in 1991 by programmer Guido van Rossum. It is a platform-independent, interpreted, object-oriented, dynamically typed, interactive language. The name Python comes from Guido's favorite comedy show, Monty Python's Flying Circus.

Python has an open, community-based development model managed by the non-profit Python Software Foundation. The CPython implementation is the de facto standard.

Category

Native

Python Enhancement Proposals (PEP) Index

  • Python Enhancement Proposals (PEP)
  • PEP 8 - Style Guide for Python Code
  • PEP 405 - Python Virtual Environments
  • PEP 425 - Compatibility Tags for Built Distributions
  • PEP 427 (wheel) - The Wheel Binary Package Format 1.0
  • PEP 432 - Restructuring the CPython startup sequence (restructuring the CPython startup sequence)
  • PEP 440 - Version Identification and Dependency Specification (Local Version Label - the part after + in a whl file name -)
  • PEP 465 - A dedicated infix operator for matrix multiplication (the matrix multiplication operator @)
  • PEP 484 - Type Hints, also covers "Stub Files", which correspond to pyi.
  • PEP 498 - Literal String Interpolation (this is f"...")
  • PEP 525 - Asynchronous Generators (__aiter__, __anext__, async for, typing.AsyncGenerator)
  • PEP 544 - Protocols: Structural subtyping (static duck typing)
  • PEP 578 - Python Runtime Audit Hooks
  • PEP 685 - Comparison of extra names for optional distribution dependencies
  • PEP 692 - Using TypedDict for more precise **kwargs typing
  • PEP 703 - Making the Global Interpreter Lock Optional in CPython (nogil)
  • PEP 750 - Template Strings
  • PEP 3333 - Python Web Server Gateway Interface v1.0.1

How to install

Install required dependencies

RHEL/CentOS Linux: 

sudo yum-builddep python python-libs
sudo yum install libffi-devel sqlite-devel zlib zlib-devel

Ubuntu/Debian Linux: 

sudo apt-get build-dep python
sudo apt-get install libffi-dev libgdbm-dev libsqlite3-dev libssl-dev zlib1g-dev

Updated. See #Updating Debian-family dependencies below.

SUSE Linux 

sudo zypper install automake fdupes gcc gcc-c++ gcc-fortran gdbm-devel gettext-tools gmp-devel intltool libbz2-devel libexpat-devel libffi-devel libnsl-devel lzma-devel make ncurses-devel netcfg openssl-devel pkgconfig readline-devel sqlite-devel xz zlib-devel

Python 2.7 on CentOS 6

CentOS uses a Linux package management tool/program called yum, which is closely tied to python 2.6, so carelessly upgrading to 2.7 leaves yum unusable.

So install it under your personal account as below. 

yum groupinstall "Development tools"
yum install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel

## Download Python 2.7 source code & extract.
cd ~/Downloads
wget --no-check-certificate https://www.python.org/ftp/python/2.7.6/Python-2.7.6.tar.xz
tar xf Python-2.7.6.tar.xz
cd Python-2.7.6
LDFLAGS=-fPIC && ./configure --enable-shared --prefix=/home/username/local
make && make altinstall

Updating Debian-family dependencies 

sudo apt-get update; sudo apt-get install make build-essential libssl-dev zlib1g-dev \
libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm \
libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev

sudo apt-get install build-essential gdb lcov pkg-config \
      libbz2-dev libffi-dev libgdbm-dev libgdbm-compat-dev liblzma-dev \
      libncurses5-dev libreadline6-dev libsqlite3-dev libssl-dev \
      lzma lzma-dev tk-dev uuid-dev zlib1g-dev

Adding a non-standard SSL path

When building, add the --with-openssl={openssl-install-root-dir} option at the ./configure step.

Tip

  • The important point is to use make altinstall, not make install. 1
  • Note that the prefix option of configure must be an absolute path. Otherwise you will see the following error.
    configure: error: expected an absolute directory name for --prefix: ~/local

How to check the compile options of an existing Python

Can be checked with Python:sysconfig. Simply: 

python3 -m sysconfig | less

Python compile

To reduce program startup (load) time, you can use byte-compiled pyc files. To compile, you can compile a specific file from the command-line interpreter as below, 

import py_compile
py_compile.compile('spam.py')

or compile all *.py files in the current directory. 

$ python -m compileall .

How to reinstall on Ubuntu 

sudo apt install --reinstall python3-minimal python3-distutils

Python files

Describes Python-related files.

  • *.py is a Python text source file.
  • *.pyc is a compiled Python binary file. It can be generated with the -O option.
  • *.pyo is an optimized compiled Python binary file. It can be generated with the -OO option.
  • *.pyd is basically a Windows DLL file.2

Bootstrap script 

#!/usr/bin/env bash

ROOT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" || exit; pwd)

OPY_PATH=$(which opy 2> /dev/null)
PYTHON3_PATH=$(which python3 2> /dev/null)
PYTHON_PATH=$(which python 2> /dev/null)

function run_base_python
{
    if [[ -x "$OPY_PATH" ]]; then
        "$OPY_PATH" "$@"
    elif [[ -x "$PYTHON3_PATH" ]]; then
        "$PYTHON3_PATH" "$@"
    elif [[ -x "$PYTHON_PATH" ]]; then
        "$PYTHON_PATH" "$@"
    else
        echo "The python executable could not be found." 1>&2
        exit 1
    fi
}

VENV_HOME=$ROOT_DIR/.venv
VENV_PYTHON=$VENV_HOME/bin/python3

if [[ ! -d "$VENV_HOME" ]]; then
    run_base_python -m venv "$VENV_HOME"
    if [[ -x "$VENV_PYTHON" ]]; then
        "$VENV_PYTHON" -m pip install -r "$ROOT_DIR/requirements.txt"
    fi
fi
if [[ ! -x "$VENV_PYTHON" ]]; then
    echo "The venv's python executable could not be found." 1>&2
    exit 1
fi

export PYTHONPATH
PYTHONPATH=$ROOT_DIR:$PYTHONPATH

"$VENV_PYTHON" "$@"

Python Compile

Dunder

Dunder (Double Underscore). A collective term for symbols used for special purposes in Python, such as __name__.

subclasshook

Must be defined as a class method.

Check whether the subclass is considered a subclass of this ABC. This means that you can customize the behavior of issubclass() further without the need to call Register() on every class you want to consider a subclass of the ABC. (This class method is called from the subclasscheck() method of the ABC.)

This method should return True, False or NotImplemented. If it returns True, the subclass is considered a subclass of this ABC. If it returns False, the subclass is not considered a subclass of this ABC, even if it would normally be one. If it returns NotImplemented, the subclass check is continued with the usual mechanism.

os.PathLike example: 

class PathLike(abc.ABC):

    """Abstract base class for implementing the file system path protocol."""

    @abc.abstractmethod
    def __fspath__(self):
        """Return the file system path representation of the object."""
        raise NotImplementedError

    @classmethod
    def __subclasshook__(cls, subclass):
        if cls is PathLike:
            return _check_methods(subclass, '__fspath__')
        return NotImplemented

    __class_getitem__ = classmethod(GenericAlias)

cause

You can get the exception specified with from, as in raise ... from e. 

try:
    try:
        raise ValueError("원래 예외")
    except ValueError as e:
        raise RuntimeError("새로운 예외") from e
except RuntimeError as new_exception:
    original_exception = new_exception.__cause__
    print(f"새 예외: {new_exception}")
    print(f"원래 예외: {original_exception}")

class_getitem

See Python:typing#Generics.

init_subclass

See Python:typing#Generics.

Structural Pattern Matching (match ~ case statement)

See Python:Match.

Getter and Setter

See Python:@property#Getter and Setter.

Make Python stack overflow and core dump 

import sys

def recursive_sum(x):
    return x + recursive_sum(x - 1) if x > 0 else 0

print(f'Default sys.getrecursionlimit()={sys.getrecursionlimit()}')
print(f'recursive_sum(100)={recursive_sum(100)}')
print(f'recursive_sum(1000)={recursive_sum(1000)}')

String interpolation

See String interpolation for details. 

name = "Mark"
print(f'Hello, {name}!')

Techniques for developing dynamic plugins based on Python modules (PIP packages)

Quick summary:

  • Discover packages using pkgutil.iter_modules.
  • Import packages using importlib.import_module.

Often when creating a Python application or library you'll want the ability to provide customizations or extra features via plugins. Because Python packages can be separately distributed, your application or library may want to automatically discover all of the plugins available.

There are three major approaches to doing automatic plugin discovery:

  • Using naming convention
  • Using namespace packages
  • Using package metadata

Using naming convention 

import importlib
import pkgutil

discovered_plugins = {
    name: importlib.import_module(name)
    for finder, name, ispkg
    in pkgutil.iter_modules()
    if name.startswith('flask_')
}

The most recommended method. (Easiest to debug)

Using namespace packages 

import importlib
import pkgutil

import myapp.plugins

def iter_namespace(ns_pkg):
    # Specifying the second argument (prefix) to iter_modules makes the
    # returned name an absolute name instead of a relative one. This allows
    # import_module to work without having to do additional modification to
    # the name.
    return pkgutil.iter_modules(ns_pkg.__path__, ns_pkg.__name__ + ".")

discovered_plugins = {
    name: importlib.import_module(name)
    for finder, name, ispkg
    in iter_namespace(myapp.plugins)
}

WARNING

Namespace packages are a complex feature and there are several different ways to create them. It's highly recommended to read the Packaging namespace packages documentation and clearly document which approach is preferred for plugins to your project.

WARNING

With this method, a conflict error about the source code Root directory setting may occur when debugging in PyCharm.

Using package metadata

Set the entry point in the setup.py of setuptools. 

setup(
    ...
    entry_points={'myapp.plugins': 'a = myapp_plugin_a'},
    ...
)

You can then discover and load all of the registered entry points: 

import sys
if sys.version_info < (3, 10):
    from importlib_metadata import entry_points
else:
    from importlib.metadata import entry_points

discovered_plugins = entry_points(group='myapp.plugins')

WARNING

This method requires the package to be installed.

Obfuscation

Ship it as a binary when distributing. Use tools such as Cython, Nuitka, Shed Skin to compile to C code.

Also see #Obfuscation libraries above.

Download package

You can get all the package files as below. 

$ pip3 list | awk '{print $1}' | sed -n '3,$p' > requirements
$ pip3 download -r requirements

Python underscore

See dunder.

Python gdb debugging

When a C-level Runtime Error occurs, it is hard to see a proper StackTrace. In this case, see Gdb#Python gdb debugging.

How to check Debugging Mode 

# -*- coding: utf-8 -*-

import sys


def is_debugging_mode() -> bool:
    """
    Return if the debugger is currently active
    """

    get_trace = getattr(sys, "gettrace", lambda: None)
    return get_trace() is not None

future.annotations.compiler_flag

How to get __future__.annotations.compiler_flag portably: 

# -*- coding: utf-8 -*-

from importlib import import_module


def get_annotations_compiler_flag() -> int:
    future = import_module("__future__")
    assert future is not None
    annotations = getattr(future, "annotations")
    assert annotations is not None
    compiler_flag = getattr(annotations, "compiler_flag")
    assert isinstance(compiler_flag, int)
    return compiler_flag

Caveats for avoiding Python's axis of evil

Do not use function calls in default arguments 

from datetime import datetime
from time import sleep


def get_update_task_query_by_uid(
    updated_at=datetime.now().astimezone(),
):
    print(updated_at)


get_update_task_query_by_uid()
sleep(1)

get_update_task_query_by_uid()
sleep(1)

get_update_task_query_by_uid()
sleep(1)

get_update_task_query_by_uid()
sleep(1)

In the code above, the updated_at argument is initialized with datetime.now().astimezone() once at the start, and it is never called again after that.

So it is better to use something like Optional[datetime] = None.

For reference, running the code above prints the following. 

2021-10-07 16:44:12.581059+09:00
2021-10-07 16:44:12.581059+09:00
2021-10-07 16:44:12.581059+09:00
2021-10-07 16:44:12.581059+09:00

Class member initialization is not per instance

When you initialize class members, do not mistake this for member initialization: 

class _Test0:
    value = list()

class _Test1(_Test0):
    def __init__(self):
        pass

a = _Test1()
a.value.append(0)
b = _Test1()
b.value.append(1)
b.value.append(2)
c = _Test1()
c.value = [3, 4]
d = _Test1()
d.value.append(5)
_Test1.value.append(100)

print(a) # [0, 1, 2, 5, 100]

The same goes for POD data types. 

class _Test0:
    value = 100

class _Test1(_Test0):
    def __init__(self):
        pass

a = _Test1()
b = _Test1()
print(a.value) # 100
print(b.value) # 100

_Test1.value = 200
print(a.value) # 200
print(b.value) # 200

Automatic initialization of Mixin classes

See Python:Mixin#How to automatically initialize Mixin classes.

Cheat Sheet

Python 3 Cheat Sheet
https://perso.limsi.fr/pointal/_media/python:cours:mementopython3-english.pdf
Memento-python3-english.pdf

Restricting arguments

Restricting to keyword-only arguments

Put '*' where the positional arguments end. 

def foo(a, *, b):
    print(a, b)

foo(1, 2)  # 에러
foo(1, b=2)  # 유효

Restricting to positional-only arguments

In Python, putting '/' at the end of the argument list in a function definition forces the function's parameters to be accepted only as positional arguments. This is new syntax added in Python 3.8. 

def foo(a, b, /):
    print(a, b)

foo(1, 2)  # 유효한 호출
foo(a=1, b=2)  # 에러: foo()는 위치 인자만 받아들입니다.

"is None" vs "

None" == In python, why do we compare None with is None and not with == None?

is operator
Checks whether two objects refer to the same memory location.
  • That is, it checks whether two objects are the same object (identical).
  • e.g. x is y is equivalent to id(x) == id(y).
== operator
Checks whether the values of two objects are equal.
  • It compares by calling the object's __eq__ method.
  • So it can be True even if two objects are in different memory locations, as long as their values are equal.
  • None is a single object
    • None is a single global singleton object in Python. That is, None is one object that always has the same memory address while the program runs.
    • So using is None is more intuitive and accurate.
  • Advantages of is None
    • Performance: is is faster because it simply compares object IDs.
    • Clear intent: is None conveys the clear meaning "check whether this object is None".
    • On the other hand, == None calls __eq__, so it can be affected by overloaded methods and may behave differently than intended.

Example

Main module

This means that when you run the file directly, as in C:/Python> python mod1.py, __name__ == "__main__" is true and the statements after the if statement are executed,
while when this module is imported and used from the interactive interpreter or another file, __name__ == "__main__" is false and the statements under the if statement are not executed. 

if __name__ == "__main__":
    # TODO: Insert to source code.

Current script name

The module where __main__ was invoked can be obtained as below. 

import __main__ as main
print(main.__file__)

## Absolute path:
import os
print os.path.abspath(main.__file__)

The current file (script) name can be obtained as below. 

os.path.realpath(__file__)

Encoding setting

Specify the file's encoding on the first line of the script file. 

# -*- coding: cp949 -*-

Method Overload 

    @overload
    def get(self, key: _KT) -> Optional[_VT_co]: ...
    @overload
    def get(self, key: _KT, default: Union[_VT_co, _T]) -> Union[_VT_co, _T]: ...

Split key-value tokens

Parses a string like the one below... 

"remove_unknoen=true; rtsp_transport=tcp; defined=key=val;"

...and returns a dictionary like the one below. 

{
    'remove_unknoen': 'true',
    'rtsp_transport': 'tcp',
    'defined': 'key=val'
}

The implemented function is as follows. 

def split_option(text: str, kv_split='='):
    i = text.find(kv_split)
    if i == -1:
        return None
    return text[0:i].strip(), text[i+1:].strip()

def str_to_dict(text: str, item_split=';', kv_split='='):
    items = [x for x in text.split(item_split)]
    items = list(filter(lambda x: x.strip(), items))
    result = {}
    for x in items:
        opt = split_option(x, kv_split)
        if opt:
            assert len(opt) == 2
            result[opt[0]] = opt[1]
    return result

The function that joins the dictionary above back into a string is as follows. 

from functools import reduce

def dict_to_str(items: dict, item_split=';', kv_split='='):
    return reduce(lambda x, y: x+item_split+y,
                  [str(k)+kv_split+str(v) for k, v in items.items()])

Appending to Your PYTHONPATH

To add to PYTHONPATH in code: 

import sys
sys.path.append("/home/me/mypy")

To check the list of applied PYTHONPATH entries: 

import sys
print(sys.path)

A usage example: 

import os.path as osp
import sys

def add_path(path):
    if path not in sys.path:
        sys.path.insert(0, path)

this_dir = osp.dirname(__file__)

# Add caffe to PYTHONPATH
caffe_path = osp.join(this_dir, '..', 'python')
add_path(caffe_path)

Python STDOUT Buffer

See the environment variable PYTHONUNBUFFERED.

Check platform 

import os
print os.name

## OR
import platform
print platform.platform()

Generics

For how to write generic classes, see Python:typing#Generics.

Checking File Permissions 

import os
os.access('my_file', os.R_OK) # Check for read access
os.access('my_file', os.W_OK) # Check for write access
os.access('my_file', os.X_OK) # Check for execution access
os.access('my_file', os.F_OK) # Check for existence of file

Python version which command 

def which(program):
    path = os.getenv('PATH')
    for cursor in path.split(os.path.pathsep):
        cursor = os.path.join(cursor, program)
        if os.path.exists(cursor) and os.access(cursor, os.X_OK):
            return cursor
    return ''

Python version variable 

import sys
version_info = sys.version_info
version_text = f"{version_info[0]}.{version_info[1]}.{version_info[2]}"
print(version_text)

Elapsed Time Calculator

For how to get the elapsed time (Duration), see Python:time#Elapsed Time Calculator.

Input 

val1 = input("Input : ")
val2 = raw_input("Input : ")

Note, however, that with input() you can use the names of Python variables that are already declared.

Most Bytes Copy method

The benchmark code is as follows: 

import copy
import timeit

original_bytes = b"some bytes data" * 1000

def using_bytes_constructor():
    return bytes(original_bytes)

def using_slicing():
    return original_bytes[:]

def using_bytearray():
    return bytearray(original_bytes)

def using_copy():
    return copy.copy(original_bytes)

def using_memoryview():
    return memoryview(original_bytes).tobytes()

# Benchmark each method
count = 10_000_000
print("Using slicing:", timeit.timeit(using_slicing, number=count))
print("Using bytearray:", timeit.timeit(using_bytearray, number=count))
print("Using copy:", timeit.timeit(using_copy, number=count))
print("Using memoryview:", timeit.timeit(using_memoryview, number=count))
print("Using bytes constructor:", timeit.timeit(using_bytes_constructor, number=count))

Results tested on an AMD Ryzen 7 4700U CPU 2.0GHz:

  • Using slicing: 0.7010392720003438
  • Using bytearray: 4.187463062000461
  • Using copy: 2.724746272999255
  • Using memoryview: 4.6049892060000275
  • Using bytes constructor: 1.6053336320001108

Result: Slicing is the fastest.

Converting between Bytes and HEX strings

bytes to HEX String 

a = bytes([10, 20, 30, 40, 50])
a.hex()

HEX String to bytes 

bytes.fromhex("deadbeef")

Serializing an Integer to Bytes 

data = 9999
serialized_data = data.to_bytes(4, "big", signed=False)
print(serialized_data.hex())  ## 0000270f
result = int.from_bytes(serialized_data, "big", signed=False)
print(result)  ## 9999

If you want to convert using Python:struct, see Python:struct#Serializing an Integer to Bytes.

Current working directory

Return a string representing the current working directory. 

os.getcwd()

List of files 

import os
import re

def get_child_directories(path=os.curdir):
    result = []
    for parent, sub_dirs, files in os.walk(path):
        result.append(parent)
    return result


def get_children(path=os.curdir):
    result = []
    for parent, sub_dirs, files in os.walk(path):
        for name in files:
            # result.append(os.path.join(os.path.abspath(parent), name))
            result.append(os.path.join(parent, name))
    return result


def get_children_with_match(path=os.curdir, regexp='.*'):
    result = []
    for cursor in get_children(path):
        if not re.match(regexp, cursor) is None:
            result.append(cursor)
    return result


def get_files(path=os.curdir):
    return list(filter(lambda x: os.path.isfile(os.path.join(path, x)), os.listdir(path)))


def get_dirs(path=os.curdir):
    return list(filter(lambda x: os.path.isdir(os.path.join(path, x)), os.listdir(path)))

Command-Line Arguments 

import sys
print 'Number of arguments:', len(sys.argv), 'arguments.'
print 'Argument List:', str(sys.argv)

staticmethod vs classmethod

You can add a staticmethod and a classmethod as below. 

class Object:
    @staticmethod
    def method(content):
        print content

    factor = 1
    @classmethod
    def mul(cls, x):
        return cls.factor*x
  • staticmethod is the same as its C++ counterpart. The class only lends its name; it implements the function in a kind of namespace, and it does not run on the instance.
  • classmethod receives the class itself as an argument. By convention it is written as cls. So when called from an instance, the class is passed as the argument.

args and kwargs

See Python:Function#args and kwargs.

instance check

To check a Type: 

x = 'string'
isinstance(x, basestring) # Python 2.x, you would do
isinstance(x, str) # In Python 3.x, it would be
isinstance(x, unicode) # In Python 3.x, it would be
isinstance(x, list)
# ...

To print the list of properties of a class: 

an = Animal()
dir(a)

To print the list of member variables of a class: 

an = Animal()
an.__dict__
## or
attrs = vars(an)

access class property from string

To access a class attribute by a string:

  • getattr(x, 'y') is equivalent to x.y
  • setattr(x, 'y', v) is equivalent to x.y = v
  • delattr(x, 'y') is equivalent to del x.y

Forward declarations (referring to the same class type inside the class)

Inside of the class, the class is not defined yet, causing a NameError (and PyCharm to complain).

To get around this, use forward declarations: <syntaxhighlight lang="python> class Foo:

def foo_method(self, other_foo: "Foo"):
return "Hello World!"

</syntaxhighlight>

Basically, if a type annotations is a string, it is evaled after the whole module is loaded, so it can evaluate to the Foo class.

Execute a string

To execute a string directly: 

exec 'print "hello world"'
## OR:
x = eval("2+2")

Singleton 

class SingletonInstane:
  __instance = None

  @classmethod
  def __getInstance(cls):
    return cls.__instance

  @classmethod
  def instance(cls, *args, **kargs):
    cls.__instance = cls(*args, **kargs)
    cls.instance = cls.__getInstance
    return cls.__instance

class MyClass(BaseClass, SingletonInstane):
  pass

c = MyClass.instance()

Environment variables in Python

How to use environment variables: 

import os

# Set environment variables
os.environ['API_USER'] = 'username'
os.environ['API_PASSWORD'] = 'secret'

# Get environment variables
USER = os.getenv('API_USER')
PASSWORD = os.environ.get('API_PASSWORD')

# Getting non-existent keys
FOO = os.getenv('FOO') # None
BAR = os.environ.get('BAR') # None
BAZ = os.environ['BAZ'] # KeyError: key does not exist.

Python install path

The Python install path can be checked indirectly as below. 

>>> import sys
>>> sys.path
['', '/Users/username/Project/opm/python', '/Library/Python/2.7/site-packages/scons-2.3.0', ... ]

Path of imported module 

>>> import sys
>>> sys.modules['os']
<module 'os' from 'c:\Python26\lib\os.pyc'>

Find Python libraries

For how to find Python libraries, see the relevant section.

Python PIPE 

import os
import errno

FIFO = 'mypipe'

try:
    os.mkfifo(FIFO)
except OSError as oe: 
    if oe.errno != errno.EEXIST:
        raise

print("Opening FIFO...")
with open(FIFO) as fifo:
    print("FIFO opened")
    while True:
        data = fifo.read()
        if len(data) == 0:
            print("Writer closed")
            break
        print('Read: "{0}"'.format(data))

As others have pointed out, there is no built-in mount function. However, it is easy to create one using ctypes, and this is a bit lighter weight and more reliable than using a shell command.

How to mount in python

Here's an example: 

import ctypes
import ctypes.util
import os

libc = ctypes.CDLL(ctypes.util.find_library('c'), use_errno=True)
libc.mount.argtypes = (ctypes.c_char_p, ctypes.c_char_p, ctypes.c_char_p, ctypes.c_ulong, ctypes.c_char_p)

def mount(source, target, fs, options=''):
  ret = libc.mount(source.encode(), target.encode(), fs.encode(), 0, options.encode())
  if ret < 0:
    errno = ctypes.get_errno()
    raise OSError(errno, f"Error mounting {source} ({fs}) on {target} with options '{options}': {os.strerror(errno)}")

mount('/dev/sdb1', '/mnt', 'ext4', 'rw')

How to change the User in Python

See the following functions.

  • os.getgid()
  • os.getuid()
  • os.setgid()
  • os.getuid()

Or see the Stackoverflow - Changing user within python shell page.

If you want to get the UID from a user name, see the Python:pwd package.

How to inherit from Dict 

from typing import Dict

class User(Dict[str, str]):
    pass

Initialization order with multiple inheritance

See Python:MethodResolutionOrder#Initialization order with multiple inheritance.

How to implement a destructor

See weakref.finalize for details.

How to check site-packages

There are the following methods.

  • python -m site
  • python -c 'import site; print(site.getsitepackages())'
  • python3 -c 'import sysconfig; print(sysconfig.get_paths()["purelib"])'
  • python -m site --user-site

User Directory/Data

Either use one of the following libraries,

  • appdirs - A small Python module for determining appropriate platform-specific dirs, e.g. a "user data dir".
  • appdata - AppData is a Python library to access app data paths based on the operating system the app is running on.

or implement the function from the example: 

import sys
import logging
import pathlib
from typing import Optional, Union, List, Tuple


logger = logging.getLogger(__name__)


def get_user_data_dir(appending_paths: Union[str, List[str], Tuple[str, ...]] = None) \
        -> pathlib.Path:
    """
    Returns a parent directory path where persistent application data can be stored.
    Can also append additional paths to the return value automatically.

    Linux: ~/.local/share
    macOS: ~/Library/Application Support
    Windows: C:/Users/<USER>/AppData/Roaming

    :param appending_paths: Additional path (str) or paths (List[str], Tuple[str]) to append to return value
    :type appending_paths: Un

    :return: User Data Path
    :rtype: str
    """
    logger.debug(f'Getting Home Path...')
    home = pathlib.Path.home()
    logger.debug(f'Home Path: {home}')

    system_paths = {
        'win32': home / 'AppData/Roaming',
        'linux': home / '.local/share',
        'darwin': home / 'Library/Application Support'
    }

    logger.debug(f'Getting System Platform...')
    if sys.platform not in system_paths:
        raise SystemError(f'Unknown System Platform: {sys.platform}. Only supports {", ".join(list(system_paths.keys()))}')
    data_path = system_paths[sys.platform]

    if appending_paths:
        if isinstance(appending_paths, str):
            appending_paths = [appending_paths]
        for path in appending_paths:
            data_path = data_path / path

    logger.debug(f'System Platform: {sys.platform}')
    logger.debug(f'User Data Directory: {system_paths[sys.platform]}')
    logger.debug(f'Return Value: {data_path}')
    return data_path

Read-only property

This will make Foo.number a read-only property: 

class MetaFoo(type):
    @property
    def number(cls):
        return cls.x

class Foo(object, metaclass=MetaFoo):
    x = 4

print(Foo.number)
# 4

Foo.number = 6
# AttributeError: can't set attribute

== A guide to implementing Defense in Depth for Python supply chain security

Countering supply chain attacks on the Python package ecosystem calls for a multi-layered defense strategy that does not rely on any single control. The article recommends blocking static vulnerabilities with Ruff's S (Bandit) rules, pinning dependencies by cryptographic hash with uv, and running pip-audit and generating SBOMs (CycloneDX) in CI environments. For package publishing, it adopts OIDC-based Trusted Publishing instead of long-lived API tokens, and presents a practical pipeline that deliberately delays the adoption of new packages (Delayed Ingestion) to give the community time to vet them for malicious packages.

In-depth analysis

  • Supply chain attack vectors and Transitive Dependencies: PyPI currently hosts more than 740,000 packages, and large-scale security incidents keep happening, such as the ctx domain takeover, the Ultralytics (YOLO) CI script injection, and the GhostAction token theft. Even installing a single package (e.g. Flask) pulls in many transitive dependencies (e.g. MarkupSafe), so even packages the developer never explicitly declared form a huge Attack Surface.
  • Blocking vulnerabilities in your own code up front: Defects in internal code must be prevented before worrying about external packages. Hardcoded secrets, weak hash algorithms (MD5, SHA1), network requests without timeouts (enabling DoS), and unsafe serialization (pickle) are easy to miss in code review. To prevent them, it is essential to integrate Ruff's Bandit security rules into CI/CD pipelines and IDEs so they are automatically detected and blocked.
  • Dependency pinning and Delayed Ingestion: When installing dependencies, go beyond simply specifying versions and pin the packages' cryptographic hashes to block tampering in the runtime environment. Also, to reduce the risk during the window before a newly released malicious package is blocked, an effective strategy is to use uv's --exclude-newer flag, or to put a '7-day Ingestion Queue' on the internal mirror repository so that only packages that have passed the community's initial vetting are let into the internal network.
  • Secure package publishing: Open-source maintainers and package publishers should retire static API tokens, which are always at risk of theft, and switch to OIDC (OpenID Connect)-based Trusted Publishing using Sigstore. This automatically produces an Attestation that cryptographically proves the link to the source repository.

Checking Transitive Dependencies

Check the hidden dependency tree that comes along when installing a single package (Flask) 

uv pip install flask
uv pip tree

Output: 

flask v3.1.0
├── blinker v1.9.0
├── click v8.1.8
├── itsdangerous v2.2.0
├── jinja2 v3.1.5
│   └── markupsafe v3.0.2
└── werkzeug v3.1.3
    └── markupsafe v3.0.2

Applying the security ruleset (Bandit) with Ruff

pyproject.toml configuration example 

[tool.ruff]
line-length = 120
# E(Error), F(Pyflakes)와 함께 S(Bandit 보안 규칙) 활성화
lint.select = ["E", "F", "S"]

Examples of typical vulnerability patterns automatically detected by the Ruff 'S' ruleset: 

# FLAGGED: S301 - pickle.loads()는 임의 코드 실행 위험이 있음 (json.loads() 권장)
import pickle
data = pickle.loads(untrusted_input)

# FLAGGED: S608 - 문자열 포매팅을 통한 SQL 인젝션 취약점
<syntaxhighlight lang="python">
cursor.execute(f"SELECT * FROM users WHERE name = '{user_input}'")

# FLAGGED: S113 - 타임아웃이 설정되지 않은 외부 요청 (무한 대기 및 DoS 공격 노출)
import requests
response = requests.get("[https://api.example.com/data](https://api.example.com/data)")

Controlling new packages with Delayed Ingestion

Compile dependencies only from packages released before a given date (e.g. 7 days ago) to avoid early malware/bugs 

uv pip compile --exclude-newer 2026-03-02 requirements.in -o requirements.txt

Ingestion Control pipeline within an organization

Stage

System component

Security purpose and description

Intake

PyPI ➜ Ingestion Queue

New packages published on PyPI are brought into a queue instead of being distributed to internal systems right away

Wait

Wait (ex. 7 days)

Secures a guaranteed window for the community and security researchers to analyze whether the package is malicious and whether it has vulnerabilities

Verification

Security Scan ➜ Approved

After the waiting period, approved only if it passes scans for known vulnerabilities (CVE) and malware

Distribution

Internal Mirror ➜ Developers

Only verified packages are cached in the internal mirror repository, so development teams can use them safely

The data validation landscape in Python in 2025

  • The data validation landscape in 2025 | GeekNews
  • An overview of the main data validation tools as of now (2025) and recommendations by situation
  • Data validation is the process of checking data quality automatically or semi-automatically
    • Checking data types, counting missing values, detecting abnormal values
  • Can validate not only DataFrame rows but also API inputs and form submissions
  • Users can define rules, such as that the values in a given column must fall within a given range
  • On validation failure: raise an error, or generate a validation report and then handle it manually or automatically

Why data validation matters

  • Analysis work in public institutions falls into two types:
    • Ad hoc analysis – one-off analysis work
    • Regular statistics production – regularly collecting and processing new data
  • Validation is needed before data errors affect analysis results
  • Data validation is effective at reducing the risk of errors and improving accuracy

Main data validation tools

  • Great Expectations - An open-source framework for Data Quality
  • Pointblank - The newest Python data validation tool, released in 2024 (made by RStudio → Posit)
  • Pandera - Provides an API similar to Great Expectations
  • Pydantic - A dictionary-based validation tool, not DataFrame-based
  • Cerberus - A dictionary-based validation tool
  • jsonschema - A JSON data validation tool

Which tool to use in the public sector

  • Validating DataFrames or databases:
    • Used in production systems → Great Expectations recommended
    • Simple validation → Pandera recommended
    • Trying the newest tool → Pointblank recommended
  • Validating APIs or user input:
    • Unstructured data → Pydantic recommended
  • Simple JSON validation:
    • jsonschema recommended
  • If you need simple validation:
    • Cerberus recommended

Troubleshooting

TypeError: 'type' object is not subscriptable

For example, remove the type arguments from something like dict[int] and change it to dict.

TypeError: get() takes no keyword arguments

Remove the keyword arguments used in the function call.

For example, if it was used like get(size=10), change it to get(10).

TypeError: 'method' object is not subscriptable

Check whether you used square brackets [...] and parentheses (...) correctly, whether you used a Type Annotation in a place where it is not allowed, and so on.

For example, given the following code: 

from multiprocessing import SimpleQueue


def _receiver_main(queue: SimpleQueue[bytes]):
    ...

SimpleQueue[bytes] must be written as SimpleQueue.

UnicodeDecodeError 

import codecs
with codecs.open(file_name, 'r', encoding='utf-8', errors='ignore') as f:
    print(f.read())

Unsafe use of relative rpath

On Mac OSX, the following can happen when you import certain libraries. 

ImportError: dlopen(/Users/username/Project/fast-rcnn/tools/../caffe-fast-rcnn/python/caffe/_caffe.so, 2): Library not loaded: libboost_system.dylib
  Referenced from: /Users/username/Project/fast-rcnn/libboost_thread.dylib
  Reason: unsafe use of relative rpath libboost_system.dylib in /Users/username/Project/fast-rcnn/libboost_thread.dylib with restricted binary

You can fix this by changing the install name of the library, but a simple fix is to copy the Python binary as below. 

$ cp System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python .

Runtime error R6034

Among the suggested methods, if you use Anaconda, there is also the following method. 

$ conda install -c anaconda msvc_runtime=1.0.1

PyThreadState_Get: no current thread

The program exits after printing the following error. 

Fatal Python error: PyThreadState_Get: no current thread
[kilojoules-20160s:64471] *** Process received signal ***
[kilojoules-20160s:64471] Signal: Abort trap: 6 (6)
[kilojoules-20160s:64471] Signal code:  (0)
[kilojoules-20160s:64471] *** End of error message ***

This can happen when separate libraries each carry their own separate Python implementation. It is a problem that typically arises when you build several libraries as static libraries. Switching to shared libraries fixes it.

Custom OpenSSL library

When multiple OpenSSL libraries exist and the -lssl flag does not link against the SSL at the desired path, here is a fix.

Modify the configure.ac and Modules/Setup.dist files as below. 

diff -urN Python-3.7.3-ori/configure.ac Python-3.7.3/configure.ac
--- Python-3.7.3-ori/configure.ac   2019-03-26 05:21:05.000000000 +0900
+++ Python-3.7.3/configure.ac   2019-06-12 10:13:29.831577474 +0900
@@ -5612,6 +5612,14 @@
 # Check for usable OpenSSL
 AX_CHECK_OPENSSL([have_openssl=yes],[have_openssl=no])

+have_openssl=yes
+OPENSSL_INCLUDES="/usr/local/c2core/include"
+OPENSSL_LDFLAGS="/usr/local/c2core/lib/libssl.so /usr/local/c2core/lib/libcrypto.so"
+OPENSSL_LIBS=""
+AC_SUBST([OPENSSL_INCLUDES])
+AC_SUBST([OPENSSL_LIBS])
+AC_SUBST([OPENSSL_LDFLAGS])
+
 if test "$have_openssl" = yes; then
     AC_MSG_CHECKING([for X509_VERIFY_PARAM_set1_host in libssl])

diff -urN Python-3.7.3-ori/Modules/Setup.dist Python-3.7.3/Modules/Setup.dist
--- Python-3.7.3-ori/Modules/Setup.dist 2019-03-26 05:21:05.000000000 +0900
+++ Python-3.7.3/Modules/Setup.dist 2019-06-12 10:14:53.375633702 +0900
@@ -204,14 +204,14 @@
 #_csv _csv.c

 # Socket module helper for socket(2)
-#_socket socketmodule.c
+_socket socketmodule.c

 # Socket module helper for SSL support; you must comment out the other
 # socket line above, and possibly edit the SSL variable:
-#SSL=/usr/local/ssl
-#_ssl _ssl.c \
-#  -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
-#  -L$(SSL)/lib -lssl -lcrypto
+SSL=/usr/local/c2core
+_ssl _ssl.c \
+   -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
+   -L$(SSL)/lib $(SSL)/lib/libssl.so $(SSL)/lib/libcrypto.so

 # The crypt module is now disabled by default because it breaks builds
 # on many systems (where -lcrypt is needed), e.g. Linux (I believe).

Then run the commands as below. The important point is that you must use the -Wl,-rpath,$PREFIX/lib option to specify the path searched first for shared libraries. 

PKG_CONFIG_PATH=$PREFIX/lib/pkgconfig \
    CFLAGS="-I/opt/X11/include -I$PREFIX/include -fPIC" \
    LDFLAGS="-L/opt/X11/lib -L$PREFIX/lib -Wl,-rpath,$PREFIX/lib" \
    ./configure --enable-shared --enable-optimizations --prefix=$PREFIX | tee $BUILD_LOG1
make -j8 install

make error: ModuleNotFoundError: Not module named '_ctypes' 

ModuleNotFoundError: Not module named '_ctypes'
maek: *** [install] Error 1

If you get the error above when building from source code, try installing libffi. On Ubuntu, use sudo apt-get install libffi-dev.

One more word, full necessary dependencies on my Ubuntu 18.04: 

sudo apt-get install build-essential libsqlite3-dev sqlite3 bzip2 libbz2-dev zlib1g-dev libssl-dev openssl libgdbm-dev libgdbm-compat-dev liblzma-dev libreadline-dev libncursesw5-dev libffi-dev uuid-dev

CERTIFICATE_VERIFY_FAILED

See CERTIFICATE_VERIFY_FAILED.

ImportError: dlopen: cannot load any more object with static TLS

Working on it ...

Can't connect to HTTPS URL because the SSL module is not available

You must install it together with OpenSSL when building Python.

NameError from the typing package in exec

See Python:builtins#NameError from the typing package in exec.

coverage.py malfunctions after exec

See Python:builtins#coverage.py malfunctions after exec.

When FileNotFoundError occurs in os.getcwd()

See Python:os#FileNotFoundError in os.getcwd.

ModuleNotFoundError: No module named '../pydevd'

See Python:ensurepip#ModuleNotFoundError in pydevd.

Annotation loss right after running eval

See Python:Eval#Annotation loss problem.

Missing api-ms-win-core-path-l1-1.0.dll

It seems python 3.9.x and later is not supported on Windows 7. (Compile it yourself if needed)

That DLL (api-ms-win-core-path-l1-1.0.dll) seems to be supported from Windows 10 onward. (Windows 8.1 needs checking)

object.new() takes exactly one argument

The following error is printed: 

TypeError: object.__new__() takes exactly one argument (the type to instantiate)

It means object.__new__() needs only a single Argument, the type. If the following part of the code is the problem: 

class ModuleOpen(ModuleBase):
    _opened: bool

    def __new__(cls, *args, **kwargs):
        instance = super().__new__(cls, *args, **kwargs)
        instance._opened = False
        return instance

Change it to instance = super().__new__(cls).

map result does not loop in zip 

areas = map(lambda c: contour_area(c, area_oriented), contours)
for contour, area in zip(contours, areas):
    ...

In the situation above, areas is a Generator created by map. This thing does not loop in python 3.10.x (exact version and circumstances unknown). So you need to wrap map() in a list, as in list(map()).

See also

  • Lua
  • Ruby
  • Perl
  • Go
  • Mojo - A new programming language for AI developers
  • PyXL - A hardware processor that runs Python
  • PyO3 - Rust bindings for the Python interpreter
  • Monty - A lightweight Python interpreter for safely running AI-generated code

Favorite site

Basic

Online Python Coding

Guide

Parsing C++

Tutorials

Articles

References

  1. Detailed differences between "make install" and "make altinstall" ↩

  2. http://docs.python.org/faq/windows.html#is-a-pyd-file-the-same-as-a-dll ↩

  3. Python_underscore_-_mingrammers_note.pdf ↩

  4. Python_Numbers_Every_Programmer_Should_Know_-_Michael_Kennedy's_Thoughts_on_Technology.pdf ↩

  5. 14_Advanced_Python_Features_-_Edward_Li's_Blog.pdf ↩

author Zer0 sitemap