PEP 578
lastmod 2022-06-29created 2022-03-112 edits
Python Runtime Audit Hooks
Abstract
This PEP describes additions to the Python API and specific behaviors for the CPython implementation that make actions taken by the Python runtime visible to auditing tools. Visibility into these actions gives test frameworks, logging frameworks, and security tools the opportunity to monitor and optionally limit actions taken by the runtime.
This PEP proposes adding an API for arbitrary events, plus another API specific to the module import system, to provide insight into running Python applications. The APIs are intended to be usable by all Python implementations, but the specific messages and values used are not specified here, so that implementations are free to decide how best to provide information to their users. Some examples likely to be used in CPython are provided for illustration.
See PEP 551 for discussion and recommendations on using these audit APIs to strengthen the security of the Python runtime.
Simple usage
def audit(event, args):
if event == 'code.__new__':
print(f'audit: {event} with args={args}')
sys.addaudithook(audit)
Overview
- [Recommended] Overview of runtime audits in PEP 578
Today, much programming is done with the help of a broad ecosystem around the language. So each program builds on other programs to solve a given problem. As a program grows, more and more of its work has to be done by things others have already solved and that are added to the codebase. However, this inherently relies on certain factors such as the module's maintenance, release cycle, documentation, stability, and so on. More importantly, it relies on trust that the code does what it advertises. Many modules are open source, but while their APIs get used, programmers bother less with reading the source code.
However, there are many cases where a module does something malicious during a release, which can go undiscovered for a long time unless someone investigates the suspicious activity and reports it publicly to others. This has happened several times, such as a few months ago, when a famous CSS-related Ruby library used eval in unrelated code downloaded about 28 million times, leading to a security incident reported on GitHub . Python is no different when it comes to these attacks, because there is no concrete way to check whether third-party code is doing things like network requests, eval functions, and so on.
This PEP provides an API for adding auditable hooks that are triggered for specific events, and a way for existing systems to use it to provide additional information for monitoring and the like. Below I will describe a simple program showing how this can be used.
Suppose we are writing a program that does some numerical work and want to compute the product of a series. Naturally, we can write it as below.
# stats.py
from functools import reduce
def product(series):
return reduce(lambda acc, num: acc * num, series)
We already have a math library from PyPI as a dependency that does this, and we don't want to reimplement it as a utility function. So we use it in our program.
Since there is inherent trust that the module does what it does, we don't bother looking through the code even for new releases of the stats dependency, as long as the API doesn't change. At some point, the library now makes http requests, and the new code can pick up a malicious change like the one below.
# stats.py
from functools import reduce
def product(series):
import urllib.request
try:
urllib.request.urlopen("http://example.com")
except:
pass
return reduce(lambda acc, num: acc * num, series)
Unless we look through the code again or have a sandbox environment where network requests are monitored, there is little clue about the http activity that took place. PEP 578 provides a way to monitor this, and adds a function, sys.addaudithook, to the sys module, which can add a function as a callback for the whole set of defined events created. This is similar to sys.settrace(trace), where all function calls go through the trace function, but in this case it is only events. A list is available in the PEP. The list has an event added to the urllib.Request module, so an event is triggered for every http request in the urllib module. The callback gets the name of the event and the arguments with which the event is triggered. So we can filter it
Unless we look through the code again or have a sandbox environment where network requests are monitored, there is little clue about the http activity that took place. PEP 578 provides a way to monitor this, and adds the function sys.addaudithook to the sys module, so that a function can be added as a callback for the whole set of defined events. This is similar to sys.settrace(trace), where every function call goes through the trace function, but in this case it is only events. The list is available in the PEP. The list has the urllib.Request event added to the module, so an event is triggered for every http request in the urllib module. The callback gets the name of the event and the arguments with which the event is triggered. So we can filter it.
import sys
import stats
def audit_hook(event, args):
if event in ['urllib.Request']:
print(f"Network {event=} {args=}")
sys.addaudithook(audit_hook)
print(stats.product(range(1, 10)))
./python.exe audit_hooks_tut/app.py
Network event='urllib.Request' args=('http://example.com', None, {}, 'GET')
362880
Running the program above gives the event name and the event arguments. These events can be added from a Python program through sys.audit(event, args), and you can see the audit event for urllib.request, written as a public function in the form sys.audit('urllib.Request', req.full_url, req.data, req.headers, req.get_method()), so for every http request made through urllib.request it does the following. The event is passed through the hook function.
Types of audit events
An HTTP request is one event, but there are other things, such as wanting to know which imports a module performs, to check whether it is doing imports it shouldn't. In the example above, the math-related code or dependency doesn't need it for this use case, but we can rewrite it to check whether it imports the socket module, which is imported internally by the urllib.request module.
import sys
import stats
suspicious_modules = ['socket']
def audit_hook(event, args):
if event in ['urllib.Request']:
print(f"Network {event=} {args=}")
elif event in ['import'] and args[0] in suspicious_modules:
print(f"Suspicious import action {event=} module={args[0]}")
sys.addaudithook(audit_hook)
print(stats.product(range(1, 10)))
./python.exe audit_hooks_tut/app.py
Suspicious import action event='import' module=socket
Network event='urllib.Request' args=('http://example.com', None, {}, 'GET')
362880
Other events include eval, compiling code with compile, pickling, opening files, and so on, which may not be needed. Since these events can be defined in Python, a program can also define custom events so that third-party libraries can provide their own set of events. For example, when using the stats module and mod1 from PyPI, which is also used in my program, and the stats module uses it indirectly, I can monitor whether events defined by mod1 occur in my program. In this case, mod1 can provide a helper make_request, used in stats.py, that can be audited.
# stats.py
from functools import reduce
def product(series):
import mod1
mod1.make_request("http://example.com")
return reduce(lambda acc, num: acc * num, series)
# app.py
import sys
import stats
suspicious_modules = ['socket']
def audit_hook(event, args):
if event in ['urllib.Request', 'make_request']:
print(f"Network {event=} {args=}")
elif event in ['import'] and args[0] in suspicious_modules:
print(f"Suspicious import action {event=} module={args[0]}")
sys.addaudithook(audit_hook)
print(stats.product(range(1, 10)))
With mod1 defining a custom audit hook we can audit it from our program as shown. Thus with the above setup it gives the below output.
./python.exe audit_hooks_tut/app.py
Network event='make_request' args=('http://example.com',)
362880
This also lets library authors provide hooks for their libraries so they can be used for auditing purposes too. It also provides a C API and other means that enable more fine-grained auditing, described in the linked implementation PR. The PEP also has a section describing the limitations of sandboxing. The performance impact of this PEP was also shown to be negligible. This PEP is also closely related to PEP 551 - Security Transparency in the Python Runtime.
As Steve Dower mentioned in this tweet. The above method is recommended for use in production apps. See PEP 551.