Cython

lastmod 2024-09-23created 2016-05-0227 edits

Cython is a compiled language designed to make it easy to create CPython extension modules. It is based on Python syntax, with additions such as a foreign function interface for calling C/C++ routines and static typing for faster execution. These features let you keep Python's high productivity while integrating concisely with external C libraries or speeding up execution.

Categories

• Recc:cython

Simple Example

• A simple Cython example
• Cython: adding speed to Python

First, save the following as helloworld.pyx.

print "Hello World"

Create a setup.py file and enter the name of the file to compile, as below.

from distutils.core import setup
from Cython.Build import cythonize
setup( ext_modules = cythonize("helloworld.pyx") )  

Running the following builds a dynamic library.

$ python setup.py build_ext --inplace

In practice, use it just like regular Python.

import helloworld

More simple

Entering the following generates a *.c file. Compile this file with gcc, etc.

$ cython helloworld.pyx

Flags

--embed

Embeds into a single file? - needs more checking.

Source Files and Compilation

• Source Files and Compilation — Cython 3.0a6 documentation

C-Flags

• Stackoverflow - How to remove -pthread compiler flag from cython setup file - Did not work well on Python 3.7.

Packaging

• Integrating Cython extension with setuptools and unit testing
• Stackoverflow - Package only binary compiled .so files of a python library compiled with Cython
• [Recommended] How should I structure a Python package that contains Cython code
• Stackoverflow - How should I structure a Python package that contains Cython code
• [Recommended] Distributing python packages protected with Cython

After configuring setup.py, the command generally used for distribution in tox, devpi, etc. is sdist.

Single file compile

extensions = [
    Extension(
        name="your_module",  # 모듈 이름
        sources=["module1.pyx", "module2.pyx", "module3.py"],  # 파일들
        extra_compile_args=["-O3"],  # 컴파일러 최적화 플래그
        extra_link_args=[]  # 추가적인 링킹 플래그
    )
]

setup(
    ext_modules=cythonize(extensions)
)

Pure Python Benchmark

I checked the performance difference when running the following test.py code as a Cython version and a Python version.

import math

def test(a):
    mean = sum(a) / len(a)
    return math.sqrt((sum(((x - mean)**2 for x in a)) / len(a)))

The setup.py file for building the Cython module:

from setuptools import setup
from Cython.Build import cythonize

setup(
    ext_modules = cythonize("test.py")
)

Compile as follows.

python setup.py build_ext --inplace

The test code is as follows:

import datetime
import sys

from test import test as test1  # Cython method.
from test2 import test as test2  # Python method.


if __name__ == "__main__":
    a = [i for i in range(100000000)]

    now = datetime.datetime.now()
    k = test1(a)
    print("cython:", datetime.datetime.now() - now)

    now = datetime.datetime.now()
    k = test2(a)
    print("python:", datetime.datetime.now() - now)

    exit(0)

The performance difference is as follows:

cython: 0:00:25.082879
python: 0:00:30.424469

Over a 100-million-iteration loop, Cython wins by 5 seconds!

Troubleshooting

FutureWarning language_level not set

• Cython FutureWarning language_level not set #8102

Set language_level. Add the following comment to the first line of the *.pyx file.

# cython: language_level=3

Customize location of .so file generated by Cython

• Stackoverflow - Customize location of .so file generated by Cython

Apply setup.py as follows.

from setuptools import setup, find_packages, Extension
from Cython.Distutils import build_ext

ext_modules=[
    Extension("package.wrapper.wrap",    # location of the resulting .so
             ["package/wrapper/wrap.pyx"],) ]


setup(name='package',
      packages=find_packages(),
      cmdclass = {'build_ext': build_ext},
      ext_modules = ext_modules,
     )

Then compile as follows.

python setup.py build_ext --inplace

How to place cython files in setup.py

• Stackoverflow - How to include header file in Cython correctly (setup.py)?

Resolved through corrected MANIFEST.in:

global-include *.txt *.rst *.pyx *.pxd *.c *.h

setup.py:

setup(
    include_package_data=True,
    package_data={'': ['data/*', '*.pyx', '*.pxd', '*.c', '*.h']},
)

MANIFEST.in and setup.py must both cover all file patterns to support bdist and sdist.

error: each element of 'ext_modules' option must be an Extension instance or 2-tuple

• Stackoverflow - error: each element of 'ext_modules' option must be an Extension instance or 2-tuple

error: each element of 'ext_modules' option must be an Extension instance or 2-tuple

When importing the Cython package, it must not be added before setuptools:

# from Cython.Build import cythonize  # BED CODE !!!!
from setuptools import find_packages, setup
from setuptools.command.build_py import build_py

from Cython.Build import cythonize  # Correct CODE !!!!

See also

• Python
• Numba
• Nuitka

Favorite site

• Cython web site
• Wikipedia (en) Cython
• Using PyInstaller and Cython to create a Python executable | peterspython.com - Adding static Cython type declarations for two variables improved performance 50 times.

Basic

• Basic Tutorial — Cython 3.0a6 documentation
• Cython: adding speed to Python
• Getting started with Cython. | binaryflavor techblog
• cython => C++ wrapping