codon README

The project's README from github.com/exaloop/codon. As one markdown file: README.md.

Codon banner

Docs  ·  FAQ  ·  Blog  ·  Discord  ·  Roadmap  ·  Benchmarks

Build Status

What is Codon?

Codon is a high-performance Python implementation that compiles to native machine code without any runtime overhead. Typical speedups over vanilla Python are on the order of 10-100x or more, on a single thread. Codon's performance is typically on par with (and sometimes better than) that of C/C++. Unlike Python, Codon supports native multithreading, which can lead to speedups many times higher still.

Think of Codon as Python reimagined for static, ahead-of-time compilation, built from the ground up with best possible performance in mind.

Goals

Non-goals

How it works

Codon figure

Quick start

Download and install Codon with this command:

/bin/bash -c "$(curl -fsSL https://exaloop.io/install.sh)"

After following the prompts, the codon command will be available to use. For example:

Many more options are available and described in the docs.

Alternatively, you can build from source.

Examples

Basics

Codon supports much of Python, and many Python programs will work with few if any modifications. Here's a simple script fib.py that computes the 40th Fibonacci number...

from time import time

def fib(n):
    return n if n < 2 else fib(n - 1) + fib(n - 2)

t0 = time()
ans = fib(40)
t1 = time()
print(f'Computed fib(40) = {ans} in {t1 - t0} seconds.')

... run through Python and Codon:

$ python3 fib.py
Computed fib(40) = 102334155 in 17.979357957839966 seconds.
$ codon run -release fib.py
Computed fib(40) = 102334155 in 0.275645 seconds.

Using Python libraries

You can import and use any Python package from Codon via from python import. For example:

from python import matplotlib.pyplot as plt
data = [x**2 for x in range(10)]
plt.plot(data)
plt.show()

(Just remember to set the CODON_PYTHON environment variable to the CPython shared library, as explained in the the Python interoperability docs.)

Parallelism

Codon supports native multithreading via OpenMP. The @par annotation in the code below tells the compiler to parallelize the following for-loop, in this case using a dynamic schedule, chunk size of 100, and 16 threads.

from sys import argv

def is_prime(n):
    factors = 0
    for i in range(2, n):
        if n % i == 0:
            factors += 1
    return factors == 0

limit = int(argv[1])
total = 0

@par(schedule='dynamic', chunk_size=100, num_threads=16)
for i in range(2, limit):
    if is_prime(i):
        total += 1

print(total)

Note that Codon automatically turns the total += 1 statement in the loop body into an atomic reduction to avoid race conditions. Learn more in the multithreading docs.

Codon also supports writing and executing GPU kernels. Here's an example that computes the Mandelbrot set:

import gpu

MAX    = 1000  # maximum Mandelbrot iterations
N      = 4096  # width and height of image
pixels = [0 for _ in range(N * N)]

def scale(x, a, b):
    return a + (x/N)*(b - a)

@gpu.kernel
def mandelbrot(pixels):
    idx = (gpu.block.x * gpu.block.dim.x) + gpu.thread.x
    i, j = divmod(idx, N)
    c = complex(scale(j, -2.00, 0.47), scale(i, -1.12, 1.12))
    z = 0j
    iteration = 0

    while abs(z) <= 2 and iteration < MAX:
        z = z**2 + c
        iteration += 1

    pixels[idx] = int(255 * iteration/MAX)

mandelbrot(pixels, grid=(N*N)//1024, block=1024)

GPU programming can also be done using the @par syntax with @par(gpu=True). See the GPU programming docs for more details.

NumPy support

Codon includes a feature-complete, fully-compiled native NumPy implementation. It uses the same API as NumPy, but re-implements everything in Codon itself, allowing for a range of optimizations and performance improvements.

Here's an example NumPy program that approximates $\pi$ using random numbers...

import time
import numpy as np

rng = np.random.default_rng(seed=0)
x = rng.random(500_000_000)
y = rng.random(500_000_000)

t0 = time.time()
# pi ~= 4 x (fraction of points in circle)
pi = ((x-1)**2 + (y-1)**2 < 1).sum() * (4 / len(x))
t1 = time.time()

print(f'Computed pi~={pi:.4f} in {t1 - t0:.2f} sec')

... run through Python and Codon:

$ python3 pi.py
Computed pi~=3.1417 in 2.25 sec
$ codon run -release pi.py
Computed pi~=3.1417 in 0.43 sec

Codon can speed up NumPy code through general-purpose and NumPy-specific compiler optimizations, including inlining, fusion, memory allocation elision and more. Furthermore, Codon's NumPy implementation works with its multithreading and GPU capabilities, and can even integrate with PyTorch. Learn more in the Codon-NumPy docs.

Documentation

Please see docs.exaloop.io for in-depth documentation.

Acknowledgements

This project would not be possible without:

codon (exaloop/codon)

Searching Database…

16.9K ·GitHub
Codon
这是什么？Codon 是一个高性能的 Python 实现，能把你的代码直接编译成原生机器码——单线程下通常比原版 Python 快 10 到 100 倍，媲美 C/C++。
那我是要学一门新语言，还是照常写普通 Python？你写的就是普通 Python:Codon 会对它做提前编译，很多 Python 程序只需少量修改甚至不用改就能运行。它基本不引入新语法，只有少数几处例外，比如表达并行的地方。
怎么让它跑起来？用 Quick start 部分的一行命令安装，装好后你就有了 codon 命令。用 codon run file.py 运行脚本(加 -release 开启优化)，或者用 codon build -release file.py 编译成独立的可执行文件。
那我已有的 Python 包还能用吗？可以，通过 'from python import' 来用。把 CODON_PYTHON 环境变量指向 CPython 共享库，这些模块就会在真正的 CPython 上运行，而你的其余代码仍然编译成原生执行。
任何 Python 程序都能跑，还是有些做不到？它不是 CPython 的直接替代品：不适合静态编译的 Python 特性不受支持(参见文档里的 'Differences with Python')。对于较大的代码库，它提供了 JIT 装饰器、Python 扩展后端以及完整的 Python 互操作。
能在 GPU 上运行代码吗？可以——Codon 支持编写并运行 GPU 内核，包括通过同样的 @par 注解，即 @par(gpu=True)。README 里演示了一个 Mandelbrot 内核，它的 NumPy 实现同样支持 GPU 和多线程。
它真的自带 NumPy 吗？是的——一个功能完整、完全编译成原生的 NumPy 重新实现，API 完全相同。由于它内置于编译器中，Codon 能应用内联、融合、省略内存分配等优化，甚至还能和 PyTorch 集成。
并行是怎么做的——我要自己管线程吗？不用。Codon 基于 OpenMP 提供原生多线程，而且没有 GIL:给 for 循环加上 @par 注解(调度方式、分块大小、线程数都是可选项)，编译器就会把它并行化，并自动把循环体里的累加转成原子归约，避免竞态条件。
什么情况下我该选 Codon 而不是直接用 CPython?适合计算密集型的场景：数值计算、大规模 NumPy 数组代码，或者原本需要手动优化的循环。凡是原版 Python 太慢、但用 C/C++ 或 Rust 重写又嫌太麻烦的地方——Codon 保留 Python 语法，同时带来底层语言的速度，用上多线程或 GPU 后提升更大。
What are similar projects?numba, spinel, taskflow, cython, wasmtime, CalcKernel, cupy, TornadoVM
Which topics is it in?aot, parallel-programming, llvm, gpu-programming, scientific-computing, multithreading, runtime, numpy, optimization, high-performance, compiler, pytorch
Creator
exaloop
Size
13.2 MB
Stars
16.9K
Updated
CommitOct 14Releasev0.20.3Sep 28
Age
5y
Commits · 90 Days