Skip to content

wolph/speedups

Repository files navigation

Speedups

CI Build Wheels PyPI Python License

C and Cython extensions for fast STL I/O and PostgreSQL-to-NumPy conversion.

Performance

Cython extensions run ~3x faster than pure Python for STL I/O and 2-3.5x faster for PostgreSQL array conversion, with the advantage growing at larger data sizes.

STL I/O Performance

Operation Facets Pure Python speedups Speedup
Read 10,000 22.3 ms 7.2 ms 3.1x
Write 10,000 28.2 ms 9.3 ms 3.0x
Read 100,000 220.3 ms 71.5 ms 3.1x
Write 100,000 289.5 ms 91.6 ms 3.2x
Read 1,000,000 2.26 s 720.8 ms 3.1x
Write 1,000,000 2.86 s 924.7 ms 3.1x
Read 10,000,000 22.61 s 7.23 s 3.1x
Write 10,000,000 28.65 s 9.39 s 3.1x

PostgreSQL COPY to NumPy Performance

Type Elements Pure Python speedups Speedup
int32 100K 7.4 ms 2.5 ms 3.0x
float64 100K 7.6 ms 3.5 ms 2.2x
int32 1M 71.0 ms 22.9 ms 3.1x
float64 1M 75.6 ms 29.9 ms 2.5x
int32 10M 748.1 ms 214.6 ms 3.5x
float64 10M 782.2 ms 268.6 ms 2.9x

Benchmarked on Apple M2 Pro, Python 3.14, macOS 15.4. PG data pre-populated in tables to isolate COPY+conversion time. Array dimensionality (1D/2D/3D) has no significant effect on performance.

Install

pip install speedups

With PostgreSQL support:

pip install speedups[postgres]

PostgreSQL Array → NumPy

Convert PostgreSQL arrays directly to NumPy ndarrays using psycopg's binary COPY protocol. Bypasses Python object creation for a significant speedup over the default loader.

Supports float4, float8, smallint, integer, and bigint arrays, from 1D to N-D.

import psycopg
from speedups.psycopg_loaders import NumpyLoader

with psycopg.connect("dbname=mydb") as conn:
    cursor = conn.cursor(binary=True)
    NumpyLoader.install(cursor)

    query = """
    COPY (
        SELECT array_agg(x)
        FROM generate_series(1, 100000) x
    ) TO STDOUT WITH BINARY
    """

    with cursor.copy(query) as copy:
        copy.set_types(["integer[]"])

        for row in copy.rows():
            print(row)  # numpy.ndarray

ASCII STL I/O

Read and write ASCII STL files at C speed. This module is used internally by numpy-stl — if you want to read or write STL files, use numpy-stl for a full-featured API. The Cython implementation uses direct sscanf/fprintf calls, avoiding Python string overhead entirely.

from speedups._stl import ascii_read, ascii_write

# Read
with open("model.stl", "rb") as f:
    buf = f.read(8192)
    name, mesh = ascii_read(f, buf)

# Write
with open("output.stl", "wb") as f:
    ascii_write(f, b"my_model", mesh)

Supported Types

PostgreSQL NumPy Dimensions
float4 float32 1D – ND
float8 float64 1D – ND
smallint int16 1D – ND
integer int32 1D – ND
bigint int64 1D – ND

Compatibility

  • Python 3.10, 3.11, 3.12, 3.13, 3.14
  • NumPy 1.x and 2.x

License

BSD-3-Clause

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Packages

 
 
 

Contributors