Python to Cython Compilation Engine
ObjCompile is a powerful and integral part of the project's build pipeline. Its primary responsibility is to transpile standard Python (.py) modules into compiled C extensions (.so on Linux, .pyd on Windows) using Cython. This process provides significant performance improvements and helps protect the source code by distributing compiled binaries instead of raw Python files.
The script is designed to be highly configurable, handling different services, packages, and build environments (local vs. Docker).
The ObjCompile class orchestrates the entire compilation workflow:
Architecture Detection (get_architecture): Before compiling, it determines the target system's architecture by compiling a small test file. This ensures that the generated binaries are compatible with the environment they will run in.
Staging (stage_factory): It prepares a "factory" (a directory containing a set of related modules, like factory.core or factory.service) for compilation. This involves copying the necessary source files into a dedicated build directory (resource.build). This step is crucial for isolating builds and preventing source tree pollution.
Cleaning (clean_binaries): It provides a utility to recursively find and delete previously compiled binaries (.so, .pyd, .c files), ensuring a clean build from scratch.
Compilation (compile_factory): This is the core method where the magic happens. It recursively traverses a staged factory directory and:
.py).cythonize to convert each Python file into a C file and then compiles it into a shared object file.def_module table)..py source file after a successful compilation, leaving only the compiled binary. This is a key step for performance and source code obfuscation.factory.core, factory.pages).package.llm, package.image). The compiler can be instructed to only compile modules belonging to certain packages.ObjCompile.py is executable and provides a typer-based CLI for invoking the compilation process.
serviceCompiles a single, specified service. This is the most common command for targeted builds.
python3 factory.deploy/ObjCompile.py service <service-name> [OPTIONS]
<service-name>: The name of the service to compile (e.g., core, pages, ai).--package-name / -p: (Optional) Compile only a specific package within the service.--build-dir / -b: (Optional) Specify a custom directory for the build output.--package-set / -ps: (Optional) Restrict the compilation to a predefined set of packages.--offline: (Optional) Skip all DB writes to def_module / Mongo deploy_compile. Required for Docker-image builds where no database is reachable. When set, compile progress is not logged to the registry — only Cython output is produced.allCompiles all available services listed in the SERVICE_LIST.
python3 factory.deploy/ObjCompile.py all [OPTIONS]
cleanRemoves all compiled binaries from the project, allowing for a fresh build.
python3 factory.deploy/ObjCompile.py clean
testRuns a set of internal system tests for the compilation process.
python3 factory.deploy/ObjCompile.py test