_____ _____ ____ _ _ _ _ ___ ____ ___ ____ _____
|_ _| ____/ ___| | | | \ | |/ _ \ / ___/ _ \| _ \| ____|
| | | _|| | | |_| | \| | | | | | | | | | |_) | _|
| | | |__| |___| _ | |\ | |_| | |__| |_| | _ <| |___
|_| |_____\____|_| |_|_| \_|\___/ \____\___/|_| \_\_____|
Automate • Innovate • Transform
https://www.technocore.co.za
Deployment, Compilation, and Infrastructure Management Modules
The factory.deploy package provides a comprehensive set of tools for managing the build, deployment, and infrastructure lifecycle of the Axion system. It includes compilation tools, packaging utilities, configuration management, and unified interfaces for various infrastructure substrates.
Python and Cython compilation module for optimizing code performance.
Features:
.pyc bytecode.so native extensionsUsage:
# ⚙️ Compile all services
dev-env/bin/python factory.deploy/ObjCompile.py all
# ⚙️ Compile specific service
dev-env/bin/python factory.deploy/ObjCompile.py service <name>
# ⚙️ Clean compiled binaries
dev-env/bin/python factory.deploy/ObjCompile.py clean
Key Optimizations:
shell=True in subprocess callsos.chdir() usageBuild pipeline orchestrator for compile → image → push → deploy.
Features:
ObjCompilecompile, image, push, deploy, status)Usage:
# 📦 Full pipeline
dev-env/bin/python factory.deploy/ObjBuild.py pipeline <package_name>
# 📦 Build image only
dev-env/bin/python factory.deploy/ObjBuild.py image <package_name>
# 🚀 Deploy only
dev-env/bin/python factory.deploy/ObjBuild.py deploy <package_name>
Abstract base class providing a unified interface for managing various infrastructure types.
Supported Substrates:
Core Features:
Abstract Methods:
connect() # Connect to substrate
list_instances() # List all instances
get_instance(name) # Get instance details
create_instance(cfg) # Create new instance
start_instance(name) # Start instance
stop_instance(name) # Stop instance
delete_instance(name) # Delete instance
execute_command(name) # Execute command in instance
health_check() # Check substrate health
get_metrics(name) # Get instance metrics
Data Classes:
ResourceLimits - CPU, memory, disk limitsResourceUsage - Current resource consumptionNetworkConfig - Network configurationInstanceTemplate - Infrastructure-as-code templatesDocker container management and image building.
Features:
/v2/_catalog)--build-arg against a static per-package DockerfileKey Methods:
build(package) - Build image using <package>.deployment.image.* from config.yamlpush(package) - Push image to configured registryconnect_registry(host) - Probe the configured registryLXC/LXD container management.
Features:
Configuration:
substrate_config.yamlProxmox VE cluster management (fully enhanced).
Features:
Status: ✅ 100% Complete - All abstract methods implemented
Configuration:
proxmox:
url: "https://proxmox.local:8006"
user: "root@pam"
verify_ssl: false
node_preference: ["node1", "node2"]
Libvirt/KVM virtual machine management.
Features:
Configuration:
libvirt:
uri: "qemu:///system" # or qemu+ssh://user@host/system
default_network: "default"
Supervisor process manager integration.
Features:
Configuration:
supervisor:
url: "http://localhost:9001/RPC2"
Unified configuration management for all substrate types.
Features:
docker.registry.host)Usage:
from SubstrateConfig import SubstrateConfig
config = SubstrateConfig("substrate_config.yaml")
host = config.get('docker', 'registry.host', '10.0.0.69')
memory = config.get('docker', 'default_resources.memory_mb', 512)
Configuration File: substrate_config.yaml (project root)
Configuration and cluster orchestration with full substrate integration.
Features:
Usage:
from factory.deploy.ObjConfig import Config
config = Config()
config.build("mypackage")
# ❤️ Run health checks
health = config.health_check_all_components()
# 📊 Get metrics
metrics = config.get_metrics()
CLI:
# 🏗️ Build cluster
dev-env/bin/python factory.deploy/ObjConfig.py build mypackage
# ❤️ Health check
dev-env/bin/python factory.deploy/ObjConfig.py health
# 📊 Display metrics
dev-env/bin/python factory.deploy/ObjConfig.py metrics
Main configuration orchestrator used by the build/deploy pipeline.
Notes:
ObjConfig_v2.py and ObjConfigProxy.py were removed.ObjConfig.py, ObjBuild.py, and ObjEnvironment.py for new workflows.Data configuration utilities now live in:
factory.deploy/extend.environment/ObjDataConfig.pyDNS management module with retry logic and event emission.
Features:
Usage:
from ObjDNSManager import ObjDNSManager
dns = ObjDNSManager()
# 🌐 Create single DNS record
dns.create_dns_record("www", "example.com", "10.0.0.1", api_token, api_email)
# 🛠️ Setup cluster DNS from YAML
results = dns.setup_from_config(yaml_config)
# 📈 Get statistics
stats = dns.get_statistics()
CLI:
# 🌐 Create DNS record
dev-env/bin/python factory.deploy/ObjDNSManager.py create \
www example.com 10.0.0.1 $TOKEN $EMAIL
# 🛠️ Setup cluster
dev-env/bin/python factory.deploy/ObjDNSManager.py setup-cluster config.yaml
Canonical implementation: factory.deploy/extend.environment/ObjDNSManager.py
Legacy import/CLI compatibility shim: factory.deploy/ObjDNSManager.py
Storage mounting and management module for S3 and NFS.
Status: legacy compatibility wrapper retained for older integrations.
For new deployments, prefer ObjEnvironment + substrate/build orchestration.
Usage:
from ObjStorageManager import ObjStorageManager, S3MountConfig
storage = ObjStorageManager()
# 📌 Mount S3 bucket
config = S3MountConfig(
bucket="my-bucket",
url="https://s3.amazonaws.com",
access_key="KEY",
secret_key="SECRET",
mount_point="/mnt/s3"
)
storage.mount_s3_bucket(config)
# 📌 Check if mounted
if storage.is_mounted("/mnt/s3"):
print("Mounted!")
SSH-based remote host provisioning module.
Status: legacy compatibility wrapper retained for older integrations.
For new deployments, prefer ObjEnvironment + substrate/build orchestration.
Usage:
from ObjRemoteProvisioner import ObjRemoteProvisioner, RemoteHost
provisioner = ObjRemoteProvisioner()
# 📌 Provision single host
host = RemoteHost(
name="web-01",
ip="10.0.0.10",
user="admin",
password="secret"
)
result = provisioner.provision_host(host)
# 📌 Provision cluster
results = provisioner.provision_cluster(hosts)
# ❤️ Health check
health = provisioner.health_check(host)
The following modules were removed from active use:
ObjPack.pyObjTerraform.pyObjPackage.pyObjConfig_v2.pyObjConfigProxy.pyUse ObjBuild.py and ObjEnvironment.py as the supported orchestration entrypoints.
# ⚙️ Compile all services in parallel
dev-env/bin/python factory.deploy/ObjCompile.py all
# 📌 Test compilation
dev-env/bin/python factory.deploy/ObjCompile.py test
# 📦 Run full build pipeline
dev-env/bin/python factory.deploy/ObjBuild.py pipeline homechoice
# 📦 Build all per-service targets
dev-env/bin/python factory.deploy/ObjBuild.py targets homechoice
from factory.deploy.ObjProxmox import ObjProxmox
# 📌 Connect to Proxmox
px = ObjProxmox()
# 💻 List VMs
vms = px.list_instances()
for vm in vms:
print(f"{vm['name']}: {vm['status']}")
# 💻 Start a VM
px.start_instance("my-vm")
# 📊 Get metrics
metrics = px.get_metrics("my-vm")
print(f"CPU: {metrics.cpu_percent}%")
from ObjSubstrate import InstanceTemplate, ResourceLimits
template = InstanceTemplate(
name="web-01",
base_image="ubuntu:22.04",
resources=ResourceLimits(
cpu_cores=2.0,
memory_mb=4096,
disk_gb=50
)
)
substrate.create_from_template(template)
from ObjSubstrate import InstanceEvent
def on_start(event):
print(f"Started: {event.instance_name}")
substrate.event_handler.subscribe(InstanceEvent.STARTED, on_start)
docker:
registry:
host: "10.0.0.69"
port: 9443
default_resources:
memory_mb: 512
cpu_cores: 1.0
proxmox:
url: "https://proxmox.local:8006"
user: "root@pam"
verify_ssl: false
lxc:
default_profile: "default"
default_image: "ubuntu/22.04"
libvirt:
uri: "qemu:///system"
supervisor:
url: "http://localhost:9001/RPC2"
global:
enable_metrics: true
enable_events: true
enable_caching: true
factory.deploy/
├── Build & Compilation
│ ├── ObjCompile.py # Cython compilation
│ └── ObjBuild.py # Build pipeline orchestration
│
├── Infrastructure (Substrates)
│ ├── ObjSubstrate.py # Base class (abstract)
│ ├── ObjDocker.py # Docker containers
│ ├── ObjLxc.py # LXC/LXD containers
│ ├── ObjProxmox.py # Proxmox VMs/containers ✅
│ ├── ObjLibvirt.py # Libvirt/KVM VMs
│ └── ObjSupervisor.py # Supervisor processes
│
├── Environment
│ ├── SubstrateConfig.py # Unified config (new)
│ ├── ObjConfig.py # Runtime/build configuration
│ ├── ObjEnvironment.py # DNS + secrets orchestration
│ └── extend.environment/
│ ├── ObjDNSManager.py
│ ├── ObjDataConfig.py
│ └── ObjSecretMigrate.py
│
└── Compatibility Wrappers
├── ObjDNSManager.py
├── ObjStorageManager.py
└── ObjRemoteProvisioner.py
| Feature | ObjSubstrate | ObjProxmox | ObjDocker | ObjLxc | ObjLibvirt | ObjSupervisor |
|---|---|---|---|---|---|---|
| Core CRUD | ✅ | ✅ | ✅ | ✅ | ✅ | ⚠️ |
| Health Check | ✅ | ✅ | ⏳ | ⏳ | ⏳ | ⏳ |
| Metrics | ✅ | ✅ | ⏳ | ⏳ | ⏳ | ⏳ |
| Events | ✅ | ✅ | ⚠️ | ⚠️ | ⚠️ | ⚠️ |
| Caching | ✅ | ✅ | ⚠️ | ⚠️ | ⚠️ | ⚠️ |
| Retry Logic | ✅ | ✅ | ⚠️ | ⚠️ | ⚠️ | ⚠️ |
| Templates | ✅ | ✅ | ⏳ | ⏳ | ⏳ | ⏳ |
Legend:
Single API for managing Docker, LXC, Proxmox, Libvirt, and Supervisor - write once, deploy anywhere.
# ⚙️ 1. Compile everything
dev-env/bin/python factory.deploy/ObjCompile.py all
# 📦 2. Build + deploy pipeline
dev-env/bin/python factory.deploy/ObjBuild.py pipeline mypackage
# 🏛️ 3. Deploy to infrastructure
# 📦 (Use substrate modules to deploy the package)
# 📌 1. Create from template
template = InstanceTemplate.from_yaml("templates/web-server.yaml")
substrate.create_from_template(template)
# ⚙️ 2. Configure resources
limits = ResourceLimits(cpu_cores=4.0, memory_mb=8192)
substrate.set_resource_limits("web-01", limits)
# 📌 3. Start and monitor
substrate.start_instance("web-01")
metrics = substrate.get_metrics("web-01")
# ❤️ Check substrate health
health = substrate.health_check()
if not health["healthy"]:
for issue in health["issues"]:
print(f"⚠️ {issue}")
# 📌 Monitor all instances
for instance in substrate.list_instances():
metrics = substrate.get_metrics(instance["name"])
if metrics.cpu_percent > 80:
print(f"⚠️ High CPU: {instance['name']}")
# 📌 Test compilation
dev-env/bin/python factory.deploy/ObjCompile.py test
# 📌 Test Proxmox connection
dev-env/bin/python factory.deploy/ObjProxmox.py list
# ❤️ Test health check
dev-env/bin/python -c "
from factory.deploy.ObjProxmox import ObjProxmox
px = ObjProxmox()
print(px.health_check())
"
ObjSubstratesubstrate_config.yamlExample:
from ObjSubstrate import ObjSubstrate, ResourceUsage
class ObjMySubstrate(ObjSubstrate):
def connect(self) -> bool:
# Implementation
pass
def health_check(self) -> Dict[str, Any]:
# Implementation
pass
# ... implement all abstract methods
When contributing to factory.deploy:
# 🏗️ Clean and rebuild
dev-env/bin/python factory.deploy/ObjCompile.py clean
dev-env/bin/python factory.deploy/ObjCompile.py all
# 📌 Check connection
if not substrate.ensure_connected():
health = substrate.health_check()
print(health["issues"])
# 📌 Fix PEM permissions
find data.code -name "*.pem" -exec chmod 600 {} \;
# 📌 Fix git log permissions
sudo chown -R $USER:$USER .git/
For issues, questions, or contributions:
substrate_config.yamlDO_DEBUG = True in module