NOTICE: All information contained herein is, and remains
the property of TechnoCore.
The intellectual and technical concepts contained
herein are proprietary to TechnoCore and dissemination of this information or reproduction of this material
is strictly forbidden unless prior written permission is obtained
from TechnoCore.
The ObjDataStore object is a comprehensive data management and schema maintenance tool. It is designed to be run as a service or a CLI tool to perform regular cleanup, refactoring, and organization of the database and related document storage.
def_* tables to identify and fix schema inconsistencies. This includes adding missing primary keys and ensuring that a package column is present to standardize the schema across the application.tablename_history_202510), keeping the operational tables lean and performant.local.documents directory to a long-term storage location (data.documents) based on Time-To-Live (TTL) settings defined in the def_document table.The behavior of ObjDataStore is driven by two main configuration sources:
config.yaml: The datastore section of the main config.yaml file contains lists of tables to be considered for trim, history, and redacted operations.ObjDataStore.yaml: This file, located in factory.core/, contains all the SQL queries used by the class. This separation of code and SQL makes the queries easier to manage and modify.The script provides a powerful CLI for executing specific data management tasks.
historyThis is the main, all-in-one command that runs a sequence of data maintenance operations:
def_* tables to ensure schema consistency.Upon completion, it displays a summary of its work in a rich table format, showing the source and history tables and their respective sizes.
Usage:
python3 factory.core/ObjDataStore.py history
validateThis command checks the def_* tables for schema compliance without making any changes. It is useful for validating the database schema against the application's expectations.
Usage:
python3 factory.core/ObjDataStore.py validate
move-documentsThis command triggers the process to move files from local.documents to data.documents based on the TTLs defined in the def_document table.
Usage:
python3 factory.core/ObjDataStore.py move-documents
history-check & truncate-checkThese commands check if there is any historical data that needs to be moved or truncated, respectively. They print a "PASSED" or "FAILED" message to the console, making them useful for monitoring and automated scripting.
Usage:
python3 factory.core/ObjDataStore.py history-check
python3 factory.core/ObjDataStore.py truncate-check