NOTICE: All information contained herein is, and remains
the property of TechnoCore Automate.
- python3 ObjDataConfig.py EXPORTCORE
- To be used to update the coredb.axion logic - for example if changes have been made to sys_, def_, internet_, meta_, list_, stage_, track_ and sysdate_ tables
- NOTE the server ashpool.co.za is hardcoded as the only legitimate source for this.
- The steps are:
** export_core_tables_to_yaml(): get the structure for all tables in coredb.axion (this would have all the sys_, def_, internet_, meta_, list_, stage_, track_ and system data tables like data_go, etc. (later these should be changed to sysdata)) and write that to yaml files in the resource.schema/package.coredb/tables folder
** export_coredata_to_config(): get data from all def_ tables that have package or block equal to 'CORE', 'SYSTEM' or 'BASE' and write that to csv files in the resource.schema/package.coredb/data folder
-
python3 ObjDataConfig.py COLDSTART
- Cold start will
** read from the config.yaml file and pull the active package and the database connection info for that package
** create a new database schema for that package (e.g. xyz would create a new schema xyzdb.axion)
** look in resource.schema/package.core/tables folder and create a set of CREATE TABLE statements in data_test/create_tables folder
** look in resource.schema/package.core/data folder and create a set of REPLACE INTO statements in data_test/replace_data folder
** Run through the create_tables and replace_data folders and run them against the DB to create the tables and populate them with the core data
** Manually populate
*** the def_package table using the config.yaml information
*** create the PrimaryDB record in the Def_RemoteConnections table
*** IF the role in config.yaml is DEV: Create 3 users (Francois, Michael, Paul) in the sys_user table
-
python3 ObjDataConfig.py EXPORT: Export from a package all the required system files to get a clone setup without client data
- def ExportCoreTablesToConfig():
** Load the structure of all system tables (def, internet, list, sys, data, meta, stage, track) into YAML files
** target is resource.schema/package.[package name]/tables/[table_name].yaml
- def ExportCoreDataToConfig():
** Load the data of all system tables WITH data (def (excl def_remote_connections), internet, list, sys (excl sys_user) and excl all checkout/delete/archive versions) into YAML files
** target is resource.schema/package.[package name]/data/[table_name].yaml
-
python3 ObjDataConfig.py COMPARE: Run a compare of that against the latest git pushed yaml files from the master and create files that can be run to implement any changes in the new db
- Create a _test DB for the package to be tested (e.g. referencedb.axion.test)
- Load the structures and data from the YAML files into the new database
** craft a create table statement from the yaml file
- Compare the table names
** use show tables (e.g. show tables from referencedb.axion) to get list of tables from test and target dbs as set(test_tables), set(target_tables)
** use sets to see what tables are in the test that are not in the target ( set(test_tables) - set(target_tables) )
** create a script that will create all of those tables in the target that are missing (create table commands)
*** target is table_changes.sql
- For each table that is in both databases ( set(test_tables).union(set(target_tables)) ), compare the tables structure
** create sets of the names of the columns in each table from test and target
*** use [SHOW COLUMNS FROM] to get a tuple of the base column information. Then use [SHOW INDEX] to add to the tuple whether the column is a PRIMARY, UNIQUE or normal KEY, and add a column with the key_name
*** set(target column names) - set(test column names) would show if the target has columns not in the new test db - this should be highlighted for manual intervention
**** target is column_warnings.sql
*** set(test column names) - set(target column names) shows the test has new columns - provide recommendations ito alter table add column statements file
**** target is column_additions.sql
** create sets of tuples of the column information (use describe table) excluding any columns that were created in the previous step
*** set(test column tuple) - set(target column tuple)
-
python3 ObjDataConfig.py APPLY: Run the accepted changes to get the new DB into the same system state as the master (i.e. latest git pushed yaml files)
- Run the scripts that have been created in the previous steps
cythonize -3 -a -i ObjDataConfig.py
warning: ObjDataConfig.py:395:53: Tuples cannot be declared as simple tuples of types. Use 'tuple[type1, type2, ...]'..Compiling /home/axion/projects/axion/factory.core/ObjDataConfig.py because it changed..[1/1] Cythonizing /home/axion/projects/axion/factory.core/ObjDataConfig.py.
Updated : 2024-09-09