¶ (c) Technocore - All Rights Reserved.
NOTICE: All information contained herein is, and remains
the property of TechnoCore.
The intellectual and technical concepts contained
herein are proprietary to TechnoCore and dissemination of this information or reproduction of this material
is strictly forbidden unless prior written permission is obtained
from TechnoCore.
¶ ObjWorkflowSimul
¶ Overview
The WorkflowSimul class extends the base Workflow class to provide a comprehensive simulation environment for testing, performance analysis, and validation of workflows. It is designed to run a defined workflow against a set of test data, either in-process ('direct' mode) or by simulating API calls ('api' mode).
This class leverages multi-threading to execute simulations in parallel, allowing for efficient testing of high-volume scenarios. Key features include preserving the initial state of data, tracking execution paths, and generating summaries to visualize workflow behavior under test conditions.
¶ Key Methods
¶ run_simulation_mode(...)
Orchestrates and executes a multi-threaded workflow simulation. This is the main entry point for running a simulation. It handles all setup, including preparing the database, fetching the list of GUIDs for the run, and populating the thread-safe queue. It then spawns a pool of worker threads to process the GUIDs in parallel and waits for their completion.
Parameters (notable):
| Parameter | Type | Default | Description |
|---|---|---|---|
workflow_name |
str |
— | Workflow to simulate |
simulation_size |
int |
— | Maximum GUIDs to process |
mode |
str |
"direct" |
"direct" or "api" |
report_webhook |
str |
"" |
Webhook code for post-simulation bloom table generation (see below) |
Module-level flags:
| Flag | Default | Purpose |
|---|---|---|
DO_DEBUG |
False |
Enable verbose debug output |
DO_TESTING |
False |
Skip post-simulation AI analysis and email (for pipeline testing) |
When DO_TESTING = True, the post-simulation steps (bloom table,
AI review, report generation, email) are all bypassed. The
simulation still runs and exports FinalOutputTables.
Post-simulation steps (in order, skipped when DO_TESTING = True):
- Display table-tracking summary (SIM deployment only)
- Benchmark metrics collected
_build_webhook_bloom_table()— usingdef_workflow.ReportWebhook(CLI--report-webhookoverrides)_run_ai_workflow_review()— ifdef_workflow.SimReviewPromptis set_run_ai_outcome_review()— ifbloom_tableis populated anddef_workflow.SimOutcomePromptis set_run_ai_interpretation()— ifbloom_tableis populated anddef_workflow.SimPromptis set_generate_simulation_report()— markdown report written tolocal.documents/simulations/_generate_email_html()— styled HTML email from report_send_simulation_email()— todef_workflow.SimNotifyEmailnotify.Read("MSG_SIMUL_END")
¶ _run_ai_workflow_review(workflow_name, package, sim_review_prompt)
Feeds the workflow's structural definition to Ollama and returns a review.
Data collected from the database and passed to the model:
| Source | Fields |
|---|---|
def_workflow |
Description, ExecutionMode, Async, Active, Version, Module |
def_workflows |
Rank, Type, Name, BranchDirect / BranchSql, Description (all active steps, ordered) |
def_webhook via def_webhook_workflow_map |
WebhookCode, Direction, Type, Method, URL, PayloadType, Notes |
def_service (SERVICE-type steps only) |
ServiceCode, ServiceType, Description, DataTable, FinalOutputTable, Result columns |
Long SQL fields are truncated to 200 characters in the context to keep the prompt within model context limits.
Returns the model's response, or "" on failure. Failures are logged as note.
¶ SimReviewPrompt field
def_workflow.SimReviewPrompt (TEXT) — free-text prompt guiding the AI workflow review. Added automatically via add_column in development deployment. See the Reference Workflow: HCSCORE section for the full production prompt.
¶ _run_ai_outcome_review(bloom_table, simul_guid, sim_outcome_prompt)
Analyses the actual outcome rows written into data_bloom_simul_{DataTable} for the current simulation run.
Data fed to the model:
| Source | Detail |
|---|---|
| Column metadata | Distinct value count and blank/null count per column |
| Top-10 distributions | Per-column value frequency, filtered to the current SimulationGuid |
| Sample rows | Up to 50 raw rows (all columns) formatted as a markdown table |
Returns the model's response, or "" on failure. Failures are logged as note.
The three AI prompts are complementary:
| Field | Type | Focus |
|---|---|---|
ReportWebhook |
VARCHAR(255) |
Webhook whose HookLookupSql populates data_bloom_simul_{DataTable}; CLI --report-webhook overrides |
SimOutcomePrompt |
TEXT |
What did the workflow actually produce? Patterns, anomalies, outliers in the output data |
SimPrompt |
TEXT |
How did the simulation perform? Distribution statistics, coverage, quality |
SimReviewPrompt |
TEXT |
Is the workflow well-designed? Structure, completeness, dependencies |
¶ SimOutcomePrompt field
def_workflow.SimOutcomePrompt (TEXT) — prompt focused on interpreting what the workflow computed for each GUID. See the Reference Workflow: HCSCORE section for the full production prompt.
¶ _run_ai_interpretation(bloom_table, simul_guid, sim_prompt)
Uses ObjAI (Ollama/mistral) to interpret the simulation results stored in the bloom table.
- Reads column names from
bloom_tableviasql_get_normalized_headings. - Generates on-the-fly summary queries — row count + top-10 value distributions per column — filtered to the current
simul_guid. - Formats the summaries as structured text and appends them to
sim_prompt(loaded fromdef_workflow.SimPrompt). - Calls
ObjAI(model="mcp:ollama:mistral").prompt()with an analyst role. - Returns the model's response text, or
""on any failure.
Only runs when both bloom_table and sim_prompt are non-empty. Failures are logged as note — the simulation report is still written without the AI section.
¶ SimPrompt field
def_workflow.SimPrompt (TEXT) — free-text prompt/question that guides the AI interpretation for a specific workflow's simulation. Added automatically via add_column in development deployment. Example:
You are analysing the results of a HomeChoice HCSCORE credit-scoring simulation ...
See the Reference Workflow: HCSCORE section for the full production prompt.
¶ _build_webhook_bloom_table(guids, report_webhook)
Runs the webhook's HookLookupSql for every simulation GUID and stores results in
data_bloom_simul_{DataTable}.
- Looks up
HookLookupSqlandDataTablefromdef_webhookfor the givenWebhookCode. - Creates
data_bloom_simul_{DataTable}if it does not exist (singleGuidPK column). - Runs the
HookLookupSqlper GUID by substituting the$guid$placeholder. - Dynamically adds columns (
mediumtext) discovered from the result rows viaadd_column. - In simulation deployment mode,
add_columnfailures are logged asnoterather thanerror. - Returns the bloom table name, or
""when skipped or on failure.
¶ _generate_simulation_report(...)
Generates a markdown simulation report and writes it to local.documents/simulations/.
Report sections: overview table, GUID selection SQL, outcome distribution, performance metrics,
errors (capped at 50), and — if a bloom table was generated — the bloom table name.
Returns the file path, or "" on failure.
¶ _generate_email_html(...)
Converts the markdown simulation report into a styled HTML file for email delivery. Saves the
result as a .html file alongside the .md file and returns the HTML file path.
- Reads the
.mdreport, strips the fenced text-chart block, and converts the remainder to HTML
viamarkdown.markdown()withtablesandfenced_codeextensions. - Applies palette-based inline styles to all content elements (
<h1>,<h2>,<h3>,<th>,
<td>,<table>,<pre>,<p>). - Injects coloured HTML bar charts (from
_build_bloom_charts_html) before the
Workflow Reviewsection.
Brevo SMTP inline-CSS rules — Brevo's HTML sanitizer applies a strict CSS whitelist to
heading elements (<h1>, <h2>, <h3>) and truncates the style attribute after its
whitelist is exhausted, leaking the remaining CSS text as visible content. Rules that must be
followed to avoid this:
| Rule | Reason |
|---|---|
Only color:{value} on heading elements |
Brevo whitelist for <h1>/<h2>/<h3> is very narrow; any additional property leaks as text |
No comma-separated font-family values |
Brevo treats commas in CSS values as separators — font-family:Verdana,Arial leaks Arial as text |
<th> and <td> elements accept full inline CSS |
Brevo does not apply the same restriction to table-cell elements |
No max-width or margin:auto |
These are outside Brevo's supported inline CSS set for most elements |
¶ _send_simulation_email(...)
Sends the generated HTML report as an email. Reads SMTP credentials from config.yaml under
the smtp section (server, port, username, password, sender). Recipient is read from
def_workflow.SimNotifyEmail.
Important: uses MIMEText(html, "html", "utf-8") and calls mailer.send() directly —
not WebMail.send_html(). send_html wraps the content in MIMEMultipart("related"),
which Brevo SMTP collapses to text/html during relay and corrupts inline CSS in the process.
Bypassing the multipart wrapper ensures the HTML is delivered intact.
¶ _build_bloom_charts_html(...)
Generates coloured HTML horizontal bar charts for each analytical column in the bloom table.
Uses the active package palette (colour_1…colour_N) for bar colours. Columns whose average
value length exceeds 60 characters are skipped (raw/encoded data). Returns an HTML <div>
string, or "" on failure.
¶ run_simulation(...)
A convenience method that runs a simulation in 'direct' mode. Accepts an optional
report_webhook parameter that is forwarded to run_simulation_mode.
¶ run_simulationapi(...)
A convenience method that runs a simulation in 'api' mode. This method first determines the correct API endpoint for the workflow before initiating the simulation.
¶ simulation_summary(...)
Analyzes and summarizes the results of a simulation run. This method aggregates the transition data from a completed simulation to identify the most common paths taken through the workflow. It then generates a Mermaid flowchart diagram where the thickness of the connecting lines represents the frequency of a path, providing a clear visual representation of the workflow's behavior.
¶ preserve_input(...)
Saves the initial state of data before it is modified by a simulation. To ensure that simulations are non-destructive and repeatable, this method creates a copy of the relevant input data in a separate table with a _simul suffix.
¶ Simulation SQL Configuration in def_workflow
A key aspect of the simulation is controlled by several SQL fields in the def_workflow table. These fields allow for powerful and flexible control over the simulation's data selection, setup, teardown, and validation.
¶ simulation_sql
This field contains the SQL query responsible for selecting the GUIDs to be processed in the simulation. The get_simulation_guids method retrieves this query for the specified workflow.
- Purpose: To define the dataset for the simulation run.
- Expected Output: A list of GUIDs.
¶ simulation_presql
This field contains one or more SQL statements that are executed before the main simulation loop begins.
- Purpose: To set up the database environment for the simulation. This can include creating temporary tables, inserting fixture data, or updating records to a known state.
- Execution: Executed once at the beginning of
run_simulation_mode. Multiple statements can be separated by a semicolon (;).
¶ simulation_postsql
This field contains one or more SQL statements that are executed after the main simulation loop has completed.
- Purpose: To clean up any artifacts created by the pre-sql script or the simulation itself. This can include dropping temporary tables or deleting test data.
- Execution: Executed once at the end of
run_simulation_mode. Note: This functionality is planned and not yet implemented.
¶ simulation_validation_sql
This field contains an SQL query that is executed after each individual GUID is processed.
- Purpose: To validate the outcome of the workflow for a single record. This allows for immediate feedback on whether the workflow produced the expected result.
- Expected Output: The query should return
1for success and any other value for failure. The placeholder$guid$can be used in the query and will be replaced with the GUID of the record that was just processed. - Execution: Executed at the end of
_process_guid_for_simulationfor each GUID.
¶ Reference Workflow: HCSCORE
HCSCORE is the reference workflow for simulation configuration. It is a multi-bureau
customer credit scoring process that routes applicants through Experian (RSA, LES, NAM, ESW),
selects a scorecard (PayingSeg / NonPayingSeg / ThinFile / NoReturn), applies credit limits,
and returns a risk rating. All inbound channels (AO, CR, WA) normalise into bloom_hcscore
before triggering this workflow.
¶ def_workflow configuration
| Field | Value |
|---|---|
| WorkflowName | HCSCORE |
| simulation_sql | SELECT _guid FROM bloom_hcscore WHERE Customersource = 'SIMULATION' LIMIT 100 |
| simulation_presql | UPDATE bloom_hcscore SET ScorecardNo = NULL, HC_Rating = NULL WHERE Customersource = 'SIMULATION' |
| simulation_validation_sql | SELECT CASE WHEN ScorecardNo IS NOT NULL AND HC_Rating IS NOT NULL THEN 1 ELSE 0 END FROM bloom_hcscore WHERE _guid = '$guid$' |
| ReportWebhook | HCSCORE — HookLookupSql reads scoring output from data_bloom_hcscore, stored in data_bloom_simul_hcscore |
¶ ReportWebhook
UPDATE def_workflow
SET ReportWebhook = 'HCSCORE'
WHERE WorkflowName = 'HCSCORE' AND Package = 'HOMECHOICE';
¶ SimOutcomePrompt
UPDATE def_workflow
SET SimOutcomePrompt = '...'
WHERE WorkflowName = 'HCSCORE' AND Package = 'HOMECHOICE';
Prompt text:
You are a credit risk analyst reviewing the output of a HomeChoice HCSCORE
workflow simulation. The data below comes from the data_bloom_simul_hcscore table
and represents the actual scoring decisions produced for each applicant GUID.
Analyse the outcome data and provide a structured review covering:
1. DECISION QUALITY
- What proportion of applicants received a valid score vs. no-score outcomes?
- Are the HC_Rating and ScorecardNo values consistent with the expected
scorecard logic (PayingSeg A–C, NonPayingSeg D–E, ThinFile, NoReturn)?
- Identify any combinations of HC_Rating and ScorecardNo that appear
contradictory or unexpected.
2. CREDIT LIMIT DISTRIBUTION
- Describe the spread of CreditLimit values across the population.
- Identify any concentration at specific limit thresholds (e.g. R0, R500,
R1000, R2500) that may indicate hard-coded limits or policy overrides.
- Flag applicants where CreditLimit is zero despite a non-null HC_Rating.
3. SCORECARD COVERAGE
- Which scorecard (PayingSeg / NonPayingSeg / ThinFile / NoReturn) was
applied to the most and least applicants?
- Are there GUIDs that appear to have missed scorecard assignment
(ScorecardNo NULL or unexpected value)?
4. EXCLUSION AND NO-SCORE ANALYSIS
- List the top exclusion reason codes and their frequencies.
- Are there patterns in which applicant attributes (Applicationchannel,
Department, Ctryno) correlate with exclusion?
- Estimate the business impact: what credit limit total was withheld due
to no-score outcomes?
5. ANOMALIES AND DATA QUALITY
- Identify GUIDs with NULL or empty values in critical output fields
(HC_Rating, ScorecardNo, CreditLimit).
- Flag any duplicate GUIDs in the sample.
- Note any outcome values that fall outside expected ranges.
6. REPRESENTATIVE SAMPLE CHECK
- Does the sample of 50 rows reflect the overall distribution?
- Are there any obvious outliers in the sample that warrant investigation?
Present findings as a concise report with a summary table of key metrics
followed by detailed observations for each section.
¶ SimReviewPrompt
The SimReviewPrompt field drives the AI workflow review step. Apply with:
UPDATE def_workflow
SET SimReviewPrompt = '...'
WHERE WorkflowName = 'HCSCORE' AND Package = 'HOMECHOICE';
Prompt text:
You are a senior workflow architect and credit risk engineer reviewing the
HomeChoice HCSCORE credit-scoring workflow. The workflow definition below was
extracted directly from the production database (def_workflow, def_workflows,
def_webhook, def_service).
Please produce a structured review covering:
1. WORKFLOW STRUCTURE
- Summarise the overall flow: entry point → bureau routing → scorecard
selection → post-processing → output.
- Identify the main decision branches and what drives each branch.
- Comment on whether the step sequencing is logical and complete.
2. BRANCH COVERAGE
- List every named branch (BranchDirect values) and confirm each has a
corresponding step.
- Identify any dead-ends, unreachable branches, or missing fallback paths.
- Flag any steps with BranchSql that may produce NULL or empty results
and thus cause silent routing failures.
3. BUREAU INTEGRATION REVIEW
- Review the Experian integration steps (RSA, LES, NAM, ESW).
- Comment on whether error paths (no-return, invalid-country, recent-score)
are handled robustly.
- Assess whether the bureau timeout and retry strategy appears adequate.
4. SCORECARD SELECTION LOGIC
- Review the ScorecardSelection step and the downstream scorecard nodes
(PayingSeg, NonPayingSeg, ThinFile, NoReturn).
- Identify whether the selection criteria are clearly defined.
- Note any scorecard paths that appear under-specified.
5. SERVICE DEPENDENCIES
- List every referenced service and its role.
- Identify any services that are single points of failure.
- Comment on whether result column mappings (Result1Col, etc.) are complete.
6. WEBHOOK INTEGRATION
- Review the inbound webhook trigger(s) and outbound calls.
- Check that payload types, HTTP methods, and base URLs are consistent
with the expected data flow.
- Note any webhooks missing validation rules or post-SQL.
7. DATA INTEGRITY
- Identify any steps that read or write the same table (potential race
conditions with parallel instances).
- Comment on whether pre-SQL and post-SQL steps are appropriately guarded.
8. RECOMMENDATIONS
- List the top 3–5 improvement actions in priority order.
- For each, state the specific step or field that needs attention and the
risk if left unaddressed.
Present your findings as a structured technical report. Use the step names and
ranks from the definition when referencing specific nodes.
¶ SimPrompt
The SimPrompt field drives the AI interpretation step after the bloom table is populated.
Apply it with:
UPDATE def_workflow
SET SimPrompt = '...'
WHERE WorkflowName = ''HCSCORE'' AND Package = ''HOMECHOICE'';
Prompt text (copy verbatim into the SQL above):
You are analysing the results of a HomeChoice HCSCORE credit-scoring simulation.
The data comes from the post-simulation bloom table produced by the HCSCORE webhook
HookLookupSql. Each row represents one applicant processed through the workflow.
Please provide a structured analysis covering the following areas:
1. SCORECARD DISTRIBUTION
- Which scorecard was selected most often (PayingSeg, NonPayingSeg, ThinFile, NoReturn)?
- What does the distribution suggest about the composition of the applicant population?
- Flag any unexpected concentration in a single scorecard.
2. RISK RATING BREAKDOWN
- Summarise the HC_Rating distribution (A = lowest risk → E = highest risk).
- Identify whether the risk profile is skewed toward high-risk or low-risk bands.
- Comment on any risk bands with unusually high or low counts.
3. CREDIT LIMIT ANALYSIS
- Describe the distribution of CreditLimit values.
- Identify the proportion of applicants receiving a zero or near-zero limit.
- Highlight any concentration at specific limit thresholds.
4. BUREAU ROUTING
- Which bureau path was most frequently taken (RSA, LES, NAM, ESW, InvalidCountry, RecentScore)?
- Are there anomalies in country-code routing that warrant investigation?
5. NO-SCORE AND EXCLUSION ANALYSIS
- What proportion of applicants received no score (ThinFile, NoReturn, PostNoScore)?
- What are the leading exclusion reasons (ExclusionReason field)?
- Are there patterns linking specific channels (Applicationchannel) to higher no-score rates?
6. CHANNEL AND LEAD ORIGIN
- Compare outcomes across application channels (AO=1, WA/CR=2) and Leadorigin codes.
- Note whether any channel shows systematically different approval or risk-band rates.
7. ERRORS AND ANOMALIES
- Identify any ScorecardNo or HC_Rating values that are NULL or unexpected.
- Flag GUIDs where the workflow appears to have reached an error path (PostError node).
8. SIMULATION QUALITY
- Comment on whether the simulation GUID population appears representative
(e.g. balanced across channels, no obvious bias in the input data).
- Suggest any data quality or configuration issues surfaced by the simulation.
Present your findings as a concise executive summary followed by bullet-point detail for
each section. Where data is insufficient for a conclusion, say so explicitly.
¶ Command-Line Interface
The class provides a command-line interface using typer for easy execution of simulations.
simul: Runs a workflow simulation in 'direct' mode. Accepts--report-webhook <CODE>to generate a post-simulation bloom table from the webhook'sHookLookupSql.simulapi: Runs a workflow simulation in 'api' mode.summary: Generates and displays a summary of a simulation run.interactive: Runs a workflow simulation interactively using arich-based UI. This command prompts the user to select a workflow from a list (including a 'None' option to exit) and specify a simulation size (capped at 50,000).preflight: Sends a test notification to verify that theObjNotifysystem is working correctly.