NOTICE: All information contained herein is, and remains
the property of TechnoCore.
The intellectual and technical concepts contained
herein are proprietary to TechnoCore and dissemination of this information or reproduction of this material
is strictly forbidden unless prior written permission is obtained
from TechnoCore.
IMAP email scanner with lifecycle tracking, workflow integration, and performance-optimised batch processing.
IMAP Server
|
v
ObjImapSet.Scan() ← ObjRecurringMixin schedules
|
v
ObjImap.scan_folder()
|-- SEARCH (UIDs)
|-- Prefetch existing IDs (1 query)
|-- Fetch + cache headers (batch 50, bloom_imap_headers)
|-- Sender filter (in-memory via def_imap_senders)
|-- Batch fetch RFC822 (only passing emails)
|-- _parse_mail_raw() per email
| |-- INSERT into bloom_imap_{code}
| |-- Status: NEW → PROCESSING → DONE/ERROR
| |-- _trigger_workflow() (async via engine)
|-- Publish MQTT metrics
| Table | Purpose |
|---|---|
def_imap |
Hook configuration (scan rules) |
def_imap_senders |
Sender allow/ignore rules |
bloom_imap_headers |
Cached email headers (PK: ImapId, ImapCode, Package) |
bloom_imap_{code} |
Processed email content per hook |
stage_imap |
Execution audit trail |
IMAP credentials in config.yaml. Supports multiple named mailboxes:
imap:
mailboxes:
default:
server: imap.gmail.com
port: 993
username: alerts@domain.com
apppassword: xxxx xxxx xxxx xxxx
support:
source: google_group
server: imap.gmail.com
port: 993
username: support@domain.com
group_email: support@domain.com
auth_method: xoauth2
read_only: true
query: "newer_than:{timeslip}d (to:{group_email} OR listid:support.domain.com)"
oauth:
client_id: GOOGLE_CLIENT_ID
client_secret: GOOGLE_CLIENT_SECRET
refresh_token: GOOGLE_REFRESH_TOKEN
token_uri: https://oauth2.googleapis.com/token
scopes:
- https://mail.google.com/
For Gmail IMAP, OAuth authentication uses XOAUTH2.
Gmail's IMAP OAuth scope is https://mail.google.com/,
so Axion enforces a read-only view operationally by opening
folders with IMAP EXAMINE and suppressing move/delete
operations when read_only: true or auth_method: xoauth2
is configured.
For Google Groups, use source: google_group. Google does not
provide a supported API that reads the Groups conversation archive
directly as a mailbox. Axion uses the Gmail API read-only scope
against a group member or delegated Workspace user and filters for
messages delivered to the group. Messages are staged into the same
bloom_imap_{code} tables as IMAP messages, so ticket creation and
thread matching behave the same way.
Hooks reference a mailbox by name via def_imap.Mailbox. If unset or empty, uses default. Falls back to flat config (legacy) when no mailboxes section exists:
imap:
server: imap.gmail.com
username: user@domain.com
apppassword: xxxx xxxx xxxx xxxx
| Field | Description |
|---|---|
| ImapCode | Unique hook identifier |
| Package | Package scope (hook only visible to this package) |
| IsActive | Y/N — enable/disable |
| SearchSubject | Y/N — filter by subject |
| RegexSubject | Subject match string |
| SearchBody | Y/N — filter by body |
| RegexContent | Body match string |
| Folder | IMAP folder (default: INBOX) |
| Timeslip | Lookback window in days |
| DoDelete | Y/N — move to Processed after scan |
| RunFrequency | Seconds between scans |
| Workflow | Workflow code to trigger per email |
| Mailbox | Named mailbox from config (default: default) |
| PreSql / PostSql | SQL to run before/after scan |
NEW → PROCESSING → DONE
→ ERROR (retry possible)
Status tracked in bloom_imap_{code}.Status column.
Rules in def_imap_senders:
Filtering happens after header fetch, before body download.
SELECT IN(...) skips all processed emailsbloom_imap_headers stores Subject/From/Date; only fetches headers (not body) for new emailsdef_workflow.Async='Y' for non-blocking executionpython ObjImap.py scan [--imap-code %] # Run scan for hooks
python ObjImap.py check-due # Recurring schedule check
python ObjImap.py check-auth [--mailbox X] # Verify IMAP credentials
python ObjImap.py stats [--imap-code X] # Email processing stats
python ObjImap.py tui # Live monitoring dashboard
python ObjImap.py list-headings [--folder] [--mailbox X]
python ObjImap.py process <code> # Process single hook
python ObjImap.py match-and-list <code> # Search and list matches
python ObjImap.py inspect-hook <code> # Show hook config
# Mailbox management
python ObjImap.py mailbox-list # List configured mailboxes
# Sender management
python ObjImap.py sender-list # List sender rules
python ObjImap.py sender-add <email> <action>
python ObjImap.py sender-remove <email>
# Hook YAML seed (export/import between environments)
python ObjImap.py hook-export # Export to local.documents/imap_hooks/
python ObjImap.py hook-import <file> # Import from YAML
When def_imap.Workflow is set, _trigger_workflow() fires after each email is processed with context:
| Key | Value |
|---|---|
| Param1 | Bloom GUID (e.g. UPTIME_V2_135635) |
| Param2 | ImapCode |
| Param3 | Email subject |
| ticket_incident_date | Email sent timestamp (YYYY-MM-DD HH:MM:SS) |
| data.subject | Subject |
| data.sent_on | Sent date |
| data.bloom_guid | Bloom GUID |
STAGE_TABLE = "stage_imap"
BLOOM_PREFIX = "bloom_imap_"
HEADERS_TABLE = "bloom_imap_headers"
PROCESSED_FOLDER = "Processed"
CONFIG_SECTION = "imap"
DEFAULT_FOLDER = "inbox"
DEFAULT_LOOKBACK_DAYS = 7
DEFAULT_TIMESLIP_DAYS = 8
class ImapStatus(StrEnum):
NEW, PROCESSING, DONE, ERROR
class ImapSenderAction(StrEnum):
IGNORE, ALLOW
Emitted via MqttEventType.EMAIL_METRICS after each scan (the ImapCode is the
event_state). All payload data is passed via named context keys rather than
positional param1-9 slots or a hand-built axion/imap/... topic:
{
"emails_found": 245,
"emails_processed": 3,
"emails_skipped": 240,
"emails_filtered": 2,
"emails_errored": 0,
"duration_seconds": 4.21
}
Per-email receipt is emitted via MqttEventType.EMAIL_RECEIVED with
context = {"folder", "imap_id", "imap_code", "sender", "subject", "size"}.
python ObjImap.py tui — live terminal dashboard showing: