Unified XLSX Importer

Added in version 1.5.0.

The UnifiedXLSXImporter reads a single em_data.xlsx workbook produced by StratiMiner (or hand-authored against the shipped template at s3dgraphy/templates/em_data_template.xlsx) and builds a complete s3dgraphy Graph in memory: stratigraphic units, epochs, qualia, relations, and the full paradata chain (PropertyNode → ExtractorNode → DocumentNode, with AuthorNode / AuthorAINode attached to the extractor or directly to the PropertyNode).

It is the default workflow for s3dgraphy 1.5 and supersedes the legacy two-file pipeline (MappedXLSXImporter + QualiaImporter). Use the unified importer for any new project.

See also

Unified XLSX Exporter for the inverse round-trip.

The 5-sheet canonical schema

em_data.xlsx ships five typed sheets. The importer fails fast if any is missing.

Units

Stratigraphic-unit declarations — the skeleton.

Column

Meaning

ID

Unit identifier (e.g. US001, USV014). Used as edge_source / edge_target in Claims rows.

TYPE

Stratigraphic abbreviation (US, USVs, USVn, SF, VSF, USD, TSU, RSF, USN, UL, serSU, serUSD, serUSVn, serUSVs). Mapped via s3dgraphy.utils.utils.STRATIGRAPHIC_CLASS_MAP.

NAME

Free-text label rendered in graph editors.

Aliases

The importer normalises common AI-agent drift on load: UNIT_ID / UNITS_ID → ID; UNIT_TYPE → TYPE.

Epochs

Swimlane / chronology declarations.

Column

Meaning

ID

Epoch identifier (e.g. EP_ROMAN, EP_MEDIEVAL). Referenced from Claims rows.

NAME

Display label.

START

Earliest TPQ (signed integer year — -44 for 44 BCE). Optional.

END

Latest TAQ (signed integer year). Optional.

COLOR

Hex colour for yEd swimlane rendering. Optional.

Claims

The heart of the workbook: one row per asserted fact. Each row declares (a) a claim target (a unit, possibly with a secondary target for relational claims), (b) the property/relation type, (c) the value, and (d) up to two attribution triples (EXTRACTOR_i / DOCUMENT_i / AUTHOR_i / AUTHOR_KIND_i).

Column

Meaning

TARGET_ID

Unit (or epoch) the claim is about.

TARGET2_ID

Secondary target for relational claims (e.g. X is_after Y).

PROPERTY_TYPE

Semantic class of the claim (see Dispatch below).

VALUE

Claim payload (string or numeric).

UNITS

Unit of measure for numeric qualia (cm, m, …). Optional.

COMBINER_REASONING

Free-text justification for multi-source (Combiner) claims. Triggers Combiner insertion when both extractor slots are populated.

EXTRACTOR_1 /

Free text describing what the AI

EXTRACTOR_2

extractor (or human reasoning step) concluded from the source document.

DOCUMENT_1 /

Document catalog ID

DOCUMENT_2

(e.g. D.01) the claim was extracted from.

AUTHOR_1 /

Authors catalog ID

AUTHOR_2

(e.g. A.01 or AI.01).

AUTHOR_KIND_1 /

"author" (transcribed from the

AUTHOR_KIND_2

document author — the claim is already in the source) or "extractor" (newly derived by an AI tool — the claim is new).

Dispatch

The importer dispatches a Claims row by the semantic class of PROPERTY_TYPE:

  • Scalar qualia (definition, material_type, length, width, height, shape, conservation_state, comparanda, interpretation, …) — create a PropertyNode attached to TARGET_ID via has_property. VALUE is the claim content.

  • Temporal qualia (absolute_time_start, absolute_time_end) — same as scalar but the PropertyNode’s property_type is set so the DP-32 resolver and A.1 compaction pick it up. See Temporal Inference Engine.

  • Epoch membership (has_first_epoch / belongs_to_epoch) — create a has_first_epoch edge from TARGET_ID (unit) to the Epochs row identified by VALUE or TARGET2_ID. The deprecated alias belongs_to_epoch is accepted for backward compat with prompt versions ≤ v5.2.

  • Stratigraphic relation (is_after, overlies, cuts, fills, abuts, bonded_to, equals, has_same_time, contrasts_with, changed_from, plus their reverses) — create a directed edge from TARGET_ID to TARGET2_ID with that edge_type. The attribution chain hangs off the edge’s attributes dict (authored_by_id, authored_by_kind, document_id) rather than creating PropertyNodes — edges are the natural subject of a relational claim’s attribution.

Authors

Normalized catalog of claim authors.

Column

Meaning

ID

Short code (A.01, AI.01).

KIND

"human" → AuthorNode; "ai" → AuthorAINode.

DISPLAY_NAME

Human label.

ORCID

ORCID identifier (optional).

AFFILIATION

Institution (optional).

The Authors catalog feeds the has_author resolution: per-claim attribution triples reference rows by ID; the importer reuses one AuthorNode per ID, avoiding catalog duplication.

Documents

Normalized catalog of source PDFs / reports.

Column

Meaning

ID

Short code (D.01).

FILENAME

On-disk filename.

TITLE

Document title.

YEAR

Year of publication.

AUTHOR_IDS

Comma-separated list of Authors.ID values (semantic only, no FK).

Three-axis document classification (ROLE / CONTENT_NATURE / GEOMETRY) is accepted on input under the 1.6 nomenclature but is not required by the 1.5 importer.

Combiner row pattern

When both EXTRACTOR_1 / EXTRACTOR_2 and COMBINER_REASONING are populated, the importer inserts a CombinerNode between the PropertyNode and the two ExtractorNode instances:

PropertyNode  --has_data_provenance-->  CombinerNode
                                            |
                          combines / combines
                                            |
                          ExtractorNode_1, ExtractorNode_2

This is the canonical pattern for Combiner-mediated synthesis (one qualia value supported by two independent extractions). The COMBINER_REASONING text is stored on CombinerNode.description.

Workflow

from s3dgraphy.importer.unified_xlsx_importer import UnifiedXLSXImporter

importer = UnifiedXLSXImporter(
    "em_data.xlsx",
    graph_id="great_temple",
)
graph = importer.parse()

print(f"Loaded {len(graph.nodes)} nodes, {len(graph.edges)} edges")
print("Per-sheet stats:", importer.stats)
for w in graph.warnings:
    print(f"  ! {w}")

Public attributes after parse():

  • importer.graph — populated Graph (also the return value).

  • importer.warnings — non-fatal issues (unknown references, duplicate declarations, …). Mirrored into graph.warnings for downstream consumers.

  • importer.stats — per-sheet row counts plus nodes_total / edges_total.

Mapping registry interaction

Unlike MappedXLSXImporter, the unified importer is schema-fixed — it does not consult mapping_registry. The 5-sheet schema is the mapping. Use MappedXLSXImporter only for legacy free-form spreadsheets that pre-date the unified workbook; new projects should emit em_data.xlsx directly.

API Reference

UnifiedXLSXImporter — single-file xlsx importer for the Extended Matrix.

Reads a em_data.xlsx file produced by StratiMiner (DP-02) and builds a complete s3dgraphy Graph in memory: stratigraphic units, epochs, qualia, relations, and the full paradata chain (PropertyNode → ExtractorNode → DocumentNode, with AuthorNode / AuthorAINode attached to the extractor or directly to the PropertyNode).

Unlike the legacy two-file pipeline (MappedXLSXImporter + QualiaImporter), this importer consumes a single workbook with five typed sheets:

  1. Units — stratigraphic unit declarations (ID, TYPE, NAME).

  2. Epochs — swimlane declarations (ID, NAME, START, END, COLOR).

  3. Claims — long-table, one row per claim about a unit/epoch:

    qualia values, epoch assignment, stratigraphic relations, each with its own attribution.

  4. Authors — normalized catalog of claim authors (human A.xx or

    AI AI.xx).

  5. Documents — normalized catalog of source PDFs/reports.

The schema and semantics are described in the template s3dgraphy/templates/em_data_template.xlsx.

Claim row semantics

Each Claims row is one of:

  • Scalar qualia (PROPERTY_TYPE ∈ {definition, material_type, length, width, height, shape, conservation_state, comparanda, interpretation, …}): creates a PropertyNode attached to TARGET_ID via has_property. VALUE is the qualia content.

  • Temporal qualia (PROPERTY_TYPE ∈ {absolute_time_start, absolute_time_end}): same as scalar, but the PropertyNode’s property_type is set so the DP-32 resolver and A.1 compaction pick it up.

  • Epoch membership (PROPERTY_TYPE = has_first_epoch, or the deprecated alias belongs_to_epoch — kept for backward compat with prompt ≤ v5.2): creates a has_first_epoch edge from TARGET_ID (unit) to the Epochs row identified by VALUE or TARGET2_ID.

  • Stratigraphic relation (PROPERTY_TYPE ∈ _RELATION_TYPES): creates a directed edge from TARGET_ID to TARGET2_ID with that edge_type.

Attribution

For every claim row, one or two attribution triples (EXTRACTOR_i, DOCUMENT_i, AUTHOR_i) may be present:

  • If EXTRACTOR_i populated: create an ExtractorNode with that text, link ExtractorNode → extracted_from → DocumentNode and PropertyNode → has_data_provenance → ExtractorNode. If AUTHOR_i is also populated, link ExtractorNode → has_author → AuthorNode/AuthorAINode (subclass chosen from AUTHOR_KIND_i and the Authors sheet).

  • If EXTRACTOR_i is empty but AUTHOR_i is populated: attach has_author directly to the PropertyNode (claim stated without an AI extraction step — e.g. the user declared it themselves).

  • If both extractor_1 and extractor_2 populated AND COMBINER_REASONING present: create a CombinerNode between the PropertyNode and the two extractors (PropertyNode → has_data_provenance → CombinerNode, CombinerNode → combines → ExtractorNode_i).

For relational claims the extractor chain hangs off the edge’s attributes dict (authored_by_id / authored_by_kind / document_id) rather than creating nodes — edges are the natural subject of a relational claim’s attribution.

class s3dgraphy.importer.unified_xlsx_importer.UnifiedXLSXImporter(filepath: str, graph_id: str | None = None)[source]

Bases: object

Import a unified em_data.xlsx file into a fresh s3dgraphy Graph.

Usage:

importer = UnifiedXLSXImporter("em_data.xlsx", graph_id="great_temple")
graph = importer.parse()

After parse():

  • importer.graph is the populated Graph (also returned).

  • importer.warnings collects non-fatal issues (unknown references, duplicate declarations, …). They are also appended to graph.warnings.

  • importer.stats reports how many rows of each sheet were read and how many nodes/edges were created.

__init__(filepath: str, graph_id: str | None = None)[source]
parse() → Graph[source]