Import Module Overview

The Import Module (implemented in pxm/importer) is a core component of the prodexa PXM platform. It acts as an ingestion engine for product, catalog, and classification data. It enables you to fetch external data sources via various communication protocols, map flat or hierarchical datasets, and process imports dynamically through predefined pipelines or customized FreeMarker scripts.

Imports can be scheduled as automated background jobs or triggered interactively by an administrator or partner.

---

Import Job Lifecycle

Every import job executes through a standardized six-step lifecycle:

┌──────────────────────────────────────────┐
│          1. Import Job Triggered         │
└────────────────────┬─────────────────────┘
                     │
                     ▼
┌──────────────────────────────────────────┐
│  2. Communication Channel Fetches File   │
└────────────────────┬─────────────────────┘
                     │
                     ▼
┌──────────────────────────────────────────┐
│  3. Data Mapping System Evaluates File   │
└────────────────────┬─────────────────────┘
                     │
                     ▼
┌──────────────────────────────────────────┐
│ 4. Transformation to PXM Target Schema   │
└────────────────────┬─────────────────────┘
                     │
                     ▼
┌──────────────────────────────────────────┐
│ 5. Serialization of Commands in WorkArea │
└────────────────────┬─────────────────────┘
                     │
                     ▼
┌──────────────────────────────────────────┐
│   6. Command Evaluation & DB Ingestion   │
└──────────────────────────────────────────┘

The 6 Core Steps

1. Triggering: The job is triggered immediately (manually by a user) or dynamically via a scheduled cron rule configured in the Job Scheduling Manager.
2. Receiving: The designated Communication Channel connects to the external source and retrieves the import file.
3. Mapping & Ingestion: The parser evaluates the file structure. It executes Flat Mapping (for table-structured files like CSV or Excel) or Hierarchy Mapping (for XML or JSON structures).
4. Command Creation: The mapped elements are converted into granular transactions using the internal PXM Command API.
5. Serialization: The generated commands are serialized as JSON payloads inside the designated WorkArea directory on the server.
6. Execution: The PXM Command Engine evaluates the serialized JSON commands to securely persist or update entities in the database.

---

User Stories

To understand how different actors interact with the Import Module, review these common workflows:

Administrator / Integrator

1. Define Channels: Creates and tests a REST Web Service communication channel to pull daily supplier CSV updates.
2. Write Mapping Rules: Sets up translation maps to normalize supplier colors and sizes into standard corporate attributes.
3. Automate: Schedules the job to run nightly at 2:00 AM and emails a summary status report to the data quality team.

Business User / Product Manager

1. Interactive Upload: Uploads an ad-hoc Excel spreadsheet with updated supplier pricing for the upcoming season.
2. Pre-Validate: Runs a dry-run in the visual editor to check for empty columns or mapping warnings.
3. Ingest & Verify: Triggers the ingestion immediately and monitors the PXM UI import log report to ensure all prices mapped correctly to active contracts.