Multi-Tenant Custom Pricing System
Revamped a custom pricing ETL platform — hybrid serverful/edge approach with self-hosted Inngest, Cloudflare Workers for public routes, and PlanetScale. Designed to cut $1K–$7K/month serverless costs to ~$400 total. Code handed off to Four13, not yet deployed.
Overview
Designed the architecture for revamping the custom pricing ETL system for a multi-tenant integration platform serving US-based e-commerce clients. The system handles complex pricing rules across multiple Acumatica ERP instances, transforming and loading pricing data to storefronts. This was a proposed replacement for the existing serverless system — designed but not deployed.
Combined with the migration app, this architecture was projected to reduce compute costs from $1K–$7K/month down to ~$400/month total for both systems.
The Problem
The existing pricing pipeline ran on serverless infrastructure. ETL processes are long-running by nature — they don’t fit the serverless model of short-lived, stateless invocations. The per-invocation billing was driving costs up significantly as client volume grew across multiple Acumatica instances.
Proposed Architecture
The design splits the system into two compute tiers — serverful for ETL, serverless for public routes — each optimized for its workload characteristics.
Serverful Tier: Application Server on Digital Ocean
A TanStack Start/Next.js application server running on a Digital Ocean Droplet handles both the management app and ETL jobs:
- Acumatica instances (A, B, C) send data to the server via dynamic cron per tenant
- Self-hosted Inngest (with its own Postgres and Redis) orchestrates the workflows — Inngest handles scheduling, retries, and step-level durability, but the application server runs the actual operations
- ETL jobs transform pricing data and write to a PlanetScale database with replication
Why serverful for ETL: pricing ETL processes are long-running and predictable. A persistent server avoids per-invocation costs and cold start penalties. Self-hosting Inngest gives full control over the orchestration layer.
Serverless Tier: Edge Public Routes on Cloudflare Workers
A Hono server on Cloudflare Workers serves the public-facing API routes:
- Cloudflare KV for caching frequently accessed pricing data
- PlanetScale connected via Cloudflare Hyperdrive to support serverless edge calls and avoid connection pooling issues
- Separate from the application server to reduce load on the ETL tier
Why serverless for public routes: storefront traffic is random and short-lived — perfect for serverless. Edge deployment supports global customers for clients. Having a separate server means public API traffic doesn’t compete with long-running ETL jobs for resources.
The Key Insight
It would be better to have a separate edge-based server for public routes because traffic is random and short-lived — perfect for serverless. ETL processes are long-running, so we need a serverful approach for those.
With this approach, the only thing we would need to scale is the database, which PlanetScale can handle. Scale depends on how many products and different pricing plans clients have — no need for a full data engineering stack.
This hybrid approach means each tier is optimized for its actual workload:
- ETL tier: consistent, long-running, cost-sensitive → serverful on a cheap VPS
- Public tier: bursty, short-lived, latency-sensitive → serverless at the edge
Status
Code and architecture were completed and handed off to Four13 Group, but not yet deployed to production.