Custom Queue Runtime

Creating a custom queue runtime in Pikku involves implementing two classes: QueueService for job publishing and QueueWorkers for job processing. Here's how to build one using pg-boss as an example.

Implementation Pattern

1. Create Your QueueService Implementation

Handle job publishing and retrieval:

PgBossQueueService

loading...

See full example on GitHub

2. Create Your QueueWorkers Implementation

Handle job processing and worker lifecycle:

PgBossQueueWorkers

loading...

See full example on GitHub

3. Create Mapping Utilities

Convert between Pikku types and your queue library:

pg-boss Utils

loading...

See full example on GitHub

Key Implementation Requirements

Job Execution

All queue implementations must call runQueueJob() to execute jobs:

import { runQueueJob } from '@pikku/core/queue'

// In your worker processor
const result = await runQueueJob({
  queueName: 'my-queue',
  data: jobData,
  singletonServices,
  createSessionServices
})

Worker Registration

Use the registration helper to validate and register workers:

Queue Registration Helper

loading...

See full example on GitHub

Configuration Mapping

Define supported and unsupported configurations:

const configMappings: QueueConfigMapping = {
  supported: {
    batchSize: 'teamSize',         // Maps to pg-boss teamSize
    pollInterval: 'intervalSeconds', // Maps to polling interval
  },
  unsupported: {
    lockDuration: 'Managed internally by pg-boss',
    visibilityTimeout: 'Uses PostgreSQL locks instead'
  }
}