Middleware

Middleware provides a powerful way to enhance the content system with additional functionality. This document provides an overview of the middleware system, its core concepts, and available implementations.

Middleware Fundamentals

What is Middleware?

Middleware are functions that intercept content operations, allowing you to add functionality before and after operations are executed. Middleware can:

• Modify input parameters before operation execution
• Transform content after operation execution
• Short-circuit operations under specific conditions
• Add cross-cutting concerns like logging, caching, and validation
• Extend system functionality without modifying core code

Middleware Function Signature

Middleware follows a consistent function signature:

type Middleware<T = ContentContext> = (
  context: T,
  next: () => Promise<T>
) => Promise<T>

Where:

• context is an object containing the operation parameters and state
• next() is a function that invokes the next middleware in the chain
• The returned promise resolves to the (potentially modified) context

Content Context

The ContentContext object contains information about the current operation:

interface ContentContext {
  // Operation information
  operation: 'read' | 'write' | 'delete' | 'list' | 'exists' | 'watch'
  uri: string
  options?: OperationOptions

  // Content data (for write operations or after read)
  content?: Content

  // Results (for list operations)
  results?: string[]

  // Error information
  error?: Error

  // Middleware state
  state: Record<string, any>

  // Metadata
  startTime?: number
  endTime?: number
}

Middleware Pipeline

Composition

Middleware functions are composed into a pipeline:

import { composeMiddleware } from '@lib/content/middleware'

const pipeline = composeMiddleware([
  loggingMiddleware({ level: 'info' }),
  cacheMiddleware({ ttl: 3600 }),
  validationMiddleware({ schema }),
  transformMiddleware({ transforms }),
])

The pipeline executes middleware in the specified order for the request phase (entering the chain) and in reverse order for the response phase (exiting the chain).

Execution Flow

When a pipeline executes, it follows this flow:

1. Create initial context with operation details
2. Enter first middleware function with context
3. Middleware may modify context and call next()
4. Enter second middleware with (possibly modified) context
5. Process continues until last middleware is reached
6. Last middleware completes and returns context
7. Return to previous middleware after its next() call
8. Continue unwinding the stack until returning to first middleware
9. Return final context from pipeline

This bidirectional flow allows middleware to process both before and after the operation.

Core Middleware Types

The content system includes several categories of middleware:

Enhancement Middleware

Enhancement middleware adds functionality to operations:

• Caching: Store and retrieve content from memory or persistent cache
• Validation: Ensure content meets schema requirements
• Transformation: Apply content transformations during operations
• Composition: Combine content from multiple sources

Observability Middleware

Observability middleware provides insights into operations:

• Logging: Record operation details for debugging and auditing
• Metrics: Collect performance and usage metrics
• Tracing: Track operation flow across system boundaries
• Profiling: Measure operation performance characteristics

Control Flow Middleware

Control flow middleware manages operation execution:

• Rate Limiting: Restrict operation frequency
• Circuit Breaking: Prevent operations during failure conditions
• Retrying: Automatically retry failed operations
• Timeout: Cancel operations that take too long

Security Middleware

Security middleware enforces security policies:

• Authentication: Verify identity for operations
• Authorization: Check permissions for operations
• Sanitization: Clean potentially dangerous content
• Encryption: Encrypt sensitive content during operations

Using Middleware

Adding Middleware to a Store

Middleware is typically added when creating a content store:

import { createContentStore } from '@lib/content'
import {
  cacheMiddleware,
  validationMiddleware,
  loggingMiddleware,
} from '@lib/content/middleware'

const store = createContentStore({
  adapter: createFileSystemAdapter({ basePath: '/content' }),
  middleware: [
    loggingMiddleware({ level: 'info' }),
    cacheMiddleware({ ttl: 3600 }),
    validationMiddleware({ schema }),
  ],
})

Middleware Execution Order

The order of middleware is significant. Consider these principles:

1. Early Exits First: Middleware that might short-circuit (like caching) should come early
2. Cross-Cutting First: Logging and metrics should typically come first to capture all operations
3. Transformations Last: Content transformations should typically be closest to the adapter
4. Consider Both Phases: Remember middleware unwinds in reverse order

Conditional Middleware

Middleware can be applied conditionally:

import { conditionalMiddleware } from '@lib/content/middleware'

// Apply specific middleware only for Markdown content
const markdownOnlyMiddleware = conditionalMiddleware(
  context => context.uri.endsWith('.md'),
  markdownProcessingMiddleware
)

Creating Custom Middleware

Basic Middleware Template

Custom middleware follows this basic pattern:

function customMiddleware(options?: Options): Middleware {
  // Initialize middleware with options
  return async (context, next) => {
    // Before next middleware (request phase)
    console.log(`Starting operation: ${context.operation} ${context.uri}`)

    try {
      // Call next middleware
      const result = await next()

      // After next middleware (response phase)
      console.log(`Completed operation: ${context.operation} ${context.uri}`)

      return result
    } catch (error) {
      // Error handling
      console.error(
        `Error in operation: ${context.operation} ${context.uri}`,
        error
      )
      throw error
    }
  }
}

Middleware Factory Pattern

Middleware typically follows the factory pattern:

interface LoggingOptions {
  level?: 'error' | 'warn' | 'info' | 'debug'
  logger?: Logger
}

function loggingMiddleware(options?: LoggingOptions): Middleware {
  // Default options
  const { level = 'info', logger = console } = options || {}

  // Return the middleware function
  return async (context, next) => {
    // Implementation
  }
}

Middleware State

Middleware can store state in the context:

function timingMiddleware(): Middleware {
  return async (context, next) => {
    // Store start time in context state
    context.state.timingStart = Date.now()

    // Call next middleware
    const result = await next()

    // Calculate duration
    const duration = Date.now() - context.state.timingStart

    // Store in state for other middleware
    context.state.operationDuration = duration

    return result
  }
}

Error Handling in Middleware

Error Propagation

Errors in middleware propagate up the chain:

function errorHandlingMiddleware(): Middleware {
  return async (context, next) => {
    try {
      return await next()
    } catch (error) {
      // Convert to specific error type if needed
      if (error instanceof ValidationError) {
        throw new ContentValidationError(
          `Validation failed for ${context.uri}: ${error.message}`,
          { cause: error }
        )
      }

      // Rethrow other errors
      throw error
    }
  }
}

Recovery Middleware

Middleware can recover from errors:

function recoveryMiddleware(): Middleware {
  return async (context, next) => {
    try {
      return await next()
    } catch (error) {
      // For certain error types, provide fallback content
      if (
        error instanceof ContentNotFoundError &&
        context.operation === 'read'
      ) {
        console.warn(`Content not found: ${context.uri}, using fallback`)

        context.content = {
          data: '# Not Found\n\nThe requested content was not found.',
          contentType: 'text/markdown',
          metadata: { title: 'Not Found' },
        }

        return context
      }

      // Rethrow other errors
      throw error
    }
  }
}

Middleware Performance

Performance Considerations

Consider these performance factors when using middleware:

1. Middleware Count: Each middleware adds processing overhead
2. Execution Frequency: Hot paths should have minimal middleware
3. Asynchronous Operations: Each await has overhead; minimize when possible
4. State Management: Large context objects can impact performance
5. Early Returns: Short-circuit operations when possible to avoid unnecessary processing

Middleware Optimization

Techniques for optimizing middleware:

function optimizedMiddleware(): Middleware {
  // Prepare reusable resources during initialization
  const cache = new Map()

  return async (context, next) => {
    // Early return for operations that don't need processing
    if (context.operation === 'exists') {
      return next()
    }

    // Process only specific content types
    if (context.content && !isProcessableType(context.content.contentType)) {
      return next()
    }

    // Use cached results when possible
    const cacheKey = `${context.operation}:${context.uri}`
    if (cache.has(cacheKey)) {
      return cache.get(cacheKey)
    }

    // Continue with normal processing
    const result = await next()

    // Cache result for future use
    cache.set(cacheKey, result)

    return result
  }
}

Available Middleware

Core Middleware

• Caching Middleware: Store and retrieve content from cache
• Validation Middleware: Validate content against schemas
• Logging Middleware: Log operation details
• [TODO] Error Handling Middleware: Enhance error information and recovery

Composition Utilities

• Middleware Composition: Combine middleware functions
• Conditional Middleware: Apply middleware based on conditions
• Middleware Factories: Create configurable middleware

Best Practices

1. Focused Middleware: Keep each middleware focused on a single responsibility
2. Error Handling: Handle errors appropriately in each middleware
3. Performance Awareness: Be mindful of performance impact in hot paths
4. Idempotent Operations: Ensure middleware can be safely applied multiple times
5. Clear Naming: Use descriptive names that reflect middleware purpose
6. Configuration Options: Provide sensible defaults but allow configuration
7. Documentation: Document middleware behavior, options, and side effects
8. Testing: Test middleware in isolation and in composition

Related Resources

• Composition Architecture
• Data Flow Concepts
• Content Operations
• Middleware Patterns