HTTP Adapter

The HTTP adapter provides remote content storage and retrieval through HTTP/HTTPS protocols for the ReX content system. It enables integration with remote content APIs, headless CMS systems, and other HTTP-based content services.

Overview

The HTTP adapter enables the content system to interact with remote content endpoints, providing a bridge between local applications and remote content repositories. It supports RESTful operations, authentication, caching, and resilient network handling.

typescript

import { createHttpAdapter } from '@lib/content/adapters/common/http'

const adapter = createHttpAdapter({
  baseUrl: 'https://api.example.com/content',
  headers: {
    Accept: 'application/json',
  },
  authentication: {
    type: 'bearer',
    token: 'your-access-token',
  },
})

API Reference

Creation

typescript

function createHttpAdapter(options?: HttpAdapterOptions): ContentAdapter

Creates a new HTTP adapter instance.

Options

typescript

interface HttpAdapterOptions extends ContentAdapterOptions {
  /**
   * Base URL for HTTP requests (required)
   */
  baseUrl?: string

  /**
   * Default HTTP headers to include with every request
   */
  headers?: Record<string, string>

  /**
   * Default fetch options (RequestInit)
   */
  fetchOptions?: RequestInit

  /**
   * Retry configuration
   */
  retryOptions?: {
    /**
     * Maximum number of retry attempts (default: 3)
     */
    maxRetries?: number

    /**
     * Delay between retries in ms (default: 1000)
     */
    retryDelay?: number

    /**
     * HTTP status codes that trigger a retry (default: [408, 429, 500, 502, 503, 504])
     */
    retryStatusCodes?: number[]

    /**
     * Use exponential backoff for retries (default: true)
     */
    useExponentialBackoff?: boolean
  }

  /**
   * Caching configuration
   */
  cacheOptions?: {
    /**
     * Enable client-side caching (default: true)
     */
    enabled?: boolean

    /**
     * Cache TTL in milliseconds (default: 60000)
     */
    maxAge?: number

    /**
     * Maximum cache items (default: 100)
     */
    maxItems?: number

    /**
     * Storage mechanism (default: 'memory')
     */
    storage?: 'memory' | 'localStorage' | 'sessionStorage'

    /**
     * Cache key generator function
     */
    keyGenerator?: (uri: string, options?: any) => string
  }

  /**
   * Authentication configuration
   */
  authentication?: {
    /**
     * Authentication type
     */
    type: 'none' | 'basic' | 'bearer' | 'custom'

    /**
     * Basic auth credentials
     */
    credentials?: {
      username: string
      password: string
    }

    /**
     * Bearer token
     */
    token?: string

    /**
     * Function to generate authentication headers
     */
    headerFactory?: () => Promise<Record<string, string>>

    /**
     * Refresh token functionality
     */
    refreshToken?: {
      /**
       * Token refresh URL
       */
      url: string

      /**
       * Refresh token value
       */
      token: string

      /**
       * Function to process refresh response
       */
      handleResponse: (response: any) => {
        token: string
        refreshToken?: string
      }
    }
  }

  /**
   * URL mapping for HTTP endpoints
   */
  urlMapping?: {
    /**
     * Read operation URL template (default: '{baseUrl}/{uri}')
     */
    read?: string

    /**
     * Write operation URL template (default: '{baseUrl}/{uri}')
     */
    write?: string

    /**
     * Delete operation URL template (default: '{baseUrl}/{uri}')
     */
    delete?: string

    /**
     * List operation URL template (default: '{baseUrl}?pattern={pattern}')
     */
    list?: string

    /**
     * Exists operation URL template (default: '{baseUrl}/{uri}')
     */
    exists?: string
  }
}

Methods

read

typescript

async read(uri: string, options?: ReadOptions): Promise<Content>;

Reads content from the remote endpoint.

Parameters:

uri: Content URI
options: Optional read configuration

Returns:

A Promise resolving to a Content object

Example:

typescript

const content = await adapter.read('articles/welcome.md')
console.log(content.data) // Content data
console.log(content.metadata) // Associated metadata

Implementation Details:

Constructs request URL from baseUrl and URI
Adds authentication headers if configured
Makes HTTP GET request to the endpoint
Parses response based on content type
Caches response if caching is enabled
Handles network errors and HTTP status codes
Implements retry logic for transient failures

write

typescript

async write(uri: string, content: Content, options?: WriteOptions): Promise<void>;

Writes content to the remote endpoint.

Parameters:

uri: Content URI
content: Content object to write
options: Optional write configuration

Example:

typescript

await adapter.write('articles/welcome.md', {
  data: '# Welcome\n\nThis is a welcome article.',
  contentType: 'text/markdown',
  metadata: {
    title: 'Welcome',
    createdAt: new Date(),
  },
})

Implementation Details:

Constructs request URL from baseUrl and URI
Adds authentication headers if configured
Serializes content based on endpoint requirements
Makes HTTP PUT or POST request to the endpoint
Handles network errors and HTTP status codes
Implements retry logic for transient failures
Invalidates cache entries if caching is enabled

delete

typescript

async delete(uri: string, options?: DeleteOptions): Promise<void>;

Deletes content from the remote endpoint.

Parameters:

uri: Content URI
options: Optional delete configuration

Example:

typescript

await adapter.delete('articles/outdated-article.md')

Implementation Details:

Constructs request URL from baseUrl and URI
Adds authentication headers if configured
Makes HTTP DELETE request to the endpoint
Handles network errors and HTTP status codes
Implements retry logic for transient failures
Invalidates cache entries if caching is enabled

list

typescript

async list(pattern?: string, options?: ListOptions): Promise<string[]>;

Lists content URIs matching the pattern from the remote endpoint.

Parameters:

pattern: Optional glob pattern for content URIs
options: Optional list configuration

Returns:

A Promise resolving to an array of content URIs

Example:

typescript

const uris = await adapter.list('articles/**/*.md')
console.log(uris) // ['articles/welcome.md', 'articles/guide.md', ...]

Implementation Details:

Constructs request URL with pattern parameter
Adds authentication headers if configured
Makes HTTP GET request to the list endpoint
Parses response to extract URI list
Handles network errors and HTTP status codes
Implements retry logic for transient failures
Optionally caches results based on configuration

exists

typescript

async exists(uri: string, options?: ExistsOptions): Promise<boolean>;

Checks if content exists at the remote endpoint.

Parameters:

uri: Content URI
options: Optional exists configuration

Returns:

A Promise resolving to a boolean indicating existence

Example:

typescript

const exists = await adapter.exists('articles/welcome.md')
if (exists) {
  console.log('Article exists on the remote server')
}

Implementation Details:

Constructs request URL from baseUrl and URI
Makes HTTP HEAD request to check existence
Returns true for successful responses (2xx)
Returns false for 404 Not Found responses
Throws errors for other HTTP status codes
Implements retry logic for transient failures

dispose

typescript

async dispose(): Promise<void>;

Cleans up resources used by the adapter.

Example:

typescript

await adapter.dispose()

Implementation Details:

Cancels any pending requests
Clears internal caches
Revokes any active authentication tokens
Releases other resources

URL Templating

The HTTP adapter uses URL templates to construct endpoint URLs:

typescript

function resolveUrl(template: string, params: Record<string, string>): string {
  return template.replace(/\{(\w+)\}/g, (_, key) => {
    return encodeURIComponent(params[key] || '')
  })
}

// Example
const readUrl = resolveUrl(options.urlMapping.read, {
  baseUrl: options.baseUrl,
  uri: uri,
})

Authentication

The adapter supports multiple authentication methods:

Basic Authentication

typescript

function applyBasicAuth(headers: Headers): void {
  const { username, password } = options.authentication.credentials
  const authValue = btoa(`${username}:${password}`)
  headers.set('Authorization', `Basic ${authValue}`)
}

Bearer Authentication

typescript

function applyBearerAuth(headers: Headers): void {
  const { token } = options.authentication
  headers.set('Authorization', `Bearer ${token}`)
}

Custom Authentication

typescript

async function applyCustomAuth(headers: Headers): Promise<void> {
  const { headerFactory } = options.authentication
  const authHeaders = await headerFactory()

  for (const [key, value] of Object.entries(authHeaders)) {
    headers.set(key, value)
  }
}

Token Refresh

typescript

async function refreshAuthToken(): Promise<void> {
  const { url, token, handleResponse } = options.authentication.refreshToken

  const response = await fetch(url, {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({ refresh_token: token }),
  })

  if (!response.ok) {
    throw new ContentAccessError(
      `Failed to refresh authentication token: ${response.statusText}`
    )
  }

  const data = await response.json()
  const tokens = handleResponse(data)

  // Update tokens
  options.authentication.token = tokens.token
  if (tokens.refreshToken) {
    options.authentication.refreshToken.token = tokens.refreshToken
  }
}

Caching System

The HTTP adapter implements client-side caching for read operations:

typescript

class HttpCache {
  private cache: Map<string, { content: Content; expires: number }>
  private maxItems: number

  constructor(options: { maxItems?: number; maxAge?: number }) {
    this.cache = new Map()
    this.maxItems = options.maxItems || 100
    this.maxAge = options.maxAge || 60000
  }

  get(key: string): Content | undefined {
    const cached = this.cache.get(key)

    if (!cached) {
      return undefined
    }

    // Check if expired
    if (cached.expires < Date.now()) {
      this.cache.delete(key)
      return undefined
    }

    return cached.content
  }

  set(key: string, content: Content): void {
    // Enforce capacity limits
    if (this.cache.size >= this.maxItems) {
      // Remove oldest item
      const oldestKey = this.cache.keys().next().value
      this.cache.delete(oldestKey)
    }

    this.cache.set(key, {
      content,
      expires: Date.now() + this.maxAge,
    })
  }

  invalidate(key: string): void {
    this.cache.delete(key)
  }

  clear(): void {
    this.cache.clear()
  }
}

Error Handling

The adapter maps HTTP errors to content errors:

typescript

function handleHttpError(response: Response, uri: string): ContentError {
  switch (response.status) {
    case 400:
      return new ContentError(
        `Bad request: ${response.statusText}`,
        uri,
        'http',
        false
      )

    case 401:
    case 403:
      return new ContentAccessError(
        `Access denied: ${response.statusText}`,
        uri,
        'http'
      )

    case 404:
      return new ContentNotFoundError(uri, 'http')

    case 409:
      return new ContentError(
        `Conflict: ${response.statusText}`,
        uri,
        'http',
        true,
        { canRetry: false }
      )

    case 422:
      return new ContentValidationError(
        `Validation failed: ${response.statusText}`,
        uri,
        'http',
        [`Server validation failed: ${response.statusText}`]
      )

    default:
      return new ContentError(
        `HTTP error ${response.status}: ${response.statusText}`,
        uri,
        'http',
        response.status >= 500, // Server errors are potentially recoverable
        { canRetry: response.status >= 500 }
      )
  }
}

Retry Mechanism

The adapter implements a retry mechanism for transient failures:

typescript

async function executeWithRetry<T>(operation: () => Promise<T>): Promise<T> {
  const {
    maxRetries = 3,
    retryDelay = 1000,
    useExponentialBackoff = true,
  } = options.retryOptions

  let attempts = 0
  let lastError: Error

  while (attempts <= maxRetries) {
    try {
      return await operation()
    } catch (error) {
      lastError = error

      // Check if error is retryable
      if (!isRetryableError(error)) {
        throw error
      }

      // Last attempt failed
      if (attempts >= maxRetries) {
        break
      }

      // Delay before retry
      const delay = useExponentialBackoff
        ? retryDelay * Math.pow(2, attempts)
        : retryDelay

      await sleep(delay)
      attempts++
    }
  }

  // All retries failed
  throw new ContentError(
    `Operation failed after ${attempts} attempts: ${lastError.message}`,
    undefined,
    'http',
    false
  )
}

function isRetryableError(error: any): boolean {
  // Network errors are retryable
  if (error instanceof TypeError && error.message.includes('network')) {
    return true
  }

  // Check if HTTP status is retryable
  if (error instanceof ContentError && error.recoverable) {
    return true
  }

  // Specific HTTP status codes
  if (
    error.status &&
    options.retryOptions.retryStatusCodes.includes(error.status)
  ) {
    return true
  }

  return false
}

Content Serialization

The adapter handles content serialization based on content type:

typescript

function serializeContent(content: Content): string | FormData {
  const contentType = content.contentType.toLowerCase()

  // JSON content
  if (contentType.includes('json')) {
    return JSON.stringify({
      data: content.data,
      metadata: content.metadata,
    })
  }

  // Form data
  if (contentType.includes('form')) {
    const formData = new FormData()
    formData.append('data', content.data as string)

    // Add metadata as separate fields
    for (const [key, value] of Object.entries(content.metadata)) {
      formData.append(`metadata[${key}]`, String(value))
    }

    return formData
  }

  // Default to plain text
  return String(content.data)
}

function deserializeContent(
  data: any,
  contentType: string,
  responseHeaders: Headers
): Content {
  // Extract content type from response if not provided
  if (!contentType) {
    contentType = responseHeaders.get('Content-Type') || 'text/plain'
  }

  // JSON response
  if (contentType.includes('json')) {
    // Handle various JSON response formats
    if (typeof data === 'string') {
      data = JSON.parse(data)
    }

    // Format 1: { data, metadata } structure
    if (data.data !== undefined && data.metadata !== undefined) {
      return {
        data: data.data,
        contentType,
        metadata: data.metadata,
      }
    }

    // Format 2: { content, meta } structure
    if (data.content !== undefined && data.meta !== undefined) {
      return {
        data: data.content,
        contentType,
        metadata: data.meta,
      }
    }

    // Format 3: Complete object is the data
    return {
      data: JSON.stringify(data),
      contentType,
      metadata: {},
    }
  }

  // Handle text content
  return {
    data: String(data),
    contentType,
    metadata: {},
  }
}

Performance Considerations

The HTTP adapter includes performance optimizations:

Caching: Client-side caching reduces network requests
Connection Pooling: Reuses connections when possible
Request Batching: Groups operations to reduce overhead
Compression: Supports gzip/deflate for reduced bandwidth
Streaming: Handles large content with response streaming

Environment Compatibility

This adapter works in all JavaScript environments that support the Fetch API:

typescript

import { createHttpAdapter } from '@lib/content/adapters'
import { isNode } from '@lib/utils/env'

// Choose the appropriate fetch implementation
const adapter = createHttpAdapter({
  baseUrl: 'https://api.example.com/content',
  fetchOptions: {
    // In Node.js environments, may need to disable SSL verification for development
    ...(isNode()
      ? { agent: new https.Agent({ rejectUnauthorized: false }) }
      : {}),
  },
})

Archive

Archive

HTTP Adapter

Overview

API Reference

Creation

Options

Methods

read

write

delete

list

exists

dispose

URL Templating

Authentication

Basic Authentication

Bearer Authentication

Custom Authentication

Token Refresh

Caching System

Error Handling

Retry Mechanism

Content Serialization

Performance Considerations

Environment Compatibility

HTTP Adapter ​

Overview ​

API Reference ​

Creation ​

Options ​

Methods ​

read ​

write ​

delete ​

list ​

exists ​

dispose ​

URL Templating ​

Authentication ​

Basic Authentication ​

Bearer Authentication ​

Custom Authentication ​

Token Refresh ​

Caching System ​

Error Handling ​

Retry Mechanism ​

Content Serialization ​

Performance Considerations ​

Environment Compatibility ​

Related Concepts ​

HTTP Adapter

Overview

API Reference

Creation

Options

Methods

read

write

delete

list

exists

dispose

URL Templating

Authentication

Basic Authentication

Bearer Authentication

Custom Authentication

Token Refresh

Caching System

Error Handling

Retry Mechanism

Content Serialization

Performance Considerations

Environment Compatibility

Related Concepts