Services Architecture

LiveCodes is built on a modular service architecture that separates concerns and enables flexible deployment options. This guide explains each service and how they interact.

Service Overview

LiveCodes consists of several core services:

┌─────────────────────────────────────────────────────┐
│                  LiveCodes App                      │
│                                                     │
│  ┌──────────┐  ┌──────────┐  ┌───────────────┐   │
│  │ Sandbox  │  │ Modules  │  │    GitHub     │   │
│  │ Service  │  │ Service  │  │    Service    │   │
│  └──────────┘  └──────────┘  └───────────────┘   │
│                                                     │
│  ┌──────────┐  ┌──────────┐  ┌───────────────┐   │
│  │  Share   │  │Broadcast │  │   Firebase    │   │
│  │ Service  │  │ Service  │  │    Service    │   │
│  └──────────┘  └──────────┘  └───────────────┘   │
│                                                     │
│  ┌──────────┐  ┌──────────┐                       │
│  │   CORS   │  │  Allowed │                       │
│  │  Proxy   │  │  Origins │                       │
│  └──────────┘  └──────────┘                       │
└─────────────────────────────────────────────────────┘

Sandbox Service

The sandbox service provides secure, isolated execution of user code.

Purpose

Executes untrusted user code in a separate origin to prevent:

Access to parent application cookies and storage
Cross-site scripting (XSS) attacks
Malicious code affecting the main application

Implementation

Location: src/livecodes/services/sandbox.ts

const cfPagesBaseUrl = 'https://livecodes-sandbox.pages.dev';
const ghPagesBaseUrl = 'https://live-codes.github.io/livecodes-sandbox/dist';
const selfHostedBaseUrl = `https://${process.env.SANDBOX_HOST_NAME}:${process.env.SANDBOX_PORT}`;
const localBaseUrl = 'http://127.0.0.1:8085';

const serviceBaseUrl =
  location.hostname === 'localhost' || location.hostname === '127.0.0.1'
    ? localBaseUrl
    : process.env.SELF_HOSTED === 'true'
      ? selfHostedBaseUrl
      : process.env.CI === 'true'
        ? ghPagesBaseUrl
        : cfPagesBaseUrl;

export const sandboxService = {
  getResultUrl: () => `${serviceBaseUrl}/${version}/`,
  getCompilerUrl: () => `${serviceBaseUrl}/${version}/compiler${ext}`,
  getOrigin: () => new URL(serviceBaseUrl).origin,
};

Server Implementation

Location: server/src/sandbox.ts

export const sandbox = async ({ hostname, port }) => {
  const app = express();
  
  app.use(cors());
  app.disable('x-powered-by');
  
  // Dynamically determine sandbox version
  const version = dirs
    .filter((v) => v.startsWith('v'))
    .map((v) => Number(v.slice(1)))
    .sort((a, b) => a - b)
    .map((v) => 'v' + v)
    .pop() || '';
  
  // Serve sandbox HTML files
  app.use('/', (req, res) => {
    res.set('Content-Type', 'text/html');
    res.status(200).sendFile(filePath);
  });
  
  app.listen(port);
};

Sandbox Versioning

Sandbox files are versioned (e.g., v9/) to allow:

Cache-busting on updates
Parallel version support
Rollback capability

The sandbox must be hosted on a different origin (scheme + domain + port) than the main app for security isolation.

Modules Service

Resolves and loads external modules from various CDNs.

Supported CDN Providers

Module CDNs (ESM):

esm.sh (default)
skypack
jspm
jsdelivr.esm
esm.run

Package CDNs (NPM):

jsdelivr (default)
unpkg
npmcdn

GitHub CDNs:

jsdelivr.gh
statically

Module Resolution

Location: src/livecodes/services/modules.ts

export const modulesService = {
  getModuleUrl: (moduleName, { isModule = true, defaultCDN = 'esm.sh' }) => {
    // Remove bundling hints
    moduleName = moduleName.replace(/#nobundle/g, '');
    
    // Get CDN URL based on prefix
    const moduleUrl = getCdnUrl(moduleName, isModule, defaultCDN);
    
    return isModule
      ? 'https://esm.sh/' + moduleName
      : 'https://cdn.jsdelivr.net/npm/' + moduleName;
  },
};

URL Transformation Patterns

The service transforms various input formats:

// npm: packages
'npm:react' → 'https://esm.sh/react'

// GitHub repos
'github:user/repo/file.js' → 'https://deno.bundlejs.com/?file&q=https://cdn.jsdelivr.net/gh/user/repo@file.js'

// JSR packages
'jsr:@scope/package' → 'https://esm.sh/jsr/@scope/package'

// Deno modules
'deno:module/mod.ts' → 'https://deno.bundlejs.com/?file&q=https://deno.land/x/module/mod.ts'

// Direct URLs
'https://example.com/file.js' → (unchanged)

CDN Health Checking

checkCDNs: async (testModule, preferredCDN) => {
  const cdns = [preferredCDN, ...modulesService.cdnLists.npm];
  for (const cdn of cdns) {
    try {
      const res = await fetch(modulesService.getUrl(testModule, cdn), {
        method: 'HEAD',
      });
      if (res.ok) return cdn;
    } catch {
      // Try next CDN
    }
  }
  return modulesService.cdnLists.npm[0]; // Fallback
}

Enables saving and loading projects via short URLs.

Architecture

Storage: Valkey/Redis key-value store
ID Length: 14 characters (reduces collision probability)

Implementation

Location: server/src/share.ts

const valkey = new Valkey(
  Number(process.env.VALKEY_PORT || 6379),
  process.env.VALKEY_HOST || 'valkey'
);

const saveProject = async (req, res) => {
  const value = JSON.stringify(req.body);
  
  // Generate unique 14-character ID
  let id = generateId(14);
  
  // Avoid collision
  while (await valkey.get(id)) {
    id = generateId(14);
  }
  
  await valkey.set(id, value);
  res.status(200).send(id);
};

const getProject = async (req, res) => {
  const id = req.query.id;
  const value = await valkey.get(String(id).toLowerCase());
  
  if (!value) {
    res.status(404).send('Not Found!');
    return;
  }
  
  res.status(200).json(JSON.parse(value));
};

Data Persistence

Configured in docker-compose.yml:

valkey:
  command:
    - valkey-server
    - --save 60 1  # Save if ≥1 change in 60 seconds
    - --loglevel warning
  volumes:
    - valkey-data:/data

ID Comparison

Service	ID Length	Example
dpaste	9 chars	`xY3kL9pQm`
API	11 chars	`xY3kL9pQmR2`
Self-hosted	14 chars	`xY3kL9pQmR2v5A`

Longer IDs reduce collision probability: 14 chars = 62^14 ≈ 1.1 × 10^25 combinations

Broadcast Service

Enables real-time collaborative coding and live broadcasts.

Features

Real-time code streaming
WebSocket-based communication
Channel-based isolation
Optional token authentication
Automatic channel cleanup (20-minute timeout)

Implementation

Location: server/src/broadcast/index.ts

export const broadcast = ({ hostname, port, appUrl, userTokens }) => {
  const app = express();
  const httpserver = http.createServer(app);
  const io = new socketio.Server(httpserver);
  
  const channels = {}; // In-memory channel storage
  
  // Create/update broadcast
  app.post('/', (req, res) => {
    const channel = req.body.channel || generateId();
    const channelToken = channels[channel]?.channelToken || generateId();
    
    // Emit to all subscribers
    io.in(channel).emit('receive', result, data);
    
    // Store reduced data (limit sizes)
    channels[channel] = {
      channelToken,
      result: result.length < 300000 ? result : '',
      data: JSON.stringify(data).length < 500000 ? data : {},
      lastAccessed: Date.now(),
    };
    
    res.json({ channel, channelUrl: `${broadcastUrl}/channels/${channel}` });
  });
  
  // WebSocket connections
  io.on('connection', (socket) => {
    socket.on('join', (channel) => {
      if (!channels[channel]) return;
      socket.join(channel);
      const { result, data } = channels[channel];
      socket.emit('receive', result, data);
    });
  });
};

Authentication

const hasPermission = (req) => {
  if (!userTokens.trim()) return true; // No auth required
  
  const userToken = req.body.userToken || req.query.userToken;
  const validTokens = userTokens.split(',').map(t => t.trim());
  
  return validTokens.includes(userToken);
};

Channel Lifecycle

// Automatic cleanup on response finish
res.on('finish', () => {
  Object.keys(channels).forEach((key) => {
    const timeout = 1000 * 60 * 20; // 20 minutes
    if (Date.now() - channels[key].lastAccessed > timeout) {
      delete channels[key];
    }
  });
});

Data Size Limits

Result HTML: 300KB max
Project data: 500KB max (JSON stringified)
Compiled code: Stripped from stored data

GitHub Service

Integrates with GitHub API for repository operations.

Features

Create/update repositories
Commit files (single or batch)
Read repository contents
Deploy to GitHub Pages
List user repositories

Authentication

Location: src/livecodes/services/github.ts

export const getGithubHeaders = (user, mediaType) => ({
  Accept: `application/vnd.github.v3${mediaType ? '.' + mediaType : ''}+json`,
  'Content-Type': 'application/json',
  Authorization: 'token ' + user.token,
});

Batch Commits

Uses Git Tree API for atomic multi-file commits:

export const commitFiles = async ({
  files, user, repo, branch, message
}) => {
  // 1. Get last commit SHA
  const lastCommit = await getLastCommit(user, repo, branch);
  
  // 2. Get current tree
  const baseTree = await getTree(user, repo, lastCommit);
  
  // 3. Create new tree with all files
  const tree = await createTree(user, repo, files, baseTree);
  
  // 4. Create commit pointing to new tree
  const commit = await createCommit(user, repo, message, tree, lastCommit);
  
  // 5. Update branch to point to new commit
  await updateBranch(user, repo, branch, commit);
  
  return { tree, commit };
};

Content API Size Limits

GitHub limits file content to 1MB via the Contents API:

// For files >1MB, use raw API
if (result.content === '' && result.encoding === 'none') {
  const rawRes = await fetch(url, {
    headers: getGithubHeaders(user, 'raw'),
  });
  result.content = encode(await rawRes.text());
  result.encoding = 'base64';
}

Firebase Service

Handles user authentication via GitHub OAuth.

Configuration

Location: src/livecodes/services/firebase.ts

export const firebaseConfig = selfHostedConfig || {
  apiKey: 'AIzaSyB352dJ_NKCZ43G5kv9Lt-sb5nMXTJRONQ',
  authDomain: 'livecodes-io.firebaseapp.com',
  projectId: 'livecodes-io',
  // ...
};

Self-Hosted Override

Provide custom Firebase config via environment variable:

FIREBASE_CONFIG='{"apiKey":"...","authDomain":"..."}'

CORS Proxy Service

Proxies external requests to avoid CORS restrictions.

Security

Location: server/src/cors.ts

export const corsProxy = (req, res) => {
  const origin = req.get('Origin');
  const hostname = process.env.HOST_NAME || 'localhost';
  
  // Only allow requests from configured hostname
  if ((origin && !origin.includes(hostname)) || 
      !['GET', 'POST'].includes(req.method)) {
    res.status(403).send('Forbidden!');
    return;
  }
  
  const url = req.body?.url || req.query?.url;
  
  fetch(url, { method: req.method, ... })
    .then(r => res.send(body))
    .catch(err => res.send(err));
};

Content Type Handling

const contentType = r.headers.get('Content-Type') || '';

if (contentType === 'application/json') {
  return r.json();
}
if (['application/zip', 'application/octet-stream'].includes(contentType) ||
    contentType.startsWith('image/')) {
  return r.arrayBuffer();
}
return r.text();

Allowed Origins Service

Validates origins for security-sensitive operations. Location: src/livecodes/services/allowed-origin.ts

export const allowedOrigin = (origin = location.origin) =>
  Boolean(
    origin &&
      (origin.endsWith('livecodes.io') ||
       origin.endsWith('livecodes.pages.dev') ||
       origin.includes('127.0.0.1') ||
       origin.includes('localhost:') ||
       origin.endsWith('localhost') ||
       origin.endsWith('.test')),
  );

Whitelist Targets

Allows specific external domains:

export const whitelistTarget = (url) =>
  /^(?:(?:http|https):\/\/(?:\w+.)?)(githubusercontent.com|jsbin.com)\/(.*)/g
    .test(url);

Service Communication

Message Passing

Services communicate via postMessage for sandbox isolation:

// App → Sandbox
const message = {
  type: 'compile',
  payload: { content, language, config },
};
compilerSandbox.postMessage(message, sandboxOrigin);

// Sandbox → App
window.addEventListener('message', (event) => {
  if (event.origin === sandboxOrigin && 
      event.data.type === 'compiled') {
    resolve(event.data.payload.compiled);
  }
});

HTTP Endpoints

Endpoint	Method	Purpose
`/api/share`	GET	Retrieve shared project
`/api/share`	POST	Save new project
`/api/cors`	GET/POST	CORS proxy
`/oembed`	GET	oEmbed metadata
`/`	GET	Serve main app

Next Steps

Security Model

Learn about security implementation

Custom Compilers

Create custom language compilers

Performance

Optimize service performance

Docker Setup

Configure services in Docker

Get Started

Features

SDK

Languages

Configuration

Advanced

Resources

​Service Overview

​Sandbox Service

​Purpose

​Implementation

​Server Implementation

​Sandbox Versioning

​Modules Service

​Supported CDN Providers

​Module Resolution

​URL Transformation Patterns

​CDN Health Checking

​Share Service

​Architecture

​Implementation

​Data Persistence

​ID Comparison

​Broadcast Service

​Features

​Implementation

​Authentication

​Channel Lifecycle

​Data Size Limits

​GitHub Service

​Features

​Authentication

​Batch Commits

​Content API Size Limits

​Firebase Service

​Configuration

​Self-Hosted Override

​CORS Proxy Service

​Security

​Content Type Handling

​Allowed Origins Service

​Whitelist Targets

​Service Communication

​Message Passing

​HTTP Endpoints

​Next Steps

Security Model

Custom Compilers

Performance

Docker Setup

Service Overview

Sandbox Service

Purpose

Implementation

Server Implementation

Sandbox Versioning

Modules Service

Supported CDN Providers

Module Resolution

URL Transformation Patterns

CDN Health Checking

Share Service

Architecture

Implementation

Data Persistence

ID Comparison

Broadcast Service

Features

Implementation

Authentication

Channel Lifecycle

Data Size Limits

GitHub Service

Features

Authentication

Batch Commits

Content API Size Limits

Firebase Service

Configuration

Self-Hosted Override

CORS Proxy Service

Security

Content Type Handling

Allowed Origins Service

Whitelist Targets

Service Communication

Message Passing

HTTP Endpoints

Next Steps