Custom Language Compilers

LiveCodes supports 90+ languages out of the box, but you can extend it with custom compilers for any language or transpiler.

Compiler Architecture

LiveCodes uses a plugin-based compiler system with these components:

┌─────────────────────────────────────────────────────┐
│              Compiler System                         │
│                                                     │
│  ┌────────────────────────────────────────────┐   │
│  │     Language Specification                   │   │
│  │  - Name, title, extensions                 │   │
│  │  - Compiler factory function               │   │
│  │  - Editor configuration                    │   │
│  └────────────────────────────────────────────┘   │
│                      │                            │
│                      ▼                            │
│  ┌────────────────────────────────────────────┐   │
│  │     Compiler Sandbox (Isolated)          │   │
│  │  - Loads compiler library                  │   │
│  │  - Executes compilation                    │   │
│  │  - Returns compiled code                   │   │
│  └────────────────────────────────────────────┘   │
│                      │                            │
│                      ▼                            │
│  ┌────────────────────────────────────────────┐   │
│  │     Post-Processors                       │   │
│  │  - Babel, PostCSS, etc.                    │   │
│  │  - Applied after compilation               │   │
│  └────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────┘

Language Specification

Basic Structure

Every language is defined by a LanguageSpecs object: Location: src/livecodes/languages/<language>/lang-<language>.ts

import type { LanguageSpecs } from '../../models';

export const typescript: LanguageSpecs = {
  name: 'typescript',
  title: 'TS',
  longTitle: 'TypeScript',
  
  // Compiler configuration
  compiler: {
    url: 'https://cdn.jsdelivr.net/npm/typescript@5.4.5/lib/typescript.js',
    factory: () => async (code, { config }) => {
      return (window as any).ts.transpile(code, {
        target: 'es2020',
        jsx: 'react',
        ...getLanguageCustomSettings('typescript', config),
      });
    },
  },
  
  // File extensions
  extensions: ['ts', 'mts', 'typescript'],
  
  // Which editor pane (markup, style, or script)
  editor: 'script',
  
  // Editor support
  editorSupport: {
    codemirror: {
      languageSupport: async () => {
        const { javascript } = await import('@codemirror/lang-javascript');
        return javascript({ typescript: true });
      },
    },
  },
  
  // Prettier formatter
  formatter: {
    prettier: {
      name: 'babel-ts',
      pluginUrls: [parserPlugins.babel, parserPlugins.html],
    },
  },
};

Required Properties

Property	Type	Description
`name`	`string`	Unique identifier (lowercase, no spaces)
`title`	`string`	Short display name
`compiler`	`object`	Compiler configuration
`extensions`	`string[]`	File extensions
`editor`	`'markup' \| 'style' \| 'script'`	Target editor pane

Optional Properties

Property	Type	Description
`longTitle`	`string`	Full language name
`editorSupport`	`object`	Syntax highlighting, autocomplete
`formatter`	`object`	Code formatting configuration
`preset`	`string`	Base language to extend

Compiler Configuration

Compiler Factory Pattern

The compiler.factory function returns the actual compiler function:

compiler: {
  // URL to load compiler library
  url: 'https://cdn.jsdelivr.net/npm/sass@1.70.0/dist/sass.js',
  
  // Factory returns async compiler function
  factory: () => async (code, { config, language }) => {
    // Wait for library to load
    await waitForLibrary('Sass');
    
    // Compile code
    const result = (window as any).Sass.compile(code, {
      syntax: language === 'sass' ? 'indented' : 'scss',
    });
    
    // Return compiled code or CompileResult object
    return result.css;
  },
}

CompileResult Object

Compilers can return enhanced results:

interface CompileResult {
  code: string;           // Compiled code
  info?: CompileInfo;     // Additional metadata
}

interface CompileInfo {
  importedContent?: string[];  // Imported files
  cssModules?: { [key: string]: string };  // CSS module mappings
  sourceMap?: any;        // Source map
}

Example:

factory: () => async (code, { config }) => {
  const result = await compiler.compile(code);
  
  return {
    code: result.output,
    info: {
      importedContent: result.imports,
      sourceMap: result.map,
    },
  };
}

Creating a Custom Compiler

Example: Adding CoffeeScript Support

Let’s create a complete CoffeeScript compiler:

Create language directory

mkdir src/livecodes/languages/coffeescript
touch src/livecodes/languages/coffeescript/lang-coffeescript.ts

Define language specification

// src/livecodes/languages/coffeescript/lang-coffeescript.ts
import type { LanguageSpecs } from '../../models';
import { getLanguageCustomSettings } from '../../utils';

export const coffeescript: LanguageSpecs = {
  name: 'coffeescript',
  title: 'Coffee',
  longTitle: 'CoffeeScript',
  
  compiler: {
    url: 'https://cdn.jsdelivr.net/npm/coffeescript@2.7.0/lib/coffeescript-browser-compiler-legacy/coffeescript.js',
    
    factory: () => async (code, { config }) => {
      // Access global CoffeeScript object
      const CoffeeScript = (window as any).CoffeeScript;
      
      if (!CoffeeScript) {
        throw new Error('CoffeeScript compiler not loaded');
      }
      
      try {
        // Get custom settings from config
        const customSettings = getLanguageCustomSettings('coffeescript', config);
        
        // Compile to JavaScript
        const result = CoffeeScript.compile(code, {
          bare: true,  // Don't wrap in function
          ...customSettings,
        });
        
        return result;
      } catch (error) {
        throw new Error(`CoffeeScript compilation failed: ${error.message}`);
      }
    },
  },
  
  extensions: ['coffee', 'coffeescript'],
  editor: 'script',
  
  editorSupport: {
    codemirror: {
      // CoffeeScript syntax highlighting
      languageSupport: async () => {
        const { StreamLanguage } = await import('@codemirror/language');
        const { coffeescript } = await import('@codemirror/legacy-modes/mode/coffeescript');
        return StreamLanguage.define(coffeescript);
      },
    },
  },
};

// src/livecodes/languages/index.ts
import { coffeescript } from './coffeescript/lang-coffeescript';

export const languages: LanguageSpecs[] = [
  // ... existing languages
  coffeescript,
];

Add to build system

// Ensure the language is included in builds
export { coffeescript } from './coffeescript/lang-coffeescript';

Advanced: Async Compilation

For compilers that use Workers or async operations:

compiler: {
  url: 'https://cdn.jsdelivr.net/npm/some-compiler@1.0.0/dist/compiler.js',
  
  factory: () => async (code, { config }) => {
    const compiler = (window as any).SomeCompiler;
    
    // Initialize compiler (one-time setup)
    if (!compiler.initialized) {
      await compiler.init();
      compiler.initialized = true;
    }
    
    // Compile asynchronously
    const result = await compiler.compileAsync(code, {
      mode: config.customSettings?.someCompiler?.mode || 'production',
    });
    
    return {
      code: result.output,
      info: {
        sourceMap: result.map,
      },
    };
  },
}

Custom Settings

Language-Specific Configuration

Users can provide custom compiler settings:

// In user config
config.customSettings = {
  typescript: {
    target: 'es2022',
    strict: true,
    jsx: 'react-jsx',
  },
};

Accessing Custom Settings

import { getLanguageCustomSettings } from '../../utils';

factory: () => async (code, { config }) => {
  const customSettings = getLanguageCustomSettings('typescript', config);
  
  return compiler.compile(code, {
    ...defaultOptions,
    ...customSettings,  // User overrides
  });
}

Editor Integration

Syntax Highlighting (CodeMirror)

editorSupport: {
  codemirror: {
    languageSupport: async () => {
      // Import language mode
      const { python } = await import('@codemirror/lang-python');
      return python();
    },
  },
}

Monaco Editor

editorSupport: {
  monaco: {
    language: 'python',
    // Optional: custom monarch tokenizer
    monarchTokenProvider: async () => {
      return {
        tokenizer: {
          root: [
            [/\bdef\b/, 'keyword'],
            [/\b\d+\b/, 'number'],
          ],
        },
      };
    },
  },
}

TypeScript Definitions

Provide autocomplete for libraries:

editorSupport: {
  compilerOptions: {
    checkJs: true,
    lib: ['ES2022', 'DOM'],
  },
  types: {
    // Auto-load type definitions
    'react': 'https://esm.sh/v135/@types/react@18.2.0/index.d.ts',
  },
}

Processors

Processors run after compilation to transform output:

Creating a Processor

export const autoprefixer: ProcessorSpecs = {
  name: 'autoprefixer',
  title: 'Autoprefixer',
  
  editor: 'style',  // Processes CSS output
  
  compiler: {
    url: 'https://cdn.jsdelivr.net/npm/autoprefixer@10.4.14/dist/autoprefixer.js',
    
    factory: () => async (code, { config }) => {
      const autoprefixer = (window as any).autoprefixer;
      const postcss = (window as any).postcss;
      
      const result = await postcss([autoprefixer]).process(code, {
        from: undefined,
      });
      
      return result.css;
    },
  },
  
  isPostcssPlugin: true,  // Special flag for PostCSS plugins
};

Enabling Processors

Users enable processors in config:

config.processors = ['autoprefixer', 'postcss'];

Testing Your Compiler

Unit Tests

// src/livecodes/languages/coffeescript/__tests__/lang-coffeescript.spec.ts
import { coffeescript } from '../lang-coffeescript';
import { createCompiler } from '../../../compiler/create-compiler';

describe('CoffeeScript Compiler', () => {
  test('compiles basic syntax', async () => {
    const compiler = await createCompiler({
      config: mockConfig,
      baseUrl: 'http://localhost',
    });
    
    const code = 'square = (x) -> x * x';
    const result = await compiler.compile(code, 'coffeescript', mockConfig, {});
    
    expect(result.code).toContain('function');
    expect(result.code).toContain('square');
  });
  
  test('handles syntax errors', async () => {
    const compiler = await createCompiler({ ... });
    const invalidCode = 'square = (x) ->';
    
    await expect(
      compiler.compile(invalidCode, 'coffeescript', mockConfig, {})
    ).rejects.toThrow('compilation failed');
  });
});

Integration Testing

Test in the live playground:

// Create test project
const testProject = {
  title: 'CoffeeScript Test',
  script: {
    language: 'coffeescript',
    content: `
      greet = (name) ->
        console.log "Hello, #{name}!"
      
      greet 'World'
    `,
  },
};

// Load in playground
createPlayground('#container', { config: testProject });

Distributing Custom Compilers

As NPM Package

{
  "name": "livecodes-lang-mylang",
  "version": "1.0.0",
  "main": "dist/index.js",
  "types": "dist/index.d.ts",
  "peerDependencies": {
    "livecodes": "^0.48.0"
  }
}

Usage

import { createPlayground } from 'livecodes';
import { mylang } from 'livecodes-lang-mylang';

// Register custom language
window.deps = window.deps || {};
window.deps.languages = window.deps.languages || [];
window.deps.languages.push(mylang);

createPlayground('#container', { ... });

Real-World Examples

TypeScript Compiler

File: src/livecodes/languages/typescript/lang-typescript.ts Features:

Custom JSX runtime detection
Custom settings support
TypeScript API integration

SCSS Compiler

File: src/livecodes/languages/scss/lang-scss.ts Features:

Sass library integration
Import resolution
Source maps

Python Compiler (Pyodide)

File: src/livecodes/languages/python/lang-python.ts Features:

WebAssembly execution
Package installation
Async initialization

Troubleshooting

Compiler library not loading

Check:

URL is accessible (test in browser)
Library exposes expected global variable
CORS headers allow loading
Use await waitForLibrary('LibraryName')

Compilation errors not showing

Wrap compiler in try/catch:

try {
  return compiler.compile(code);
} catch (error) {
  throw new Error(`Compilation failed: ${error.message}`);
}

Syntax highlighting not working

Verify:

CodeMirror mode is imported correctly
Mode name matches registration
Check browser console for import errors

Next Steps

Services Architecture

How compilers run in sandbox

Security Model

Compiler isolation security

Performance

Optimize compilation speed

Language Reference

See all built-in languages

Get Started

Features

SDK

Languages

Configuration

Advanced

Resources

​Compiler Architecture

​Language Specification

​Basic Structure

​Required Properties

​Optional Properties

​Compiler Configuration

​Compiler Factory Pattern

​CompileResult Object

​Creating a Custom Compiler

​Example: Adding CoffeeScript Support

​Advanced: Async Compilation

​Custom Settings

​Language-Specific Configuration

​Accessing Custom Settings

​Editor Integration

​Syntax Highlighting (CodeMirror)

​Monaco Editor

​TypeScript Definitions

​Processors

​Creating a Processor

​Enabling Processors

​Testing Your Compiler

​Unit Tests

​Integration Testing

​Distributing Custom Compilers

​As NPM Package

​Usage

​Real-World Examples

​TypeScript Compiler

​SCSS Compiler

​Python Compiler (Pyodide)

​Troubleshooting

​Next Steps

Services Architecture

Security Model

Performance

Language Reference

Compiler Architecture

Language Specification

Basic Structure

Required Properties

Optional Properties

Compiler Configuration

Compiler Factory Pattern

CompileResult Object

Creating a Custom Compiler

Example: Adding CoffeeScript Support

Advanced: Async Compilation

Custom Settings

Language-Specific Configuration

Accessing Custom Settings

Editor Integration

Syntax Highlighting (CodeMirror)

Monaco Editor

TypeScript Definitions

Processors

Creating a Processor

Enabling Processors

Testing Your Compiler

Unit Tests

Integration Testing

Distributing Custom Compilers

As NPM Package

Usage

Real-World Examples

TypeScript Compiler

SCSS Compiler

Python Compiler (Pyodide)

Troubleshooting

Next Steps