Overview

Relevant source files

• README.md
• ast.png
• map.png

Purpose and Scope

This document provides an overview of the tiny-compiler system, a minimal implementation designed to demonstrate fundamental compiler principles through a simple but complete compilation pipeline. The system transforms Lisp-like syntax into JavaScript-like output using approximately 200 lines of code, making it an ideal educational tool for understanding compilation concepts.

For detailed information about individual pipeline components, see Core Components. For architectural patterns and design decisions, see Architecture. For project setup and development workflow, see Development Environment.

System Description

The tiny-compiler implements a classic compilation pipeline that processes source code through five distinct stages: tokenization, parsing, AST traversal, transformation, and code generation. The system transforms mathematical expressions from Lisp-style prefix notation to JavaScript-style function calls.

Input and Output Example

The compiler transforms expressions like:

Into JavaScript-compatible output:

Compilation Pipeline

The system implements a traditional compilation pipeline where each stage transforms data into increasingly structured representations.

Pipeline Overview

Sources: README.md8-88

Data Transformation Flow

Sources: README.md11-88

Core Pipeline Stages

Tokenizer (Lexical Analysis)

The tokenizer() function performs lexical analysis, converting raw source strings into structured token arrays. Each token contains a type (Paren, Name, Number) and corresponding value.

Parser (Syntax Analysis)

The parser() function builds Abstract Syntax Trees from token streams, creating hierarchical representations with Program, CallExpression, and NumberLiteral nodes.

Traverser (Tree Navigation)

The traverser implements the visitor pattern for AST navigation, enabling systematic tree traversal operations required by the transformation stage.

Transformer (AST Transformation)

The transformer() function converts the original AST structure into a target-ready format, wrapping call expressions in ExpressionStatement nodes where the parent context requires it.

Code Generator (Output Generation)

The codegen() function produces the final executable code by traversing the transformed AST and generating appropriate string representations for each node type.

Compiler Orchestration

The main compiler() function coordinates the entire pipeline, calling each stage in sequence and passing intermediate results between stages.

Sources: README.md8-88

Key Architectural Concepts

AST Node Types

The system defines specific node types for representing different language constructs:

• Program: Root container for all statements
• CallExpression: Function call representations
• NumberLiteral: Numeric value nodes
• ExpressionStatement: Statement wrappers for expressions

Token Types

The tokenizer recognizes three fundamental token types:

• TokenTypes.Paren: Parentheses for grouping
• TokenTypes.Name: Function and identifier names
• TokenTypes.Number: Numeric literals

Transformation Strategy

The transformer adds ExpressionStatement wrappers when parent nodes are not expressions, ensuring proper JavaScript syntax generation in the final output.

Sources: README.md44-87

Educational Value

This implementation serves as a practical introduction to compilation theory, demonstrating how modern tools like ESLint, Webpack, and Vue's template compiler utilize similar AST-based transformations. The system's simplicity makes compiler concepts accessible while maintaining the essential complexity found in production compilers.

Sources: README.md94-98