Lazy graph based tensor execution runtime

Why bother?

The purpose of the DSL is not merely to shorten kernel syntax. It moves repetitive implementation work out of every operation and into one compiler.

T = any behaves like a constrained generic type. The compiler monomorphizes the operation into concrete dtype variants such as matrix_mul<int32> and matrix_mul<real32>. Shared symbolic dimensions become runtime tensor-shape bindings, algorithm metadata becomes generated dispatch logic, and operation-count expressions become profiler code.

Without this layer, each operation would need to manually reproduce dtype switches, pointer casts, shape extraction, parameter unpacking, dispatch tables, symbol registration, and profiler metadata.

human-maintained code
        |
        v
compact operation definition
        |
        v
mechanically generated boilerplate

Compiler pipeline

              matrix_mul.op
                    |
                    v
             +-------------+
             |    Lexer    |
             +-------------+
                    |
                    v
        Line / AlgorithmBlock stream
                    |
                    v
             +-------------+
             |   Parser    |
             +-------------+
                    |
                    v
              Operation AST
                    |
                    v
             +-------------+
             |  Validator  |
             +-------------+
                    |
                    v
          ValidatedOperation
                    |
                    v
        +-----------------------+
        | Type specialization   |
        | and C lowering        |
        +-----------------------+
                    |
                    v
              operations.c
                    |
                    v
             clang -emit-llvm
                    |
                    v
              operations.bc
                    |
          +---------+---------+
          |                   |
          v                   v
      LLVM ORC           registry.c
          |                   |
          +---------+---------+
                    |
                    v
             runtime lookup
                    |
                    v
               execution

Frontend · lexical structure

lexer.py performs the first compiler pass. DSL statements are normalized into logical source lines, while algorithm bodies are preserved as complete C blocks. A declaration such as dim M; becomes a lexical line, while an algorithm block becomes an AlgorithmBlock. The lexer deliberately does not interpret the C inside an algorithm; it establishes the outer structure of the operation language.

Parser · syntax to AST

parser.py converts the lexical stream into an abstract syntax tree. Definitions such as type T = any;, dim M;, and in a: T [M, K]; become structured type definitions, dimensions, shapes, values, algorithms, and hints under one Operation node.

Semantic analysis · validation and resolution

validator.py resolves named types and shapes, verifies dimensions, checks that algorithms and hints agree, validates operation IDs and names, and resolves backend selection rules into a ValidatedOperation. This is the compiler boundary between syntactically valid input and semantically valid operation metadata. For matrix_mul, it establishes that all tensors share T and that both occurrences of K describe the same logical dimension.

Specialization · generic types become concrete kernels

The operation compiler enumerates legal assignments for generic type variables. One source algorithm therefore expands into concrete functions for the supported integer and floating-point types before execution.

matrix_mul<T>
      |
      +--> matrix_mul<int8>
      +--> matrix_mul<int16>
      +--> matrix_mul<int32>
      +--> ...
      +--> matrix_mul<real32>
      +--> matrix_mul<real64>

This is compile-time monomorphization: generic source is expanded into concrete functions before runtime.

Lowering · DSL concepts become C machinery

The lowering pass makes implicit DSL information explicit. For an int32 specialization, a declaration such as in a: T [M, K]; becomes a typed input pointer and concrete shape bindings:

const int32 *restrict a =
    (const int32 *)inputs[0];

extent M = input_tensors[0]->shape[0];
extent K = input_tensors[0]->shape[1];

Output pointers, dimensions, tensor ranks, sizes, parameter blocks, algorithm conditions, and operation metadata are generated in the same way. This is why a small .op definition can expand into a much larger C translation unit: the compiler is materializing details that would otherwise be handwritten repeatedly.

LLVM lowering · C becomes SSA-based IR

Clang compiles the generated C into LLVM bitcode. Source variables and loops are represented in LLVM’s SSA-oriented form. The inner matrix accumulation becomes an explicit loop-carried data dependency:

%acc = phi i32 [ 0, %entry ], [ %next_acc, %loop ]
%k   = phi i64 [ 0, %entry ], [ %next_k, %loop ]

%product  = mul i32 ...
%next_acc = add i32 %product, %acc
%next_k   = add i64 %k, 1

The original accumulation, acc += a[i * K + k] * b[k * N + j], is now explicit dataflow that LLVM can analyze and optimize.

Registry generation · compiler output becomes runtime-addressable

The generated registry embeds the combined bitcode module and maps operation identities and dtype signatures to symbols within it. The runtime does not need to understand .op syntax: parsing and semantic validation have already happened during the shared-library build.

Runtime lookup starts from the operation ID, output dtype, and input dtype, then resolves the corresponding function inside the embedded LLVM module.

Build-time language, runtime artifact

LIBRARY BUILD TIME

.op source
   ↓
operation compiler
   ↓
validated / specialized generated code
   ↓
LLVM bitcode + registry

RUNTIME

tensor graph
   ↓
registry lookup
   ↓
LLVM specialization / prepared function
   ↓
execution

The custom language adds compiler structure without requiring the application to parse or interpret DSL source during normal tensor execution.