← portfolio  /  DMCTP
Viewing: RTL
DMCTP · FULL-STACK SOC

A custom hardware-software project, from ISA to shell.

A parameterized multithreaded SoC and Unix-like kernel, developed alongside an LLVM toolchain, cycle-accurate simulator, accelerator, and verification environment.

See RTL → Sources
boot: BL1 ROM loads BL2
[KERNEL] DMCTP kernel initialized
[INIT] slab allocator initialized
[INIT] page allocator initialized
[INIT] vfs initialized
[INIT] traps initialized
[INIT] switch to uart0
dmctp# ps
PID  PPID  STATE    NAME
0    0     running  shell
dmctp# run /bin/utest.elf
RTL

SoC

FIG. 1 · 2026-07-05
2D Mesh NoC (2x2 Grid) Cores Systolic Array Memory+TLB Memory+TLB Cores Systolic Array UART MMIO DMA Engine Storage
Architecture at a glance

Parameterized mesh SoC

Core clusters, memory and TLB clusters, DMA, storage, and UART communicate across the 2D mesh.

Core count, hardware threads, grid dimensions, memory regions, ISA width, and accelerator dimensions are configuration parameters rather than fixed limits.

Live message and data transfers in one simulated mesh configuration.
Running system

Traffic across the mesh

The simulation view connects the structural diagram to actual activity across cores, memories, and attached devices.

The same organization scales through configuration rather than a fixed physical topology.

Core study

Latency hiding and throughput

IPC · PORTS · UTILIZATION

These cores are designed to be inexpensive. There is no branch predictor, out of order execution, or superscalar instruction issue. The microarchitecture is best described as a barrel processor. It keeps several hardware thread contexts and selects a ready thread each cycle. When one thread stalls, another can use the pipeline. This hides latency without spending hardware to predict or avoid it. The same broad principle appears when a CUDA streaming multiprocessor selects another ready warp.

This study measures instructions per cycle (IPC), retirement behavior, and utilization of the major shared resources. The main limits are one 64 bit instruction port and one 64 bit data port.

Reproduce the study

The benchmark is provided as source, and the complete interactive report contains the sweep data and utilization timelines.

Download workload.c Open report → Download report
2.00 IPCTheoretical fetch ceiling
1.8589 IPCMeasured peak at 16 threads
32 cyclesProbe sampling interval

Workload model: one program, many thread contexts

The benchmark uses the SPMD model, meaning single program, multiple data. Every hardware thread runs the same program, obtains its own thread ID, and operates on a distinct row of packet data. SPMD describes the software execution model. It is not the same as one SIMD instruction operating on several packed elements.

The fetch ceiling

Instructions are fixed at 32 bits while the instruction memory port is 64 bits wide, so one accepted fetch can carry at most two adjacent instructions. Under an ideal backend and perfectly useful sequential fetches (and no instruction cache inside the core), the architectural upper bound is therefore 2 IPC. Branches, redirects, dependencies, finite queues, and memory contention can only reduce the sustained result.

Line chart showing IPC increasing from one through sixteen threads, peaking at 1.8589 IPC, then declining at thirty-two threads.
Thread count sweep. Throughput rises from 0.1591 IPC with one thread to 1.8589 IPC with sixteen, demonstrating that thread switching successfully fills otherwise idle pipeline slots.
Why does IPC fall from 1.8589 at 16 threads to 1.5859 at 32?
Timeline showing sparse instruction-port and data-port activity with one hardware thread.
One thread
Timeline showing nearly continuous instruction-port activity and increased data-port activity with sixteen hardware threads.
Sixteen threads
Shared port utilization

More threads fill idle pipeline slots

With one thread, dependencies and memory latency leave long idle gaps.

At sixteen threads, ready work keeps instruction traffic almost continuous. Data traffic remains bursty because it follows the program's load and store mix.

Commit rate

IPC is calculated as total committed instructions divided by elapsed cycles. Commit is the architectural retirement point: only instructions that complete and become visible count toward IPC. The heatmap shows the distribution of commits per cycle moving right as thread count increases.

More than one retirement in a cycle does not make this a conventional superscalar core. The extra commits come from different hardware threads completing through different functional units in parallel, rather than multiple independently issued scalar lanes from one thread.

Heatmap showing the distribution of zero, one, two, and higher commits per cycle across the thread-count sweep.
Commit rate distribution. With more runnable contexts, cycles containing two completed instructions become much more common.
Memory banks: Their occupancy traces are included in the full report. Bank behavior is left uninterpreted here because it will be covered in a separate memory system study.

Background reading

Floating point unit

Broad arithmetic, deliberately compact

2 LANES · SHARED FMA · 5 FORMATS

Every core receives an FPU. This makes area efficiency a primary design goal. Two lanes share a common internal representation and reuse the same FMA datapaths across fused and nonfused arithmetic. This keeps the implementation compact while retaining an initiation interval of one cycle for ordinary pipelined operations.

The operation set covers add, subtract, multiply, FMA, FMS, FNMA and FNMS; minimum and maximum; conversions, classification and comparisons; plus divide and square root. Operands and results may be FP32, FP16, BF16, FP8 E4M3 or FP8 E5M2.

Supported formats

FP32 FP16 BF16 FP8 E4M3 FP8 E5M2
5 cyclesPipelined ADD, MUL, FMA, CMP and conversion latency
1 cyclePipelined initiation interval
2 FMA lanesReused by regular and iterative operations

Goldschmidt division and square root

Divide and square root are simple, blocking iterative operations on lane 1. They do not duplicate a large arithmetic datapath: instead, the controller feeds work back through the existing FMA lanes. Lane 1 owns the iterative operation; lane 0 may help when idle, but normal work on lane 0 always takes priority. A long divide or square root therefore never blocks every lane.

Division refinement

A reciprocal seed brings the denominator near one. Each step applies the same correction to numerator and denominator, converging on the quotient.

// q = numerator / denominator
F = reciprocal_seed(denominator);
N = numerator   * F;
D = denominator * F;

repeat(iterations_for(format)) {
    F = 2.0 - D;
    N = N * F;
    D = D * F;
}
q = final_round_and_correct(N);

Square root refinement

An inverse square root seed produces an approximate root and half reciprocal. The shared correction improves both values together.

// root = sqrt(radicand)
y = inverse_sqrt_seed(radicand);
x = radicand * y;
h = 0.5 * y;

repeat(iterations_for(format)) {
    F = 1.5 - (x * h);
    x = x * F;
    h = h * F;
}
root = final_round_and_correct(x);
Full measurement report

FPU latency and initiation interval

Pipeline cases, iterative latency, format scaling, and lane blocking measurements.

Open report → Download
Systolic array study

Tiling for reuse, not array size

8×8 ARRAY · 256×256 GEMM · INT8

This systolic array targets tiled matrix multiplication with comparatively small physical arrays instead of scaling one array to the full problem. The architectural focus is reducing memory bandwidth: keep an 8×8 compute fabric busy, reuse tiles close to it, and avoid repeatedly moving the same operands and partial results through the memory system.

Native mode exposes individual 8×8 tile operations and leaves sequencing to software. Dense mode accepts the matrix dimensions and moves the same tiled traversal and reuse policy into hardware. Native remains useful for flexible or sparse schedules. Dense mode is the high reuse path for regular GEMM.

Full counter report

The interactive report includes workload source, accelerator and MAC cycles, stalls, memory traffic, bandwidth, FIFO occupancy, and sampled timelines.

Open report → Download report
0.128 TOPS8×8 INT8 peak at 1 GHz
99.88%Dense 256×256 peak compute usage
70.74%Native 256×256 peak compute usage
Performance table comparing native and dense 8 by 8 and 256 by 256 INT8 matrix multiplication workloads.
Workload comparison. Select the table to inspect it at full resolution.
Counter chart comparing MAC cycles, stalls, A and B loads, and D stores across native and dense workloads.
Dense 256×256 nearly eliminates stalls and sharply reduces A traffic.
Native: 532,480 moves
Dense: 270,592 moves 1.97× less movement

A matrix traffic

262,144 → 8,192 reads

Exactly 32× fewer reads through hardware tile reuse.

B and output traffic

B reads fall from 262,144 to 254,208. Output remains at 8,192 writes in both modes.

Open question

Why is B matrix traffic not reduced as significantly as A matrix traffic?

#define TILE 8

for (int ii = 0; ii < M; ii += TILE) {
    for (int jj = 0; jj < N; jj += TILE) {
        for (int kk = 0; kk < K; kk += TILE) {
            bool start = (kk == 0);
            bool last  = (kk + TILE == K);

            // C[ii:ii+8][jj:jj+8] += A[ii:ii+8][kk:kk+8]
            //                           × B[kk:kk+8][jj:jj+8]
            sysarr_native_tile(&A[ii][kk], &B[kk][jj], start, last);
        }
        sysarr_writeback_tile(&C[ii][jj]);
    }
}
Native mode

Tiling in software

Software walks output and reduction tiles explicitly. The first reduction initializes the accumulator, the last completes the output tile, and software then writes it back.

Dense mode moves this traversal and operand reuse into hardware.

Peak throughput

128 operations per cycle

2 × 8 × 8 operations from 64 MAC cells. At 1 GHz this is 0.128 TOPS for INT8 or 0.128 TFLOPS for floating point.

Dense reaches 99.88% of peak. Native reaches 70.74%.

Side clusters

Implemented peripherals

WATCHDOG · MTIME · UART

Watchdog

A lockable watchdog with a protected key sequence, configurable timeout and grace periods, interrupt and reset outputs, reset cause state, and MMIO access controlled by privilege.

Machine timer

A machine mode mtime counter and mtimecmp comparator provide the architectural timer interrupt used by the kernel and scheduler.

UART

The UART supplies the boot console and interactive shell through MMIO, with buffered transmit and receive paths integrated into the side cluster interconnect.

RTL cross-links

The SoC RTL described in the RTL section based on the UART path that was developed in UART and based on the accelerator family that appears in Systolic Array.

Toolchain & Simulator · 2026-07-05

Compiler

The platform features a toolchain backend built on top of the LLVM compiler infrastructure. The target ISA closely resembles the RISC-V instruction set architecture (allowing to adapt the existing RISC-V backend for rapid bring up with minimal modification), featuring floating point and integer math, CSR support, and atomics.

Shared Host Tooling

The simulator provides a rich set of host utilities that manipulate ELF files, generate memory hex images, and build the initial filesystem image for the block storage model. These exact host tools are shared directly by the hardware verification environment to load memory arrays and drive automated test scenarios, guaranteeing consistent memory initialization across both the simulator and verification flows.

Cycle-Accurate Simulation & Checkpointing

For performance, the simulator compiles the SystemVerilog SoC into a fast, cycle-accurate model using Verilator. The simulator supports deep object serialization, allowing developers to checkpoint the entire SoC state at any time and instantly restore it later. This eliminates the need to restart OS boot sequences while debugging userspace applications.

Event-Based Verification

In contrast to the cycle-accurate Verilator model used for software development, the hardware verification flow relies on an event-based simulator with full UVM support.

Verification

RTL and ISA validation

FIG. 2 · 2026-07-05
Verification approach: The core verification environment compares committed architectural state against an Instruction Set Simulator (ISS) through a UVM scoreboard. Directed and generated programs exercise instruction behavior at the core boundary. The wider RTL flow also includes property-based formal verification, lint, and structural clock/reset-domain crossing analysis.
top_tb (module) dmctp_test (uvm_test) prog_seq / rand_seq (uvm_sequence) dmctp_env (uvm_environment) ISS Model (C++) via DPI-C core_arch_commit_scb (uvm_scoreboard) core_mon (uvm_monitor) DUT (dmctp_soc)
[2] DMCTP UVM Architecture. The Instruction Set Simulator (ISS) maintains golden architectural state and feeds expected commits to the scoreboard.

UVM Environment

The testbench supports both constrained random testing and executing full compiled ELF programs, including the DMCTP kernel and OS boot flow.

verification/uvm
├── agents
│   └── core
│       └── arch_commit
├── coverage
│   └── core_arch_commit_cov.sv
├── env
│   ├── scb
│   │   └── core_arch_commit_scb.sv
│   └── soc_env.sv
├── filelist.f
├── models
│   ├── core
│   │   ├── dpi
│   │   └── iss
│   └── page_engine
│       └── page_engine_model.sv
├── README.txt
├── scripts
│   ├── run_dsim.sh
│   └── run_uvm.sh
├── sequencers
│   └── soc_seqr.sv
├── tb
│   ├── arch_commit_if.sv
│   ├── core_arch_commit_binder.sv
│   └── soc_tb_top.sv
├── tests
│   ├── base_test.sv
│   └── random
│       ├── gen.py
│       ├── Makefile
│       ├── test.ld
│       └── test.S
├── tools
│   └── sim_uart_dpi.cpp
└── transactions
    └── arch_commit
        ├── arch_commit_txn.sv
        ├── csr_commit_txn.sv
        ├── mm_update_txn.sv
        ├── pc_commit_txn.sv
        └── rf_commit_txn.sv

Instruction Set Simulator

The ISS is a custom C++ reference model of the ISA integrated via DPI-C bindings. When the RTL commits an instruction, the ISS steps alongside it, validating the architectural state.

models/core
├── dpi
│   ├── iss_dpi.cpp
│   └── iss_dpi.sv
└── iss
    ├── include
    │   ├── cpu.h
    │   ├── isa.h
    │   ├── iss.h
    │   ├── memory.h
    │   └── trace.h
    ├── src
    │   ├── cpu.cpp
    │   ├── isa
    │   │   ├── float_isa.cpp
    │   │   ├── integer_isa.cpp
    │   │   ├── isa.cpp
    │   │   ├── load_store_isa.cpp
    │   │   └── system_isa.cpp
    │   ├── iss.cpp
    │   ├── memory.cpp
    │   └── trace.cpp
    └── test
        ├── test_iss
        └── test_main.cpp
Kernel

Unix-Like

Boots into a, Unix like kernel written in C and assembly.

Core Features

The kernel supports a rich set of features including: Virtual File System (VFS) over block storage, Virtual Memory & Memory Management (slab allocation, frame/page mapping, mmap), Process Management (fork, exec, wait, exit, context switching, PID tracking), Inter Process Communication (IPC) (pipes, shared memory, signals), and hardware Trap/Interrupt handling for timer ticks and UART character devices.

Interactive Shell

At the end of the boot sequence, the kernel launches init and a usable interactive shell. The shell supports pipelines, input/output redirection, background jobs, and foreground job control. Commands include help, pwd, cd, ls, lscpu, free, cat, nano, grep, ps, kill, and the usual small file utilities.

Current scope: The kernel includes process and user-thread contexts, ELF loading, virtual memory and faults, VFS and block storage, pipes, shared memory, signals, device I/O, and SMP support. It remains a developing research system rather than a production Unix implementation.

User

Userspace Execution

Userspace binaries running on the simulated SoC, shown at 5× speed.
/bin/test_sys.elf running matrix multiplication on the systolic array at 10× speed.
Systolic array accelerator · 2026-07-24

From userspace to the 8×8 array

64 MAC units accept INT8 or FP8, accumulate with extended internal precision, and emit requantized INT8 output.

The CPU submits commands through /dev/sysarr0. The driver translates them into MMIO and DMA, then wakes the waiting process when the completion interrupt arrives.

Native mode exposes one 8×8 pass. Dense mode performs tiled traversal and reuse in hardware. See the RTL performance study.

Sources · 2026-07-05

Availability

Sources are not yet open.