Tiny Computers GitHub

Real chips,
rebuilt small.

Tiny Computers is a series of small projects that rebuild GPUs, TPUs, CPUs and compilers from scratch. Explore the ideas here, then go deeper with the real projects and source code.

tiny-tpu, running its real RTL in the browser.

Five projects.
One idea.

Choose a project to explore a small interactive explanation, then visit its project site or source code.

  1. tiny-tpu Live

    A real 4×4 systolic array, compiled to WebAssembly.

  2. tiny-gpu Live

    A GPU simulator you can step through, with threads drawn as heatmaps.

  3. tiny-cpu Live visualizer

    A synthesizable RV32I CPU, with the five-stage pipeline shown here.

  4. tiny-attention Live visualizer

    A FlashAttention engine with online softmax, checked bit-for-bit against a golden model.

  5. tiny-compiler Live explorer

    One tiny tensor language, compiled to tiny-gpu or tiny-tpu.

tiny-tpu

A real 4×4 systolic array, compiled to WebAssembly.

tiny-gpu

A GPU simulator you can step through, with threads drawn as heatmaps.

tiny-cpu

A synthesizable RV32I CPU, with the five-stage pipeline shown here.

tiny-attention

A FlashAttention engine with online softmax, checked bit-for-bit against a golden model.

tiny-compiler

One tiny tensor language, compiled to tiny-gpu or tiny-tpu.

From SystemVerilog
to your browser.

The same files, four steps. Choose a step to see what changes between them.

always_ff @(posedge clk or negedge rst_n) begin
  if (!rst_n) begin
    weight_reg <= '0;
    act_out    <= '0;
    psum_out   <= '0;
  end else begin
    if (load_weight)
      weight_reg <= weight_in;
    act_out  <= act_in;
    psum_out <= psum_in + ACC_W'(product);
  end
end

rtl/pe.sv, one multiply-accumulate cell of the array.

Built to be real.

Small enough to understand. Real enough to explore. Each project reveals a different layer of computing.

Small on purpose

Each project is cut down until the whole design fits in your head: a 4×4 array, a short instruction set, one pipeline. Nothing is there to look impressive.

Clear about what is real

The project sites show the real implementations. The interactive sketches here simplify ideas and are labeled accordingly.

Readable

Each project has a README that explains the design, and tests you can run. Follow the links to explore the implementation, tests and documentation.

161 KB
tiny-tpu, compiled to WebAssembly
16
multiply-accumulate cells in the array, each one visible