Real chips,
rebuilt small.
Tiny Computers is a series of small projects that rebuild GPUs, TPUs, CPUs and compilers from scratch. Explore the ideas here, then go deeper with the real projects and source code.
Five projects.
One idea.
Choose a project to explore a small interactive explanation, then visit its project site or source code.
-
tiny-tpu
A real 4×4 systolic array, compiled to WebAssembly.
-
tiny-gpu
A GPU simulator you can step through, with threads drawn as heatmaps.
-
tiny-cpu
A synthesizable RV32I CPU, with the five-stage pipeline shown here.
-
tiny-attention
A FlashAttention engine with online softmax, checked bit-for-bit against a golden model.
-
tiny-compiler
One tiny tensor language, compiled to tiny-gpu or tiny-tpu.
tiny-attention
A FlashAttention engine with online softmax, checked bit-for-bit against a golden model.
From SystemVerilog
to your browser.
The same files, four steps. Choose a step to see what changes between them.
always_ff @(posedge clk or negedge rst_n) begin
if (!rst_n) begin
weight_reg <= '0;
act_out <= '0;
psum_out <= '0;
end else begin
if (load_weight)
weight_reg <= weight_in;
act_out <= act_in;
psum_out <= psum_in + ACC_W'(product);
end
end
rtl/pe.sv, one multiply-accumulate cell of the array.
$ verilator --cc rtl/pe.sv rtl/systolic_array.sv \
rtl/controller.sv rtl/tiny_tpu_top.sv \
--top-module tiny_tpu_top -Wall
Verilated model written to wasm/obj_dir/
The same files, now C++ that any compiler can build.
$ em++ -O3 -std=c++17 bindings.cpp V*.cpp \
verilated.cpp -lembind -s MODULARIZE=1 …
web/public/tiny_tpu.wasm 161 KB (62 KB gzipped)
The whole chip, compiled to WebAssembly and shipped as one file.
Every number here is read from the compiled chip, not drawn by hand.
Built to be real.
Small enough to understand. Real enough to explore. Each project reveals a different layer of computing.
Small on purpose
Each project is cut down until the whole design fits in your head: a 4×4 array, a short instruction set, one pipeline. Nothing is there to look impressive.
Clear about what is real
The project sites show the real implementations. The interactive sketches here simplify ideas and are labeled accordingly.
Readable
Each project has a README that explains the design, and tests you can run. Follow the links to explore the implementation, tests and documentation.
- 161 KB
- tiny-tpu, compiled to WebAssembly
- 16
- multiply-accumulate cells in the array, each one visible
Explore the work behind the ideas.
Visit the organization to explore the source, follow development and contribute.