Skip to content
ichen522Public

About

4-channel AXI4 DMA in SystemVerilog. Verified with Verilator, a DPI-C reference model, SVA, 100% functional coverage, mutation testing and a formal fairness proof. Signed off at 500 MHz in FreePDK45 with DC + PrimeTime.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

1 Commit

Folders and files

Repository files navigation

AXI4 Multi-Channel DMA Engine

A 4-channel memory-to-memory DMA in SystemVerilog — designed, verified (simulation + formal),
signed off at 500 MHz in FreePDK45 with Design Compiler and PrimeTime, and characterised with a bandwidth / area trade-off study.

SystemVerilog AXI4 Verilator Coverage Formal Mutation Sign-off


Highlights

Design 4 channels · AXI4-Lite CSRs with maskable W1C interrupts · AXI4 INCR bursts split at 4 KB · any byte alignment on source and destination (2-word realigner + WSTRB head/tail masking) · up to 8 outstanding reads · round-robin arbitration · SLVERR/DECERR handling with write poisoning
Verification Verilator constrained-random testbench + DPI-C C reference model (golden memory + exact burst-sequence predictor) · 3,820 transfers / 525k AXI bursts checked over 8 seeds · 14 AXI SVA properties · 34 / 34 functional cover points · 3 extra parameter configurations
Formal Round-robin arbiter starvation-freedom proved unbounded (SymbiYosys + ABC PDR) for N = 2, 3, 4, 8
Mutation testing 8 / 8 injected RTL bugs caught — and the first run exposed a real coverage hole (an IRQ event racing a W1C write) that was then closed
Sign-off Synopsys DC Ultra + PrimeTime, OSU gscl45nm (FreePDK45), 500 MHz: setup +16 ps, hold +12 ps, 0 DRV · 52,347 µm² · 30.4 mW (PrimePower, SAIF-annotated)
Analysis Outstanding × burst-length sweep at 32-cycle latency: 2.9 % → 99.1 % bus utilisation; analytical model within ~1 %; 8 × 8-beat reaches 99 % with 4 % less area than 4 × 16

Table of contents

  1. Overview
  2. Architecture
  3. Programming model
  4. Verification
  5. Implementation & sign-off
  6. Performance analysis
  7. Getting started
  8. Repository layout
  9. Reproducing every number
  10. Design decisions & trade-offs
  11. Limitations & roadmap
  12. Lessons learned
  13. Author

1. Overview

A DMA engine moves blocks of memory without the CPU touching every byte. Software programs a source address, a destination address and a length through memory-mapped registers, sets START, and gets an interrupt when the copy is finished (or failed).

This project implements that end to end and goes through the whole front-end flow:

spec ─▶ RTL ─▶ lint ─▶ simulation (random + directed, C reference, SVA, coverage)
                    └▶ formal (arbiter fairness) ─▶ mutation testing
     ─▶ synthesis (DC) ─▶ STA + power (PrimeTime) ─▶ performance / area trade-off study
Feature Detail
Channels 4 by default (parameter NCH, 2–8), independent register sets, round-robin bus sharing at burst granularity
Register interface AXI4-Lite slave, 12-bit address, 32-bit data; AW and W accepted in any order; byte strobes honoured; unmapped address → SLVERR
Memory interface AXI4 master, 32-bit data (parameter DW), INCR bursts of 1 – BL_MAX beats (runtime-selectable per channel), never crossing a 4 KB boundary
Alignment Source and destination may start at any byte and differ in alignment; length is any number of bytes
Outstanding reads Up to MAX_OUTS (runtime register, ≤ OUTS_MAX) read bursts in flight
Interrupts Per-channel done and error bits, W1C status, per-bit enable, single level irq output
Errors RRESP/BRESP = SLVERR or DECERR → error status + interrupt; channel stops issuing and drains; poisoned writes (WSTRB = 0) guarantee bad read data never reaches memory
Shadow registers The next transfer can be programmed while the current one runs; START while busy is ignored

2. Architecture

2.1 Block diagram

flowchart LR
  SW([Software])
  MEM[(AXI4 memory)]
  subgraph DMA[dma_top]
    direction LR
    CSR["dma_csr<br/>registers · IRQ"]
    CH["dma_channel × 4<br/>state + next chunk<br/>(src, dst, n)"]
    subgraph ENG[dma_engine]
      direction LR
      ARB["round-robin<br/>arbiter"]
      ARREG["AR issue<br/>register"]
      RQ[("rq<br/>realign info")]
      AWQ[("awq<br/>AW info")]
      BQ[("bq<br/>channel id")]
      PK["realigning packer<br/>shift + WSTRB"]
      WF[("W buffer")]
    end
  end
  SW -- "AXI4-Lite" --> CSR
  CSR -- irq --> SW
  CSR -- "start / cfg" --> CH
  CH -- "done / error" --> CSR
  CH -- req --> ARB --> ARREG
  ARREG -- "on AR hs" --> RQ & AWQ
  ARREG -- AR --> MEM
  MEM -- R --> PK
  RQ --> PK --> WF -- W --> MEM
  AWQ -- AW --> MEM
  AWQ -- "on AW hs" --> BQ
  MEM -- B --> BQ
  BQ -- "chunk done" --> CH
Loading

2.2 Module hierarchy

dma_top                         top level, parameters, wiring
├── dma_csr        u_csr        AXI4-Lite slave, register file, IRQ logic
├── dma_channel    g_ch[c].u_ch per-channel state, next-chunk computation
└── dma_engine     u_eng        shared AXI4 master datapath
    ├── dma_rr_arb u_arb        round-robin arbiter
    ├── dma_fifo   u_rq         read-return descriptors (realignment control)
    ├── dma_fifo   u_awq        write-address descriptors
    ├── dma_fifo   u_bq         channel id per outstanding write burst
    └── dma_fifo   u_wf         W data buffer {last, strb, data}
File Lines Role
rtl/dma_top.sv 176 Top level
rtl/dma_csr.sv 286 Registers, AXI4-Lite, interrupts
rtl/dma_channel.sv 206 Channel state machine, chunking, address update
rtl/dma_engine.sv 342 AR issue, descriptor queues, realigning packer, AW/W/B
rtl/dma_rr_arb.sv 52 Round-robin arbiter
rtl/dma_fifo.sv 54 Flop-based FWFT FIFO with overflow/underflow assertions

RTL total 1,116 lines; testbench + C model ≈ 1,500 lines; scripts / formal / analysis ≈ 700 lines.

2.3 Top-level interface

Port group Signals Notes
Clock / reset clk, rst_n Single clock; asynchronous active-low reset
s_axil_* awaddr[11:0] awvalid awready wdata[31:0] wstrb[3:0] wvalid wready bresp[1:0] bvalid bready araddr[11:0] arvalid arready rdata[31:0] rresp[1:0] rvalid rready CSR slave
m_axi_ar* / r* arid araddr arlen[7:0] arsize arburst arvalid arready · rid rdata rresp rlast rvalid rready Read master; ID always 0
m_axi_aw* / w* / b* awid awaddr awlen awsize awburst awvalid awready · wdata wstrb wlast wvalid wready · bid bresp bvalid bready Write master; ID always 0
Interrupt irq Level, registered: |(IRQ_STATUS & IRQ_ENABLE)

2.4 Parameters

Parameter Default Range Meaning
NCH 4 2 – 8 Number of channels
AW 32 Address width
DW 32 32, 64, … Data width (bytes per beat BPB = DW/8)
IDW 4 AXI ID width (IDs are driven to 0)
BL_MAX 16 2 – 256, power of 2 Maximum beats per burst
OUTS_MAX 4 ≥ 1 Maximum outstanding read bursts (descriptor queues are 2·OUTS_MAX deep)
DBUF_DEPTH 2·BL_MAX power of 2 W data buffer depth (words)

2.5 Life of a transfer

sequenceDiagram
  participant SW as Software
  participant CSR as dma_csr
  participant CH as dma_channel
  participant ENG as dma_engine
  participant MEM as AXI memory
  SW->>CSR: write SRC, DST, LEN, CTRL{BURST_LOG2, START}
  CSR->>CH: start pulse (copies shadow registers)
  loop until all bytes issued
    CH->>ENG: req + next chunk (src, dst, n)
    ENG->>ENG: round-robin grant, load AR register
    ENG->>MEM: AR (one read burst)
    ENG->>MEM: AW (matching write burst)
    MEM-->>ENG: R beats (in order)
    ENG->>ENG: realign bytes, apply WSTRB head/tail mask
    ENG->>MEM: W beats
    MEM-->>ENG: B
    ENG-->>CH: bdone (pend − 1)
  end
  CH->>CSR: ev_done (or ev_err)
  CSR->>SW: irq
  SW->>CSR: read IRQ_STATUS, write-1-to-clear
Loading

2.6 Chunking: one AR + one AW per chunk

Each channel continuously proposes its next chunk — the largest piece of the remaining transfer that fits in exactly one read burst and one write burst:

n = min( remaining,
         4096 − src[11:0],          4096 − dst[11:0],           // neither side crosses 4 KB
         burst_bytes − src % BPB,   burst_bytes − dst % BPB )   // ≤ BL beats on each side

AR = { src & ~(BPB−1),  ceil((src % BPB + n) / BPB) − 1 }
AW = { dst & ~(BPB−1),  ceil((dst % BPB + n) / BPB) − 1 }

Worked example — src = 0x00FF1, dst = 0x09FFA, len = 200, 16-beat bursts (64 B), 32-bit bus. Both ends are unaligned and both cross a 4 KB page at different points:

# src dst remaining n limited by AR addr / beats AW addr / beats
1 0x00FF1 0x09FFA 200 6 dst 4 KB 0x00FF0 / 2 0x09FF8 / 2
2 0x00FF7 0x0A000 194 9 src 4 KB 0x00FF4 / 3 0x0A000 / 3
3 0x01000 0x0A009 185 63 dst burst length 0x01000 / 16 0x0A008 / 16
4 0x0103F 0x0A048 122 61 src burst length 0x0103C / 16 0x0A048 / 16
5 0x0107C 0x0A085 61 61 remaining 0x0107C / 16 0x0A084 / 16

The C reference model computes this sequence independently and the testbench checks every AR/AW on the bus against it.

2.7 Byte realignment

With source byte offset s and destination offset d (each 0–3), write word j is a 4-byte window of two consecutive read beats:

Case Window First read beat
s > d {R[j+1], R[j]} shifted by s − d bytes only loads prev, emits nothing
s ≤ d {R[j], R[j−1]} shifted by s − d + 4 bytes emits a word

So the packer keeps one register prev, forms win = {current beat, prev} and takes win >> (sel × 8). If the tail bytes are already all in prev after the last read beat, one extra flush word is emitted. Only the first and last write word are masked: WSTRB[l] = (l ≥ d) on the first, (l ≤ (d + n − 1) % 4) on the last. sel and the case bit are computed when the AR is issued and travel in the rq descriptor, so the R → W path is just a byte shifter.

Example — s = 1, d = 3, n = 7 bytes (b0…b6): 2 read beats, 3 write beats, sel = 2.

Step Read beat (lanes 3‥0) Window shifted by 2 B W word (lanes 3‥0) WSTRB
R0 arrives b2 b1 b0 · {R0, –} b0 · – – 1000
R1 arrives b6 b5 b4 b3 {R1, R0} b4 b3 b2 b1 1111
flush — {0, R1} – – b6 b5 0011

2.8 Ordering and completion

All bursts use ID 0, so AXI guarantees R and B return in issue order. Three small FIFOs stay in lock-step with the bus:

  • rq — pushed on AR handshake, popped when the packer finishes that burst's data
  • awq — pushed on AR handshake, popped on AW handshake
  • bq — pushed on AW handshake with the channel id, popped on B → tells the channel one chunk is done

Each channel counts chunks in flight (pend). It finishes when all bytes are issued (or an error stopped it) and pend == 0 — i.e. every write has its B response.

Global limits: rd_cnt < MAX_OUTS (outstanding reads) and infl_cnt < 2·OUTS_MAX (chunks between AR issue and B) gate new issues, which also guarantees the descriptor queues can never overflow (checked by assertions in dma_fifo).

2.9 Error handling

Event Action
RRESP = SLVERR / DECERR Channel ERR + RD_ERR + response code; that beat and every later W beat of the channel are sent with WSTRB = 0
BRESP = SLVERR / DECERR Channel ERR + WR_ERR + response code
After an error Channel stops issuing new chunks, waits for in-flight ones (AXI requires every AW to get its W beats), then raises the error interrupt instead of done
Other channels Unaffected (tested: a healthy channel runs alongside all four error types)
Recovery A new START clears the status and the channel works normally (tested)

2.10 Timing-driven microarchitecture

Three changes came out of timing closure (details in §5.6):

// (1) chunk size is registered; a channel skips one request cycle after each grant
always_ff @(posedge clk) begin
  n_q   <= n_comb;                 // 5-way min computed from the current state
  n_vld <= !(grant || load);       // stale for the cycle after a state change
end
assign req = busy_q && n_vld && (rem_q != 0) && !err_q;

// (2) carry-select address update: n < 2^13, so only a 13-bit adder is needed
wire [13:0]    src_lo  = {1'b0, src_q[12:0]} + {1'b0, n_q};
wire [AW-14:0] src_hi1 = src_q[AW-1:13] + 1'b1;               // in parallel
wire [AW-1:0]  src_nx  = {src_lo[13] ? src_hi1 : src_q[AW-1:13], src_lo[12:0]};

// (3) adder-free burst limit: burst_bytes is a power of two ≥ BPB, offset < BPB
assign mx_off = (s_off > d_off) ? s_off : d_off;
assign lim_bl = (mx_off == '0) ? blb_q : {blbm_q, OFFW'(~mx_off + 1'b1)};

3. Programming model

3.1 Register map

Offset Name Access Bits
0x000 ID RO 0x444D4131 ("DMA1")
0x004 HWCFG RO [31:24] OUTS_MAX · [23:16] log2(BL_MAX) · [15:8] bytes/beat · [7:0] NCH
0x008 IRQ_STATUS W1C bit 2c = channel c done · bit 2c+1 = channel c error
0x00C IRQ_ENABLE RW same layout; 0 = masked
0x010 MAX_OUTS RW outstanding read bursts; writes clamped to 1 … OUTS_MAX (reset = OUTS_MAX)
0x100 + 0x20·c SRC RW source byte address (any alignment)
+0x04 DST RW destination byte address (any alignment)
+0x08 LEN RW length in bytes; 0 completes immediately
+0x0C CTRL RW [0] START (write 1, self-clearing, reads 0) · [7:4] BURST_LOG2 (beats = 2ⁿ, clamped to BL_MAX)
+0x10 STATUS RO [0] BUSY · [1] DONE · [2] ERR · [3] RD_ERR · [4] WR_ERR · [9:8] RESP of the error
+0x14 REMAIN RO bytes not yet issued

Any other address returns SLVERR (reads return 0).

3.2 Example driver

#define DMA          ((volatile uint32_t *)DMA_BASE)
#define REG(off)     DMA[(off) >> 2]
#define CH(c, off)   REG(0x100 + 0x20 * (c) + (off))

void dma_init(void) {
    REG(0x00C) = 0xFF;          // IRQ_ENABLE: done + error for 4 channels
    REG(0x010) = 4;             // MAX_OUTS
}

void dma_copy(int c, uint32_t src, uint32_t dst, uint32_t len) {
    CH(c, 0x00) = src;
    CH(c, 0x04) = dst;
    CH(c, 0x08) = len;
    CH(c, 0x0C) = (4u << 4) | 1u;   // 16-beat bursts, START
}

void dma_isr(void) {
    uint32_t st = REG(0x008);
    REG(0x008) = st;                 // write-1-to-clear what we are about to handle
    for (int c = 0; c < 4; c++) {
        if (st & (1u << (2 * c)))      on_done(c);
        if (st & (2u << (2 * c))) {
            uint32_t s = CH(c, 0x10);  // STATUS: RD_ERR/WR_ERR, RESP = (s >> 8) & 3
            on_error(c, s);
        }
    }
}

3.3 Rules

  • SRC/DST/LEN/CTRL are shadow registers: the channel copies them on START, so the next transfer can be programmed while one is running. START while BUSY is ignored.
  • An event and a W1C in the same cycle: set wins — no interrupt is ever lost (this exact case is covered by a directed test, see §4.4).
  • Change MAX_OUTS only while all channels are idle.
  • Overlapping source/destination regions (within or across channels) are undefined.

4. Verification

4.1 Environment

               ┌──────────────────────────── tb_top.sv ────────────────────────────┐
  directed  ─▶ │ AXI4-Lite BFM          ┌───────────┐  AXI4   ┌────────────────────┐ │
  + random     │ (random AW/W order, ──▶│  dma_top  │ ══════▶ │  axi_mem_model.sv  │ │
  tests        │  random delays)        └───────────┘         │  random READY,     │ │
               │                          │     │             │  bubbles, latency, │ │
               │      axi_checker.sv (SVA)┘     └ dma_cov.sv  │  SLVERR / DECERR   │ │
               │                                              └────────────────────┘ │
               │  every AR/AW ─▶ DPI-C  tb/ref/dma_ref.c  ◀─ completed transfers      │
               │                 • burst planner: predicted (addr, len) per channel  │
               │                 • golden memory: memmove replay, byte compare       │
               └────────────────────────────────────────────────────────────────────┘
Component What it does
tb/axi_mem_model.sv 256 KB AXI4 slave, unlimited outstanding, in-order. Per round randomises ARREADY / AWREADY / WREADY probability (30–100 %), RVALID / BVALID bubbles (0–40 %), read and write latency (1–45 cycles). 0x4000_0000 reads and 0x5000_0000 writes return SLVERR; anything outside memory returns DECERR. Accepts W before its AW.
tb/ref/dma_ref.c C reference over DPI-C, written from the spec independently of the RTL. (1) Predicts each channel's exact AR and AW burst sequence → the testbench matches every burst on the bus. (2) Keeps a golden memory image; each completed transfer is replayed with memmove; the destination is compared after every transfer and the whole 256 KB at the end.
tb/axi_checker.sv AXI protocol assertions (below).
tb/dma_cov.sv 34 SVA cover property points (below).
tb/tb_top.sv AXI4-Lite BFM, interrupt-driven completion handling, directed tests, random regression, performance mode. TB drives and samples on the negative edge to avoid races with the DUT.
tb/sim_main.cpp Verilator harness (timing mode) that writes coverage.dat.

4.2 Checks

AXI SVA (axi_checker.sv)

Property Checks
ap_ar_hold ap_aw_hold ap_w_hold ap_r_hold ap_b_hold VALID stays high and payload stays stable until READY (all five channels)
ap_ar_4k ap_aw_4k No burst crosses a 4 KB boundary
ap_ar_fmt ap_aw_fmt INCR, full-width size, aligned address, len < BL_MAX
ap_wlast_beat WLAST on exactly beat AWLEN (works whether W arrives before or after AW)
ap_w_maxlen No W burst longer than BL_MAX without WLAST
ap_rlast, ap_r_unexpected RLAST matches ARLEN; no R without an outstanding AR
ap_outs_limit Outstanding read bursts never exceed MAX_OUTS
ap_no_overflow ap_no_underflow (RTL) Descriptor / data FIFOs never overflow or underflow

Per transfer: STATUS bits and RESP code, REMAIN = 0 on success, all planned bursts issued, destination bytes equal the golden image. At the end: whole-memory compare, zero protocol errors seen by the memory model.

Directed tests

Test Covers
t_csr ID/HWCFG, byte-strobe writes, unmapped → SLVERR, MAX_OUTS clamp, CTRL read-back
t_zero_len LEN = 0 completes with no bus traffic
t_basic Aligned copy on every channel
t_4k 4 KB crossings on src, dst, both, at different points
t_align_sweep 4 src offsets × 4 dst offsets × 13 lengths (1…65 B)
t_burst_sweep Every burst length 1 … BL_MAX
t_errors RD SLVERR, RD DECERR, WR SLVERR, WR DECERR alongside a healthy channel; destination untouched on read errors; recovery afterwards
t_irq_mask Pending-while-masked, unmask → irq, W1C → irq low
t_irq_race Completion events landing in the same cycle as W1C writes
t_shadow Reprogram + START while busy (ignored), then run the shadowed transfer
random rounds Random MAX_OUTS and memory behaviour, 1–4 concurrent channels, 1 B – 12 KB, random offsets / burst sizes, ~8 % error transfers

4.3 Results

Regression (make regress): 8 seeds × (directed suite + 60 random rounds), all PASS.

Total over 8 seeds
Successful transfers 3,667
Expected-error transfers 153
AR bursts matched against the C planner 525,291
Simulated cycles 5.9 M
Per-seed results
seed 1: PASS  transfers ok=457  expected-errors=22  AR=51470 AW=51470  R beats=108659 W beats=108670
seed 2: PASS  transfers ok=463  expected-errors=19  AR=78625 AW=78625  R beats=148135 W beats=148129
seed 3: PASS  transfers ok=443  expected-errors=18  AR=52044 AW=52044  R beats=114532 W beats=114527
seed 4: PASS  transfers ok=461  expected-errors=19  AR=60719 AW=60719  R beats=136717 W beats=136733
seed 5: PASS  transfers ok=461  expected-errors=21  AR=88218 AW=88218  R beats=152454 W beats=152455
seed 6: PASS  transfers ok=466  expected-errors=20  AR=51921 AW=51921  R beats=122710 W beats=122709
seed 7: PASS  transfers ok=473  expected-errors=17  AR=73290 AW=73290  R beats=161925 W beats=161915
seed 8: PASS  transfers ok=443  expected-errors=17  AR=69004 AW=69004  R beats=142505 W beats=142512

(W beats can exceed R beats by the realignment flush words.)

Other parameter configurations (scripts/run_configs.sh, 3 seeds each, all PASS):

NCH BL_MAX OUTS_MAX Coverage
2 2 1 27 / 27 (1 point waived: W buffer can never fill — unreachable by construction)
3 64 8 34 / 34
4 256 2 38 / 38

Functional coverage (8 seeds merged — results/coverage_report.txt): 34 / 34

Group Cover points Min hits
Per channel cp_done[ch0‥3], cp_error[ch0‥3] 25
4 KB split was the limiting factor cp_split4k 1,026
Alignment head / tail / both partial WSTRB, realign s>d, s<d, tail flush, poisoned beat 415
Burst length 1 / 2 / 4 / 8 / 16 beats 11,067
Error responses RRESP SLVERR / DECERR, BRESP SLVERR / DECERR 110
Concurrency all channels busy, outstanding at hardware max 133,084
Back-pressure AR / AW / W stalled by slave, R stalled by DMA 147,127
Interrupts pending while masked, W1C clear, event racing W1C 76

4.4 Mutation testing

Coverage only says the scenarios you thought of were exercised. To show the checkers actually fail on bugs, make mutants injects one bug at a time into a copy of the RTL and requires the run to fail:

Mutant Bug Caught by First message
no_src_4k_split source-side 4 KB limit removed SVA ap_ar_4k u_chk.ap_ar_4k
wlast_off_by_one WLAST missing on 1-beat bursts memory-model WLAST check / SVA u_mem … WLAST mismatch
head_strobe_unmasked head bytes written when s = d C reference memory compare final memory compare: 65 bytes differ
no_error_poisoning read-error data written to memory C reference memory compare final memory compare: 6 bytes differ
outstanding_off_by_one < → <= on the outstanding limit SVA ap_outs_limit u_chk.ap_outs_limit
arvalid_dropped ARVALID dropped before ARREADY SVA ap_ar_hold u_chk.ap_ar_hold
irq_event_lost_on_w1c event lost when it coincides with W1C t_irq_race irq_race 4: ch0 DONE lost
arbiter_fixed_priority arbiter pointer never advances formal bounded-wait assertion arb_fairness.sv:78

The first run caught only 7 / 8. The IRQ mutant escaped: random traffic never lined a completion event up with the exact cycle of a W1C write, and no cover point measured it. That was a real coverage hole. Adding cp_irq_ev_on_w1c and the t_irq_race directed test (48 short transfers while hammering IRQ_STATUS with W1C writes) hit the race 76 times and the mutant is now caught.

4.5 Formal: arbiter starvation-freedom

formal/arb_fairness.sv + arb_fairness.sby (SymbiYosys). The environment may drive req and en arbitrarily, except that a request is held until it is granted.

Liveness ("eventually granted") is replaced by a stronger, provable safety property — bounded wait: while channel i is waiting, at most N − 1 grants go to other channels. An inductive invariant makes the proof go through:

wait_cnt[i] + dist(ptr → i) ≤ N − 1

(every grant to someone else moves the priority pointer at least one step closer to i).

Task Engine Result
prove_n2 prove_n3 prove_n4 prove_n8 ABC PDR (unbounded) PASS
cover_n4 smtbmc z3 bound N − 1 reached in 5–6 steps → the bound is tight
bmc_n4 smtbmc z3, depth 12 PASS (and FAIL with a trace for the fixed-priority mutant)

Also proved: grant is one-hot, only to requesters, and work-conserving.


5. Implementation & sign-off

Step-by-step guide for the USC servers: syn/RUN_DC_PT.md · full report: results/DC_PT_RESULTS.md.

5.1 Flow and constraints

Library FreePDK45, OSU gscl45nm (typical, 1.1 V); the library's seven max_capacitance : 0 pins are removed
Synthesis Design Compiler Ultra Y-2026.03, compile_ultra -no_autoungroup (hierarchy kept)
STA / power PrimeTime Y-2026.03-SP2, PrimePower (averaged, SAIF)
Clock 2.0 ns (500 MHz), 0.1 ns uncertainty, ideal (pre-CTS)
I/O input / output delay 20 % of the period, INVX1 drive, INVX4 input-pin load
Reset rst_n ideal network + false path
Cross-check Yosys 0.33 + ABC + OpenSTA with identical constraints (make syn)

5.2 Results

Flow Setup WNS Hold WNS Cell area Flops Cells Power
DC + PrimeTime +0.016 ns +0.012 ns 52,346.7 µm² 2,810 13,956 30.4 mW
Yosys + ABC + OpenSTA +0.342 ns +0.009 ns 62,773.6 µm² 2,894 16,879 39.1 mW (vectorless)

0 setup / 0 hold violations, 0 max-transition / max-capacitance violations. DC and PT setup slack agree to < 1 ps. With identical constraints DC turns the spare slack into 16.6 % less area (area recovery).

5.3 Area by block (DC)

Block Area (µm²) Share
u_eng/u_wf — W data buffer (37 b × 32) 18,692 35.7 %
u_csr — registers 9,502 18.2 %
u_eng/u_awq — AW descriptors (38 b × 8) 4,636 8.9 %
u_eng/u_rq — realign descriptors (18 b × 8) 2,436 4.7 %
u_eng/u_bq — channel ids (2 b × 8) 452 0.9 %
u_eng/u_arb — arbiter 87 0.2 %
g_ch[c].u_ch — each channel (×4) 3,269 6.2 % each
Total 52,347

The four flop-based queues are half the design; the W buffer alone is a third.

5.4 Power (PrimeTime, 500 MHz)

Activity: a 4-channel concurrent copy at 98 % read-bus utilisation (make vcd), converted with vcd2saif and annotated through DC's SAIF name map (97.9 % of flops, 100 % of inputs annotated).

Group Internal Switching Leakage Total Share
Clock network 27.6 mW 0 0 27.6 mW 90.6 %
Registers 0.545 mW 0.100 mW 0.216 mW 0.861 mW 2.8 %
Combinational 0.823 mW 1.132 mW 0.059 mW 2.015 mW 6.6 %
Total 28.9 mW 1.23 mW 0.275 mW 30.4 mW

By block: u_wf 11.6 mW (38 %), u_csr 6.37 mW (21 %), u_awq 2.48 mW, each channel 1.59 mW. 2,810 un-gated flops toggle their clock pin every cycle, while data-path switching is only 1.2 mW even with the bus 98 % busy → clock gating is the clear next step.

The testbench clock is 10 ns. PrimeTime derives toggle rates from the dump's total time, so the VCD is re-timed to a 2 ns cycle first (syn/pt/dma_500m.vcd: every timestamp ÷ 5, value changes identical). The raw 100 MHz dump reports 6.4 mW — ~5× too low. read_vcd -rtl annotates only ~40 % of flops (RTL vs. DC names); SAIF with the name map reaches 97.9 %.

5.5 Critical path (DC / PT)

g_ch[2].u_ch/rem_q_reg[13]
  → remain != 0   (15-level NOR3/NAND3 chain after area recovery)   0.00 → 0.85 ns
  → req → round-robin arbiter                                       0.85 → 1.03 ns
  → grant                                                           1.03 → 1.09 ns
  → next-src mux / carry-select add → g_ch[1].u_ch/src_q_reg[29]    1.09 → 1.81 ns
  required 1.83 ns  →  slack +0.016 ns (MET)

Registering a remain_nz flag alongside rem_q would make req a flop output and take ~0.8 ns off this path — the first step before synthesising at 1.8 ns for P&R.

5.6 Timing closure history

Open-source flow, 2.0 ns target, before the DC run:

Step Change Setup WNS
0 First netlist −5.409 ns
1 Buffer trees on high-fanout flop outputs — ABC leaves the W-buffer read pointer driving ~600 pins (DC fixes this itself); syn/open/fanout_buf.py −1.412 ns
2 Register the chunk size — the 5-way min over 13-bit limits leaves the grant cycle; a channel skips one request cycle after a grant (round-robin fills it) −0.165 ns
3 Realignment sel / case precomputed at AR issue; carry-select src/dst ± n (13-bit add + parallel ±1 of the upper bits) −0.197 ns *
4 Adder-free burst limit (blb − max(s_off, d_off) is a bit-splice because blb is a power of two) + balanced min tree +0.254 ns
5 Store only the non-zero bits of blb − BPB +0.342 ns

* the critical path moved to the burst-limit logic, which step 4 removed.


6. Performance analysis: outstanding × burst length

Setup. Fixed 32-cycle read and write latency, no back-pressure, one 32 KB aligned copy per point, 32-bit bus. Every point is a real RTL simulation with the data checked. Utilisation = R-channel data beats ÷ cycles from START to completion. make perf.

6.1 Measured utilisation (model in parentheses)

outstanding \ burst 1 beat 2 beats 4 beats 8 beats 16 beats
1 2.9 % (3) 5.6 % (6) 10.5 % (11) 19.0 % (19) 32.0 % (32)
2 5.7 % (6) 11.1 % (11) 21.0 % (21) 38.0 % (38) 63.7 % (64)
4 11.4 % (11) 22.2 % (22) 42.0 % (42) 75.8 % (76) 99.1 % (100)
8 22.8 % (23) 44.3 % (44) 83.7 % (84) 99.1 % (100) 99.1 % (100)

At 500 MHz the 32-bit bus peaks at 2 GB/s: 57 MB/s (1 × 1) up to 1,983 MB/s.

6.2 Model

A read burst occupies the read pipeline for L + BL + c cycles and N of them can be in flight, so

util = min(1, N · BL / (L + BL + c))

The bus saturates once the bytes in flight cover the latency × bandwidth product. Fitting c on the N = 1 column gives c = 2.03 cycles (issue register + descriptor release); the model then reproduces all 20 points within ~1 %.

Latency sensitivity — smallest N reaching ≥ 90 %:

Latency (cycles) BL = 4 BL = 16
8 N = 4 N = 2
16 N = 8 N = 2
32 > 8 (best 84 %) N = 4
64 > 8 (best 45 %) N = 8
128 > 8 (best 24 %) > 8 (best 85 %)

6.3 Area vs bandwidth

Each hardware configuration synthesised (open-source flow, 500 MHz, all timing-clean) and run at its own limits (N = OUTS_MAX, BL = BL_MAX), latency 32:

Full table (12 configurations)
OUTS_MAX BL_MAX Area (µm²) Flops WNS (ns) Utilisation MB/s Pareto
1 4 39,109 1,620 +0.409 10.5 % 210 ✔
2 4 41,236 1,733 +0.381 21.0 % 421 ✔
1 8 44,728 1,928 +0.409 19.0 % 381
4 4 45,318 1,946 +0.409 42.0 % 840 ✔
2 8 47,096 2,047 +0.409 38.0 % 760
4 8 51,405 2,272 +0.403 75.8 % 1,515 ✔
8 4 52,992 2,359 +0.288 83.7 % 1,673 ✔
1 16 55,823 2,532 +0.407 31.9 % 639
2 16 58,246 2,657 +0.392 63.7 % 1,275
8 8 60,200 2,709 +0.271 99.1 % 1,983 ✔
4 16 62,774 2,894 +0.342 99.1 % 1,983
8 16 71,254 3,355 +0.217 99.1 % 1,983

6.4 Take-aways

  1. Outstanding requests buy bandwidth more cheaply than longer bursts. A longer burst doubles the W buffer (2 · BL words × 37 bits); one more outstanding read only adds a few descriptor entries. 8 outstanding × 8-beat bursts reach 99 % with 4 % less area than the default 4 × 16, and 8 × 4 already gives 84 % with 16 % less area.
  2. Long bursts coarsen arbitration. A 16-beat burst holds the read bus for 16 cycles, so latency-sensitive channels prefer shorter bursts with more outstanding reads.
  3. Latency moves the knee. At L = 64, BL = 16 needs N = 8; at L = 128 even 8 × 16 only reaches 85 % (the model asks for N ≥ 10).
  4. Single-channel issue bubble. With 1-beat bursts one channel tops out at 49.9 % (it can be re-granted only every other cycle because its chunk size is registered); with 4 channels round-robin fills the gap: 72.6 %, exactly the model's value.

7. Getting started

7.1 Prerequisites

Tool Tested version Used for
Verilator 5.020 (needs --timing) lint, simulation, coverage
g++ / make GCC 13 Verilator C++ build, DPI-C
Python 3 + matplotlib 3.13 / 3.11 coverage report, sweeps, plots
Yosys + ABC 0.33 open-source synthesis, formal front-end
SymbiYosys tag yosys-0.33 (must match Yosys) formal
z3 4.8.12 formal cover / BMC
OpenSTA 3.1.0 (built with CUDD) open-source STA
Synopsys DC / PT Y-2026.03 / Y-2026.03-SP2 sign-off (optional)

On Ubuntu 24.04: apt install verilator yosys z3 python3-matplotlib; SymbiYosys and OpenSTA from source (see docs/ISSUES_AND_FIXES.md §1). The gscl45nm library is downloaded automatically by syn/open/run_open.sh.

7.2 Quick start

make lint          # Verilator -Wall → "LINT CLEAN"
make sim           # one seed (SEED=3 ROUNDS=100 to change)
make regress       # 8 seeds + merged coverage report

Expected end of a simulation log:

[T] random regression: 60 rounds
------------------------------------------------------------
 transfers ok=457  expected-errors=22  AR=51470 AW=51470  R beats=108659 W beats=108670
 max outstanding reads seen = 4 (OUTS_MAX=4)
 cycles = 575450
 TEST PASSED
------------------------------------------------------------

7.3 All targets

Command What it runs Output
make lint Verilator -Wall on the RTL console
make sim one seed, directed + random logs/sim_<seed>.log, logs/cov_<seed>.dat
make regress 8 seeds + coverage merge logs/coverage_report.txt
bash scripts/run_configs.sh 3 other NCH / BL_MAX / OUTS_MAX configs logs/coverage_<cfg>.txt
make formal all SymbiYosys tasks formal/arb_fairness_*/
make mutants 8 mutants logs/mutants.md
bash scripts/verify_all.sh lint → regress → configs → formal → mutants logs/verify_all.log
make perf utilisation sweep analysis/results.csv, tradeoff.md, plots
bash analysis/area_sweep.sh then python3 analysis/area_tradeoff.py 12 synthesised configs analysis/area.csv, area_vs_util.png
make syn Yosys + OpenSTA (PERIOD=2.0 NCH=4 BL_MAX=16 OUTS_MAX=4) syn/open/out/<tag>/summary.txt
make vcd activity dump for PrimeTime syn/pt/dma.vcd, syn/pt/dma_500m.vcd
cd syn && ./run_dc_pt.sh 2.0 DC → PT (Synopsys environment) syn/dc/reports/, syn/pt/reports/

Simulation plusargs: +verilator+seed+<n>, +rounds=<n>, +test=perf +lat=<cycles> +bl=<log2 beats> +outs=<n> +plen=<bytes> +pch=<channels>.


8. Repository layout

.
├── rtl/                      synthesizable RTL (6 files)
├── tb/
│   ├── tb_top.sv             testbench: BFM, directed tests, random regression, perf mode
│   ├── axi_mem_model.sv      randomised AXI4 slave memory
│   ├── axi_checker.sv        AXI protocol SVA
│   ├── dma_cov.sv            functional coverage (cover properties)
│   ├── sim_main.cpp          Verilator harness (coverage output, tracing)
│   └── ref/dma_ref.c         DPI-C reference model
├── formal/                   arb_fairness.sv + .sby
├── syn/
│   ├── dc/dc_syn.tcl         Design Compiler
│   ├── pt/pt_sta.tcl         PrimeTime STA + power
│   ├── pt/dma_500m.vcd       activity for PrimeTime (re-timed to 2 ns)
│   ├── open/                 Yosys + OpenSTA flow, fanout_buf.py
│   ├── run_dc_pt.sh          DC → PT wrapper
│   └── RUN_DC_PT.md          step-by-step DC/PT guide (zh-TW)
├── analysis/                 sweep scripts, CSVs, plots
├── scripts/                  coverage report, config regression, mutation testing, verify_all
├── results/                  DC/PT report, coverage, regression, formal, mutation, open-flow STA
├── docs/ISSUES_AND_FIXES.md  every problem met during the project and its fix (zh-TW)
└── Makefile

9. Reproducing every number

Claim Command Where to look
Lint clean make lint console
3,820 transfers, all PASS make regress logs/sim_*.log, results/regression_summary.txt
34 / 34 coverage make regress results/coverage_report.txt
Other configs pass bash scripts/run_configs.sh results/configs_summary.txt
Arbiter proofs make formal results/formal_summary.txt
8 / 8 mutants make mutants results/mutants.md
Utilisation table, model make perf analysis/tradeoff.md, analysis/results.csv
Area vs bandwidth analysis/area_sweep.sh + area_tradeoff.py analysis/area_tradeoff.md
Open-source timing / area make syn results/open_syn_summary.txt, results/open_sta.rpt
DC / PT sign-off syn/run_dc_pt.sh results/DC_PT_RESULTS.md

10. Design decisions & trade-offs

Decision Why Cost
One AR + one AW per chunk Read and write sides stay in lock-step; one descriptor per burst; 4 KB and burst limits enforced in one place With different src/dst offsets, consecutive chunks may re-read one boundary word
Single AXI ID (0) AXI guarantees in-order R/B → three small FIFOs replace a reorder buffer A slow slave blocks other channels (head-of-line)
2-word window realigner Any src/dst alignment with one register and a byte shifter One flush cycle on some tails
Write poisoning after errors Bad read data never lands in memory, AXI still gets its W beats Bytes written before the error stay written
Registered chunk size Removes the 5-way min from the grant cycle (timing) Same channel re-granted at most every 2nd cycle (only visible with 1-beat bursts)
Round-robin at burst granularity Provably starvation-free, cheap (87 µm²) A long burst holds the bus for BL cycles
W buffer = 2·BL_MAX Absorbs a full burst while the next one fills under write back-pressure 35.7 % of area
Flop-based FIFOs Portable, no memory compiler needed for FreePDK45 Area and clock power — SRAM macro is the next step

11. Limitations & roadmap

  • Register remain_nz so req is a flop output (~0.8 ns off the critical path) → synthesise at 1.8 ns
  • Clock gating on FIFO entries and CSR registers (clock network is 90.6 % of power)
  • Replace the W buffer with an SRAM / register-file macro
  • Multiple AXI IDs + reorder buffer (no head-of-line blocking)
  • Keep a carry word between consecutive chunks of the same channel (no boundary re-read)
  • Scatter-gather descriptor ring; AxCACHE / AxPROT / QoS outputs
  • Generate the VCD at 2 ns directly in the testbench
  • Place & route in Innovus (CTS, post-route sign-off)

12. Lessons learned

Every issue met while building this — tool-chain problems, testbench bugs, coverage gaps, formal set-up, timing closure, DC/PT sign-off — is written up with the error message, root cause and fix in docs/ISSUES_AND_FIXES.md. The most instructive ones:

  • A DPI function called twice. Every AXI burst was reported as unexpected. The C model was fine; Verilator was re-evaluating a non-context DPI import that has side effects. Fix: import "DPI-C" context function … and store the result before testing it.
  • 100 % coverage, one bug still escaped. Mutation testing showed an interrupt lost when a completion event coincides with a W1C write — a scenario no cover point measured.
  • Tool limits vs. design limits. −5.4 ns was mostly unbuffered fanout that DC fixes on its own; the remaining −1.4 ns was real and needed microarchitecture changes.
  • Power numbers are only as good as the activity. A 100 MHz testbench dump made PrimeTime report 5× too little power; SAIF + name mapping raised flop annotation from 40 % to 98 %.

13. Author

Ian Chen — M.S. Electrical Engineering, University of Southern California · GitHub · LinkedIn

About

4-channel AXI4 DMA in SystemVerilog. Verified with Verilator, a DPI-C reference model, SVA, 100% functional coverage, mutation testing and a formal fairness proof. Signed off at 500 MHz in FreePDK45 with DC + PrimeTime.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages