Day 6 · Verification Methodology

File-Driven Testing

Video 4 of 4 · ~10 minutes

Dr. Mike Borowczak · Electrical & Computer Engineering · CECS · UCF

AnatomySelf-CheckingTasksFile-Driven

🌍 Where This Lives

In Industry

Real verification suites use randomized input generators that produce hundreds of thousands of vectors, written to files, then replayed. Golden reference models in C or Python produce expected outputs; the testbench reads both input and expected-output files, runs the DUT, and compares. This is how Apple's CPU teams validate instruction decoders — against a C reference model fed millions of generated vectors.

In This Course

Day 11 UART bring-up uses a file of received-byte test patterns. Day 14 capstone verification leans on this pattern. Any time you have “I need to test N cases and N is big,” file-driven testing is the answer.

⚠️ Separate Test Data From Test Logic

❌ Wrong Model

“I'll add new test cases as new lines of Verilog in my testbench. If I need 10,000 cases I'll generate a 10,000-line testbench.”

✓ Right Model

The testbench logic (apply input, wait, check output) stays the same. The test data (what input, what expected output) lives in an external file. Your testbench loops over the file's contents. Adding new cases = editing the data file, not the Verilog.

The receipt: Verilog's $readmemh loads hex values from a text file into a Verilog memory array at simulation start. Combined with a loop, one initial block runs arbitrarily many vectors.

👁️ I Do — $readmemh and the Test Loop

// vectors.hex:  16 lines, 8 hex chars each = {a[7:0], b[7:0], expected_sum[8:0]}
// 0503_0008
// 0709_0010
// 0F0F_001E
//  ...

reg [31:0] vec_mem [0:15];   // 16 vectors
integer i;

initial begin
    $readmemh("vectors.hex", vec_mem);

    for (i = 0; i < 16; i = i + 1) begin
        a = vec_mem[i][31:24];
        b = vec_mem[i][23:16];
        #10;
        check_eq(sum, vec_mem[i][8:0],
                 $sformatf("vec[%0d]: %h+%h", i, a, b));
    end

    $display("=== %0d passed, %0d failed ===",
             tests_run - tests_failed, tests_failed);
    $finish;
end
My thinking: Pack each test vector as a single hex word (inputs + expected). $readmemh loads all vectors at time 0. The loop does the work — adding 1,000 more cases requires zero Verilog changes, only vectors.hex changes.

🤝 We Do — Generating Vectors from Python

#!/usr/bin/env python3
# gen_vectors.py — produces vectors.hex for adder testbench
import random

random.seed(42)
with open("vectors.hex", "w") as f:
    for _ in range(1000):
        a = random.randint(0, 255)
        b = random.randint(0, 255)
        s = a + b                              # golden reference in Python
        f.write(f"{a:02X}{b:02X}_{s:04X}\n")
print("Wrote 1000 vectors.")
Together: Python is your golden reference model. It produces the expected output the way you know it should be computed (just a + b). The Verilog testbench checks the DUT against Python's answers. If the DUT disagrees, the DUT is wrong — because the Python reference is trivially correct.

🧪 You Do — Debug the Hex Format

Your vectors.hex looks like this. Your testbench reports “all zeros, all fail.” What's wrong?

# vectors.hex
Test 1: 0503_0008
Test 2: 0709_0010
Answer: $readmemh doesn't parse “Test 1:” prefixes or colons — it expects hex digits only (with optional // comments and _ as a visual separator). Fix: strip the labels, keep only the hex.
// Test 1
0503_0008
// Test 2
0709_0010
▶ LIVE DEMO

1000-Vector Adder Test

~5 minutes

▸ COMMANDS

cd labs/week2_day06/ex4_file_driven/
python3 gen_vectors.py      # 1000 random cases
head -3 vectors.hex
make sim                    # < 1 second
tail -1 logs/sim.log

▸ EXPECTED STDOUT

Wrote 1000 vectors.
4F37_0086
1CE9_0105
...
=== 1000 passed, 0 failed ===

▸ KEY OBSERVATION

1000 cases, fraction of a second. Change the Python seed or bump to 10,000 — still seconds. Now change the DUT to break it (try a + b + 1) and watch all 1000 fail instantly.

🔧 Constrained vs Directed Testing

Directed

Hand-picked specific cases. Testing known edge conditions: overflow, carry, zero, max. Small, precise. What you write by hand.

Constrained-Random

Randomly generated within constraints (e.g. “both inputs 0–255”). Catches combinations you didn't think of. Large scale. What file-driven testing enables.

Best practice: Use both. Directed cases cover the specific corners you know. Constrained-random coverage catches the ones you don't. The file-driven pattern lets you mix them in one .hex file.

🤖 Check the Machine

Ask AI: “Write a Python script that generates 1000 random test vectors for a Verilog adder, and a Verilog testbench that consumes them with $readmemh.”

TASK

Python generator + Verilog consumer.

BEFORE

Predict: Python writes hex lines; Verilog packs input+expected into one memory word.

AFTER

Strong AI sets a seed. Weak AI omits it — reproducibility bug.

TAKEAWAY

Seeded randomness = reproducible failures. Always seed.

Key Takeaways

 Separate test data from test logic.

$readmemh loads hex from file into a Verilog memory.

 Python (or any language) is your golden reference.

 Seed your randomness — reproducibility beats cleverness.

Directed tests catch what you know. Randomized tests catch what you don't.

🔗 End of Day 6

Tomorrow: Finite State Machines

Day 7 · FSM Design & The 3-Block Template

▸ WHY THIS MATTERS NEXT

You can now verify any design you build. Day 7 is the week's intellectual peak: finite state machines. The sequential control pattern that underlies every protocol, every controller, every CPU. You'll learn the 3-block template that makes FSMs reliable — and your self-checking testbench skills will earn their keep, because FSMs have lots of states to verify.