Week 5 test guide
A test failed and you don't know why? Find it here: what it checks, and the plausible bugs that are known to fail it. Then read the matching part of the week 5 spec.
How to use this
Run python tools/test.py --compiler PATH --week 5 -v, copy the failing test's name, and search below. "Bugs known to fail it" lists mistakes that were planted in the reference compiler and caught by this test, so one of them is a good first guess. It's not a complete list, and the debugging guide covers what to do when none fit.
| test | kind | what it checks | bugs known to fail it |
|---|---|---|---|
render/gemm_movement_3x4x5 | render | the spec's GEMM: reshape, permute, expand, mul, reduce pop=1 | end doesn't close its range final version not followed loads never move into loops render elementwise at 0 render expand reads i render flat_of order render load at offset 0 render loop starts at 1 render loop uses <= render reduce from zero render reduce reversed render skips rangeify render store loop short render unflat stride values hoisted to outer loops |
render/pad_2d | render | pad a 2x3 with a ring of zeros: 4x5 | C cmplt as <= C where swapped end doesn't close its range final version not followed render flat_of order render load at offset 0 render pad mask off by one render skips rangeify render unflat stride |
render/r_column_normalize | render | divide each column by its sum: reduce, reshape, expand, mul | end doesn't close its range final version not followed loads never move into loops render elementwise at 0 render flat_of order render load at offset 0 render loop starts at 1 render loop uses <= render reduce from zero render reduce reversed render skips rangeify render store loop short render unflat stride values hoisted to outer loops |
render/r_conv1d_same | render | same-size conv1d with a 5-tap kernel | C cmplt as <= C where swapped end doesn't close its range final version not followed loads never move into loops render elementwise at 0 render expand reads i render load at offset 0 render loop starts at 1 render loop uses <= render pad mask off by one render reduce from zero render reduce reversed render shrink offset sign render skips rangeify render stack chain off by one render store loop short values hoisted to outer loops |
render/r_conv2d_16x16 | render | 3x3 same-size conv2d on 16x16 | C cmplt as <= C where swapped end doesn't close its range final version not followed loads never move into loops render elementwise at 0 render expand reads i render flat_of order render load at offset 0 render loop starts at 1 render loop uses <= render pad mask off by one render reduce from zero render reduce reversed render shrink offset sign render skips rangeify render stack chain off by one render store loop short render unflat stride values hoisted to outer loops |
render/r_elementwise_rank3 | render | elementwise mul and max on rank 3 | end doesn't close its range final version not followed render elementwise at 0 render flat_of order render load at offset 0 render loop starts at 1 render skips rangeify render store loop short render unflat stride |
render/r_expand_scalar | render | broadcast a scalar to a matrix | end doesn't close its range final version not followed render expand zero render loop starts at 1 render skips rangeify render store loop short |
render/r_expand_size1 | render | broadcast a column | end doesn't close its range final version not followed render expand reads i render load at offset 0 render loop starts at 1 render skips rangeify render store loop short render unflat stride |
render/r_inplace_transpose | render | transpose a 3x3 matrix in place: the result is built from the old contents, not half-overwritten ones | end doesn't close its range final version not followed render flat_of order render in-place store fused render load at offset 0 render loop uses <= render skips rangeify render stage reads x[0] render unflat stride |
render/r_matmul_1x8x5 | render | a row vector times a matrix | end doesn't close its range final version not followed loads never move into loops render elementwise at 0 render expand reads i render flat_of order render load at offset 0 render loop starts at 1 render loop uses <= render reduce from zero render reduce reversed render skips rangeify render store loop short render unflat stride values hoisted to outer loops |
render/r_matmul_2x3x4 | render | matmul via movement ops | end doesn't close its range final version not followed loads never move into loops render elementwise at 0 render expand reads i render flat_of order render load at offset 0 render loop starts at 1 render loop uses <= render reduce from zero render reduce reversed render skips rangeify render store loop short render unflat stride values hoisted to outer loops |
render/r_matmul_64x64 | render | 64x64 by 64x64: three nested loops in C, well under 20 KB | end doesn't close its range final version not followed loads never move into loops render elementwise at 0 render expand reads i render load at offset 0 render loop starts at 1 render loop uses <= render reduce from zero render reduce reversed render skips rangeify render store loop short render unflat stride values hoisted to outer loops |
render/r_matmul_i32_wrap | render | i32 matmul with wraparound | C plain signed add end doesn't close its range final version not followed loads never move into loops render elementwise at 0 render expand reads i render flat_of order render load at offset 0 render loop starts at 1 render loop uses <= render reduce from zero render skips rangeify render store loop short render unflat stride values hoisted to outer loops |
render/r_pad_32x32 | render | pad a 32x32 image by one: 34x34 | C cmplt as <= C where swapped end doesn't close its range final version not followed render flat_of order render load at offset 0 render pad mask off by one render skips rangeify render unflat stride |
render/r_pad_3d | render | pad a rank-3 value | C cmplt as <= C where swapped end doesn't close its range final version not followed render flat_of order render load at offset 0 render pad mask off by one render pad shift wrong side render skips rangeify render unflat stride |
render/r_pad_bool | render | bool pad fills false | C cmplt as <= C where swapped end doesn't close its range final version not followed render flat_of order render load at offset 0 render pad mask off by one render pad shift wrong side render skips rangeify render unflat stride |
render/r_pad_left_right | render | pad both sides of a vector | C cmplt as <= C where swapped end doesn't close its range final version not followed render load at offset 0 render pad mask off by one render pad shift wrong side render skips rangeify |
render/r_pad_then_shrink | render | pad then an off-center window: some zeros, some data | C cmplt as <= C where swapped end doesn't close its range final version not followed render flat_of order render load at offset 0 render pad mask off by one render shrink offset sign render skips rangeify render unflat stride |
render/r_permute_rank3 | render | a rank-3 permute | end doesn't close its range final version not followed render flat_of order render load at offset 0 render loop starts at 1 render permute inverse render skips rangeify render store loop short render unflat stride |
render/r_permute_rank4 | render | a rank-4 permute | end doesn't close its range final version not followed render flat_of order render load at offset 0 render loop starts at 1 render permute inverse render skips rangeify render store loop short render unflat stride |
render/r_reduce_broadcast_axis | render | reduce over an axis that expand created: every copy is still added, so the result is 4 times x | end doesn't close its range final version not followed render expand reads i render flat_of order render load at offset 0 render loop starts at 1 render loop uses <= render permute inverse render reduce from zero render reduce of a broadcast hoisted render skips rangeify render store loop short render unflat stride values hoisted to outer loops |
render/r_reduce_pop1 | render | column sums of a matrix | end doesn't close its range final version not followed loads never move into loops render flat_of order render load at offset 0 render loop starts at 1 render loop uses <= render reduce from zero render reduce reversed render skips rangeify render store loop short values hoisted to outer loops |
render/r_reduce_pop2_64x64 | render | sum of a 64x64 matrix in row-major order | end doesn't close its range final version not followed loads never move into loops render flat_of order render load at offset 0 render loop starts at 1 render loop uses <= render reduce column-major render reduce reversed render skips rangeify render store loop short render unflat stride |
render/r_reduce_pop2_rank3 | render | reduce the two leading axes of a rank-3 value | end doesn't close its range final version not followed loads never move into loops render flat_of order render load at offset 0 render loop starts at 1 render reduce from zero render skips rangeify render store loop short render unflat stride values hoisted to outer loops |
render/r_reduce_rowmajor_order | render | row-major fold order is visible in f32 bits | end doesn't close its range final version not followed loads never move into loops render flat_of order render load at offset 0 render loop starts at 1 render loop uses <= render reduce column-major render reduce reversed render skips rangeify render store loop short render unflat stride |
render/r_reshape_permute_2d | render | transpose via permute, stored flat | end doesn't close its range final version not followed render flat_of order render load at offset 0 render loop starts at 1 render skips rangeify render store loop short render unflat stride |
render/r_reshape_permute_4096 | render | 4096 elements reshaped to 16x16x16 and reversed axis order | end doesn't close its range final version not followed render flat_of order render load at offset 0 render loop starts at 1 render skips rangeify render store loop short render unflat stride |
render/r_reshape_same_rank_then_transpose | render | reshape [2 3] to [3 2] is not a no-op: transposing afterwards shows the new row boundaries | end doesn't close its range final version not followed render load at offset 0 render loop starts at 1 render reshape same rank skipped render skips rangeify render store loop short render unflat stride |
render/r_shrink_symbolic | render | a run-time but provable offset: mod (max o 0) 4 with o = 13 is 1 | end doesn't close its range final version not followed render load at offset 0 render loop starts at 1 render shrink offset sign render skips rangeify render store loop short |
render/r_shrink_window | render | a value window | end doesn't close its range final version not followed render flat_of order render load at offset 0 render loop starts at 1 render shrink offset sign render skips rangeify render store loop short render unflat stride |
render/r_stack_index | render | index a stack of vectors | C cmplt as <= C where swapped end doesn't close its range final version not followed render index pops the end render load at offset 0 render loop starts at 1 render skips rangeify render store loop short |
render/r_stack_matrices | render | stack of matrices rendered as a where chain | C cmplt as <= C where swapped end doesn't close its range final version not followed render flat_of order render load at offset 0 render loop starts at 1 render skips rangeify render stack chain off by one render store loop short render unflat stride |
render/r_stage_rank2 | render | stage a transposed matrix, then use it twice | end doesn't close its range final version not followed render elementwise at 0 render load at offset 0 render loop starts at 1 render loop uses <= render skips rangeify render stage reads x[0] render store loop short render unflat stride |
render/r_transpose_64x64 | render | 64x64 transpose: loops, not 4096 unrolled stores | end doesn't close its range final version not followed render flat_of order render load at offset 0 render loop starts at 1 render skips rangeify render store loop short render unflat stride |
run/assert_rank2_fails | run: assert_fail | assert compares every element of a rank-2 value | evaluates everything |
run/broadcast_column_sums_back | run: pass | reduce over rows, reshape to [1 4], expand back, add: the normalization shape | elementwise uses x[0] evaluates everything expand reads i not 0 reduce pops trailing axes reshape column-major |
run/conv1d_same | run: pass | same-size conv1d: pad, a stack of shifted shrinks, multiply by the broadcast kernel, reduce over taps | elementwise uses x[0] evaluates everything expand reads i not 0 f32 ops not rounded fold right to left pad fill not zero reduce pops trailing axes shrink offset sign |
run/conv2d_same | run: pass | 3x3 same-size conv2d on a 4x5 image via pad and nine shrinks | elementwise uses x[0] evaluates everything expand reads i not 0 pad fill not zero reduce pops trailing axes reshape column-major shrink offset sign |
run/elementwise_rank3 | run: pass | add, mul, cmplt, where and cast work elementwise on rank-3 values | elementwise uses x[0] evaluates everything f32 ops not rounded f32->i32 rounds where branches swapped |
run/err_expand_non1_axis | run: invalid | a size-2 axis cannot grow to 4 | expand grows non-1 axes |
run/err_expand_rank_mismatch | run: invalid | a non-scalar expand keeps the rank | none recorded |
run/err_index_runtime_oob | run: invalid | a computed index past the end is a run-time error | none recorded |
run/err_index_too_many | run: invalid | at most one index per axis | index count unchecked |
run/err_pad_length | run: invalid | pads has two numbers per axis | pads length unchecked |
run/err_pad_negative | run: invalid | pads are >= 0 (cut with shrink) | none recorded |
run/err_param_rank2 | run: invalid | params stay rank 0 or 1 | none recorded |
run/err_permute_duplicate | run: invalid | order must be a permutation | permute any order |
run/err_permute_out_of_range | run: invalid | order must name each axis once | permute any order |
run/err_permute_wrong_length | run: invalid | order has one entry per axis | none recorded |
run/err_reduce_pop_too_big | run: invalid | pop can drop at most every axis | reduce takes scalars |
run/err_reshape_count | run: invalid | reshape must keep the element count | reshape count unchecked |
run/err_reshape_zero_dim | run: invalid | shape sizes are positive | none recorded |
run/err_shrink_offset_count | run: invalid | one offset per axis | shrink offset count unchecked |
run/err_shrink_past_end | run: invalid | offset 3 plus 2 elements passes the end of 4 | shrink bounds unchecked |
run/err_shrink_unprovable | run: invalid | a raw loaded offset could be anything | shrink bounds unchecked |
run/err_stack_mixed_shape | run: invalid | stack operands share one shape | stack shapes unchecked |
run/expand_rank3_middle | run: pass | a middle size-1 axis broadcasts | evaluates everything expand reads i not 0 reshape column-major |
run/expand_scalar_rank2 | run: pass | a scalar expands to any shape | evaluates everything expand scalar is zero |
run/expand_size1_axes | run: pass | only size-1 axes grow, and they repeat the single element | evaluates everything expand reads i not 0 reshape column-major |
run/expand_then_reduce | run: pass | broadcast then reduce: every column gets the same column sum | evaluates everything expand reads i not 0 reduce pops trailing axes |
run/gemm_movement_value | run: pass | C[2][4] computed independently with per-op f32 rounding | elementwise uses x[0] evaluates everything expand reads i not 0 f32 ops not rounded reduce pops trailing axes reshape column-major |
run/matmul_movement_1x5x3 | run: pass | (1x5) @ (5x3) with reshape, permute, expand, mul, reduce pop=1; sums over k in order | elementwise uses x[0] evaluates everything expand reads i not 0 f32 ops not rounded fold right to left reduce pops trailing axes reshape column-major |
run/matmul_movement_2x3x2 | run: pass | (2x3) @ (3x2) with reshape, permute, expand, mul, reduce pop=1; sums over k in order | elementwise uses x[0] evaluates everything expand reads i not 0 f32 ops not rounded fold right to left reduce pops trailing axes reshape column-major |
run/matmul_movement_3x3x1 | run: pass | (3x3) @ (3x1) with reshape, permute, expand, mul, reduce pop=1; sums over k in order | elementwise uses x[0] evaluates everything expand reads i not 0 f32 ops not rounded fold right to left reduce pops trailing axes |
run/matmul_movement_4x1x2 | run: pass | (4x1) @ (1x2) with reshape, permute, expand, mul, reduce pop=1; sums over k in order | elementwise uses x[0] evaluates everything expand reads i not 0 f32 ops not rounded reduce pops trailing axes reshape column-major |
run/matmul_syllabus_form | run: pass | the syllabus GEMM: A stored transposed as K x M, reshape [K M 1], no permute | elementwise uses x[0] evaluates everything expand reads i not 0 f32 ops not rounded fold right to left reduce pops trailing axes reshape column-major |
run/pad_2d_mixed | run: pass | pad rows on top and columns on the right, f32 fill 0.0 | evaluates everything pad fill not zero pad sides swapped reshape column-major |
run/pad_3d | run: pass | pads apply per axis, in axis order | evaluates everything pad fill not zero pad sides swapped reshape column-major |
run/pad_bool_false | run: pass | bool pads with false | evaluates everything pad fill not zero pad sides swapped reshape column-major |
run/pad_left_only | run: pass | pads=[2 0] adds two zeros on the left | evaluates everything pad fill not zero pad sides swapped |
run/pad_right_only | run: pass | pads=[0 3] adds three zeros on the right | evaluates everything pad fill not zero pad sides swapped |
run/pad_shrink_roundtrip_3d | run: pass | shrinking a pad at its left pads recovers the value | evaluates everything pad sides swapped shrink offset sign |
run/pad_shrink_value | run: pass | pad then shrink back recovers the original, and an off-center window sees the zeros | evaluates everything pad fill not zero shrink offset sign |
run/pad_zero | run: pass | pads of zero change nothing | evaluates everything |
run/permute_identity | run: pass | the identity order changes nothing | evaluates everything |
run/permute_inverse_roundtrip | run: pass | order [1 2 0] then its inverse [2 0 1] gives x back | evaluates everything permute inverse |
run/permute_rank3_all_orders | run: pass | all six axis orders of a [2 3 4] value | evaluates everything permute inverse reshape column-major |
run/permute_rank4 | run: pass | shape[k] = x.shape[order[k]]: [2 3 1 2] with order [3 1 0 2] is [2 3 2 1] | evaluates everything permute inverse |
run/permute_then_reshape | run: pass | reshape after a transpose reads the transposed (row-major) order | evaluates everything reshape column-major |
run/permute_transpose | run: pass | permute [2 3] with order [1 0] is the transpose | evaluates everything reshape column-major |
run/reduce_last_axis_via_permute | run: pass | to reduce the last axis, permute it to the front (row sums of x) | evaluates everything f32 ops not rounded fold right to left reduce pops trailing axes reshape column-major |
run/reduce_max_nan_mul_wrap | run: pass | max over rows propagates NaN per column, and i32 mul wraps (65536 * 65536 = 0) | NaN != NaN in assert evaluates everything fold from zero fold skips nan max ignores NaN mul doesn't wrap reduce op ignored reduce pops trailing axes reshape column-major |
run/reduce_pop1_pop2 | run: pass | pop=1 sums over rows (one total per column), and pop=2 sums everything | evaluates everything reduce pops trailing axes reshape column-major |
run/reduce_rank3_pops | run: pass | pop=1, 2, 3 on a rank-3 value, and pop defaults to 1 | evaluates everything fold from zero reduce op ignored reduce pops trailing axes reshape column-major |
run/reduce_rowmajor_order | run: pass | pop=2 folds x00, x01, x10, x11 in row-major order: 1e8+1 rounds to 1e8, so the total is 1, not 2 | evaluates everything f32 ops not rounded fold right to left reduce folds column-major reshape column-major |
run/reshape_from_scalar | run: pass | a scalar reshapes to [1 1] and back down | evaluates everything |
run/reshape_rank_down | run: pass | reshape a rank-3 value flat keeps row-major order | evaluates everything reshape column-major |
run/reshape_rank_up | run: pass | reshape [12] to [3 4] is row-major: element (2, 1) is 9 | evaluates everything reshape column-major |
run/reshape_same_rank | run: pass | reshape [2 3] to [3 2] is not a transpose: same flat order | evaluates everything reshape column-major |
run/reshape_to_scalar | run: pass | reshape [1] to shape=[] gives a scalar | evaluates everything |
run/shrink_constant | run: pass | a [2 3] window starting at (1, 2) | evaluates everything reshape column-major shrink offset sign |
run/shrink_of_shrink | run: pass | offsets of nested shrinks add up | evaluates everything reshape column-major shrink offset sign |
run/shrink_symbolic_provable | run: pass | mod (max o 0) 2 is in [0, 1], so a 3-wide window of 4 is provably in range (o = 5 gives 1) | evaluates everything shrink offset sign |
run/stack_matrices_runtime_index | run: pass | stack of matrices is rank 3; a run-time index picks one row | evaluates everything reshape column-major |
run/stack_vectors_and_index | run: pass | stack of vectors is [3 4]; index pops leading axes | evaluates everything reshape column-major |
run/stage_rank2 | run: pass | stage is the identity on a rank-2 value | elementwise uses x[0] evaluates everything stage zeroes |