codellama:13b | RAG on

13_rtllm_miscellaneous / task_17 | verified_RAM

benchmarkrtllm

task_idrtllm/13_rtllm_miscellaneous/task_17

samples_total10

final_success_samples0

final_success_rate_samples0.0%

functional_pass_rate_samples10.0%

synth_pass_rate_samples0.0%

source_runsrun_20260426_075435_225972_f32200

Что показывает эта страница

Здесь перечислены все реальные попытки этой модели на одной задаче. По каждой попытке можно увидеть, до какой стадии она дошла, где именно сломалась и какое сообщение об ошибке вернулось из syntax, functional или synthesis.

Samples

sample	run	extract_ok	syntax_ok	functional_ok	openlane_ok	final_success	failure_stage	failure_reason	details
sample_1	run_20260426_075435_225972_f32200	failed	failed	failed	skipped	failed	extract	No module/endmodule block found in model response	Open details sample_dir: E:\Work\MIEM\VKR\Analyse\data\model_dataset\models\codellama-13b\rag_on\benchmarks\rtllm\tasks\rtllm__13_rtllm_miscellaneous__task_17\samples\run_20260426_075435_225972_f32200__sample_1 synthesis details [error] None errors[] [ { "stage": "extract", "code": "extract.no_verilog_found", "message": "No module/endmodule block found in model response", "retryable": false, "details": {} } ] outcome { "llm_ok": true, "extract_ok": false, "syntax_ok": false, "functional_ok": false, "openlane_ok": false, "syntax_required": true, "functional_required": true, "openlane_required": true, "final_success": false, "success": false }
sample_2	run_20260426_075435_225972_f32200	failed	failed	failed	skipped	failed	extract	No module/endmodule block found in model response	Open details sample_dir: E:\Work\MIEM\VKR\Analyse\data\model_dataset\models\codellama-13b\rag_on\benchmarks\rtllm\tasks\rtllm__13_rtllm_miscellaneous__task_17\samples\run_20260426_075435_225972_f32200__sample_2 synthesis details [error] None errors[] [ { "stage": "extract", "code": "extract.no_verilog_found", "message": "No module/endmodule block found in model response", "retryable": false, "details": {} } ] outcome { "llm_ok": true, "extract_ok": false, "syntax_ok": false, "functional_ok": false, "openlane_ok": false, "syntax_required": true, "functional_required": true, "openlane_required": true, "final_success": false, "success": false }
sample_3	run_20260426_075435_225972_f32200	failed	failed	failed	skipped	failed	extract	No module/endmodule block found in model response	Open details sample_dir: E:\Work\MIEM\VKR\Analyse\data\model_dataset\models\codellama-13b\rag_on\benchmarks\rtllm\tasks\rtllm__13_rtllm_miscellaneous__task_17\samples\run_20260426_075435_225972_f32200__sample_3 synthesis details [error] None errors[] [ { "stage": "extract", "code": "extract.no_verilog_found", "message": "No module/endmodule block found in model response", "retryable": false, "details": {} } ] outcome { "llm_ok": true, "extract_ok": false, "syntax_ok": false, "functional_ok": false, "openlane_ok": false, "syntax_required": true, "functional_required": true, "openlane_required": true, "final_success": false, "success": false }
sample_4	run_20260426_075435_225972_f32200	failed	failed	failed	skipped	failed	extract	No module/endmodule block found in model response	Open details sample_dir: E:\Work\MIEM\VKR\Analyse\data\model_dataset\models\codellama-13b\rag_on\benchmarks\rtllm\tasks\rtllm__13_rtllm_miscellaneous__task_17\samples\run_20260426_075435_225972_f32200__sample_4 synthesis details [error] None errors[] [ { "stage": "extract", "code": "extract.no_verilog_found", "message": "No module/endmodule block found in model response", "retryable": false, "details": {} } ] outcome { "llm_ok": true, "extract_ok": false, "syntax_ok": false, "functional_ok": false, "openlane_ok": false, "syntax_required": true, "functional_required": true, "openlane_required": true, "final_success": false, "success": false }
sample_5	run_20260426_075435_225972_f32200	failed	failed	failed	skipped	failed	extract	No module/endmodule block found in model response	Open details sample_dir: E:\Work\MIEM\VKR\Analyse\data\model_dataset\models\codellama-13b\rag_on\benchmarks\rtllm\tasks\rtllm__13_rtllm_miscellaneous__task_17\samples\run_20260426_075435_225972_f32200__sample_5 synthesis details [error] None errors[] [ { "stage": "extract", "code": "extract.no_verilog_found", "message": "No module/endmodule block found in model response", "retryable": false, "details": {} } ] outcome { "llm_ok": true, "extract_ok": false, "syntax_ok": false, "functional_ok": false, "openlane_ok": false, "syntax_required": true, "functional_required": true, "openlane_required": true, "final_success": false, "success": false }
sample_6	run_20260426_075435_225972_f32200	failed	failed	failed	skipped	failed	extract	No module/endmodule block found in model response	Open details sample_dir: E:\Work\MIEM\VKR\Analyse\data\model_dataset\models\codellama-13b\rag_on\benchmarks\rtllm\tasks\rtllm__13_rtllm_miscellaneous__task_17\samples\run_20260426_075435_225972_f32200__sample_6 synthesis details [error] None errors[] [ { "stage": "extract", "code": "extract.no_verilog_found", "message": "No module/endmodule block found in model response", "retryable": false, "details": {} } ] outcome { "llm_ok": true, "extract_ok": false, "syntax_ok": false, "functional_ok": false, "openlane_ok": false, "syntax_required": true, "functional_required": true, "openlane_required": true, "final_success": false, "success": false }
sample_7	run_20260426_075435_225972_f32200	failed	failed	failed	skipped	failed	extract	Model response is empty	Open details sample_dir: E:\Work\MIEM\VKR\Analyse\data\model_dataset\models\codellama-13b\rag_on\benchmarks\rtllm\tasks\rtllm__13_rtllm_miscellaneous__task_17\samples\run_20260426_075435_225972_f32200__sample_7 synthesis details [error] None errors[] [ { "stage": "model_request", "code": "model_request.failed", "message": "HTTPConnectionPool(host='ollama', port=11434): Read timed out. (read timeout=120)", "retryable": true, "details": {} }, { "stage": "extract", "code": "extract.no_verilog_found", "message": "Model response is empty", "retryable": false, "details": {} } ] outcome { "llm_ok": false, "extract_ok": false, "syntax_ok": false, "functional_ok": false, "openlane_ok": false, "syntax_required": true, "functional_required": true, "openlane_required": true, "final_success": false, "success": false }
sample_8	run_20260426_075435_225972_f32200	failed	failed	failed	skipped	failed	extract	Model response is empty	Open details sample_dir: E:\Work\MIEM\VKR\Analyse\data\model_dataset\models\codellama-13b\rag_on\benchmarks\rtllm\tasks\rtllm__13_rtllm_miscellaneous__task_17\samples\run_20260426_075435_225972_f32200__sample_8 synthesis details [error] None errors[] [ { "stage": "model_request", "code": "model_request.failed", "message": "HTTPConnectionPool(host='ollama', port=11434): Read timed out. (read timeout=120)", "retryable": true, "details": {} }, { "stage": "extract", "code": "extract.no_verilog_found", "message": "Model response is empty", "retryable": false, "details": {} } ] outcome { "llm_ok": false, "extract_ok": false, "syntax_ok": false, "functional_ok": false, "openlane_ok": false, "syntax_required": true, "functional_required": true, "openlane_required": true, "final_success": false, "success": false }
sample_9	run_20260426_075435_225972_f32200	ok	ok	ok	failed	failed	synthesis	OpenLane failed: 2	Open details sample_dir: E:\Work\MIEM\VKR\Analyse\data\model_dataset\models\codellama-13b\rag_on\benchmarks\rtllm\tasks\rtllm__13_rtllm_miscellaneous__task_17\samples\run_20260426_075435_225972_f32200__sample_9 generated.v preview module RAM ( input clk, input rst_n, input write_en, input [WIDTH - 1 : 0] write_addr, input [WIDTH - 1 : 0] write_data, input read_en, input [WIDTH - 1 : 0] read_addr, output reg [WIDTH - 1 : 0] read_data ); parameter WIDTH = 6; parameter DEPTH = 8; reg [DEPTH - 1 : 0] RAM [2**WIDTH-1:0]; always @(posedge clk or negedge rst_n) begin if (!rst_n) begin read_data <= '0; end else begin if (read_en) begin read_data <= RAM[read_addr]; end else begin read_data <= '0; end end end always @(posedge clk or negedge rst_n) begin if (!rst_n) begin for (int i = 0; i < DEPTH; i++) begin RAM[i] <= '0; ... functional details [stdout] ===========Your Design Passed=========== /workspace/banchmarks/rtllm/13_rtllm_miscellaneous/task_17/tb.v:79: $finish called at 11000 (1ns) synthesis details [error] OpenLane failed: 2 errors[] [ { "stage": "openlane", "code": "openlane.failed", "message": "OpenLane failed: 2", "retryable": false, "details": {} } ] outcome { "llm_ok": true, "extract_ok": true, "syntax_ok": true, "functional_ok": true, "openlane_ok": false, "syntax_required": true, "functional_required": true, "openlane_required": true, "final_success": false, "success": false }
sample_10	run_20260426_075435_225972_f32200	ok	ok	failed	skipped	failed	functional	/workspace/banchmarks/rtllm/13_rtllm_miscellaneous/task_17/tb.v:21: error: port ``rst_n'' is not a port of uut.	Open details sample_dir: E:\Work\MIEM\VKR\Analyse\data\model_dataset\models\codellama-13b\rag_on\benchmarks\rtllm\tasks\rtllm__13_rtllm_miscellaneous__task_17\samples\run_20260426_075435_225972_f32200__sample_10 generated.v preview module RAM ( input clk, input [AWIDTH-1:0] addr0, input [DWIDTH-1:0] data0, output reg [DWIDTH-1:0] q0, input we0, input [AWIDTH-1:0] addr1, input [DWIDTH-1:0] data1, output reg [DWIDTH-1:0] q1, input we1 ); parameter AWIDTH = 8; parameter DWIDTH = 6; reg [DWIDTH-1:0] mem[2**AWIDTH]; always @(posedge clk) begin if (we0) begin q0 <= data0; mem[addr0] <= data0; end else begin q0 <= mem[addr0]; end end always @(posedge clk) begin if (we1) begin q1 <= data1; mem[addr1] <= data1; end else begin q1 <= mem[addr1]; ... functional details [stderr] /workspace/banchmarks/rtllm/13_rtllm_miscellaneous/task_17/tb.v:21: error: port ``rst_n'' is not a port of uut. /workspace/banchmarks/rtllm/13_rtllm_miscellaneous/task_17/tb.v:21: error: port ``write_en'' is not a port of uut. /workspace/banchmarks/rtllm/13_rtllm_miscellaneous/task_17/tb.v:21: error: port ``write_addr'' is not a port of uut. /workspace/banchmarks/rtllm/13_rtllm_miscellaneous/task_17/tb.v:21: error: port ``write_data'' is not a port of uut. /workspace/banchmarks/rtllm/13_rtllm_miscellaneous/task_17/tb.v:21: error: port ``read_en'' is not a port of uut. /workspace/banchmarks/rtllm/13_rtllm_miscellaneous/task_17/tb.v:21: error: port ``read_addr'' is not a port of uut. /workspace/banchmarks/rtllm/13_rtllm_miscellaneous/task_17/tb.v:21: error: port ``read_data'' is not a port of uut. 7 error(s) during elaboration. synthesis details [error] None errors[] [ { "stage": "functional", "code": "functional.testbench_failed", "message": "/workspace/banchmarks/rtllm/13_rtllm_miscellaneous/task_17/tb.v:21: error: port ``rst_n'' is not a port of uut.\n/workspace/banchmarks/rtllm/13_rtllm_miscellaneous/task_17/tb.v:21: error: port ``write_en'' is not a port of uut.\n/workspace/banchmarks/rtllm/13_rtllm_miscellaneous/task_17/tb.v:21: error: port ``write_addr'' is not a port of uut.\n/workspace/banchmarks/rtllm/13_rtllm_miscellaneous/task_17/tb.v:21: error: port ``write_data'' is not a port of uut.\n/workspace/banchmarks/rtllm/13_rtllm_miscellaneous/task_17/tb.v:21: error: port ``read_en'' is not a port of uut.\n/workspace/banchmarks/rtllm/13_rtllm_miscellaneous/task_17/tb.v:21: error: port ``read_addr'' is not a port of uut.\n/workspace/banchmarks/rtllm/13_rtllm_miscellaneous/task_17/tb.v:21: error: port ``read_data'' is not a port of uut.\n7 error(s) during elaboration.\n", "retryable": false, "details": {} } ] outcome { "llm_ok": true, "extract_ok": true, "syntax_ok": true, "functional_ok": false, "openlane_ok": false, "syntax_required": true, "functional_required": true, "openlane_required": true, "final_success": false, "success": false }