Кумулятивный профиль модели по завершённым sample. Синий набор столбцов на графике — текущая модель. Если выбрать вторую модель ниже, на графике появится красный набор столбцов для прямого сравнения.
Выбери другой профиль, чтобы добавить его в график и сравнить кумулятивные метрики напрямую.
| Metric | Что означает |
|---|---|
extract_pass_at_k | Доля задач, где модель хотя бы один раз выдала извлекаемый Verilog-код. |
syntax_pass_at_k | Доля задач, где модель хотя бы один раз сгенерировала синтаксически корректный RTL. |
functional_pass_at_k | Доля задач, где модель хотя бы один раз прошла functional-проверку. |
synth_pass_at_k | Доля задач, где модель хотя бы один раз сгенерировала синтезируемый RTL. Это главный итоговый рейтинг на странице. |
final_success_at_k | Доля задач, где модель хотя бы один раз полностью прошла всю цепочку проверки. |
functional_pass_rate_samples | Какой процент всех sample прошёл functional. Это метрика стабильности на уровне отдельных попыток. |
synth_pass_rate_samples | Какой процент всех sample прошёл synthesis. Это метрика устойчивости на уровне отдельных попыток. |
samples_per_task | Среднее число sample на одну задачу в данном профиле модели. |
final_success_samples | Сколько sample конкретной задачи полностью прошли всю цепочку. |
final_success_rate_samples | Доля полностью успешных sample для конкретной задачи. |
| Бенчмарк | Задач | Sample | functional_pass_at_k | synth_pass_at_k | final_success_at_k |
|---|---|---|---|---|---|
| ilyasov | 27 | 702 | 100.0% | 92.6% | 92.6% |
| rtllm | 50 | 1050 | 48.0% | 48.0% | 48.0% |
Эта таблица показывает не только факт “задача закрылась”, но и насколько стабильно модель её решала по всем sample.
| Бенчмарк | Задача | Название | Sample | final_success_samples | final_success_rate_samples | functional_pass_rate_samples | synth_pass_rate_samples | synth_pass_at_k | final_success_at_k | samples_page |
|---|---|---|---|---|---|---|---|---|---|---|
| ilyasov | 05_dft_testability / task_51 |
scan_reset_override_reg | 26 | 0 | 0.0% | 23.1% | 0.0% | нет | нет | Open samples |
| ilyasov | 12_cdv / task_121 |
queue_credit_tracker | 26 | 0 | 0.0% | 100.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_01 |
verified_accu | 21 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_03 |
verified_adder_32bit | 21 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_06 |
verified_adder_64bit | 21 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_09 |
verified_div_16bit | 21 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_10 |
verified_radix2_div | 21 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_11 |
verified_multi_16bit | 21 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_13 |
verified_multi_booth_8bit | 21 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_14 |
verified_multi_pipe | 21 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_15 |
verified_multi_pipe_8bit | 21 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_control / task_01 |
verified_counter_12 | 21 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_control / task_03 |
ring_counter | 21 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_control / task_06 |
sequence_detector | 21 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_memory / task_01 |
verified_asyn_fifo | 21 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_memory / task_04 |
LFSR | 21 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_memory / task_05 |
verified_right_shifter | 21 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_05 |
verified_calendar | 21 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_06 |
verified_edge_detect | 21 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_07 |
verified_parallel2serial | 21 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_08 |
verified_pulse_detect | 21 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_09 |
verified_serial2parallel | 21 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_10 |
verified_synchronizer | 21 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_11 |
verified_traffic_light | 21 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_12 |
verified_width_8to16 | 21 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_13 |
verified_alu | 21 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_16 |
verified_pe | 21 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_17 |
verified_RAM | 21 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_19 |
verified_signal_generator | 21 | 2 | 9.5% | 9.5% | 9.5% | да | да | Open samples |
| rtllm | 13_rtllm_arithmetic / task_04 |
verified_adder_8bit | 21 | 3 | 14.3% | 14.3% | 14.3% | да | да | Open samples |
| rtllm | 13_rtllm_control / task_05 |
verified_fsm | 21 | 3 | 14.3% | 14.3% | 14.3% | да | да | Open samples |
| ilyasov | 10_fault_tolerant_safety / task_100 |
ecc_guarded_byte | 26 | 5 | 19.2% | 23.1% | 19.2% | да | да | Open samples |
| ilyasov | 01_ip_integration / task_12 |
width_upsizer | 26 | 6 | 23.1% | 23.1% | 23.1% | да | да | Open samples |
| ilyasov | 06_reset_domain_crossing / task_63 |
delayed_reset_release | 26 | 6 | 23.1% | 23.1% | 23.1% | да | да | Open samples |
| ilyasov | 08_spec_clarification / task_80 |
event_counter_alert | 26 | 6 | 23.1% | 23.1% | 23.1% | да | да | Open samples |
| ilyasov | 11_side_channel_constant_time / task_110 |
consttime_word_compare | 26 | 6 | 23.1% | 23.1% | 23.1% | да | да | Open samples |
| ilyasov | 11_side_channel_constant_time / task_111 |
consttime_byte_search | 26 | 6 | 23.1% | 23.1% | 23.1% | да | да | Open samples |
| rtllm | 13_rtllm_control / task_02 |
verified_JC_counter | 21 | 5 | 23.8% | 23.8% | 23.8% | да | да | Open samples |
| ilyasov | 07_csr_register_map_generation / task_72 |
axil_sensor_csr | 26 | 9 | 34.6% | 34.6% | 34.6% | да | да | Open samples |
| ilyasov | 03_low_power_design / task_34 |
soc_top | 26 | 10 | 38.5% | 38.5% | 38.5% | да | да | Open samples |
| ilyasov | 09_constraint_aware_timing / task_90 |
dual_mac_timing | 26 | 12 | 46.2% | 46.2% | 46.2% | да | да | Open samples |
| rtllm | 13_rtllm_arithmetic / task_18 |
float_multi | 21 | 14 | 66.7% | 66.7% | 66.7% | да | да | Open samples |
| ilyasov | 05_dft_testability / task_52 |
jtag_tap_debug | 26 | 19 | 73.1% | 73.1% | 73.1% | да | да | Open samples |
| ilyasov | 05_dft_testability / task_53 |
mbist_ram_wrapper | 26 | 19 | 73.1% | 73.1% | 73.1% | да | да | Open samples |
| ilyasov | 07_csr_register_map_generation / task_71 |
apb_lock_cfg | 26 | 21 | 80.8% | 80.8% | 80.8% | да | да | Open samples |
| ilyasov | 07_csr_register_map_generation / task_70 |
apb_timer_csr | 26 | 22 | 84.6% | 84.6% | 84.6% | да | да | Open samples |
| rtllm | 13_rtllm_arithmetic / task_17 |
fixed_point_subtractor | 21 | 18 | 85.7% | 85.7% | 85.7% | да | да | Open samples |
| ilyasov | 10_fault_tolerant_safety / task_101 |
lockstep_event_counter | 26 | 25 | 96.2% | 96.2% | 96.2% | да | да | Open samples |
| ilyasov | 00_security_vulnerability_detection / task_01 |
secure_reg | 26 | 26 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| ilyasov | 00_security_vulnerability_detection / task_06 |
shared_reg | 26 | 26 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| ilyasov | 01_ip_integration / task_11 |
axi2apb_write_bridge | 26 | 26 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| ilyasov | 02_clock_domain_crossing / task_24 |
cdc_bus_sync | 26 | 26 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| ilyasov | 03_low_power_design / task_40 |
power_aware_fsm | 26 | 26 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| ilyasov | 04_chiplet_die_to_die_integration / task_41 |
ucie_flit_packager | 26 | 26 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| ilyasov | 05_dft_testability / task_50 |
scan_reg8 | 26 | 26 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| ilyasov | 06_reset_domain_crossing / task_60 |
reset_sync_2ff | 26 | 26 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| ilyasov | 06_reset_domain_crossing / task_61 |
ordered_reset_ctrl | 26 | 26 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| ilyasov | 06_reset_domain_crossing / task_62 |
triple_reset_sequencer | 26 | 26 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| ilyasov | 07_csr_register_map_generation / task_73 |
apb_sample_csr | 26 | 26 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_arithmetic / task_02 |
verified_adder_16bit | 21 | 21 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_arithmetic / task_05 |
adder_bcd | 21 | 21 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_arithmetic / task_07 |
comparator_3bit | 21 | 21 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_arithmetic / task_08 |
comparator_4bit | 21 | 21 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_arithmetic / task_12 |
multi_8bit | 21 | 21 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_arithmetic / task_16 |
fixed_point_adder | 21 | 21 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_arithmetic / task_19 |
sub_64bit | 21 | 21 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_control / task_04 |
up_down_counter | 21 | 21 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_memory / task_02 |
LIFObuffer | 21 | 21 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_memory / task_03 |
barrel_shifter | 21 | 21 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_01 |
freq_div | 21 | 21 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_02 |
freq_divbyeven | 21 | 21 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_03 |
freq_divbyfrac | 21 | 21 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_04 |
freq_divbyodd | 21 | 21 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_14 |
clkgenerator | 21 | 21 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_15 |
instr_reg | 21 | 21 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_18 |
ROM | 21 | 21 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_20 |
square_wave | 21 | 21 | 100.0% | 100.0% | 100.0% | да | да | Open samples |