Кумулятивный профиль модели по завершённым sample. Синий набор столбцов на графике — текущая модель. Если выбрать вторую модель ниже, на графике появится красный набор столбцов для прямого сравнения.
Выбери другой профиль, чтобы добавить его в график и сравнить кумулятивные метрики напрямую.
| Metric | Что означает |
|---|---|
extract_pass_at_k | Доля задач, где модель хотя бы один раз выдала извлекаемый Verilog-код. |
syntax_pass_at_k | Доля задач, где модель хотя бы один раз сгенерировала синтаксически корректный RTL. |
functional_pass_at_k | Доля задач, где модель хотя бы один раз прошла functional-проверку. |
synth_pass_at_k | Доля задач, где модель хотя бы один раз сгенерировала синтезируемый RTL. Это главный итоговый рейтинг на странице. |
final_success_at_k | Доля задач, где модель хотя бы один раз полностью прошла всю цепочку проверки. |
functional_pass_rate_samples | Какой процент всех sample прошёл functional. Это метрика стабильности на уровне отдельных попыток. |
synth_pass_rate_samples | Какой процент всех sample прошёл synthesis. Это метрика устойчивости на уровне отдельных попыток. |
samples_per_task | Среднее число sample на одну задачу в данном профиле модели. |
final_success_samples | Сколько sample конкретной задачи полностью прошли всю цепочку. |
final_success_rate_samples | Доля полностью успешных sample для конкретной задачи. |
| Бенчмарк | Задач | Sample | functional_pass_at_k | synth_pass_at_k | final_success_at_k |
|---|---|---|---|---|---|
| ilyasov | 27 | 540 | 22.2% | 18.5% | 18.5% |
| rtllm | 50 | 1000 | 32.0% | 32.0% | 32.0% |
Эта таблица показывает не только факт “задача закрылась”, но и насколько стабильно модель её решала по всем sample.
| Бенчмарк | Задача | Название | Sample | final_success_samples | final_success_rate_samples | functional_pass_rate_samples | synth_pass_rate_samples | synth_pass_at_k | final_success_at_k | samples_page |
|---|---|---|---|---|---|---|---|---|---|---|
| ilyasov | 01_ip_integration / task_11 |
axi2apb_write_bridge | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 01_ip_integration / task_12 |
width_upsizer | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 03_low_power_design / task_34 |
soc_top | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 03_low_power_design / task_40 |
power_aware_fsm | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 05_dft_testability / task_51 |
scan_reset_override_reg | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 05_dft_testability / task_52 |
jtag_tap_debug | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 05_dft_testability / task_53 |
mbist_ram_wrapper | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 06_reset_domain_crossing / task_60 |
reset_sync_2ff | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 06_reset_domain_crossing / task_61 |
ordered_reset_ctrl | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 06_reset_domain_crossing / task_62 |
triple_reset_sequencer | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 06_reset_domain_crossing / task_63 |
delayed_reset_release | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 07_csr_register_map_generation / task_70 |
apb_timer_csr | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 07_csr_register_map_generation / task_71 |
apb_lock_cfg | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 07_csr_register_map_generation / task_72 |
axil_sensor_csr | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 07_csr_register_map_generation / task_73 |
apb_sample_csr | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 08_spec_clarification / task_80 |
event_counter_alert | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 09_constraint_aware_timing / task_90 |
dual_mac_timing | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 10_fault_tolerant_safety / task_100 |
ecc_guarded_byte | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 10_fault_tolerant_safety / task_101 |
lockstep_event_counter | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 11_side_channel_constant_time / task_110 |
consttime_word_compare | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 11_side_channel_constant_time / task_111 |
consttime_byte_search | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 12_cdv / task_121 |
queue_credit_tracker | 20 | 0 | 0.0% | 100.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_01 |
verified_accu | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_02 |
verified_adder_16bit | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_03 |
verified_adder_32bit | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_04 |
verified_adder_8bit | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_05 |
adder_bcd | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_06 |
verified_adder_64bit | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_09 |
verified_div_16bit | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_10 |
verified_radix2_div | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_11 |
verified_multi_16bit | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_12 |
multi_8bit | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_13 |
verified_multi_booth_8bit | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_14 |
verified_multi_pipe | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_15 |
verified_multi_pipe_8bit | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_control / task_01 |
verified_counter_12 | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_control / task_02 |
verified_JC_counter | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_control / task_03 |
ring_counter | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_control / task_05 |
verified_fsm | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_control / task_06 |
sequence_detector | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_memory / task_01 |
verified_asyn_fifo | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_memory / task_03 |
barrel_shifter | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_memory / task_04 |
LFSR | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_memory / task_05 |
verified_right_shifter | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_05 |
verified_calendar | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_06 |
verified_edge_detect | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_07 |
verified_parallel2serial | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_08 |
verified_pulse_detect | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_09 |
verified_serial2parallel | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_10 |
verified_synchronizer | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_11 |
verified_traffic_light | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_12 |
verified_width_8to16 | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_13 |
verified_alu | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_16 |
verified_pe | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_17 |
verified_RAM | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_19 |
verified_signal_generator | 20 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_17 |
fixed_point_subtractor | 20 | 2 | 10.0% | 10.0% | 10.0% | да | да | Open samples |
| ilyasov | 00_security_vulnerability_detection / task_01 |
secure_reg | 20 | 3 | 15.0% | 15.0% | 15.0% | да | да | Open samples |
| rtllm | 13_rtllm_arithmetic / task_16 |
fixed_point_adder | 20 | 3 | 15.0% | 15.0% | 15.0% | да | да | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_03 |
freq_divbyfrac | 20 | 3 | 15.0% | 15.0% | 15.0% | да | да | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_04 |
freq_divbyodd | 20 | 3 | 15.0% | 15.0% | 15.0% | да | да | Open samples |
| ilyasov | 04_chiplet_die_to_die_integration / task_41 |
ucie_flit_packager | 20 | 5 | 25.0% | 25.0% | 25.0% | да | да | Open samples |
| ilyasov | 00_security_vulnerability_detection / task_06 |
shared_reg | 20 | 7 | 35.0% | 35.0% | 35.0% | да | да | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_20 |
square_wave | 20 | 7 | 35.0% | 35.0% | 35.0% | да | да | Open samples |
| ilyasov | 05_dft_testability / task_50 |
scan_reg8 | 20 | 8 | 40.0% | 40.0% | 40.0% | да | да | Open samples |
| ilyasov | 02_clock_domain_crossing / task_24 |
cdc_bus_sync | 20 | 11 | 55.0% | 55.0% | 55.0% | да | да | Open samples |
| rtllm | 13_rtllm_memory / task_02 |
LIFObuffer | 20 | 12 | 60.0% | 60.0% | 60.0% | да | да | Open samples |
| rtllm | 13_rtllm_arithmetic / task_08 |
comparator_4bit | 20 | 13 | 65.0% | 65.0% | 65.0% | да | да | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_14 |
clkgenerator | 20 | 13 | 65.0% | 70.0% | 65.0% | да | да | Open samples |
| rtllm | 13_rtllm_arithmetic / task_18 |
float_multi | 20 | 14 | 70.0% | 70.0% | 70.0% | да | да | Open samples |
| rtllm | 13_rtllm_arithmetic / task_07 |
comparator_3bit | 20 | 18 | 90.0% | 90.0% | 90.0% | да | да | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_01 |
freq_div | 20 | 18 | 90.0% | 90.0% | 90.0% | да | да | Open samples |
| rtllm | 13_rtllm_arithmetic / task_19 |
sub_64bit | 20 | 19 | 95.0% | 95.0% | 95.0% | да | да | Open samples |
| rtllm | 13_rtllm_control / task_04 |
up_down_counter | 20 | 20 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_02 |
freq_divbyeven | 20 | 20 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_15 |
instr_reg | 20 | 20 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_18 |
ROM | 20 | 20 | 100.0% | 100.0% | 100.0% | да | да | Open samples |