Кумулятивный профиль модели по завершённым sample. Синий набор столбцов на графике — текущая модель. Если выбрать вторую модель ниже, на графике появится красный набор столбцов для прямого сравнения.
Выбери другой профиль, чтобы добавить его в график и сравнить кумулятивные метрики напрямую.
| Metric | Что означает |
|---|---|
extract_pass_at_k | Доля задач, где модель хотя бы один раз выдала извлекаемый Verilog-код. |
syntax_pass_at_k | Доля задач, где модель хотя бы один раз сгенерировала синтаксически корректный RTL. |
functional_pass_at_k | Доля задач, где модель хотя бы один раз прошла functional-проверку. |
synth_pass_at_k | Доля задач, где модель хотя бы один раз сгенерировала синтезируемый RTL. Это главный итоговый рейтинг на странице. |
final_success_at_k | Доля задач, где модель хотя бы один раз полностью прошла всю цепочку проверки. |
functional_pass_rate_samples | Какой процент всех sample прошёл functional. Это метрика стабильности на уровне отдельных попыток. |
synth_pass_rate_samples | Какой процент всех sample прошёл synthesis. Это метрика устойчивости на уровне отдельных попыток. |
samples_per_task | Среднее число sample на одну задачу в данном профиле модели. |
final_success_samples | Сколько sample конкретной задачи полностью прошли всю цепочку. |
final_success_rate_samples | Доля полностью успешных sample для конкретной задачи. |
| Бенчмарк | Задач | Sample | functional_pass_at_k | synth_pass_at_k | final_success_at_k |
|---|---|---|---|---|---|
| ilyasov | 27 | 270 | 22.2% | 18.5% | 18.5% |
| rtllm | 50 | 500 | 70.0% | 38.0% | 38.0% |
Эта таблица показывает не только факт “задача закрылась”, но и насколько стабильно модель её решала по всем sample.
| Бенчмарк | Задача | Название | Sample | final_success_samples | final_success_rate_samples | functional_pass_rate_samples | synth_pass_rate_samples | synth_pass_at_k | final_success_at_k | samples_page |
|---|---|---|---|---|---|---|---|---|---|---|
| ilyasov | 01_ip_integration / task_11 |
axi2apb_write_bridge | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 01_ip_integration / task_12 |
width_upsizer | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 03_low_power_design / task_34 |
soc_top | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 03_low_power_design / task_40 |
power_aware_fsm | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 05_dft_testability / task_50 |
scan_reg8 | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 05_dft_testability / task_51 |
scan_reset_override_reg | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 05_dft_testability / task_52 |
jtag_tap_debug | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 05_dft_testability / task_53 |
mbist_ram_wrapper | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 06_reset_domain_crossing / task_60 |
reset_sync_2ff | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 06_reset_domain_crossing / task_61 |
ordered_reset_ctrl | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 06_reset_domain_crossing / task_62 |
triple_reset_sequencer | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 06_reset_domain_crossing / task_63 |
delayed_reset_release | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 07_csr_register_map_generation / task_70 |
apb_timer_csr | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 07_csr_register_map_generation / task_71 |
apb_lock_cfg | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 07_csr_register_map_generation / task_72 |
axil_sensor_csr | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 07_csr_register_map_generation / task_73 |
apb_sample_csr | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 08_spec_clarification / task_80 |
event_counter_alert | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 10_fault_tolerant_safety / task_100 |
ecc_guarded_byte | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 10_fault_tolerant_safety / task_101 |
lockstep_event_counter | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 11_side_channel_constant_time / task_110 |
consttime_word_compare | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 11_side_channel_constant_time / task_111 |
consttime_byte_search | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 12_cdv / task_121 |
queue_credit_tracker | 10 | 0 | 0.0% | 100.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_01 |
verified_accu | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_02 |
verified_adder_16bit | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_03 |
verified_adder_32bit | 10 | 0 | 0.0% | 30.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_04 |
verified_adder_8bit | 10 | 0 | 0.0% | 20.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_06 |
verified_adder_64bit | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_08 |
comparator_4bit | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_09 |
verified_div_16bit | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_10 |
verified_radix2_div | 10 | 0 | 0.0% | 80.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_11 |
verified_multi_16bit | 10 | 0 | 0.0% | 60.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_13 |
verified_multi_booth_8bit | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_14 |
verified_multi_pipe | 10 | 0 | 0.0% | 90.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_arithmetic / task_15 |
verified_multi_pipe_8bit | 10 | 0 | 0.0% | 30.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_control / task_01 |
verified_counter_12 | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_control / task_02 |
verified_JC_counter | 10 | 0 | 0.0% | 80.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_control / task_03 |
ring_counter | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_control / task_05 |
verified_fsm | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_memory / task_01 |
verified_asyn_fifo | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_memory / task_04 |
LFSR | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_memory / task_05 |
verified_right_shifter | 10 | 0 | 0.0% | 100.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_05 |
verified_calendar | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_06 |
verified_edge_detect | 10 | 0 | 0.0% | 100.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_07 |
verified_parallel2serial | 10 | 0 | 0.0% | 100.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_08 |
verified_pulse_detect | 10 | 0 | 0.0% | 100.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_09 |
verified_serial2parallel | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_10 |
verified_synchronizer | 10 | 0 | 0.0% | 100.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_11 |
verified_traffic_light | 10 | 0 | 0.0% | 40.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_12 |
verified_width_8to16 | 10 | 0 | 0.0% | 100.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_13 |
verified_alu | 10 | 0 | 0.0% | 70.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_16 |
verified_pe | 10 | 0 | 0.0% | 100.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_17 |
verified_RAM | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_19 |
verified_signal_generator | 10 | 0 | 0.0% | 0.0% | 0.0% | нет | нет | Open samples |
| ilyasov | 04_chiplet_die_to_die_integration / task_41 |
ucie_flit_packager | 10 | 1 | 10.0% | 10.0% | 10.0% | да | да | Open samples |
| ilyasov | 09_constraint_aware_timing / task_90 |
dual_mac_timing | 10 | 2 | 20.0% | 20.0% | 20.0% | да | да | Open samples |
| rtllm | 13_rtllm_arithmetic / task_05 |
adder_bcd | 10 | 2 | 20.0% | 20.0% | 20.0% | да | да | Open samples |
| rtllm | 13_rtllm_arithmetic / task_18 |
float_multi | 10 | 2 | 20.0% | 20.0% | 20.0% | да | да | Open samples |
| rtllm | 13_rtllm_control / task_06 |
sequence_detector | 10 | 3 | 30.0% | 30.0% | 30.0% | да | да | Open samples |
| ilyasov | 02_clock_domain_crossing / task_24 |
cdc_bus_sync | 10 | 5 | 50.0% | 50.0% | 50.0% | да | да | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_04 |
freq_divbyodd | 10 | 6 | 60.0% | 90.0% | 60.0% | да | да | Open samples |
| rtllm | 13_rtllm_arithmetic / task_12 |
multi_8bit | 10 | 7 | 70.0% | 70.0% | 70.0% | да | да | Open samples |
| ilyasov | 00_security_vulnerability_detection / task_01 |
secure_reg | 10 | 8 | 80.0% | 80.0% | 80.0% | да | да | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_14 |
clkgenerator | 10 | 8 | 80.0% | 80.0% | 80.0% | да | да | Open samples |
| rtllm | 13_rtllm_arithmetic / task_16 |
fixed_point_adder | 10 | 9 | 90.0% | 90.0% | 90.0% | да | да | Open samples |
| rtllm | 13_rtllm_memory / task_02 |
LIFObuffer | 10 | 9 | 90.0% | 90.0% | 90.0% | да | да | Open samples |
| rtllm | 13_rtllm_memory / task_03 |
barrel_shifter | 10 | 9 | 90.0% | 90.0% | 90.0% | да | да | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_01 |
freq_div | 10 | 9 | 90.0% | 90.0% | 90.0% | да | да | Open samples |
| ilyasov | 00_security_vulnerability_detection / task_06 |
shared_reg | 10 | 10 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_arithmetic / task_07 |
comparator_3bit | 10 | 10 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_arithmetic / task_17 |
fixed_point_subtractor | 10 | 10 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_arithmetic / task_19 |
sub_64bit | 10 | 10 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_control / task_04 |
up_down_counter | 10 | 10 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_02 |
freq_divbyeven | 10 | 10 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_03 |
freq_divbyfrac | 10 | 10 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_15 |
instr_reg | 10 | 10 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_18 |
ROM | 10 | 10 | 100.0% | 100.0% | 100.0% | да | да | Open samples |
| rtllm | 13_rtllm_miscellaneous / task_20 |
square_wave | 10 | 10 | 100.0% | 100.0% | 100.0% | да | да | Open samples |