Fix several issues in ulsch_mmse_2layers
Fix incorrect array type of determ_fin which was set to int32_t while the array holds the results of simde_mm_abs_epi32 which produces unsigned values. Remove assert on right-shifting negative integers as it is impossible for determ_fin to hold negative values. Change type of sum_det to uint32_t to prevent possible overflow. Fix the type cast (int -> uint32_t) to prevent casting before shift in nr_ulsch_comp_muli_sum which might have resulted in right shifting negative integers.
Showing
Please register or sign in to comment