Co-authored-by: Luis Pereira <lpereira@allbesmart.pt>

Prior to this commit, after RA failure, the DU would send a UE context
release complete, without actually requesting one. This could lead to
inconsistent state between DU and CU, because the CU might just discard
this (unrequested) message, and not actually trigger a release [1].

Thus, refactor the release request into a function, and use it to
request release of a UE for which RA has not been completed. If the CU
does not know the UE, release the UE immediately.

[1] One possibility would be that during reestablishment, the CU would
request to look up an old DU UE ID when it has been released, resulting
in

    Assertion (oldUE) failed!
    In dl_rrc_message_transfer() /home/oaicicd/robert/openairinterface5g/openair2/LAYER2/NR_MAC_gNB/mac_rrc_dl_handler.c:860
    CU claims we should know UE bc7f, but we don't

In handover, we might need to remove a UE that we knew under it's
C-RNTI, for which we also have the RA process. Hence, when removing a UE
by RNTI, remove it also from RA.

Co-authored-by: rmagueta <rmagueta@allbesmart.pt>

Improve synchronization to also handle handover between gNB-DUs with
differing carrier freqs.

Refactor the RRC reconfiguration function to reuse it also later for
sending the reconfiguration including the handover trigger.

After RA, in the case of HO, we need to make sure to use the right radio
configuration (MCS table, etc). This commit is important to make HO
actually work.

Our initial CellGroupConfig always contains LCID1, because in normal
connection setup, the DU has to automatically set it up anyway. In
handover/UE Context Setup, the CU requests to set up SRB1 (which we
assume to be LCID1), but we reuse the initial CellGroupConfig. To not
have SRB1 added twice, skip in this case.

This commit implements adding a new UE on the target DU in case of
handover. We identify this case by the fact that the gNB-CU did not send
a gNB-DU UE ID (which is optional in F1AP, but not in our
implementation), such that we know there is no corresponding UE, which
therefore has to be created.

If it happens that there is a handover preparation information, we read
the UE capabilities from this information, and subsequently trigger the
creation of a new UE.

Another change is that for all bearers that we are asked to setup, we
add reestablishRLC so that the UE does a reestablishment during the
reconfiguration.

As we have to remove the reconfigurationWithSync and reestablishRLC IE
after application of the CellGroupConfig (because we might reuse the
CellGroupConfig later), free these IEs. As there are now multiple fields
to free, introduce a separate nr_mac_clean_cellgroup() function that
handles cleaning of the CellGroupConfig.
Co-authored-by: batuhanduyuler <batuhan.duyuler@firecell.io>
Co-authored-by: rmagueta <rmagueta@allbesmart.pt>

The next commit will modify the UE context setup request procedure to
(possibly) receive UE capabilities through Handover Preparation
Information, which extracts the UE capabilities for the UE to be setup.
To avoid code duplication, extract the code that will be needed in UE
capability extraction from Handover Preparation Information.

No logic changes in this commit; to be used for the handover case later.
Co-authored-by: rmagueta <rmagueta@allbesmart.pt>

This function will be used in the case of a handover for getting a new
RNTI (checking that we do not reuse an already used RNTI).

Will be reused later for handover.
Co-authored-by: batuhanduyuler <batuhan.duyuler@firecell.io>

Co-authored-by: batuhanduyuler <batuhan.duyuler@firecell.io>

Implement the Transmission Action Indicator F1AP forwarding logic, and
add it to the F1 internal representation structure. After the
transmission interrupt timer (during which SRB1 can send messages), the
UE will be out-of-sync (we assume it's not there anymore).
Co-authored-by: batuhanduyuler <batuhan.duyuler@firecell.io>
Co-authored-by: rmagueta <rmagueta@allbesmart.pt>

Implement the HandoverPreparationInformation F1AP forwarding logic, and
add it to the F1 internal representation structure.

No logic is implemented in this commit.
Co-authored-by: batuhanduyuler <batuhan.duyuler@firecell.io>
Co-authored-by: rmagueta <rmagueta@allbesmart.pt>

Refactor the RRC processing timer to be a generic "transmission
interrupt" timer, and use NR_timer_t for it. Also,
depending on the kind of RRC processing (see 38.331 table 12.1-1),
timing requirements might be different. However, the DU does not always
know which RRC processing is ongoing, and does not get this information
via F1. Hence, introduce a "worst case" delay time, which aligns with
the spec (it says these times are "minima") and is accepted by COTS UE.

The gNB MAC might need to stop transmission either for some time (RRC
processing), or continuously (transmission stop requested by CU). Those
can be handled in one common timer implementation. The continuous case
will be used in a follow-up commit.
Co-authored-by: Luis Pereira <lpereira@allbesmart.pt>

CI: remove unused code, clean up imports

* Remove pyshark, matplotlib imports, as they are not used
* Remove stats_monitor.py, as the CI does not use it
* Clean up many warnings shown by pyflakes regarding includes that we don't use

pyflakes warns about various errors, addressed here. For cls_cluster.py,
remove (unknown) sys.exit and replace through Exception, as sys.exit()
has the unwanted effect that the CI script would stop.  Instead, raise
an Exception that would show the problem in the HTML, and ensures that
the script runs until the end.

These are the warnings flagged by pyflakes.

    cls_cluster.py:297:4: undefined name 'sys'

    cls_containerize.py:39:1: 'subprocess' imported but unused
    cls_containerize.py:41:1: 'threading' imported but unused
    cls_containerize.py:43:1: 'multiprocessing.Process' imported but unused
    cls_containerize.py:43:1: 'multiprocessing.Lock' imported but unused
    cls_containerize.py:43:1: 'multiprocessing.SimpleQueue' imported but unused
    cls_containerize.py:49:1: 'cls_cluster as OC' imported but unused
    cls_containerize.py:50:1: redefinition of unused 'cls_cmd' from line 42

    cls_module.py:28:1: 'os' imported but unused
    cls_module.py:29:1: 'sys' imported but unused
    cls_module.py:34:1: 'subprocess' imported but unused
    cls_module.py:35:1: 'datetime.datetime' imported but unused
    cls_module.py:211:11: undefined name 'sshconnection'

    cls_oaicitest.py:37:1: 'pexpect' imported but unused
    cls_oaicitest.py:47:1: 'cls_cluster as OC' imported but unused
    cls_oaicitest.py:54:1: 'matplotlib.pyplot as plt' imported but unused
    cls_oaicitest.py:55:1: 'numpy as np' imported but unused

    cls_oai_html.py:34:1: 'sys' imported but unused
    cls_oai_html.py:41:1: 'multiprocessing.Process' imported but unused
    cls_oai_html.py:41:1: 'multiprocessing.Lock' imported but unused
    cls_oai_html.py:41:1: 'multiprocessing.SimpleQueue' imported but unused

    cls_physim.py:37:1: 'multiprocessing.SimpleQueue' imported but unused

    cls_static_code_analysis.py:39:1: 'time' imported but unused
    cls_static_code_analysis.py:40:1: 'multiprocessing.Process' imported but unused
    cls_static_code_analysis.py:40:1: 'multiprocessing.Lock' imported but unused
    cls_static_code_analysis.py:40:1: 'multiprocessing.SimpleQueue' imported but unused

    epc.py:41:1: 'multiprocessing.Process' imported but unused
    epc.py:41:1: 'multiprocessing.Lock' imported but unused
    epc.py:41:1: 'multiprocessing.SimpleQueue' imported but unused

    main.py:60:1: 'pexpect' imported but unused
    main.py:66:1: 'datetime' imported but unused
    main.py:68:1: redefinition of unused 'subprocess' from line 63
    main.py:70:1: 'multiprocessing.Process' imported but unused
    main.py:70:1: 'multiprocessing.Lock' imported but unused
    main.py:70:1: 'multiprocessing.SimpleQueue' imported but unused

    provideUniqueImageTag.py:2:1: 'os' imported but unused

Integration: `2024.w42`

Closes #547, #853, #854, #858, #859, #862, #844, and #746

See merge request oai/openairinterface5g!3053

* !3047 Resolve compilation warnings on ARM
* !3052 Optimize T_ID() macro.
* !3045 CI: Reduce tested DL throughput on Aerial
* !3039 fix for Other SI reception when BWP start != 0
* !2997 websrv regression
* !3041 Update NR SA Tutorials
* !3042 Update TESTBenches.md to reflect current testbench
* !3004 Hotfix to remove an unwanted TX power reduction when using multiple TX antenna ports.
* reduce ssPBCH_BlockPower in gNB config of FHI7.2 pipeline
* !3046 A testsuite and a benchmark for RLC AM mode
* !3038 Various bugfixes for gNB stability, cleanup
* !3051 PUCCH2 DTX detection
* nr_psbchsim: fix segv by providing correct get_nrUE_params()
* !3049 Improve PUSCH and PUCCH SNR stability
* !3054 Allocate and add sD to DRB NSSAI in case sD != 0xffffff

Currently, sD has not been allocated, so the octet string is empty. Fix the
allocation to correctly write&read NSSAIs in UE context setup/modification
request messages.

Reset TPC to 0 dB, after sending it to the UE, to not request new gain multiple
times before computing new value for SNR.

When running, nr_psbchsim, we get this UBsan error:

    openair1/PHY/NR_UE_ESTIMATION/nr_adjust_synch_ue.c:79:16: runtime error: member access within misaligned address 0x0ff4278c791c for type 'struct nrUE_params_t', which requires 8 byte alignment

The problem is that the corresponding definition of
get_nrUE_params() in psbchsim.c does not actually return the UE params,
but void, so we might or might not get something that resembles valid
data in nr_adjust_synch_ue.c. Fix this by including nr-uesoftmodem.h
(that provides the correct declaration) and define it correctly. Due to
type mismatches, also adapt downlink_frequency, uplink_frequency_offset,
and openair_cfg (which might result in the same type of bugs).

This adds DTX detection in PUCCH 2 receiver. This is detected by MAC
already which will drop PUCCH 2 indications that are detected as DTX.
For the moment this uses the same pucch0_dtx_threshold value indicated
in L1 section of the configuration file. The feature firstly reduces
unnecessary logging on stdout and the use of invalid CSI measurements in
the MAC scheduler.

Closes: #547

- Fixes #853, #854, #858, #859, #862
- handle repeated NGAP Initial context setup requests, thereby fixing assert:

      Assertion (!ue_p->as_security_active) failed!
      In rrc_gNB_generate_SecurityModeCommand() /home/eurecom/raymond/openairinterface5g/openair2/RRC/NR/rrc_gNB.c:2590
      logic error: security already activ

- A lot of cleanup: remove almost all use of protocol_ctxt_t in 5G RRC and use
  existing pointers instead

Introduce two tests for NR RLC AM entity:

- test_nr_rlc_am_entity
- benchmark_nr_rlc_am_entity

The benchmark can be used to verify processing time improvements for NR RLC AM
entity. The benchmark creates two entities and creates an unbalanced
communication channel between them with 10% loss packet loss and 100:1 PDU sent
ratio. This allows the lists in RLC to build up and triggers RLC AM code
related to looking through the segment/pdu lists.

Example benchmark output:

    ubuntu@zeus:~/bpodrygajlo/openairinterface5g/cmake_targets/build$ ./openair2/LAYER2/nr_rlc/tests/benchmark_nr_rlc_am_entity
    [LOG] init aborted, configuration couldn't be performed
    log init done
    CMDLINE: "./openair2/LAYER2/nr_rlc/tests/benchmark_nr_rlc_am_entity"
    [CONFIG] debug flags: 0x00400000
    2024-10-16T14:58:20+02:00
    Running ./openair2/LAYER2/nr_rlc/tests/benchmark_nr_rlc_am_entity
    Run on (8 X 4700 MHz CPU s)
    CPU Caches:
      L1 Data 48 KiB (x4)
      L1 Instruction 32 KiB (x4)
      L2 Unified 1280 KiB (x4)
      L3 Unified 12288 KiB (x1)
    Load Average: 0.48, 0.27, 0.20
    ***WARNING*** CPU scaling is enabled, the benchmark real time measurements may be noisy and will incur extra overhead.
    --------------------------------------------------------------------
    Benchmark                          Time             CPU   Iterations
    --------------------------------------------------------------------
    BM_nr_rlc_am_entity/100        50096 ns        50097 ns        13101
    BM_nr_rlc_am_entity/256       140386 ns       140387 ns         4868
    BM_nr_rlc_am_entity/1024      536323 ns       536311 ns         1309
    BM_nr_rlc_am_entity/4096     2084464 ns      2084385 ns          338
    BM_nr_rlc_am_entity/16384    8309318 ns      8309458 ns           81
    BM_nr_rlc_am_entity/20000    9843792 ns      9843774 ns           68

Tests showed that a UE connected much faster, and did not fail as often
on Msg3.

Threshold allows detection with -9 dB SNR with 8 RX antennas

Merge remote-tracking branch 'origin/remove_power_reduction_for_MIMO' into integration_2024_w42 (!3004)

This hotfix removes division by num_antenna_ports in PDSCH precoding matrix generation.
These matrices are defined for CSI reporting in this way but here we are using them for
transmission and this results in an unwanted 10*log10(num_antenna_ports) reduction in
transmit power (3dB for 2 antenna ports, 6dB for 4 antenna ports, etc.)

Reset TPC to 0 dB, after sending it to the UE, to not request new gain multiple times before computing new value for SNR