After handover, the last CellGroupConfig received is the one for
handover, containing a reconfigurationWithSync. Resending this
CellGroupConfig to the UE after reestablishment would trigger a
handover. Free the reconfigurationWithSync to have the UE complete
reestablishment at the current cell.

The CU implementation of reestablishment modifies the CellGroupConfig.
It should not do this, but does as of now. The specification (38.473 Sec
8.7) foresees an optional F1 UE context modification procedure (which
can be used to receive the CellGroupConfig). This should be done instead
to retrieve the correct CellGroupConfig.

Note that the F1 UE context setup procedure, similar to the UE context
modification procedure, would also allow to perform reestablishment on a
DU that is not the original one.

If we release a request from a DU during, handle as follows:

- from the source: Handover is still ongoing, so ignore it by
  acknowledging the release, and marking handover as done. In this case,
  we did not receive an acknowledgement from the target, but since the
  source DU does not keep the UE context, we could not cancel the
  handover anymore either. The UE might come back on any DU through
  reestablishment (or simply be connected on the target)
- from the target: consider this to be a genuine release, and forward to
  AMF.
Co-authored-by: batuhanduyuler <batuhan.duyuler@firecell.io>
Co-authored-by: rmagueta <rmagueta@allbesmart.pt>

The HO is completed (from a pure RAN perspective) when receiving
reconfiguration complete. In that case, remove the HO context, and
signal successful handover through the callback. For the F1 handover,
this means releasing the UE at the source DU.
Co-authored-by: batuhanduyuler <batuhan.duyuler@firecell.io>
Co-authored-by: rmagueta <rmagueta@allbesmart.pt>

We expect the UE's RRC Reconfiguration Complete to come through the
target DU, as the "new UE" at the target CU. Also, function
CU_handle_UL_RRC_MESSAGE_TRANSFER() checks that the UL RRC Message,
through which we will receive RRC Reconfiguration Complete, matches the
DU UE ID stored in RRC. Since the DU UE ID will be the one at the target
DU, while up to now, we stored the DU UE ID at the source DU.

Correspondingly, update the DU UE ID to the one at the target DU; also
update the corresponding assoc_id.
Co-authored-by: batuhanduyuler <batuhan.duyuler@firecell.io>
Co-authored-by: rmagueta <rmagueta@allbesmart.pt>

In this commit, handle the reception of a UE context setup response, in
the case of handover. In that case, store the gNB-DU UE ID and C-RNTI of
the UE at the target DU, and trigger a reconfiguration towards the UE at
the source DU via the callback.

To ensure that the CU-UP's PDCP is SN-aligned with the UE, use
cuup_notify_reestablishment() to reset the bearers with PDCP
reestablishment, which has already been implemented in the last commit.
Co-authored-by: batuhanduyuler <batuhan.duyuler@firecell.io>
Co-authored-by: rmagueta <rmagueta@allbesmart.pt>

This is preparatory work to handle the UE context setup response from
the target DU. There are two cases:

- "Initial" UE context setup response (during connection setup): we need
  to inform the CU-UP via an E1 message of the new bearers (GTP tunnel
  update)
- F1 handover:  we need to inform the CU-UP via an E1 message that there
  are DRBs to reestablish

There are two different functions, e1_send_bearer_updates() and
cuup_notify_reestablishment(), to do this. Harmonize the code, before
implementing the actual handover logic in the next commit.

Add a function to trigger an F1 handover, including the corresponding
telnet function implementation. The function looks up source&target DU,
creates a HO context, and reads PDU information to send a UE context
setup request to the target DU towards which to handover.
Co-authored-by: batuhanduyuler <batuhan.duyuler@firecell.io>
Co-authored-by: rmagueta <rmagueta@allbesmart.pt>

Functions get_qos_char_from_qos_flow_param() and
get_qos_characteristics() will be used in rrc_gNB_mobility.c to read PDU
information for handover. Move the definitions to a header to reuse
later.

HandoverPreparationInformation will be used in the follow-up commit to
trigger a handover.
Co-authored-by: batuhanduyuler <batuhan.duyuler@firecell.io>
Co-authored-by: rmagueta <rmagueta@allbesmart.pt>

This F1 message sends us the CellGroupConfig of the UE. Store it in case
we need it. For instance, at least in OAI, it is possible to make a
handover without a DRB (this might or might not be standard compliant).
However, in this case, we did never receive a UE context setup request,
which stores the CellGroupConfig. Hence, store the first CellGroupConfig
we receive, to be able to do e.g., handover.

RETURN_IF_INVALID_ASSOC_ID() checks if an assoc_id (of a DU) is valid.
It will be also used in the mobility RRC module later. Move it to the DU
header (where it logically belongs) to move it from there.

Co-authored-by: Luis Pereira <lpereira@allbesmart.pt>

Prior to this commit, after RA failure, the DU would send a UE context
release complete, without actually requesting one. This could lead to
inconsistent state between DU and CU, because the CU might just discard
this (unrequested) message, and not actually trigger a release [1].

Thus, refactor the release request into a function, and use it to
request release of a UE for which RA has not been completed. If the CU
does not know the UE, release the UE immediately.

[1] One possibility would be that during reestablishment, the CU would
request to look up an old DU UE ID when it has been released, resulting
in

    Assertion (oldUE) failed!
    In dl_rrc_message_transfer() /home/oaicicd/robert/openairinterface5g/openair2/LAYER2/NR_MAC_gNB/mac_rrc_dl_handler.c:860
    CU claims we should know UE bc7f, but we don't

In handover, we might need to remove a UE that we knew under it's
C-RNTI, for which we also have the RA process. Hence, when removing a UE
by RNTI, remove it also from RA.

Co-authored-by: rmagueta <rmagueta@allbesmart.pt>

Improve synchronization to also handle handover between gNB-DUs with
differing carrier freqs.

Refactor the RRC reconfiguration function to reuse it also later for
sending the reconfiguration including the handover trigger.

After RA, in the case of HO, we need to make sure to use the right radio
configuration (MCS table, etc). This commit is important to make HO
actually work.

Our initial CellGroupConfig always contains LCID1, because in normal
connection setup, the DU has to automatically set it up anyway. In
handover/UE Context Setup, the CU requests to set up SRB1 (which we
assume to be LCID1), but we reuse the initial CellGroupConfig. To not
have SRB1 added twice, skip in this case.

This commit implements adding a new UE on the target DU in case of
handover. We identify this case by the fact that the gNB-CU did not send
a gNB-DU UE ID (which is optional in F1AP, but not in our
implementation), such that we know there is no corresponding UE, which
therefore has to be created.

If it happens that there is a handover preparation information, we read
the UE capabilities from this information, and subsequently trigger the
creation of a new UE.

Another change is that for all bearers that we are asked to setup, we
add reestablishRLC so that the UE does a reestablishment during the
reconfiguration.

As we have to remove the reconfigurationWithSync and reestablishRLC IE
after application of the CellGroupConfig (because we might reuse the
CellGroupConfig later), free these IEs. As there are now multiple fields
to free, introduce a separate nr_mac_clean_cellgroup() function that
handles cleaning of the CellGroupConfig.
Co-authored-by: batuhanduyuler <batuhan.duyuler@firecell.io>
Co-authored-by: rmagueta <rmagueta@allbesmart.pt>

The next commit will modify the UE context setup request procedure to
(possibly) receive UE capabilities through Handover Preparation
Information, which extracts the UE capabilities for the UE to be setup.
To avoid code duplication, extract the code that will be needed in UE
capability extraction from Handover Preparation Information.

No logic changes in this commit; to be used for the handover case later.
Co-authored-by: rmagueta <rmagueta@allbesmart.pt>

This function will be used in the case of a handover for getting a new
RNTI (checking that we do not reuse an already used RNTI).

Will be reused later for handover.
Co-authored-by: batuhanduyuler <batuhan.duyuler@firecell.io>

Co-authored-by: batuhanduyuler <batuhan.duyuler@firecell.io>

Implement the Transmission Action Indicator F1AP forwarding logic, and
add it to the F1 internal representation structure. After the
transmission interrupt timer (during which SRB1 can send messages), the
UE will be out-of-sync (we assume it's not there anymore).
Co-authored-by: batuhanduyuler <batuhan.duyuler@firecell.io>
Co-authored-by: rmagueta <rmagueta@allbesmart.pt>

Implement the HandoverPreparationInformation F1AP forwarding logic, and
add it to the F1 internal representation structure.

No logic is implemented in this commit.
Co-authored-by: batuhanduyuler <batuhan.duyuler@firecell.io>
Co-authored-by: rmagueta <rmagueta@allbesmart.pt>

Refactor the RRC processing timer to be a generic "transmission
interrupt" timer, and use NR_timer_t for it. Also,
depending on the kind of RRC processing (see 38.331 table 12.1-1),
timing requirements might be different. However, the DU does not always
know which RRC processing is ongoing, and does not get this information
via F1. Hence, introduce a "worst case" delay time, which aligns with
the spec (it says these times are "minima") and is accepted by COTS UE.

The gNB MAC might need to stop transmission either for some time (RRC
processing), or continuously (transmission stop requested by CU). Those
can be handled in one common timer implementation. The continuous case
will be used in a follow-up commit.
Co-authored-by: Luis Pereira <lpereira@allbesmart.pt>

CI: remove unused code, clean up imports

* Remove pyshark, matplotlib imports, as they are not used
* Remove stats_monitor.py, as the CI does not use it
* Clean up many warnings shown by pyflakes regarding includes that we don't use

pyflakes warns about various errors, addressed here. For cls_cluster.py,
remove (unknown) sys.exit and replace through Exception, as sys.exit()
has the unwanted effect that the CI script would stop.  Instead, raise
an Exception that would show the problem in the HTML, and ensures that
the script runs until the end.

These are the warnings flagged by pyflakes.

    cls_cluster.py:297:4: undefined name 'sys'

    cls_containerize.py:39:1: 'subprocess' imported but unused
    cls_containerize.py:41:1: 'threading' imported but unused
    cls_containerize.py:43:1: 'multiprocessing.Process' imported but unused
    cls_containerize.py:43:1: 'multiprocessing.Lock' imported but unused
    cls_containerize.py:43:1: 'multiprocessing.SimpleQueue' imported but unused
    cls_containerize.py:49:1: 'cls_cluster as OC' imported but unused
    cls_containerize.py:50:1: redefinition of unused 'cls_cmd' from line 42

    cls_module.py:28:1: 'os' imported but unused
    cls_module.py:29:1: 'sys' imported but unused
    cls_module.py:34:1: 'subprocess' imported but unused
    cls_module.py:35:1: 'datetime.datetime' imported but unused
    cls_module.py:211:11: undefined name 'sshconnection'

    cls_oaicitest.py:37:1: 'pexpect' imported but unused
    cls_oaicitest.py:47:1: 'cls_cluster as OC' imported but unused
    cls_oaicitest.py:54:1: 'matplotlib.pyplot as plt' imported but unused
    cls_oaicitest.py:55:1: 'numpy as np' imported but unused

    cls_oai_html.py:34:1: 'sys' imported but unused
    cls_oai_html.py:41:1: 'multiprocessing.Process' imported but unused
    cls_oai_html.py:41:1: 'multiprocessing.Lock' imported but unused
    cls_oai_html.py:41:1: 'multiprocessing.SimpleQueue' imported but unused

    cls_physim.py:37:1: 'multiprocessing.SimpleQueue' imported but unused

    cls_static_code_analysis.py:39:1: 'time' imported but unused
    cls_static_code_analysis.py:40:1: 'multiprocessing.Process' imported but unused
    cls_static_code_analysis.py:40:1: 'multiprocessing.Lock' imported but unused
    cls_static_code_analysis.py:40:1: 'multiprocessing.SimpleQueue' imported but unused

    epc.py:41:1: 'multiprocessing.Process' imported but unused
    epc.py:41:1: 'multiprocessing.Lock' imported but unused
    epc.py:41:1: 'multiprocessing.SimpleQueue' imported but unused

    main.py:60:1: 'pexpect' imported but unused
    main.py:66:1: 'datetime' imported but unused
    main.py:68:1: redefinition of unused 'subprocess' from line 63
    main.py:70:1: 'multiprocessing.Process' imported but unused
    main.py:70:1: 'multiprocessing.Lock' imported but unused
    main.py:70:1: 'multiprocessing.SimpleQueue' imported but unused

    provideUniqueImageTag.py:2:1: 'os' imported but unused

Integration: `2024.w42`

Closes #547, #853, #854, #858, #859, #862, #844, and #746

See merge request oai/openairinterface5g!3053

* !3047 Resolve compilation warnings on ARM
* !3052 Optimize T_ID() macro.
* !3045 CI: Reduce tested DL throughput on Aerial
* !3039 fix for Other SI reception when BWP start != 0
* !2997 websrv regression
* !3041 Update NR SA Tutorials
* !3042 Update TESTBenches.md to reflect current testbench
* !3004 Hotfix to remove an unwanted TX power reduction when using multiple TX antenna ports.
* reduce ssPBCH_BlockPower in gNB config of FHI7.2 pipeline
* !3046 A testsuite and a benchmark for RLC AM mode
* !3038 Various bugfixes for gNB stability, cleanup
* !3051 PUCCH2 DTX detection
* nr_psbchsim: fix segv by providing correct get_nrUE_params()
* !3049 Improve PUSCH and PUCCH SNR stability
* !3054 Allocate and add sD to DRB NSSAI in case sD != 0xffffff

Currently, sD has not been allocated, so the octet string is empty. Fix the
allocation to correctly write&read NSSAIs in UE context setup/modification
request messages.

Reset TPC to 0 dB, after sending it to the UE, to not request new gain multiple
times before computing new value for SNR.