1. 28 Sep, 2024 24 commits
    • Robert Schmidt's avatar
      Implement CreateWorkspace() through script, print dir in HTML · 60b9649e
      Robert Schmidt authored
      Use a script, which can be tested independently of CI code (by running
      it on the host on which it will be used), and print the directory that
      has been prepared in the HTML.  Also, in the description, use the
      infinitive/imperative(?) of the verb to harmonize with other steps.
      60b9649e
    • Robert Schmidt's avatar
      Implement runScript in cls_cmd for LocalCmd/RemoteCmd, add test · e541aee4
      Robert Schmidt authored
      A lot of CI code is python mixed with bash, e.g.,
      
          ssh = getConnection(host)
          ssh.run('ls')
          ssh.run('echo')
      
      At least some of this CI code would benefit if it was written in a
      simple bash script, returning error codes and potentially other
      information either through stdout/stderr or files, to the calling Python
      code:
      
          ssh = runScript(host, script)
          # script does: ls; echo
      
      This commit introduces the possibility to run entire scripts. The idea
      is that the executor has a script (on localhost), which is either
      executed locally or on a remote host. For the remote host, the script is
      not copied but piped into a remotely executed bash. In both cases,
      output is either returned like the Cmd.run() function with returncode
      and mixed stdout/stderr, or optionally redirected into a file on the
      (remote) host, which can be treated further by the Python code in later
      steps.
      e541aee4
    • Robert Schmidt's avatar
    • Robert Schmidt's avatar
      51bf5c0f
    • Robert Schmidt's avatar
      Reimplement DeployObject()/UndeployObject() · 5201a5c3
      Robert Schmidt authored
      This reimplements the CI deployment functions, and makes them a bit more
      robust.  Concretely:
      
      - Introduce a generic "deployment tag" option that can be used to
        override the tag to use for specific images. By default, it is the
        current "branch-commitID[0:8]" tag name (requires the change from
        function ImageTagToUse() to CreateTag(), impacting also pull/push
        image functions)
      - Avoid sed for image names, and use an .env file that docker-compose
        picks up automatically; the deployment analyzes a potentially existing
        .env file and updates instead of simply overriding. For instance, some
        pipelines might use -asan images for the gNB and "normal" (non-asan)
        images for UE, and a simple overwriting might make the -asan image
        name tag disappear for the gNB, resulting in deployment failures).
        Finally, undeployment removes the .env file, so that there are no
        modifications in the repository after undeployment.
      - Redo the previous behavior of autodetecting asan, and use that (the
        current function always uses asan, no matter what)
      - Remove deployKind/displayedNewTags globals, as they are not necessary
      - Make the usedImage output in HTML slimmer
      - On undeployment, print service names next to undeploy analysis, and
        return success/failure.
      - Make the functions generally less verbose and easier to read
      
      Note that as of and only in this commit, deployment does not work, as
      all the YAML files have not been updated to work with this updated
      version.  The next commit adds tests for the new deployment, and updates
      one YAML file (also being used in the tests). The follow-up commit then
      modifies all YAML files.
      5201a5c3
    • Robert Schmidt's avatar
      Reimplement Native Build (CI step Build_eNB) · d81abdaf
      Robert Schmidt authored
      The BuildeNB steps either does things that are already implemented
      (created workspace), or things that we don't use (separate build_wait),
      and is generally quite convoluted when all it does is calling build_oai.
      Reimplement by reusing Create_Workspace, and make build_oai in a single
      step. To clean up, make a new module in which this function does not
      need a class instantiation. Instead, we give all necessary parameters
      directly.
      d81abdaf
    • Robert Schmidt's avatar
      3feaa93b
    • Robert Schmidt's avatar
      0d63861b
    • Robert Schmidt's avatar
      CI: Add basic unittest for cls_cmd · f21b5d2e
      Robert Schmidt authored
      f21b5d2e
    • Robert Schmidt's avatar
      cls_cmd: execute commands through bash · 606e2c74
      Robert Schmidt authored
      The next commit will use "echo -e". The problem with that is that bash
      and sh differ:
      
      - in bash: this results in "a\nb" (a+newline+b)
      - in sh: this results in -e a\nb (-e a+newline+b)
      
      The problem is that by default, commands are executed through sh, which
      nobody expects. Change to bash, which is likely more aligned with what
      people want to use.
      606e2c74
    • Robert Schmidt's avatar
      Remove superfluous cleanup/terminate/teardown XMLs · 5d5ade16
      Robert Schmidt authored
      After the previous commit, the CI will always do an undeployment step
      even if the CI failed. Hence, these XMLs are not necessary anymore. Some
      XMLs are updated to do what their "cleanup XML" counterparts did, such
      as removing additional images.
      5d5ade16
    • Robert Schmidt's avatar
    • Robert Schmidt's avatar
      Remove prematureExit/exitStatus, and implement in task main loop · 041b2b2d
      Robert Schmidt authored
      Use the previous commit on returning success from each task execution
      function, and implement handling of errors in the main loop. Add a
      script with which this is visible using Custom_Commands.
      041b2b2d
    • Robert Schmidt's avatar
      7b895488
    • Robert Schmidt's avatar
      IdleSleep(): no need for class members · e969ddb8
      Robert Schmidt authored
      e969ddb8
    • Robert Schmidt's avatar
      71807db4
    • Robert Schmidt's avatar
      Remove CI repeatCount XML property · 61ea76e7
      Robert Schmidt authored
      The repeatCount property is useless: it would be enough to specify the
      same XML multiple times to achieve the same thing.
      61ea76e7
    • Robert Schmidt's avatar
      Basic test runner script · 77a8f5fa
      Robert Schmidt authored
      77a8f5fa
    • Robert Schmidt's avatar
      Handle parameter reading and test case execution together · d1086483
      Robert Schmidt authored
      For reasons I don't understand, the XML option reading and actual test
      case execution are at different places in the code. This is problematic,
      because what this led to is that options are not passed as function
      arguments, but indirectly through class members. This led to a lot of
      "indirect code"; it's a total mess
      
      To improve this, move them together, so that parameters can be passed
      directly as function arguments later on.
      
      Because we reference the globals, mark them inside
      ExecuteActionWithParam(). That is a Python antipattern, but necessary at
      this moment. I believe that we can remove the following globals over
      time:
      
      - SSH (with new cls_cmd, which we don't need to pass)
      - EPC (replaced through a class similar to ue_module, i.e., refer to
        core networks on specific hosts through an ID, instead of a global)
      - HTML (should use templating, where individual functions give back a
        dictionary of what they want to print, and a final pass assembles the
        entire HTML file)
      - RAN/SCA/PHYSIM/CLUSTER/ldpc (can be replaced with stateless scripts
        build/run eNB/gNB on a specific node)
      d1086483
    • Jaroslava Fiedlerova's avatar
    • Jaroslava Fiedlerova's avatar
      CI: Adjust regex patterns for Iperf3 UDP log parsing · 5fd5ed43
      Jaroslava Fiedlerova authored
      Update the regex for both sender and receiver data to allow optional
      bitrate unit prefixes - accepts also no prefix. Modify the patterns for
      lost and total packets to accept negative values (`-?\d+`).
      5fd5ed43
    • Jaroslava Fiedlerova's avatar
    • Jaroslava Fiedlerova's avatar
    • Robert Schmidt's avatar
      CI Ping/Iperf: add tests, print without lock, reduce code lines · 0da2f27f
      Robert Schmidt authored
      Add some basic unit tests to verify that Iperf/Ping work. Harmonize the
      logging by only print in the "main" iperf/ping function, to avoid that
      if we test with multiple UEs, the output intermixes. Also, do one common
      mkdir before starting multiple pings/iperfs. Finally, harmonize the
      paths so that Iperf and Ping log their files in the same place.
      0da2f27f
  2. 27 Sep, 2024 7 commits
  3. 26 Sep, 2024 6 commits
  4. 24 Sep, 2024 1 commit
    • Robert Schmidt's avatar
      Merge branch 'integration_2024_w38b' into 'develop' · ccdabb4c
      Robert Schmidt authored
      Integration: `2024.w38b`
      
      Closes #839
      
      See merge request oai/openairinterface5g!2977
      
      * !2704 nr pdcp: add support for nea1
      * !2800 UE improvements
      * !2920 Receive SIB19 on nrUE
      * !2975 Remove hardcoding of CPU measurements (respect option `-q`)
      * !2976 Update latest SIMDE version
      * !2957 T: increase max number of args
      * !2916 NR UE PUSCH resource mapping optimization
      * !2787 CI: increase max number of concurrent workers to 64
      * !2974 fhi_72: allow to provide dpdk_mem_size to xran
      * !2964 CI: Replacing the UE on Aerial setup
      * !2973 CI: remove unused code and files
      ccdabb4c
  5. 23 Sep, 2024 2 commits