Easy methods to arrange SonarQube and combine it with Codemagic


This put up is written by Kalgi Shah

What’s SonarQube?

SonarQube by SonarSource is the main software for constantly inspecting the code high quality and safety of your codebase and guiding growth groups throughout code critiques. It’s an open-source software that has help for 29 programming languages as of the time of writing this text, and the quantity is rising.

SonarQube’s key options embrace:

  • Code high quality checkups: SonarQube checks the general well being of your code and, extra importantly, highlights code-related points. This makes it an awesome software for checking code high quality.
  • Clever bug detection: SonarQube gives code analyzers and makes use of highly effective path-sensitive dataflow engines that may level out errors like null deferences, logical errors, and useful resource leaks.
  • Multilanguage help: SonarQube has greater than 29 code analyzers for various languages/platforms, like C/C++, JavaScript, C#, Java, COBOL, PL/SQL, PHP, ABAP, VB.NET, Python, RPG, Flex, Goal-C, Swift, net, and extra.
  • DevOps integration: It may be simply built-in with CI/CD instruments utilizing webhooks and REST APIs.

SonarQube might be run in your native machine or as a Docker container. You may also host it on an on-premises or cloud-based server. SonarQube is available in Neighborhood, Developer, and Enterprise editions. The Neighborhood Version is free and open supply. Alternatively, although it isn’t free, the Developer Version comes with C, C++, Goal-C, Swift, ABAP, T-SQL, and PL/SQL help, department evaluation, and pull request ornament.

On this article, we are going to stroll you thru internet hosting SonarQube domestically and on an AWS EC2 occasion, in addition to implementing it right into a CI/CD pipeline in Codemagic.

Utilizing SonarQube in your native system

You’ll be able to both obtain the SonarQube Neighborhood/Developer Version ZIP file from right here or use their Docker picture. On this article, we might be utilizing SonarQube Developer Version.

You’ll be able to examine the directions right here to learn to set up the native occasion utilizing the ZIP file.

To make use of their Docker picture for testing, begin the Docker server utilizing:

docker run -d --name sonarqube -e SONAR_ES_BOOTSTRAP_CHECKS_DISABLE=true -p 9000:9000 sonarqube:newest

Now, log in to http://localhost:9000 with system administrator credentials (login=admin, password=admin).

Click on the Create Venture button. When requested the way you need to create your mission, choose Manually.

Enter a Venture show title and a Venture key and click on Set Up.

Below Present a token, enter a token title and click on Generate. Copy the token and click on Proceed. You have to this whereas operating the evaluation CLI command.

Choose your mission’s important language and observe the directions.

SonarQube has a devoted Gradle plugin referred to as SonarScanner for Gradle, which you should use to generate the SonarQube evaluation on your Android mission.

SonarQube gives Swift help within the Developer Version. Sadly, Swift just isn’t supported within the Neighborhood version. Nonetheless, you should use their SonarScanner as a CLI software for producing the SonarQube evaluation on your iOS mission.

Additionally, you possibly can at all times request a free trial of Developer Version and take a look at it out for your self.

SonarQube Android integration

Integrating SonarQube with an Android mission is fairly simple. Observe the steps under:

  1. Navigate to your app/construct.gradle and open it.

  2. Add the SonarQube Gradle plugin:

plugins {
    id "org.sonarqube" model "3.0"
}
  1. Run mission sync should you’re utilizing Android Studio or simply run ./gradlew --refresh-dependencies within the terminal.

  2. Then run the next command from the terminal:

./gradlew sonarqube 
  -Dsonar.projectKey=<project_key> 
  -Dsonar.host.url=http://localhost:9000 
  -Dsonar.login=<login_token>

iOS integration

Obtain SonarScanner from right here, and add the bin listing to the PATH atmosphere variable. With a purpose to obtain that, begin Terminal and run the next command:

These instructions will open your bash_profile in vi editor. Then add the next strains on the finish:

export PATH=$PATH:/Functions/SonarScanner/bin
export PATH=$PATH:/Functions/SonarQube/bin

Press ESC key and a colon will seem on the bottom-left nook in vi editor.
Enter wq to save lots of & give up.

Use the next command to add the evaluation outcomes:

sonar-scanner 
  -Dsonar.projectKey=<project_key> 
  -Dsonar.sources=. 
  -Dsonar.host.url=http://localhost:9000 
  -Dsonar.login=<login_token>

You will note the code evaluation standing displayed on the SonarQube dashboard. With a purpose to join Codemagic to your localhost SonarQube, you will have to make it accessible to the web. On this case, you should use ngrok. Obtain the software and observe the directions on their web site.

Connecting to SonarQube with Codemagic utilizing an AWS Linux EC2 occasion

If you want SonarQube to be obtainable to the entire staff and plan to combine it into the CI/CD pipeline, internet hosting it on the server is the best choice. Right here, we are going to look into easy methods to deploy SonarQube on an AWS EC2 occasion and combine it with Codemagic to generate a code evaluation of your Android and iOS initiatives.

We have to get a Linux EC2 server up and operating with sudo privileges earlier than putting in a sonar server. You should use a machine sort of t2 medium or bigger, as we want at the very least 3 GB of RAM to run SonarQube effectively. Additionally, add a customized TCP safety rule for the EC2 occasion to permit inbound site visitors to the chosen SonarQube port (default: 9000).

Step 1: Arrange the AWS Linux EC2 occasion

Hook up with the EC2 occasion utilizing a safe shell:

ssh -i <<path to your .pem file>> [email protected]<<ip deal with of your EC2>>

Replace the system packages on Amazon Linux 2:

Set up vim, wget, and curl on Amazon Linux 2:

sudo yum set up vim wget curl -y

Improve the vm.max_map_count kernel, file descriptor, and ulimit for the present session at runtime:

sudo sysctl -w vm.max_map_count=262144
sudo sysctl -w fs.file-max=65536
ulimit -n 65536
ulimit -u 4096

If you wish to enhance these completely, open the limits.conf config file and insert the right values, as proven under:

 sudo nano /and many others/safety/limits.conf
 sonarqube   -   nofile   65536
 sonarqube   -   nproc    4096

Set up Java. We’d like JDK 11 or greater to run SonarQube 7.9, which we’re utilizing on this weblog put up.

sudo amazon-linux-extras set up java-openjdk11
java -version  // To examine java model

You may also set up OpenJDK 11 utilizing curl.

Step 2: Set up and configure PostgreSQL 13 on Amazon Linux 2 for SonarQube

We have now to arrange a database for SonarQube to save lots of the report evaluation. This additionally helps in sustaining the report variations. We might be utilizing PostgreSQL as our database, which might be configured on EC2:

First, allow the EPEL repository on Amazon Linux 2 utilizing the command under:

sudo amazon-linux-extras set up epel

Add the PostgreSQL 13 repo in Amazon Linux 2:

sudo tee /and many others/yum.repos.d/pgdg.repo<<EOF
[pgdg13]
title=PostgreSQL 13 for RHEL/CentOS 7 - x86_64
baseurl=https://obtain.postgresql.org/pub/repos/yum/13/redhat/rhel-7-x86_64
enabled=1
gpgcheck=0
EOF

Now let’s set up and initialize PostgreSQL 13 on Amazon Linux utilizing the command under:

sudo yum set up postgresql13 postgresql13-server
sudo /usr/pgsql-13/bin/postgresql-13-setup initdb
sudo systemctl allow --now postgresql-13
sudo systemctl standing postgresql-13  //To examine PostgreSQL service

Change the password for the default PostgreSQL consumer:

sudo passwd postgres
su - postgres  //Swap to postgres consumer

Create a brand new consumer by typing:

Swap to the PostgreSQL shell:

Create a consumer and database for sonar:

ALTER USER sonar WITH ENCRYPTED password 'sonar_password';
CREATE DATABASE sonarqube OWNER sonar;
GRANT ALL PRIVILEGES ON DATABASE sonarqube to sonar;

Exit the PostgreSQL shell:

q

Swap again to the sudo consumer by operating the exit command:

exit

Step 3: Set up SonarQube on Amazon Linux 2

Now that we’ve got all of the stipulations in place, we’re going to obtain the binaries for SonarQube and use them to put in it.

Observe: Please watch out when deciding on the version that you will set up. As talked about earlier than, we’re going to use the Developer Version on this article. Extra particulars might be discovered right here. If you wish to set up a unique version, right-click on the respective obtain button, and replica the hyperlink deal with.

sudo wget https://binaries.sonarsource.com/CommercialDistribution/sonarqube-developer/sonarqube-developer-9.3.0.51899.zip

Unzip the SonarQube setup file and transfer it to the /decide listing:

sudo unzip sonarqube-developer-9.3.0.51899.zip
sudo mv -v sonarqube-9.3.0.51899 /decide/sonarqube

Step 4: Configure SonarQube on Amazon Linux 2

Working the SonarQube occasion as a root consumer causes it to cease operating. You’ll be able to create a brand new group and consumer to beat this challenge.

Create the group first. We’ll name it sonar:

Now, add the consumer with listing entry:

sudo useradd -c "consumer to run SonarQube" -d /decide/sonarqube -g sonar sonar 
sudo chown -R sonar:sonar /decide/sonarqube

Open the SonarQube configuration file utilizing your favourite textual content editor. We’ll be utilizing nano on this instance:

sudo nano /decide/sonarqube/conf/sonar.properties

Discover the next strains:

#sonar.jdbc.username=
#sonar.jdbc.password=

Uncomment and sort the PostgreSQL database username and password that we created within the steps above. Add the PostgreSQL connection string.

sonar.jdbc.username=sonar
sonar.jdbc.password=sonar
sonar.jdbc.url=jdbc:postgresql://localhost:5432/sonarqube

Within the sonar script file, uncomment RUN_AS_USER and alter it to RUN_AS_USER=sonar.

sudo nano /decide/sonarqube/bin/linux-x86-64/sonar.sh

Kind CTRL+X to save lots of and shut the file.

Step 5: Begin SonarQube

Now, it’s time to start out SonarQube.

Swap to the sonar consumer:

Transfer to the script listing:

cd /decide/sonarqube/bin/linux-x86-64/

Run the script to start out SonarQube:

Test SonarQube’s operating standing:

To examine the SonarQube logs, navigate to the /decide/sonarqube/logs/sonar.log listing.

Step 6: Configure the systemd service for SonarQube

First, cease the SonarQube service, as we began it manually utilizing the steps above. Navigate to SonarQube’s set up path:

cd /decide/sonarqube/bin/linux-x86-64/
./sonar.sh cease

Create a systemd service file for SonarQube to run at system startup:

sudo nano /and many others/systemd/system/sonar.service

Add the next strains:

[Unit]
Description=SonarQube service
After=syslog.goal community.goal

[Service]
Kind=forking

ExecStart=/decide/sonarqube/bin/linux-x86-64/sonar.sh begin
ExecStop=/decide/sonarqube/bin/linux-x86-64/sonar.sh cease

Person=sonar
Group=sonar
Restart=at all times

LimitNOFILE=65536
LimitNPROC=4096

[Install]
WantedBy=multi-user.goal

Save and shut the file.

Now, let’s cease the SonarQube script we began to run:

Begin the SonarQube daemon by operating:

sudo systemctl begin sonar

Allow the SonarQube service to mechanically begin at system startup:

sudo systemctl allow sonar
sudo systemctl standing sonar

Step 7: Entry the SonarQube UI

If you’re already inside your occasion, you will get the general public IP of your Linux EC2 occasion utilizing the command under:

curl -s v4.ident.me

Your SonarQube ought to be up now. You’ll be able to entry the SonarQube UI at http://<<EC2 occasion public ip>>:9000/sonarqube. By default, the credentials stay login=admin and password=admin.

The subsequent step is so as to add SonarQube to our CI/CD atmosphere in order that code high quality checks might be mechanically triggered by sure occasions.

Step 8: Utilizing SonarQube with Codemagic

We will simply combine SonarQube with Codemagic utilizing the codemagic.yaml file. Codemagic just lately labored with Christophe Havard (Product Supervisor at SonarSource) so as to add Codemagic to the checklist of supported CIs for department and pull-request detection. You’ll be able to examine the SonarQube launch notes right here.

To combine SonarQube with Codemagic, we might want to set the SONAR_TOKEN, SONARQUBE_URL, and SONAR_PROJECT_KEY atmosphere variables within the Codemagic UI, as proven under. Mark the atmosphere variables as safe, and add the respective group (sonarqube) to the codemagic.yaml file.

Additionally, navigate to your app/construct.gradle and add the SonarQube Gradle plugin:

plugins {
    id "org.sonarqube" model "3.0"
}

Let’s outline the construct pipeline script within the codemagic.yaml file for each the Android and iOS initiatives. In these examples we’re utilizing a premium Codemagic’s Mac Professional construct machine. We wish our code high quality checks to be triggered on each commits and pull requests, so we specify that within the triggering part. You may also reference the pattern Android and iOS YAML file configurations.

Android mission with SonarQube integration

workflows:
  android-workflow:
    title: Android Workflow
    instance_type: mac_pro
    cache:
      cache_paths:
        - ~/.sonar
    atmosphere:
      teams:
        - sonarqube # consists of SONAR_TOKEN, SONARQUBE_URL, SONAR_PROJECT_KEY
    triggering:
      occasions:
        - push
        - pull_request
      branch_patterns:
        - sample: '*'
          embrace: true
          supply: true
    scripts:     
      - title: Construct Android app
        script: |
                    ./gradlew assembleDebug
      - title: Generate and add code evaluation report
        script: |
           ./gradlew sonarqube 
           -Dsonar.projectKey=$SONAR_PROJECT_KEY 
           -Dsonar.host.url=$SONARQUBE_URL 
           -Dsonar.login=$SONAR_TOKEN           

As soon as the construct is profitable, you possibly can examine your code evaluation on the SonarQube UI.

iOS mission with SonarQube integration

For the iOS construct evaluation, we first must obtain and add the SonarScanner to the trail.

SonarScanners operating in Codemagic can mechanically detect branches and merge or pull requests in sure jobs.

workflows:
  ios-workflow:
    title: ios_workflow
    instance_type: mac_pro
    cache:
      cache_paths:
        - ~/.sonar
    atmosphere:
      teams:
        - sonar
      vars:
        XCODE_WORKSPACE: "Sonar.xcodeproj"  # PUT YOUR WORKSPACE NAME HERE
        XCODE_SCHEME: "Sonar" # PUT THE NAME OF YOUR SCHEME HERE
      xcode: newest
      cocoapods: default
    triggering:
      occasions:
        - push
        - pull_request
      branch_patterns:
        - sample: '*'
          embrace: true
          supply: true
    scripts:
      - title: Run checks
        script: |
          xcodebuild 
          -project "$XCODE_WORKSPACE" 
          -scheme "$XCODE_SCHEME" 
          -sdk iphonesimulator 
          -destination 'platform=iOS Simulator,title=iPhone 12,OS=15.4' 
          clear construct take a look at CODE_SIGN_IDENTITY="" CODE_SIGNING_REQUIRED=NO          
      - title: Construct debug app
        script: |
          xcodebuild construct -project "$XCODE_WORKSPACE" 
          -scheme "$XCODE_SCHEME" 
          CODE_SIGN_IDENTITY="" CODE_SIGNING_REQUIRED=NO CODE_SIGNING_ALLOWED=NO          
      - title: Sonar
        script: |
            # obtain and set up the SonarScanner
            wget -O $FCI_BUILD_DIR/sonar-scanner.zip https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-4.4.0.2170-macosx.zip
            unzip $FCI_BUILD_DIR/sonar-scanner.zip
            mv sonar-scanner-* sonar-scanner            
      - title: Protection checks
        script: |
            xcodebuild 
            -project "$XCODE_WORKSPACE" 
            -scheme "$XCODE_SCHEME" 
            -sdk iphonesimulator 
            -destination 'platform=iOS Simulator,title=iPhone 11 Professional,OS=15.4' 
            -derivedDataPath Construct/ 
            -enableCodeCoverage YES 
            clear construct take a look at CODE_SIGN_IDENTITY="" CODE_SIGNING_REQUIRED=NO            
      - title: convert protection report back to sonarqube format
        script: |
                        bash xccov-to-sonarqube-generic.sh Construct/Logs/Take a look at/*.xcresult/ > sonarqube-generic-coverage.xml
      - title: Generate and add code evaluation report
        script: |
            export PATH=$PATH:$FCI_BUILD_DIR/sonar-scanner/bin
            sonar-scanner 
            -Dsonar.projectKey=$SONAR_PROJECT_KEY 
            -Dsonar.host.url=$SONARQUBE_URL 
            -Dsonar.c.file.suffixes=- 
            -Dsonar.cpp.file.suffixes=- 
            -Dsonar.coverageReportPaths=sonarqube-generic-coverage.xml             
    artifacts:
      - $HOME/Library/Developer/Xcode/DerivedData/**/Construct/**/*.app
      - /tmp/xcodebuild_logs/*.log
      - $HOME/Library/Developer/Xcode/DerivedData/**/Construct/**/*.dSYM
    publishing:
      electronic mail:
        recipients:
            - [email protected]

Protection studies can’t be uploaded on this format, so that you’ll have to make use of the next script to transform it from an .xcresult to an .xml file:

File title: xccov-to-sonarqube-generic.sh

#!/usr/bin/env bash
set -euo pipefail

operate convert_file  
    sed -n '
    s/^ *([0-9][0-9]*): 0.*$/    <lineToCover lineNumber="1" lined="false"/>/p;
    s/^ *([0-9][0-9]*): [1-9].*$/    <lineToCover lineNumber="1" lined="true"/>/p
    '
  echo '  </file>'


operate xccov_to_generic  whereas learn -r file_name; do
      convert_file "$xccovarchive_file" "$file_name"
    executed
  executed
  echo '</protection>'


xccov_to_generic "[email protected]"

Run this script utilizing:

bash xccov-to-sonarqube-generic.sh Construct/Logs/Take a look at/*.xcresult/ > sonarqube-generic-coverage.xml

Move the outcome to SonarQube by specifying the next properties:

-Dsonar.cfamily.build-wrapper-output.bypass=true 
-Dsonar.coverageReportPaths=sonarqube-generic-coverage.xml 
-Dsonar.c.file.suffixes=- 
-Dsonar.cpp.file.suffixes=- 
-Dsonar.objc.file.suffixes=-

And that’s it! We have now efficiently built-in SonarQube with Codemagic to run for each Android and iOS initiatives. Earlier than we wrap up, let’s talk about two minor particulars you could have observed within the YAML configurations above which might be associated to automated triggering and caching.

Robotically detecting pull requests

For SonarQube to mechanically detect pull requests when utilizing Codemagic, it’s essential to add an occasion within the triggering part of your codemagic.yaml file, as proven within the following snippet:

    triggering:
      occasions:
        - pull_request

You’ll have observed this within the YAML scripts above. However that’s not all: For triggering to work, you additionally must arrange a webhook between Codemagic and your repository (e.g., Bitbucket, GitHub).

Caching the .sonar folder

Caching the .sonar folder will save construct time on subsequent analyses. To do that, add the next snippet to your codemagic.yaml file:

    cache:
      cache_paths:
        - ~/.sonar

Conclusion

Integrating SonarQube with Codemagic is absolutely easy when utilizing the codemagic.yaml file. On this put up, we’ve got lined the fundamental configuration wanted to generate the code evaluation report, however there are numerous different properties that you could specify utilizing SonarQube, particularly with SonarQube Developer Version. Nonetheless, even the free tier could also be sufficient to considerably enhance the code high quality of your mission by automating reporting with SonarQube and Codemagic.

Attempt it! And when you have any questions or recommendations, we’re at all times comfortable to listen to them on our Slack or on Twitter (simply tag @codemagicio).

Study extra



Related Articles

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Latest Articles