Gitlab

SimplifyQA pipeline execution through GitLab CI/CD enables seamless test automation integrated into your DevOps workflow. This setup uses Docker containers to run SimplifyQA pipelines in a consistent and isolated environment.

Setting up GitLab Runners

Step 1: Install a GitLab Runner

1. Log in to your runner machine.

2. Download and install the GitLab Runner:

curl -L --output /usr/local/bin/gitlab-runner https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64
chmod +x /usr/local/bin/gitlab-runner 

1. Register the runner:

gitlab-runner register

• URL: Enter the GitLab instance URL (e.g., https://gitlab.com or self-hosted URL).

• Registration Token: Obtain it from Settings → CI/CD → Runners in your GitLab project.

• Description: Provide a meaningful name for the runner.

• Executor: Choose docker as the executor.

Step 2: Configure the Runner

1. Edit the configuration file (/etc/gitlab-runner/config.toml) and update the executor with Docker configuration:

concurrent = 4 
[[runners]] 
name = "gitlab-ci-demo" 
url = "https://gitlab.devopsark.com" 
id = 10 
token = "glrt-Dp-dHWyCxGzZ1ZmNbxTn" 
token_obtained_at = 2025-01-20T11:30:48Z 
token_expires_at = 0001-01-01T00:00:00Z 
executor = "docker" 
[runners.docker] 
  tls_verify = true 
  image = "registry.devopsark.com/simplifyqa/pipeline-executor/v2:latest" 
  privileged = true 
  disable_entrypoint_overwrite = false 
  oom_kill_disable = false 
  disable_cache = false 
  volumes = [ 
    "/cache", 
  ] 
  shm_size = 0 
  [runners.docker.volume_driver_ops] 
    "size" = "50G" 
  [runners.docker.auth_config] 
    "username" =  
    "password" =  

1. Restart the runner to apply the changes:

gitlab-runner restart

1. To start the gitlab runner:

gitlab-runner run

Creating a Pipeline in SimplifyQA

• Log in to your SimplifyQA account and navigate to the Pipelines section from the left-hand panel. Click on the +Pipeline button to create a new pipeline.

• Provide a meaningful and descriptive name for the pipeline to ensure easy identification.

• Choose whether you want to execute a Test Case or a Suite, and specify the corresponding Test Case ID or Suite ID.

• Select the appropriate Version and Execution Type for the pipeline.

• Configure additional details:

  • Machine Name: Specify the machine where the execution will take place.

  • Environment: Choose the desired testing environment.

  • Release and Sprint: Define the release and sprint for which you want to execute the pipeline.

• Once all details have been filled in, click Save to finalize and create the pipeline.

Configuring Variables for Pipeline Execution

• INPUT_PIPELINEID: The ID of the pipeline to execute in SimplifyQA.

• INPUT_APIKEY: Your SimplifyQA key for authentication.

  • Click on Manage account from by clicking on your profile dropdown

  • Go to Security -> Create & Manage API Token

  • Click Generate Token, name your token, and click Save.

  • Copy the generated API key and paste it into the configuration.

• INPUT_APIURL: The URL of the SimplifyQA instance.

• INPUT_THRESHOLD: The threshold for execution results.

• INPUT_VERBOSE: Toggle for detailed logs (true or false).

Modify and Execute the Pipeline in GitLab

• Navigate to the project in GitLab.

• If a new branch is required:

  • Click on the + icon dropdown and select New Branch.

  • Create the branch as needed.

• Click on the Edit icon and choose Web IDE to open the editor.

• The editor will redirect you to the .gitlab-ci.yml file.

• Make the necessary changes to the .gitlab-ci.yml file.

Below is a breakdown:

Global Configuration

• Stages: Defines the stage in which the job will execute:

simplifyqa-pipeline-execution-v2 

• Variables: Specifies global environment variables for Docker setup:

DOCKER_DRIVER: overlay2  
	DOCKER_TLS_CERTDIR: ' ' 
	DOCKER_HOST: tcp://docker:2375  
	DOCKER_IMAGE: registry.devopsark.com/simplifyqa/pipeline-executor/v2:latest 

• Services: Configures Docker-in-Docker (docker:dind) for containerized builds:

name: docker:dind 
alias: docker 

Pipeline Execution Jobs

Job: [Job ID]

Runs a SimplifyQA pipeline with specific variables:

[Job ID]:

stage: simplifyqa-pipeline-execution-v2  
	image: docker:latest  
before_script:  
	- echo "$CI_REGISTRY_PASSWORD" | docker login -u $CI_REGISTRY_USER --		password-stdin "$CI_REGISTRY" - docker pull $DOCKER_IMAGE  
script:  
	- docker run --rm  
	-e INPUT_PIPELINEID="$INPUT_PIPELINEID"  
	-e INPUT_APIKEY="$INPUT_APIKEY"  
	-e INPUT_APIURL="$INPUT_APIURL"  
	-e INPUT_THRESHOLD="$INPUT_THRESHOLD"  
	-e INPUT_VERBOSE="$INPUT_VERBOSE"  
	"$DOCKER_IMAGE"  
variables:  
	INPUT_PIPELINEID: [Job ID]  
	INPUT_APIKEY: [SimplifyQA Key] 
	INPUT_APIURL: [Simplifyqa URL] 
	INPUT_THRESHOLD: 100  
	INPUT_VERBOSE: true 

• Once changes are complete:

  • Click on Source Control.

  • Select Commit and push the changes to the respective branch.

• Return to the GitLab project page.

• From the left panel, navigate to Build > Pipeline.

• The pipeline will run automatically on this page.

• To view the job log, click on the Status of the pipeline.

• To check the pipeline status with individual logs, click on the respective pipeline entry.

• If needed, you can rerun the pipeline by clicking the Rerun icon.

Frequently Asked Questions (FAQs)

Can I run test suites automatically on code push?

Yes. You can configure GitLab pipelines to trigger SimplifyQA test runs when code is pushed, a merge request is created, or a build is deployed.

How do I authenticate SimplifyQA in GitLab?

Use a SimplifyQA API token, which should be stored as a GitLab CI/CD secret variable for secure access. This token will be used in the API request to trigger the test execution.

Do I need a specific GitLab version or tier for integration?

No specific tier is required. Integration works with any GitLab CI/CD-enabled repository, including self-hosted and GitLab.com.

What happens if a test fails during the pipeline?

If a test fails:

• The pipeline step can be configured to fail the entire job

• Defects can be auto-logged to tools like Jira (if integrated)

• Notifications can be sent to the relevant stakeholders