Pipeline Issues¶

This page provides solutions to some common issues you may encounter when using the pipeline feature.

Error when running a pipeline¶

High network communication delay can lead to pipeline running errors, if Jenkins and the application are deployed in different data centers. The error message is similar to:

E0113 01:47:27.690555 50 request.go:1058] Unexpected error when reading response body: net/http: request canceled (Client.Timeout or context cancellation while reading body)
error: unexpected error when reading response body. Please retry. Original error: net/http: request canceled (Client.Timeout or context cancellation while reading body)

Solution :

In the pipeline's Jenkinsfile, change the deployment command from kubectl apply -f to kubectl apply -f. --request-timeout=30m .

Update podTemplate image of built-in Labels¶

The Workbench declares 7 labels through podTemplate: base , maven , mavenjdk11 , go , go16 , node.js , and python . You can specify a specific Agent label to use the corresponding podTemplate.

If the image in the built-in podTemplate does not meet your needs, you can replace or add the container image in the following ways:

Go to the Container Management module and click the name of the cluster where the Jenkins component is running.
In the left navigation bar, click ConfigMaps & Secrets -> ConfigMaps .
Search for jenkins-casc-config and click Edit YAML in the Actions column.
Under data -> jenkins.yaml -> jenkins.clouds.kubernetes.templates , select the podTemplate whose image you want to change.
Once you have made the necessary updates, go to Workloads section and restart Jenkins deployment.

Modify dependency source in settings.xml in Maven?¶

When use Maven as the pipeline build environment, most users need to modify settings.xml file to change the dependency source. You can follow these steps:

Go to the Container Management module and click the name of the cluster where the Jenkins component is running.
In the left navigation bar, click ConfigMaps & Secrets -> ConfigMaps .
Search for amamba-devops-agent and click Edit YAML in the Actions column.
Modify the MavenSetting under the data section as per your requirement.
Once you have made the necessary updates, go to Workloads and restart Jenkins.

Unable to access private image repositories when building images through Jenkins¶

Podman runtime¶

Go to the Container Management module and click the name of the cluster where the Jenkins component is running.
Click ConfigMaps & Secrets -> ConfigMaps in the left navigation bar in order.
Search for insecure-registries and click Edit YAML in the Actions column.
Configure under the registries.conf file in the data section.

Pay attention to the formatting and indentation when making modifications. Each registry should have a separate [[registry]] section, as shown in the image below:
Note

The value of the registries keyword should be the complete domain name or IP address of the container registry, without adding the http or https prefix. If the container registry uses a non-standard port number, you can add a colon : followed by the port number after the address.
```
[registries]
location = "registry.example.com:5000"
insecure=true

[registries]
location = "192.168.1.100:8080"
insecure=true
```
Refer to Podman documentation.

Cluster runtime is Docker¶

Open the Docker configuration file. On most Linux distributions, the configuration file is located at /etc/docker/daemon.json . If it doesn't exist, please create this configuration file.
Add the repository address to the insecure-registries field.
```
{
  "insecure-registries": ["10.16.10.120:4443"]
}
```

After saving, restart Docker by executing the following commands:

sudo systemctl daemon-reload
sudo systemctl restart docker

Note

Refer to Docker documentation.

How to modify the number of concurrent running of Jenkins pipelines¶

Currently, the number of concurrent running of Jenkins pipelines after DCE 5.0 deployment is 2. The following will describe how to change the number of concurrent running:

Go to the Container Management module, find the cluster where the Jenkins component is located, and click the cluster name.
Click ConfigMaps & Secrets -> ConfigMaps in the left navigation bar in order.
Search for jenkins-casc-config, and click Edit YAML in the operation column.
Modify the value under the data -> jenkins.yaml -> jenkins.clouds.kubernetes.containerCapStr field.
After the update is completed, go to Workloads to restart Jenkins.

Unable to update pipeline running status in time¶

In the pod of Jenkins, there will be a sidecar container named event-proxy, which sends Jenkins events to the workbench through this container. Currently, Jenkins installed by dce5-installer will enable this container by default. Of course, you can also choose to create Jenkins yourself in the Addon module of the Container Management Platform (this is usually used when Jenkins is deployed in the working cluster). When creating, you can also choose whether to enable this container.

Based on whether this container is enabled or not, please check whether the different configmaps are correct:

The event-proxy container is enabled¶

Go to the Container Management module, find the cluster where the Jenkins component is located, and click the cluster name.
Click ConfigMaps & Secrets -> ConfigMaps in the left navigation bar in order.
Search for jenkins-casc-config, and click Edit YAML in the operation column.
Search for eventDispatcher.receiver in data -> jenkins.yaml, its value should be http://localhost:9090/event.

If Jenkins is deployed in the working cluster (need to penetrate the gateway of DCE 5.0), you also need to check the following configmaps.

Query the configmap named event-proxy-config again, check YAML, and the configmap description:

eventProxy:
  host: amamba-devops-server.amamba-system:80   # This is the gateway address of dce5. If it is Jenkins installed by dce5-installer, this does not need to be modified
  proto: http                                   # This is the gateway protocol of dce5 (http or https)

In the cluster where Jenkins is located, search for the secret amamba-jenkins in ConfigMaps & Secrets -> ConfigMaps.
Check whether the event-proxy-token in the secret is correct. This Token is used for the gateway authentication of DCE 5.0. If it is incorrect, Jenkins will not be able to send events to the workbench. For how to generate this Token, you can check Access Key.

If all the above configmaps are correct, but the status of Jenkins pipeline still cannot be updated, please first check the container log of event-proxy in Jenkins.

The event-proxy container is not enabled¶

Go to the Container Management module, find the cluster where the Jenkins component is located, and click the cluster name.
Click ConfigMaps & Secrets -> ConfigMaps in the left navigation bar in order.
Search for jenkins-casc-config, and click Edit YAML in the operation column.
Search for eventDispatcher.receiver in data -> jenkins.yaml, its value should be http://amamba-devops-server.amamba-system:80/apis/internel.amamba.io/devops/pipeline/v1alpha1/webhooks/jenkins where amamba-system is the namespace where the workbench is deployed.