How to set up CI/CD server using Jenkins connected to a Bitbucket repository for an ASP.NET Core Web API running in an EC2 instance?

In my previous article, i have shared on how to host ASP.NET Core Web application on Linux using Nginx. Today, i will try to set up a CI/CD server using Jenkins for my application.

The question is: Why CI/CD server?

Continuous integration combines together the work of multiple developers at most companies and is usually done at least daily to help detect any bugs early on. Continuous delivery helps you build your app in a way so that it can be ready whenever you want it to be released at any point in time. Continuous deployment is an automation process that lets the changes you make instantly get deployed through the pipeline.

Jenkins, originally founded in 2006 as “Hudson”, is the leading open-source continuous integration server. Using an extensible, plugin-based architecture, developers have created over 300 plugins to adapt Jenkins to a multitude of build, test, and deployment automation workloads. In 2015, Jenkins surpassed 100,000 known installations, making it the most widely deployed automation server.

So, we will install Jenkins on our EC2 instance. Let’s connect to our EC2 instance using PuTTY.

At first, we need to install the OpenJdk.

$ sudo apt-get install openjdk-8-jre

Then, we’ll add the repository key to the system:

$ wget -q -O — https://pkg.jenkins.io/debian/jenkins-ci.org.key | sudo apt-key add -

When the key is added, the system will return OK. Next, we'll append the Debian package repository address to the server's sources.list:

$ echo deb https://pkg.jenkins.io/debian-stable binary/ | sudo tee /etc/apt/sources.list.d/jenkins.list

When both of these are in place, we’ll run update so that apt-get will use the new repository:

$ sudo apt-get update

Finally, we’ll install Jenkins and its dependencies.

$ sudo apt-get install jenkins

Now that Jenkins and its dependencies are in place, we’ll start the Jenkins server. Using systemctl we'll start Jenkins:

$ sudo systemctl start jenkins

Since systemctl doesn't display output, we'll use its status command to verify that it started successfully:

$ sudo systemctl status jenkins

If everything went well, the beginning of the output should show that the service is active and configured to start at boot:

Output
● jenkins.service - LSB: Start Jenkins at boot time
  Loaded: loaded (/etc/init.d/jenkins; bad; vendor preset: enabled)
  Active:active (exited) since Thu 2017-04-20 16:51:13 UTC; 2min 7s ago
    Docs: man:systemd-sysv-generator(8)

For start setting up our Jenkins, we will need to visit its web dashboard running on port, 8080. so let’s open browser and paste below url:

http://YOUR_IP_OR_DOMAIN:8080

Let’s copy our IP address from EC2 dashboard:

Now, we will see below screen when we go to http://54.175.26.11:8080.

In the terminal window, we’ll use the cat command to display the password:

$ sudo cat /var/lib/jenkins/secrets/initialAdminPassword

We’ll copy the 32-character alphanumeric password from the terminal and paste it into the Administrator password field, then click Continue. The next screen presents the option of installing suggested plugins or selecting specific plugins.

We’ll click the “Install suggested plugins” option, which will immediately begin the installation process:

When the installation is complete, we’ll be prompted to set up the first administrative user. It’s possible to skip this step and continue as admin using the initial password we used above, but we'll take a moment to create the user.

Note: The default Jenkins server is NOT encrypted, so the data submitted with this form is not protected. When you’re ready to use this installation, follow the guide How to Configure Jenkins with SSL using an Nginx Reverse Proxy. This will protect user credentials and information about builds that are transmitted via the Web interface.

Once the first admin user is in place, you should see a Jenkins is ready! confirmation screen.

Click Start using Jenkins to visit the main Jenkins dashboard:

At this point, Jenkins has been successfully installed.

Now, we will integrate our bitbucket repository with Jenkins.

We will create a Webhook in the bitbucket repository that will notify jenkins if any push is made to the repository.

Let’s click the Manage Jenkins sections from the jenkins dashboard and then click on the Manage Plugins section. Let’s install the Bitbucket plugin from the Available section.

Then it will install bitbucket plugin with all of it’s dependencies. Let’s check the checkbox with Restart Jenkins when installation is complete and no jobs are running

We will wait till the dependencies are installed and it will automatically move us to the jenkins dashboard when finishes.

Now, let’s go to our bitbucket repository and click on the Settings section. Then, let’s go to the Webhook section and click Add Webhook.

Let’s give it a title such as jenkins. The URL will be the address of our jenkins server but with bitbucket-hook. As the address of our jenkins server is http://54.175.26.11:8080 and we will add bitbucket-hook with it at the end. So, the final url will be http://54.175.26.11:8080/bitbucket-hook/. Let’s save the hook. Now, if any changes happens in the repository, it will trigger the jenkins server using this webhook.

Now, if the repositories are private in bitbucket, then we will have to add credentials so that jenkins can connect to bitbucket. Otherwise, jenkins won’t be able to connect with bitbucket private repositories. But, if the repositories are public, then we don’t need to do add any credentials.

Now, let’s go to our terminal of EC2 server and start a session with jenkins user with below command:

$ sudo su jenkins

Now, we will generate a new ssh key-pair using below command:

$ ssh-keygen

It will ask for a passphrase. We will just leave it for now and hit enter.

Now, we can see that our public key has been saved in /var/lib/jenkins/.ssh/id_rsa.pub file and private key has been saved in /var/lib/jenkins/.ssh/id_rsa file.

Now, we need to bitbucket know that it should accept this key. So, we will run below command and copy the public key.

$ cat /var/lib/jenkins/.ssh/id_rsa.pub

Then, we will go the Settings/Access keys and Add key in the bitbucket repository.

Now, we will paste the copied public key into the Key section and give it a Label.

Now, jenkins server will be allowed to read all the source codes from this private repository. But, we need to set the private key in the jenkins server also. Let’s go to the Credentials section and click the System section. Then, we will click the Global credentials and Add Credentials.

Let’ go to our terminal window of jenkins server and run below command to get the private key.

$ cat /var/lib/jenkins/.ssh/id_rsa

Let’s add a new credential in the jenkins server.

We will select SSH Username with private key and paste the copied private key in the Private Key section. Now, we will use this credential when we create job in the jenkins server.

I have mentioned before, if it is a public repository, then we don’t have to do this above section.

Now, let’s go the jenkins dashboard and create a new job. Let’s call it my-project and select as Freestyle project.

Let’ select Git as Source Code Management and give the repository SSH url from bitbucket. But, it will not show this error if the repository is public. But, it is showing this error as the repository is private.

So, we will add the jenkins credentials we have made before in the Credentials section and the error will be gone means that jenkins can connect to the bitbucket repository.

But, if the error still present, then replace the “:” with ” /” in front of the username like below:

ssh://git@bitbucket.org/Savjee/jenkins-private.git

Then, we will select Build when change is pushed to BitBucket.

Now, we want to add some build steps. At first, we want to build our asp.net core application and publish to the folder from where the API service is running. And, then we will restart the service which we have made in the previous article.

So, let’s select Execute Shell from Add Build Step selection list and add some commands.

Here, we are publishing to a specific folder and restarting our web api service.

Now, let’s change something in the bitbucket repository and it will trigger a build in jenkins server.

We can manually trigger build by clicking Build Now.

We can click on the build number from Build History section to see the builds full information. If we go to the Console Output, we may get more information about the build.

If we see that the build has been failed for using sudo command, we need to do some extra work. You can follow this SO link.

Sudo: No tty present and no askpass program specified’ error

Let’s go to teminal window of EC2 server and run below command:

$ sudo visudo

It will open the soders file and add below line after all the line in the file.

jenkins ALL=(ALL) NOPASSWD: ALL

Let’s save the file and build again. Now, the build will be successful.

That’s it. We have set Jenkins as a CI/CD server and connected with bitbucket repository. Also, Jenkins server is publishing the ASP.NET Core web application changes in the EC2 server.

Thanks for reading :)