Use Cloud Build to do Continuous Delivery for your Java project on App Engine

In this post I’m going to show to set up Continuous Delivery for a simple Java micro service built with Maven. I’ll be using the sample project I have been using in five blog posts I wrote earlier this year, so if you wonder why things work the way they do in the sample project, read the previous blog posts.

I want that every commit to the master branch of my GitHub repository triggers a maven build in Cloud Build and that the produced jar files are deployed to App Engine.

Prerequisites

• Oracle’s JDK 11, with the Java command available in a terminal. (verify that it isn’t some old JDK 8 you forgot to uninstall by running java -version)
• A recent version of git
• Maven 3.6.2 or newer usable on the command line
• IntelliJ IDEA (I use the Ultimate edition, but the free one should work too)
• A credit card to be able to get $300 of free credits in Google Cloud.
• You need to have the Google Cloud SDK installed. As soon as you installed it, run gcloud components install app-engine-java

Clone my sample project and run it locally

Run git clone https://github.com/Leejjon/SimpleJerseyService and switch to the 4.2 tag by running: git checkout tags/4.2

Verify that it works locally by running mvn clean install exec:exec and visit http://localhost:8080/ to see “Got it”.

Creating an App Engine project

To be able to host this sample service, you need to create your own App Engine project. If you haven’t already, grab your free credits on your google account.

You´ll need to set up a billing account using a credit card, but doing so grants you $300 free credits to spend on resources on Google Cloud Platform. Even if you already used it I would not worry about costs. I run blindpool.com on App Engine and Google Datastore. This is what I pay each month:

To create a project, go to https://console.cloud.google.com/appengine

Click on the NEW PROJECT button to create a new project.

For this blog I’m creating a new project with ID: simple-jersey-service, choose your own unique project name.

I’m not sure why the Font is so ugly.

Run gcloud init on the command line and follow the steps to authenticate with your Google account. Select the project you just created in the web interface.

Now run gcloud app create . You have to select a region, I picked europe-west-1 in Belgium (the region in the Netherlands doesn’t support App Engine for some reason). In the pom.xml of the SimpleJerseyService you just cloned, you can see I have put the projectId of my App Engine project in the configuration of the appengine-maven-plugin:

<plugin>
    <groupId>com.google.cloud.tools</groupId>
    <artifactId>appengine-maven-plugin</artifactId>
    <version>2.2.0</version>
    <configuration>
        <deploy.projectId>simple-jersey-service</deploy.projectId>
        <version>1</version>
    </configuration>
</plugin>

You can simply run mvn clean install appengine:deploy to compile everything from source and push it to App Engine.

If your deploy was successful, you can test the url via your newly generated appspot.com url. Mine is: https://simple-jersey-service.appspot.com/

In the above picture I deployed two versions. Currently https://simple-jersey-service.appspot.com/ points to version 2. But you can easily migrate traffic back to version 1 via the web interface:

Each version has it’s own url (version 1 for example has https://1-dot-simple-jersey-service.appspot.com/), so you can test it without affecting the users of your production version.

Logging

You’ll need no more digging through log files with VI. No more risk of full hard drives crashing applications because of too much logging. You don’t have to maintain logstash/elasticsearch/kibana and whatever servers they run on either. All requests to your service will result in a log entry in the Google Cloud Logging. All log statements in your code that are executed will also be shown here.

I find this Logs Viewer easier to use than Logstash/Kibana. Sure, it vendor locks me into the Google stack, but for my hobby projects this is perfect.

Continuous Delivery

I originally planned to host a Jenkins server, install Java, Maven and the Google Cloud SDK that can run the mvn clean install appengine:deploy command to deploy every time a commit is merged to master.

However, I postponed starting with that because I personally hate maintaining Jenkins servers. They need OS updates, Java updates, maven updates, they can run out of storage. Don’t get me wrong, Jenkins is great for what it does, but if we are able to run our applications on serverless technology such as App Engine, why would we bother to maintain servers for CI/CD purposes?

Cloud Build

Just like AWS has CodeBuild and CodePipeline, Google also released a way to execute CI/CD tasks on Google Cloud Platform. You should read the docs for more information.

So let’s take a look on Cloud Build in our brand new simple-jersey-service project:

Wait what? There’s already two builds in there. Are we already using it? It seems that when we run the mvn appengine:deploy command, the underlying gcloud command creates a build in Cloud Build.

Connect Cloud Build with GitHub

You can create triggers with the gcloud command, but it’s easier for me to show how to do it with a screenshot of the web UI. First we need to connect my GitHub repository with Cloud Build.

Go through the Connect Repository wizard to connect your GitHub repository (you can skip creating a default trigger):

After you give Cloud Build access to you GitHub repository, create a trigger:

Creating a cloudbuild.yaml

To make sure Cloud Build actually does something with our repository, we need to create a cloudbuild.yaml in the project:

I found daninge98’s cloudbuild.yaml configuration on this GitHub issue worked really well:

steps:
  - id: 'Stage app using mvn appengine plugin on mvn cloud build image'
    name: 'gcr.io/cloud-builders/mvn'
    args: ['package', 'appengine:stage']
  - id: 'Deploy to app engine using gcloud image'
    name: 'gcr.io/cloud-builders/gcloud'
    args: ['app', 'deploy', 'target/appengine-staging/app.yaml']

One thing I noticed is that it does not pick up the `<version>1</version>` we specified in the maven-appengine-plugin configuration of the pom.xml.

Instead it will auto generate a version based on the timestamp of deployment like 20200328t11282. But to be honest I like that as it means you don’t need to forget to update the version in the pom.xml every time you push commits.

Now push this this new file to GitHub using:

git add -A
git commit -m "Adding cloudbuild file for continuous delivery"
git push

This should already trigger a Cloud Build! Let’s check it out:

It will complain about permissions, luckily it also provides a solution:

ERROR: (gcloud.app.deploy) User [156238929146@cloudbuild.gserviceaccount.com] does not have permission to access app [simple-jersey-service] (or it may not exist): App Engine Admin API has not been used in project 156238929146 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/appengine.googleapis.com/overview?project=156238929146 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.

Simply follow the url in the logs and click Enable:

Now just use the “Retry” button in your failed build to see whether it works now:

You will most likely run into the next Permission error:

Already have image (with digest): gcr.io/cloud-builders/gcloud
ERROR: (gcloud.app.deploy) Permissions error fetching application [apps/simple-jersey-service]. Please make sure you are using the correct project ID and that you have permission to view applications on the project.

Fortunately I already ran into this problem and figured out what permissions you need. Go to the IAM:

You need to edit the cloudbuild.gserviceaccount.com account and add the following two rules:

• App Engine Deployer (for deploying the project)
• App Engine Service Admin (if you want to transfer all traffic to your new deployed service)

After you added these roles, hit the retry button again on the build that failed.

And finally, we have deployed to production! Please provide any feedback if you think I can make any of these steps even simpler.

What’s next?

The team at Google that works on App Engine is beta testing a new option in the gcloud tool:

This might simplify our cloudbuild.yaml even more. I have tried to use this on Cloud Build, but the gcloud version on Cloud Build didn’t contain the beta command yet. I might write a new blog post that includes this command when it’s ready for use.

See you next time!