Monitoring Central

3 Posts authored by: vitosalvaggio Employee

Look back into almost any online technology businesses 10, or even 5 years ago and you’d see a clear distinction between what the CTO and CMO did in their daily roles. The former would oversee the building of technology and products whilst the latter would drive the marketing that brought in the customers to use said technology. In short, the two together took care of two very different sides of the same coin.

 

Marketing departments traditionally measure their success against KPIs such as the number of conversions a campaign brought in versus the cost of running it. Developers measure their performance on how quickly and effectively they develop new technologies.

 

Today, companies are shifting focus towards a customer-centric approach, where customer experience and satisfaction is paramount. After all, how your customers feel about your products can make, or break a business.

Performance diagnostic tools can help you optimize a slow web page but won’t show you whether your visitors are satisfied.

So where do the classic stereotypes that engineers only care about performance and marketers only care about profit fit into the customer-centric business model? The answer is they don’t: in a business where each department works against the same metrics — increasing their customers’ experience — having separate KPIs is as redundant as a trap door in a canoe.

 

The only KPI that matters is “are my customers happy?”

 

Developers + Marketing KPIs = True

With technology being integral to any online business, marketers are now in a position where we can gather so much data and in such detail that we are on the front line when it comes to gauging the satisfaction and experience of our customers. We can see what path a visitor took on our website, how long they took to complete their journey and whether they achieved what they set out to do.

 

Armed with this, we stand in a position to influence the technologies developers build and use.

 

Support teams, no longer confined to troubleshooting customer problems have become Customer Success teams, and directly impact on how developers build products, armed with first-hand data from their customers.

 

So as the lines blur between departments, it shouldn’t come as a surprise that engineering teams should care about marketing metrics. After all, if a product is only as effective as the people who use it, engineers build better products and websites when they know how customers intend to use them.

 

Collaboration is King

“How could engineers possibly make good use of marketing KPIs?” you might ask. After all, the two are responsible for separate ends of your business but can benefit from the same data.

 

Take a vital page on your business’s website: it’s not the fastest page on the net but its load time is consistent and it achieves its purpose: to convert your visitors to customers. Suddenly your bounce rate has shot up from 5% to 70%.

Ask an engineer to troubleshoot the issue and they might tell you that the page isn’t efficient. It takes 2.7 seconds to load, which is 0.7 seconds over the universal benchmark and what’s more is that some of the file sizes on your site are huge.

 

Ask a marketer the same question and they might tell you that the content is sloppy, making the purpose of the page unclear. The colors are off-brand and what’s more is that an important CTA is missing.

 

Even though both have been looking at the same page, they’ve come to two very different results, but the bottom line is that your customer doesn’t care about what went wrong. What matters is that the issue is identified and solved, quickly.

 

Unified Metrics Mean Unified Monitoring

Having unified KPIs across the various teams internal to your organisation means that they should all draw their data from the same source: a single, unified monitoring tool.

 

For businesses where the customer comes first, a new breed of monitoring is evolving that offers organizations this unified view, centred on how your customer experiences your site: Digital Experience Monitoring, or seeing as everything we do is digital, how about we just call it Experience Monitoring?

With Digital Experience Monitoring, your marketers and your engineering teams can follow a customer’s journey through your site, see how the navigated through it and where and why interest became a sale or a lost opportunity.

 

Let’s go back to our previous example: both your marketer and your engineer will see that although your bounce rate skyrocketed, the page load time and size stayed consistent. What they might also see is that onboarding you implemented that coincides with your bounce rate spike is confusing to your customers meaning that they leave, frustrated and unwilling to convert.

 

Digital Experience Monitoring gives a holistic view of your website’s health and helps you answer questions like:

  • Where your visitors come from
  • When are they visiting your site
  • What they visit and the journey they take to get there
  • How your site’s performance impacts on your visitors

By giving your internal teams access to the same metrics, you foster greater transparency across your organization which leads to faster resolution of issues, a deeper knowledge of your visitors and better insights into what your customers love about your products.

 

Pingdom’s Digital Experience Monitoring, Visitor Insights, bridges the gap between site performance and customer satisfaction, meaning you can guess less and know more about how your visitors experience your site.

Version 1.1 of the venerable HTTP protocol powered the web for 18 years.

 

Since then, websites have emerged from static, text-driven documents to interactive, media-rich applications. The fact that the underlying protocol remained unchanged throughout this time just goes to show how versatile and capable it is. But as the web grew bigger, its limitations became more obvious.

 

We needed a replacement, and we needed it soon.

 

Enter HTTP/2. Published in early 2015, HTTP/2 optimizes website connections without changing the existing application semantics. This means you can take advantage of HTTP/2’s features such as improved performance, updated error handling, reduced latency, and lower overhead without changing your web applications.

 

Today nearly 84% of modern browsers and 27% of all websites support HTTP/2, and those numbers are gradually increasing.

 

How is HTTP/2 Different from HTTP/1.1?

HTTP/2’s biggest changes impact the way data is formatted and transported between clients and servers.

 

Binary Data Format

HTTP/2 encapsulates data using a binary protocol. With HTTP/1.1, messages are transmitted in plaintext. This makes requests and responses easy to format and even read using a packet analysis tool, but results in increased size due to unnecessary whitespace and inefficient compression.

 

The benefit of a binary protocol is it allows for more compact, more easily compressible, and less error-prone transmissions.

 

Persistent TCP Connections

In early versions of HTTP, a new TCP connection had to be created for each request and response. HTTP/1.1 introduced persistent connections, allowing multiple requests and responses over a single connection. The problem was that messages were exchanged sequentially, with web servers refusing to accept new requests until previous requests were fulfilled.

 

HTTP/2 simplifies this by allowing for multiple simultaneous downloads over a single TCP connection. After a connection is established, clients can send new requests while receiving responses to earlier requests. Not only does this reduce the latency in establishing new connections, but servers no longer need to maintain multiple connections to the same clients.

 

Multiplexing

Persistent TCP connections paved the way for multiplexed transfers. With HTTP/2, multiple resources can be transferred simultaneously. Clients no longer need to wait for earlier resources to finish downloading before the next one begins. Website developers used workarounds such as domain sharding to “trick” browsers into opening multiple connections to a single host; however, this led to browsers opening multiple TCP connections. HTTP/2 makes this entire practice obsolete.

 

Header Compression and Reuse

In HTTP/1.1, headers are incompressible and repeated for each request. As the number of requests grows, so does the volume of duplicate header information. HTTP/2 eliminates redundant headers and compresses the remaining headers to drastically decrease the amount of data repeated during a session.

 

Server Push

Instead of waiting for clients to request resources, servers can now push resources. This allows websites to preemptively send content to users, minimizing wait times.

 

Does My Site Already Support HTTP/2?

Several major web servers and content delivery networks (CDNs) support HTTP/2. The fastest way to check if your website supports HTTP/2 is to navigate to the website in your browser and open Developer Tools. In Firefox and Chrome, press Ctrl-Shift-I or the F12 key and click the Network tab. Reload the page to populate the table with a list of responses. Right-click the column names in the table and enable the “Protocol” header. This column will show HTTP/2.0 in Firefox or h2 in Chrome if HTTP/2 is supported, or HTTP/1.1 if it’s not.

 

What is HTTP/2, and Will It Really Make Your Site Faster?
The network tab after loading 8bitbuddhism.com©. The website fully supports HTTP/2 as shown in the Protocol column.

 

Alternatively, KeyCDN provides a web-based HTTP/2 test tool. Enter the URL of the website you want to test, and the tool will report back on whether it supports HTTP/2.

 

How Do I Enable HTTP/2 on Nginx?

As of version 1.9.5, Nginx fully supports HTTP/2 via the ngx_http_v2 module. This module comes included in the pre-built packages for Linux and Windows. When building Nginx from source, you will need to enable this module by adding –with-http_v2_module as a configuration parameter.

You can enable HTTP/2 for individual server blocks. To do so, add http2 to the listen directive. For example, a simple Nginx configuration would look like this:

# nginx.conf
server {
listen 443 ssl http2;
server_name mywebsite.com;

root /var/www/html/mywebsite;
}

Although HTTP/2 was originally intended to require SSL, you can use it without SSL enabled. To apply the changes, reload the Nginx service using:

$ sudo service nginx reload

or by invoking the Nginx CLI using:

$ sudo /usr/sbin/nginx -s reload

Benchmarking HTTP/2

To measure the speed difference between HTTP/2 and HTTP/1.1, we ran a performance test on a WordPress site with and without HTTP/2 enabled. The site was hosted on a Google Compute Engine instance with 1 virtual CPU and 1.7 GB of memory. We installed WordPress 4.9.6 in Ubuntu 16.04.4 using PHP 7.0.30, MySQL 5.7.22, and Nginx 1.10.3.

 

To perform the test, we created a recurring page speed check in SolarWinds®Pingdom® to contact the site every 30 minutes. After four measurements, we restarted the Nginx server with HTTP/2 enabled and repeated the process. We then dropped the first measurement for each test (to allow Nginx to warm up), averaged the results, and took a screenshot of the final test’s Timeline.

 

 

The metrics we measured were:
  • Page size: the total combined size of all downloaded resources.
  • Load time: the time until the page finished loading completely.

 

Results Using HTTP/1.1

 

What is HTTP/2, and Will It Really Make Your Site Faster?
Timeline using HTTP/1.1

 

Results Using HTTP/2

 

What is HTTP/2, and Will It Really Make Your Site Faster?
Timeline using HTTP/2

 

And the Winner Is…

With just a simple change to the server configuration, the website performs noticeably better over HTTP/2 than HTTP/1.1. The page load time dropped by over 13% thanks to fewer TCP connections, resulting in a lower time to first byte. As a result of only using two TCP connections instead of four, we also reduced the time spent performing TLS handshakes. There was also a minor drop in overall file size due to HTTP/2’s more efficient binary data format.

 

Conclusion

HTTP/2 is already proving to be a worthy successor to HTTP/1.1. A large number of projects have implemented it and, with the exception of Opera Mini and UC for Android, mainstream browsers already support it. Whether it can handle the next 18 years of web evolution has yet to be seen, but for now, it’s given the web a much-needed performance boost.

 

You can try this same test on your own website using the Pingdom page speed check. Running the page speed check will show you the size and load time of every element. With this data you can tune and optimize your website, and track changes over time.

Jenkins X (JX) is an exciting new Continuous Integration and Continuous Deployment (CI/CD) tool for Kubernetes users. It hides the complexities of operating Kubernetes by giving developers a simpler experience to build and deploy their code. You can think of it as creating a serverless-like environment in Kubernetes. As a developer, you don’t need to worry about all the details of setting up environments, creating a CI/CD pipeline, or connecting GitHub to your CI pipeline. All of this and much more is handled by JX. In this article, we’ll introduce you to JX, show you how to use it, and how to monitor your builds and production deployments.

 

What is Jenkins X?

JX was created by James Strachan (creator of Groovy, Apache Camel, and now JX) and was first announced in March 2018. It’s designed from the ground up to be a cloud-native, Kubernetes-only application that not only supports CI/CD, but also makes working with Kubernetes as simple as possible. With one command you can create a Kubernetes cluster, install all the tools you’ll need to manage your application, create build and deployment pipelines, and deploy your application to various environments.

Jenkins is described as an “extensible automation server” that can be configured, via plugins, to be a Continuous Integration Server, a Continuous Deployment hub, or a tool to automate just about any software task. JX provides a specific configuration of Jenkins, meaning you don’t need to know which plugins are required to stand up a CI/CD pipeline. It also deploys numerous applications to Kubernetes to support building your docker container, storing the container in a docker registry, and deploying it to Kubernetes.

Jenkins pipeline builds are driven by adding a Jenkinsfile to your project. JX automates this for you. JX can create new projects (and the required Jenkinsfile) for you or import your existing project and create a Jenkinsfile if you don’t already have one. In short, you don’t need to know anything about Jenkins or Kubernetes to get started with JX. JX will do it all for you.

 

Overview of How JX Works

JX is designed to take all of the guesswork or trial and error approach many teams have used to create a fully functional CI/CD pipeline in Kubernetes. To make a tailored developer experience, JX had to choose which Kubernetes technologies to use. In many ways, JX is like a Linux distribution, but for Kubernetes. JX had to decide, from the plethora of tools available, which ones to use to create a smooth and seamless developer experience in Kubernetes.

To make the transition to Kubernetes simpler, the command line tool jx can drive most of your interactions with Kubernetes. This means you don’t need to know how to use kubectl right away; instead you can slowly adopt kubectl as you become more comfortable in Kubernetes. If you are an experienced Kubernetes user, you’ll use jx for interacting with JX (CI/CD, build logs, and so on) and continue to use kubectl for other tasks.

When you create or import a project using the jx command line tool, JX will detect your project type and create the appropriate Jenkinsfile for you (if it doesn’t already exist), define the required Kubernetes resources for your project (like Helm charts), add your project to GitHub and create the necessary webhooks for your application, build your application in Jenkins, and if all tests pass, deploy your application to a staging environment. You now have a fully integrated Kubernetes application with a CI/CD pipeline ready to go.

Your interaction with JX is driven by a few jx commands to set up and env, create or import an application, and monitor the state of your build pipelines. The developer workflow is covered in the next section. Generally speaking, once set up, you don’t need to interact with JX that much; it works quietly in the background, providing you CI and CD functionality.

 

Install Jenkins X

To get started using JX, install the jx binary. For Mac OS, you can use brew:

brew tap jenkins-x/jx brew install jx

Note: When I first tried to create a cluster using JX, it installed kops for me. However, the first time jx tried to use kops, it failed because kops wasn’t on my path. To address this, install kops as well:

brew install kops

Create a Kubernetes Cluster

JX supports most major cloud environments: Google GKE, Azure AKS, Amazon EKS, minikube, and many others. JX has a great video on installing JX on GKE. Here, I’m going to show you how to install JX in Amazon without EKS. Creating a Kubernetes cluster from scratch is very easy:

jx create cluster aws

Since I wasn’t using JX for a production application, I ran into a few gotchas during my install:

  1. When prompted with, “No existing ingress controller found in the kube-system namespace, shall we install one?” say yes.
  2. Assuming you are only trying out JX, when prompted with, “Would you like to register a wildcard DNS ALIAS to point at this ELB address?” say no.
  3. When prompted with, “Would you like wait and resolve this address to an IP address and use it for the domain?” say yes.
  4. When prompted with, “If you don’t have a wildcard DNS setup then set up a new CNAME and point it at: XX.XX.XX.XX.nip.io. Then, use the DNS domain in the next input” accept the default.

The image below shows you the EC2 instances that JX created for your Kubernetes Cluster (master is an m3.medium instance and the nodes are t2.medium instances):

LG IntroJenkinsX 1
AWS EC2 Instances. © 2018 Amazon Web Services, Inc. or its affiliates. All rights reserved.

When you are ready to remove the cluster you just created, you can use this command (JX currently does not provide a delete cluster command):

kops delete cluster

Here’s the full kops command to remove the cluster you just created (you’ll want to use the cluster name and S3 bucket for all kops commands):

kops delete cluster --name aws1.cluster.k8s.local \ --state=s3://kops-state-xxxxxx-ff41cdfa-ede6-11e8-xx6-acde480xxxx

To add Loggly integration to your Kubernetes cluster, you can follow the steps outlined here.

 

Create an Application

Now that JX up and running, you are ready to create an application. The quickest way to do this is with the JX quickstart. In addition to the quickstart applications that come with JX, you can also create your own.

To get started, run create quickstart, and pick the spring-boot-http-gradle quick start (see the screenshot below for more details):

jx create quickstart

 

LG IntroJenkinsX 2
Creating a Kubernetes cluster using jx create cluster © 2018 Jenkins Project

Note: During the install process, I did run into one issue. When prompted with, “Which organization do you want to use?” make sure you choose a GitHub Org and not your personal account. The first time I ran this, I tried my personal account (which has an org associated with it) and jx create quickstart failed. When I reran it, I chose my org ripcitysoftware and everything worked as expected.

Once your application has been created, it will automatically be deployed to the staging environment for you. One thing I really like about JX is how explicit everything is. There isn’t any confusion between temporary and permanent environments because the environment name is embedded into the application URL (http://spring-boot-http-gradle.jx-staging.xx.xx.xx.xx.nip.io/).

The Spring Boot quickstart application provides you with one rest endpoint:

LG IntroJenkinsX 3
Example Spring Boot HTTP © 2018 Google, Inc

 

Developer Workflow

JX has been designed to support a trunk-based development model promoted by DevOps leaders like Jez Humble and Gene Kim. JX is heavily influenced by the book Accelerate (you can find more here), and as such it provides an opinionated developer workflow approach. Trunk-based development means releases are built off of trunk (master in git). Research has shown that teams using trunk-based development are more productive than those using long-lived feature branches. Instead of long-lived feature branches, teams create branches that live only a few hours, making a few small changes.

Here’s a short overview of trunk-based development as supported by JX. To implement a code change or fix a bug, you create a branch in your project, write tests, and make code changes as needed. (These changes should only take a couple of hours to implement, which means your code change is small.) Push your branch to GitHub and open a Pull Request. Now JX will take over. The webhook installed by JX when it imported your project will trigger a CI build in Jenkins. If the CI build succeeds, Jenkins will notify GitHub the build was successful, and you can now merge your PR into master. Once the PR is merged, Jenkins will create a released version of your application (released from the trunk branch) and deploy it (CD) to your staging environment. When you are ready to promote your application from stage to production, you’ll use the jx promote command.

The development workflow is expected to be:

  1. In git, create a branch to work in. After you’ve made your code changes, commit them and then push your branch to your remote git repository.
  2. Open a Pull Request in your remote git repo. This will trigger a build in Jenkins. If the build is successful, JX will create a preview environment for your PR so you can review and test your changes. To trigger the promotion of your code from Development to Staging, merge your PR.
  3. By default, JX will automatically promote your code to Stage. To promote your code to Production, you’ll need to run this command manually: jx promote app-name --version x.y.z --env production

Monitoring Jenkins X

Monitoring the status of your builds gives you insight into how development is progressing. It will also help you keep track of how often you are deploying apps to various environments.

JX provides you multiple ways to track the status of a build. JX configures Jenkins to trigger a build when a PR is opened or updated. The first place to look for the status of your build is in GitHub itself. Here is a build in GitHub that resulted in a failure. You can clearly see the CI step has failed:

LG IntroJenkinsX 4
GitHub PR Review Web Page. © 2018 GitHub Inc. All rights reserved.

The next way to check on the status of your build is in Jenkins itself. You can navigate to Jenkins in your browser or, from GitHub, you can click the “Details” link to the right of “This commit cannot be built.” Here is the Jenkins UI. You will notice Jenkins isn’t very subtle when a build fails:

LG IntroJenkinsX 5
Jenkins Blue Ocean failed build web page. © 2018 Jenkins Project

A third way to track the status of your build is from the command line, using the jx get activity command:

LG IntroJenkinsX 6
iTerm – output from jx get activity command © 2018 Jenkins Project

If you want to see the low-level details of what Jenkins is logging, you’ll need to look at the container Jenkins is running in. Jenkins is running in Kubernetes like any other application. It’s deployed as a pod and can be found using the kubectl command:

$ kubectl get pods NAME                      READY     STATUS    RESTARTS   AGE jenkins-fc467c5f9-dlg2p   1/1       Running   0          2d

Now that you have the name of the Pod, you can access the log directly using this command:

$ kubectl logs -f jenkins-fc467c5f9-dlg2p

 

LG IntroJenkinsX 7
iTerm – output from kubectl logs command © 2018 Jenkins Project

Finally, if you’d like to get the build output log, the log that’s shown in the Jenkins UI, you can use the command below. This is the raw build log that Jenkins creates when it’s building your application. When you have a failed build, you can use this output to determine why the build failed. You’ll find your test failures here along with other errors like failures in pushing your artifacts to a registry. The output below is not logged to the container (and therefore not accessible by Loggly):

$ jx get build log ripcitysoftware/spring-boot-http-gradle/master view the log at: http://jenkins.jx.xx.xx.xxx.xxx.nip.io/job/ripcitysoftware/job/spring-boot-http-gradle/job/master/2/console tailing the log of ripcitysoftware/spring-boot-http-gradle/master #2 Push event to branch master Connecting to https://api.github.com using macInfinity/****** (API Token for accessing https://github.com Git service inside pipelines)

Monitoring in Loggly

One of the principles of a microservice architecture, as described by Sam Newman in Building Microservices, is being Highly Observable. Specifically, Sam suggests that you aggregate all your logs. A great tool for this is SolarWinds® Loggly. Loggly is designed to aggregate all of your logs into one central location. By centralizing your logs, you get a holistic view of your systems. Deployments can trigger a change in the application that can generate errors or lead to instability. When you’re troubleshooting a production issue, one of the first things you want to know is whether something changed. Being able to track the deployments in your logs will let you backtrack deployments that may have caused bugs.

To monitor deployments, we need to know what’s logged when a deployment succeeds or fails. This is the message Jenkins logs when a build has completed:

INFO: ripcitysoftware/spring-boot-http-gradle/master #6 completed: SUCCESS

From the above message, we get a few pieces of information: the name of the branch, which contains the Project name ripcitysoftware/spring-boot-http-gradle and the branch master, the build number #6, and finally the build status SUCCESS.

The metrics you should monitor are:

  • Build status – Whether a build was a success or failure
  • The project name – Which project is being built
  • The build number – Tracks PRs and releases

By tracking the build status, you can see how often builds are succeeding or failing. The project name and build number tell you how many PRs have been opened (look for “PR” in the project name) and how often a release is created (look for “master” in the name).

To track all of the above fields, create one Derived Field in Loggly called jxRelease. Each capture group (the text inside of the parentheses) defines a unique Derived Field in Loggly. Here is the regex you’ll need:

^INFO:(.*)\/.*(master|PR.*) #(.*\d) completed: ([A-Z]+$)$

Here’s the Jenkins build success log-message above as it appears in Loggly after we’ve created the Derived Field. You can see all the fields we are defining highlighted in yellow below the Rule editor:

LG IntroJenkinsX
Loggly – Derived Field editor web page.  © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Please note that Derived Fields use past logs only in the designer tool. Loggly only adds new derived fields to new log messages. This means if you’ve got an hour of Jenkins output already sent to Loggly and you create the jxBuildXXX fields (as shown above), only new log messages will include this field.

In the image below, you can see all the Derived Fields that have been parsed in the last 30 minutes. For jxBuildBranchName, there has been one build to stage, and it was successful, as indicated by the value SUCCESS. We also see that nine (9) builds have been pushed to stage, as indicated by the jxBuildNumber field.

 

LG IntroJenkinsX 9
Loggly Search Results web page.  © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Now that these fields are parsed out of the logs, we can filter on them using the Field Explorer. Above, you can see that we have filtered on the master branch. This shows us each time the master branch has changed. When we are troubleshooting a production bug, we can now see the exact time the code changed. If the bug started after a deployment, then the root cause could be the code change. This helps us narrow down the root cause of the problem faster.

We can also track when master branch builds fail and fire an alert to notify our team on Slack or email. Theoretically, this should never happen, assuming we are properly testing the code. However, there could have been an integration problem that we missed, or a failure in the infrastructure. Setting an alert will notify us of these problems so we can fix them quickly.

 

Conclusion

JX is an exciting addition to Jenkins and Kubernetes alike. JX fills a gap that has existed since the rise of Kubernetes: how to assemble the correct tools within Kubernetes to get a smooth and automated CI/CD experience. In addition, JX helps break down the barrier of entry into Kubernetes and Jenkins for CI/CD. JX itself gives you multiple tools and commands to navigate system logs and track build pipelines. Adding Loggly integration with your JX environment is very straightforward. You can easily track the status of your builds and monitor your apps progression from development to a preview environment, to a staging environment and finally to production. When there is a critical production issue that you are troubleshooting, you can look at the deployment time to see if changes in the code caused the issue.

SolarWinds uses cookies on its websites to make your online experience easier and better. By using our website, you consent to our use of cookies. For more information on cookies, see our cookie policy.