Monthly Archives: October 2017

Java Docker Pull Travails

Just had a problem that I thought I’d document the solution to for the world…

In our build pipeline at work, we use maven and the fabric8 docker-maven-plugin to manage our builds.  We love it, developers can just “mvn install” locally and then the Atlassian Bamboo build system just “mvn deploy”s in the exact same way.

Well, so we had some builds that suddenly weren’t able to pull the base images specified in our Dockerfiles down from Dockerhub, breaking the build with 500 error messages like:

[ERROR] DOCKER> Unable to pull 'library/debian:sid' from registry 'docker.io' : received unexpected HTTP status: 500 Server Error (Internal Server Error: 500) [received unexpected HTTP status: 500 Server Error (Internal Server Error: 500)]

But it worked fine on our local box. And it could pull our custom images from Artifactory fine. What’s the problem here?  Bamboo?  The plugin? Well, some helpful community folks helped home in on it, it turns out that for some versions of Java 1.8, 8u131 and prior at least going back to 112, where there’s some problem (TLS? Root certs? Not really sure) that messes up when pulling a docker.io container from inside Java during our docker build step.  My team’s microservices aren’t Java based so the Java version doesn’t come up much – but of course maven uses Java.

Upgrading the JDK version to 8u144 made the problem go away.  We actually have an up to date curated Java version we use in Bamboo for our Java builds, but folks doing Python builds were just using the default “JDK 1.8” that Atlassian is putting on their Bamboo build agent AMI, which is of course old and suffers from this issue.

 

Leave a comment

Filed under DevOps

Long live ChatOps, RIP AOL IM!

I grew up in Muscat, Oman, and it was an exciting time when we got Internet at home in 1996. By 1998, all of my friends who had Internet at home were first on ICQ and then on AOL IM. AOL IM was huge when I went to college in the early 2000’s and was the primary way to connect friends together to chat. Back then, it was rare to have chat rooms, and the rooms that existed were usually long-running things set up to talk about general topics.

The first time I saw value in a chat room in a professional setting was when I got invited to a Basecamp “deploy room” by fellow Agile Admin Peco (or was it Ernest?) at NI when our quarterly release cycle was going super poorly, and all of us (100 other people) were waiting around at hour #34 trying to figure out why some random enterprise application was holding up the rest of the release process. Post invitation to the room, I was able to look at the past messages between the ops team about application failures, and then realized pretty quickly that our databases weren’t actually responding like they should. It took all of 10 minutes to ask someone on the ops side with credentials to run a database query, and figure out that the db creds were all wrong. 2 hours later, the release was all done…

That moment made me realize that 1×1 chats were great, but having a persistent chat rooms with teams of people added value to an organization.

Recently, a colleague asked me a simple question that made me reflect. He asked, “What’s the big deal about Slack?”. At work, there’s been a big push to move towards Slack, when we’ve had 1×1 chat forever. Here are my 5 most compelling reasons for doing so:

1) Collaboration++: 15 years ago, software was a simpler, and there was no cloud/microservices. You’d have 1 large binary to deploy for a platform, and typically have a few folks who understood the overall workings of platform. Today, with microservices, you require a bunch of applications to deploy, and each of these have specific owners who understand specifics. Thus, you’re going to have to have conversations with multiple folks to figure out any issues. Having this in a room setting versus a 1×1 setting gets you to a resolution faster.

2) Chat metadata: Chat is less about words, and more about conversations that include images, links, slash commands, workflows etc. Chatops tools make pasting these much easier than before, and looking at formatted code in Slack is so much easier to read than looking at the same in pidgin.

3) Chat History: Chat apps now give you history – even from when you were not online or in the chat room. This is valuable from the perspective that you can see everything from when you weren’t around, and don’t have to ask someone to keep repeating the problem over and over again. You can just scroll up, read the context, and be ready to help if you can. This is my one knock against IRC (or at least the implementation of IRC at a company I worked at); it was nice to have everyone in a spot, but it only worked when we were VPN’ed in, and had no history.

4) Pipelining with chatbots: Continuous Integration/Delivery is all the rage these days! Having a chat system that allows for your devops systems to push data is a primary requirement in order to build a pipeline of this sort. Responses to broken builds, tests, alerts are quicker when the data associated with these are transmitted to a chatroom that you’re looking at, than having to look at Jenkins all the time. Chatbots are invaluable in this scenario, and help you with information flow.

5) The new normal: A new generation of engineers already do this. It’s already part of the culture for the next generation of engineers who work on open source (for example, kubernetes slack) and there’s even chatter about slack at Universities now. The world is evolving towards broader conversation, and not having chatops tools will hurt your company in terms of hiring and retention.

 

Agree/Disagree, or have a different perspective? Let me know by commenting below!

2 Comments

Filed under Agile, DevOps

Docker 101

Working at Stackengine, and now at Oracle, I’ve been working in the Docker ecosystem for the last 5 years!

While containerization has taken the IT and devops world by storm, a lot of larger enterprises might still be on the outside looking in. If you find yourself in that boat, you’re in luck!

Here’s a quick video on getting you running your very first Docker container on your Mac in under 5 minutes.

Also, I had the pleasure of traveling back to my childhood hometown of Bengaluru and presenting a workshop at Code Conf this year. I’ll create a separate post about my travels, but I got to present a workshop lab that is an Introduction to Containers. This lab is a perfect follow on to the video above, and will help you get started on your Docker journey! Let me know if you have questions.

If you’re more of a product manager, or just looking for why you’d want to use Docker, and understand its usecases, you can take a look at this presentation I had published on Why to docker? as shown below.

Questions, comments, or concerns? Hit us up by leaving a comment below…

2 Comments

Filed under DevOps

DevOps Foundations: Lean and Agile

Well you’re in for a treat – we’re getting all of the Agile Admins in on making DevOps courses, and Karthik and I did a course that’s just released today – DevOps Foundations: Lean and Agile.

It’s available both on LinkedIn Learning:
https://www.linkedin.com/learning/devops-foundations-lean-and-agile

and on Lynda.com:
https://www.lynda.com/JIRA-tutorials/DevOps-Foundations-Lean-Agile/622078-2.html

After James and I did DevOps Foundations, the “101” course, we were focused on building out courses for the three major practice areas of DevOps – Continuous Deployment, Infrastructure Automation, and Site Reliability Engineering (in progress now). But our lynda.com content manager said there was interest in us also expanding on the use of Agile and Lean especially as it relates to DevOps.

Karthik is our agile admin Agile expert; he’s presented at several Agile conferences and the like, so he and I decided to take it on.  But how would we bring a DevOps specific take to it?  We started outlining a course and realized it could turn into a giant boring encyclopedia of every Lean and Agile term ever. Most of what we have to add isn’t reading definitions, it’s sharing our experiences actually doing this (my Scrum for Operations series on this blog is perennially popular).

So we decided to take a tip from both Eliahu Goldratt’s The Goal and Gene Kim’s The Phoenix Project by framing the course as a fictional story!  By stitching a narrative together of a Lean, Agile, DevOps transformation of a hypothetical company out of our real world stories from a variety of implementations, we figured we could explain the concepts in context and make them more interesting.  Let us know what you think!

Lynda Course Description:

By applying lean and agile principles, engineering teams can deliver better systems and better business outcomes—both of which are crucial to the success of DevOps. In this course, instructors Ernest Mueller and Karthik Gaekwad discuss the theories, techniques, and benefits of agile and lean. Learn how they can be applied to operations teams to create a more effective flow from development into operations and accelerate your path of “concept to cash.” In addition to key concepts, you can hear in-the-trenches examples of implementing lean and agile in real-world software organizations.

Topics Include:
What is agile?
What is lean?
Measuring success
Learning and adapting
Building a culture of metrics
Continuous learning
Advanced concepts

Duration:
1h 26m

Leave a comment

Filed under Agile