Just had a problem that I thought I’d document the solution to for the world…
In our build pipeline at work, we use maven and the fabric8 docker-maven-plugin to manage our builds. We love it, developers can just “mvn install” locally and then the Atlassian Bamboo build system just “mvn deploy”s in the exact same way.
Well, so we had some builds that suddenly weren’t able to pull the base images specified in our Dockerfiles down from Dockerhub, breaking the build with 500 error messages like:
[ERROR] DOCKER> Unable to pull 'library/debian:sid' from registry 'docker.io' : received unexpected HTTP status: 500 Server Error (Internal Server Error: 500) [received unexpected HTTP status: 500 Server Error (Internal Server Error: 500)]
But it worked fine on our local box. And it could pull our custom images from Artifactory fine. What’s the problem here? Bamboo? The plugin? Well, some helpful community folks helped home in on it, it turns out that for some versions of Java 1.8, 8u131 and prior at least going back to 112, where there’s some problem (TLS? Root certs? Not really sure) that messes up when pulling a docker.io container from inside Java during our docker build step. My team’s microservices aren’t Java based so the Java version doesn’t come up much – but of course maven uses Java.
Upgrading the JDK version to 8u144 made the problem go away. We actually have an up to date curated Java version we use in Bamboo for our Java builds, but folks doing Python builds were just using the default “JDK 1.8” that Atlassian is putting on their Bamboo build agent AMI, which is of course old and suffers from this issue.
We use Oracle Application Server as our Java app server at NI. Yeah, yeah, I’ll wait till you stop laughing.
Why not JBoss or WebLogic or WebSphere? Well, a couple reasons. We made the decision five years ago, and JBoss wasn’t solid then, and we needed J2EE support so plain Tomcat wasn’t enough. And we’re a huge Oracle shop and figured that if we were using the same app server on the Web and our ERP tiers there’d be leverage in terms of developer knowledge etc. Would we make that same decision today? I’m not sure about that (I can hear my team members shouting “hell no” over the cube walls). Although since we’ve also gone with Oracle’s SOA suite for ESB and BPEL it would be harder to switch. But still tempting – Oracle has done a horrible job in getting their app server supported by other vendors. Every time we buy something and look at the supported app server section of their support matrix, and we ask “What about Oracle’s OAS?” we get expressions of mixed horror and pity from the supplier. (I liked it when the Chinese technical guy from one eComm vendor we had in responded to this question with, “You know, the Tomcat is good, and free! Maybe you use that!”)
Anyway, Oracle bought BEA a while back, which got keen interest from us. Stay with Oracle *and* use a good app server that other people support? Tempting! But Oracle’s been farting around for six months without coming out with a statement on what this will mean for the products. Oracle’s finally done a Webcast describing their strategy. Well, it’s half marketing and a celebration of how many million dollars they have. But there’s also a lot of product strategy in there. I’ll sum it up for you because the damn webcast is nearly two hours long, and I don’t want other people to have to waste that much time on it. Unless you like to hear someone go on about “strategic clarity” and “customer profiles,” in which case this is two hours of bliss for you and you should watch it. Although I also had the stream break a bunch of times while watching. Who the heck uses RealPlayer any more? Anyway, here’s a list of the interesting product facts from the Webcast. Some are marked with their timestamp if you want to fast forward to them and see more.