Monthly Archives: March 2016

Classy up your curl with curl-trace

 

Let’s say you are debugging some simple web requests and trying to discern where things are slowing down.  Curl is perfect for that.  Well, sort of perfect. I don’t know about you but I forget all the switches for curl to make it work like I want.  Especially in a situation where you need to do something quickly.

Let me introduce you to curl-trace.

It’s not a new thing to install, its just an opinionated way to run curl.  To give you a feel for what it does, lets start with the output from curl-trace.

Screenshot 2016-03-11 10.15.35

As you can see, this breaks up the request details like response code, redirects and IP in the Request Details section and then breaks down the timing of the request in the Timing Analysis section.  This uses curl’s --write-out option and was inspired by this post, this post, and my co-worker Marcus Barczak.

The goal of curl-trace is to quickly expose details for troubleshooting web performance.

How to setup curl-trace

Step 1

Download .curl-format from github (or copy from below)

\n
 Request Details:\n
 url: %{url_effective}\n
 num_redirects: %{num_redirects}\n
 content_type: %{content_type}\n
 response_code: %{response_code}\n
 remote_ip: %{remote_ip}\n
 \n
 Timing Analysis:\n
 time_namelookup: %{time_namelookup}\n
 time_connect: %{time_connect}\n
 time_appconnect: %{time_appconnect}\n
 time_pretransfer: %{time_pretransfer}\n
 time_redirect: %{time_redirect}\n
 time_starttransfer: %{time_starttransfer}\n
 ----------\n
 time_total: %{time_total}\n
 \n

And put that in your home directory as .curl-format or wherever you find convenient.

Step 2

Add an alias to your .bash_profile (and source .bash_profile) for curl-trace like this:


alias curl-trace='curl -w "@/path/to/.curl-format" -o /dev/null -s'

Be sure to change the /path/to/.curl-format to the location you saved .curl-format. Once you do that and source your .bash_profile you are ready to go.

Usage

Now you can run this:

$ curl-trace https://google.com

Or follow redirects with -L

$ curl-trace -L https://google.com

Thats it…

Now you are ready to use curl-trace. If you have anything to add to it, just send me an issue on github or a PR or ping me on twitter: https://twitter.com/wickett.

Enjoy!

UPDATE: 3/17/2016

There was a lot of good feedback on curl-trace so it has now been moved to its own repo: https://github.com/wickett/curl-trace

 

1 Comment

Filed under DevOps

RSAC gets down with the Rugged DevOps!

This year at RSAC—you know RSA, its the really big InfoSec conference that isn’t BlackHat/DefCon—there was a mini-conference on Rugged DevOps. For the last few years they have been featuring this mini-conference as a way to bring DevOps into the InfoSec community.

I did a writeup of the event over on Medium and I hope you find it interesting. One of my favorite parts of the event is summed up from that article:

To give you a feel for how well it went, I think it is easily summed up with what happened at the closing. To a mostly full room of about 500 people the question was asked, “How many of you have been here all day?” Over 80% of the hands went up. For being a conference within a conference that number is surprising, for doing that with the InfoSec crowd, it is proof that the industry culture is truly shifting.

Security is ready to join the DevOps tribe.  It’s our duty as stewards of DevOps to do this right.

In the article on Medium I link to all the talks and speakers at the event. Check out the presos on SlideShare, follow the speakers on Twitter and most importantly be part of the joining of the tribes.

Leave a comment

Filed under DevOps