Tag Archives: DevOps

How To Learn DevOps?

Hey all! James and I are preparing to revise our LinkedIn Learning course, DevOps Foundations, a three hour set of videos designed to orient beginners in the whole scope of DevOps. 

We created the course in 2016 primarily because at the time there were no good introductions to DevOps. You needed to know what blogs to follow and what events to go to and that was it. Even the DevOps Handbook hadn’t come out yet. And this provided a very high barrier to entry to the field. And we believe in learning and collaboration so we knew what we had to do!

Since then, it’s been one of the top tech courses on LinkedIn Learning with over 400,000 learners so far and has generated a dozen other courses drilling down into detail in specific areas. The things that make it worth it to me is the people we run across who say “this helped me improve my career.” My favorite was one gentleman who pulled me aside at the Aqua Security booth at RSA back before the pandemic and said “Hey, I had just gotten out of the Army and was trying to get a good job, and so was looking at tech. Your course oriented me enough that I got a sales job here!” Being able to help people like that is a rare privilege and we really value it.

Please fill out our survey to let us know what you think are the key things someone needs to learn about DevOps – whether they have some existing dev or ops knowledge or are just getting into it!

Here’s the old table of contents for reference… A lot of this hasn’t changed, the basics are still the basics, but it has been 7 years and a lot has changed, some things to add, some things to change, some things to cut. Let us know your opinion!

  • DevOps Basics
    • What Is DevOps? – Understand the meaning of DevOps and why you might care about it.
    • DevOps Core Values: CAMS – Culture, Automation, Measurement, and Sharing are the core values of DevOps.
    • DevOps Principles: The Three Ways – The Three Ways can guide your strategic approach to DevOps.
    • Your DevOps Playbook – There’s a developing list of patterns and methodologies that can help you transition to DevOps.
    • Ten Practices for DevOps Success: 10 through 6 – Tactical, pragmatic tips for DevOps success in your organization
    • Ten Practices for DevOps Success: 5 through 1 – Tactical, pragmatic tips for DevOps success in your organization
    • DevOps Tools – the Cart Or The Horse? – The role of tools in DevOps and tips for selecting and using tooling to achieve your end goal.
  • DevOps: A Culture Problem
    • The IT Crowd and the Coming Storm – Existing IT culture has both internal and external problems. Meanwhile, new challenges of scale and business cadence are pressing technology departments to change.
    • Use Your Words – Communication is the key to collaboration and solving problems when the stakes are high.
    • Do Unto Others – Build trust and respect and eliminate blame and hostility in your teams.
    • Throwing Things Over Walls – Break down the silos and establish a culture of responsibility and ownership, and align your teams to support the flow of concept to cash.
    • Kaizen: Continuous Improvement – Everything can be iterated upon to make it better – even yourself!
  • The Building Blocks of DevOps
    • DevOps Building Block: Agile – DevOps extends Agile principles to include deployment and operations.
    • DevOps Building Block: Lean – Understanding Lean can be the difference between a DevOps implementation that helps you achieve your company’s goals and one that’s just “the same but different.”
    • ITIL, ITSM, and the SDLC – Where does the “old school” fit in to a DevOps world?
  • Infrastructure Automation
    • Infrastructure As Code – Take a fundamentally different approach to building distributed systems whether in the datacenter or in the cloud.
    • Golden Image to Foil Ball – Learn about configuration mangement, automated provisioning, deployment and orchestration.
    • Immutable Deployment – With the rise of containers, different CM patterns are gaining currency.
    • Your Infrastructure Toolchain – Common tools in this space include Chef, Puppet, and Ansible but new container-based approaches like docker are on the rise. [Yes, this was before terraform and kubernetes, definitely places to update]
  • Continuous Delivery
    • Small + Fast = Better – Delivering small batches of change quickly reduces risk, improves quality, and restricts technical debt.
    • Continuous Integration Practices – Learn about Continuous Integration, Delivery, and Deployment, which you need and how to get there.
    • The Continuous Delivery Pipeline
    • The Role Of QA – Move from manual testing to automated with Test Driven Development (TDD) and Behavior Driven Development (BDD).
    • Your CI Toolchain – From Github to Jenkins, your code pipeline consists of many different parts with specific functions.
  • Reliability Engineering
    • Engineering Doesn’t End With Deployment – If you build it, you run it and other patterns for reliability engineering.
    • Design For Operation – Theory – Building a system to be resilient is the highest leverage step in ensuring high uptime and low MTTR.
    • Design For Operation – Practice – Ops has learned hard lessons about resiliency over the years – take it into account when building your applications.
    • Operate For Design: Metrics and Monitoring – Operational support isn’t just keeping the systems up, it provides crucial feedback back into the development cycle. {Yes, the kids call this observability now]
    • Operate for Design: Logging
    • Your SRE Toolchain – Monitoring, troubleshooting, and metrics are a vital space in your tooling strategy.
  • Additional DevOps Resources
    • Unicorns, Horses, and Donkeys, Oh My – In an emerging discipline, going to events to learn from other expert practitioners is your fastest route to success.
    • Ten Best DevOps Books You Need to Read – There’s a growing number of books on DevOps, here’s our top 10 reading list.
    • Navigating The Series of Tubes – DevOps information on the Web is fragmented and hard to find sometimes; here’s some of the best places to watch.
  • The Future of DevOps
    • Cloud to Containers to Serverless – Profound changes to our computing model have arrived to challenge many of our established practices.
    • The Rugged Frontier of DevOps: Security – Security is changing and is rapidly uptaking the DevOps movement, we cover the major implications here. {Yes. the kids call this DevSecOps now]
  • Conclusion
    • Next Steps: Am I a DevOp now? – Learn what next steps you should pursue for growing in DevOps understanding and practice.

Leave a comment

Filed under DevOps

New DevOps Courses Are Out!

James and I were revising one of our LinkedIn Learning courses, DevOps Foundations: Infrastructure as Code to keep up with the times, and while we were out there filming we knocked out some new courses as well!

DevOps for Managers talks about DevOps from the management perspective. What do you need to know, and how can you best unlock the success of the teams you’re working with when they are – or want to – excel at DevOps?

DevOps Antipatterns explores some common pitfalls that people fall into when starting out (or, even, later…)

Check them out! For reference here’s our whole curriculum that the agile admins have put out to help you in your path to thriving in tech.

Agile Admin LinkedIn Learning Courses

DevOps 101

DevOps 200-level

DevOps 300-level

DevOps 400-level

DevSecOps

Cloud Native

Observability

Leave a comment

Filed under DevOps

DevOps for Managers Library

James and I are working on a LinkedIn Learning course entitled “DevOps for Managers” and I wanted to share some of the books we love that we’ve found helpful in preparing it! We’d love to hear books you think are indispensable for DevOps managers. We’ve generally omitted general management books like First, Break All The Rules and DevOps non-management-specific books like Continuous Delivery, trying to focus on the specific intersection of tech and management.

Here’s our list, post yours in comments!

The Phoenix Project: A Novel about IT, DevOps, and Helping Your Business Win and The Unicorn Project: A Novel About Developers, Digital Disruption, and Thriving in the Age of Data by Gene Kim et al. demonstrate the benefits of DevOps transformations on an organization in a story format.

Accelerate: Building and Scaling High Performing Technology Organizations by Forsgren, Humble, and Kim gathers the DORA research on DevOps into a summary of how to practice high performance leadership and management.

The DevOps Handbook: How To Create World-Class Agility, Reliability, & Security in Technology Organizations by Kim, Humble, Debois, and Willis is an encyclopedic guide to implementing the Three Ways in an organization.

An Elegant Puzzle: Systems of Engineering Management by Will Larson is specifically about managing modern engineering teams.

Team Topologies: Organizing Business and Technology Teams for Fast Flow by Matthew Skelton and Manuel Pais focuses on team organization and the communication and outcome ramifications thereof.

Turn the Ship Around!: A True Story of Turning Followers into Leaders by David Marquet isn’t DevOps specific but is a great description of how to enable meaningful decision making at the lowest level. 

Measure What Matters: OKRs – The SImple Idea That Drives 10x Growth by John Doerr describes how to set goals using OKRs and avoid many of the naive goal-setting pitfalls that beset organizations that decide they want to be goal driven.

Smart & Gets Things Done: Joel Spoksly’s Concise Guide To Finding The Best Technical Talent by Joel Spolsky talks about how to attract, hire, and retain the best engineers.

The No Asshole Rule: Building a Civilized Workplace and Surviving One That Isn’t by Robert Sutton talks about traits to screen out to ensure a collaborative organization.

Managing Humans: Biting and Humorous Tales of a Software Engineering Manager by Michael Lopp has a good bit of engineer-managing wisdom.

The Lean Mindset: Ask The Right Questions by Mary and Tom Poppendieck shows how to focus your thoughts and iterate towards good products, including your internal products and services.

Building a StoryBrand: Clarify Your Message So Customers Will Listen by Donald Miller is intended for Marketing but for DevOps, especially platform teams, being able to concisely define and communicate your value is key.

A Seat at the Table: IT Leadership in the Age of Agility by Mark Schwartz helps IT leaders take on Agile, Lean, and DevOps.

I’ve heard about Camille Fournier’s The Manager’s Path, Julie Zhuo’s The Making of a Manager, and Lara Hogan’s Resilient Management but haven’t read any of them yet so can’t vouch.

1 Comment

Filed under DevOps, Management

DevOpsDays Austin 2022 Retrospective

DevOpsDays Austin was back in person in 2022 for our 10 year anniversary! So how’d it go? We are committed to transparency and metrics here so enjoy our retrospective.

Financially it went well; we sold 317 tickets and had 296 people show up (an almost unheard of 93% show rate), we were able to donate $28,000 to support the LGBT+ members of our community via Out Youth and The Trevor Project, bringing us to $100,000 donated to charity total over the history of the event, and still ended up with a $1000 increase in our bank account.

We also send out retrospective surveys to our attendees, speakers, sponsors, volunteers, and even our fellow organizers to find out how we’re doing and get an idea of what we should do better on (or keep doing) in the future!

Here’s the deeper details if you want them: DevOpsDays 2022 Retrospective

To sum up, however, it’s looking good! Our last surveys were from 2019, from our last pre-pandemic DevOpsDays Austin, so we have a previous number to compare to.

Our attendee NPS was up to 77 (44 responses) from 62 (50 responses). And the things people loved were, basically, the personal interaction. Community, people, discussions, and openspaces were the most cited positives by far. We knew people had been missing that for a couple years now so our retrospective format and event plan were specifically designed to promote small group interactions.

The gripes were more varied. Primary was the food, which is fair enough. While we don’t intend to change from a boxed lunch format – it leaves so much more time for the actual conference, so we left fancy catered lunches behind long ago – we were forced to use the venue caterer and they ran out of food especially veggie options, and we had asked for breakfast tacos both mornings and on day two the breakfast was what I can only call “leftover meat bits.” So room for improvement, with the understanding that boxed lunches are here to stay, but we’ll definitely see what we can do about better options and especially making sure there’s not shortages of anyone’s dietary needs.

The other leading concern we’re just plain going to ignore, and here’s why. It was “the retro format – but what about technical talks, what about content for newbies?”

At DevOpsDays Austin we explicitly reject the assumption that all events must be the same generic thing every time. We specifically change our format every year. We’ve had the Monsters of DevOps where we went for flying in big keynoters (including all the authors of the DevOps Handbook) and doing everything up huge; we’ve had DevOps Unplugged where talks were voted in on site and there were no sponsor tables. This year we had a “class reunion” format with talks only being 20 minute “retrospectives” from what the speaker’s learned over their time in DevOps (some speakers were experienced, others were new voices). We very, very clearly talked about this format on our website, social media, and emails to attendees and sponsors. In the end, people just don’t read, and there’s nothing really to do about that. And we won’t be doing the retro format two years in a row, we’ll keep mixing it up!

Organizer feedback was good (we have 20 organizers), up slightly, with everyone enjoying working with the group, and some concerns about unclear roles and roles already taken. That’s always a challenge – we have a lot of organizers but not all of them are up to actually leading something. We have people volunteer to own roles and then encourage them to reach out to the others/the others to chip in when we need something, but that doesn’t always go well, which is frustrating for everyone. In the end, most roles need someone who can commit to consistent participation over the planning period (there’s a couple specialty roles like making signage that can be backloaded, but not many). But we want to be inclusive and not tell people “no, take the year off if you can’t be putting in a couple hours a week including making the organizer calls, and truly own something.” We’ve wrestled with this for 10 years, no clear answer is in sight.

Speakers love speaking at the event. NPS was 92, slightly up from 90 in 2019. They love the audience, how supportive and welcoming they are, and how low stress and chill the experience is. There’s always some AV issues as a fly in the ointment – we do AV checks but not everyone shows up for them.

Volunteers have a good experience too. NPS was 88, slightly down from 94 but still good; we try to make sure that the load isn’t too much on any given volunteer so they can also enjoy the event. Posting the openspace topics is always a challenge each year; we tweet out photos and then desperately type them into the sessionize, but a bunch of attendees are social media impaired I guess and it’s hard to get the schedule to everyone, but there’s not a lot of options given that openspaces are predicated on doing the agenda immediately previous; I’m not sure more time would help short of printing out copies or having live monitors everywhere.

Sponsor feedback was down from 60 to 50 NPS. They do like the audience and authentic content. The main problem was the new venue and unclear flow meant that the platinum sponsor rooms were more out of the way than we’d planned (we gave them tables in the gold area as well once this became clear). And then the general sponsor gripes about it not being a good lead gen event. We always tell sponsors this is a good participation event, not a good lead gen event, no badge scanners, no sponsor list, etc., but a previously mentioned people don’t read, plus the teams being sent out aren’t the people buying the sponsorships and often just assume they’ll be getting a standard conference experience. We sell out every year so I’ll worry about it when that stops.

There’s one other thing worth mentioning, which is that we did require masking at the event and asked people to either be vaccinated or test prior to the event.

One the one hand, a couple sponsors and attendees griped about the masking.

On the other hand, despite other events resulting in superspreading (Kubecon EU, RSA, even some DevOpsDays events):

So, with all due respect, we are very happy with our choice and that we had a safe event. No one likes wearing masks. If you don’t like it enough to not come – don’t come. Hopefully it won’t be necessary next year.

Everything was pretty good! There was one issue, though – in all the survey sub-questions, there was a drop in the perceived friendliness of the organizer team, so we’re going to make some changes there – stay tuned to hear what!

Leave a comment

Filed under Conferences, DevOps

DevOpsDays Austin Donates to Out Youth and The Trevor Project To Support LGBTQ+ Youth

We were psyched to be back and meeting in person for DevOpsDays Austin 2022 this year! Charitable donation is a part of our Austin tech culture and very important to us, and since it was our 10th anniversary we aimed to hit a total of $100,000 in charitable donations since we started the event in 2012. And we did it! We’re happy to have $28,000 to give to local charities after our event this year.

That left us with the question of who to donate to. We like to choose things that fill the greatest need in our community at the time, and strongly bias towards supporting Austin area charities. Our state government decided to help us make our decision by starting a pogrom of discrimination against LGBTQ+ Texans.

Many of our colleagues in the Austin technology community are gay, lesbian, transgender, nonbinary, or identify with other nontraditional gender identities and sexual preferences, or have family members that are. I myself have a transgender son who I’ve loved and supported through his transition, and now he’s a happy, healthy adult. We find the attempts of the Texas government to institutionalize hostile behavior towards them deeply unacceptable and want to find ways to support them.

We looked initially at charities like Equality Texas that are working to ensure the rights of LGBTQ+ Texans, but as we discussed we wanted our funds to go directly to the benefit of people in need, and not all go to the lawyers fighting the long fight.

Since $28,000 is a lot, we decided to split it up into two $14,000 donations. After some research we nominated two recipients, Out Youth and The Trevor Project, and brought them to the DoDA organizer team for a vote, which they enthusiastically approved.

We selected Out Youth for our first donation as a deeply local Austin organization directly supporting LGBT youth.

For 32 years, Out Youth has supported Central Texas LGBTQIA+ youth and young adults by providing safe places where they are loved, acknowledged, and accepted for exactly who they are. Their life-saving and life-changing programs and services ensure these promising young people develop into happy, healthy, successful adults. Out Youth hosts a variety of programs to keep their youth mentally and physically safe such as drop-in times at their youth community center, by offering free individual counseling and group therapy sessions, and through their in-school-support services. To find out more information about ways to get involved and about their services, please visit outyouth.org today.

It’s funny that our relationships with these charities usually start with us showing up with a big surprise donation and then after that getting deeply involved with the organization; we plan to go tour their house and spread the word about volunteer opportunities with them to the Austin technology community.

Not only do I have a transgender son, but also fellow Agile Admin and conference organizer James lost a brother to suicide. So while The Trevor Project isn’t Austin based per se, it does help many Austinites, and its mission of suicide prevention among LGBTQ+ youth is deeply and personally meaningful to us both. Therefore we chose them for our second donation this year.

The Trevor Project has worked to save young lives by providing support through our free and confidential crisis programs on platforms where young people spend their time — online and on the phone: TrevorLifeline, TrevorChat and TrevorText. We also run TrevorSpace, the world’s largest safe space social networking site for LGBTQ youth, and operate innovative education, research, and advocacy programs.

  • The Trevor Project’s research has found that having at least one accepting adult in an LGBTQ young person’s life reduces their risk of suicide by 40%.
  • Transgender and nonbinary youth who reported having pronouns respected by all of the people they lived with attempted suicide at half the rate of those who did not have their pronouns respected by anyone with whom they lived.
  • “You are lovable” – this is one of the most common phrases The Trevor Project’s crisis counselors share with youth in crisis.
  • According to Trevor’s research 42% of LGBTQ youth seriously considered attempting suicide in the past year, including more than half of transgender and nonbinary youth.

You can sign up to become a volunteer counselor on their site; there’s extensive training and it requires a year commitment.

In closing, we appreciate the work Out Youth and The Trevor Project are doing and hope that others will look into finding ways to support them as well.

To the LBGTQ+ technologists in Austin, you are welcome, and both DevOpsDays Austin and other user groups we run like CloudAustin have published codes of conduct that don’t allow any hostile behavior towards you at our events, and we look forward to interacting with you there. Happy Pride Month!

Leave a comment

Filed under Conferences, DevOps

DevOpsDays Austin Hits $100,000 In Charitable Donations

DevOpsDays Austin believes in supporting the local community that supports us, the techies of Austin. As we’ve grown and we’ve been able to end some of our years with extra money from our sponsors. We are careful to be thrifty with the event and rely on our volunteers to do a lot, and all DevOpsDays events run on a nonprofit basis; one of the few core rules of all DevOpsDays is that the events are not for individual or corporate profit.

The Austin technical community is a cross-section of, and part of, our community. We have a diverse set of individuals from many backgrounds and neighborhoods all around the area. As technologists we are largely blessed with decent salaries and technology companies have lots of money, and many of the keenest needs in the area need relatively little funding to make a real difference. We believe it’s our responsibility as a part of the community to use part of what we make to give back to support the most vulnerable among us. Therefore we’ve made it a point to use our excess funds to give back to charities that directly help those most in need.

We started DevOpsDays Austin in 2012 from nothing, and relied on a free venue (courtesy of my employer at the time, National Instruments). We made enough to make a down payment on a venue in 2014, and by 2015 we were confident enough of our finances that we considered our first charitable donation.

The Charity Page In Our 2022 Yearbook

The natural choice was the Central Texas Food Bank (at the time, known as the Capital Area Food Bank), a well known long time Austin charity that combats hunger in the area. We gave $5000, and we also did a charity drive at the event, handing out leftover t-shirts and swag from the previous two years to those who made a donation of their own, which sent another $2600 to the food bank.

In 2016 we moved to a new, larger, much more expensive venue (the Darrell K. Royal Texas Memorial Stadium out at UT Austin) so we could let more people attend DevOpsDays. That put us at the limit of our finances for a couple years, especially with our 2017 “Monsters of DevOps” blowout conference with many international speakers and great fun events. But by 2018 we had that venue dialed in as well and had $20,000 left over that we decided to donate to charity. That year, instead of sending all the donation to one charity, we let each of our 20 organizers send $1000 to a charity of their choice. This ended up driving our helpful accountant Laura at ConferenceOps batty trying to get proper tax receipts from everyone, however, and we promised her we wouldn’t do that again.

Then in 2019 we had a great event, sponsors were in a right frenzy to get in and we had to add more sponsor booths to accommodate them – which was a lot of work, but left us also with a lot of leftover money ($25,000, off a conference with only a $200k total budget). We weren’t sure how we could be the most effective with a gift like this, and one of our organizers said “Did you know… There’s a new project here in Austin that builds miniature houses for the homeless in a beautiful community?” And that’s how we were introduced to the Community First! Village, a quickly growing and very effective outgrowth of Mobile Loaves and Fishes to house the homeless. And it turns out $25,000 is exactly how much it takes to build one of those homes. Our organizers enthusiastically approved the donation and we went out and did a great tour of the site, and many of us have returned as volunteers since.

And then the hard times came. During the pandemic, DevOpsDays Austin was on ice. In fact, we had planned on moving to an even more expensive hotel venue and had a down payment in place when the lockdown came, and we had to get a lawyer into play to get our deposit back.

But the needs of the community weren’t on hold. We have many Black and brown technologists in our community, and the high profile brutality directed at them was completely unacceptable. Long time friend of DevOpsDays Austin, John Wills, started a fundraiser for Black Lives Matter around making a quilt with all his DevOpsDays shirts (many of which were from Austin) in 2020, and we felt compelled to donate $2000 of the more than $12,000 he raised.

DevOpsDays Shirt Quilt

Then we were in the long night of lockdown. We weren’t doing anything from a DevOpsDays Austin perspective in 2021, though there was a virtual DevOpsDays Texas conference to fill in some of the gap. But as jobs and aid dried up, hunger became a critical need again in Austin. Fellow organizer Boyd Hemphill encouraged people to help out and volunteer during a virtual meetup, and his words made my conscience burn till I brought it to my fellow DevOpsDays Austin organizers to see if we could dip into our reserves and help. They all enthusiastically approved a $10,000 donation to our old friends the Central Texas Food Bank again.

Two donations without any revenue, that’s good enough till we can have an event again, right? Well, you’d think, and then Russia went and invaded Ukraine.

While we’re an Austin organization and we’ve always given to help out in Austin directly, we have many Ukrainians as part of our local tech community. I worked with many of them hand in hand while I was running teams at Bazaarvoice, as we had a great relationship with the Ukrainian consulting company Softserve. We brought many of them here to Austin to work with us, we went out together, I had toasted them with “Slava Ukraini!” Many of our organizers had similar experiences. And since we don’t like bullies around here, that riled us up. After a discussion along the lines of “well, we started from nothing once, we can do it again if we have to,” we donated $10,000 to Ukrainian relief organizations Razom for Ukraine and Nova Ukraine.

And that brings us up to date with the past, but we finally managed to have DevOpsDays Austin again! In May, we got a great venue (the University of Texas Alumni Center, at half price courtesy of Bill our venue lead being an UT alum) and planned for a slightly smaller than usual (350 masked attendees, to hedge against super-spreading) conference on our 10 year anniversary – the DevOpsDays Austin 10 Year Class Reunion.

Since it was our 10th anniversary, we did a yearbook. And when I put the charitable donation page together for the yearbook, I realized we’d given $72,000 to charity over the years. 10 years and an even $100,000 sounded mighty nice.

The conference went great, and all those sponsors have been saving up their marketing money wondering what to do with it. After some laborious running of numbers I realized we could free up the $28,000 donation to get to that bar and leave enough for us to make a venue down payment the next year.

As we contemplated this year’s donation our Texas state government decided to openly attack the LGBT+ citizens of our state. Many of those in our technical community we meet with every month in user groups are gay, lesbian, transgender, binary, and so on, and this is a direct attack on many of our coworkers, colleagues, and friends. And not just them, but their children.

As a result we gave $14,000 to The Trevor Project, a national service that provides suicide prevention hotlines for LGBT+ youth, $14,000 to Out Austin, a local place for youth of all sexual orientations and gender identities. I’ll write a separate post about those organizations and why them, and more importantly how you can help.

But in the end we’ve been very happy that we’ve been able to use our position as techies in the tech hotspot of Austin to consistently give back. We’d challenge other conferences, tech companies, and individual technologists to do so as well – especially to reputable charities that directly help those who need it.

I hope that DevOpsDays Austin can continue to give back in this way over the next ten years too!

1 Comment

Filed under Conferences, DevOps

DevOpsDays Austin is Back For 2022!

Your DevOpsDays Austin 2022 Organizers!

Well, we had to skip 2 years in a row due to the pandemic, but we were finally able to have DevOpsDays Austin in person again in 2022! It’s our tenth anniversary of DoD Austin, we had the first one at National Instruments back in 2012, one of the early DevOpsDays in the US.

We had to move to a new venue, and used the beautiful University of Texas Etter-Harbin Alumni Center (our site lead Bill is a UT alum which makes it half price!). The Etter-Harbin Center is right across from the stadium where we had DoDA in the years leading up to the hiatus. It has plenty of great outdoor spaces, which we used for lunches and happy hours, as well as a great main ballroom with views of the outside. It worked great for our target of 350 attendees, and we think we could make it work for more in the future.

We were thinking and came across the perfect theme – it’s our tenth anniversary, and we’re just back from the pandemic, and DevOps is also just a little more than ten years old and at a weird inflection point that has people asking “is DevOps dead? Where does DevOps go from here?” So we decided that since we were also in the Alumni Center, the obvious theme was our 10 Year Class Reunion!

We don’t take our themes lightly at DevOpsDays Austin. We settled on a new theme for our talks – instead of the normal RFC for whatever technical and culture topics, we required all talks to be a retrospective format – reflecting on what you’ve learned over the years of DevOps and what you think the future holds. We had lots of great speakers, many of whom are long time parts of the DoD Austin community, both locals like Rob Hirschfeld, Christa Meck, Ross Dickey, and Victor Trac, as well as folks from other parts of the Earth like Patrick Debois, Damon Edwards, J. Paul Reed, Pete Cheslock, and Michael Cote, who all frequently come to Austin to share with us.

And we printed a yearbook, with pics of speakers from all the events, our tshirts over the years, and more! Very snazzy, and we had people sign each others’ yearbooks to add some fun to the hallway track. In fact, you can view the yearbook online and buy a hardcopy here if you want!

The DevOpsDays 2022 Yearbook

We did require COVID protocols – masking inside and (honor system) vaccine or test, and while it is a bummer to not see each others’ faces, it also resulted in only one person I know of getting COVID the week after, so well worth it.

We didn’t have to worry about sponsor interest! We sold out quickly. Here’s the ones I got snaps of!

Everything went great, and it was super to finally get back together and interact with our local DevOps community. J. Paul Reed led a great session where a retro was done on DevOps in general!

And one of the best things is that we managed to carry on our tradition of giving our excess proceeds to charity! I’ll do a separate post on that, but the short form is that we contributed $28,000 to LGBT-supporting charities, half to The Trevor Project and half to Out Youth here in Austin, bringing us to $100,000 given to charity over our 10 years in existence! Stay tuned for more details on that…

2 Comments

Filed under Conferences, DevOps

SRE: The Biggest Lie Since Kanban

There is a lot of discussion lately about how SRE fits into or competes with or whatever-s with DevOps.  I’m scheduled to speak on a “SRE vs DevOps Smackdown” panel today here at Innotech Austin, and at the exact same time I see Bridget tweeting Liz Fong-Jones’ slides from Velocity on using SRE to implement DevOps. And the more I think about it, and see what people are doing, the more I’m getting worried.

The Big Lie

Just to get the easily provoked to put up their pitchforks, I don’t dislike SRE and I don’t dislike Kanban.  The reason I call Kanban a “Big Lie” is because really doing Kanban correctly and getting the value out of it requires even more discipline that doing something like Scrum.  But it looks so close to doing nothing new that many lazy teams out there say “they’re doing Kanban” and by that they mean they’re doing nothing, but they’ve turned on the Kanban view in JIRA for your convenience.  They have no predictability, they’re not managing WIP, they’re not identifying bottlenecks – they just have a visible board now and that’s it. I strongly believe from my experience that most teams “doing Kanban” are really doing mostly nothing.  There’s articles on this blog about how I make my teams I’m teaching Agile do Scrum first if they want to get to Kanban to build up the required discipline.  And I’m not just a crank, David Hawks from Agile Velocity just told our management team the same thing yesterday, which brought this back to mind for me and spurred this article.

Because I’m starting to see the same thing with SRE.  It’s not surprising – there was and is plenty of “DevOps-washing” of existing teams out there.  Rename your ops team DevOps, done. Well, at least DevOps was able to say “it’s a methodology not a job description or group name stop it” to force deeper thought – it’s why my team at work is the “Engineering Operations” team not “DevOps”, Lee Thompson insisted on that when he set it up! But SRE – yeah, it’s a team just like your own ops team, from an “org chart” viewpoint it looks the same. So doing SRE can – and in many shops does – mean doing nothing new. You just call your existing ops team SRE and figure you’re done.

A brief personal history lesson – my last job before DevOps hit was running the Web Systems team at National Instruments, an ops team.  That’s where we agile admins met, Peco and James were both ops engineers on that team! (Karthik was a dev we worked with.) We had smart people and did ops all right.  We had automation, monitoring, we had “definition of done” standards for new services. You wouldn’t have to squint too hard to just call that team a SRE team and call it a day. But, I wouldn’t wish that job on my worst enemy. It was brutal trying to do ops for just 4-5 dev teams, and that’s with business support, some shared goals, and so on. Our quality of life was terrible, we weren’t empowered, and no matter how hard we tried, success was always right out of our grasp. When we actually started a team using DevOps thinking at NI after that, the difference was night and day, and we actually began to enjoy our jobs as ops engineers. I would hate for anyone to deceive themselves into thinking they’re getting the goodness they should be able to get from a DevOps/”real” SRE approach while still just doing it the way we were doing it.

I have a friend at a local legal software firm, who told me they’re going through and just renaming all the QA folks to SWET (Software Engineer In Test), whether they can code or not, and all the Ops folks to SREs in this manner. One might be charitable and say they’re leaning forward and they intend to loop around and back that up with retraining or something, but… will they? Probably not, it’s just a rename to the hot new term without any of the changes to help those engineers succeed more in their jobs!

SRE isn’t “an implementation of DevOps” if you just apply it as a name for a hopped-up ops team.  Properly understood, it can be an implementation of one of the three parts of DevOps, Infrastructure As Code, Continuous Integration/Deployment, and Site Reliability Engineering. But note that reliability engineering doesn’t start with deploy to production; so much of it is Michael Nygard-esque techniques to write your app reliably in the first place; reliability engineering, in usual DevOps fashion, requires dev and ops work both way back in the dev cycle and out in production to work right. It doesn’t need to be a different team.  If it is, and that team doesn’t get to decide if it takes over ops for a given app, and it’s not allowed to spend 50% of its time on reducing toil and you’re not comping SREs like you do dev engineers – it’s not SRE and you’re a liar for calling it SRE. If you don’t keep DevOps principles in mind, you’re just going to get your old ops team with its old problems again.

That’s why SRE is a Big Lie – because it enables people to say they’re doing a thing that could help their organization succeed, and their dev and ops engineers to have a better career and life while doing so – but not really do it.  Yes, there have been Big Lies before, which is why I cite Kanban as another example – but even if the new criminal is pretty much like the old criminal, you still put their picture up on the post office wall.

Frankly, anyone pushing SRE that doesn’t put warning labels on it is contributing to the problem.  “Well but it mentions in chapter 20 of the second book,” said someone responding to the first version of this article on Twitter.  Not good enough. If something you’re selling is profoundly misused it’s your responsibility to be more up front about the issues.

The Little Problems

Now there are legitimate issues to have even with the “real SRE” model, at least the way that it’s usually being described.  The Google books kinda try to have it both ways, describing it as an engineering practice (how I describe it above and in the SRE course I did for LinkedIn) and describing it as “a team that works this way.”  Even among those not SRE-washing classical ops, the generally understood model is that SRE is a org/job title for a production operations team.

There’s an issue here, the problem of specialization.  If you are Google scale, well, then you’re going to have to specialize and a separate ops team makes sense.  But – first of all, you are not Google scale.  In my opinion, if you are under 100 engineers, you are committing an error by having a separate ops team. You need your product teams to own their products. Second of all – I don’t want to make an enemy of all the lovely Google engineers out there, but is your experience with Google services that they evolve quickly and get better once they go to wide release?  It’s not mine.  They rot.  Have you used Google Hangouts lately without it ending up with cursing and moving off to someone’s Zoom? That kind of specialization still has its downsides in terms of hindering your feedback loops that let you improve (the Second Way). Is SRE just Google-ese for “sustaining?”

I get that the Google folks say they still get feedback and innovation using the SRE model, I’m sure they do and they work hard at that, but that doesn’t change the fact that running a separate ops team is making a deliberate tradeoff between innovation and efficiency. There is no way in that you get as much feedback or improve as quickly with a separate team, you can compensate for it, but you’re still saying “look… Not as important.” Which is fine if that’s your situation, I worked at many companies with 200 abandoned apps in production and you had to do something.  But “not getting there in the first place” is better.

Some of the draw of the model, and why Google is highly aligned with it, is Kubernetes itself. k8s is very complex to run drives people back a little bit to the old priest-in-the-tabernacle model of “someone maintains the infrastructure and you write the app and then you have them deploy it,” but now there’s some standards (like deploying as a container) that make that OK – I guess? But if you think reliability, and observability, are the primary responsibility of an ops team that is not involved in constructing the application, you either have deep and profound company standards that allow seamless plugging of the one into the other or you’re fooling yourself. 90% of you are fooling yourself.

At this conference I heard “Service meshes!  They get you observability so your devs don’t have to think about it.” Do you not see how dangerous that mindset is?

SRE, as interpreted as “a separate newfangled ops team,” may work for some but you need to be realistic about the issues and tradeoffs you’re making.  Consider whether product teams supporting their product, maybe with aid from a platform team making tooling and an enabling/consulting/center of excellence team that can give expert advice?  DevOps helped us see how the “throw it over the wall from dev to ops” model was profoundly harming our industry.  Throwing over the wall from dev to SRE doesn’t improve that, it’s profoundly regressive. Doing SRE “right” to compensate for this, like doing Kanban right, requires more skill and discipline, not less – be realistic about whether you have Google levels of skill and discipline in your org, eh?

Conclusion

SRE (and Kanban) aren’t bad, they have their pros and cons, but they are easy to “pretend to do” in some minimal, cargo cult-ey way that gets you little of the benefits. And if you think spinning up an ops team and calling it SRE is “an implementation of DevOps” you’ve swallowed the worst poison pill the DevOps talk circuit can deal to you.

15 Comments

Filed under DevOps

DevOps Foundations: Lean and Agile

Well you’re in for a treat – we’re getting all of the Agile Admins in on making DevOps courses, and Karthik and I did a course that’s just released today – DevOps Foundations: Lean and Agile.

It’s available both on LinkedIn Learning:
https://www.linkedin.com/learning/devops-foundations-lean-and-agile

and on Lynda.com:
https://www.lynda.com/JIRA-tutorials/DevOps-Foundations-Lean-Agile/622078-2.html

After James and I did DevOps Foundations, the “101” course, we were focused on building out courses for the three major practice areas of DevOps – Continuous Deployment, Infrastructure Automation, and Site Reliability Engineering (in progress now). But our lynda.com content manager said there was interest in us also expanding on the use of Agile and Lean especially as it relates to DevOps.

Karthik is our agile admin Agile expert; he’s presented at several Agile conferences and the like, so he and I decided to take it on.  But how would we bring a DevOps specific take to it?  We started outlining a course and realized it could turn into a giant boring encyclopedia of every Lean and Agile term ever. Most of what we have to add isn’t reading definitions, it’s sharing our experiences actually doing this (my Scrum for Operations series on this blog is perennially popular).

So we decided to take a tip from both Eliahu Goldratt’s The Goal and Gene Kim’s The Phoenix Project by framing the course as a fictional story!  By stitching a narrative together of a Lean, Agile, DevOps transformation of a hypothetical company out of our real world stories from a variety of implementations, we figured we could explain the concepts in context and make them more interesting.  Let us know what you think!

Lynda Course Description:

By applying lean and agile principles, engineering teams can deliver better systems and better business outcomes—both of which are crucial to the success of DevOps. In this course, instructors Ernest Mueller and Karthik Gaekwad discuss the theories, techniques, and benefits of agile and lean. Learn how they can be applied to operations teams to create a more effective flow from development into operations and accelerate your path of “concept to cash.” In addition to key concepts, you can hear in-the-trenches examples of implementing lean and agile in real-world software organizations.

Topics Include:
What is agile?
What is lean?
Measuring success
Learning and adapting
Building a culture of metrics
Continuous learning
Advanced concepts

Duration:
1h 26m

Leave a comment

Filed under Agile

But How Can IT Do DevOps?

img_2455

That’s how!

While James and I were filming our lynda.com course, DevOps Fundamentals, I was impressed by this LinkedIn tech vending machine in their offices.  Swipe badge, get doodad.  I had just come off two different jobs where it was more like “pester IT for the thing you need, wait a couple weeks, maybe someone will drop by with one.”

It’s not “Chef or Puppet,” but it is self service automation!  Think outside the box.  Or in this case, I guess, inside a box. Kudos to LinkedIn IT for a user friendly solution.

2 Comments

Filed under DevOps