Tag Archives: austin

Why ha.ckers.org Doesn’t Get Hacked

The first LASCON session I went to was Why ha.ckers.org Doesn’t Get Hacked by James Flom (who with rsnake is ha.ckers.org).  By its nature, it gets like 500-1000 hack attempts a week, but they’ve kept it secure for six years now.

From the network perspective, they use dual firewalls running the openBSD open source pf, which does Cisco-style traffic inspection.  Systems inside have no egress, and they have the user traffic and admin traffic segmented to different firewalls  sets and switches.

On the systems, they use chroot jails mounted read only.  Old school!  Jails are virtualization on the cheap, and if combined with a read only filesystem, give you a single out of band point of update, and you can do upgrades with minimal downtime.  They monitor them from the parent host.

Rsnake has done a whole separate presentation on how he’s secured his browser – the biggest attack vector is often “compromise the browser of an admin” and not direct attack on the asset.

They went to WordPress for their software – how to secure that?  Obviously code security’s a nightmare there.  So they set up a defense in depth scheme where they check the source ip, cert, and user/pass auth at the firewall, then to admin proxy check source IP, path, htaccess user/pass, and finally do the app auth.

Other stuff they do:

• Secure logging to OSSEC – pflogd, waf logs, os logs, apache logs, parent logs, it goes off host so it’s reasonably tamper-proof
• On-host WAF – custom, more of a “Web IDS” really, which feeds back “naughty people” to the firewall for blocking
• For Apache – have your content owned by a different user, in their case there’s not even a user in the jail that can write to the files.
• Use file ACLs, too.

Use case – they found an Apache flaw, reported it, and as is too often the case, Apache didn’t care.  So they modded their pf to detect the premise of the attack and block it (not just the specific attack).  (Heard of slowloris?)

Their ISP has been an issue – as they’ve moved their ISPs have shut them down out of cluelessness sometimes (Time Warner Business Class FTL).

They are moving to relayd for load balancing and SSL.  The PCI rule about “stay encrypted all the way to the box” is dumb, because it would prevent them from doing useful security inspection at that layer.

A good talk, though sadly a lot of the direct takeaways would mean “go to FreeBSD,” which I would rather not do.  But a lot of the concepts can port to other OSes and pure virtualization/cloud scenarios.  And note how joining network security, OS security, and appsec gets you way more leverage than having “separate layers” where each layer only worries about itself.n

And may I just say that I love how Apache can be run “read only” – sadly, most software, even other open source software like Tomcat, can’t be.  It all wants to write down into its config and its running directories itself, and it’s a horrible design practice and security risk.  If you’re writing software, remember that if it’s compromised and it can write to its own exes/config/etc. you’re owned.  Make your software run on a read only FS (with read/write in /tmp for stuff acceptable).  It’s the right thing to do.

Why does bad software happen to good people?

First up at LASCON was the keynote by Matt Tesauro from Praetorian (and OWASP Foundation board member), speaking on “Why does bad software happen to good people?”  The problem in short is:

• Software is everywhere, in everything
• Software has problems
• Why do we have these problems, and why can’t we create a secure software] ecosystem?

The root causes boil down to:

• People trust software a lot nowadays
• Blame developers for problems
• Security of software is hidden
• Companies just CYA in their EULAs
• Lack of market reward for secure software
• First mover advantage, taking time on security often not done
• Regulation can’t keep up

So the trick is to address visibility of application security, and in a manner that can take root despite the market pressures against it.  We have to break the “black box” cycle of trust and find ways to prevent problems rather than focusing on coping with the aftermath.

He made the point that the physical engineering disciplines figured out safety testing long ago, like the “slump test” for concrete.  We don’t have the equivalent kind of standards and pervasive testability for software safety.  How do we make software testable, inspectable, and transparent?

Efforts underway:

• They got Craig Youngkins, a big python guy, to start Python Security.org, which has been successful as a developer-focused grass roots effort
• The Rugged Software Manifesto at ruggedsoftware.org is similar to the Agile Manifesto and it advocates resilient (including secure) software at the ideological level.

I really liked this talk and a number of things resonated with me.  First of all, working for a test & measurement company that serves the “real engineering” disciplines, I often have noted that software engineering needs best practices taken from those disciplines.  If it happens for jumbo jets then it can happen for your shitty business application.  Don’t appeal to complexity as a reason software can’t be inspected.

Also, the Rugged Software Manifesto dovetails well with a lot of our internal discussion on reliability.  And having “rugged” combine reliability, security, and other related concepts and make it appealing to grass roots developers is great.  “Quality initiatives” suck.  A “rugged manifesto” might just work.  It’s how agile kicked “CMMI”‘s ass.

The points about how pervasive software are now are well taken, including the guy with the mechanical arms who died in a car crash – software fault?  We’ll never know.  As we get more and more information systems embedded with/in us we have the real possibility of a “Ghost In The Shell” kind of world, and software security isn’t just about your credit card going missing but about your very real physical safety.

He threw in some other interesting tidbits that I noted down to look up later, including the ToorCon “Real Men Carry Pink Pagers” presentation about hacking the Girl Tech IM-Me toy into a weaponized attack tool, and some open source animated movie called Sintel.

It was a great start to the conference, raised some good questions for thought and I got a lot out of it.

LASCON 2010 was awesome.  It’s an Austin app security conference put on by the Austin OWASP chapter. Josh Sokol and James Wickett did a great job of putting the thing together; for a first time convention it was really well run and went very smoothly.  The place was just about full up, about 200 people.  I saw people I knew there from Austin Networking, the University of Texas, HomeAway, and more.  It was a great crowd, all sorts of really sharp people, both appsec pros and others.

And the swag was nice, got a good quality bugout bag and shirt, and the OWASP gear they were selling was high quality – no crappy black geek tshirts.

I wish I had more time to talk with the suppliers there; I did make a quick run in to talk to Fortify and Veracode.  Both now have SaaS offerings where you can buy in for upload scanning of your source (Fortify) or your binaries (Veracode) without having to spring for their big ass $100k software packages, which is great – if proper security is only the purview of billion dollar companies, then we’ll never be secure.

At the happy hour they brought in a mechanical bull!  We had some friends in from Cloudkick in SF and they asked me with some concern, “Do all conferences in Austin do this?”  Nope, first time I’ve seen it, but it was awesome!  After some of the free drinks, I was all about it.  They did something really clever with the drinks – two drink tickets free, but you could get more by going and talking to the vendors at their booths.  That’s a win-win!  No “fill out a grade school passport to get entered into a drawing” kind of crap.

Speaking of drawings, they had a lot of volunteers working hard to run the con, they did a great job.

I took notes from the presentations I went to, they’re coming as separate posts.  I detected a couple common threads I found very interesting.  The Rugged Software Manifesto was mentioned by speakers in multiple sessions including by the Department of Homeland Security.  It’s clear that as software becomes more and more pervasive in our lives that health, safety, national security, and corporate livelihood are all coming to depend on solid, secure software and frankly we’re not well on the right track towards that happening.

Also, the need for closer cooperation between developers, appsec people, and traditional netsec people was a clear call to action.  This makes me think about the ongoing call for developer/ops collaboration from DevOps – truly, it’s a symptom of a larger need to find a better way for everyone to work together to generate these lovely computerized monstrosities we work on.

So check out my notes from the sessions – believe me, if it was boring I wouldn’t bother to write it down.

I hear the conference turned a profit and it was a big success from my point of view, so here’s hoping it’s even bigger and better in 2011!  Two days!  It’s calling to you!