h1

Praise for CCDC

January 28, 2013

Over on r/netsec there’s a discussion debating the merits and realism of the Collegiate Cyber Defense Competition.

I’ve volunteered at the North East Collegiate Cyber Defense competition since 2008. I’ve also participated with several CCDC regional events since 2010 and I was on the National CCDC red team last year. I’ve seen more of CCDC than most. I believe in it as an event or else I wouldn’t put so much time into it.

CCDC is a cyber defense competition league. Student teams qualify to participate in a regional event. The winners of these regional events move on to a national event with the winner taking bragging rights.

Each region organizes itself. Some regions mirror the National CCDC rules very closely. Others, do not. Right now, regional events are not happening. They do not start until March. The reflection happening on reddit is about two events that happened over the weekend: a qualifier event and a practice event organized by students at Capitol College.

The qualifier events are simple filters to invite the most prepared teams to the regional event. They’re usually throw aways and many times the qualifiers are no reflection of the rules or organization of the regional.

As for the student run practice events, these are a clue that something special is happening. Student run practice events means students organized an event, reached out to their professional security community, invited people in, and they asked for a lesson.

Why does this matter? We’ll get to that in a moment…

There are many opinions on CCDC’s rules, restrictions, and artificialities. As someone who participates, I see the rules shift year-to-year to make a more engaging game. No organizer wants students sitting bored or idle throughout the event. Everyone’s hair should be on fire.

No two students take the same thing out of CCDC. The team captains have to deal with people issues. They have to motivate their fellow students to take time away from video games and parties to sit down and drill through checklists.

The teams that win assign roles. They have to. One smart person doing everything won’t win a CCDC event for you. Too much is happening. Some students will become Cisco IOS whizzes. Others will learn how to administer and perform intrusion response on UNIX systems. Some Windows. Students from the winning teams will understand the value of staging a secure configuration and migrating production stuff to it with minimal downtime.

The red vs. blue battle aside, students must also write policy and effectively communicate with judges, who act as executive leadership. This is a big part of their score. There’s a lot that happens in a CCDC weekend event.

The teams that will win their regional events are probably spending 10-30 hours each week, practicing as a team, right now.

The success of CCDC isn’t in its rules or how closely it mirrors sitting on a NOC floor for 12 hours with nothing happening. The success of CCDC is in what it motivates the students to do, on their own time, to prepare themselves to enter our field as peers. Last weekend’s student run practice event is an example of this.

Since 2008, I’ve seen the student teams get better by leaps and bounds. I’ve never been part of a regional red team that had access on all teams at the end of an event. Please don’t let the chest thumping of some red team volunteers lead you to believe that students are lost and engaged in an unfair game. Most student teams are well prepared and I’m in awe of them each year.

I run into these students at conferences. We have a good laugh about CCDC. For them and for me as a volunteer, it’s one of the high points of our year.

CCDC works. Students learn leadership, teamwork, and they’re motivated to pick up skills. CCDC is a good thing for our professional community

9 comments

  1. Hey Raphi, I’ve been involved in NECCDC since 2008 as either a student or a white team member and it doesn’t sound like you’re describing the same competition to me. Students are explicitly not allowed to prepare for the competition by writing scripts or preparing tools ahead of time. Internet access is only allowed through a VNC console that blocks access to certain websites (sometimes unintentionally like this year when microsoft.com was blocked for more than 1/4 of the competition and no patches or security tools could be downloaded from it) and hampers your ability to react quickly to changing conditions. There are limits on the rate at which you can change passwords to systems that you’re in charge of maintaining (only once per hour) and all such changes need to be made via PDF requests e-mailed to the organizers. As a white team member, when I asked to get access to the internal scoreboard so I could see statistics from the competition I was told “The scoreboard is too insecure and giving you access would allow you to take down the competition. No remote white team members can have access to the scoreboard.” The irony! I asked because scoring for the competition is so opaque that no team ever knows how well they’re doing until the competition is over. Until then, you are at the mercy of a finicky scoring bot that the organizers don’t document in any way, resulting in services you think are up being scored as down without recourse. After the competition, there are little to no methods for self-examination for students who want to improve. Unlike a traditional CTF, there are no solution writeups to be found and teams rarely, if ever, discover why they lost points, how they were hacked, or learn what they did that worked or did not work, vastly limiting opportunities for learning. Rather, the most positive outcomes I have heard from CCDC are from red team members like yourself since they can see the entire competition as it unfolds, can count the number of shells they have, and can use this awareness to improve your abilities in following years.

    The concept of CCDC has merit but the current implementation of it, at least in NECCDC, limits opportunities for students to learn and grow. I’ve recommended that our university de-prioritize anything related to CCDC until how it runs changes in such a way that our students can gain something out of it besides a beating. I think it’s possible to address the issues I pointed out above but I don’t see any interest or momentum from the organizers to do so.


    • Dan, I think what you say has some truth in it especially with some of the technical issues I have run into while competing. I’ve noticed things such as the scoreboard not being completely live and up to date or reporting differently than what is being seen by the judges. My first year in CCDC was as a freshman in college and I really didn’t have a great understanding of a lot of concepts in security. I am currently in a student organization called The 49th Security Division and CC-DC is something that we all look forward to each year. My first trip to SECCDC I knew almost nothing and was no good for much more than paperwork. Since then, I’ve learned the competition and the set up…CCDC has given me a reason to more intensely dive into setting up IDS and hardening servers/applications.

      One big step that I feel CCDC has taken to address your concerns of not letting students train is that they have started the National Cyber League competition in the fall. These CTF type challenges go over things such as reviewing system logs to find attackers, basic web pentesting, simple forensics, and a few other key topics. While some of the challenges would seem very easy to someone who has been doing this stuff awhile, for a beginner there is nothing like it. I know for one that I can’t learn the things I get from CCDC or the National Cyber League through my institution. I think that as the NCL starts to develop itself more, the lessons taught in these competitions will become the off-season training for CCDC. They also had VERY thorough write-ups for the challenges.

      Nationals last year was only two days of competitions followed by a third day talks and awards. In my opinion, this needs to be adjusted. It was enough time for us to get beaten up and owned and then the competition ended. At SECCDC we go for three days and the end of the second/third day is when we start to learn how to fix the holes we were getting owned through.

      I know that CCDC hasn’t been perfected but there is no other experience like it where students can get some hands on time against a professional Red Team and get the exposure to recruiters. To us security nerds it’s like our super bowl and it’s not something we take for granted. We look forward to seeing how the competition grows and how the students develop.

      PS. Dan, many thanks to your videos on Reverse Eng. Myself and many from The 49th Security Division have seen them and they are awesome.


      • It sounds like there is a ton of variability between regions from CCDC. I’ve never had a positive experience with the NorthEast one.

        btw, I’m glad you’re finding those videos useful! I’m planning on relaunching that website in a few months. Keep an eye on it!


    • Dan, I think we’re going to have to agree to disagree. We could debate the mechanics of a regional event all day. I don’t feel this is very productive. I’m sorry to hear that scoring and networking issues in one region tainted you so much against the competition as a whole. Each region is different in polish and execution.

      The most important part of the event is the students. CCDC motivates them. I see it on Twitter, I hear it when I talk to them in person, and I experience it when I play red against them. The effort put in by the students is the output of CCDC. Even if volunteers like myself fall short, the event is an amazing success so long as students continue to put in so much effort to come together as a team and better themselves.


      • If the competition could be modified in such a way that lets students examine their performance, compare it to others, and plan out a path forward then I would feel much more positive about CCDC. Like I said, I think this type of competition has merit but I’m not happy at all with the current way that it’s implemented. Here are several changes to CCDC that I think would make it a much more effective teaching tool (essentially the reverse of my rant above):

        1) Red teams should be required to document the systems they’ve compromised, how, and when and provide this to the student teams after the event for them to learn from.

        2) Similarly, organizers should post various ways in which the student computers were pre-made to be insecure after the competition ends. I want to be able to re-create a play-by-play of what happened to learn from.

        3) User functionality requirements should be published for each service and used as the scoring system. Teams should explicitly understand why one of their services are marked down.

        4) The nonsense with password change requests and writing user acceptance policies during the competition is not teaching anyone anything and should be eliminated. These type of injects are tangential to the competition and hardly relate to anything else that goes on during it.

        5) Arbitrary restrictions on preparation and internet access during the competition should be eliminated. This limits the ability of student teams to plan and prepare. Teams should be able to do anything they want while maintaining the require user functionality.

        6) The scoreboard should provide detailed statistics and information about not only your team performance but the performance of your competitors. This increases competition and decreases the isolation that many student teams feel when competing.

        I think each of these represents substantial issues with the way they are implemented now and that the above changes would reflect positively on the knowledge gained by playing and make it more fun overall.


      • Honestly I have to agree with Dan Guido to a degree. Don’t get me wrong I love the CCDC and the experience I get from it but honestly some of my newer team members walked away feeling disappointed. They wanted to know what we placed, what we did wrong, or right, etc. But when we asked for our placement we got a resounding “No only the first and second place teams get to know”. To them it was a smack in the face to feel like with all their time and effort invested that they didn’t even get to know the net result of it. I believe a improved response and information system should be implemented.

        Unless you really know what the root cause was for losing points here and there(how or where you got hacked) and can determine where it all originated from then you walk away knowing none of your mistakes. The scoring machine was having problems for most the day we competed and so we didn’t know when a service was up or if something was wrong, along with this the red team had gotten ahold of most our AD users, but didn’t do any damaging hacks so we chalked the scoring engines instability on service scores as being only the scoring engine. We wern’t aware until after the competition that the AD databse might have been compromised.

        My teamates loved the experience of the competition but feel somewhat like their time was waited because they have no definitive answer on how good/ or bad they did. I would propose most of the same changes as Dan because honestly I have competed two years now and even I felt like I waisted my time to a degree because I had no idea what we placed or even how we did compared to Rose Holmen. Most our services stayed up for the whole competition but not knowing how we could have improved or where we failed was a major letdown. I figured out the AD hack from my own investigation towards the end of the competition but honestly if I hadn’t figured that out we would have left not having any idea as to where we failed.

        I don’t see what would be so terrible about at the end of the competition listing who placed what and heck even a set of scores, how do we know how much more to improve ourselves before the next years competition unless we know how were doing in the first place….

        I am not saying the CCDC isn’t important all I am saying is that it needs some major overhaul on the information the teams get back(we would like to know what we place and a score to know if we improve or not each year, aswell as maybe a basic hint at how/where we failed), the scoring engine(needs major performance upgrades and more realtime listing of services) I think the scoring engine should inform you in very basic detail on why a service appears down(such as login failure), and access to the Netlabs+ network for “Security+ Excersises” needs to be for more than just a few months before the competition. Those excersises had great information in them and got our team more used to the Netlabs+ interface.

        My school/team will continue to compete in the CCDC but those changes I listed above would make it much more productive from both a learning perspective and a competition perspective. Every other type of competition in sports, olympics, etc lists how well you did, what your score is and your placement. Why can’t the CCDC be the same so people know if they improve or not each year?


      • Now that this CCDC season is kicking off, I am already starting to see some terrible changes to the way the competition works. SECCDC has a forced proxy and they only allow each team 20 URL’s to have open. This is a terrible, terrible, terrible idea. a good fourth or more of the list is dedicated to update sites. Add a couple AV sites, wireshark, putty, and a few other basic tools and thats all you can really use going into the competition. Moves like this make training for the competition feel useless. We spend hours and hours of our weeks searching for open source tools, learning how to configure them, sharpening our skills, and now a majority of it has gone to waste. I have to say that this is a low move on their part.


  2. Coming from someone who competes in CCDC and has used Armitage for a plethora of things, I’d like to say thanks for all your work.


    • Thanks Max. These comments are *always* appreciated. 🙂



Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s