When discussing DevOps processes such as “continuous delivery” and associating them with “security standards” this normally springs to mind analogies like Leonardo Di Caprio and an Oscar win, or Jon Snow and knowledge of everything, or for those well-read individuals Einstein's theory of gravity and quantum mechanics, or to the everyday person square pegs in round holes. It is two ideologies that are deemed not to mesh or an anti-pattern even. But let’s investigate the reasons for this perception, why do people think this concept doesn't mesh and why do people deem it something that won't work?...
So what is the perception of security teams in the DevOps community? Ironically,the one thing development and operations staff will tend to agree on is security teams, if nothing else, and how irritating they are, with analogies of a mosquito that won’t go away often perpetuated. When discussing security teams with IT and playing the word association game, some of the common words you will hear will be “bureaucratic”, “blockers”, “process monkeys” or just plain “annoying”. This summation needs to be taken with a pinch of salt like any stereotype, but is this an unfair assessment, does everyone really need to hate security teams? Security teams are surely no different from other teams and a security team can be as poorly run and managed as the next team. Tin hat on, there are good security teams and security practitioners out there, I have met them and worked with them at Betfair and other companies I have worked for. I don’t believe for a minute anyone comes to work every morning to do a bad job, or set out to annoy people by setting meaningless tasks. The security team have different priorities such as passing audits, ISO accreditation's, detecting vulnerabilities, preventing loss of data to name a few and most importantly keeping the company in business by proving the business is complying with the necessary rules and regulations. It is a very important role and not an easy one given the security teams priorities sometimes don't match the other teams priorities.
So if we flip the coin, how are DevOps practitioners viewed from a security teams perspective, when they implement processes such as continuous delivery? Security staff generally look upon IT staff with suspicion, as they believe they don’t give a damn about security, and do little to give them any visibility of what is going on and believe they are hiding what they are doing. Typically security tooling is bolted onto existing processes as an afterthought, in an attempt to try and give the security team the information they need to do their job and they often have to beg for information to gain visibility of what is going on, which eats up IT peoples time. Using common word association the common words associated with developers and operations teams by security staff will be “cowboys”, “unhelpful”, “uncaring”, “naive” or just “idiots” when talking in the context of appreciating security requirements. The issue is that IT staff have different priorities from the security team, they want to develop a quality product to market as quickly as possible, maintain up-time of the platform and make changes in a repeatable, reproducible way, and it is a general consensus that they don’t have time for a multitude of meetings around security.
Are continuous delivery and deployment something that cannot sit happily alongside Security? Let’s take a step back and analyse what continuous delivery and deployment set out to do. The main reason for these processes are that they promote IT staff to develop processes that are repeatable and reproducible, which in turns allows products to be delivered to market faster. Is this not exactly what security teams want, consistent processes that can be measured and are transparent and visible so that they can audit processes and make improvements to that process continually?
If we re-trace our steps as to why a DevOps philosophy was originally adopted the sole reason was not tooling, automation or anything else, it was in fact allowing developers and operations staff to collaborate daily and remove the “chucking it over the fence” mentality from daily routines. It was viewed that working in silos was no longer productive and engagement earlier on in the development life-cycle was beneficial for all parties. So by involving Operations staff in scrum meetings and properly assigning work rather than having a reactive model, this allowed operations staff to know what changes were coming, adequately plan any necessary infrastructure changes that were required, rather than finding out at the last minute and having a massive fight and associated finger pointing exercise which delayed the product reaching customers. It also worked exactly the same way with developers and QA testers, involving test teams in scrum teams to start writing tests while developers were writing code earlier in the development life-cycle, allowed testers to not have code “chucked over the fence” that they had no idea how to test. This cut out delays incurred by developers having to spend valuable time explaining to a tester what the feature was meant to do and how to test it and a tester explaining to a developer how something actually needed to be tested. Up front discussions and collaboration between teams helped solve these issues, instead teams could plan features and necessary testing as part of one scrum team. So to mitigate having issues in all these areas, we have learned that if we involve the stakeholders in sprint planning sessions then we have happier staff and more joined up processes.
So today we still have two silos which are the IT and security team, so how can we solve this issue? The solution is already there and has been proven to work. That solution is early engagement and collaboration which could be as simple as inviting your security team to be part of your scrum team or attending stand-ups so teams can build processes with security in mind from the outset, thus collaborating with security practitioners earlier on in the development cycle. This keeps everyone informed and happy, allowing teams to appreciate each others goals. Collaborative projects could be created that even go as far as including vulnerability detection or security steps in a quality gate on deployment pipelines and sharing knowledge on how processes could be automated. The possibilities are endless to improve this relationship and help security staff meet their goals and the winner will be the business.
This is why I am proud to be a part of a new conference in London called DevSecCon, which aims to promote the DevOps and SecOps collaboration. This is a fairly niche conference, that hasn’t been done before, so it will ensure some very interesting talks. At the DevSecCon conference we will discuss topics such as integrating vulnerability scanning into your continuous delivery process and adopting a mind-set of inclusiveness and collaboration with regards security and DevOps processes. The mission statement is that in a continuous delivery process teams should have nothing to hide, so there is no reason not to include security and bring them into the circle of trust. So if you are not familiar with this topic, I urge you to come along or even take part: http://www.devseccon.com/
As well as the sponsorship from my company Betfair and the security sponsorship from Qualys and MWR, we have some really cutting edge companies that have really embraced a DevOps mind-set and pushed forward automation in their respective fields.
At DevSecCon in the networking space, Nuage Networks will be presenting their software defined networking and how it allows secure and automated networking in private and public clouds using it's overlay technology, which allows both DevOps practitioners and security teams peace of mind, that network changes can be rapidly changed without compromising security standards. Arista will also be talking about how they have automated their top of rack switches, which can be racked and cabled in the datacenter and then set-up using a fully automated process. Both companies are really setting an example of how to change perception of their disciplines, when just a few years ago you would have shuddered having networking and hardware vendors presenting at an event associated with promoting DevOps processes. So things can change and one day we will look back and think about the time when security processes weren’t seamlessly integrated into the continuous delivery processes and laugh or potentially cry at the stupidity of it and all the time we wasted. But today could be the day we change things for the better, it’s too important for your business to wait and play catch up, so instead of fighting security we should be embracing it, as DevOps practitioners should really have nothing to hide. So let's help push through the next evolution in DevOps which has security integrated from the outset, let's not allow it to remain an after thought.