Fintechs and Security – Part One

  • Prologue – covers the overall challenge at a high level
  • Part One – Recruiting and Interviews
  • Part Two – Threat and Vulnerability Management – Application Security
  • Part Three – Threat and Vulnerability Management – Other Layers
  • Part Four – Logging
  • Part Five – Cryptography and Key Management, and Identity Management
  • Part Six – Trust (network controls, such as firewalls and proxies), and Resilience

Recruiting and Interviews

In the prologue of this four-stage process, I set the scene for what may come to pass in my attempt to relate my experiences with fintechs, based on what i am hearing on the street and what i’ve seen myself. In this next instalment, i look at how fintechs are approaching the hiring conundrum when it comes to hiring security specialists, and how, based on typical requirements, things could maybe be improved.

The most common fintech setup is one of public-cloud (AWS, Azure, GCP, etc), They’re developing, or have developed, software for deployment in cloud, with a mobile/web front end. They use devops tools to deploy code, manage and scale (e.g. Kubernetes), collaborate (Git variants) and manage infrastructure (Ansible, Terraform, etc), perhaps they do some SAST. Sometimes they even have different Virtual Private Clouds (VPCs) for different levels of code maturity, one for testing, and one for management. And third party connections with APIs are not uncommon.

Common Pitfalls

  • Fintechs adopt the stance: “we don’t need outside help because we have hipsters. They use acronyms and seem quite confident, and they’re telling me they can handle it”. While not impossible that this can work – its unlikely that a few devops peeps can give a fintech the help they need – this will become apparent later.
  • Using devops staff to interview security engineers. More on this problem later.
  • Testing security engineers with a list of pre-prepared questions. This is unlikely to not end in tears for the fintech. Security is too wide and deep an area for this approach. Fintechs will be rejecting a lot of good candidates by doing this. Just have a chat! For example, ask the candidate their opinions on the usefulness of VA scanners. The length of the response is as important as its technical accuracy. A long response gives an indication of passion for the field.
  • Getting on the security bandwagon too late (such as when you’re already in production!) you are looking at two choices – engage an experienced security hand and ignore their advice, or do not ignore their advice and face downtime, and massive disruption. Most will choose the first option and run the project at massive business risk.

The Security Challenge

Infosec is important, just as checking to see if cars are approaching before crossing the road is important. And the complexity of infosec mandates architecture. Civil engineering projects use architecture. There’s a good reason for that – which doesn’t need elaborating on.

Collapsing buildingWhenever you are trying to build something complex with lots of moving parts, architecture is used to reduce the problem down to a manageable size, and help to build good practices in risk management. The end goal is protective monitoring of an infrastructure that is built with requirements for meeting both risk and compliance challenges.

Because of the complexity of the challenge, it’s good to split the challenge into manageable parts. This doesn’t require talking endlessly about frameworks such as SABSA. But the following six capabilities (people, process, technology) approach is sleek and low-footprint enough for fintechs:

  • Threat and Vulnerability Management (TVM)
  • Logging – not “telemetry” or Threat intelligence, or threat hunting. Just logging. Not even necessarily SIEM.
  • Cryptography and Key Management
  • Identity Management
  • Business Continuity Management
  • Trust (network segmentation, firewalls, proxies).

I will cover these 6 areas in the next two articles, in more detail.

The above mentioned capabilities have an engineering and architecture component and cover very briefly the roles of security engineers and architects. A SABSA based approach without the SABSA theory can work. So an architect takes into account risk (maybe with a threat modelling approach) and compliance goals in a High Level Design (HLD), and generates requirements for the Low Level Design (LLD), which will be compiled by a security engineer. The LLD gives a breakdown of security controls to meet the requirements of the HLD, and how to configure the controls.

Security Engineers and Devops Tools

What happens when a devops peep interviews a security peep? Well – they only have their frame of reference to go by. They will of course ask questions about devops tools. How useful is this approach? Not very. Is this is good test of a security engineer? Based on the security requirements for fintechs, the answer is clear.

Security engineers can use devops tools, and they do, and it doesn’t take a 2 week training course to learn Ansible. There is no great mystery in Kubernetes. If you hire a security engineer with the right background (see the previous post in this series) they will adapt easily. The word on the street is that Terraform config isn’t the greatest mystery in the world and as long as you know Linux, and can understand what the purpose of the tool is (how it fits in, what is the expected result), the time taken to get productive is one day or less.

The point is: if i’m a security engineer and i need to, for example, setup a cloud SIEM collector: some fintechs will use one Infrastructure As Code (IaC) tool, others use another one – one will use Chef, another Ansible, and there are other permutations. Is a lack of familiarity with the tool a barrier to progress? No. So why would you test a security engineer’s suitability for a fintech role by asking questions about e.g. stanzas in Ansible config? You need to ask them questions about the six capabilities I mentioned above – i.e. security questions for a security professional.

Security Engineers and Clouds

Again – what was the transition period from on-premise to cloud? Lets take an example – I know how networking works on-premise. How does it work in cloud? There is this thing called a firewall on-premise. In Azure it’s called a Network Security Group. In AWS its called a …drum roll…firewall. In Google Cloud its called a …firewall. From the web-based portal UI for admin, these appear to filter by source and destination addresses and services, just like an actual non-virtual firewall. They can also filter by service account (GCP), or VM tag.

There is another thing called VPN. And another thing called a Virtual Router. On the world of on-premise, a VPN is a …VPN. A virtual router is a…router. There might be a connection here!

Cloud Service Providers (CSP) in general don’t re-write IT from the ground up. They still use TCP/IP. They host virtual machines (VM) instead of real machines, but as VMs have operating systems which security engineers (with the right background) are familiar with, where is the complication here?

The areas that are quite new compared to anything on-premise are areas where the CSP has provided some technology for a security capability such as SIEM, secrets management, or Identity Management. But these are usually sub-standard for the purpose they were designed for – this is deliberate – the CSPs want to work with Commercial Off The Shelf (COTS) vendors such as Splunk and Qualys, who will provide a IaaS or SaaS solution.

There is also the subject of different clouds. I see some organisations being fussy about this, e.g. a security engineer who worked a lot with Azure but not AWS, is not suitable for a fintech that uses AWS. Apparently. Well, given that the transition from on-premise to cloud was relatively painless, how painful is it to transition from Azure to AWS or …? I was on a project last summer where the fintech used Google Cloud Platform. It was my first date with GCP but I had worked with AWS and Azure before. Was it a problem? No. Do i have an IQ of 160? Hell no!

The Wrap-up

Problems we see in fintech infosec hiring represent what is most likely a lack of understanding of how they can best manage risk with a budget that is considerably less than a large MNC for example. But in security we haven’t been particularly helpful for fintechs – the problem is on us.

The security challenge for fintechs is not just about SAST/DAST of their code. The challenge is wider and be represented as six security capabilities that need to be designed with an architecture and engineering view. This sounds expensive, but its a one-off design process that can be covered in a few weeks. The on-going security challenge, whereby capabilities are pushed through into the final security operations stage, can be realised with one or two security engineers.

The lack of understanding of requirements in security leads to some poor hiring practices, the most common of which is to interview a security engineer with a devops guru. The fintech will be rejecting lots of good security engineers with this approach.

In so many ways, the growth of small to medium development houses has exposed the weaknesses in the infosec sector more than they were ever exposed with large organisations. The lack of the sector’s ability to help fintechs exposes a fundamental lack of skilled personnel, more particularly at the strategic/advisory level than others.

On Hiring For DevSecOps

Based on personal experience, and second hand reports, there’s still some confusion out there that results in lots of wasted time for job seekers, hiring organisations, and recruitment agents.

There is a want or a need to blame recruiters for any hiring difficulties, but we need to stop that. There are some who try to do the right thing but are limited by a lack of any sector experience. Others have been inspired by Wolf Of Wall Street while trying to sound like Simon Cowell.

It’s on the hiring organisation? Well, it is, but let’s take responsibility for the problem as a sector for a change. Infosec likes to shift responsibility and not take ownership of the problem. We blame CEOs, users, vendors, recruiters, dogs, cats, “Russia“, “China” – anyone but ourselves. Could it be we failed as a sector to raise awareness, both internally and externally?

So What Are Common Understandings Of Security Roles?

After 25 years+ we still don’t have universally accepted role descriptions, but at least we can say that some patterns are emerging. Security roles involve looking at risk holistically, and sometimes advising on how to deal with risk:

  • Security Engineers assess risk and design and sometimes also implement controls. BTW some sectors, legal in particular, still struggle with this. Someone who installs security products is in an IT ops role. Someone who upgrades and maintains a firewall is an IT ops role. The fact that a firewall is a security control doesn’t make this a security engineering function.
  • Security Architects take risk and compliance goals into account when they formulate requirements for engineers.
  • Security Analysts are usually level 2 SOC analysts, who make risk assessments in response to an alert or vulnerability, and act accordingly.

This subject evokes as much emotion as CISSP. There are lots of opinions out there. We owe to ourselves to be objective. There are plenty of sources of information on these role definitions.

No Aspect Of Risk Assessment != Security. This is Devops.

If there is no aspect of risk involved with a role, you shouldn’t looking for a security professional. You are looking for DEVOPS peeps. Not security peeps.

If you want a resource to install and configure tools in cloud – that is DEVOPS. It is not Devsecops. It is not Security Engineering or Architecture. It is not Landscape Architecture or Accounting. It is not Professional Dog Walker. it is DEVOPS. And you should hire a DEVOPS person. If you want a resource to install and configure appsec tools for CI/CD – that is DEVOPS. If you want a resource to advise on or address findings from appsec tools, that is a Security Analyst in the first case, DEVSECOPS in the 2nd case. In the 2nd case you can hire a security bod with coding experience – they do exist.

Ok Then So What Does A DevSecOps Beast Look Like?

DevSecOps peeps have an attack mindset from their time served in appsec/pen testing, and are able to take on board the holistic view of risk across multiple technologies. They are also coders, and can easily adapt to and learn multiple different devops tools. This is not a role for newly graduated peeps.

Doing Security With Non-Security Professionals Is At Best Highly Expensive

Another important point: what usually happens because of the skills gap in infosec:

  • Cloud: devops fills the gap.
  • On-premise: Network Engineers fill the gap.

Why doesn’t this work? I’ve met lots of folk who wear the aforementioned badges. Lots of them understand what security controls are for. Lots of them understand what XSS is. But what none of them understand is risk. That only comes from having an attack mindset. The result will be overspend usually – every security control ever conceived by humans will be deployed, while also having an infrastructure that’s full of holes (e.g. default install IDS and WAF is generally fairly useless and comes with a high price tag).

Vulnerability assessment is heavily impacted by not engaging security peeps. Devops peeps can deploy code testing tools and interpret the output. But a lack of a holistic view or an attack mindset, will result in either no response to the vulnerability, or an excessive response. Basically, the Threat And Vulnerability Management capability is broken under these circumstances – a sadly very common scenario.

SIEM/Logging is heavily impacted – what will happen is either nothing (default logging – “we have Stackdriver, we’re ok”), or a SIEM tool will be provisioned which becomes a black hole for events and also budgets. All possible events are configured from every log source. Not so great. No custom use cases will be developed. The capability will cost zillions while also not alerting when something bad is going down.

Identity Management – is not deploying a ForgeRock (please know what you’re getting into with this – its a fork of Sun Microsystems/Oracle’s identity management show) or an Azure AD and that’s it, job done. If you just deploy this with no thought of the problem you’re trying to solve in identity management, you will be fired.

One of the classic risk problems that emerges when no security input is taken: “there is no personally identifiable information in development Virtual Private Clouds, so there is no need for security controls”. Well – intelligence vulnerability such as database schema – attackers love this. And don’t you want your code to be safe and available?

You see a pattern here. It’s all or nothing. Either of which ends up being very expensive or worse. But actually come to think of it, expensive is the goal in some cases. Hold that thought maybe.

A Final Word

So – if the word risk doesn’t appear anywhere in the job description, it is nothing to do with security. You are looking for devops peeps in this case. And – security is an important consideration for cloud migrations.

How To Break Into Information Security

I’ve been asked a few times recently, usually by operations folk, to give some advice about how to break into the security sector, so under much pain I decided to commit my thoughts on the subject to this web log post. I’ve commented on this subject before and more extensively in chapter 6 of Security De-engineering, but this version is more in line with the times (up to 2012 I was advising a wide pass-by trajectory of planet infosec) and it will be shorter – you have my word(s).

blog-image

First I’d just be wary about trying to get into security just because of financial reasons (David Froud has an excellent blog and one of his posts covered this point well). At the time of writing it is possible to get into the field just by having an IT background and a CISSP. But don’t do that unless you have what’s REALLY required (do not judge what is REALLY required for the field based on job descriptions – at the time of writing, there are still plenty of mistakes being made by organisations). Summarising this in a very brief way:

  • You feel like you have grown out of pure IT-based roles and sort of excelled in whatever IT field you were involved in. You’re the IT professional who doesn’t just clear their problem tickets and switch off. You are, for example, looking for ways to automate things, and self-teach around the subject.
  • Don’t think about getting into security straight from higher education. Whereas it is possible, don’t do it. Just…don’t. Operational Security (or opsec/devopssec) is an option but have some awareness of what this is (scroll down to the end for an explanation).
  • Flexibility: can jump freely from a Cisco switch to an Oracle Database on any Operating System. Taking an example: some IT folk are religious about Unix and experience a mental block when it comes to Windows – this doesn’t work for security. Others have some kind of aversion to Cloud, whereas a better mindset for the field is one that embraces the challenge. Security pros in the “engineer” box should be enthusiastic about the new opportunities for learning offered by extended use of YAML, choosing the ideal federated identity management solution, Puppet, Azure Powershell, and so on. [In theory] projects where on-premise applications are being migrated to Cloud are not [in theory] such a bad place to be in security [in theory].
  • You like coding. Maybe you did some Python or some other scripting. What i’ve noticed is that coding skills are more frequently being seen as requirements. In fact I heard that one organisation went as far as putting candidates through a programming test for a security role. Python, Ruby, Shell ([Li,U]nix) and Powershell are common requirements these days. But even if role descriptions don’t mention coding as a requirement – having these skills demonstrates the kind of flexibility and enthusiasm that go well with infosec. “Regex” comes up a lot but if you’ve done lots of Python/Ruby and/or Unix sed/awk you will be more than familiar with regular expressions.

There is a non-tech element to security (sometimes referred to as “GRC”) but this is something you can get into later. Being aware of international standards and checking to see what’s in a typical corporate security policy is a good idea, but don’t be under the impression that you need to be able to recite verses from these. Generally speaking “writing stuff” and communication is more of a requirement in security than other fields, but you don’t need to be polished at day zero. There are some who see the progression path as Security Analyst –> Security Consultant (Analyst who can communicate effectively).

Another common motivator is hacker conferences or Mr Robot. Infosec isn’t like that. Even the dark side – you see Elliott with a hoody writing code with electronic techno-beats in the background, but hackers don’t write code to compromise networks to any huge degree, if at all. All the code is written for them by others mostly. And as with the femtocell and Raspberry Pi incidents, they usually have to assume a physical presence on the inside, or they are an internal employee themselves, or they dupe someone on the inside of the organisation under attack. Even if you’re in a testing role on the light side, the tests are vastly restricted and there’s a very canned approach to the whole thing with performance KPIs based on reports or something else that doesn’t link to actual intellectual value. Its far from glamorous. There’s an awful lot of misunderstanding out there. What is spoken about at hacker confz is interesting but its not usually stuff that is required to prove the existence of vulnerability in a commercial penetration test – most networks are not particularly well defended, and very little attention is given to results, more so because in most cases the only concern is getting ticks in boxes for an audit – and the auditors are often 12 years old and have never seen a command shell. Quality is rarely a concern.

Its a good practice to build up a list of the more influential bloggers and build up a decent Twitter feed and check what’s happening daily, but also, here are the books that I found most useful in terms of starting out in the field:

  • TCP/IP Illustrated – there are 3 volumes. 1 and 2 are the most useful. Then…
  • Building Internet Firewalls – really a very good way to understand some of the bigger picture ideas behind network architecture design and data flows. I hear rolling of eyes from some sectors, but the same principles apply to Cloud and other “modern” ideas that are from the 90s. With Cloud you have less control over network aspects but network access control and trust relationships are still very much a concern.
  • Network Security Assessment – the earlier versions are also still pertinent unless you will never see a Secure Shell or SMB port (hint: you will).
  • Security Engineering – there’s a very good chapter on Cryptography and Key Management.
  • The Art of Software Security Assessment – whether or not you will be doing appsec for a living you should look at OWASP‘s site and check out Webgoat. They are reportedly looking to bolster their API security coverage, which is nice (a lot of APIs are full of the same holes that were plugged in public apps by the same orgs some years ago). But if you are planning on network penetration testing or application security as a day job, then read this book, its priceless and still very applicable today.
  • The Phoenix Project – a good background illustrative for gaining a better understanding of the landscape in devops.

Also – take a look at perhaps a Windows security standard from the range of CIS benchmarks.

Finally – as i alluded earlier – opsec is not security. Why do i say this? Because i did come across many who believe they made it as a security pro once they joined a SOC/NOC team and then switched off. Security is a holistic function that covers the entire organisation – not just its IT estate, but its people, management, availability and resilience concerns, and processes. As an example – you could be part of a SOC team analysing the alerts generated by a SIEM (BTW some of the best SIEM material online is that written by Dr Anton Chuvakin). This is a very product centric role. So what knowledge is required to architect a SIEM and design its correlation rules? This is security. The same applies to IDS. Responding to alerts and working with the product is opsec. Security is designing the rulebase on an internal node that feeds off a strategically placed network tap. You need to know how hackers work among other areas (see above). Security is a holistic function. A further example: opsec takes the alerts generated by vulnerability management enterprise suites and maybe does some base false positives testing. But how does the organisation respond effectively to a discovered vulnerability? This is security.

Addressing The Information Security Skills Gap

We are told there is a skills gap in information security. I agree – there is, but recent suggestions to address the gap take us to dangerous places that are great for recruitment agencies, but not so great for the business world.

I want to steer away from use of the phrase ‘skills’ in this article because its too micro and the phrase has been violated by modern hiring practices. We are not looking for ‘Websense’, ‘DLP’ skills or as i saw recently ‘HSM’ skills. These requirements are silly unless it is the plan for organisations to spend 10 to 50 times more than they need on human resource, and have a security team of 300. Its healthier for organisations to look at ‘habits’ or ‘backgrounds’, and along those lines, in information security we’re looking for the following:

  • At least 5 years in an IT discipline: sys admin, DBA, devops bod, programmer for example
  • Evidence of having excelled in those positions and sort of grown out of them
  • Flexibility: for example, the crusty Radagast BSD-derivative disciple who has no fundamentalist views of other operating systems (think ‘Windows’) and not only can happily work with something like Active Directory, but they actually love working with Active Directory
  • A good-to-have-but-not-critical is past evidence of breaking or making things, but this should seen as a nice bonus. In its own right, it is insufficient – recruiting from hacker confz is far from guaranteed to work – too much to cover here

So really it should be seen that a career in infosec is a sort of ‘graduation’ on from other IT vocations. There should be an entrance exam based on core technologies and penetration testing. The career progression path goes something like: Analyst (5 years) –> Consultant -> Architect/Manager. Managers and architects cannot be effective if they do not have a solid IT background. An architect who doesn’t know her way around a Cisco router, implement a new SIEM correlation rule, or who cannot run or interpret the output from a packet sniffer is not an architect.

Analysts and Consultants should be skilled with the core building blocks to the level of being confidently handed administrative access to production systems. As it is, security pros find it hard to even get read-only access to firewall management suites. And having fast access to information on firewall rules – it can be critical.

Some may believe that individuals fitting the above profile are hard to find, and they’d be right. However, with the aforementioned model, the workforce will change from lots of people with micro-skills or product-based pseudo skills, to fewer people who are just fast learners and whose core areas complement each other. If you consider that a team of 300 could be reduced to 6 – the game has changed beyond recognition.

Quoting a recent article: “The most in demand cyber security certifications were Security+, Ethical Hacking, Network+, CISSP, and A+. The most in demand skills were Ethical Hacking, Computer Forensics, CISSP, Malware Analysis, and Advanced Penetration Testing”. There are more problems with this to describe in a reasonable time frame but none of these should ever be called ‘skills’. Of these, Penetration Testing (leave out the ‘ethics’ qualifier because it adds a distasteful layer of judgment on top of the law) is the only one that should be called a specification in its own right.

And yes, Governance, Risk and Control (GRC) is an area that needs addressing, but this must be the role of the Information Security Manager. There is a connection between Information Security Manage-ment and Information Security Manage-er.  Some organisations have separate GRC functions, the UK public sector usually has dedicated “assurance” functions, and as i’ve seen with some law firms, they are separated from the rest of security and IT.  Decision making on risk acceptance or mitigation, and areas such as Information Classification, MUST have an IT input and this is the role of the Information Security Manager. There must be one holistic security team consisting of a few individuals and one Information Security Manager.

In security we should not be leaving the impression that one can leave higher education, take a course in forensics, get accreditation, and then go and get a job in forensics. This is not bridging the security skills gap – its adding security costs with scant return. If you know something about forensics (usually this will be seen as ‘Encase‘ by the uninitiated) but don’t even have the IT background, let alone the security background, you will not know where to look in an investigation, or have a picture of risk. You will not have an inkling of how systems are compromised or the macro-techniques used by malware authors. So you may know how to use Encase and take an integral disk image for example, but that will be the limit of your contribution. Doesn’t sound like a particularly rewarding way to spend 200 business days per year? You’d be right.

Sticking with the forensics theme: an Analyst with the right mindset can contribute effectively in an incident investigation from day one. There are some brief aspects of incident response for them to consider, but it is not advisable to view forensics/incident response as a deep area. We can call it a specification, just as an involuntary action such as breathing is a specification, but if we do, we are saying that it takes more than one person to change a light bulb.

Incident response from the organisational / Incident Response Plan (IRP) formation point of view is a one-day training course or a few hours of reading. The tech aspects are 99% not distinct from the core areas of IT and network security. This is not a specialisation.

Other areas such as DLP, Threat Intelligence, SIEM, Cryptography and Key Management – these can be easily adopted by the right security minds. And with regard security products – it should be seen that security professionals are picking up new tools on-the-fly and don’t need 2 week training courses that cost $4000. Some of the tools in the VM and proxy space are GUIs for older open source efforts such as Nessus, OpenVAS, and Squid with which they will be well-versed, and if they’re not, it will take an hour to pick up the essentials.

There’s been a lot of talk of Operating Systems (OS) thus far. Operating Systems are not ‘a thing from 1998’. Take an old idea that has been labelled ‘modern’ as an example: ok, lets go with ‘Cloud’. Clouds have operating systems. VMs deployed to clouds have operating systems. When we deploy a critical service to a cloud, we cannot ignore the OS even if its a PaaS deployment. So in security we need people who can view an OS in the same way that a hacker views an OS – we need to think about Kill Chains and local privilege elevations. The Threat and Vulnerability Management (TVM) challenge does not disappear just because you have PaaS’d everything. Moreover if you have PaaS’d everything, you have immediately lost the TVM battle. As Beaker famously said in his cloud presentation – “Platforms Bitches”. Popular OS like Windows, *nix, Linux, and popular applications such as Oracle Database are going to be around for some time yet and its the OS where the front-lines are drawn.

Also what is a common misconception and does not work: a secops/network engineer going straight into security with no evidence of interest in other areas. ‘Secops’ is not good preparation for a security career, mainly because secops is sort of purgatory. Just as “there is no Dana, only Zool“, so “there is no secops, only ops”. There is only a security element to these roles because the role covers operational processes with security products. That is anti-security.

All Analyst roles should have an element of penetration testing and appsec, and when I say penetration testing, i do mean unrestricted testing as in an actual simulation. That means no restrictions on exploit usage or source address – because attackers do not have such restrictions. Why spend on this type of testing if its not an actual simulation?

Usage of Cisco Discovery Protocol (CDP) offers a good example of how a lack of penetration testing experience can impede a security team. If security is being done even marginally professionally in an organisation, there will exist a security standard for Cisco network devices that mandates the disabling of CDP.  But once asked to disable CDP, network ops teams will want justification. Any experienced penetration tester knows the value of intelligence in expediting the attack effort and CDP is a relative gold mine of intelligence that is blasted multicast around networks. It can, and often does, reveal the identity and IP address of a core switch. But without the testing experience or knowledge of how attacks actually go down, the point will be lost, and the confidence missing from the advisory.

The points i’ve just covered are not actually ground-breaking at all. Analysts with a good core background of IT and network security can easily move into any new area that marketeers can dream up.

There is an intuition that Information Security has a connection with Information Technology, if only for the common word in them both (that was ‘Information’ by the way, in case you didn’t get it). However, as Upton Sinclair said “It is difficult to get a man to understand something, when his salary depends upon his not understanding it”.

And please don’t create specialisations for Big Data or Internet of Things…woops, too late.

So, consider a small team of enthusiastic, flexible, fast learners, rather than a large team of people who can be trained at a high cost to understand the UI of an application that was designed in the international language and to be intuitive and easy to learn.

Consider using one person to change a light bulb, and don’t be the butt of future jokes.

Information Security Pseudo-skills and the Power of 6

How many Security Analysts does it take to change a light bulb? The answer should be one but it seems organisations are insistent on spending huge amounts of money on armies of Analysts with very niche “skills”, as opposed to 6 (yes, 6!) Analysts with certain core skills groups whose abilities complement each other. Banks and telcos with 300 security professionals could reduce that number to 6.

Let me ask you something: is Symantec Control Compliance Suite (CCS) a “skill” or a product or both? Is Vulnerability Management a skill? Its certainly not a product. Is HP Tippingpoint IPS a product or a skill?

Is McAfee Vulnerability Manager 7.5 a skill whereas the previous version is another skill? So if a person has experience with 7.5, they are not qualified to apply for a shop where the previous version is used? Ok this is taking it to the extreme, but i dare say there have been cases where this analogy is applicable.

How long does it take a person to get “skilled up” with HP Arcsight SIEM? I was told by a respected professional who runs his own practice that the answer is 6 months. My immediate thought is not printable here. Sorry – 6 months is ridiculous.

So let me ask again, is Symantec CCS a skill? No – its a product. Its not a skill. If you take a person who has experience in operational/technical Vulnerability Management – you know, vulnerability assessment followed by the treatment of risk, then they will laugh at the idea that CCS is a skill. Its only a skill to someone who has never seen a command shell before, tested manually for a false positive, or taken part in an unrestricted manual network penetration test.

Being a software product from a major vendor means the GUI has been designed to make the software intuitive to use. I know that in vulnerability assessment, i need to supply the tool with IP addresses of targets and i need to tell the tool which tests I want to run against those targets. Maybe the area where I supply the addresses of targets is the tab which has “targets” written on it? And I don’t want to configure the same test every time I run it, maybe this “templates” tab might be able to help me? Do i need a $4000 2-week training course and a nice certificate to prove to the world that I can work effectively with such a product? Or should there be an effective accreditation program which certifies core competencies (including evidence of the ability to adapt fast to new tools) in security? I think the answer is clear.

A product such as a Vulnerability Management product is only a “Window” to a Vulnerability Management solution. Its only a GUI. It has been tailored to be intuitive to use. Its the thin layer on top of the Vulnerability Management solution. The solution itself is much bigger than this. The product only generates list of vulnerabilities. Its how the organisation treats those vulnerabilities that is key – and the product does not help too much with the bigger picture.

Historically vulnerability management has been around for years. Then came along commercial products, which basically just slapped a GUI on processes and practices that existed for 20 years+, after which the jobs market decided to call the product the solution. The product is the skill now, whereas its really vulnerability management that is the skill.

The ability to adapt fast to new tools is a skill in itself but it also is a skill that should be built-in by default: a skill that should be inherent with all professionals who enter the field. Flexibility is the key.

The real skills are those associated with areas for large volumes of intellectual capital. These are core technologies. Say a person has 5 years+ experience of working in Unix environments as a system administrator and has shown interest in scripting. Then they learn some aspects of network penetration testing and are also not afraid of other technologies (such as Windows). I can guarantee that they will be up and running in less than one day with any new Vulnerability Management tool, or SIEM tool, or [insert marketing buzzphrase here] that vendors can magic up.

Different SIEM tools use different terms and phrases for the same basic idea. HP uses “flex connectors” whilst Splunk talks about “Forwarders” and “Heavy Forwarders” and so on. But guess what? I understand English but If i don’t know what the words mean, i can check in an online dictionary. I know what a SIEM is designed to do and i get the data flows and architecture concept. Network aggregation of syslog and Windows Events is not an alien concept to me, and neither are all layers of the TCP/IP stack (a really basic requirement for all Analysts – or should be). Therefore i can adapt very easily to new tools in this space.

IPS/IDS and Firewalls? Well they’re not even very functional devices. If you have ever setup Snort or iptables you’ll be fine with whatever product is out there. Recently myself and another Consultant were asked to configure a Tippingpoint device. We were up and running in 10 minutes. There were a few small items that we needed to check against the product documentation. I have 15 years+ experience in the field but the other guy is new. Nonetheless he had configured another IPS product before. He was immediately up and running with the product – no problem. Of course what to configure in the rule base – that is a bigger story and it requires knowledge of threats, attack techniques and vulnerabilities – but that area is GENERIC to security – its not specific to a product.

I’ve seen some truly crazy job specifications. One i saw was Websense Specialist!! Come on – its a web proxy! Its Squid with extra cosmetic functions. The position would be filled by a Websense “Olympian” probably. But what sort of job is that? Carpe Diem my friends, Carpe Diem.

If you run a security consultancy and you follow the usual market game of micro-boxed, pigeon-holed security skills, i don’t know how you can survive. A requirement comes up for a project that involves a number of different products. Your existing consultants don’t have those products written anywhere on their CVs, so you go to market looking for contractors at 600 USD per day. You either find the people somehow, or you turn the project down.  Either way you lose out massively. Or – you could have a base of 6 (its that number again) consultants with core skills that complement each other.

If the over-specialisation issue were addressed, businesses would save considerably on human resource and also find it easier to attract the right people. Pigeon-holed jobs are boring. It is possible and advisable to acquire human resource able to cover more bases in risk management.

There are those for and against accreditation in security. I think there is a solution here which is covered in more detail of Chapter 11 of Security De-engineering.

So how many Security Analysts does it take to change a light bulb? The answer is 6, but typically in real life the number is the mark of the beast: 666.

Information Security Careers: The Merits Of Going In-house

Job hunting in information security can be a confusing game. The lack of any standard nomenclature across the sector doesn’t help in this regard. Some of the terms used to describe open positions can be interpreted in wildly different ways. “Architect” is a good example. This term can have a non-technical connotation with some, and a technical connotation with others.

There are plenty of pros who came into security, perhaps via the network penetration testing route, who only ever worked for consultancies that provide services, mainly for businesses such as banks and telcos. The majority of such “external” services are centered around network penetration testing and application testing.

I myself started out in infosec on the consultancy path. My colleagues were whiz kids and some were well known in the field. Some would call them “hackers”, others “ethical” or “white hat” network penetration testers. This article does not cover ethics or pander to some of the verdicts that tend to be passed outside of the law.

Many Analysts and Consultants will face the decision to go in-house at some point in their careers, or remain in a service provider capacity. Others may be in-house and considering the switch to a consultancy. This post hopefully can help the decision making process.

The idea of going in-house and, for example, taking up an Analyst position with a big bank – it usually doesn’t hold much appeal with external consultants. The idea prevails that this type of position is boring or unchallenging. I also had this viewpoint and it was largely derived from the few visions and sound bytes I had witnessed behind the veil. However, what I discovered when I took up an analyst position with a large logistics firm was that nothing could be further from the truth. Going in-house can benefit one’s career immensely and open the eyes to the real challenges in security.

Of course my experiences do not apply across the whole spectrum of in-house security positions. Some actually are boring for technically oriented folk. Different organisations do things in different ways. Some just use their security department for compliance purposes with no attention to detail. However there are also plenty that engage effectively with other teams such as IT operations and development project teams.

As an Analyst in a large, complex environment, the opportunity exists to learn a great deal more about security than one could as an external consultant.  An external consultant’s exposure to an organisation’s security challenges will only usually come in the form of a network or application assessment, and even if the testing is conducted thoroughly and over a period of weeks, the view will be extremely limited. The test report is sent to the client, and its a common assumption that all of the problems described in the report can be easily addressed. In the vast majority of cases, nothing could be further from the truth. What becomes apparent at a very early stage in one’s life as an in-house Analyst, is that very few vulnerabilities can be mitigated easily.

One of the main pillars of a security strategy is Vulnerability Management. The basis of any vulnerability management program is the security standard – the document that depicts how, from a security perspective, computer operating systems, DBMS, network devices, and so on, should be configured. So an Analyst will put together a list of configuration items and compose a security standard. Next they will meet with another team, usually IT operations, in an attempt to actually implement the standard in existing and future platforms. For many, this will be the point where they realize the real nature of the challenges.

Taking an example, the security department at a bank is attempting to introduce a Redhat Enterprise Linux security standard as a live document. How many of the configuration directives can be implemented across the board with an acceptable level of risk in terms of breaking applications or impacting the business in any way? The answer is “not many”. This will come as a surprise for many external consultants. Limiting factors can come from surprising sources. Enlightened IT ops and dev teams can open security’s eyes in this regard and help them to understand how the business really functions.

The whole process of vulnerability management, minus VM product marketeers’ diatribe, is basically detection, then deduce the risk, then take decisions on how to address the risk (i.e. don’t address the vulnerability and accept the risk, or address / work around the vulnerability and mitigate the risk). But as an external consultant, one will only usually get to hand a client a list of vulnerabilities and that will be the end of the story. As an in-house Security Analyst, one gets to take the process from start to finish and learn a great deal more in the process.

As a security consultant passing beyond the iron curtain, the best thing that can possibly happen to their careers is that they find themselves in a situation where they get to interface with the enlightened ones in IT operations, network operations (usually there are a few in net ops who really know their security quite well), and application architects (and that’s where it gets to be really fun).

For the Security Consultant who just metamorphosized into an in-house Analyst, it may well be the first time in their careers that they get to encounter real business concerns. IT operations teams live in fear of disrupting applications that generate huge revenues per minute. The fear will be palpable and it encourages the kind of professionalism that one may never have a direct need to have as an external consultant. Generally, the in-house Analyst gets to experience in detail how the business translates into applications and then into servers, databases, and data flows. Then the risks to information assets seem much more real.

The internal challenge versus the external challenge in security is of course one of protection versus breaking-in. Security is full of rock stars who break into badly defended customer networks and then advertise the feat from the roof tops. In between commercial tests and twittering school yard insults, the rock stars are preparing their next Black Hat speech with research into the latest exotic sploit technique that will never be used in a live test, because the target can easily be compromised with simple methods.

However the rock stars get all the attention and security is all about reversing and fuzzing so we hear. But the bigger challenge is not breaking in, its protection, but then protection is a lot less exotic and sexy than breaking in. So there lies the main disadvantage of going in-house. It could mean less attention for the gifted Analyst. But for many, this won’t be such an issue, because the internal job is much more challenging and interesting, and it also lights up a CV, especially if the names are those in banking and telecoms.

How about going full circle? How about 3 years with a service provider, then 5 years in-house, then going back to consulting? Such a consultant is indeed a powerful weapon for consultancies and adds a whole new dimension for service providers (and their portfolio of services can be expanded). In fact such a security professional would be well positioned to start their own consultancy at this stage.

So in conclusion: going in-house can be the best thing that a Security Consultant can do with their careers. Is going in-house less interesting? Not at all. Does it mean you will get less attention? You can still speak at conferences probably.

One Infosec Accreditation Program To Bind Them All

May 2013 saw a furious debate ensue after a post by Brian Honan (Is it time to professionalize information security?) that suggested that things need to be improved, which was followed by some comments to the effect that accreditation should be removed completely.

Well, a suggestion doesn’t really do it for me. A strong demand doesn’t really do it either, in fact we’re still some way short. No – to advocate the strength of current accreditation schemes is ludicrous. But to then say that we don’t need accreditations at all is completely barking mad.

Brian correctly pointed out “At the moment, there is not much that can be done to prevent anyone from claiming to be an information security expert.” Never a truer phrase was spoken.

Other industry sectors have professional accreditation and it works. The stakes are higher in areas such as Civil Engineering and Medicine? Well – if practitioners in those fields screw up, it cost lives. True, but how is this different from Infosec? Are the stakes really lower when we’re talking about our economic security? We have adversaries and that makes infosec different or more complex?

Infosec is complex – you can bet ISC2’s annual revenue on that. But doesn’t that make security even more deserving of some sort of accreditation scheme that works and generates trust?

I used the word “trust”, and I used it because that what’s we’re ultimately trying to achieve. Our customers are C-levels, other internal departments, end users, home users, and so on. At the moment they don’t trust infosec professionals and who can blame them? If we liken infosec to medicine, much of the time, forget about the treatment, we’re misdiagnosing. Infosec is still in the dark ages of drilling holes in heads in order to cure migraine.

That lack of trust is why, in so many organizations, security has been as marginalized as possible without actually being vaporized completely. Its also why security has been reduced down to the level of ticks in boxes, and “just pass the audit”.

Even though an organization has the best security pros in the world working for them, they can still have their crown jewels sucked out through their up-link in an unauthorized kinda way. Some could take this stance and advocate against accreditation because ultimately, the best real-world defenses can fail. However, nobody is pretending that the perfect, “say goodbye to warez – train your staff with us” security accreditation scheme can exist. But at the same time we do want to be able to configure detection and cover some critical areas of protection. To say that we don’t need training and/or accreditation in security is to say the world doesn’t need accreditation ever again. No more degrees and PhDs, no more CISSPs, and so on.

We certainly do need some level of proof of at least base level competence. There are some practices and positions taken by security professionals that are really quite deceptive and result in resources being committed in areas where there is 100% wastage. These poor results will emerge eventually. Maybe not tomorrow, but eventually the mess that was swept under the carpet will be discovered. We do need to eliminate these practices.

So what are we trying to achieve with accreditation? The link with IT needs to be re-emphasized. The full details of a proposal are covered in chapter 11 of Security De-engineering, but basically what we need first is to ensure the connection at the Analyst level with IT, mainly because of the information element of information technology and information security (did you notice the common word in IT and IT security? Its almost as though there might be a connection between them). 80% of information is now held in electronic form. So businesses need expertise to assist them with protection of that information.

Security is about both business and IT of course. Everybody knows this even if they can’t admit it. There is an ISMS element that is document and process based, which is critical in terms of future proofing the business and making security practices more resource-efficient. A baseline security standard is a critical document and cannot be left to gather dust on a shelf – it does need to be a “living” document. But the “M” in ISMS stands for Management, and as such its an area for…manage-ers. What is quite common is to find a security department of 6 or more Analysts who specialize in ISMS and audits. That does not work.

There has to be a connection with IT and probably the best way to ensure that is to advocate that a person cannot metamorphosize into a Security Analyst until they have 5 years served in IT operations/administration, network engineer, or as a DBA, or developer. Vendor certs such as those from IBM, Microsoft, Cisco – although heavily criticized they can serve to indicate some IT experience but the time-served element with a signed off testimonial from a referee is critical.

There can be an entrance exam for life as an Analyst. This exam should cover a number of different bases. Dave Shackleford’s assertion that creative thinkers are needed is hard to argue with. Indeed, what i think is needed is a demonstration of such creativity and some evidence of coding experience goes a long way towards this.

Flexibility is also critical. Typically IT ops folk cover one major core technology such as Unix or Windows or Cisco. Infosec needs people who can demonstrate flexibility and answer security questions in relation to two or more core technologies. As an Analyst, they can have a specialization with two major platforms plus an area such as application security, but a broad cross-technology base is critical. Between the members of a team, each one can have a specialization, but the members of the team have knowledge that compliments each other, and collectively the full spectrum of business security concerns can be covered.

There can be specializations but also proportional rewards for Analysts who can demonstrate competence in increasing numbers of areas of specialization. There is such a thing as a broad-base experienced Security Analyst and such a person is the best candidate for niche areas such as forensics, as opposed to a candidate who got a forensics cert, learned how to use Encase, plastered forensics on their CV, and got the job with no other Analyst experience (yes – it does happen).

So what emerges is a pattern for an approximate model of a “graduation”-based career path. And then from 5 years time-served as an Analyst, there can be another exam for graduation into the position of Security Manager or Architect. This exam could be something similar to the BCS’s CISMP or ISACA’s CISA (no – I do not have any affiliations with those organizations and I wasn’t paid to write this).

Nobody ever pretended that an accreditation program can solve all our problems, but we do need base assurances in order for our customers to trust us.

The Search For Infosec Minds

Since the early 2000s, and in some of my other posts, I have commented in different forms on the state of play, with a large degree of cynicism, which was greeted with cold reservation, smirks, grunts, and various other types of un-voiced displeasure, up to around 2009 or so. But since at least 2010, how things have changed.

If we fast forward from 2000 to 2005 or so, most business’s security function was reduced down to base parrot-fashion checklists, analysis and thinking were four letter words, and some businesses went as far as outsourcing security functions.

Many businesses who turned their backs on hackers just after the turn of the millennium have since found a need to review their strategies on security hiring. However 10 years is a long time. The personnel who were originally tasked with forming a security function in the late 90s, have since risen like phoenixes from the primordial chasm, and assisted by thermals, they have swooped up to graze on higher plains. Fast forward again to 2012, and the distance between security and IT is in the order of light years in most cases. The idea that security is purely a compliance game hasn’t changed, but unlike the previous decade, it is in many cases seen as no longer sufficient to crawl sloth-like over the compliance finishing line every year.

Businesses were getting hacked all through the 2000s but they weren’t aware of it. Things have changed now. For starters the attacks do seem to be more frequent and now there is SIEM, and audit requirements to aggregate logs. In the past, even default log settings were annulled with the result that there wasn’t even local logging, let alone network aggregation! Mind you, even after having been duped into buying every well-marketed detection product, businesses are still being hacked without knowing it. Quite often the incident comes to light after a botnet command and control system has been owned by the good guys.

Generally there is more nefarious activity now, as a result of many factors, and information security programs are under more “real” focus now (compliance-only is not real focus, in fact it’s not real anything, apart from a real pain the backside).

The problem is that with such a vast distance between IT and security for so long, there is utter confusion about how to get tech’d up. Some businesses are doing it by moving folk out of operations into security. This doesn’t work, and in my next post I will explain why it doesn’t work.

As an example of the sort of confusion that reigns, there was one case I came across earlier in 2012 where a company in the movies business was hacked and they were having their trailers, and in some cases actual movies, put up on various torrent sites for download. Their response was to re-trench their outsourced security function and attempt to hire in-house analysts (one or two!). But what did they go looking for? Because they had suffered from malware problems, they went looking for, and I quote, “Malware Reverse Engineers”. Malware Reverse Engineers? What did they mean by this? After some investigation, it turns out they are really were looking for malware reverse engineers, there was no misnomer – malware reverse engineers as in those who help to develop new patterns for anti-virus engines!! They had acquired a spanking new SIEM, but there was no focus on incident response capability, or prevention/protection at all.

As it turns out “reverse engineer[ing]” is now a buzzword. Whereas in the mid-2000s, buzzwords were “governance” and “identity management” (on the back of…”identity theft” – neat marketing scam), and so on. Now there are more tech-sounding buzzwords which have different connotations depending on who you ask. And these tech sounding buzzwords find their way into skills requirements sent out by HR, and therefore also on CVs as a response. And the tech-sounding buzzwords are born from…yes, you got it…Black Hat conferences, and the multitude of other conferences, B-sides, C-sides, F-sides and so on, that are now as numerous as the stars in the sky.

The segue into Black Hat was quite deliberate. A fairly predictable development is the on-going appearance of Infosec managers at Black Hats, who previously wouldn’t touch these events with a barge pole. They are popping up at these events looking to recruit speakers primarily, because presumably the speakers are among the sharper of the crayons in the box, even if nobody has any clue what they’re talking about.

Before I go on, I need to qualify that I am not going to cover ethics here, mainly because it’s not worth covering. I find the whole ethics brush to be somewhat judgmental and divisive. I prefer to let the law do the judgment.

Any attempt to recruit tech enthusiasts, or “hackers”, can’t be dismissed completely because it’s better than anything that could have been witnessed in 2005. But do businesses necessarily need to go looking for hackers? I think the answer is no. Hackers have a tendency to take security analysis under their patronage, but it has never been their show, and their show alone. Far from it.

In 2012 we can make a clear distinction between protection skills and breaking-in skills. This is because as of 2012, 99.99…[recurring to infinity]% of business networks are poorly defended. Therefore, what are “breaking-in skills”? So a “hacker” breaks into networks, compromises stuff, and posts it on pastebin.com. The hackers finds pride and confidence in such achievements. Next, she’s up on the stage at the next conference bleating about “reverse engineering”, “fuzzing”, or “anti forensics tool kits”…nobody is sure which language is used, but she’s been offered 10 jobs after only 5 minutes into her speech.

However, what is actually required to break into networks? Of the 20000+ paths which were wide open into the network, the hacker chose one of the many paths of least resistance. In most cases, there is no great genius involved here. The term “script kiddy” used to refer to those who port scan, then hunt for public declared exploits for services they find. There is IT literacy required for sure (often the exploits won’t run out-of-the-box, they need to be compiled for different OSs or de-bugged), but no creativity or cunning or …whatever other mythical qualities are associated with hacking in 2012.

The thought process behind hiring a hacker is typically one of “she knows how to break into my network, therefore she can defend against others trying to break in”, but its quite possible that nothing could be further from the truth. In 2012, being a hacker, or possessing “breaking-in skills”, doesn’t actually mean a great deal. Protection is a whole different game. Businesses should be more interested about protection as of 2012, and for at least the next decade.

But what does it take to protect? Protection is a more disciplined, comprehensive IT subject. Collectively, the in-house security teams needs to know the all the nooks and crannies, all the routers, databases, applications, clouds, and operating systems and how they all interact and how they’re all connected. They also collectively need to know the business importance of information assets and applications.

The key pillars of focus for new-hire Security Analysts should be Operating Systems and Applications. When we talk about operating systems and security, the image that comes to mind is of auditors going thru a checklist in some tedious box-ticking exercise. But OS security is more than that, and it’s the front line in the protection battle. The checklists are important (I mean checklists as in standards and policies) but there are two sides to each item on the checklist: one is in the details of how to practically exploit the vulnerability and the potential tech impact, the other are the operational/business impacts involved with the associated safeguard. In other words, OS security is far more than a check-list, box-ticking activity.

In 12 years I never met a “hacker” who could name more than 3 or 4 local privilege elevation vectors for any popular Operating System. They will know the details of the vulnerability they used to root a server last month, but perhaps not the other 100 or so that are covered off by the corporate security standard for that Operating System. So the protection skills don’t come by default just because someone has taken to the stand at a conference.

Skills such as “reverse engineering” and “fuzzing” – these are hard to attain and can be used to compromise systems that are well defended. But the reality is that very few systems are so well defended that such niche skills are ever needed. In 70+ tests for which i have either taken part or been witness, even if the tests were quite unrestricted, “fuzzing” wouldn’t be required to compromise targets – not even close.

A theoretical security team for a 10000+ node business, could be made of a half dozen or so Analysts, plus a Security Manager. Analysts can come from a background of 5 years in admin/ops or devs. To “break into” security, they already have their experience in a core technology (Unix, Windows, Oracle, Cisco etc), then they can demonstrate competence in one or more other core technologies (to demonstrate flexibility), programming/scripting, and security testing with those platforms.

Once qualified as a Security Analyst, the Analyst has a specialization in at least two core technologies. At least 2 analysts can cover application security, then there are other areas such as incident handling and forensics. As for Security Managers, once in possession of 5 years “time served” as an Analyst, they qualify for a manager’s exam, which when passed qualifies them for a role as a Security Manager. The Security Manager is the interface, or agent, between the technical artist Analysts and the business.

Overall then, it is far from the case that Hackers are not well-suited for vocational in-house security roles (moreover I always like to see “spare time” programming experience on a resume because it demonstrates enthusiasm and creativity). But it is also not the case that Analyst positions are under the sole patronage of Black Hat speakers. Hackers still need to demonstrate their capabilities in protection, and doing “grown-up” or “boring” things before being hired. There is no great compelling need for businesses to hire a hacker, although as of today, it could be that a hooligan who throws security stones through security windows is as close as they can get to effective network protection.

How To Break Into Security – Planet Earth Edition

The venerable Brian Krebs has recently been running some stories from various demigods of the infosec world, aimed at those wishing to enter the information security field – aspiring graduate ninjas, and others seeking the mythical pot of gold at the end of the rainbow.

First up there was there was Tomas Ptacek’s edition, then we had some pearls of wisdom from Bruce Schneier, Jeremiah Grossman, Richard Bejtlich, and then Charlie Miller.

Thomas Ptacek claimed about the security field: “It’s one of the few technology jobs where the most fun roles are well compensated”, and “if you watched “Sneakers” and ideated a life spent breaking or defending software, great news: infosec can be more fun in real life, and it’s fairly lucrative.” Well…I am not refuting any of this, but it is certainly quite unusual for jobs in information security to be fun.

Thomas talks of the benefits of having extensive programming experience – and this is something I advocate myself quite strenuously (more on that later). Thomas’s viewpoint was centered around appsec, which is fine. I think for myself, in terms of defending networks, we need to look at two main areas: appsec and operating systems / databases. There is some good advice in Thomas’s article about breaking into application security, although I wouldn’t say that this area is everything. There’s a little too much religious fervor about appsec in the article for my liking, just as one often sees a lack of balance in other areas, such as CISSP-worship, and malware “reverse engineering” – basically – “my area is the alpha and omega – all that was, is, and ever will be”.

There are other areas that matter in security other than application security. I wouldn’t say its all about appsec, but I would say though that the two main areas are appsec and operating system and database configuration. “But operating systems and databases are also applications” I hear you say. Yes, but when we’re talking appsec in infosec, we’re usually talking about web applications. There are few times when we suffer web attacks where that one single exploit leads to something really bad happening, apart from perhaps a SQLi that in itself reveals sensitive database hosted information.

With regard web applications I think Thomas is spot-on with his comments about learning about web application security assessment, and how to get clued up in this area. Also the comments about Nessus and getting into penetration testing – sad but true.

With regard Bruce Schneier’s “breaking into” edition – nothing he says is factually incorrect (most of what we talk about is subjective, neither black or white, but grey) but the comments are not at all close to the coal-face realities of most business’s in-house or service providers’ practices. Wannabe security pros reading this will be sure to get grandiose visions of their future lives as a security pro – but in 90%-plus of cases the vocational activities of security professionals do not match the picture painted by Bruce Schneier. I’ll explain more on this later.

Richard Bejtlich’s response was centered around getting into penetration testing with Metasploit and Jeremiah Grossman’s was the most all-encompassing and in my opinion, the response that had the most value for security pro wannabes – although Charlie Millar’s wasn’t far behind. In particular Mr Grossman plays down the effectiveness of accreditation programs, in favor of practical experience – wise words indeed. Charlie Millar had similar opinions.

In Charlie Millar’s response there was a lot of talk of really specialized nieche areas like reverse engineering and so on, but he does temper this with “I really do a lot of reverse engineering and binary analysis, which is unusual.” and “for those starting out, it probably makes more sense to learn some languages more useful for web applications, like PHP or Java or something.  The majority of jobs I come across in application security are web applications, so unless you’re a dinosaur like me, you probably want to become a web app expert.  Web application security is a lot easier to get started in as well.”

Charlie Millar’s response sort of segues me into the wider scope here, and that is of the realities “out there”. The articles are based on responses by folk who’ve rightly become esteemed professionals in their field and there is some really valuable insight there. The thing is – there is a lot of talk of the security field being a place for artists and magicians, and of being technically demanding, but there are very few places where technical acrobatics skills in security are seen as having any value to businesses, or even security line managers for that matter – and therefore such intellectual capital just does not appear on the balance books of these businesses.

We’ve been through a vicious 70 to 80% of a sine wave of pain in security since the late 90s. The security world painted by the fellowship of five assembled by Brian Krebs (speaking of whom, it would be nice to hear his version of the “breaking into” story) seems much more like the world of the late 90s than today. Security was heavily de-engineered through the 2000s. Things started to change around 2010, but we’re still very much in non-tech territory, and many of the security line managers who will interview prospective Security Analysts will not have an IT background, and their security practice will be hands-off, non-tech, check-list based. Anything “tech” to do with information risk management will be handled by an ops team, but there won’t be any “reverse engineering” or “fuzzing” over there…far from it. More like – firewall configuration, running bad vulnerability management suites, monitoring IDS/SIEM logs.

Picking up on some of Bruce Schneier’s comments: “You can be an expert in viruses, or policies, or cryptography.” Policies? OK, this part is true – if you want to be an expert in policies, whatever that is, you can certainly find this in 90%+ of businesses – but is this something that is an economically viable and sustainable position, or even for that matter anything that any homo sapien would ever really want to do? Probably accountancy would be a better bet.

“Viruses?” Hmm. There are increasing numbers of openings for “malware reverse engineers”, where really what they’re looking for is incident response – they want to know what happened after they discovered that some of their laptops were connecting out to various addresses in places they hadn’t heard of, prior to the click of doom. If you get interviewed for one of these positions, be prepared to answer questions about SIEM technologies and incident response. These openings are not usually associated with reverse engineering to the level of detail of those pattern-makers in the anti-virus software market- and if they are, they needn’t be, and the line manager will get to realize this after a while.

And “cryptography”? We have Bruce’s comments and then we have a title heading from a book by Shostack and Stewart: “Amateurs Study Cryptography; Professionals Study Economics”. So who do you believe here? To be fair, the book was published at the height of the de-engineering phase of security and it sort of fitted with the agenda of the times. I would go with Bruce Schneier again here, but with some qualification (the final paragraph also talks about math): most security departments won’t ever go anywhere near anything mathematical or even crypto-related, and when they do, it will be with a checklist approach that goes something like “is a strong key used?”, “yes”, “ok, good, tick in a box”, or “is DES used?”, “yes”, “ok, that’s bad, I think, anyway use triple DES please” – with no further assistance for the dev team.

With regard Cryptography, right or wrong, it’s really only on tiny islands where the math is seen as relevant – places where they code the apps and people are assigned to review the security of the app. And these territories are keenly disputed. Most of the concerns in the rest of the business world will never get more techy than discussions about key management – which is the more common challenge with crypto anyway (mostly we’re using public crypto algorithms in security, so the challenge is in protection of the key).

There were comments from all respondents about testing applications and breaking into networks. Again, the places where skills like reverse engineering are actually relevant are so small. Bruce Schneier painted a grand picture of thinking like a hacker, not just a mere engineer, in order to be able to create systems that are difficult to compromise by the most advanced hackers. But most humans who design systems are not even thinking about security, or they’re on such a tight deadline (with related KPIs and bonuses) that they side-step security. So as a pen test guru wannabe, you may possess extremely high levels of fuzzing, exploit coding, and reversing skills, but you will never get to use them, in fact you will intimidate most interviewers, and you’ll be over-qualified. There will be easier ways to break into systems in most cases. In fact in general, as I commented in an earlier post, security is insufficiently mature in most organizations to warrant any manual penetration testing whatsoever.

So really, what I have had to say here may sound harsh or “negative”, but I would hate for anyone to get into a field that they thought was challenging, only to discover that it’s anything but. I believe things are changing, but it’s at rather a slow pace, and the field of security has been broken for so long, that there are very few around who know how to fix it. Security is getting more challenging, that’s for sure, but for the security pro who goes looking for a job in this field because of the tech challenge aspect: just be very careful about what you’re getting into. Many jobs sound great from the job descriptions posted to recruitment agents, but this is only a show. The reality inside the team is that you may be sent to Siberia if you so much as use a tech-sounding word like “computer” or “IP address” – while this sounds unreal I can assure the reader that such a scenario is most certainly real, although it is of course more often the case that the job would just not be offered to a “techie”.

Its impossible to cover the jobs aspect of information security in just this article. I had a more comprehensive stab at it in Chapter Six of Security De-engineering. I would say to the prospective security pro though, that the advice given by the five mentioned in this article is not bad advice at all – it’s just that you may push yourself to higher levels, and not see significant benefit from it in your careers any time soon. There will always be some benefit, just not as much as you might expect. Certainly, you will have more confidence, but also probably over-qualify yourself for your current position.

As a security pro with a tech inclination, getting into security might not be as hard as you thought. Thomas Ptacek mentioned “A good way to move into penetration testing: grab some industry standard tools and use an Amazon EC2 account to set up a “shooting range” to attack. Some of the best-known tools are available for free: the Nessus scanner, for instance, while not an application security tool, is free and can land you a network penetration testing role that you can use as a springboard to breaking applications.” Believe me, this is not a difficult target, but because of the way the security industry is, you could very well land a penetration testing job with the preparation as described by Mr Ptacek.

All I had to say here is aimed at managing expectations. You may well find that you have to market yourself down a bit in order just to get a foothold in the industry. Once there, by pushing yourself to learn more and get more advanced skills, it could be that you would eventually osmosize towards the ideal job of your dreams. However, these positions are so rare in reality. Many of the folk I worked with in the earlier days of my career had these ninja skills that have been discussed in the five articles mentioned here. Once we got to the early 2000s they realized that security was no longer a place for them. Has the demand for these advanced skills returned? It has, to some extent, but still the demand is miniscule compared to the usual skills required the vast majority of businesses.