What saddens me is that, while red team pen testing is a very "hot" (high employer demand, high salaries) job market, people don't generally care about the blue team. It's easy to get a pentesting gig that pays well, but employers don't ask for/value people with the competence to build/maintain secure applications/networks/solutions in my experience. Instead they pay for recurring pen tests which results in internal tickets/issues/BUGs, while the development/operation practices are kept the same.
It's because internal econometrics are resulting in perverse incentives.
Defensive security is a cost center without clear, deterministic metrics for success. Let's say you pay X on defensive security (which is an oversimplification when you're talking about a cultural change, but that cultural change involves people learning how to pay attention to security, and paying attention is a form of man-hours, for which a cost can be calculated). If you don't get attacked, is it because the X you paid is high enough to deter/foil attackers, or could you have paid less and achieved the same result? If you are attacked and the attackers get past your defenses, is it because the X you paid wasn't enough, or if you had spent more, would the attackers have succeeded anyway, because of their relative power and motivation? For defensive security, it's very, very hard to justify to bean counters that X was the correct amount of money spend, no matter what the real outcome is, because it's hard to understand X's affect on that outcome.
Pentests which result in tickets/issues/etc. are much easier to justify. The company spent X on the pentest, and it got Y feedback in return. Simple, and effective, at least in the short-term.
It's part of the overall challenge that organizations face when they become metrics-driven. People choose the path of least resistance, so if you ask people to measure data, they'll measure the data that's easiest to measure. Data that's harder to measure - culture and social attitudes - becomes "not a priority" to measure.
it's like everything in a big company. every round of red team on my team's applications we just sit and laugh as they find nothing, yet we serve data to billions of customers every hour, from a myriad of complex entry points. nobody cares, and when I mention that on my self reviews it looks like I am padding it.
then the other teams only handle requests from the ios app they own, and red team finds tons of amateur attacks that work. they spend a quarter fixing it, and boast that they worked with the red team to patch hundreds of vulnerabilities. and everyone is promoted.
but that is not new. it always happened with teams that causes outages, or teams that miss out obvious revenues stream for years. remedial action for some reason is always rewarded in troubled big corps.
My experience is inconsistent with yours. In my 12 years of experience in the security industry, I have found that the blue teams (engineers who develop, maintain and secure network, data and applications) have higher demand and higher salaries than the red teams or pen-testing teams.
My experience is limited to security software development in e-banking, e-commerce, network security and data security domains in technology areas like cryptography, PKI, deep packet inspection, and network protocols. I know my experience may not be representative of the entire security industry and there is a possible selection bias too (i.e. I may have seen more demand for blue team engineers because I have belonged to blue teams myself), but I thought I should share my experience here to present the other side of the story.
I used to work for a pentest company and everytime we engage with a customer, we discover a lot of problems and security issues which we document and submit to the customer.
We discover after a while that another company gets a contract 10x our price fixing the issues we discovered
I'm hiring for blue team at Akamai. We've picked up some fantastic early-career people into our training program; also looking for mid-career architects, senior architects.
Honestly, as someone who is leaving "blue-team" network security work for a "DevOps" production team (I know, I know), it's really a mixed bag. I've done blue-team for 2 companies now and honestly the job is more project management than anything else. I found that I was very rarely actually getting hands-on with technology. When implementing a new piece of security tech, we were simply directing other teams to perform most of the the actual technical work (this was the case at both of my "security engineering" jobs). I didn't get any of the satisfaction of building anything, solving problems etc.
The other big thing to note is that a lot of companies have security teams solely to meet audit requirements. If you find yourself on a team like that, you'll be spending a lot of time just gathering evidence for audits, remediating findings and writing policy. I really loved security intellectually, but in practice, the blue-team side of things wasn't my cup of tea.
This will change with time. I have thought, studied, talked, and written extensively on this general theme. What comes to mind to frame it for you is my "four principles:" [a]
1. Compromise is inevitable
2. Default-allow products always fail
3. 1 and 2 are not opinion or marketing spin, just simple truths
4. As an industry, we are still learning 1 and 2
Security is slowly shifting from an administrative IT function to an operational function. In IT, the business value comes from the products and people are a tax required to administer the products. In security operations, the business value comes from the people, products are just tools in their toolbag. [c]
Keep walking this dog and you realize basic IT activities for core infrastructure are critical for security, to the point the CIO will report to the CISO -- unless the CIO steps up. [b]
So - in short - your frustrations are accurate, but the winds are shifting. Companies will incresingly value top people for their internal staff/blue teams. It's going to take a few more years, but I believe it is inevitable.
I'm lucky enough to be with a company with a couple people on both sides. Originally we were all blue team (that was VERY poorly integrated with the engineering group), but ive started shifting one guy over to full time pentesting and working on integration with engineering.
> results in internal tickets/issues/BUGs, while the development/operation practices are kept the same.
You could not be more accurate; this also applies to groups that maybe started out as corporate infosec (virus protection, simple application scanning, etc...) and were never really tightly coupled with engineering. We have identified essentially identical authorization issues in a pre-release version of one of our products two or three times this year, which was also present in the last 3rd party pentest of the same product before my time (which was pretty scathing). Its incredible.
The company I work for is aggressively hiring blue teamers. The core of our product is essentially a security product, so we need someone who has at least some professional engineering experience as you would be responsible for helping to lead teams building certain parts of our infrastructure and product. Someone who can hop onto code reviews that deal with sensitive areas would be fantastic.
We're located in Boulder but for the right candidate we'd consider remote, although that might involve relatively frequent travel.
cGhpbGlwLmRldWNobGVyQGp1bXBjbG91ZC5jb20= for contact
In theory you could take advantage of this by starting a company that hires people from the blue side at a premium and create products with top notch security.
Unfortunately I am not sure if currently consumers care that much about security of their products relative to convenience, price, and eye candy.
But in theory those who spend lots on red side will end up having a more expensive product and a bad reputation, so perhaps investing more on blue will win in the long run.
This. Not interested in pen testing at all, just want to build things that don't suck from a security perspective. Happy to break things along the way to gain perspective, and I like to know how things work, but I'm a software engineer who likes security. A builder, not a breaker, by nature.
I am totally into this field as a bystander. When many people would watch late night TV or listen to music or a podcast, I'll scour YouTube for defcon and CCC talks I haven't seen yet. I am good with python, Javascript, web and graphic design, technical writing, all kinds of stuff. I live in rural neighbor-island Hawaii and the only tech jobs I ever see out here are military, which I deeply respect but don't think would be a good culture fit for me. I've just transitioned to working part time and intend to dive in to some open source projects with my extra time. Is there a good path of entry for someone with a deep natural curiosity about the field, self-trained in coding but with no industry connections or much in the way of related professional experience?
3) Finally, each of these popular "Getting Started in Security" guides has a slightly different, but useful, opinion on the specifics of the path to take:
I advise companies on tech security, and talent is very much needed.
What's surprising (at first glance) is that the security talent need is very strong in UI/UX/CX.
For example, security is needed to gradually escalate a user's own identity verification -- think of things like two-factor auth and multi-factor auth, that can phase in (or ramp up) when a user's actions enter a gray area of risk.
Some examples: when a user signs in from a new location, or a user does an especially large money transfer, or a user resumes an account that's been dormant for years, etc.
The UI/UX/CX is especially necessary to 1) explain to the user that there's a security issue, 2) how the user can fix it, and 3) how to improve backend systems to handle users who ask for help.
I work in infosec as a security engineer, I agree with this more than anything else anyone has posted about the state of the security industry on HN, ever.
For security being a human problem, I have yet to meet infosec types with strong backgrounds in human factors or product design. Nearly everyone came to this industry from netsec/IT/SOC work, or low-level programming, and neither group has a decent understanding of the usability issues that plague the security posture of common users. What works for a CLI junkie with deep systems knowledge absolutely fails people who barely know how to navigate their Android phone.
If anyone's interested in attempting to solve some of these design pattern issues, please reply here or DM me on Twitter. I'd love to actually get a group of people together trying to come up with standard, secure UX paradigms that can be referenced by others.
There are so many sources of information and learning grounds available now - bug bounties, certifications, war games, online tutorials, blogs, conferences etc.
I would suggest choosing a particular area of interest to begin with and deep-diving on that subject. Look for mentors or perhaps someone to knowledge share / skill exchange with.
You could do pretty well with a base in C#. Through pentest engagements, I've come across quite a few C# apps in my time and even with my limited knowledge of the language, found some interesting vulnerabilities ;)
I'm in my late 20s and got into the field professionally just a couple years ago. Prior to that I had been working as a software developer.
I believe what helped was a handful of personal projects related to security: reverse engineering firmware, finding bugs in web apps. Also I emphasised the parts of my software development work that had some overlap, such as debugging Windows kernel drivers, and doing security reviews of network services we were writing and deploying.
Now I'm doing full-time vulnerability research and writing software to help do that. Much more enjoyable and pays better too.
You don't need a college degree to do pentesting. I don't think anyone credible in the industry cares where you got your education, as long as you know what you're doing (or in the case of juniors, are able to learn).
Always cracks me up thinking about how much the press talks about cybersecurity being in demand then looking at the actual IOT and embedded cybersecurity market.
Everyone should be very, very scared about infrastructure/medical security and the effective lack of anyone doing anything about it.
I'm sure web/network pentesting is doing well though.
Slightly skewed towards consulting; notch it up a bit to account for stock grants in addition to the bonus if you're thinking of something like Google, Facebook or Amazon. Sometimes the consulting firm pays separate bonuses for research time and bench projects that result in something useful or beneficial for the firm's brand.
That's based on my own experience and talking with probably well over a hundred people in the industry about their salary by now.
You can do remote, I've been remote almost my entire career thus far. Remote is more likely in consulting than it is at one of the reputable internal security teams, but it's not uncommon at smaller tech companies.
Regarding onsite/remote, I think it depends on what you choose to specialise in. Most of the web application assessments I conduct are remote but there is still demand for onsite work.
I'm looking to get out of security after being in IR and security research at a large startup for 5 years. I am having no luck finding anything besides glorified helpdesk jobs and other entry level positions.
[+] [-] sebcat|8 years ago|reply
[+] [-] solatic|8 years ago|reply
Defensive security is a cost center without clear, deterministic metrics for success. Let's say you pay X on defensive security (which is an oversimplification when you're talking about a cultural change, but that cultural change involves people learning how to pay attention to security, and paying attention is a form of man-hours, for which a cost can be calculated). If you don't get attacked, is it because the X you paid is high enough to deter/foil attackers, or could you have paid less and achieved the same result? If you are attacked and the attackers get past your defenses, is it because the X you paid wasn't enough, or if you had spent more, would the attackers have succeeded anyway, because of their relative power and motivation? For defensive security, it's very, very hard to justify to bean counters that X was the correct amount of money spend, no matter what the real outcome is, because it's hard to understand X's affect on that outcome.
Pentests which result in tickets/issues/etc. are much easier to justify. The company spent X on the pentest, and it got Y feedback in return. Simple, and effective, at least in the short-term.
It's part of the overall challenge that organizations face when they become metrics-driven. People choose the path of least resistance, so if you ask people to measure data, they'll measure the data that's easiest to measure. Data that's harder to measure - culture and social attitudes - becomes "not a priority" to measure.
[+] [-] gcb0|8 years ago|reply
then the other teams only handle requests from the ios app they own, and red team finds tons of amateur attacks that work. they spend a quarter fixing it, and boast that they worked with the red team to patch hundreds of vulnerabilities. and everyone is promoted.
but that is not new. it always happened with teams that causes outages, or teams that miss out obvious revenues stream for years. remedial action for some reason is always rewarded in troubled big corps.
[+] [-] susam|8 years ago|reply
My experience is limited to security software development in e-banking, e-commerce, network security and data security domains in technology areas like cryptography, PKI, deep packet inspection, and network protocols. I know my experience may not be representative of the entire security industry and there is a possible selection bias too (i.e. I may have seen more demand for blue team engineers because I have belonged to blue teams myself), but I thought I should share my experience here to present the other side of the story.
[+] [-] hexc0de|8 years ago|reply
We discover after a while that another company gets a contract 10x our price fixing the issues we discovered
[+] [-] brians|8 years ago|reply
https://akamaijobs.referrals.selectminds.com/jobs/senior-lea...
https://akamaijobs.referrals.selectminds.com/jobs/security-a...
https://akamaijobs.referrals.selectminds.com/jobs/manager-in...
[+] [-] SandwichTeeth|8 years ago|reply
The other big thing to note is that a lot of companies have security teams solely to meet audit requirements. If you find yourself on a team like that, you'll be spending a lot of time just gathering evidence for audits, remediating findings and writing policy. I really loved security intellectually, but in practice, the blue-team side of things wasn't my cup of tea.
[+] [-] jjguy|8 years ago|reply
1. Compromise is inevitable 2. Default-allow products always fail 3. 1 and 2 are not opinion or marketing spin, just simple truths 4. As an industry, we are still learning 1 and 2
Security is slowly shifting from an administrative IT function to an operational function. In IT, the business value comes from the products and people are a tax required to administer the products. In security operations, the business value comes from the people, products are just tools in their toolbag. [c]
Keep walking this dog and you realize basic IT activities for core infrastructure are critical for security, to the point the CIO will report to the CISO -- unless the CIO steps up. [b]
So - in short - your frustrations are accurate, but the winds are shifting. Companies will incresingly value top people for their internal staff/blue teams. It's going to take a few more years, but I believe it is inevitable.
[a] - https://www.linkedin.com/pulse/my-four-cybersecurity-princip... [b] - https://www.linkedin.com/pulse/cio-report-ciso-j-j-guy [c] - https://www.linkedin.com/pulse/cio-report-ciso-why-j-j-guy
[+] [-] jdc0589|8 years ago|reply
> results in internal tickets/issues/BUGs, while the development/operation practices are kept the same.
You could not be more accurate; this also applies to groups that maybe started out as corporate infosec (virus protection, simple application scanning, etc...) and were never really tightly coupled with engineering. We have identified essentially identical authorization issues in a pre-release version of one of our products two or three times this year, which was also present in the last 3rd party pentest of the same product before my time (which was pretty scathing). Its incredible.
[+] [-] unknown|8 years ago|reply
[deleted]
[+] [-] pdeuchler|8 years ago|reply
We're located in Boulder but for the right candidate we'd consider remote, although that might involve relatively frequent travel.
cGhpbGlwLmRldWNobGVyQGp1bXBjbG91ZC5jb20= for contact
[+] [-] j7ake|8 years ago|reply
Unfortunately I am not sure if currently consumers care that much about security of their products relative to convenience, price, and eye candy.
But in theory those who spend lots on red side will end up having a more expensive product and a bad reputation, so perhaps investing more on blue will win in the long run.
[+] [-] sporkenfang|8 years ago|reply
[+] [-] taylorfinley|8 years ago|reply
[+] [-] mr_overalls|8 years ago|reply
https://www.schneier.com/blog/archives/2012/07/how_to_become...
2) The Reddit NetSec FAQ has a good list of resources for beginners (and those starting to specialize):
https://www.reddit.com/r/netsec/wiki/start
3) Finally, each of these popular "Getting Started in Security" guides has a slightly different, but useful, opinion on the specifics of the path to take:
https://medium.freecodecamp.org/so-you-want-to-work-in-secur...
https://danielmiessler.com/blog/build-successful-infosec-car...
https://www.trustwave.com/Resources/SpiderLabs-Blog/Getting-...
https://tisiphone.net/2015/10/12/starting-an-infosec-career-...
[+] [-] wepple|8 years ago|reply
[+] [-] ohm|8 years ago|reply
[+] [-] davedx|8 years ago|reply
They're based in Amsterdam, but she said that a lot of their pentesters and engineers are remote (all over the world).
Might be worth reaching out!
[+] [-] user5994461|8 years ago|reply
[+] [-] jph|8 years ago|reply
What's surprising (at first glance) is that the security talent need is very strong in UI/UX/CX.
For example, security is needed to gradually escalate a user's own identity verification -- think of things like two-factor auth and multi-factor auth, that can phase in (or ramp up) when a user's actions enter a gray area of risk.
Some examples: when a user signs in from a new location, or a user does an especially large money transfer, or a user resumes an account that's been dormant for years, etc.
The UI/UX/CX is especially necessary to 1) explain to the user that there's a security issue, 2) how the user can fix it, and 3) how to improve backend systems to handle users who ask for help.
[+] [-] micaksica|8 years ago|reply
For security being a human problem, I have yet to meet infosec types with strong backgrounds in human factors or product design. Nearly everyone came to this industry from netsec/IT/SOC work, or low-level programming, and neither group has a decent understanding of the usability issues that plague the security posture of common users. What works for a CLI junkie with deep systems knowledge absolutely fails people who barely know how to navigate their Android phone.
If anyone's interested in attempting to solve some of these design pattern issues, please reply here or DM me on Twitter. I'd love to actually get a group of people together trying to come up with standard, secure UX paradigms that can be referenced by others.
[+] [-] aphextron|8 years ago|reply
Which technologies specifically?
[+] [-] jomkr|8 years ago|reply
I used to love doing pen-testing when I was a teenager, and paid for my first car out of bug-bounties.
Unfortunately, I got distracted by girls and booze at university and didn't keep it up, now I work in sigh enterprise C#/WPF land.
[+] [-] janfry|8 years ago|reply
There are so many sources of information and learning grounds available now - bug bounties, certifications, war games, online tutorials, blogs, conferences etc.
I would suggest choosing a particular area of interest to begin with and deep-diving on that subject. Look for mentors or perhaps someone to knowledge share / skill exchange with.
You could do pretty well with a base in C#. Through pentest engagements, I've come across quite a few C# apps in my time and even with my limited knowledge of the language, found some interesting vulnerabilities ;)
Edit: Added tptacek link
[+] [-] secu|8 years ago|reply
I believe what helped was a handful of personal projects related to security: reverse engineering firmware, finding bugs in web apps. Also I emphasised the parts of my software development work that had some overlap, such as debugging Windows kernel drivers, and doing security reviews of network services we were writing and deploying.
Now I'm doing full-time vulnerability research and writing software to help do that. Much more enjoyable and pays better too.
[+] [-] devwastaken|8 years ago|reply
[+] [-] tetrep|8 years ago|reply
[+] [-] unixhero|8 years ago|reply
(Similar story:) )
[+] [-] ohm|8 years ago|reply
[+] [-] rhexs|8 years ago|reply
Everyone should be very, very scared about infrastructure/medical security and the effective lack of anyone doing anything about it.
I'm sure web/network pentesting is doing well though.
[+] [-] jerrylives|8 years ago|reply
Excellent talk from last year's HOPE about the vulnerability of medical devices - he even connects to some live
[+] [-] laurencei|8 years ago|reply
And is it largely on site work, or is it more common as a remote consultant?
[+] [-] dsacco|8 years ago|reply
Junior: $120k base. $10-15k annual bonus.
Mid: $150k base. $20-30k annual bonus.
Senior: $180k base. $30k annual bonus.
Slightly skewed towards consulting; notch it up a bit to account for stock grants in addition to the bonus if you're thinking of something like Google, Facebook or Amazon. Sometimes the consulting firm pays separate bonuses for research time and bench projects that result in something useful or beneficial for the firm's brand.
That's based on my own experience and talking with probably well over a hundred people in the industry about their salary by now.
You can do remote, I've been remote almost my entire career thus far. Remote is more likely in consulting than it is at one of the reputable internal security teams, but it's not uncommon at smaller tech companies.
[+] [-] janfry|8 years ago|reply
[+] [-] unethical_ban|8 years ago|reply
[+] [-] kdbg|8 years ago|reply
[+] [-] mahyarm|8 years ago|reply
[+] [-] drraid0|8 years ago|reply
[+] [-] unknown|8 years ago|reply
[deleted]
[+] [-] unknown|8 years ago|reply
[deleted]
[+] [-] goshx|8 years ago|reply