top | item 26056136

Ask HN: Any tips for a programmer wanting to switch into security?

148 points| Murgen_ | 5 years ago | reply

I've got ~5 years experience as a business app developer with some networking / DevOps experience in there as well. The more I learn about the networking side the more interested I am in how to secure this, and I'm less interested in writing code.

Obviously security is becoming more and more important, and I'd like to focus my career toward this. In terms of talent, I'm an average Developer, and I know there are roles that focus toward knowing how to secure applications at the code level, which could be interesting, but I also would be interested in securing networks.

I've read that OSCP certification is very good for getting a role in Penetration Testing. Is PenTesting a good place to enter the field?

Any general advice would be much appreciated.

110 comments

order
[+] robcohen|5 years ago|reply
My advice is to stay as a software engineer, and slowly move towards secure software development. 98% of the security world is complete and utter bullshit, and no one is interested in actually securing anything. Ever wonder why EVERY bank has terrible 2FA practices? It's because they don't care. Same with law firms, hedge funds, governments, etc. Yes you can get great job security with security, but at the end of the day you simply won't be producing anything of value unless you are in the top 2% of the field.

Secure software development is different. Go make high quality software for firms that write in functional languages and use advanced methods for ensuring high code quality and safety.

Source: I did penetration testing for four years, also served in a cyber position in the military. What a giant waste of my time that whole effort was.

[+] gotaran|5 years ago|reply
I'm a burnt out security focussed software engineer of 9 years. I worked at 2 well-known Bay Area tech companies. The first 7 years were incredibly rewarding, but now I'm taking a year off the job market. At the end of my tenure, I felt like I was a few days away from being committed to an insane asylum.

Doing anything security related is amazing if you work with the right people, but more often than not the amount of snake oil and petty politics is intolerable, more so than in any normal software engineering gig.

I know a few incredibly competent ex-Uber security software engineers at Uber (disclaimer: I never worked there), who reported to Joe Sullivan, a former federal prosecutor with no technical knowhow whatsover, now facing criminal charges [1]. They're very traumatized from the incident. Trust me, this sort of sleaze is on par for the course at security orgs in major Bay Area tech companies.

[1] https://www.nytimes.com/2020/08/20/technology/joe-sullivan-u...

[+] dpeck|5 years ago|reply
Thanks for sharing that. Aging vulnerability researcher and reverse engineer here and it is both amazing and disheartening to see what most of security work has turned into.

Layers and layers of policies, procedures, and abstractions that mean no one is ever accountable for anything and anyone who knows the details is marginalized as being too much “in the weeds” so they’re left out of the conversation.

[+] dstroot|5 years ago|reply
CIO in financial services here. Most security activity is/will be driven be external forces. Auditors, regulators, etc. This tends to drive organizations towards “checklist” security. Real security professionals know that it’s not really about “can we check the MFA box” it’s about HOW we implement MFA. Unfortunately, that discussion gets squeezed in favor of “we have to close this finding by the next risk committee meeting...”.
[+] humbleMouse|5 years ago|reply
Exactly this. Software development and security are basically the same thing. If you’re not writing secure software, and you don’t understand enough about network layers to secure them, then you have a lot to learn just to get your knowledge up to par with a lot of devs/arch’s/“security people”
[+] dfgdghdf|5 years ago|reply
My perspective as a software engineer is that security is a much more technically challenging field that SWE. Is this true? Is security just smoke and mirrors? And why is SWE so much better remunerated?
[+] formerly_proven|5 years ago|reply
Same for security and audit processes. Most of these are a largely pointless waste of money and time. Some are well-intentioned, but utterly impractical to the point of questioning the well-intentioned part (e.g. "all releases of all software needs to be audited").
[+] guerrilla|5 years ago|reply
> 98% of the security world is complete and utter bullshit, and no one is interested in actually securing anything

As someone who did it for six years before leaving for better things, I couldn't agree more. It really is a circus that sells feelings at best.

[+] bluedino|5 years ago|reply
It’s a joke, but it can pay really well.
[+] GartzenDeHaes|5 years ago|reply
From an executive management point of view, security incident insurance is often seen as cheaper and safer than investing in actual security. That often leaves compliance as the only real security emphasis.
[+] fma|5 years ago|reply
I was in the same position as OP...subscribed to 2600 as a high schooler (maybe I'm on a list)...then after college became a software engineer w/ interest in security. Even got a Security+ certification 10 years ago. Afterwards, I learned most of the security work is policy...at least, that's what people that I knew in the field told me.

Now I'm still a software engineer and get the occasional rush when I need to fix something that our Fortify scanner picks up (rarer now, since we rely so much in frameworks nowadays)

[+] u678u|5 years ago|reply
> EVERY bank has terrible 2FA practices? It's because they don't care.

Good point, but its more they tradeoff losses vs convenience. HSBC pushed a hardware security fob for every time you wanted to check your balance. It was a PITA until they relented. If they kept requiring the device for every log in I'd have closed my account.

[+] Spooky23|5 years ago|reply
Good advice.

Security is a vertical that tends to attract really really smart people and people looking for a landing zone, functioning as unaccredited auditors or folks performing monk-like transcription of NIST or compliance guide. There’s no middle.

The only really technical person I know who is really happy in the space transitioned into a security-focused solution architect type role with a VAR. But he is one of those rare people who is very deep in a few tech disciplines AND loves engaging with people.

[+] catacombs|5 years ago|reply
> Source: I did penetration testing for four years, also served in a cyber position in the military. What a giant waste of my time that whole effort was.

But the money must've been really, really good.

[+] pjsilvestre|5 years ago|reply
Why functional languages? I have some experience with Haskell via a class I took in uni, although we didn't go into detail re: security.
[+] AznHisoka|5 years ago|reply
Why was it a waste of time? If you found just 1 security flaw that could’ve resulted in a ransomware attack, wouldnt it have been worth it?
[+] markus_zhang|5 years ago|reply
Thanks for sharing. Do u think reverse engineering in a certain field (e.g. Hardware or Windows applications) could be a better path?
[+] ubertoop|5 years ago|reply
> for firms that write in functional languages

What does the language being a functional language have to do with it?

[+] woodruffw|5 years ago|reply
I'm a security researcher. From my vantage point:

* There's a world of difference between the work I do and the world of pentesting. Are you interested in building secure systems, researching ways to secure (or break) systems, or applying security techniques? That answer to that will probably inform the kind of security track you want to head down.

* IME, the best security engineers are diligent software engineers. Others have already said it, but: good software engineering is secure engineering, and the skills you pick up as a normal engineer who thinks a little bit harder about security will take you much further than a certificate or special training in security itself.

[+] rtkaratekid|5 years ago|reply
Thanks for the comment. You answered my question before asking. I was looking for how best to transition but it sounds like just interest and diligence is fundamental.
[+] alangibson|5 years ago|reply
Don't do it. Execs see security as pure cost, so your head will be on the block every time they need to trim the budget. There's also constant pressure to offshore it since it's not a core competency unless your company is a cybersecurity consultancy.

And as others have pointed out, unless you're top 2% you'll be running scripts until you're required to train your outsourced replacement.

[+] joshka|5 years ago|reply
+1 being on the "build stuff to increase revenue" rather than "build stuff to decrease costs" side of the business is anecdotally a path to general developer happiness.
[+] Kalium|5 years ago|reply
Good execs understand that security is risk management. I've found that this is better understood outside startups trying to hockey-stick. Finance in particular is a place where reasoning about risk often comes naturally to leaders.
[+] bluedino|5 years ago|reply
On the other hand, companies need compliance so they pay out the ass for SIM tools they will basically never use.
[+] gwittel|5 years ago|reply
I’ve worked in computer security for over 15 years as an engineer. In my case I often work closely with threat researches building infrastructure or things to make their lives easier (eg automate or detect the “easier” attacks so they can focus on the novel). I’ve had the same Internal career debate recently, in my case going back to roots. I enjoy CTFs and working with our threat researchers. With this skill set I figure out how to scale them. If I know more, can I help more? Probably.

Beyond general security it sounds more like you’re interested in the blue team side (defense). To that I’d look into things like https://cyberdefenders.org/ hack the box and other CTF work first. OSCP is nice and maybe even mandatory for pen testing (Hr) but hands on is key. From there you can figure out if certain are worth it to you. The ejpt is a cheaper starter as well.

In practice you need both technical skills and report writing skills. You have to tell a security story to technical and non technical people. The better you are at both, the further you can go. As a counterpoint, we still have a hard time finding solid security minded engineers. You can be a triple threat :)

[+] riskable|5 years ago|reply
Note: I'm a security engineer. Have been doing security work for ~15 years now.

Penetration testing is a good way to get real security experience. You'll learn pretty quickly just how vulnerable everything is and how attackers use the tools to exploit said vulnerabilities. If penetration testing is your job though know that it doesn't often pay very well. Most companies that hire "penetration testers" are really just looking for folks to run a bunch of scripts/tools against a list of IP addresses/hostnames and generate a template-based report. That is tedious, mindless work.

There's "security consulting" too which often involves at lot of actual penetration testing (not just running scripts) and that can pay pretty well but probably not as much as you'd think. The real money in security consulting is in governance work, sadly (because it's not as fun haha). There's a million companies offering "penetration testing" (even if its awful/useless) so the price for that has been driven down quite a lot of the years but companies offering consultants that can write your company's security policies and procedures are much more rare (and expensive!). That's why one pays better than the other... Even though becoming a good penetration tester requires 1000x more knowledge and experience than the skills necessary to write a policy document.

Penetration experience is important though if you want to be serious about security. I think penetration testing experience is so important that I'd say that anyone that claims to be a Chief Information Security Officer (CISO) that hasn't performed some form of penetration testing doesn't have the requisite knowledge to do the job. They're an imposter, IMHO.

At the very least learn how to use Metasploit and actually use it to successfully run a payload on something (anything). Then--rather than getting a job as a penetration tester--I'd use your software engineering skills to develop some security tools. For example, there's a huge gap in the market for open source password management tools (think CyberArk, not Hashicorp).

[+] cmehdy|5 years ago|reply
Naive question from an outsider with an interest in understanding pentesting and information security (mostly out of curiosity, but also to contribute to projects with security in mind): do you think putting time into a cert like OSCP is worth it?

Not arguing about any kind of market or resume worth, but more so about the actual value provided by the path to complete that cert.

[+] bluedino|5 years ago|reply
>> The more I learn about the networking side the more interested I am in how to secure this, and I'm less interested in writing code.

Be warned, most 'security' jobs are running scripts and programs and filling out checklists. If you were interested in writing code I'd suggest books like Reversing: Secrets of Reverse Engineering or Hacking: The Art of Exploitation

That said, a good way to get into it is to find any kind of local user groups, either in industries or at colleges, and find ones that offer security classes and do capture the flag (CTF) events.

Here's one in Michigan, for example:

https://www.merit.edu/security/training/

This is a good way to get familiar with the tools you would be using and even better, a good way to meet other people in your area who might know of job openings and such.

Here are some details on CTF events:

https://cybersecurity.att.com/blogs/security-essentials/capt...

[+] cj|5 years ago|reply
I'm not a security engineer, although I've managed security programs and triage security issues. If I were you, I'd probably start by pen testing via HackerOne. You likely won't earn much doing it, but it's an easy way to access many companies inviting you to break their systems.

Many reports on HackerOne are disclosed publicly. Reading through public reports will expose you to what application errors are most commonly found with specific reproduction steps, what tools were used to discover the issue (Burp Suite is very common), and use that as a jumping off point for what to learn and discovering where your knowledge gaps are.

[+] austincheney|5 years ago|reply
The best advise about security you will ever get: Don't take security advise from software developers.

I cannot stress that enough.

I worked in the security domain from the military for about 10 years as an Army Reservist while simultaneously launching my career as a software developer in the corporate world. I have the rare experience of working in both worlds simultaneously yet separately. I have passed the Security+, CISSP, and CASP exams each on the first attempt without taking boot camps.

Taking security advise from software developers is like taking marriage advise from some guy at the bar who is looking for his fourth wife.

[+] sedgjh23|5 years ago|reply
Security Engineer here… after reading through this trainwreck of a comment section it was nice to get a laugh, thanks for that.
[+] totorovirus|5 years ago|reply
You can see many brogrammers in so called the biggest security conference DEFCON. You'll soon find out that only top 5 percent actually know what they are doing and other 90 percent complete bullshit people who just claims themselves as security expert while not knowing how to exploit a program. It is one of the few political jobs you can get among computer science world.
[+] appsecengthrow|5 years ago|reply
Application Security Engineer here. For a developer I would recommend the OSWE certification which takes a more code-centric approach and resources such as https://portswigger.net/web-security and The Art of Software Security Assessment book. You can also leverage your DevOps experience for learning about server, app and cloud misconfigurations and their impact on security.

The demand for network security is not as high as for appsec people and I personally don't see network security as very rewarding (intellectually and financially).

For the Pentesting route I recommend trying some HackTheBox and watching Ippsec's channel (https://www.youtube.com/channel/UCa6eh7gCkpPo5XXUDfygQQA). OSCP is fine, but it is a beginner certification and definitely not enough for getting a Pentest job.

[+] jmartrican|5 years ago|reply
Stay as a developer and practice good security in your projects. At the end if the day, it's the developers that have to implement security standards into their code.

Furthermore, certain projects tend to have more security requirements than others. So maybe keep your eyes open for those opportunities. The most security intensive project in worked on was a standalone video game where there was a global leaderboard and save points in the game. Trying to protect the leaderboard from hackers was a fun security challenge. The leaderboard needed to be protected from networked API attacks, local file manipulation, and in memory variable manipulation. It really taught me a lot.

[+] xwdv|5 years ago|reply
As someone who once considered the same thing: Don't.

While security can be a fascinating field to learn about on your own, the actual work you do day to day is dull as hell, and the pay is no where near as good as what you get as a software developer.

You already have 5 years experience as a developer, if you start a security career now you'll be 5 years behind where you should be for your age, and you'll start with shitty entry level jobs that pay very little, and can also be easily outsourced to dedicated security companies anyway.

You have to be really exceptional in the field to make what an average senior software developer makes. Trust me, if you're like most software developers I know whose virtue is laziness and strive to do the most effective work with the least amount of effort, you don't want to be exceptional in the field. It takes so much time and effort and brand building, fuck it man. You're not getting any younger.

Securing networks isn't even a big deal, like anything when you're first learning it it seems super interesting but then it's just the same bullshit over and over.

My recommendation, sticking to writing code and actually building stuff as a career and do the hacking on the side as a hobby, and occasionally use what you learn in your career.

[+] ppierald|5 years ago|reply
I was/am a software engineer who had a wonderful opportunity to join a central security team at a very large internet company focusing on security engineering.

My recommendation on switching is to latch onto any SMEs in your company who you look up to, go to their classes and brown bags, research topics and make presentations to the company, be sure to include security decisions in your architecture designs, then once there's an opportunity in their team, you will be a natural choice for the team.

If there is an opportunity for your current product development team to be a Security Champion (i.e the person primarily responsible for security in your team and liaison to your security team for issues that you are unsure of), then jump on that if possible. Security Champions are a great way to dip your toes into security without having to go all in and also for your company to build a "bench" of talent. They can use this as a career lattice rather than a strict career ladder in the engineering org. Many companies are embracing this model as they grow because security folks are hard to find, hard to retain, and hard to scale as the engineering team grows.

[+] throwawaysecuri|5 years ago|reply
Don’t. I don’t think anyone should attempt to get into security, or at least work in a security team.

First, in many stages, a company needs to move fast. This is not a cliche, if you do not move fast you die. This is what a start up is about. Even as a small business trying to grow, you might often be in a position where you constantly need new prospects, new cash flows, etc. Security is inherently about moving slow, and about friction. This is why security is badly seen by management, and it is badly seen by developers and site reliability engineers. The less you work, the happier people will be around you.

Because of that, being in a security team is often an unrewarding job where you’re moving against the courant. Your team is often understaffed, because again, you’re not producing clear value, so the overton window is shifting against you. You’re outnumbered after all, so most of what you say is unreasonable, and perhaps you team and you start forming an “us against them” mindset, as other teams are doing the same with you.

The job is also not that interesting to be honest, you spend more time reading docs and attending meetings trying to keep up, as well as trying to create connections, as your job is to cover as much ground as possible as well as creating connections so that you can convince and delegate better.

[+] slickrick216|5 years ago|reply
You should check out the OSWE it’s harder for “security” people but for devs it’s a lot easier. Do that and then get the OSCP. You could also pick up a forensics course or two and network forensics course.

Experience wise I would suggest starting with incident handling in a large companies in-house blue team. Ask them about scope and duties. Try get a job where it’s a mix of the tasks within DFIR and the teams scope is wide protecting many different environments from IT to cloud etc. The more variety the more incidents the more experience you’ll get faster.

Given your previous work you’ll likely get asked to work on an app sec team. It’s not for everyone and quite close to testing for some folks. I prefer operations as it has a higher pace.

Like any tech job try to automate things people do manually from forensic analysis to security solutions.

Whatever type of team you are on don’t be a snob and look down on other teams be they security or non security. This is particularly common quite hilariously for red teams who should epitomise hacker culture. Having been on these teams I can tell you they get particularly huffy about elitism.

Also don’t look down on the role of security analyst. Mind you not all analyst roles are created equal. I’ve found though that bar a few large companies if you work for an MSSP (managed security service provider) you probably won’t get the same quality of experience unless you are on a few of their consulting teams. The issue I’ve seen is they have no remit to actually remediate the incidents they find so miss the full journey.

Most of all like anything in life enjoy it. You are choosing this.

[+] GoldenStake|5 years ago|reply
tl;dr; Take the online courses for Cloud Security is the best bang for your buck IMO.

As a Security Engineer that works on network/devops stuff at a modern Saas company. I think 90% of what I do is Cloud DevOps with a focus on Security. That could mean: making frameworks to make security easier, or advise other teams on how to secure their pieces of infra, or identifying insecure configs and pushing to get those fixed. The other 10% is understanding security risks and what designs/implementations of the infra are good/bad. Pen-testing might help with the later skill, but at ~10% it's a surprisingly small factor.

I would like to echo the points made by other posts, that there are a lot of different fields of security. Pentesting is one field, Application security is another, there's also compliance, red-team, IT-security, threat hunting, etc. The list goes on, and there are a lot of different skills you could build, certifications you could get, and areas to specialize in (or distract you from your specialization)

It does sound as if you enjoy the InfraSec/SecDevOps parts of the problem. So learning more about AWS/GCP Security in detail is probably the best way to improve your skill set in the area.

[+] tlavoie|5 years ago|reply
I went the software developer -> security route, so maybe I can weigh in a little.

Security takes a different mind-set in some ways; not incompatible, but not necessarily there to start with. I've always been interested in how things fail, and with what might happen when it does.

Many school-mates going through CS were interested in how to build things, so their goal was achieved if things were constructed and seemed to run well. Edge cases might seem then to resemble the background noise when talking about big-O issues in an algorithm. In reality though, small gaps might lead to larger consequences.

In that way, I think security might more closely resemble QA, except that the consequences of failure have more interesting implications.

To that end, how can you incorporate security analysis into the software development work you do today? For me, it was a matter of trying to help secure our own product; from there I could bounce to some level of consulting.

[+] kevin_nisbet|5 years ago|reply
While maybe not really actionable advice, I've been making a slow transition in this direction over the last few years, so I'm just sharing what happened for me incase it's of use.

The networking side came from that being my background, I used to work in cellular telecom, and my role was to solve complex network problems.

The security side has been a more natural transition, and it really came for working for companies that had security problems but weren't really equipped or able to solve them. But the bit of its that luck or I don't really understand, is somehow I had developed better mental models of technical security that allowed me to break apart other proposed solutions and develop my own, which was probably just religious reading of HN security content.

This better understanding put me into a position to solve organization objectives where the skills were otherwise missing, and then the organization started asking to solve other security problems, and without really trying I've been a security SME in my previous and current role.

From a tips perspective, I think the most important tip is approaching almost any problem with the statement "I don't know what I don't know". A lot of devs can get away with brute forcing their way through tech and applying similar solutions they know to solve new problems. Such as a personal pet peeve, the number of devs who think a crypto or password hash will anonymize data. But starting with know our mental model is incomplete, and trying to figure out why, I think helps me out alot.

So the second tip is, reach out to someone who does security and get free advice for particular problems. Write a design doc and get them to review it, or just converse with them at a high level what's around to solve a particular problem. I've done complete 180's on particular choices based on just a conversation about what exists and then go and do a bunch of research.

[+] codykochmann|5 years ago|reply
Every field of CS has a security aspect to it. Unless you are a legitimate network engineer, learn how to use the network right before learning how to break it.

I'd say start with learning how security works in the world you know. Defensive programming is a very real thing and translates to just about every other field of CS since... Its all running on code. :D

There are quite a few good defensive coding guides out there. Redhat has some really nicely put together guides for you to start learning from.

https://developers.redhat.com/articles/defensive-coding-guid...

Remember: Learning how to use a gun and becoming licensed to use the gun isnt gonna teach you nearly as much about security as learning how to build Fort Knox.

[+] bawolff|5 years ago|reply
The best is if there is a security team at your current role that needs help, which you could transfer to, or even unofficially help out.

There are lots of different types of computer security jobs. Lots are really boring compliance jobs (i mean, some people are into that, all the power to you, if that's what they like). From a corporate perspective security is all about risk management, some aspects of risk management is covering your ass, so make sure the security position aligns with what you want to actually do.

> I've read that OSCP certification

That might be a good starting place. Just be careful about relying too much on certs, about 95% of them are bullshit, they definitely don't replace real world experience, and its a major red flag to have a resume with like 10 different certs on it.